A re-usable Versionable Behavior for Doctrine 2

Posted 5 months ago by beberlei

My previous post on behaviors in Doctrine 2 generated quite some discussion about the difference on behaviours that are re-usable across models and the trivial specific implementations I have shown.

In this post I will show a re-usable versionable (audit-log) behavior. For this we will need the following ingredients:

  • An interface DoctrineExtensions\Versionable\Versionable
  • A class DoctrineExtensions\Versionable\VersionManager
  • An event listener DoctrineExtensions\Versionable\VersionListener
  • A generic entity DoctrineExtensions\Versionable\ResourceVersion

The Event API is currently in the central focus of our efforts so the API shown here may change before the first Beta release.

The workflow is as follows, each Entity that is supposed to be versionable has to implement the interface Versionable which looks like this:

<?php

namespace DoctrineExtensions\Versionable;

interface Versionable
{
    /**
     * @return int
     */
    public function getCurrentVersion();

    /**
     * @return array
     */
    public function getVersionedData();

    /**
     * @return int
     */
    public function getResourceId();
}

Whenever an entity is persisted or updated the state that is persisted will also be logged in an audit table. The state is returned with an array of key value pairs in the getVersionedData() and the current version has to be the value of the @Version column of the entity.

To sum up, the requirements of an entity that can be a Versionable in this simple implementation:

  • Single Integer Primary Key.
  • Has to be versioned with an integer column.

How does such versioned data look like? The generic resource version entity looks like this. Its a Doctrine Entity, but in a domain model its an immutable value object. It should not be changed after creation.

<?php

namespace DoctrineExtensions\Versionable;

class ResourceVersion
{
    /** @Id @Column(type="integer") */
    private $id;

    /** @Column(type="string") */
    private $resourceName;

    /** @Column(type="integer") */
    private $resourceId;

    /** @Column(type="array") */
    private $versionedData;

    /**
     * @Column(type="integer") */
    private $version;

    /** @Column(type="datetime") */
    private $snapshotDate;

    public function __construct(Versionable $resource)
    {
        $this->resourceName = get_class($resource);
        $this->resourceId = $resource->getResourceId();
        $this->versionedData = $resource->getVersionedData();
        $this->version = $resource->getCurrentVersion();
        $this->snapshotDate = new \DateTime("now");
    }

    // getters
}

Now we need to solve the problem of generating the ResourceVersion whenever an Versionable entity is persisted or updated. This can be done by using the Doctrine EventManager API. We will implement the EventSubscriber interface and hook into the "onFlush" event.

<?php

namespace DoctrineExtensions\Versionable;

use Doctrine\Common\EventSubscriber,
    Doctrine\ORM\Events,
    Doctrine\ORM\Event\OnFlushEventArgs,
    Doctrine\ORM\EntityManager;

class VersionListener implements EventSubscriber
{
    public function getSubscribedEvents()
    {
        return array(Events::onFlush);
    }

    public function onFlush(OnFlushEventArgs $args)
    {
        $em = $args->getEntityManager();
        $uow = $em->getUnitOfWork();

        foreach ($uow->getScheduledEntityInsertions() AS $entity) {
            if ($entity instanceof Versionable) {
                $this->_makeSnapshot($entity);
            }
        }

        foreach ($uow->getScheduledEntityUpdates() AS $entity) {
            if ($entity instanceof Versionable) {
                $this->_makeSnapshot($entity);
            }
        }
    }

    private function _makeSnapshot($entity)
    {
        $resourceVersion = new ResourceVersion($entity);
        $class = $this->_em->getClassMetadata(get_class($resourceVersion));

        $this->_em->persist( $resourceVersion );
        $this->_em->getUnitOfWork()->computeChangeSet($class, $resourceVersion);
    }
}

How do we hook this VersionListener into the EntityManager? We will wrap the VersionManager around it that handles registration and offers some convenience methods to retrieve the versions of a resource.

<?php

namespace DoctrineExtensions\Versionable;

use Doctrine\ORM\EntityManager;

class VersionManager
{
    private $_em;

    public function __construct(EntityManager $em)
    {
        $this->_em = $em;
        $this->_em->getEventManager()->addEventSubscriber(
            new VersionListener()
        );
    }

    public function getVersions(Versionable $resource)
    {
        $query = $this->_em->createQuery(
            "SELECT v FROM DoctrineExtensions\Versionable\ResourceVersion v INDEX BY v.version ".
            "WHERE v.resourceName = ?1 AND v.resourceId = ?2 ORDER BY v.version DESC");
        $query->setParameter(1, get_class($resource));
        $query->setParameter(2, $resource->getResourceId());

        return $query->getResult();
    }
}

Now using this to retrieve all the versions of a given entity that is versionable you would go and:

<?php

// $em EntityManager, $blogPost my Blog Post

$versionManager = new VersionManager($em);
$versions = $versionManager->getVersions($blogPost);

echo "Old Title: ".$versions[$oldVersionNum]->getVersionedData('title');

// Create a new version
$blogPost->setTitle("My very new title");    
$em->flush();

This is a first example of how to use the powerful Doctrine 2 Event API. It is certainly not easy to use, as you need to understand the inner workings of the UnitOfWork and the different steps it is in during the flush process. However you can generate huge benefits in reusability.

The versionable behaviour could be extended by the following features:

  • Create a new interface Revertable that extends Versionable and add a method revert(Revertable $resource, $toVersion) to the VersionManager that handles the retrieval, invoking of revert and such.
  • Create a new interface Diffable with a method diff($aVersion, $bVersion) and new method diff(Diffable $resource, $aId, $bId) to the VersionManager that handles the delegation of a difference computation between two versions to the Diffable implementor.

Another approach would be not to save the complete state of an entity during the flush operation, but only the fields that changed. This is generally called an AuditLog. We could add an Auditable interface much in the same manner than the Versionable and retrieve the ChangeSets of each entity during flush using the following event listener:

<?php

class AuditListener implements EventSubscriber
{
    public function getSubscribedEvents()
    {
        return array(Events::onFlush);
    }

    public function onFlush(OnFlushEventArgs $args)
    {
        $em = $args->getEntityManager();
        $uow = $em->getUnitOfWork();

        $changeDate = new DateTime("now");
        $class = $em->getClassMetadata('DoctrineExtensions\Auditable\AuditEntry');

        foreach ($uow->getScheduledEntityUpdates() AS $entity) {
            if ($entity instanceof Auditable) {
                $changeSet = $uow->getEntityChangeSet($entity);

                foreach ($changeSet AS $field => $vals) {
                    list($oldValue, $newValue) = $vals;
                    $audit = new AuditEntry(
                        $entity->getResourceName(),
                        $entity->getId(),
                        $oldValue,
                        $newValue,
                        $changeDate
                    );

                    $em->persist($audit);
                    $em->getUnitOfWork()
                       ->computeChangeSet($class, $audit);
                }
            }
        }
    }
}

This approach can also be re-used or combined with several similiar behaviours, like Taggable, Blamable, Commentable.


Comments (14) [ add comment ]

thx Posted by Tom about 5 months ago.

Thx for reacting on the discussion about your last post. This very much helps in understanding what you as the devs consider best practise in dealing with behaviours and doctrine 2.

well done :-)

Great! Posted by Nico about 5 months ago.

Great post, thank you!

I have a question - what effect does the "computeChangeSet" call in VersionListener::_makeSnapshot have?

Great Work! Posted by Jonathan Nieto about 5 months ago.

Even though it seems a little more complicated, I'm glad to see the flexibility and the cleanness of this new architecture, leaving behind the intrusive architecture of doctrine 1.

Great work guys!

@Nico Posted by beberlei about 5 months ago.

The UnitOfWork computes the difference between the original and the current state of an object when "flush" is called. This helps to detect only those fields that changed, and only those entities that changed at all, which significantly enhances write performance.

When "onFlush" is triggered these computations already took place, that means you have to re-compute the change-sets in order for the UnitOfWork to recognize them. Otherwise the already calculated ones would be inserted into the database.

@Jonathan:

Hopefully we will have talented extensions writers that share their code so that as a developer you can pick from a set of well planned extensions of this kind. However i doubt that the DC1s Versionable behaviours code is as simple as this code here :-)

@Tom:

More posts in this fashion will follow in the next weeks and month.

Using SQL on Version table Posted by Johannes about 5 months ago.

I'm wondering if there is a way to have an entity be automatically generated as I don't see any way to use SQL on the versioned table as all the data is serialized and sitting in a single column?

@Johannes Posted by beberlei about 5 months ago.

You have to add the ResourceVersion class to your path that is checked by Schema-Tool to generate the SQL schema for this entity.

includeing behaviours within Doctrin 2 Posted by alexey_baranov about 5 months ago.

Thanks for the clarification. Event model - it is a great opportunity. With it we can easily implement our own models of behavior simply and elegantly. It is much cleaner than behaviours in the first version. I hope that the most popular behaviors will go within the Doctrine. Because this requires deep knowlige of Doctrine core and it is easier to write once the author than to force every user to write their own version.

Concerning Doctrine/Extension/Versionable: Is it possible to simplify the interface Versionable? Why can not determine the version of the model by the field labeled @version at run time instead of getCurrentVersion()? And the same @id instead getResourceId()?

And whether it is possible to determine a list of logged fields parsing annotations @column instead getVersionedData()? As EntityManager does while flushing. This mechanism will be resistant for models change.

And the interface IVersionable leave just as a marker of behaviour for

if ($entity instanceof Versionable){ //do versioning logic }

or remove it completely, replacing this marker by determine the field annotated with @version at run time.

Thanks.

@alexey Posted by beberlei about 5 months ago.

Your idea is certainly easy to implement for the Id and Version column and probably the next step for this extension. however I would go with the getVersionedData(), because in many cases you don't want to snapshot all the data, only relevant ones.

Code onGitHub Posted by beberlei about 5 months ago.

I released the modified code (tested, removed inconsistencies, plus added all the ideas from @alexey) on GitHub:

http://github.com/beberlei/DoctrineExtensions

Entity Filters Posted by David about 5 months ago.

Hi

I'm very interested in Doctrine 2 and I think it is a great step forward compared to Doctrine 1.

What I wanted to ask is if there is a mechanism, or will be in the future, to specify entity filters? For example it would be nice if one could add something like @Where("deleted <> 1") to an entity and this filter would be added to every select query.

Thanks and keep up the good work!

@beberlei Posted by alexey_baranov about 5 months ago.

Thank you for your understanding. Surprisingly quickly, which proves that with the help of events can be easily implemented behavior when you know how to use them.

The only thing it seems to me that if the VersionedManager do in class, it will be a child of a EntityManager, because VersionedManager is the entity manager, which has the additional ability to save all versions of objects IMHO.

Seems soon will see a whole library of extensions for the Doctrine for every taste. For us the most important and expected are trees and support inheritance.

Ideally, we would like to come to such a scheme: 1. download from the official repository (written and tested by experienced devs) extensions. 2. adjust entity manager as a way

import Doctrine\Extension\Versionable import Doctrine\Extension\Tree

$em=new EntityManager(); $em->addEventListrener(new VersionableEventSubscriber()); $em->addEventListrener(new TreeEventSubscriber()); Application::set('em',$em);

//doing our work with fun

Can you provide an example of entity annotation metadata parsing in you next post?

thanks.

code for last post Posted by alexey_baranov about 5 months ago.

import Doctrine\Extension\Versionable

import Doctrine\Extension\Tree

$em=new EntityManager();

$em->addEventListrener(new VersionableSEventSubscriber());

$em->addEventListrener(new TreeEventSubscriber());

Application::set('em',$em);

//doing our work with fun

@David Posted by beberlei about 5 months ago.

There exists the possibility already to hook custom sql walkers into the Query Hydration, i.e. for DQL Queries you can implement filters by modifing the DQL AST.

Its however not supported across the Persisters (where simple SQL queries are generated on demand). A general purpose Filter approach will probably have to wait for 2.1, but its certainly on our watch-list.

@alexey

We certainly want to support an extension repository with high quality plugins early on.

Relationships updates are not working Posted by anil about 3 days ago.

Hi,

i downloaded the versioning extension from github and working on it in our project.. everything is working fine except the relationship versioning. i.e if i update a users group or if i update users payment type id which is a relation in database it is just ignoring everything.

I hope you can explain about this

Create Comment