[DDC-930] A table cannot have more than one many to many relationship with the same table when using reverse engineer Created: 13/Dec/10  Updated: 13/Dec/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: Mapping Drivers
Affects Version/s: 2.0-RC2
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: Jiri Helmich Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None
Environment:

FreeBSD, PostgreSQL 8.4



 Description   

This is caused by taking the join column name as the identifier while generating a property name for annotation. The mapping driver detects that the same property is already defined and ends the convert process. A little bit smarter approach for me was to take the local table name. But this assumes a specific style of join table naming convention.

Doctrine\ORM\Mapping\Driver\DatabaseDriver::loadMetadataForClass()

Replace:

$associationMapping['fieldName'] = Inflector::camelize(str_replace('_id', '', strtolower(current($otherFk->getColumns()))));

With:

$name = explode("_",$myFk->getLocalTableName());
if (count($name) > 1)
{
array_shift($name);
}
$name = implode("_", $name);

$associationMapping['fieldName'] = Inflector::camelize(str_replace('_id', '', strtolower($name)));

Maybe to switch to this behavior with an additional option?






[DDC-919] subselect Created: 08/Dec/10  Updated: 20/Mar/11

Status: Open
Project: Doctrine 2 - ORM
Component/s: Documentation
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Documentation Priority: Major
Reporter: Mungiu Dragos Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None


 Description   

i'd like to see more example in documentation with this subselects
[23:08] <beberlei> can you open a tciket on jira? then i dont forget to do that when i have time



 Comments   
Comment by Alberto [ 20/Mar/11 ]

Subselect as columns or FROM clause should have mor examples.





[DDC-851] Automerge of detached entities passed to doctrine Created: 31/Oct/10  Updated: 30/Dec/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.0-BETA4
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: Daniel Alvarez Arribas Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None


 Description   

This is a feature request.

Currently it is not possible to assign a detached entity to a relationship. You have to manually "merge" it, and only then you are able to assign it to relationships of managed objects.

This can become complicated to do. The way it is now, when assigning an entity to a relationship in a process using a large number of entities, the entity's state needs to be checked and the entity possibly merged - all in userland code. This adds a level of complexity and potential for errors, while it could be solved transparently and elegantly within the ORM. There are ways to implement it in userland code, too, with moderate effort (see below), but this does not change the fact that responsibility for implementing a purely technical feature is delegated to the user, who could be spending his time much better writing business code. And if the user actually implements it, it will clutter the application with non-problem-domain code.

To keep things simple, I propose Doctrine be extended to simply auto-merge any detached entities passed to it. That would save the programmer the manual tracking of object states and merge() calls.

This would be especially handy when using cascades, as keeping track of deep object graphs in userland code would duplicate substantial ORM functionality.

In programs that work with massive amounts of data, it is practically impossible to keep all entities managed due to resource constraints (see e.g. the batch processing patterns documented in the Doctrine 2 reference at http://www.doctrine-project.org/projects/orm/2.0/docs/reference/batch-processing/en). In a situation like that, one would probably simply flush and clear the entity manager regularly. Doctrine 2 currently forces the user to manually "merge" all persistent objects he/she still holds references to and wants to assign e.g. to other newly created persistable objects. I can not think of any reason why Doctrine 2 should not be able to do it automatically.

Below is another comment originally attached to the GitHub proposal, containing a userland implementation of the feature as a temporary fix, for whoever cares.

Here is a userland implementation for the functionality I am proposing, though I feel it is technical clutter that belongs into the ORM. Changing doctrine to be able to auto-merge unmanaged entities would be ideal. I thought I'd share this, for use as long as Doctrine 2 does not provide equivalent functionality. The implementation assumes all entities inherit from a base class (named "YourEntityBaseClass here") and intercepts the assignment to ToOne-relationships in a __set() method provided in that base class. For ToMany-relationships we extend ArrayCollection to intercept calls to add() and set() to accomplish the same.

As an alternative to defining a __set() method in a base class you could also implement the interception by changing any mutator methods you define in your entities. But that would bloat your code quickly as you define more and more relationship attributes on your entities.

The following __set() method implementation relies on reflection to parse the DocBlock-Comment with the Annotation and determine whether or not the property to be set is a ToOne-relationship.

   public function __set($name, &$value) {
      
      $reflectionClass = new ReflectionClass($this);
      
      $property = $reflectionClass->getProperty($name);

      if (   self::isToOneRelationship($property)
          && $value !== null) {
            
         $value = self::mergeIfDetached($value);
      }
      
      $this->$name = $value;
   }

The following is an implementation of mergeIfDetached(), that assumes there is a __get defined on the entity, to be able to access the protected mapped properties.

public static function mergeIfDetached(YourEntityBaseClass $dataObject) {

   $doctrineEntityManager = DB::getDoctrineEntityManager();

   if ($doctrineEntityManager->getUnitOfWork()->getEntityState($dataObject) == \Doctrine\ORM\UnitOfWork::STATE_DETACHED) {
       
      $dataObject = $doctrineEntityManager->merge($dataObject);
   }

   return $dataObject;
}

For your purposes, consider DB to be just a class holding a reference to the Doctrine entity manager.

Here are the helper methods for the reflection:

   private static function isToOneRelationship(ReflectionProperty $property) {

      return self::matchDoctrineAnnotation($property, self::$doctrineToOneRelationshipAnnotation);
   }

   private static function matchDoctrineAnnotation(ReflectionProperty $property, $pattern) {
      
      return preg_match('/\@' . $pattern . '/', $property->getDocComment()) != 0;
   }

Here is the drop-in-replacement class for use with ToMany-Relationships. It uses the static reloadIfDetached method defined in the entity base class:

use Doctrine\Common\Collections\ArrayCollection;


class Collection extends ArrayCollection {
   
    public function set($key, $value) {
       
       $value = YourEntityBaseClass::mergeIfDetached($value);
       
       parent::set($key, $value);
    }


    public function add($value) {
       
       $value = YourEntityBaseClass::mergeIfDetached($value);
       
       return parent::add($value);
    }
}

This approach keeps the amount of unnecessary code to a minimum, so that merges are not scattered throughout the problem-domain code.



 Comments   
Comment by Daniel Alvarez Arribas [ 29/Dec/10 ]

I have to note that the code I listed above turned out to be broken. There is nothing that guarantees that a data object just merged will not become detached again after being merged on assignment, unless the object is immediately persisted afterwards.

The correct solution would be to merge all data objects found through relationships for a given data object, right from the persistence manager, immediately before calling persist() on the data object.

I am currently using this solution (save() saves a data object safely for use within long-running batch jobs):


   public static function save(DataObject $dataObject) {
      
      self::mergeRelatedDataObjectsIfDetached($dataObject);
      
      self::$doctrineEntityManager->persist($dataObject);
   }
   

   public static function merge(DataObject $dataObject) {
      
      return self::$doctrineEntityManager->merge($dataObject);
   }


  protected static function mergeRelatedDataObjectsIfDetached(DataObject $dataObject) {
      
      $reflectionClass = new ReflectionClass($dataObject);
      
      $properties = $reflectionClass->getProperties();
      
      foreach ($properties as $property) {

         $propertyName = $property->getName();
         
         $propertyValue = $dataObject->__get($propertyName);
         
         
         if (MetadataReader::isToOneRelationship($property)) {
            
            if (   $propertyValue !== null
                && ! $propertyValue instanceof Proxy
                && self::isDetached($propertyValue)) {
               
               $relatedDataObject = self::merge($propertyValue);
               
               $dataObject->__set($propertyName, $relatedDataObject);
            }
         }
         else {
            
            if (MetadataReader::isToManyRelationship($property)) {
               
               $relatedDataObjects = $propertyValue->toArray();
               
               foreach ($relatedDataObjects as $index => $relatedDataObject) {
                  
                  if ( ! $relatedDataObject instanceof Proxy
                      && self::isDetached($relatedDataObject)) {
                     
                     $relatedDataObject = self::merge($relatedDataObject);
                     
                     
                     // Replace the entry in the collection with the merged copy.
                     
                     $propertyValue->set($index, $relatedDataObject);
                  }
               }
            }
         }
      }
   }
   
   
   protected static function isDetached(DataObject $dataObject) {
      
      return self::$doctrineEntityManager->getUnitOfWork()->getEntityState($dataObject) == UnitOfWork::STATE_DETACHED;
   }

I still wish there would be an automerge feature, kind of Hibernate's "update".

Comment by Daniel Alvarez Arribas [ 29/Dec/10 ]

Wrapped the code sections into proper code blocks...





[DDC-810] Issue with detaching entities and updating when using change notification Created: 17/Sep/10  Updated: 04/Jul/11

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: 2.0-BETA4
Fix Version/s: 2.x
Security Level: All

Type: Improvement Priority: Major
Reporter: Jonathan H. Wage Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None

Attachments: File DDC810Test.php    

 Description   

More information coming soon. Attaching a test case



 Comments   
Comment by Benjamin Eberlei [ 20/Sep/10 ]

From reading the issue i know what the bug is, indeed this sucks.

Comment by Roman S. Borschel [ 28/Sep/10 ]

@Jon: Any more information coming?

@Benjamin: Can you summarize the essence of the issue shortly?

Comment by Benjamin Eberlei [ 29/Sep/10 ]

@Roman: The UnitOfWork (may) still be pushed as a listener into that entity, and still recieve noticies of update. Which may throw notices because the oid hashes are removed everywhere. Additionally you cant serialize the thing because you still got the UoW inside there.

Comment by Jonathan H. Wage [ 04/Oct/10 ]

I don't have anymore information currently. The issue was relayed to me. I will try and find some more information and report back.

Comment by Benjamin Eberlei [ 03/Apr/11 ]

There is no way to "fix" this issue, i am turning it into a feature request. There needs to be a "postDetach" event that is triggered where the developer can detach the change notification objects.





[DDC-803] Create subselect queries within join statements Created: 14/Sep/10  Updated: 14/Sep/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: New Feature Priority: Major
Reporter: Martijn Evers Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None





[DDC-785] Post-Post-Persist event Created: 02/Sep/10  Updated: 14/Jan/11

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.0-BETA4
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: arnaud-lb Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None


 Description   

postPersist/postUpdate events are triggered in the middle of a unitOfWork, and querying the DB in such events causes infinite loops. Doctrine attempts to flush the entity manager before running any query, which triggers flushing of entities, and postPersist/postUpdate events are triggered again.

I did not checked, but the flush() before each query may be a performance problem too, if doctrine has to determine what has changed, depending on the changetracking policy.

Also, it would be great if postPersist / postUpdate events were triggered after all entities have been persisted. It looks like that entities are flushed by groups of same 'type', and that events for a type are triggered once all of the elements of that group have been flushed, potentially before entities of an other type have been flushed : postPersist / postUpdate events are triggered while some other entities are still not flushed.



 Comments   
Comment by Benjamin Eberlei [ 03/Sep/10 ]

That is documented and for perfomance reasons we cannot move the preUpdate/postUpdate/prePersist/postPersist events to other locations inside the UnitOfWork.

There is an onFlush event that allows for more flexibility and is triggered before any update/insert/delete is done by the UnitOfWork.

Comment by arnaud-lb [ 04/Sep/10 ]

Thanks.

I understand that. Is there any chance of getting some onPostFlush or similar, which would be triggered like onFlush, but after all update/insert/delete ? Or just some post-something event which is allowed to issue db queries.

Comment by Gediminas Morkevicius [ 24/Sep/10 ]

onFlush you can store your entity for furher processing and on postPersist you can check if there are no more insertions and process the entity if it needs additional query
I have faced all these issues and you can check http://github.com/l3pp4rd/DoctrineExtensions/tree/master/lib/DoctrineExtensions/Translatable/
for a solution to your problem

Comment by Gediminas Morkevicius [ 14/Jan/11 ]

I think this issue should be closed since the main reason of opening it was the possibility to execute additional queries when inserts were pending in unit of work. In current release it does not cause a flush during an additional query execution anymore.





[DDC-776] Persisters use a fixed "SELECT" SQL statements Created: 29/Aug/10  Updated: 23/Apr/13

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: 2.0-BETA3
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: Aaron DM Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Windows 7, Apache 2.2, MSSQL Server, PHP 5.3.3



 Description   

I am currently trying to work with BINARY columns with Doctrine 2 and MSSQL. In order to get my Entities working I had to create a custom Mapping Type for Binary columns. All went well in this case and I've got it running.

The problem arises when I am attempting to use Associative mapping (OneToOne/ManyToMany). The problem is, in order to do a select for an SQL column, I had to create a DQL function called "CONVERT" so that I use WHERE statements:

return $this->createQueryBuilder('u')
->where("u.id = CONVERT('binary', :id, 1)")
->setParameter('id', $id)
->getQuery()
->getSingleResult();

As you see, I must do this in order to get a result.

However, when I'm using associative mapping; this is what it does:

return 'SELECT ' . $this->_getSelectColumnListSQL()
. ' FROM ' . $this->_class->getQuotedTableName($this->_platform) . ' '
. $this->_getSQLTableAlias($this->_class->name)
. $joinSql
. ($conditionSql ? ' WHERE ' . $conditionSql : '')
. $orderBySql
. $lockSql;

As you can see, its some what hard coded and I cannot change it without changing the actual code in
Doctrine\ORM\Persisters\BasicEntityPersister.php

So, I would first like to know if there was maybe a way you could allow us to customize the SELECT statement that the persisters use - or maybe (though I'm not sure how this will be done) make them use user-defined repository functions?

Like $myRepo->find($identifier)

Not entirely sure if I explained this properly and I do realize my circumstance is highly odd - but this does seem like a limitation and because of this I cannot use associative mapping.



 Comments   
Comment by locs [ 23/Apr/13 ]

Hi, i try to make my custom type for binary field in MSSQL.
I don't find own, can you please show me your custom type binary?
Thks a lot.





[DDC-779] Doctrine\ORM\Configuration should be immutable after construction of EntityManager Created: 30/Aug/10  Updated: 30/Aug/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.0-BETA3
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: Benjamin Eberlei Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Currently the Doctrine\ORM\Configuration instance is not immutable after construction of the EM, which can lead to funny behavior when changing essential dependencies such as caches or others.






[DDC-769] Disabling discriminator column in WHERE clause Created: 26/Aug/10  Updated: 07/Sep/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: 2.0-BETA3
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: Lars Strojny Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 1
Labels: None


 Description   

Per default Doctrine 2 adds an IN(...)-part to the query when hydrating an entity where a discriminator column is defined. While this makes sense as a default behavior, it would be pretty helpful if one could disable the WHERE-clause for discriminator columns alltogether for performance optimization.



 Comments   
Comment by Roman S. Borschel [ 26/Aug/10 ]

That would obviously produce wrong results. Maybe you can elaborate more with an example.

Comment by Lars Strojny [ 07/Sep/10 ]

I use ENUM("foo","bar") as discriminator columns. That means, the column will contain the right values out of the box, no further result set limiting required with WHERE.





[DDC-688] Original Entity Data gets overridden by the change set Created: 12/Jul/10  Updated: 28/Dec/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.0-BETA2
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: Jasper Kuperus Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Mac OS X 10.6; PHP 5.3.2; MySQL 5.1.44



 Description   

When changing data in an entity, the UnitOfWork will call computeChangeSet on a flush event. If there is a changeset, the original data ($this->_originalEntityData) gets overridden by the new data. However, the _originalEntityData should hold the original data, that was present at the time the entity was reconstituted from the database. This does no longer hold now.

I think this can simply be fixed by commenting this line, however I do not know of any consequences this may bring with it:

$this->_originalEntityData[$oid] = $actualData; (in computeChangeSet, after if( $changeSet ));

Anyway, I ran into this problem while trying to retrieve the original data at the onFlush event of an update.



 Comments   
Comment by Roman S. Borschel [ 08/Aug/10 ]

This is actually currently expected. You can not get access to the original data in the onFlush event right now. I'm not saying that this will never be possible but it is simply the way it works at the moment.

Comment by Jasper Kuperus [ 08/Dec/10 ]

Does this mean that it is currently impossible to implement a Versionable mechanism using snapshots?

Comment by Benjamin Eberlei [ 09/Dec/10 ]

You can hold a map of them yourself if your listener also implements the "postLoad" event:

$entity = $args->getentity();
$this->originalData[spl_object_hash($entity)] = $args->getEntityManager()->getUnitOfWork()->getOriginalData($entity);
Comment by Benjamin Eberlei [ 28/Dec/10 ]

Changed into possible improvement for the future





[DDC-678] OneToMany/OneToOne + onDelete=CASCADE may corrupt UoW. Created: 10/Jul/10  Updated: 05/Jun/11

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: Roman S. Borschel Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None


 Description   

OneToMany/OneToOne associations together with an onDelete=CASCADE schema generation hint on the @JoinColumn and appropriate foreign key constraints can potentially result in a corrupt UoW if the associated objects are already managed. We need to add tests for such scenarios and settle on a well-defined behavior in such cases.



 Comments   
Comment by Benjamin Eberlei [ 31/Oct/10 ]

I think to preserve the semantics the following has to happen:

"on-delete" => "cascade" has to implicitly set cascade = remove. This hurts performance of course vs just using the on-delete, however it won't corrupt the UoW.

Comment by Benjamin Eberlei [ 02/Jan/11 ]

Not entirely would it hurt performance, you could check if on-delete => cascade is set. If this is the case you wouldnt need to do an explicit remove using the UnitOfWorks cascade.

Comment by Benjamin Eberlei [ 05/Jun/11 ]

Changed to improvement





[DDC-624] Partial object query that leaves out an association to avoid loading it fetches the association anyway. Created: 03/Jun/10  Updated: 11/Nov/11

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: 2.0-BETA1
Fix Version/s: 2.x
Security Level: All

Type: Bug Priority: Major
Reporter: Roman S. Borschel Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 2
Labels: None

Issue Links:
Duplicate
is duplicated by DDC-1465 Fetching partial objects doesn't work... Open

 Description   

Assuming:

Customer <onetoone> Cart

where Cart is the owning side.

Since the association from Customer to Cart can not be lazy, it would make sense to leave out the association in a query to avoid loading the carts like this:

select partial c.{id,name, ... anything except cart} from Customer c"

But this is ignored and the carts of all customers are fetched anyway. Query::HINT_FORCE_PARTIAL_LOAD is an alternative solution, however it has the disadvantage that it disables lazy-loading for all queried objects. If partial querying would honor associations this would allow more fine-grained control.



 Comments   
Comment by Roman S. Borschel [ 26/Aug/10 ]

Might need to be pushed back to a 2.0.x / 2.x.x bugfix release. Not clear yet.





[DDC-349] Add support for specifying precedence in joins in DQL Created: 18/Feb/10  Updated: 01/May/13

Status: Open
Project: Doctrine 2 - ORM
Component/s: DQL
Affects Version/s: 2.0-ALPHA4
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: Dennis Verspuij Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 1
Labels: None

Attachments: Text File DDC349Test.patch    
Issue Links:
Duplicate
is duplicated by DDC-1256 Generated SQL error with DQL WITH and... Resolved

 Description   

This request is in followup to my doctrine-user message "Doctrine 2.0: Nested joins'.
I am a bit surprised by the responses in that defining precedences in joins by placing parenthesis around join expressions is not well-known. Although not in the original SQL92 specification it is a major and important feature offered by all the RDBMS's that Doctrine 2 supports, and oftenly performs better than using subselects or alike. Doctrine 1 did not support it, but imho Doctrine 2 should support it to be a mature allround ORM.

As a short example the following is a SQL statement with a nested join, where the nesting is absolutely necessary to return only a's together with either both b's and c's or no b's and c's at all:

SELECT *
FROM a A
LEFT JOIN (
b B
INNER JOIN c C ON C.b_id = B.id
) ON B.a_id = A.id

In order for Doctrine 2 to support this the BNF should be something like:
Join ::= ["LEFT" ["OUTER"] | "INNER"] "JOIN" ( "(" JoinAssociationPathExpression ["AS"] AliasIdentificationVariable Join ")" | JoinAssociationPathExpression ["AS"] AliasIdentificationVariable ) [("ON" | "WITH") ConditionalExpression]
instead of the current:
Join ::= ["LEFT" ["OUTER"] | "INNER"] "JOIN" JoinAssociationPathExpression ["AS"] AliasIdentificationVariable [("ON" | "WITH") ConditionalExpression]

This would allow DQL like:

SELECT A, B, C
FROM a A
LEFT JOIN (
A.b B
INNER JOIN B.c C
) WITH B.something = 'value' AND C.something = 'othervalue'

What further needs to be done is that the DQL parser loosly couples the ConditionalExpression to any of the previously parsed JoinAssociationPathExpression's instead of tieing it explicitely to the JoinAssociationPathExpression that preceedes it according to the old BNF notation. The new BNF should however not require any changes to the hydrator. Therefore I have the feeling that improving the DQL parser for nested joins does not require extensive work, while the benefit of running these kind of queries is considerable.

As an extra substantiation here are links to (BNF) FROM clause documentations of the RDBMS's that Doctrine 2 supports, they all show support for nested joins:
MySQL: http://dev.mysql.com/doc/refman/5.0/en/join.html
PostgreSQL: http://www.postgresql.org/docs/8.4/interactive/sql-select.html#SQL-FROM and http://www.postgresql.org/docs/8.1/interactive/explicit-joins.html
MSSQL: http://msdn.microsoft.com/en-us/library/ms177634.aspx
Oracle: http://download.oracle.com/docs/cd/E11882_01/server.112/e10592/statements_10002.htm#CHDDCHGF
SQLite: http://www.sqlite.org/syntaxdiagrams.html#single-source

I surely hope you will consider implementing this improvement because it would save me and others from the hassle of writing raw SQL queries or executing multiple (thus slow) queries in DQL for doing the same. Thanks anyway for the great product so far!



 Comments   
Comment by Guilherme Blanco [ 13/Apr/10 ]

This seems to be a valid issue to me.

This implementation is the actual solution to associations retrieval that are inherited (type joined).

Example:

/** Joined */
class Base {}

class Foo extends Base {}

class Bar {
    public $foo;
}

// This causes the CTI to link as INNER JOIN, which makes the result become 0
// il if you have no Foo's defined (although it should ignore this)
$q = $this->_em->createQuery('SELECT b, f FROM Bar b LEFT JOIN b.foo f'); 
Comment by Roman S. Borschel [ 13/Apr/10 ]

Yes, this is a possible solution for DDC-512 but on the SQL level. I still don't see this as appropriate for DQL, it just doesnt make sense to me, DQL joins object associations, there is no precedence.

Comment by Roman S. Borschel [ 13/Apr/10 ]

So, no, this has nothing to do with DDC-512. DDC-512 can even be fixed differently as outlined in my comments there.

Comment by Roman S. Borschel [ 13/Apr/10 ]

On a side note I would still like to know/see the following for this issue:

  • Some realisitic DQL examples where this feature would be essential, i.e. there is no other way to do it.
    This also means explaining what the impact on the resulting object graph is and why it makes sense.
  • Which other ORMs support this on the OQL/Criteria level?

So far, my stance on this issue is:

1) It doesnt make sense (semantically) in DQL
2) Its rarely needed
3) When you really need it you can use a NativeQuery anyway and use this nesting in SQL, where it probably belongs and makes more sense
4) It would (unnecessarily) complicate DQL

Thus I am currently leaning towards "Wont fix" for this issue.

Comment by Dennis Verspuij [ 13/Apr/10 ]

Hi Roman. I understand your doubts, and I have been breaking my head over
creating a realistic example the last few hours that would hopefully convince
you for implementing this feature. But actually I cannot find one that you wouldn't
consider to be trivial. I do have a number of very complex optimized queries written
for sportskickoff dot com (using Doctrine 1.2) but they are probably hard to understand
because they may not be selfdescribing. Below is one example literally ripped from
the application. Still they often can be broken down to my example query in this
ticket's description, but applied grouping, additional other joins on the root component
and/or other criteria made them impossible to rewrite using subselects or choosing
another root component. Most often they just performed way best using the nested
syntax and saved me a number of additional queries.

SELECT A.id, A.username, A.balance, COALESCE(SUM(B.stake), 0) AS sumstake, COUNT(B.id) AS nrbets
FROM account A
LEFT JOIN (
bet B
INNER JOIN game G ON G.id = :GAMEID AND B.timestampcompletion BETWEEN G.timestampstart AND G.timestampend
) ON B.accountid = A.id AND B.timestampcompletion IS NOT NULL
WHERE A.Status & :ACTIVEORDISQUALIFIED = :ACTIVE
GROUP BY A.id, A.username, A.balance
ORDER BY A.balance DESC, sumstake ASC, nrbets ASC, A.username ASC

But let's put it another way. I would also like this feature to be supported in DQL
because I just do not want to use native queries. Why would I want to use native
queries if it can be done using DQL? In DQL I work with class names and field
names, and they may differ from the underlying table and column names. Doctrine
takes care of that mapping based on my schema/annotations and I do not
have to "know" these mappings. In native queries I suddenly do have to "know"
these mappings. I use Doctrine because it makes my application portable and
enables me to work with my database in an OOP way like I do in my model,
abstracting things. The need for native queries partly reverts the benefits Doctrine
offers in the first place.

Btw, I recall to have successfully used the nested join syntax in HQL (.NET Hibernate)
but I cannot find examples on the web or a BNF notation.

Furthermore, in reply to your stances:
1) It indeed doesnt make sense (semantically) in DQL, it only makes the result
set different, but not the way data is hydrated into objects;
2) Its indeed rarely needed for inserting, updating and populating basic lists but
it allows you to better select what combinations of associated rows are joined
and which not in more optimized queries without having to use native queries,
or because they perform better than using subseletcs and alike.
3) Not having to use native queries is just an extra reason for using Doctrine and
maintains the abstraction the ORM provides througout on'es whole application
4) Why would it complicate DQL, if people do not know about or understand
the feature it wouldn't matter because not using parenthesises is the default
way to specify joins?

Well, this is it, can't find any more words to promote and make you enthusiastic.... lol.

Comment by Dennis Verspuij [ 13/Apr/10 ]

Ok, I have not given up yet... , here's a "stupid" example.

Imagine a book store that sells books of various authors and keeps track of those sales.
Let's say you would have an admin page that lists all authors, and for each author
its also shows the books and their sales dates since january 1st, but only for those
books that were actually sold and contain an A in its name. An optimized SQL query
to fetch all the information at once would be something like:

SELECT A., B., S.*
FROM author A
LEFT JOIN (
book B
INNER JOIN sale S ON S.book_id = B.id AND S.dt >= '2010-01-01'
) ON B.author_id = A.id AND A.name LIKE '%A%'

In DQL it would then be something like:

SELECT A., B., S.*
FROM author A
LEFT JOIN (
book B
INNER JOIN sale S WITH S.dt >= '2010-01-01'
) WITH A.name LIKE '%A%'

If the database would contain thousands of books, but sales for just a
few books, this will definitely perform better than using subselects.
Off course one would like to fetch array graphs instead of objects for
further optimization, but this hopefully shows my point.

I have attached a test casefor a similar query, though without the additional
join constraints for clarity. I surely hope you can consider it.

One last note, you shouldn't be afraid that nesting joins is not in the
ansi SQL spec. Select queries are about record sets and products
between these sets, tables are just the basic means of providing record
sets to the query. This is an important terminological difference to think about.
Specifying precedence with parenthesis around joins is a logical and
natural evolution of the ansi sql standard. For example views are a good
proof of this concept, I could define book B INNER JOIN sale S as a view
and LEFT JOIN that to authors to get effectively the same result
set as the above example. The database server would internally perform the
same query (though may additionally take indexes on the view into account).
That said, rdbm's that support this syntax would certainly never drop the
feature, as its not a feature but just plain logical and smart querying!

P.S. I had a hard time finding out how to run the test cases, I could not find
it in the Doctrine 2 documentation, development wiki, cookbook or any other
place, while finally it was as easy as running phpunit Doctrine_Tests_AllTests
from within the tests/ directory, or just phpunit Doctrine_Tests_ORM_Functional_Ticket_DDC349Test
for my test. Could you please add some info about this somewhere, it might
save others some googling.

Comment by Dennis Verspuij [ 13/Apr/10 ]

Test case as SVN patch using a parenthesized join.
Just remove the parenthesises from the query to have it fail...

Comment by Roman S. Borschel [ 29/May/10 ]

@"The need for native queries partly reverts the benefits Doctrine offers in the first place."

That is something I hugely disagree with. Neither SQL abstraction, nor database vendor independence is the main purpose of an ORM like Doctrine 2.
It is the state management of your objects, the transparent change tracking, lazy-loading and synchronization of the object state with the database state and nothing of this gets lost when using native queries.

We could rip out DQL and any other querying mechanism except a basic find() (and lazy-loading, of course), only providing the native query facility and even only supporting MySQL and would still retain all the core ORM functionality.

NativeQuery is one of the best and core "features" of the project. It is even the foundation for DQL. A DQL query is nothing more than an additional (beautiful) abstraction but what comes out is a native query + a ResultSetMapping, the same thing you can build yourself in the first place, even using the mapping metadata to construct the query. Nothing forces you to hardcode table and column names in native queries if you don't want that. Just use the mapping metadata, DQL does the same.

SQL abstraction and database vendor independence is icing on the cake, not the heart of the ORM.





[DDC-265] Possibility for Nested Inheritance Created: 21/Jan/10  Updated: 16/Jan/13

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: New Feature Priority: Major
Reporter: Michael Fürmann Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Duplicate
duplicates DDC-138 Allow for mixed inheritance mapping Open

 Description   

It would be great if Doctrine had the possibility to define a further inharitance in a subclass.

Example:
There is a class DataObject managing things like created- and lastedit-
timestamps, archiving objects before updates, ...
One of the sub-objects is Content.
There are several types of content.
Written directly to a database field, read from a textfile on server,
executed php file on server, loaded from another server via xmlrpc and
so on.

I'd like to use a single table inheritance to map all information of
the different content objects in one table.
If I understand the model right the only alternate solution would be
to write each single content object to the discriminator map of
DataObject.



 Comments   
Comment by Benjamin Eberlei [ 21/Jan/10 ]

The DataObject you describe is a no-go for Doctrine 2. Its just a very bad practice.

Inheritance Mapping is for REAL inheritance only, otherwise you shouldnt go with a relational database in the first place.

You should use the Event system for such changes, it offers you roughly the same possibilities and keeps you from having to use inheritance mapping. You could still create an abstract data object and define the fields that will be used in each "implementation" and then in events do something like:

if ($entity instanceof DataObject) {
     $entity->updated();
     $archiver->makeSnapshot($entity);
}
Comment by Jonathan H. Wage [ 20/Mar/10 ]

With this patch I think you could setup a nice similar model where you can introduce new children of this parent class and have it added to the discriminator map from the child instead of having to modify the parents mapping information.

http://www.doctrine-project.org/jira/browse/DDC-447





[DDC-128] Consider adding EntityManager#link/unlink methods for direct association manipulation Created: 07/Nov/09  Updated: 29/Dec/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.0-ALPHA2
Fix Version/s: 2.x
Security Level: All

Type: New Feature Priority: Major
Reporter: Roman S. Borschel Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 1
Labels: None

Issue Links:
Reference
is referenced by DDC-546 New fetch mode EXTRA_LAZY for collect... Resolved

 Description   

A problem when working with collection-valued associations is that almost all operations except add($obj) require the collection to become initialized in order for the operation to be performed properly. While this is all correct and beautiful OO-wise it may be problematic at times with regards to performance. Hence we might want to consider to provide some convenient methods along the lines of link/unlink (name suggestions?) which allow more direct, less OO collection manipulation. Such methods obviously would bypass the normal object lifecycle and the changes done through these methods will not be reflected in the in-memory objects and collections, unless the user keeps them in-synch himself.



 Comments   
Comment by Benjamin Eberlei [ 11/Dec/09 ]

Questions

  • I suppose link and unlinked entities would then handled by UnitOfwork commit also?
  • Since the collection is not initialized, one does not know upfront if the action will be successful, what happens if:
    • an entity is linked with a collection, although they are already connected.
    • an entity is unlinked from a collection it is not in.

Regarding the naming, i like link/unlink.

Comment by Roman S. Borschel [ 17/Dec/09 ]

What do you mean by "handled by UnitOfWork commit" ? Whether the SQL is "scheduled" or executed immediately? Interesting question.
Scheduling would probably be better but also more difficult.

As far as usage is concerned, I currently imagine it as follows:

// EntityManager#link($sourceObj, $field, $targetObj)
$user = $em->getReference($userId); // $userId probably from request parameters
$address = $em->getReference($addressId); // $addressId probably from request parameters
$em->link($user, 'addresses', $address);

"What happens if: an entity is linked with a collection, although they are already connected."

Probably an SQL error which results in an exception from the driver. Depends on the database constraints though.

"What happens if: an entity is unlinked from a collection it is not in"

Probably nothing, at least not from the SQL side. An exception could be thrown from Doctrine itself if the update affected 0 rows.

Thanks for these initial questions. Thats definitely food for thought. Keep it coming.

Comment by Roman S. Borschel [ 26/Aug/10 ]

Pushed back.





[DDC-138] Allow for mixed inheritance mapping Created: 12/Nov/09  Updated: 24/Dec/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: DQL, Mapping Drivers, ORM
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: New Feature Priority: Major
Reporter: Reinier Kip Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 3
Labels: None

Issue Links:
Duplicate
is duplicated by DDC-265 Possibility for Nested Inheritance Open

 Description   

Requesting implementation of mixed inheritance mapping (class table inheritance and single table inheritance).

This would be especially handy when the difference between certain classes is only "implementational" (i.e. a subclass only functions differently/implements abstract methods and does not specify any additional fields). Using class table inheritance would result in tables only containing an id column.






[DDC-213] Persist order of collections Created: 15/Dec/09  Updated: 27/Jun/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: 2.0
Fix Version/s: 3.0
Security Level: All

Type: New Feature Priority: Major
Reporter: Roman S. Borschel Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 15
Labels: None

Issue Links:
Duplicate
is duplicated by DDC-181 Order of many-to-many relationship Resolved
Reference
is referenced by DDC-250 ArrayCollection Key Column @indexBy Resolved

 Description   

A Collection is like a php array, an ordered map. Hence there should be the possibility to persist this order.



 Comments   
Comment by Christian Heinrich [ 21/May/10 ]

Roman, I'd like to do this one as I have currently a use case for this. Do you have any idea of how to do this? What I'm wondering is whether it is possible to implement this without user intervention. (This would simply mean "store the entities as they were added"). But this would need another column in DB that we'd have to add within oneToMany / manyToMany relationships, but in this case one could save a serialized array holding "entityId => position" key / value pairs.

Afterwards, one could easily rebuild / reorder the collection via $collection->set($entity, $order[$entity->identifier]);

If you've got another thought about this, please don't hesitate to point me into the right direction!

Comment by Benjamin Eberlei [ 22/May/10 ]

this won't be implemented until 2.1, since its a pretty complex feature.

Changes are probably required in:

1. CollectionPersister - Add a new collection persister that takes the position into account
2. SchemaTool - Add a 'col_position' column to either the many-to-many or the one-to-many tables.
3. EntityPersister - Use and extend current order-by support to make the sorting happen

You can implement this already though with some performance hit in update scenarios. If you use the ORDER BY support and implement an API around your entity that abstracts those changes and always sets a "position" field on the many entity that is supposed to be sorted.

Comment by Roman S. Borschel [ 22/May/10 ]

I don't think we necessarily need a new collection persister. Simply adjusting the ManyToManyPersister to be able to deal with it might be sufficient.

For OneToMany, that is always persisted from the "many" side, thus there is no collection persister, we would need to adjust the normal persisters.

They key element for the user should be a new annotation (or corresponding xml/yaml element) @OrderColumn. By default the order should not be persistent, only when an @OrderColumn annotation is present. The name of the order column can have a default, i.e. "position". Thus this enhancement of persisting the order should be fully backwards compatible.

Comment by Roman S. Borschel [ 22/May/10 ]

On another note, the getInsertDiff/getDeleteDiff methods of PersistentCollection should already be "ready" for this. That is, when an element in the collection changed only its position, this is already tracked as a change. However the ManyToManyPersister issues no "UPDATE" queries, it simply deletes and inserts. A position change may be more effectively persisted with an UPDATE.

Comment by Benjamin Eberlei [ 30/Sep/10 ]

From a mailinglist entry, required check/changepoints:

1. ClassMetadata of Many-To-Many associations have to be extended to publish the required datastructure to the ORM.
2. All Metadata Mapping Drivers have to be extended
3. Persisters\ManyToManyCollectionPersister has to be extended to save the key in the many to many table if desired by the user.
4. Schema-Tool has to be extended to create the additional column.
5. PersistentCollection has to be extended so that lazy loading of collections with additional key works.
6. Array- and ObjectHydrator have to be extended to allow fetch join of collections with key column.
7. Discuss wheather to support this for One-To-Many also with the key-column on the many side. This is much more tricky internally though.

Comment by Benjamin Eberlei [ 24/Dec/10 ]

Push back to 2.x, we will have support for DDC-250 first and for this at a later release.

Comment by Thomas Tourlourat - Armetiz [ 07/Feb/12 ]

Hi there,
I'm looking for this feature.

Benjamin Eberlei said that : "You can implement this already", but I don't understand the "how to".

Also,
The problem should be solve if RDBMS had a "natural" order. An order based on item position inside table.

To get this feature without any change on Doctrine, I have remplace the PK defined by the target & mapped field identifier. The new PK is a new field with type "integer" and with auto-increment enable.
In this configuration, Doctrine use the "natural" order of the RDBMS. And I can change order of my item inside Collection and persist it.

It's an very bad solution, but It work before an official support.

Waiting for advices, and solutions,

Thomas.

Comment by Thomas Tourlourat - Armetiz [ 08/Feb/12 ]

Answering to Benjamin Eberlei on the "7. Discuss wheather to support this for One-To-Many also with the key-column on the many side. This is much more tricky internally though.".

I think that for One-To-Many relations, if user want to store the collection order, Doctrine can store the One-To-Many as Many-To-Many with a "model" limitation.
In that case, if storing order collection for Many-To-Many work, it should work for One-To-Many.

What do you think about it ?

Comment by Nicolas [ 29/Feb/12 ]

I think that it must be possible to have two keys ordering : the order isn't obligatory reversible.

For exemple with user and group :

  • You can order groups for one user : with preference by exemple, or importance.
  • And with a different order, users for a group : rank by example.

And maybe more, if you decide to add multi-order : an user show group by his rank in it, if his rank is identical, the order is make by love preference, and after by the importance given by the user (not necessary a number, if we imagine filter on them).

So a default order can be choice with parametized fields and could be :

 
@ManyToMany(targetEntity="Group")
...
@JoinFields ( rank: { type: int} , preference:{type:int}, importance:{type: string, length: 40} )
@OrderByJoinFields({"rank" = "ASC", "preference"="ASC", "importance"="ASC" } )

In this case the order must be optional and would be clean if another order appears in the same scope (DQL...). And manytomany became virtual entities act as other entities except they don't appears permetting in the same time a better conception.

So if the solution take in DDC-181 will become the only solution. This would a good idea to document this. Because, this seems to me a very important point.

My last point is even an unique ordering field created in the join table will be a big and usefull improvement.

Thank a lot for your beautiful work.

Comment by Thomas Tourlourat - Armetiz [ 29/Feb/12 ]

In my point of view, a collection can be order in a single way only.
If you want to add more than one order between User & Group, it's a new collection, a new relation.
Like :

  • User.memberOf() : Group[]
  • Group.members() : User[]
  • Group.importantMembers() : User[]

And it's your role to keep a consistency between members & importantMembers array.

Because ManyToMany join table is the reflection of a state of an ArrayCollection. It's not a usefull feature to be able to store all of the state of an ArrayCollection, even the order of this Array. It's just a normal feature that is really missing

Thomas.

Comment by Nicolas [ 29/Feb/12 ]

I don't think:

If you have three collection, you duplicate one relation 3 times and it's easy in consequence to lost the data integrity and unicity.

By example :

  • Thomas have rank 10 in Admin
  • Thomas think the admin group has importance noted 3 on all of his groups.
  • If a responsable of admin group decide to delete Thomas from it.
  • Thomas, in his ordered list of groups, think always to be in group admin.

So in my idea, the many to many relation isn't just an array collection, but should be an virtual entity. In UML or in Merise method this is a common problem to have a parametized relation. I think an orm should just implement this.

Comment by Thomas Tourlourat - Armetiz [ 29/Feb/12 ]

Hum,
I agree with you.. In a SQL Schema, it's a good choice to add many fields in a ManyToMany join table to description "order".

Comment by Thomas Tourlourat - Armetiz [ 07/Mar/12 ]

I just want to add a piece of Doctrine ORM Document :

"When working with collections, keep in mind that a Collection is essentially an ordered map (just like a PHP array). That is why the remove operation accepts an index/key. removeElement is a separate method that has O ( n) complexity using array_search, where n is the size of the map."

Comment by Thomas Tourlourat - Armetiz [ 23/Mar/12 ]

Hi there,
After several discussions. on IRC, I have changed my point of view.

Doctrine Documentation says : "When working with collections, keep in mind that a Collection is essentially an ordered map (just like a PHP array)".
So, I think that Doctrine have to be able to store or not the order of a Collection. By adding a new field on the Joined table to store the position of each elements.

But I not agree with @Nicolas. Because in his case, he's talking about Association Class : http://etutorials.org/Programming/UML/Chapter+6.+Class+Diagrams+Advanced+Concepts/Association+Class/
Because he's talking of a business logic, he's talking of a dedicated Entity class.

What do you think about it ?

Thomas;

Comment by Thomas Tourlourat - Armetiz [ 31/Aug/12 ]

Any news ?

Comment by Matthieu Napoli [ 16/Oct/12 ]

Hi, any news on this?

If I may add any info to this feature request, maybe something like JPA/Hibernate could be a good start?

See http://docs.oracle.com/javaee/6/api/javax/persistence/OrderColumn.html or http://docs.jboss.org/hibernate/core/3.6/reference/en-US/html/collections.html#collections-indexed

The idea in Hibernate is that you persist the order of the list in an extra column. This column is not a field of the entity however.

Comment by Albert Casademont [ 27/Jun/14 ]

+1, this would be indeed a very nice thing. We are using the @OrderBy annotation as a workaround but it's not quite the same thing

Comment by Marco Pivetta [ 27/Jun/14 ]

Moved to 3.x





[DDC-586] Repo does not find "unflushed" object Created: 14/May/10  Updated: 26/Aug/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: 2.0
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: John Kleijn Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 1
Labels: None


 Description   

The problem is this:

$bar = new \entity\content\ContentTag();
$bar->setName('bar');
$em->persist($bar);

$existingTag = $em->getRepository('entity\content\ContentTag')->findOneByName('bar');

Seeing as in EntityRepository "find()" queries the Unit of Work first, and "findBy()" goes directly to the persister, only remotely stored objects will be found. Now if I want a tag object to attach related tags, it would have to query by name to see if an object already exist, BUT it wont find one as the UoW has not been committed, resulting in a new one being created, ultimately resulting in a PDO error on the unique name constraint.

This can be "solved" by inserting a flush, but it is impossible to know whether a flush is required, without knowledge of what comes next. I.e. for one part to know it has to flush, it has to know another wants to fetch an object you just created.

This causes an unacceptable amount of coupling.

Somehow the repo will have to be able execute DQL against the objects in the UoW. This does not have to be full support (straight away), but it should fail (throw an exception) if the possibility exists that the UoW contains items that are excluded (e.g. the operation is not supported and the UoW still contains items).

For right now, this means the EntityManager should throw an exception if DQL is executed on the type when the UoW is not empty. Until the time that the EntityManager can query the UoW using DQL. The alternative would be to "flush" before every operation that goes to the database for data.



 Comments   
Comment by Roman S. Borschel [ 14/May/10 ]

Hi,

you mention a good point, however, this currently only affects findBy queries made through a repository. A DQL query already triggers a flush when there are pending insertions but this still has its own problems. First of, querying against the objects in the UoW is not a viable solution in my eyes.

For a regular find() (by identifier) the situation is clear anyway, you must flush prior to a lookup on an entity you previously persisted in the same request because, by definition, generated primary key values are only guaranteed to be available after the next flush.

Automatic flushing if the UoW has pending inserts (new objects) and a query is executed (either through DQL or a repository) currently has its own set of problems, namely that it is still subject to infinite recursion if such a query is triggered in an event (listener) that executes during commit of a UoW, and secondly, that it will easily lead to double-flushes that cause unnecessary overhead (currently a flush() even if nothing needs to be done is not free because the UoW actually has to check whether nothing needs to be done). Both of these problems could be addressed with some sort of flags, but the question still is whether its not better to flush manually in the first place.
That would mean, in your example, you should flush after persisting the new objects, irrespectively of what code comes next, you persisted (a) new object(s) and you want to make sure these are fully available to the rest of the script.

Comment by Roman S. Borschel [ 14/May/10 ]

Furthermore, automatic flushing when there is no transaction active is probably also not a great idea, as it may split a single unit of work (that was supposed to be atomic) into 2 without the user knowing about it. So auto-flushing should better only happen when a transaction is active (i.e. explicit transaction demarcation is used).

Comment by John Kleijn [ 14/May/10 ]

That would mean, in every example, you should flush after persisting new objects, period. If I flush in some cases and not in others, I'm asking for issues that may not be caught by tests. It's an inconsistency that I personally am not comfortable with.

Could be that I'm overlooking something, I've just started playing with D2.

Why is querying against registered objects not viable? It's not easy, granted, but it doesn't seem impossible. There should probably be a layer between the UoW and the "persisters" (Data Mappers?).

RE: the UoW double flush: state management on the UoW as a whole should prevent that. i.e. after a commit the whole UoW is clean? Just a suggestion, as I said, still getting my bearings.

On a side note I just want to say that what I've seen so far, for the better part, pleases me greatly. Kudos.

Comment by Roman S. Borschel [ 14/May/10 ]

@"That would mean, in every example, you should flush after persisting new objects, period."

Yes, if you want the objects to be visible to queries in the same request. Generally, you should flush when you complete a unit of work and that is usually not the whole request (but can be).

I don't want to "query" against registered objects because it is a) not easy b) likely a lot of code and c) very likely error-prone. And in addition I don't see this helping with solving any inconsistency. If you want to use find() you have to flush anyway because you can not find() without having the identifier in the first place, which is only available after a flush.

@RE: the UoW double flush:

Yes, like I said, it can be done but it is a compromise. Having a "clean/dirty" flag in addition to calculating the changesets of the work to do (which implicitly tells us whether the UoW is dirty) adds more code and more potential for errors. Forget to update the flag in one location and you get flushes that don't do anything, because the flag was not updated. A dirty-flag for the UoW is not really required for proper working. It is similar to the approach of maintaining a separate counter for the number of elements in a collection implementation: can make many size/count requests faster but complicates the internal implementation and increases the likelihood for errors (and lock contention for the counter in a thread-safe/concurrent implementation, an interesting case where performance goes against scalability, but I digress and that does not apply to php obviously). That said, I am not strongly opposed to doing this.

If you're interested in how this is specified by "big brother", take a look at section 3.8.7 of the JPA 2 specification. Shortly, with the default behavior it requires the implementation to ensure that unflushed changes are visible to queries which can be achieved by flushing these to the database automatically but only if a transaction is active, otherwise the implementation must not flush to the database. There is alternatively also a "MANUAL" flush mode, in that case the effect of updates made to entities in the UoW upon queries is unspecified. We do not have different flush modes anymore, however, in Doctrine.

So I see two possible ways to go here:

1) More effort, more code, (really better?)

  • Maintaining a dirty flag in the UoW (this could be done anyway at some point, even if 2) is chosen)
  • Maintaining a flag to avoid infinite recursion triggered from events within a UoW commit/flush
  • Flushing automatically when querying while there are pending inserts and a transaction is active

2) No effort, less code

  • Removing the current auto-flush on DQL queries which is still subject to infinite recursion
  • No automatic flushes, anywhere (less magic, so to speak?)
  • Clearly documenting that new, unflushed entities are not visible in subsequent queries issued in the same request, and if this is desired, a flush should be issued.

That's how I see it. Now we need some votes and volunteers for the implementation Personally, I am not sure yet about which version I prefer, 2) does not sound too bad for me.

Comment by Roman S. Borschel [ 14/May/10 ]

In Nr. 1) the case with the infinite recursion may actually be more problematic. I think you simply can not see unflushed new objects in queries made during a UoW commit.

Comment by John Kleijn [ 14/May/10 ]

When there's no in-memory objects inclusion, I'd say 2) as well. Again, I have no idea how this is implemented currently, but I would prefer something like this:

$repo->start();

$repo->register($object);

$repo->commit();

Why?

  • Commit instead of flush: "flush" has little semantic value IMO, "commit" leaves no questions: you're committing your changes (which implies that they are not, before)
  • Operating on the repo leaves no question to what you are committing: changes of the associated type and relations configured to cascade, made after start()
  • Register instead of persist: "persist" is misleading as the object is not immediately persisted, and as my example shows, may not be.

The way I see it "start" would create a UoW associated with the repo, "commit" would calculate changes and write (the enitity manager would make sure references in other UoWs are removed).

Because the way it is currently implemented (or so it seems), it's unclear when to flush and when not to flush, and unclear what I'm flushing at any one point in the code (because it is not locally isolated). If I have to decide whether to flush in some bit of client code, I am apparently making an assumption about the target entity, i.e. coupling.

I know, you already went beta, so it's unlikely you would consider such a large change, but anyway, for your consideration.

Finally, I realize I'm borderline nagging now as you've made it clear you see nothing it, but a Repository (as in the PoEAA pattern, p 322) may provide a method of fetching native in-memory objects using criteria, acting as a "buffer" between code and database. The Repository in D2 does effectively nothing but delegate to the UoW (or mostly to the underlying persister). Ref PoEAA 327 for an example of an in-memory strategy. As a final point of critique, the Repository does not always seem to be used as entry point for data requests, which is the whole point of the pattern. Most of what's in EntityManager, should be in EntityRepository ("manager" is a bit to abstract a concept to expect clear responsibilities anyway). EntityManager::find() delegates to EntityRepository, but pretty much everything else is the other way around. EntityManager would be better off named DataGateway, as that accurately describes its intended function.

I admit, it would be very difficult to use DQL on in memory objects, but it would be far superior and if it work lead to much more predictable behaviour. It's the ONLY way the data store is ever going to be truly transparent. A few examples (DQL from the docs):

SELECT
u,
UPPER(u.name) nameUpper
FROM MyProject\Model\User u

  • Fetch everything from the db
  • Select all objects from the User UoW
  • Iterate over the in memory ones and modify the name property to upper case
  • Merge the results and return

SELECT u FROM User u WHERE u.id = ?1 OR u.nickname LIKE ?2 ORDER BY u.surname DESC

  • Execute against database
  • Iterate over the User UoW, indexing by "surname", adding items that match the criteria
  • Merge the results and return

With joins it could get more complex, provided you want to intelligently merge results into existing objects. Question is whether that is really needed, but there's obviously a performance benefit. Actually this may already be implemented.

I suspect there are edge cases, rooted in DQL still being based on SQL, but in theory it should be possible. Likely you would still want to do start(), and delegate to the driver to start an actual transaction to prevent inconsistent reads... The only way to find out if it's truly feasible is to to try it, I think.

Ramble, ramble, ramble, I'm done. I know I seem critical, but it's positive critique, I love the direction you went with D2.

Comment by Roman S. Borschel [ 14/May/10 ]

Maybe I was not clear, with approach Nr.1 there would be in-memory objects inclusion (of new objects), in fact, there always is, due to the identity map. When you query for objects and some of them are already in memory, these are used, not again reconstructed.

The EntityRepository provided by Doctrine is just a convenient mechanism for writing your own repositories. There are many different understandings for what a repository is, you can make it whatever you want it to be. Is a PoEAA repository the same as a DDD repository? Anyway, the repository could be stripped of the project, it is optional, the state management is handled by the EntityManager and UnitOfWork. These are the core components. I agree that the delegation from EntityManager#find to the repository is suboptimal in this regard and should be the other way around.

Now to your question: "When should I flush?". Generally, you should flush at the end of a transaction, which in turn is a unit of work. That means, use explicit transaction demarcation. begin() ... flush() commit(). I've added some control abstractions recently that should make this even easier. I can only recommend to explicitly demarcate your transaction boundaries. As you probably know, you can not talk to your database outside of a transaction anyway. The default behavior (flush() wrapping all its stuff in a transaction) is for convenience mostly and so as not to alienate or confuse people even more who are used to autocommit mode.

Concerning the naming, we mostly stick with the JPA specification and I, for one, really like the naming and I don't want to invent new names. PoEAA is far more abstract (and the examples far too specific) than what is specified in JPA, so I recommend giving that a read. The patterns in PoEAA obviously and intentionally leave a lot of room for different variants of implementation and also leave open a lot of open questions (many of the difficult questions especially, it is for a reason that the author recommends using an existing tool instead of writing your own). In my opinion it is just not feasible to query in-memory objects in a generic way, all the examples in PoEAA do not have generic but rather concrete code examples, which is obviously a lot easier. The feasible strategy, and that is what we do, is to do in-memory lookups only when querying by PK, otherwise the query is executed and afterwards nevertheless any objects reused that are already in memory (based on the PK) and not reconstructed. This is the approach we use.

Thanks for your input, I do see that you are an experienced fellow in object-relational persistence, maybe we can see you as a committer some day?

Comment by Roman S. Borschel [ 14/May/10 ]

@ "SELECT u, UPPER(u.name) nameUpper FROM MyProject\Model\User u"

This selects all users and their names in uppercase, the uppercase names are scalar values, the users are not modified! Scalar values are separate from objects.

@ "... and unclear what I'm flushing at any one point in the code"

flush() means: Synchronize the in-memory state of my objects with the database, making any changes that are only in-memory persistent. Nothing more, nothing less.

Again, objects are always reused based on the identity map and the state that is in-memory prevails, unless you use refresh() or execute a query with the Query::HINT_REFRESH query hint. All objects you fetch from DQL, be it as a root object or as a joined association, are first looked up in-memory (but after the SQL query has been issued!).

Maybe we have been talking past each other here, what I refer to as not feasible is querying the in-memory objects first in some way, even before the SQL query. This is just too complicated and error-prone, except for the simple case of a PK lookup and that is where we do it already.

Comment by John Kleijn [ 14/May/10 ]

> Scalar values are separate from objects.

Right. Bad example.

> flush() means: Synchronize the in-memory state of my objects with the database, making any changes that are only in-memory persistent. Nothing more, nothing less.

I realize that it means that, but commit() would be more obvious.

> Maybe we have been talking past each other here, what I refer to as not feasible is querying the in-memory objects first in some way, even before the SQL query. This is just too complicated and error-prone, except for the simple case of a PK lookup and that is where we do it already.

Fair enough, you don't think it's feasible, so we'll keep it at that. Maybe I'll give it a shot some time.





[DDC-935] copy function needs implementation Created: 15/Dec/10  Updated: 02/Jan/11

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.0-RC2
Fix Version/s: None
Security Level: All

Type: Task Priority: Minor
Reporter: Jack van Galen Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None


 Description   

The (deep)copy function of the entity manager is not yet implemented. I assume this is known, but I could not find any open issue on it. This is a pretty powerfull feature once implemented. The function body is completely empty however. Perhaps the tried code could be added so I and others could try and resolve the known issue with this function (recursion limit reached).



 Comments   
Comment by Benjamin Eberlei [ 15/Dec/10 ]

There was never code written for that function. I don't think its too problematic that this is missing. You only have to implement __clone (and do so safely as the docs/cookbook describes) and then pass this structure to persist. Optionally making use of cascade persist.

Comment by Marcus Stöhr [ 01/Jan/11 ]

I recently came accross this. Is there any best practice if you have to clone an entity who has several associations? I thought of grabbing them and clone them one by one. Or is there a more convenient way?

Comment by Benjamin Eberlei [ 02/Jan/11 ]

no, except implementing __clone and doing it there.





[DDC-900] Insufficient Error Information for orm:validate-schema Created: 29/Nov/10  Updated: 29/Nov/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: Tools
Affects Version/s: 2.0-BETA4
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: aurorius Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 2
Labels: None
Environment:

linux, php 5.3.3



 Description   

Running "doctrine orm:validate-schema" would return -> [Database] FAIL - The database schema is not in sync with the current mapping file.

It should have at least return the name of the table/field that is not in sync.






[DDC-891] DDC-117: No sequence generation with composite foreign key Created: 25/Nov/10  Updated: 25/Nov/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: New Feature Priority: Minor
Reporter: Timo A. Hummel Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Given the following entity definitions, Doctrine does not attempt to manage generated values. For example, in MySQL, it is perfectly possible to create a composite primary key and set auto_increment on one of these. See below the code for issues that occur.

User.php
/**
 * @Entity
 */
class User {
	/**
	 * @Id
	 * @Column(type="integer")
	 * @GeneratedValue(strategy="AUTO")
	 */
	private $id;
	
	/**
	 * @Column(type="string")
	 */
	private $name;
	
	/**
	 * @OneToMany(targetEntity="PhoneNumber",mappedBy="id",cascade={"all"})
	 */
	private $phoneNumbers;
	
	public function setName ($name) {
		$this->name = $name;
	}
}
PhoneNumber.php
/**
 * @Entity
 */
class PhoneNumber {
	/**
	 * @Id
	 * @Column(type="integer")
	 * @GeneratedValue(strategy="AUTO")
	 */
	private $id;
	
	/**
	 * @Id
	 * @ManyToOne(targetEntity="User",cascade={"all"})
	 */
	private $user;
	
	/**
	 * @Column(type="string")
	 */
	private $phonenumber;
	
	public function setUser (User $user) {
		$this->user = $user;
	}
	
	public function setPhoneNumber ($phoneNumber) {
		$this->phonenumber = $phoneNumber;
	}
}
Invokation
$albert = new User;
$albert->setName("albert");
$em->persist($albert);

$phoneAlbert1 = new PhoneNumber();
$phoneAlbert1->setUser($albert);
$phoneAlbert1->setPhoneNumber("albert home: 012345");
$em->persist($phoneAlbert1);

The first issue which occurs is that Doctrine does not generate the field "id" within PhoneNumber set to auto_increment.

The second issue which occurs is that Doctrine becomes confused when inserting a new record into PhoneNumber, because of the following INSERT INTO statement:

Insert Statement
INSERT INTO PhoneNumber (user_id, phonenumber) VALUES (?, ?)
array(1) {
  [1]=>
  string(19) "albert home: 012345"
}

SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens


 Comments   
Comment by Roman S. Borschel [ 25/Nov/10 ]

I don't think this will ever be possible.

Comment by Timo A. Hummel [ 25/Nov/10 ]

Is there a technical reason for that? I mean, with DDC-117 we are aiming for composite foreign keys, or is DDC-117 cancelled?

Comment by Benjamin Eberlei [ 25/Nov/10 ]

A composite key is ALWAYS of the type "ASSIGNED" and cannot be a combination of different id generation strategies.

You could however write a prePersist Listener that does this for you.

Comment by Timo A. Hummel [ 25/Nov/10 ]

Okay, maybe this is a feature for 3.0 or so. However, I'd suggest leaving this bug open as this is something which needs to be documented once DDC-117 becomes integrated within the main branch.

Additionally, Doctrine should complain about different ID generation strategies. Right now it silently ignores it.





[DDC-878] Don't explicitly require object members (fields) to be defined in the entity class Created: 16/Nov/10  Updated: 16/Nov/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Nick Daugherty Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Currently, Doctrine REQUIRES that a given entity class have protected or private members explicitly defined in the class (even if meta data mapping is handled elsewhere, such as in YAML). This is less than optimal...for example, many class implementations prefer to store all data in a protected $fields member, as an array, accessing the members with getters and setters.

Doctrine makes this behavior impossible. An exception is thrown if a field defined in meta data is not an explicit member of the class. Instead, it should 'take the meta data's word for it' that the field exists, and is accessible via getters and setters, without explicitly checking for the member. The meta data is already the authoritative source, I don't see why the double check should (or needs to) be performed (although I am not familiar with Doctrine internals). Since Doctrine recommends making members private, I have to assume it is already hydrating them with the get/set accessors anyway...so it should just rely on them.

Quick example use case (notice 'name' is not actually a member...it is stored in $fields and assume meta data is defined in a separate yaml file):

class User

{ protected $fields = array(); public function getName() { return $this->fields['name']; } public function setName($name) { $this->fields['name'] = $name; } }

 Comments   
Comment by Benjamin Eberlei [ 16/Nov/10 ]

This maybe a potential optimization for a very future version. However currently we heavily rely on the Reflection support for properties, which kind of makes a change of this a very complex undertaking.





[DDC-838] SchemaTool - ignores the attribute uniq in relations Created: 13/Oct/10  Updated: 29/Oct/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: Tools
Affects Version/s: 2.0-BETA4
Fix Version/s: 2.x
Security Level: All

Type: Improvement Priority: Minor
Reporter: gektor Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Ubuntu, PHP 5.3.2, MySQL



 Description   
Mapper
<entity name="Default_Model_Test" table="test">
  <id name="id" type="integer" column="id">
    <generator strategy="AUTO"/>
  </id>
  <field name="blabla" column="blabla" type="boolean"/>
  <one-to-one field="user" target-entity="Users_Model_User">
    <join-column name="users_id" referenced-column-name="id" on-delete="CASCADE" on-update="CASCADE" unique="false" />
  </one-to-one>
</entity>
SQL
CREATE TABLE test (id INT AUTO_INCREMENT NOT NULL, users_id INT DEFAULT NULL, blabla TINYINT(1) NOT NULL, UNIQUE INDEX test_users_id_uniq (users_id), PRIMARY KEY(id)) ENGINE = InnoDB;
ALTER TABLE test ADD FOREIGN KEY (users_id) REFERENCES users(id) ON UPDATE CASCADE ON DELETE CASCADE;

Actual:
UNIQUE INDEX test_users_id_uniq (users_id)

Expected:
INDEX test_users_id (users_id)



 Comments   
Comment by Benjamin Eberlei [ 14/Oct/10 ]

Verified, i just don't understand why you are using a one-to-one relation and then "deactivate" the database constraint for this. You could easily use Many-To-One

Comment by gektor [ 14/Oct/10 ]

You are right. It's not a bug, it's feature.

Comment by Benjamin Eberlei [ 29/Oct/10 ]

This might still be a good improvement to allow the flexibility, but its not a bug. Updating to "Minor Improvmenet for 2.x"





[DDC-747] Add support for table, column and indexes comments Created: 13/Aug/10  Updated: 14/Aug/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: Mapping Drivers
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: New Feature Priority: Minor
Reporter: s9e Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 5
Labels: None


 Description   

It would be nice to add support for table comments, column comments and indexes comments, i.e.

@Table(
    name="ecommerce_products",
    comment="Contains products",
    indexes={@index(name="search_idx", comment="Makes finding people by name easier", columns={"name", "email"})}
)

@Column(type="integer", comment="Stores the user's age, in years")

MySQL supports comments via ALTER TABLE since 4.1.
PostgreSQL support comments via COMMENT at least since 7.4.
SQLite doesn't appear to support table, column or indexes comments as of 3.7.
Oracle supports comments on tables and columns via COMMENT at least since 10.1. It doesn't appear to support comments on indexes.

For databases that don't support those persistent comments, you could use normal SQL comments in the table definition. Of course they wouldn't be stored by the database, but at least they would appear in the generated schema, that's better than nothing.






[DDC-718] Bottleneck in computeAssociationChanges()? Created: 24/Jul/10  Updated: 24/Jul/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Benjamin Eberlei Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Reference
relates to DDC-600 Persisting Entities with unmanaged re... Closed

 Description   

It seems that since DDC-600 computeAssociationChanges() iterate over entries of an collections, even if they are not marked as cascadePersist. For large, hydrated collections this could potentially become a bottleneck.

Wouldn't it be better to save the "addedEntities" in an additional map inside "PersistentCollection" and retrieve those instead of calling $value->unwrap() ?



 Comments   
Comment by Roman S. Borschel [ 24/Jul/10 ]

Do you have any numbers to back this up? With large, hydrated collections the bottlenecks are likely elsewhere (SQL query, hydration)

Further, maintaining "addedEntities" is not as trivial as you might think. The current approach does not care about what happens in-between, it just computes a diff between the old and new state of the collection at commit time. Tracking added/removed objects as they come in and go is more cumbersome.

Comment by Benjamin Eberlei [ 24/Jul/10 ]

no numbers, i was just confused about the code, because i remembered it differently





[DDC-712] allow RIGHT JOIN or specifying the root class of the hydratation tree Created: 21/Jul/10  Updated: 22/Jul/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: DQL, ORM
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Mihai Ilinca Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Hi! Let me start by saying you guys did a great job with Doctrine 1 and that I can't wait to start using Doctrine2

I will explain this feature request with an example. I have a User entity wich relates one to many to a Picture entity. Picture has a " is main picture" boolean field. Not all users have a main picture. I would like to be able to select all Users, each with their main picutre, if that exists, or some Null value, if it does not exists, in one query, using join. I would also like for the result collection to contain Picture entities on the first level, with the User beinng accessible as an aggregate of Picture.

The way I can think doing this is by using a RIGHT or LEFT join (not INNER) as to also select Users that don't have a main picture. I can do this by selecting

SELECT Picture p, p.User u FROM p RIGHT JOIN u WITH p.main=1

but right joins afik are not available atm in either version of Doctrine, or by selecting

SELECT User u, u.Picture p FROM u LEFT JOIN p WITH p.main=1

and somehow instructing the hydrator to consider Picture as the root object for the generated object tree and User as a "child" of Picture.

For users without a picture, the Picture object would somehow indicate it is NULL, while still holding a refference to the User.

Makes sense? If there is an alternate way to achieve this, please enlighten me, tough I think it would still add felxibility if we could hint the hydrator for the root object in a tree.



 Comments   
Comment by Benjamin Eberlei [ 22/Jul/10 ]

Why don't you model that as ManyToOne for the Main Picture and OneToMany for all pictures? Makes much more sense from an ORM perspsective, you would have your own property "User::$mainPicture"

Comment by Mihai Ilinca [ 22/Jul/10 ]

Thanks for the suggestion. However, this was just an example to demonstrate some lack of flexibility, I am not strictly looking for a solution to this example, but to the concept behind it.

Also, how would I get the result with Picture on the top level and User aggregated to Picture with the model you suggested? Unless I am missing something, wouldn't I end up in the same situation?

I can post-process the results myself and create a new collection easily, ofc, but it would be better (and more optimal) if I could tell the hydrator to do this, similar to how INDEXBY is passed as an option to the hydrator.





Add the notion of read-only entities (DDC-209)

[DDC-691] doctrine.readOnly query hint Created: 15/Jul/10  Updated: 31/May/12

Status: Open
Project: Doctrine 2 - ORM
Component/s: DQL, ORM
Affects Version/s: None
Fix Version/s: 2.x
Security Level: All

Type: Sub-task Priority: Minor
Reporter: Roman S. Borschel Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 6
Labels: None


 Description   

Setting such a query hint to TRUE should result in all entities being retrieved by that query to be read-only for the purposes of change-tracking. Note that the entities themselves need not necessarily be read-only in general.

This feature is a flush performance tweak that can be used to query for objects but not let the returned objects run through change-tracking on flush. Any other managed objects are tracked as usual so you can do a read-only query for 100 entities and persist a new entity in the same unit of work with optimal flushing performance.



 Comments   
Comment by Konstantin [ 26/Dec/11 ]

Any news?
Why query hint? What about temporary switching like fetch mode changing via query object?

Comment by Gigi Largeanus [ 31/May/12 ]

Any news on this?

I think this is a must have feature. Thanks for all your work.





[DDC-677] Allow DQL DELETE statements to work with join table fk constraints Created: 10/Jul/10  Updated: 10/Jul/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: 2.0-BETA2
Fix Version/s: None
Security Level: All

Type: New Feature Priority: Minor
Reporter: Benjamin Eberlei Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Reference
is referenced by DDC-130 Cascading and @ManyToMany association... Resolved

 Description   

Currently DQL DELETE will break if any foreign key constraint restricts the delete.

Using the Metadata we can possibly detect these join table FK contstraints and delete them correctly.






[DDC-530] Create tests and documentation for possibilities of mixing inheritance mapping strategies Created: 19/Apr/10  Updated: 19/Apr/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.0-BETA1
Fix Version/s: 2.0
Security Level: All

Type: Task Priority: Minor
Reporter: Roman S. Borschel Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None


 Description   

It is (theoretically) possible to use different inheritance mapping strategies in the same class hierarchy as long as the different subtrees that use different mapping strategies do not have a common ancestor entity. We should add some tests for that and mention it in the docs about inheritance mapping in a new subsection "Mixing Inheritance Mapping Strategies".






[DDC-450] Add TableGenerator Implementation Created: 20/Mar/10  Updated: 13/Feb/12

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.0-ALPHA4
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Benjamin Eberlei Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 2
Labels: None

Issue Links:
Dependency
depends on DBAL-223 Add DBAL TableGenerator Resolved

 Description   

The TableGenerator Id Generator is not yet implemented, here is some code i came up with:

class TableGenerator extends AbstractIdGenerator
{
    private $_tableName;
    private $_sequenceName;
    private $_allocationSize;
    private $_nextValue;
    private $_maxValue;

    public function __construct($tableName, $sequenceName = 'default', $allocationSize = 10)
    {
        $this->_tableName = $tableName;
        $this->_sequenceName = $sequenceName;
        $this->_allocationSize = $allocationSize;
    }

    public function generate(EntityManager $em, $entity)
    {
        if ($this->_maxValue === null || $this->_nextValue == $this->_maxValue) {
            // Allocate new values
            $conn = $em->getConnection();
            if ($conn->getTransactionNestingLevel() == 0) {

                // use select for update
                $sql = $conn->getDatabasePlatform()->getTableHiLoCurrentValSql($this->_tableName, $this->_sequenceName);
                $currentLevel = $conn->fetchColumn($sql);
                if ($currentLevel != null) {
                    $this->_nextValue = $currentLevel;
                    $this->_maxValue = $this->_nextValue + $this->_allocationSize;

                    $updateSql = $conn->getDatabasePlatform()->getTableHiLoUpdateNextValSql(
                        $this->_tableName, $this->_sequenceName, $this->_allocationSize
                    );
                    
                    if ($conn->executeUpdate($updateSql, array(1 => $currentLevel, 2 => $currentLevel+1)) !== 1) {
                        // no affected rows, concurrency issue, throw exception
                    }
                } else {
                    // no current level returned, TableGenerator seems to be broken, throw exception
                }
            } else {
                // only table locks help here, implement this or throw exception?
                // or do we want to work with table locks exclusively?
            }
        }
        return $this->_nextValue++;
    }
}


 Comments   
Comment by Guilherme Blanco [ 04/Aug/10 ]

Already merged into core.

Comment by Benjamin Eberlei [ 04/Aug/10 ]

But it is not enabled yet Plus we need tests to verify this works in high concurrency enviroments and does not pass the same id twice.

Furthermore the DAtabase Platform Methods are completly missing. No implementations yet.

Comment by Benjamin Eberlei [ 04/Aug/10 ]

Schema-Tool support is also missing.





[DDC-445] Evaluate possible ways in which stored procedures can be used Created: 19/Mar/10  Updated: 24/Dec/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: 2.x
Security Level: All

Type: Improvement Priority: Minor
Reporter: Roman S. Borschel Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 1
Labels: None

Issue Links:
Reference
relates to DDC-391 Allow to specifiy custom Entity and C... In Progress

 Description   

We should evaluate the current situation of stored procedure support as well as where we want to go with it / how far we want to support it, if at all.



 Comments   
Comment by Benjamin Eberlei [ 19/Mar/10 ]

I think this relates to the usage of Custom Persisters





[DDC-415] Introduce UnitOfWork Stages and throw exceptions for wrong method uses Created: 12/Mar/10  Updated: 18/Mar/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.0-ALPHA4
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Benjamin Eberlei Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Currently the event architecture is fragile when used wrong. I already see lots of "bug reports" popping up on this issue due to people dont understanding what is doable and what is not.

How about we introduce an instance variable stage into the UnitOfWork and introduce an assertIsInStages($stages) protected method which is called ineach major command method of the UnitOfWork to verify its applied correctly?

Stages could be:

UNFLUSHED
PRE_COMPUTE_CHANGESETS
POST_COMPUTE_CHANGESETS
FLUSH_LOOP
TRANSACTION_COMPLETED



 Comments   
Comment by Roman S. Borschel [ 12/Mar/10 ]

I'm not sure. I'm afraid this will just add code bloat with the only goal to provide better error messages and its fragile to do right. There will surely be places missed in the code where to check for the stage and it might even constrain some valid use-cases we dont think of yet.

So I'm afraid that this would hurt more than it would help.





[DDC-280] UnitOfWork changeSet population should take advantage of Comparable technique Created: 27/Jan/10  Updated: 04/Feb/11

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Guilherme Blanco Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 1
Labels: None


 Description   

Currently our UnitOfWork computes the changeset by checking actual instances of Objects.

Unable to find source-code formatter for language: php. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
// UnitOfWork, lines 501-507 

            foreach ($actualData as $propName => $actualValue) { 
                $orgValue = isset($originalData[$propName]) ? $originalData[$propName] : null; 
                if (is_object($orgValue) && $orgValue !== $actualValue) { 
                    $changeSet[$propName] = array($orgValue, $actualValue); 
                } else if ($orgValue != $actualValue || ($orgValue === null ^ $actualValue === null))  
	                    $changeSet[$propName] = array($orgValue, $actualValue);
                } 

While this is ok when you do new object assignments, it just bypass same instances of same object, since the hash is the same.
A user on IRC (post-o-matic) has a quite complex object logic that he would like to avoid clone and even instantiate another class.
I agree with him that cloning is not the ideal technique, mainly because the changeset would always compute the object (since then hashs would be different).

He implemented this datatype:

Unable to find source-code formatter for language: php. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
class EffortGraphType extends Type 
{ 
    public function getSqlDeclaration(array $fieldDeclaration, AbstractPlatform $platform) 
    { 
        return $platform->getClobTypeDeclarationSql($fieldDeclaration); 
    } 

    public function convertToPHPValue($value, AbstractPlatform $platform) 
    { 
        return new EffortGraph(unserialize($value)); 
    } 

    public function convertToDatabaseValue($value, AbstractPlatform $platform) 
    { 
        return serialize($value->getGraphPoints()); 
    } 

    public function getName() 
    { 
        return 'effort_graph'; 
    } 
}

I was thinking in a possible alternative and it came up to me the same basic idea we have with operators overloading OR Comparable interface of Java. I know in Java it supports way more things, but at least for this situation (as a start point) it would make developer's life easier.

Basic idea is to have an interface in Doctrine\Common:

Unable to find source-code formatter for language: php. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
namespace Doctrine\Common;

interface Comparable
{
    public function compareTo($value);
}

And update our UnitOfWork to take advantage of it:

Unable to find source-code formatter for language: php. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
// UnitOfWork, lines 501-507 

            foreach ($actualData as $propName => $actualValue) { 
                $orgValue = isset($originalData[$propName]) ? $originalData[$propName] : null; 

                if (is_object($orgValue)) { 
                    $isDiff = ($orgValue instanceof Doctrine\Common\Comparable) 
                        ? $orgValue->compareTo($actualValue) :  ($orgValue !== $actualValue);

                    if ($isDiff) {
                        $changeSet[$propName] = array($orgValue, $actualValue); 
                    }
                } else if ($orgValue != $actualValue || ($orgValue === null ^ $actualValue === null))  
	                    $changeSet[$propName] = array($orgValue, $actualValue);
                } 

In this user's usecase, it'd require him to update the EffortGraph class and implement Comparable interface.
For his specific situation, he'd need to store original value and updated value, just like we do internally in UnitOfWork for Entities.



 Comments   
Comment by Guilherme Blanco [ 19/May/10 ]

What's the final status of this?

IMHO this should be incorporated, since it adds a powerful support that users can take advantage in our Types.

Cheers,

Comment by Guilherme Blanco [ 19/May/10 ]

You're the main guy that can give a final word in this subject.

I'm +1 for this

Comment by Robert [ 04/Feb/11 ]

+1 for this...

if you have datetimes in a table and are using the DateTime object, you end up with useless update queries, if you persist an unchanged object...

Comment by Benjamin Eberlei [ 04/Feb/11 ]

That is not true.

Comment by Robert [ 04/Feb/11 ]

Sorry, I wasn't clear.

It does not happen in all cases.

I have a simple object, that I save to the session. After merging it and flushing the entitiy manager, an update query is generated, which sets all the datetime fields of the object to their current value:

Unable to find source-code formatter for language: php. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
$item = $_SESSION['item'];
$item = $this->_em->merge($item);
$this->_em->persist($item);
$this->_em->flush();

This code results in the expected SELECT query, which refreshes the item from DB, but it also results in an update query which sets the datetime of the object to the same value.
This query could be omitted, if I could use a comparable interface and a custom type for datetimes, which implement it.

Comment by Benjamin Eberlei [ 04/Feb/11 ]

This rather seems like a bug with the merging. Can you open up a new ticket describing this? Thank you





[DDC-17] Ability to skip the operation from a pre-operation event handler Created: 20/Sep/09  Updated: 26/Aug/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: None
Fix Version/s: 2.x
Security Level: All

Type: New Feature Priority: Minor
Reporter: Ismo Toijala Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None


 Description   

In Doctrine 1.1 it is possible to skip the operation in the event handlers in Doctrine_Record_Listener using Doctrine_Event::skipOperation.

This no longer seems to be possible in Doctrine 2.0 Alpha 1, for example when handling a preRemove event to implement soft-delete behaviour. Perhaps a method could be added to \Doctrine\Common\EventArgs\LifecycleEventArgs to skip the operation, at least before the operation.

Without this implementing soft-delete would require the user to update deleted_at and deleted_by himself and then save the record. It could no longer be done automatically when removing a record because the record is then removed.



 Comments   
Comment by Roman S. Borschel [ 20/Sep/09 ]

The problem is, full support for soft-delete throughout the system is not feasible and very fragile. Simple soft-delete through skipping the delete operation is the easiest part. Then you will probably want to modify all DQL queries so that they adhere to it automatically and then there will always be still queries that do NOT go through DQL, like even internal lazy-load queries or native queries or others, which would need to be modified also.

To sum it up, implementing soft-delete "inside" doctrine is absolutely not worth the effort and imho a bad idea and I'm certainly not willing to make lots of adjustments to the core that have a negative impact on performance just to make this soft-delete possible.

I really recommend handling "soft" deletes yourself, the normal way, by simply abstracting entity retrieval and persistence through a DAO/repository layer. As a nice side-effect you get less magic and it still works when you swap out doctrine for another persistence provider.

I am willing to add support for skipping deletes and maybe some other operations through events but I'm not willing to go any further, as explained above.





[DDC-4] Implement support for Concrete Table Inheritance Created: 09/Sep/09  Updated: 24/Dec/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.1
Fix Version/s: 2.x
Security Level: All

Type: Improvement Priority: Minor
Reporter: Roman S. Borschel Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None


 Description   

A first implementation could probably live without support for polymorphic queries (requires SQL UNIONs to be generated).






[DDC-547] Consider allowing custom PersistentCollection implementations Created: 27/Apr/10  Updated: 20/May/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: New Feature Priority: Minor
Reporter: Roman S. Borschel Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 7
Labels: None


 Description   

We should consider allowing the configuration of custom PersistentCollection implementations on a per-association basis.
This could allow users to craft optimized (SQL) behavior for for some of their collections to improve performance without changing the domain model code.

For this, PersistentCollection needs to be designed for inheritance.



 Comments   
Comment by Roman S. Borschel [ 26/Aug/10 ]

Rescheduled for 2.1. Might be 2.x.

Comment by Benjamin Eberlei [ 24/Dec/10 ]

Reschedule for 2.x

Comment by Wouter Wiltenburg [ 20/May/14 ]

Any progress on this new feature?

Comment by Marco Pivetta [ 20/May/14 ]

This can't be solved in the 2.x series because of the tight coupling between association UnitOfWork logic and the PersistentCollection API.





[DDC-940] Entities can / can not have private properties Created: 15/Dec/10  Updated: 15/Dec/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: Documentation
Affects Version/s: 2.0-RC2
Fix Version/s: None
Security Level: All

Type: Documentation Priority: Trivial
Reporter: Ray Rehbein Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None
Environment:

PHP 5.3.3 (cli) (built: Nov 14 2010 16:54:26)



 Description   

In the note in
http://www.doctrine-project.org/docs/orm/2.0/en/reference/working-with-objects.html#merging-entities
It appears to state that private variables are not serialized for child objects

If this is the only reason entities can't have private properties, then this restriction is no longer valid, or possibly be reconsidered.

<?php

class A {
    private $a = null;

    public function setValue($value) {
        $this->a = $value;
    }
}

class B extends A {}

$b = new B();
$b->setValue("B");
var_dump($b);

$c = unserialize(serialize($b));
var_dump($c);

The output suggests private variables are serialized, and are restored fine






[DDC-605] Include Fowards Compatible Support for Scalar Type Hints Patch in ProxyFactory Created: 21/May/10  Updated: 08/Aug/10

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Trivial
Reporter: Benjamin Eberlei Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 0
Labels: None


 Description   

ProxyFactory needs foward compatible support for the scalar type hint patch:

http://ilia.ws/archives/217-Scalar-Type-Hints-are-Here!.html



 Comments   
Comment by Roman S. Borschel [ 23/May/10 ]

I would be surprised if that would not be reverted soon though





[DDC-567] Foreign Key to Unique Field Update Failure Created: 03/May/10  Updated: 08/Oct/12

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.0-BETA2
Fix Version/s: None
Security Level: All

Type: New Feature Priority: Trivial
Reporter: Michael Ridgway Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 2
Labels: None

Attachments: File DDC567Test.php    

 Description   

I am getting an error: 'Notice: Undefined index: sysname in ./libraries/Doctrine/ORM/Persisters/BasicEntityPersister.php on line 434' when I try to flush a change to a property that references a unique field on another object.

From poking around in the _prepareUpdateData function, it seems that it only allows you to use identifier fields:

$newValId = $uow->getEntityIdentifier($newVal);

..

$result[$owningTable][$sourceColumn] = $newValId[$targetClass->fieldNames[$targetColumn]];

I'll see if I can get a test case for this set up.



 Comments   
Comment by Roman S. Borschel [ 03/May/10 ]

Hi. That is right. Foreign keys (join columns) must point to primary keys, not arbitrary other columns, whether they're unique or not, Doctrine does not know.

In other words, joinColumn must always refer to an identifier/pk. I'm not sure but I think anything else would be a pretty strange relational model, too, but there may be usecases we have not yet encountered.

I'm afraid this will not be possible and would be very hard to implement. Of course if somebody has a patch we happily accept it (after reviewing).

Leaving this open in the case somebody wants to work on it.

Comment by Michael Ridgway [ 03/May/10 ]

A minimal test case. Removing the first flush produces the same error, so this seems to be a bug on inserts as well.

Comment by Roman S. Borschel [ 03/May/10 ]

Its not really a bug but rather a new feature This was not intended to work so far.

Comment by Michael Ridgway [ 03/May/10 ]

Ah, ok. Maybe it didn't work before. I don't know where I got the idea that it did.

Thanks.

Comment by Michael Ridgway [ 03/May/10 ]

Oops, closed it before I noticed you said you wanted to leave it open.

Comment by Roman S. Borschel [ 03/May/10 ]

Thanks for the testcase though, it is useful. In your concrete example, is it not an option to make the sysname the @Id ?

Comment by Michael Ridgway [ 03/May/10 ]

Yes. That is definitely the way it should be done in this case. I can't really think of a case to have a reference to a unique key while still having an Id on it, except when you're working with an existing, poorly designed database (which is our case).

The reason I assumed this was possible is that the references actually work for lazy loading, but as soon as you start changing the references it throws this error.

Comment by Roman S. Borschel [ 14/May/10 ]

Lowering priority.

Comment by Thomas Subera [ 08/Oct/12 ]

although we would also need this i would suggest adding an error message if the associated column is not found in $newValId. (class BasicEntityPersister.php _prepareUpdateData)

otherwise the field is populated with null leaving the developer debugging an hour :-/

thx





Generated at Tue Sep 30 22:00:48 UTC 2014 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.