[DDC-2133] Issue with Query::iterate and query mixed results Created: 09/Nov/12  Updated: 01/May/13

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.2.1
Fix Version/s: 3.0
Security Level: All

Type: Bug Priority: Major
Reporter: Oleg Namaka Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
is duplicated by DDC-1314 DQL permits partial select using SQL Resolved


Consider this code:

$dql = "
    SELECT Page, Product.name
    FROM Dlayer\\Entity\\Page Page
    INNER JOIN Page.Product Product
$q = ($em->createQuery($dql));
foreach ($q->iterate() as $entry) {
  $page = $entry[0][0];
  $name = $entry[0]['name'];

This results with undefined index: 'name' for the second entry.

First result keys are (notice just one array element with index 0):

array(2) {
  [0] =>
  [1] =>
  string(4) "name"

but all others are different (notice two array elements with index 0 and the other one that is incrementing):

the second one:
array(1) {
  [0] =>
array(1) {
  [0] =>
  string(4) "name"
the third one:
array(1) {
  [0] =>
array(1) {
  [0] =>
  string(4) "name"

What's wrong with this approach? Is it a bug or mixed results should not be used with the iterate method?

Comment by Benjamin Eberlei [ 12/Nov/12 ]

This is a known issue that we don't have found a BC fix for and as I understand Guilherme Blanco requires considerable refactoring.

[DDC-2089] Modify OneToMany to allow unidirectional associations without the need of a JoinTable Created: 19/Oct/12  Updated: 07/Sep/13

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.x
Fix Version/s: 3.0, 2.5
Security Level: All

Type: Improvement Priority: Major
Reporter: Enea Bette Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: onetomany, persister, unidirectional

Debian Wheezy, Mysql 5.1, Apache2, PHP 5.4


As I sayd in the title, it would be nice if the ORM layer could permit to map a 1:n association in the db as an unidirectional OneToMany in the classes, without using a JoinTable in the database.
This would permit us to get rid of the unnecessary database JoinTable, which creates disorder and decreases performance for no valuable reason.

Is it possible?

Comment by Enea Bette [ 16/Dec/12 ]

A little up... for inspiration from JPA


[DDC-536] Remove the _ prefix from private and protected members Created: 23/Apr/10  Updated: 16/Apr/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: 3.0
Security Level: All

Type: Improvement Priority: Major
Reporter: Roman S. Borschel Assignee: Guilherme Blanco
Resolution: Unresolved Votes: 1
Labels: None


The reasoning is simple: The prefix "_" is usually either used for easier distinction of instance variables from other, i.e. local variables, instead of always using "this." (often seen in C#), or it is used to signal that a member is not meant to be accessed from outside of the class when the language does not have visibility modifiers (PHP4).

Since you always have to use "$this->" in PHP5+ when accessing instance members and there are visibility modifiers, the "_" is largely superfluous and just makes the verbose OO code even more verbose.

Maybe the following find/replace steps will do the job almost completely:

"private $_" => "private $"
"protected $_" => "protected $"
"$this->_" => "$this->"

Comment by Benjamin Eberlei [ 27/Apr/10 ]

i just found a possible BC issue with this.

EntityRepository is allowed to be extended by us, it has several variables that are underscore prefixed. How to proceed in this case?

Comment by Roman S. Borschel [ 27/Apr/10 ]

I know but its not really a problem I think. We should just decide whether we make them private in the first place and provide getters instead (which would have avoided this problem in the first place).

Comment by Roman S. Borschel [ 27/Apr/10 ]

Leaving the prefixes on the repository class only is also an option... but I dont think thats necessary.

Comment by Benjamin Eberlei [ 27/Apr/10 ]

can we commit getters for Beta 1 then? We could give everyone a period until Beta 2 to fix their code and then make the change.

EntityRepository is the only class that is meant to be userland extendable to my knowledge, so this should be the only problem to adress

Comment by Roman S. Borschel [ 27/Apr/10 ]

Yes, you can add getters and commit right away if you want. Plus adding a note on the upgrade document that direct access of these properties is deprecated.

Comment by Roman S. Borschel [ 27/Apr/10 ]

Persisters will be also extensible some day in userland but they need more polish for that, I've already started with it

Comment by Johnny Peck [ 19/Nov/10 ]

Is this still planned? Searching the code base finds this is not being implemented. It would be a good idea to implement the change sooner than later if it will be done at all. Also, +1 for the change. It makes complete sense.

Comment by Guilherme Blanco [ 16/Apr/14 ]

Moving to 3.0 as this can potentially create BC breaks

[DDC-3183] Add JsonSerializable to Collections Created: 22/Jun/14  Updated: 22/Jun/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 3.0
Fix Version/s: 3.0
Security Level: All

Type: New Feature Priority: Major
Reporter: Gabriel Bull Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: collection, orm


Implement this:


Can't really make this claim if Doctrine is not implementing basic interfaces for collections:

> The missing (SPL) Collection/Array/OrderedMap interface.

[DDC-274] Class and namespace naming inconsistency Created: 24/Jan/10  Updated: 26/Jun/14

Status: In Progress
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: 3.0
Security Level: All

Type: Improvement Priority: Major
Reporter: Glen Ainscow Assignee: Guilherme Blanco
Resolution: Unresolved Votes: 1
Labels: None


There are inconsistencies with some class and namespace names that include acronyms.


Classes with upper-casing:
ORMException, DBALException, OCI8Connection, etc.

Classes with proper-casing:
RunDqlTask, CliException, MySqlPlatform, etc.

Namespaces with upper-casing:
DBAL, ORM, Doctrine\DBAL\Driver\PDOMsSql, etc.

Namespaces with proper-casing:
Doctrine\Common\Cli, Doctrine\DBAL\Tools\Cli\, Doctrine\ORM\Id, etc.

There is more proper-casing than upper-casing. IMHO, proper-casing is better as it's easier to read "SqlException" than it is to read "SQLException" (the "E" looks like part of the acronym), and things like "CLITask" can be avoided.

I discussed this a bit with Benjamin and Guilherme, and they were unsure and said that the whole team needed to reach consensus.

I'm leaving the priority as "Major" because this should probably be fixed sooner rather than later to prevent compatibility breaks.

Comment by Guilherme Blanco [ 25/Jan/10 ]

Increasing the severity and adding a fix version since this MUST be fixed before next release.

Comment by Roman S. Borschel [ 25/Jan/10 ]

I find this to be of rather minor importance. You're talking of compatibility breaks, but thats only the case if we intend to start being very nitpicky about the naming in the future. We are currently not, and we wont be, so not much risk of later breakage.

We have a rule of thumb already: Acronyms up to 4 characters all uppercase, otherwise normal camelcase.

Now, it is often a matter of taste what is an acronym and what not and also not all acronyms have a clear casing, prime example mysql.

Being too nit-picky here is just a pain in the ass and we won't reach a common consensus anyway.

Comment by Roman S. Borschel [ 25/Jan/10 ]

Oh and we better dont start arguing about whats better to read because there is no chance of agreement. If you ask me I find SQLException better than SqlException. So here we go. Lets better not run down this path.

Comment by Glen Ainscow [ 25/Jan/10 ]

No need to get upset, I'm only trying to help.

I just thought that consistency is usually a good idea, either way.

I have to disagree in that an acronym is an acronym, it's not a matter of taste, the letters either stand for something, or they don't.

As for "mysql", only the SQL part is an acronym. So, MySQL or MySql.

Close if you disagree.

Comment by Roman S. Borschel [ 25/Jan/10 ]

I'm not upset. What made you think so? Maybe I should introduce a every now and then.

There's just no point in arguing about readability.

Like I said, we do have a naming standard even if its not adhered everywhere. The standard is also not something we've made up ourselves because we try to avoid that. When we introduced namespaces, we talked about adopting either the Java or .NET naming standards. We opted for the .NET standards. And there its recommended to write acronyms with up to 4 characters all uppercase.

I dont think there are too many violations of that in the code, probably Cli => CLI and some others but most of it looks ok to me.

UPDATE: Maybe we can gather a list here in this ticket of violations? Cli => CLI would be one. MySql => MySQL another. "Id" is rather an abbreviation of Identifier and not an acronym, to me at least.

Comment by Guilherme Blanco [ 25/Jan/10 ]

It's not a case of getting anyone upset...

But we should fix it and keep consistency of our codebase.

@romanb We should all talk and reach a common sense.
That's our last chance (last alpha) to get this consistency in.

Comment by Roman S. Borschel [ 25/Jan/10 ]

@Guilherme: We do have a naming standard since a year. You want to change the standard now?

Comment by Glen Ainscow [ 25/Jan/10 ]


Just a feeling I got.

This issue was more about consistency (indicated in the title) than moving to proper-case naming.

I think it might be up to 3 characters in .NET, HtmlElement, System.Linq, etc. Anyway, not important.

I agree that Id. is an abbreviation.

There are some more violations. If you decide you want to change them, let me know and I'll compile a list.

Comment by Roman S. Borschel [ 25/Jan/10 ]

@Glen: Yes, a list would be great. I find it hard to be 100% consistent sometimes though because my taste conflicts with the rule. For example, I would prefer "Id" over "ID", especially since it comes directly after ORM "Doctrine\ORM\ID\..." would be a bit too much. But I would not like "Orm" or "Dbal" either. But I think in most cases we can clearly fix the inconsistency. Like CLI or MySQL or MSSQL (or should it be MsSQL?).

Thanks for your help!

We should probably create updated coding standards for Doctrine 2 also, since they differ quite some from Doctrine 1 due to the introduction of namespaces and such. I'll create a ticket for that.

Comment by Glen Ainscow [ 25/Jan/10 ]

I'll do the list for you by tomorrow at the latest, just running out of time ATM.

Id is correct, as mentioned above, so that would be fine. MsSQL is correct (Ms = Microsoft = abbreviation).

Comment by Glen Ainscow [ 25/Jan/10 ]

Found some time ...

Doctrine\Common\Cache\ApcCache -> APCCache
Doctrine\Common\Cli -> CLI
Doctrine\Common\Cli\Printers\AnsiColorPrinter -> ANSIColorPrinter
Doctrine\Common\Cli\CliController -> CLIController
Doctrine\Common\Cli\CliException -> CLIException

Doctrine\DBAL\Driver\PDOMsSql -> PDOMsSQL
Doctrine\DBAL\Driver\PDOMySql -> PDOMySQL
Doctrine\DBAL\Driver\PDOPgSql -> PDOPgSQL
Doctrine\DBAL\Driver\PDOSqlite -> PDOSQLite
Doctrine\DBAL\Logging\EchoSqlLogger -> EchoSQLLogger
Doctrine\DBAL\Logging\SqlLogger -> SQLLogger
Doctrine\DBAL\Platforms\MsSqlPlatform -> MsSQLPlatform
Doctrine\DBAL\Platforms\MySqlPlatform -> MySQLPlatform
Doctrine\DBAL\Platforms\PostgreSqlPlatform -> PostgreSQLPlatform
Doctrine\DBAL\Platforms\SqlitePlatform -> SQLitePlatform
Doctrine\DBAL\Schema\Visitor\CreateSchemaSqlCollector -> CreateSchemaSQLCollector
Doctrine\DBAL\Schema\Visitor\DropSchemaSqlCollector -> DropSchemaSQLCollector
Doctrine\DBAL\Schema\MsSqlSchemaManager -> MsSQLSchemaManager
Doctrine\DBAL\Schema\MySqlSchemaManager -> MySQLSchemaManager
Doctrine\DBAL\Schema\PostgreSqlSchemaManager -> PostgreSQLSchemaManager
Doctrine\DBAL\Schema\SqliteSchemaManager -> SQLiteSchemaManager
Doctrine\DBAL\Tools\Cli -> CLI
Doctrine\DBAL\Tools\Cli\Tasks\RunSqlTask -> RunSQLTask

Doctrine\ORM\Mapping\Driver\PhpDriver -> PHPDriver
Doctrine\ORM\Mapping\Driver\XmlDriver -> XMLDriver
Doctrine\ORM\Mapping\Driver\YamlDriver -> YAMLDriver
Doctrine\ORM\Query\Exec\AbstractSqlExecutor -> AbstractSQLExecutor
(Should Doctrine\ORM\Query\Expr\Andx and Orx be AndX and OrX?)
Doctrine\ORM\Query\SqlWalker -> SQLWalker
Doctrine\ORM\Tools\Cli -> CLI
Doctrine\ORM\Tools\Cli\Tasks\RunDqlTask -> RunDQLTask
Doctrine\ORM\Tools\Export\Driver\PhpExporter -> PHPExporter
Doctrine\ORM\Tools\Export\Driver\XmlExporter -> XMLExporter
Doctrine\ORM\Tools\Export\Driver\YamlExporter -> YAMLExporter

... I think that's it. More than you expected? I did say: "There is more proper-casing than upper-casing."

Comment by Roman S. Borschel [ 25/Jan/10 ]

No, not more than I expected. It's mostly SQL and CLI basically. Thanks for the list!

We should try to go through that list before beta.

Comment by Roman S. Borschel [ 05/Mar/10 ]

Methods should be adjusted accordingly also (even though they're case insensitive). I already started with that. More to come.

Comment by Roman S. Borschel [ 28/Mar/10 ]

Most of the Cli => CLI changes seem to be complete now.

Comment by Roman S. Borschel [ 26/Apr/10 ]

Pushing the rest of the name changes towards beta2.

Comment by Roman S. Borschel [ 19/May/10 ]

More renamings have been done but still more to do. Pushing remaining work to beta3.

Comment by Roman S. Borschel [ 30/Aug/10 ]

Final name changes should be done prior to going into RC1.

Comment by Benjamin Eberlei [ 31/Oct/10 ]

there is much to be done yet, however most of it is also public API class names and might cause quite some BC breaks (i.e. DBAL Platforms and Schema Manager, Mapping Driver Names). I am not sure how to proceed here.

Comment by Roman S. Borschel [ 31/Oct/10 ]

Since PHP is mostly case-insensitive on class and method names it shouldn't be much of an issue.

Comment by Guilherme Blanco [ 16/Apr/14 ]

Scheduled to 3.0 as this may potentially create BC breaks

[DDC-213] Persist order of collections Created: 15/Dec/09  Updated: 27/Jun/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: 2.0
Fix Version/s: 3.0
Security Level: All

Type: New Feature Priority: Major
Reporter: Roman S. Borschel Assignee: Roman S. Borschel
Resolution: Unresolved Votes: 15
Labels: None

Issue Links:
is duplicated by DDC-181 Order of many-to-many relationship Resolved
is referenced by DDC-250 ArrayCollection Key Column @indexBy Resolved


A Collection is like a php array, an ordered map. Hence there should be the possibility to persist this order.

Comment by Christian Heinrich [ 21/May/10 ]

Roman, I'd like to do this one as I have currently a use case for this. Do you have any idea of how to do this? What I'm wondering is whether it is possible to implement this without user intervention. (This would simply mean "store the entities as they were added"). But this would need another column in DB that we'd have to add within oneToMany / manyToMany relationships, but in this case one could save a serialized array holding "entityId => position" key / value pairs.

Afterwards, one could easily rebuild / reorder the collection via $collection->set($entity, $order[$entity->identifier]);

If you've got another thought about this, please don't hesitate to point me into the right direction!

Comment by Benjamin Eberlei [ 22/May/10 ]

this won't be implemented until 2.1, since its a pretty complex feature.

Changes are probably required in:

1. CollectionPersister - Add a new collection persister that takes the position into account
2. SchemaTool - Add a 'col_position' column to either the many-to-many or the one-to-many tables.
3. EntityPersister - Use and extend current order-by support to make the sorting happen

You can implement this already though with some performance hit in update scenarios. If you use the ORDER BY support and implement an API around your entity that abstracts those changes and always sets a "position" field on the many entity that is supposed to be sorted.

Comment by Roman S. Borschel [ 22/May/10 ]

I don't think we necessarily need a new collection persister. Simply adjusting the ManyToManyPersister to be able to deal with it might be sufficient.

For OneToMany, that is always persisted from the "many" side, thus there is no collection persister, we would need to adjust the normal persisters.

They key element for the user should be a new annotation (or corresponding xml/yaml element) @OrderColumn. By default the order should not be persistent, only when an @OrderColumn annotation is present. The name of the order column can have a default, i.e. "position". Thus this enhancement of persisting the order should be fully backwards compatible.

Comment by Roman S. Borschel [ 22/May/10 ]

On another note, the getInsertDiff/getDeleteDiff methods of PersistentCollection should already be "ready" for this. That is, when an element in the collection changed only its position, this is already tracked as a change. However the ManyToManyPersister issues no "UPDATE" queries, it simply deletes and inserts. A position change may be more effectively persisted with an UPDATE.

Comment by Benjamin Eberlei [ 30/Sep/10 ]

From a mailinglist entry, required check/changepoints:

1. ClassMetadata of Many-To-Many associations have to be extended to publish the required datastructure to the ORM.
2. All Metadata Mapping Drivers have to be extended
3. Persisters\ManyToManyCollectionPersister has to be extended to save the key in the many to many table if desired by the user.
4. Schema-Tool has to be extended to create the additional column.
5. PersistentCollection has to be extended so that lazy loading of collections with additional key works.
6. Array- and ObjectHydrator have to be extended to allow fetch join of collections with key column.
7. Discuss wheather to support this for One-To-Many also with the key-column on the many side. This is much more tricky internally though.

Comment by Benjamin Eberlei [ 24/Dec/10 ]

Push back to 2.x, we will have support for DDC-250 first and for this at a later release.

Comment by Thomas Tourlourat - Armetiz [ 07/Feb/12 ]

Hi there,
I'm looking for this feature.

Benjamin Eberlei said that : "You can implement this already", but I don't understand the "how to".

The problem should be solve if RDBMS had a "natural" order. An order based on item position inside table.

To get this feature without any change on Doctrine, I have remplace the PK defined by the target & mapped field identifier. The new PK is a new field with type "integer" and with auto-increment enable.
In this configuration, Doctrine use the "natural" order of the RDBMS. And I can change order of my item inside Collection and persist it.

It's an very bad solution, but It work before an official support.

Waiting for advices, and solutions,


Comment by Thomas Tourlourat - Armetiz [ 08/Feb/12 ]

Answering to Benjamin Eberlei on the "7. Discuss wheather to support this for One-To-Many also with the key-column on the many side. This is much more tricky internally though.".

I think that for One-To-Many relations, if user want to store the collection order, Doctrine can store the One-To-Many as Many-To-Many with a "model" limitation.
In that case, if storing order collection for Many-To-Many work, it should work for One-To-Many.

What do you think about it ?

Comment by Nicolas [ 29/Feb/12 ]

I think that it must be possible to have two keys ordering : the order isn't obligatory reversible.

For exemple with user and group :

  • You can order groups for one user : with preference by exemple, or importance.
  • And with a different order, users for a group : rank by example.

And maybe more, if you decide to add multi-order : an user show group by his rank in it, if his rank is identical, the order is make by love preference, and after by the importance given by the user (not necessary a number, if we imagine filter on them).

So a default order can be choice with parametized fields and could be :

@JoinFields ( rank: { type: int} , preference:{type:int}, importance:{type: string, length: 40} )
@OrderByJoinFields({"rank" = "ASC", "preference"="ASC", "importance"="ASC" } )

In this case the order must be optional and would be clean if another order appears in the same scope (DQL...). And manytomany became virtual entities act as other entities except they don't appears permetting in the same time a better conception.

So if the solution take in DDC-181 will become the only solution. This would a good idea to document this. Because, this seems to me a very important point.

My last point is even an unique ordering field created in the join table will be a big and usefull improvement.

Thank a lot for your beautiful work.

Comment by Thomas Tourlourat - Armetiz [ 29/Feb/12 ]

In my point of view, a collection can be order in a single way only.
If you want to add more than one order between User & Group, it's a new collection, a new relation.
Like :

  • User.memberOf() : Group[]
  • Group.members() : User[]
  • Group.importantMembers() : User[]

And it's your role to keep a consistency between members & importantMembers array.

Because ManyToMany join table is the reflection of a state of an ArrayCollection. It's not a usefull feature to be able to store all of the state of an ArrayCollection, even the order of this Array. It's just a normal feature that is really missing


Comment by Nicolas [ 29/Feb/12 ]

I don't think:

If you have three collection, you duplicate one relation 3 times and it's easy in consequence to lost the data integrity and unicity.

By example :

  • Thomas have rank 10 in Admin
  • Thomas think the admin group has importance noted 3 on all of his groups.
  • If a responsable of admin group decide to delete Thomas from it.
  • Thomas, in his ordered list of groups, think always to be in group admin.

So in my idea, the many to many relation isn't just an array collection, but should be an virtual entity. In UML or in Merise method this is a common problem to have a parametized relation. I think an orm should just implement this.

Comment by Thomas Tourlourat - Armetiz [ 29/Feb/12 ]

I agree with you.. In a SQL Schema, it's a good choice to add many fields in a ManyToMany join table to description "order".

Comment by Thomas Tourlourat - Armetiz [ 07/Mar/12 ]

I just want to add a piece of Doctrine ORM Document :

"When working with collections, keep in mind that a Collection is essentially an ordered map (just like a PHP array). That is why the remove operation accepts an index/key. removeElement is a separate method that has O ( n) complexity using array_search, where n is the size of the map."

Comment by Thomas Tourlourat - Armetiz [ 23/Mar/12 ]

Hi there,
After several discussions. on IRC, I have changed my point of view.

Doctrine Documentation says : "When working with collections, keep in mind that a Collection is essentially an ordered map (just like a PHP array)".
So, I think that Doctrine have to be able to store or not the order of a Collection. By adding a new field on the Joined table to store the position of each elements.

But I not agree with @Nicolas. Because in his case, he's talking about Association Class : http://etutorials.org/Programming/UML/Chapter+6.+Class+Diagrams+Advanced+Concepts/Association+Class/
Because he's talking of a business logic, he's talking of a dedicated Entity class.

What do you think about it ?


Comment by Thomas Tourlourat - Armetiz [ 31/Aug/12 ]

Any news ?

Comment by Matthieu Napoli [ 16/Oct/12 ]

Hi, any news on this?

If I may add any info to this feature request, maybe something like JPA/Hibernate could be a good start?

See http://docs.oracle.com/javaee/6/api/javax/persistence/OrderColumn.html or http://docs.jboss.org/hibernate/core/3.6/reference/en-US/html/collections.html#collections-indexed

The idea in Hibernate is that you persist the order of the list in an extra column. This column is not a field of the entity however.

Comment by Albert Casademont [ 27/Jun/14 ]

+1, this would be indeed a very nice thing. We are using the @OrderBy annotation as a workaround but it's not quite the same thing

Comment by Marco Pivetta [ 27/Jun/14 ]

Moved to 3.x

Generated at Mon Jul 28 04:31:28 UTC 2014 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.