A related Github Pull-Request [GH-673] was closed:
https://github.com/doctrine/doctrine2/pull/673

BC break without advantages

A related Github Pull-Request [GH-673] was reopened:
https://github.com/doctrine/doctrine2/pull/673

irrelevant

Amazon Redshift

Ok this is much more complicated to allow then i thought. The problem is that the QueryComponents are passed by value, as an array, not by reference. That prevents changing them because this change wouldn't be visible in the output walker.

I can add a method to allow this in the OutputWalker for now, but generally this requires a bigger refactoring on the Query Components.

Added setQueryComponent() in SQL Walker to allow modification in output walker.

I'm afraid that this doesn't solve the initial problem at all. I'll try to describe it in more details to show what I mean. Suppose we have two doctrine extensions each of which contain its own tree walker. Each of these tree walkers need to modify AST and add new component to it (joined with some component already existing in the query). The first problem is that each tree walker has its own queryComponents array which is not passed between them, although they not necessary need to use queryComponents - they could use only AST. The second, bigger problem is that the Parser class does not know anything about modifications of queryComponents in tree walkers and cannot pass modified version to the OutputWalker. The goal of submitting this issue was to allow adding new components to the query in tree walkers which is not achievable by your fix. I think it may be the first step in the right direction. Maybe TreeWalkerAdapter should have public method getQueryComponents() which would be used by the Parser to pass modified queryComponents between different tree walkers and finally to the OutputWalker ? This would not break backward compatibility and solve this issue. What do you think about it?

I've tried to implement the solution mentioned in previous comment but it's also not so clean and easy as I thought. Each tree walker (including TreeWalkerChain) would have to implement getQueryComponents() and setQueryComponent($alias, array $component) methods. The same with SqlWalker, so the TreeWalker interface should have these methods, which would break BC in some way (walkers that do not inherit from SqlWalker or TreeWalkerAdapter will fail to compile). So maybe my first solution (PR #464) is not so bad for now? In the future queryComponents could be replaced by a special object or could be passed by a reference to allow modifications.

Marked as improvement as its not a bug.

A solution might probably implement an object holding all the QueryComponent, implementing ArrayAccess. So that way the state can be shared.

Just hit this while developing an ast walker... Will look into it too since I need it more than soon.

As a VERY UGLY workaround, I used a static variable and a custom sql walker in combination with my AST walker.

namespace Comcom\Versioning\ORM\Query;


use Doctrine\ORM\Query\SqlWalker;

class WorkaroundSqlWalker extends SqlWalker
{
    public function __construct($query, $parserResult, array $queryComponents)
    {
        parent::__construct($query, $parserResult, $queryComponents);

        foreach (VersionWalker::$additionalAliases as $alias => $value) {
            $this->setQueryComponent($alias, $value);
        }
    }
}

PHP 5.3.10-1ubuntu3.6 with Suhosin-Patch (cli) (built: Mar 11 2013 14:31:48)
Mysql version: 5.5.31-0ubuntu0.12.04.1 (Ubuntu)

We could say a duplicate of: DDC-2208

I have changed this into a feature request because you have hit the limitations of using inheritance and self referencing entities.

Doctrine2 cannot currently lazy load UserBase#$sponsor because we don't know which proxy we have to insert. It can either be UserPerson or UserCompany. In order to know this Doctrine2 has to query the actual object to determine its type. The current strategy is then to load the actual entity because we have all data anyway.

In order to implement this feature we need to insert a proxy instead of the actual entity. If we do that there should be no recursion happening.

Reduced priority

It'd be great if this is a configurable option.

MAMP

Can you update to at least 2.2.1 and try again, because this fix here http://www.doctrine-project.org/jira/browse/DDC-1652 look like it could be related to your problem.

We're already using the 2.2.x-dev package. It does look similar to DDC-1652

Added testcase on 2.1.x (not the right one unfortunately) branch: https://github.com/doctrine/doctrine2/pull/395

BTW I was adding said testcase on master and got an error similar to DDC-979

I upgraded the testcase to master locally, and it seems to fail on Array hydration only now, with a notice:

Exception: [PHPUnit_Framework_Error] Argument 1 passed to Doctrine\ORM\Internal\Hydration\ArrayHydrator::updateResultPointer() must be of the type array, string given, called in /home/benny/code/php/workspace/doctrine2/lib/Doctrine/ORM/Internal/Hydration/ArrayHydrator.php on line 196 and defined

I remember fixing something similar for ObjectHydration (which works for your testcases). Will investigate more when I have time.

Using Doctrine in Symfony 2.2.1 on Windows Platform

Is the composite key a mix of association and field types?

I'm trying to remember the usage as it was a short term project but I believe it is.

For example a user has a userid.

The table in question might have a primary key consisting of the userid and an int representing a year..

This is documented behavior and would just be an improvement

This is actually very hard to implement, the problem is that we only have ARRAY constants for PDO::PARAM_INT and PDO::PARAM_STR - all the other types would require special handling.

Attaching failing testcase.

The idea is to have something like "datetime[]" as type and detect this in the SQLParserUtils of DBAL.

Another approach would be to convert the values in the ORM already, before passing to the DBAL.

Hi Simon,

Could you please try to write a failing test case or paste your entities ?

Cheers

Verified by code review that this issue exists, but it will be very tricky to fix, because the null check is there for other reasons as well.

Linux 3.6.6-1.fc17.x86_64

error log from orm:validate-schema

Copying from pastebin:

use \Doctrine\ORM\Mapping as ORM;
use \xxx\Doctrine\Annotation\Entity as re;
use \xxx\Doctrine\Annotation\Forms as rf;
use \Doctrine\Common\Collections;
 
/**
 * @ORM\Entity
 */
class EventPicture extends \Picture
{
 
    /**
     * @ORM\ManyToOne(targetEntity="Event", inversedBy="eventPicture")
     * @ORM\JoinColumn(name="FK_Event", referencedColumnName="id")
     */
    protected $event;
 
}

use \Doctrine\ORM\Mapping as ORM;
use \xxx\Doctrine\Annotation\Entity as re;
use \xxx\Doctrine\Annotation\Forms as rf;
use \Doctrine\Common\Collections;
 
/** @ORM\MappedSuperclass */
class Picture extends \xxx\Doctrine\Entity\BaseEntity
{
 
    /**
     * @ORM\Id
     * @ORM\Column(type="integer")
     * @ORM\GeneratedValue(strategy="IDENTITY")
     * @var type
     */
    protected $id;
 
    /**
     * @ORM\Column(type="string",unique=true, nullable=false)
     *  @rf\FileUpload(fileSize="php",uploadType="local",fieldName="link",formControl="FileUploadField",image=true)
     *
     */
    protected $link;
 
}

kluk does this happen also with any other simple custom annotation? For example following:

/**
 * @Annotation 
 * @Target({"PROPERTY","ANNOTATION"})
 */
final class Entity implements Annotation
{
    /**
     * @var string
     */
    public $value;
}

the same error when using simple annotation.

 
<?php

use \Doctrine\ORM\Mapping as ORM;
use \xxx\Doctrine\Annotation\Entity as re;
use \xxx\Doctrine\Annotation\Forms as rf;
use \Doctrine\Common\Collections;

/** @ORM\MappedSuperclass */
class Picture extends \xxx\Doctrine\Entity\BaseEntity {

    /**
     * @ORM\Id
     * @ORM\Column(type="integer")
     * @ORM\GeneratedValue(strategy="IDENTITY")
     * @var type
     */
    protected $id;

   
    /**
     * @ORM\Column(type="integer")
     * @rf\SetClass({"class","hide"})
     */
    public $value;

    /**
     * @ORM\Column(type="string",unique=true, nullable=true)
     * @rf\FileUpload(fileSize="php",uploadType="local",fieldName="link",formControl="FileUploadField",image=true)
     *
     */
    protected $link;

}

When i remove $value , $picture from class everything goes ok.
Easy fix for me is change ClassMetadataInfo.

    /**
     * INTERNAL:
     * Adds a field mapping without completing/validating it.
     * This is mainly used to add inherited field mappings to derived classes.
     *
     * @param array $fieldMapping
     *
     * @return void
     */
    public function addInheritedFieldMapping(array $fieldMapping)
    {
        if(isset($fieldMapping['fieldName'])){
        $this->fieldMappings[$fieldMapping['fieldName']] = $fieldMapping;
        $this->columnNames[$fieldMapping['fieldName']] = $fieldMapping['columnName'];
        $this->fieldNames[$fieldMapping['columnName']] = $fieldMapping['fieldName'];
        }
    }

But i dont know if this fix can break another part of doctrine.

Can you put the code of your annotations online? I can't seem to understand why this happens.

Mysql 5.1.66 / Symfony 2.2.1

Can you show the CREATE TABLE and FOREIGN KEY statements of all the tables involved? It seems the cascade of the foreign keys is not propagated between multiple tables?

CREATE TABLE Base (id INT AUTO_INCREMENT NOT NULL, PRIMARY KEY(id)) DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci ENGINE = InnoDB;
CREATE TABLE Child1 (id INT NOT NULL, foreignKey INT NOT NULL, UNIQUE INDEX UNIQ_179B6E88E992F5A (foreignKey), PRIMARY KEY(id)) DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci ENGINE = InnoDB;

ALTER TABLE Child1 ADD CONSTRAINT FK_179B6E88E992F5A FOREIGN KEY (foreignKey) REFERENCES ExternalEntity (id) ON DELETE CASCADE;
ALTER TABLE Child1 ADD CONSTRAINT FK_179B6E8BF396750 FOREIGN KEY (id) REFERENCES Base (id) ON DELETE CASCADE;

The problem is that, the SQL model never explicitely tells the DB to delete the corresponding Base when Child1 gets removed. It looks like it is handled by the doctrine entity manager layer and not the actual DB engine (Base has no on delete cascade nor foreign key to its children).
So only doctrine can add the logic here because it knows the entity schema. But in this case, when it is deleted from another table, it looks like the special treatment is not triggered.

Maybe using

cascade={"remove"}

, instead of

onDelete="CASCADE"

to force the cascading process to be handled by doctrine would workaround the bug... But I prefer to have my DB do the logic work as much as possible.

Scheduling this for 3.0, when we move to php 5.4 or higher requirement

OSX

I wrote a test showing the issue.

https://github.com/NoUseFreak/doctrine2/commit/8574b79fd3d245532bbe7e310c5cbe083892057a

This is not a bug, because restoring queries is not yet a feature of the QueryBuilder. Marking as possible improvement for future.

This is a known issue that we don't have found a BC fix for and as I understand Guilherme Blanco requires considerable refactoring.

This seems to be a valid issue to me.

This implementation is the actual solution to associations retrieval that are inherited (type joined).

Example:

/** Joined */
class Base {}

class Foo extends Base {}

class Bar {
    public $foo;
}

// This causes the CTI to link as INNER JOIN, which makes the result become 0
// il if you have no Foo's defined (although it should ignore this)
$q = $this->_em->createQuery('SELECT b, f FROM Bar b LEFT JOIN b.foo f'); 

Yes, this is a possible solution for DDC-512 but on the SQL level. I still don't see this as appropriate for DQL, it just doesnt make sense to me, DQL joins object associations, there is no precedence.

So, no, this has nothing to do with DDC-512. DDC-512 can even be fixed differently as outlined in my comments there.

On a side note I would still like to know/see the following for this issue:

• Some realisitic DQL examples where this feature would be essential, i.e. there is no other way to do it.
  This also means explaining what the impact on the resulting object graph is and why it makes sense.
• Which other ORMs support this on the OQL/Criteria level?

So far, my stance on this issue is:

1) It doesnt make sense (semantically) in DQL
2) Its rarely needed
3) When you really need it you can use a NativeQuery anyway and use this nesting in SQL, where it probably belongs and makes more sense
4) It would (unnecessarily) complicate DQL

Thus I am currently leaning towards "Wont fix" for this issue.

Hi Roman. I understand your doubts, and I have been breaking my head over
creating a realistic example the last few hours that would hopefully convince
you for implementing this feature. But actually I cannot find one that you wouldn't
consider to be trivial. I do have a number of very complex optimized queries written
for sportskickoff dot com (using Doctrine 1.2) but they are probably hard to understand
because they may not be selfdescribing. Below is one example literally ripped from
the application. Still they often can be broken down to my example query in this
ticket's description, but applied grouping, additional other joins on the root component
and/or other criteria made them impossible to rewrite using subselects or choosing
another root component. Most often they just performed way best using the nested
syntax and saved me a number of additional queries.

SELECT A.id, A.username, A.balance, COALESCE(SUM(B.stake), 0) AS sumstake, COUNT(B.id) AS nrbets
FROM account A
LEFT JOIN (
bet B
INNER JOIN game G ON G.id = :GAMEID AND B.timestampcompletion BETWEEN G.timestampstart AND G.timestampend
) ON B.accountid = A.id AND B.timestampcompletion IS NOT NULL
WHERE A.Status & :ACTIVEORDISQUALIFIED = :ACTIVE
GROUP BY A.id, A.username, A.balance
ORDER BY A.balance DESC, sumstake ASC, nrbets ASC, A.username ASC

But let's put it another way. I would also like this feature to be supported in DQL
because I just do not want to use native queries. Why would I want to use native
queries if it can be done using DQL? In DQL I work with class names and field
names, and they may differ from the underlying table and column names. Doctrine
takes care of that mapping based on my schema/annotations and I do not
have to "know" these mappings. In native queries I suddenly do have to "know"
these mappings. I use Doctrine because it makes my application portable and
enables me to work with my database in an OOP way like I do in my model,
abstracting things. The need for native queries partly reverts the benefits Doctrine
offers in the first place.

Btw, I recall to have successfully used the nested join syntax in HQL (.NET Hibernate)
but I cannot find examples on the web or a BNF notation.

Furthermore, in reply to your stances:
1) It indeed doesnt make sense (semantically) in DQL, it only makes the result
set different, but not the way data is hydrated into objects;
2) Its indeed rarely needed for inserting, updating and populating basic lists but
it allows you to better select what combinations of associated rows are joined
and which not in more optimized queries without having to use native queries,
or because they perform better than using subseletcs and alike.
3) Not having to use native queries is just an extra reason for using Doctrine and
maintains the abstraction the ORM provides througout on'es whole application
4) Why would it complicate DQL, if people do not know about or understand
the feature it wouldn't matter because not using parenthesises is the default
way to specify joins?

Well, this is it, can't find any more words to promote and make you enthusiastic.... lol.

Ok, I have not given up yet... , here's a "stupid" example.

Imagine a book store that sells books of various authors and keeps track of those sales.
Let's say you would have an admin page that lists all authors, and for each author
its also shows the books and their sales dates since january 1st, but only for those
books that were actually sold and contain an A in its name. An optimized SQL query
to fetch all the information at once would be something like:

SELECT A., B., S.*
FROM author A
LEFT JOIN (
book B
INNER JOIN sale S ON S.book_id = B.id AND S.dt >= '2010-01-01'
) ON B.author_id = A.id AND A.name LIKE '%A%'

In DQL it would then be something like:

SELECT A., B., S.*
FROM author A
LEFT JOIN (
book B
INNER JOIN sale S WITH S.dt >= '2010-01-01'
) WITH A.name LIKE '%A%'

If the database would contain thousands of books, but sales for just a
few books, this will definitely perform better than using subselects.
Off course one would like to fetch array graphs instead of objects for
further optimization, but this hopefully shows my point.

I have attached a test casefor a similar query, though without the additional
join constraints for clarity. I surely hope you can consider it.

One last note, you shouldn't be afraid that nesting joins is not in the
ansi SQL spec. Select queries are about record sets and products
between these sets, tables are just the basic means of providing record
sets to the query. This is an important terminological difference to think about.
Specifying precedence with parenthesis around joins is a logical and
natural evolution of the ansi sql standard. For example views are a good
proof of this concept, I could define book B INNER JOIN sale S as a view
and LEFT JOIN that to authors to get effectively the same result
set as the above example. The database server would internally perform the
same query (though may additionally take indexes on the view into account).
That said, rdbm's that support this syntax would certainly never drop the
feature, as its not a feature but just plain logical and smart querying!

P.S. I had a hard time finding out how to run the test cases, I could not find
it in the Doctrine 2 documentation, development wiki, cookbook or any other
place, while finally it was as easy as running phpunit Doctrine_Tests_AllTests
from within the tests/ directory, or just phpunit Doctrine_Tests_ORM_Functional_Ticket_DDC349Test
for my test. Could you please add some info about this somewhere, it might
save others some googling.

Test case as SVN patch using a parenthesized join.
Just remove the parenthesises from the query to have it fail...

@"The need for native queries partly reverts the benefits Doctrine offers in the first place."

That is something I hugely disagree with. Neither SQL abstraction, nor database vendor independence is the main purpose of an ORM like Doctrine 2.
It is the state management of your objects, the transparent change tracking, lazy-loading and synchronization of the object state with the database state and nothing of this gets lost when using native queries.

We could rip out DQL and any other querying mechanism except a basic find() (and lazy-loading, of course), only providing the native query facility and even only supporting MySQL and would still retain all the core ORM functionality.

NativeQuery is one of the best and core "features" of the project. It is even the foundation for DQL. A DQL query is nothing more than an additional (beautiful) abstraction but what comes out is a native query + a ResultSetMapping, the same thing you can build yourself in the first place, even using the mapping metadata to construct the query. Nothing forces you to hardcode table and column names in native queries if you don't want that. Just use the mapping metadata, DQL does the same.

SQL abstraction and database vendor independence is icing on the cake, not the heart of the ORM.

Benjamin Eberlei, Is this an expected behavior ?

I mean.. This issue is about dispatch the event before copy the original values into the managed instance.
But overall, should $em->detach() trigger @PrePersist events ?

Fabio B. Silva he talks about $em->merge() on a detached entity calling pre persist. This should only happen on a NEW entity, not on a DETACHED one.

I tend to disagree with the statement above about pre persist that should not happen on a detached entity being merged back in. If this event handler contains a business logic that this entity needs to be checked against and the detached entity was modified before the merge operation in a way that invalidates it in the prePersist than I will end up with the invalid entity in the identity map. If the merge operation calls persist it must run the prePersist event handler as well for consistency.

If there is a logic that prevents persisting invalid entities why should it bypassed in the merge operation?

Windows 7, Apache 2.2, MSSQL Server, PHP 5.3.3

Hi, i try to make my custom type for binary field in MSSQL.
I don't find own, can you please show me your custom type binary?
Thks a lot.

The problem is that changing ClassMetadata after generating it from the cache is not really supported and depends on the Internal State of other classes. Have you tried creating a completly new EntityManager and then directly setting this? It could be that the SQL for the entity was already generated inside Doctrine, with the ID Generator information at IDENTITY_AUTO.

> The problem is that changing ClassMetadata after generating it from the cache is not really supported

Yeah, it is a problem indeed, why set ticket status to resolved ?
Do you think it's normal to have a public method that trigger a fatal error ?

Please fix it or put setIdGeneratorType as private, or AT LEAST add a context exception ...

Almost every interaction with metadata outside the `loadClassMetadata` event will cause unexpected problems. I don't think throwing an exception there helps in any way.

@marco pivetta

The generation of the actual exception comes from DBALException on the query excetion and point a bad generated query (Invalid parameter number),
when the problem comes from setting ClassMetada, and concerns a problem of cache generated after loadClassMetadata.

Adding an exception is just the fast way pointing where the problem comes from and that "setting metadata after loadMetadata is not supported anymore". (It will spare developper's time that used to set metadata, but also help future contribution)

> Please fix it or put setIdGeneratorType as private, or AT LEAST add a context exception ...
Note: BTW, my favorite solution would be to fix it (re-generate cache, or edit cache, or disable cache or whatever)

Hi Quintenvk

Could you please try to write a failing test case ?

Thanks

I added a testcase. Please note that the database settings are to be configured in Core/simplys/simplys.php, and that the dump is in dummy.sql.

Apart from that all should run well immediately.

Fabio,

Please check the zip I just attached. I hope this helps you in finding the problem.

Thanks,
Quinten

Thanks Quintenvk,

SELECT p.barcode, p.id, p.name FROM \core\Simplys\Entity\Products p INDEX BY p.barcode JOIN p.businessid b INDEX BY p.id

In this DQL you are trying to index by scalar values,
I think we does not support that, and a single dimensional array is the expected result in this case.

Also the INDEX BY documentations seems wrong to me.

The given DQL :

 SELECT u.id, u.status, upper(u.name) nameUpper FROM User u INDEX BY u.idJOIN u.phonenumbers p INDEX BY p.phonenumber 

Show the following result :

array
  0 =>
    array
      1 =>
        object(stdClass)[299]
          public '__CLASS__' => string 'Doctrine\Tests\Models\CMS\CmsUser' (length=33)
          public 'id' => int 1
          ..
      'nameUpper' => string 'ROMANB' (length=6)
  1 =>
    array
      2 =>
        object(stdClass)[298]
          public '__CLASS__' => string 'Doctrine\Tests\Models\CMS\CmsUser' (length=33)
          public 'id' => int 2
          ...
      'nameUpper' => string 'JWAGE' (length=5)

Which IMHO represents another DQL, something like :

 SELECT u, p , upper(u.name) nameUpper FROM User u INDEX BY u.id JOIN u.phonenumbers p INDEX BY p.phonenumber

Thanks for your reply Fabio.
Do you think there could be alternatives (apart from a foreach-loop) to achieve the expected result?

Thanks,
Quinten

Not sure if it's exactly the result you need but you can try

Something like :

SELECT p, b FROM \core\Simplys\Entity\Products p INDEX BY p.barcode JOIN p.businessid b INDEX BY p.id

or something like :

SELECT PARTIAL p.{id, barcode, name}, b.{id, attributesYouNeed} FROM \core\Simplys\Entity\Products p INDEX BY p.barcode JOIN p.businessid b INDEX BY p.id

And than :

$result = $query->getArrayResult();

Both produce the same result as the query I had. I think i'll move on to loops after a bit more research, too bad it can't be done (at least for now) though... Would've been nice.

Thanks for your help though!

LAMP, debian squeeze

Enea Bette Can you attach the entities (stripped down to the fields we need here)?

Can you check guilherme? This looks really weird.

It should be:

WHERE d0_.sys_user_owner = d3_.user_id AND d0_.sys_group_owner = d4_.group_id

Enea Bette Can you please provide your entities?
I can try to reproduce the issue, but I need your entities as a base for a failing unit test.

I'm having a similar problem with the query:

SELECT um, p FROM Ciwwic\AppBundle\Entity\Provider p LEFT JOIN Ciwwic\UserBundle\Entity\UserMeta um WITH um.user = p.id WHERE p.id = 30

When you run this query DQL she returns an empty array.
I getting solve my problem by adding WHERE clauses example as:

SELECT p, um FROM Ciwwic\AppBundle\Entity\Provider p LEFT JOIN Ciwwic\UserBundle\Entity\UserMeta um WHERE p.id = 30 AND um.user = 30

Hi Enea

If i got it correctly
Your associations DmsObject#ownerUser and DmsObject#ownerGroup are flipped.
Note that ownerUser points to sys_group_owner and ownerGroup to sys_user_owner

/**
 * @ORM\ManyToOne(targetEntity="library\system\business\User", fetch="EAGER")
 * @ORM\JoinColumn(name="sys_group_owner", referencedColumnName="ID")
 */
protected $ownerUser;
/**
 * @ORM\ManyToOne(targetEntity="library\system\business\Group", fetch="EAGER")
 * @ORM\JoinColumn(name="sys_user_owner", referencedColumnName="ID")
 */
protected $ownerGroup;

It should be :

/**
 * @ORM\ManyToOne(targetEntity="library\system\business\User", fetch="EAGER")
 * @ORM\JoinColumn(name="sys_user_owner", referencedColumnName="ID")
 */
protected $ownerUser;

/**
 * @ORM\ManyToOne(targetEntity="library\system\business\Group", fetch="EAGER")
 * @ORM\JoinColumn(name="sys_group_owner", referencedColumnName="ID")
 */
protected $ownerGroup;

formated snippets nicely

I need this feature too.

But I would suggest using the same annotation used by JPA

@Embeddable

+1

You should also take into consideration different storage strategies of ValueObjects.

Martin Fowler points out - in „PoEAA" - two approaches: Embedded Value (which is the one presented above) and Serialized LOB .
Both have their pros and cons, that's why Doctrine2 should give developers choice of selecting the fittest solution.

Of course technically we can similate a serialized LOB with a new Doctrine 2 type.

I don't like that idea - Its so not generic.

VO as a pattern is important building block of domain model, which clearly indicates that VO as a feature of Doctrine2 should be tailor-made.

To anyone of dev-team reading this issue: without VOs Doctrine is not yet DDD-ready, please hurry

Serialized LOB is not very useful IMHO and has lots of problems (many mentioned in PoEEA already).

@Alan: I appreciate your nice reminder and I'm sure you mean it in a friendly way, but please keep in mind that noone is paid to work on this project. It all happens in free/spare time and the current state of the project already consumed at least 1 1/2 years spending many hours weekly on this project from me alone. Not to speak of the others.

Thus, there is no point in demanding something or telling us to hurry. The best way to get a feature in is to provide a (good) patch that we find worth including.

I started to add notes to this issue to collect all the things that need to be done for this feature.

In the meantime, its not too hard/ugly to get a half-way decent embedded value yourself:

/** @Entity @HasLifecycleCallbacks */
class Foo {
    // annotations not shown
    private $id;
    private $embedded;
    private $value1; // never reveal to public
    private $value2; // never reveal to public
    private $value3; // never reveal to public

   public function getEmbedded() {
       return $this->embedded;
   }

   public function setEmbedded($embedded) {
       $this->embedded = $embedded;
   }
   
   /** @PrePersist @PreUpdate */
   function _destructEmbedded() {
       // destruct $embedded into $value1, $value2, $value3
   }

   /** @PostLoad */
   function _constructEmbedded() {
      // construct $embedded from $value1, $value2, $value3 
   }
}

Several variations of this are possible, also with an external event listener instead of callbacks but in that case you might need to use reflection to get at the values.

I want to share my thoughts on possible VOs collections implementations.

1. As it was mentioned earlier serialized (C)LOB is one solution. Implementation of storing/retrieving object graphs alone is quite simple, but it's complex in terms of SELECTs with conditions.
Composing SQL condition would result in some nasty constructions e.g. vo_collection_column LIKE '%foo%bar%' which output format would depend on serialization target (CSV, XML, YAML, PHP serialized objects etc.). Also in most cases it would be impossible to obtain eligible result.

I'm not taking Regexp or XPath operators into consideration as only few RDBMS support them.

2. The second solution is to break VOs graph into separate related table... or tables if we consider that VO can contain another VO(s). It's not so fast as serialized LOB but more flexible and it utilize power of RDMS,
But there is one catch: Doctrine2 must preserve nature of VO. To make it happen during Entities persisting - if any change in dependant VOs graph has been made - all associated VOs rows in database should be deleted and the new/changed VOs graph should be inserted in their place.
I know it could be inefficient while dealing with large object graphs, yet faster than comparing VOs one-by-one.

In conclusion:
serialized LOB is extremely fast in CRUD-like operations on aggregates, however very search unfriendly.
Separate ValueObjects tables are better where serialized LOB lacks, but slower in exploitation.

I can't tell which approach is superior, because each of them is valid under different circumstances.

Hope this helps.

@Alan: I appreciate your nice reminder and I'm sure you mean it in a friendly way [...]

Of course I do.

It would be easy to implement value objects in userland using the XML capabilities of many RDBMS:

1. Implement an Xpath function on the Dql Parser
2. Implement a User-Defined Type for each value object that handles the translation from and to XML.

The second point can be heavily optimized when value objects are immutable with an own identiy map of value types inside the Type flyweight instance.

I more or less suggested something similar above.

ah, my bad - i must have overseen this

+1

This would be awesome.

Don't forget (especially with regard to SLOBs) that values might in turn contain references to Entities.

Example: An "Order" might be an @Entity and might have a field (an array) of OrderLineItems as value. Each OrderLineItem might e. g. carry quantity or disconunt and references a Product (@Entity).

So even if you don't need the traversal from Product to all the Orders it is contained in, serializing the OrderLineItems needs a way to "cut off" the object graph at the transition towards the Product but must place some kind of referral there so that upon unserialization (of the OrderLineItem list, that is, during Order load) the Product references in every OrderLineItem are at least initialized with proxies again.

Don't know whether/how referential integrity (OrderLineItems <-> Products) would make sense or could be implemented here.

Pushed back to 2.x, this feature is probably the largest feature request we have and we'd rather focus on small improvements for 2.1

Several thinks to consider/not to oversee here:

1) There are value objects with identity. I know that is not DDD-conform but only at first sight. It means they are technically entities but are treated like VOs.
Common examples are Zipcode or country. As they have identity (e.g. Zipcode: de-40723) they are entities but are created and interchanged like normal VOs.
On the google DDD-List they were often referenced aS Lookup Entities.

2) In virtually all (business) cases a collection of VO is an Entity. How else could you reference (add or remove) single elements of that list?
There are exceptions here like a undefinded number of VOs in a collection, but in that case you can only add or remove a quantity of it.
As a true collection (say 3 addresses for a client = Entity ClientAdresses) you would have to give them some kind of identity, even if it is only having
a sequential number in that collection.

@Matthias: OrderLineItems is an example of actually being an Entity.

Hi guys. I face this in my own way. Hope you won't wake up your neighbours with loud laugh.

Every @Entity extends my BaseEntity object which provide kind of wrap for value with ValueBase object. So when want to get/set value from entity you call $entity->getData() where you won't get value "data" but wrapping ValueBase for value "data". Then you can get bare value by getValue(). Name of value class is in annotation and would be child of ValueBase.

There's also parent class Base for EntityBase and ValueBase. In my case class Base is something like HTML element. So in the end you can use $entity->renderHtml() or $value->renderHtml() no matter if you're rendering value or @Entity. There's more features like validation, filtering and hydration value/entity from HTML forms, but it's extra.

Implementation:

 
 /* @MappedSuperclass */
abstract class Base {
  /* there're methods like _getParent(), _getPropertyName(), etc. used in code behind */
}

 
abstract class ValueBase extends Base { 
   public function getValue() {
        return $this->_getParent()->{$this->_getPropertyName()};
   }
   
   public function setValue($value) {
        $this->_getParent()->{$this->_getPropertyName()} = $value;
   }
}

 
/** @MappedSuperclass */
abstract class EntityBase extends Base {
    public function __call($name, $arguments) {
        /* get property object */
        $pattern = '/^get(.*)$/u';
        preg_match($pattern, $name, $matches);
        if (isset($matches[1])) {
            $propertyName = lcfirst($matches[1]);
            return $this->get($propertyName);
        }

        /* set entity */
        $pattern = '/^set(.*)$/u';
        preg_match($pattern, $name, $matches);
        if (isset($matches[1])) {
            $propertyName = lcfirst($matches[1]);
            return $this->set($propertyName, $arguments[0]);
        }
    }

    public function get($propertyName) {
	    $property = $this->_getElementProperty($propertyName);

	    if ($property == null)
		throw new Exception(sprintf("There isn't property like '%s'.", $propertyName));

	    /* for collections and entities */
	    if ($property["type"] == "collection" || $property["type"] == "entity") {
		$element = $this->{$propertyName};
		if ($element != null) {
		    $element->_setParent($this);
		    $element->_setPropertyName($propertyName);
		} elseif ($property["type"] == "entity") {
		    $element = new $property["class"];
		    $element->_setParent($this);
		    $element->_setPropertyName($propertyName);
		    $element->_setNullEntity();
		    $this->{$propertyName} = $element;
		}
		return $element;
	    }
	    else {
	    /* for values */
		if (!isset($this->_loadedEntities[$propertyName])) {
		    $this->_loadedEntities[$propertyName] = new $property["class"]($this, $propertyName);
		}
		return $this->_loadedEntities[$propertyName];
	    }
    }

    public function set($propertyName, $value) {
        $property = $this->_getElementProperty($propertyName);

        if ($property == null)
            throw new Exception(sprintf("There isn't property like '%s'.", $propertyName));

        /* for collections and entities */
        if ($property["type"] == "collection" || $property["type"] == "entity") {
            $this->{$propertyName} = $value;
        }
        /* for values */ else {
            throw new Exception(sprintf("Can't call set on value property '%s'.", $propertyName));
        }

        return $this;
    }
}

Note that there's something i call "NullEntity". Instead of getting bare "null" you'll get @Entity child of EntityBase, where is set property nullEntity. Then there's posibility to work with null entity (for example renderHtml with empty inputs).

It would be nice, if this is support by Doctrine natively, because i have some performace problems with my implementation. If it's interest in my whole code i can send you. But of course there's some security holes so i'll send it privetely. Thanks for understand and for Doctrine of course.

Note that Roman's workaround presented here does not work.

   /** @PrePersist @PreUpdate */
   function _destructEmbedded() {
       // destruct $embedded into $value1, $value2, $value3
   }

Doctrine tracks changes and does not perform updates when no changes are found. $embedded is not mapped, so it's not tracked and won't be taken into account by Doctrine when updating. Therefore, if $embedded is the only value that was changed, the PreUpdate event won't be triggered.

The easiest thing to do is to simply destruct the VO on every mutation:

   public function setEmbedded($embedded) {
       $this->embedded = $embedded;
       $this->_destructEmbedded();
   }

The downside is that you need to remember to call the method in every setter, but apart from that, there are no side effects, it always works and it's just one line of code

_constructEmbedded() keeps working as is, postLoad will always be triggered.

Hi,

This feature would be awesome !
VOs are really essential in a good domain design.

If you plan to implement this, please remember that you can have nested VOs.
Take the design for a Booking process for instance, you would have a DateRange object embedding two DateTime objects (in the simplest case).

I have no doubts that you've already took this in consideration, but i prefer pointing this out, just in case

work has been started, https://github.com/doctrine/doctrine2/pull/265

Does the new "complex sql types" feature help here - I mean, could that be used to map a value object to more than one column in the database?

@Benjamin Eberlei The request seems to be closed in the link you provided! Does that mean that this feature won't be implemented?!

songoko songowan no, it just probably wasn't the correct way of implementing this

I'm curious if any effort is currently being put into this. I would really love to have this feature available.

Daniel Pitts this is being developed in DDC-2374 ( https://github.com/doctrine/doctrine2/pull/634 )

Not a blocker

What's the result of `EXPLAIN` on a query without the subquery?

explain without the subquery

Sergey Gerdel that's not the same query.

Sergey Gerdel this is still using

Using index; Using temporary; Using filesort

Check your indexes

Not in the index problem

SELECT DISTINCT id0 FROM (SELECT m0_.id AS id0, m0_.title AS title1, m0_.text AS text2, m0_.price AS price3, m0_.originalPrice AS originalPrice4, m0_.condition_type AS condition_type5, m0_.image_1 AS image_16, m0_.image_2 AS image_27, m0_.image_3 AS image_38, m0_.image_4 AS image_49, m0_.image_5 AS image_510, m0_.video AS video11, m0_.contact_email AS contact_email12, m0_.contact_name AS contact_name13, m0_.contact_phone AS contact_phone14, m0_.contact_type AS contact_type15, m0_.published AS published16, m0_.type AS type17, m0_.status AS status18, m0_.highlight AS highlight19, m0_.urgent AS urgent20, m0_.topads AS topads21, m0_.period AS period22, m0_.hits AS hits23, m0_.ip AS ip24, m0_.created_at AS created_at25, m0_.updated_at AS updated_at26 FROM milla_message m0_ WHERE m0_.status = 1 ORDER BY m0_.published DESC) dctrn_result LIMIT 20 OFFSET 0

Time: 104.614s explain 3

SELECT DISTINCT m0_.id AS id0 FROM milla_message m0_ WHERE m0_.status = 1 ORDER BY m0_.published DESC LIMIT 20 OFFSET 0;

Time: 0.001s explain 4

Sergey Gerdel the ORM cannot simplify a complex query that way. There may be a conditional on one of the joined results, or generally usage of one of the joined results.

Things that could be optimized here are:

• Removal of the `ORDER BY` clause when grouping (check ORM master, I think somebody already did that)
• Trying to simplify the query by doing some serious hacking on the AST.

The problem I see here is that the chance to spawn random bugs because of the optimization is very high, and you'd have to rewrite `walkSelectStatement`

Marking as improvement

Minor?
i have 100 sec for this query.
200k items are selected for temporary table. wtf?

OK. Programmers may be mistaken in parser
expect ORDER BY m0_.published DESC LIMIT 20 OFFSET 0) dctrn_result
Time: 0.001s

reality ORDER BY m0_.published DESC) dctrn_result LIMIT 20 OFFSET 0

Sergey Gerdel this problem does not introduce security issues and can be worked around by you while using your own pagination logic. It does not stop you from doing anything, that's why it's minor.

ok)
i have already created my own paginator.
at last
please see how to fix this problem
https://github.com/Sergic/doctrine2/commit/2733c815387273d3bd199a68acb717e0cbc8ccfe

Hi SergSW

Could you try to write a failing test case ?

Thanks

SSW/TestBundle with the problem

I install the Symfony v2.0.18. and made small TestBundle.
I made schema database, by CLI "console doctrine:schema:update --force"
Result: Database schema updated successfully!
But I saw that I lost a field 'user_id' in a table 'AttachTree' (see Attach)

MySQL dump

Adjusted example formatting, don't apologize for your English, thanks for the report!

What version of 2.1 are you using? We don't actually support 2.1 anymore. Inheritance has always worked as used in hundrets of unit-tests, this changes look quite major a bug to have been missed before. I can't really explain whats happening here.

SergSW news?

Hi. I got an email notification that arbuscula has changed the status to "Awaiting Feedback". Do you need any feedback from me?

Here is the way I've implement the solution on my side: (for oracle)

into Doctrine/DBAL/Statement.php, I've add this method:

/**
     * Binds a parameter value to the statement.
     * This is implemented this way for oracle only. Other drivers are redirected to bindValue method.
     *
     * The value will be bound with to the type provided (that required to be a table type).
     *
     * @param String $name The name or position of the parameter.
     * @param Array $value The value of the parameter.
     * @param String $type The name of the type to use to bind.
     * @return boolean TRUE on success, FALSE on failure.
     */
    public function bindList($name, Array $value, $type)
    {
        if ('oracle' !== $this->platform->getName())
        {
            $this->bindValue($name, $value, $type);
        }
        else
        {
            return $this->stmt->bindList($name, $value, $type);
        }
    }

into Doctrine/DBAL/Driver/Statement.php I've add:

/**
     * @TODO: docs
     */
    function bindList($param, Array $values, $type);

into Doctrine/DBAL/Driver/OCI8/OCI8Statement.php I've add this method:

/**
     * {@inheritdoc}
     */
    public function bindList($param, Array $value, $type)
    {
        if (!($list = oci_new_collection($this->_dbh, $type)))
        {
            //throw new OCI8Exception::fromErrorInfo($this->errorInfo());
        }

        foreach ($value as $entry)
        {
            $list->append($entry);
        }
        if (!oci_bind_by_name($this->_sth, $param, $list, -1, OCI_B_NTY))
        {
            //throw new OCI8Exception::fromErrorInfo($this->errorInfo());
        }
    }

// NOTE: we should probably add the bindList to all driver Statement object.

into your code you can use it this way:

$sql = "
    SELECT *
    FROM test
    WHERE id IN
    (
        SELECT *
        FROM
        (
            CAST (: p_ids AS list_int_type)
        )
    )
";
$stmt = connection->prepare($sql);
$stmt->bindList(': p_ids', $ids, 'list_int_type');
$stmt->execute();
$rs = $stmt->fetchAll(PDO::FETCH_ASSOC);

NOTE:
list_int_type need to be a valid oracle data type. You can create one with the name you want.
example:
you can have 2 type of accepted array of values: integer and string
let's say we create one for string named: list_str_type and one for integer list_int_type

create or replace type list_str_type as table of varchar2(4000);
create or replace type list_int_type as table of number;

Hey Marc Drolet

thanks for the feedback and the solution, however i would like to have something generic that is working independent of the database driver. This code is very specific.

Can you point me to some documentation why oci collection works with more than 1000 elements and how it works in PHP?

Hi Benjamin,

The limitation is not from the oci driver, it's an oracle limitation. There are a couple of possible solution/implementation that can be done but the one I've provide is the one that perform better for the test I've done and from what I can found over the blogs I've read.

I can't find the exact documentation of oracle. oracle doc is so poor.
Here is the best description link I can provide that describe some possible implementation.
http://vsadilovskiy.wordpress.com/substituting-a-collection-for-in-list-performance-study/

I don't know if there is similar limitation with other database. With the implementation I've provided, It will be possible to implement the proper solution depending on the database limitation you face otherwise it will execute the generic IN. What's bad, we need to create the type into the database.

NOTE: In my case, I can not perform a sub-query, I get the my collection from a web service call.

Hi

Could you please attach your entities or a failing test case ?

Cheers

sure

LineRelated.php :

class LineRelated
{
    /**
     * @ORM\Id
     * @ORM\Column(type="integer")
     * @ORM\GeneratedValue(strategy="AUTO")
     */
     protected $id;

    /**
     * @ORM\ManyToOne(targetEntity="Line", inversedBy="line_related")
     * @ORM\JoinColumn(name="line_id", referencedColumnName="id",nullable=false)
     */
     protected $line;
     
    /**
     * @ORM\Column(name="line_id_related", type="integer")
     */
    protected $line_related;

    /**
     * @ORM\Column(type="smallint",nullable=false)
     */
     protected $count = 0;

###### get/set etc....... #######

Line.php

class Line
{
    /**
     * @ORM\Id
     * @ORM\Column(type="integer")
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    protected $id;
########## blablabla #############

my action:

    public function right_line_relatedAction($line = null, $title='相关线路')
    {

        $lines = $l->getByUser($user, array());
//anyway,$lines is an array,It has several elements,each element is an instance of LineEntity.

        $related = $this->getDoctrine()->getRepository('MyWebBundle:LineRelated')->findBy(array('line' => $lines), array('count' => 'DESC'), 20);
//this findBy function generate the sql which is slow.

        return $related;
    }

Hi,

How did you get this query string ?

Repository#findBy does not quote the values, It uses PDO:bindParam.
so the expected query string should be someting like :

WHERE t0.line_id IN (?, ? ,?) 

I tried to reproduce but in my tests the generated Query binds the parameters as "PDO::PARAM_INT".

I have added a test case.
Could you please can try to change it and make fails.

Cheers

reproduced :

SELECT t0.id AS id1, t0.line_id_related AS line_id_related2, t0.count AS count3, t0.line_id AS line_id4 FROM line_related t0 WHERE t0.line_id IN ('6059', 4851, '6068', 126352, '6060', '1000000') ORDER BY t0.count DESC LIMIT 20
Parameters: [['6059', 4851, '6068', 126352, '6060', '1000000']] 
[Hide runnable query]
Time: 234.53 ms [   Explain query ]

let me have a look on what's going on

interesting. I've dump(using ladybug_dump) the $lines,and I found out that when the element is a Proxies Object(Object(Proxies_GC_\My\WebBundle\Entity\Line)),then the id of that Object will be with quoted,when the elememt is an Real Entity,then It will be without quote.

for example,in my last comment, the parameters is [['6059', 4851, '6068', 126352, '6060', '1000000']]
the result of dumping $lines is :

array(6)
[0]: object(Proxies_CG_\Zuo\WebBundle\Entity\Line)
[1]: object(Zuo\WebBundle\Entity\Line)
[2]: object(Proxies_CG_\Zuo\WebBundle\Entity\Line)
[3]: object(Zuo\WebBundle\Entity\Line)
[4]: object(Proxies_CG_\Zuo\WebBundle\Entity\Line)
[5]: object(Proxies_CG_\Zuo\WebBundle\Entity\Line)

tell me if you need more information. thanks

This may be because

$_identifier

in proxies ( https://github.com/doctrine/doctrine2/blob/42e83a2716d19eada4f1cd49ece77d5f5229a239/lib/Doctrine/ORM/Proxy/ProxyFactory.php#L383 ) is not necessarily composed by integers. This could be fixed with DCOM-96. I'll add the tests to my development branch and will ping you back tomorrow

thanks

I see what is going on here... But this should not be a problem anyway, since they're bound anyway as "PDO::PARAM_INT", as Fabio B. Silva told you.

That's only a problem with the logger showing them as string. PDO will handle the conversion before the value hits the DB as far as I know.

I can understand your point,but what I don't really get is that the execute time of sql is very long,that explained the quote should be in the sql,not like what you said,that's only a problem with the logger.

scourgen can you profile the difference directly in CLI? What about checking the bound parameter type? Are those values bound as INTs in your case?

@ocramius I wish I could, but I was using doctrine2 with symfony2,So It looks like It will takes some time to simulating all environment and settings that could allow me to reproduced the problem.

but anyway,I will have a try and tell you what happen when I found something.

scourgen ok, awaiting your reply then

I've spent some time on playing with native doctrine2. It took me awhile to setup everything. but I just don't get that how to retrive data with its Proxy ojbect(for example Proxies_CG_\My\WebBundle\Entity\Line).

I mean the result of

$this->_em->getRepository("something")->findxxx()

always return an array of real object. I can't reproduced the situation(#comment-19186) that happens on symfony2+doctrine2.

anyway,I can make sure the problem is real exist,Because the execute time of that slow sql from the tool bar of symfony2 is same as I executed it at mysql cli. If the sql shows up on log with quote but running at mysql without quote,the execute time won't be same(actually It will be much more faster,in my case,20x times,from 2xxms to 10ms).

scourgen you can use

$em->getReference($className, $identifier)

(identifier being a key=>value array) to force proxies.

Give it a try

looks like I reproduced it.

    public function testIssue()
    {   
         $no_used=   $this->_em->getRepository(__NAMESPACE__. '\DDC2214Line')->findOneById(1);
        $lines=array(
            //$this->_em->getRepository(__NAMESPACE__. '\DDC2214Line')->findOneById(1),
            $this->_em->getReference(__NAMESPACE__. '\DDC2214Line',1),
            $this->_em->getReference(__NAMESPACE__. '\DDC2214Line','2'),
            $this->_em->getReference(__NAMESPACE__. '\DDC2214Line',3),
        );  
        $logger  = $this->_em->getConnection()->getConfiguration()->getSQLLogger();
        $ids     = array_map(function($r){
            return $r->id;
        }, $this->relatedList);

        //$related = $this->_em->getRepository(__NAMESPACE__ . '\DDC2214LineRelated')->findBy(array('line' => $lines), array('count' => 'DESC'), 20);
        $related = $this->_em->createQuery('select lr from '.__NAMESPACE__ . '\DDC2214LineRelated lr where lr.id in (:ids)')->setParameter('ids',$lines)->getResult();
            
        $query   = end($logger->queries);
//\Doctrine\Common\Util\Debug::dump($query['params']);

        $this->assertCount(3, $related);
        $this->assertEquals($ids, $query['params'][0]);
        $this->assertEquals(\Doctrine\DBAL\Connection::PARAM_INT_ARRAY, $query['types'][0]);
    }   
}

I use MySql Query log to see what's really happen in database(http://dev.mysql.com/doc/refman/5.5/en/query-log.html)

this is the log from table mysql.general_log

2013-01-08 12:23:44	[root] @ localhost [127.0.0.1]	59	0	Connect	root@localhost on doctrine_tests
2013-01-08 12:23:44	root[root] @ localhost [127.0.0.1]	59	0	Query	CREATE TABLE DDC2214Line (id INT AUTO_INCREMENT NOT NULL, PRIMARY KEY(id)) DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci ENGINE = InnoDB
2013-01-08 12:23:44	root[root] @ localhost [127.0.0.1]	59	0	Query	CREATE TABLE DDC2214LineRelated (id INT AUTO_INCREMENT NOT NULL, line_id INT NOT NULL, count SMALLINT NOT NULL, line_id_related INT NOT NULL, INDEX IDX_D31307994D7B7542 (line_id), PRIMARY KEY(id)) DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci ENGINE = InnoDB
2013-01-08 12:23:44	root[root] @ localhost [127.0.0.1]	59	0	Query	ALTER TABLE DDC2214LineRelated ADD CONSTRAINT FK_D31307994D7B7542 FOREIGN KEY (line_id) REFERENCES DDC2214Line (id)
2013-01-08 12:23:44	root[root] @ localhost [127.0.0.1]	59	0	Query	START TRANSACTION
2013-01-08 12:23:44	root[root] @ localhost [127.0.0.1]	59	0	Query	INSERT INTO DDC2214Line (id) VALUES (null)
2013-01-08 12:23:44	root[root] @ localhost [127.0.0.1]	59	0	Query	INSERT INTO DDC2214Line (id) VALUES (null)
2013-01-08 12:23:44	root[root] @ localhost [127.0.0.1]	59	0	Query	INSERT INTO DDC2214Line (id) VALUES (null)
2013-01-08 12:23:44	root[root] @ localhost [127.0.0.1]	59	0	Query	INSERT INTO DDC2214LineRelated (count, line_id_related, line_id) VALUES (1, 1, 1)
2013-01-08 12:23:44	root[root] @ localhost [127.0.0.1]	59	0	Query	INSERT INTO DDC2214LineRelated (count, line_id_related, line_id) VALUES (2, 2, 2)
2013-01-08 12:23:44	root[root] @ localhost [127.0.0.1]	59	0	Query	INSERT INTO DDC2214LineRelated (count, line_id_related, line_id) VALUES (3, 3, 3)
2013-01-08 12:23:44	root[root] @ localhost [127.0.0.1]	59	0	Query	COMMIT
2013-01-08 12:23:44	root[root] @ localhost [127.0.0.1]	59	0	Query	SELECT t0.id AS id1 FROM DDC2214Line t0 WHERE t0.id = 1 LIMIT 1
2013-01-08 12:23:44	root[root] @ localhost [127.0.0.1]	59	0	Query	SELECT d0_.id AS id0, d0_.count AS count1, d0_.line_id_related AS line_id_related2, d0_.line_id AS line_id3 FROM DDC2214LineRelated d0_ WHERE d0_.id IN (1, '2', 3)
2013-01-08 12:23:44	root[root] @ localhost [127.0.0.1]	59	0	Quit	

you can see,in database level,the second parameter of last query but two has quote ( (1, '2', 3) )

A related Github Pull-Request [GH-247] was opened
https://github.com/doctrine/common/pull/247

A related Github Pull-Request [GH-247] was closed
https://github.com/doctrine/common/pull/247

My pull request (https://github.com/doctrine/doctrine2/pull/548) contains an implementation of the method. Note that no exception is thrown if you query for the state of a non-existing filter - in such a case, false is returned as for disabled filters.

Symfony 2.1.8, php 5.4.7 and php 5.4.12, Windows 7

Please provide either a code example or a test case. As it stands, this issue is incomplete

Are you calling EntityManager#clear() inbetween? Because PHP reuses the hashes. The ORM accounts for this.

This is not a reproduce case, i don't want to execute your whole project.

I want to know, what is the actual bug that you see? Can you just print a list of all the hashes? Because the hashes dont differ at the end, bu tjust somewhere in the middle.

I attached a hashlogs.txt file. The last Task class hash is 0000000050ab4aba0000000058e1cb12 ( line 3 129 )

This is not unique, view the line 2 760 . The Task is not being saved and the program don't call the prePersist listener. The "UnitOfWork" believe the entity has been saved because the isset($this->entityStates[$oid]) is true. But it is an other entity.

The EntityManager::clear() fix the problem, but this is not "good" and "beautiful" solution. Shows no sign of that conflicts were and this is causing the problem. I was looking for the problem 7 hours.

Sure you are on git master? this should be optimized already with fetch=EAGER

Attach test file

I run

git clone git://github.com/doctrine/doctrine2.git
git clone git://github.com/doctrine/common.git
git clone git://github.com/doctrine/dbal.git

and run testDoctrine.php

Result

SELECT t0.user_id AS user_id1 FROM users t0

SELECT t0.post_id AS post_id1, t0.user_id AS user_id2 FROM posts t0 WHERE t0.user_id = ?

array(1) {
  [0]=>
  string(1) "1"
}
array(1) {
  [0]=>
  NULL
}
SELECT t0.post_id AS post_id1, t0.user_id AS user_id2 FROM posts t0 WHERE t0.user_id = ?

array(1) {
  [0]=>
  string(1) "2"
}
array(1) {
  [0]=>
  NULL
}
SELECT t0.post_id AS post_id1, t0.user_id AS user_id2 FROM posts t0 WHERE t0.user_id = ?

array(1) {
  [0]=>
  string(1) "3"
}
array(1) {
  [0]=>
  NULL
}

Please instead of using fetch="EAGER", please use fetch="EXTRA_LAZY". It would fix your issue.
I have successfully tested this situation in 2.2-DEV and it works like a charm. =)

Doctrine ORM 2.3.3 (Symfony2.2) - using LAZY or EXTRA_LAZY fetch mode there are only one query for:

$users = $em->getRepository('User')->findAll();

but additional users_count queries for

foreach($users as $user) $user->languages->toArray()

And if use fetch EAGER - for some reason there are 2 x users_count queries , ie each query

SELECT t0.post_id AS post_id1, t0.user_id AS user_id2 FROM posts t0 WHERE t0.user_id = ?

with unique user_id executed twice

Windows

Thats why you shouldn't generate proxies at runtime. The problem happens on windows, because the atomic rename operation doesn't work as perfectly there as on linux.

We cannot fix this in Doctrine.

Benjamin Eberlei What do you mean "you shouldn't generate proxies at runtime"? I'm not in production, this is in dev. And I'm using the default configuration.

What I don't understand is why will Doctrine regenerate proxies on every request? The warning is reproductible, and even when no PHP entity has been touched.

Benjamin Eberlei To simplify my previous message (I don't want to bury you under questions) I'll sum it up like that:

What can I do?

Thanks!

Benjamin Eberlei ping: what can be done?

Can we suppress the error with @rename($tmpFileName, $fileName); ?

https://github.com/doctrine/common/blob/master/lib/Doctrine/Common/Proxy/ProxyGenerator.php#L287

I can make a PR if you think that's a valid solution.

Matthieu Napoli no, if you have warnings, please disable them via ini setting. With error suppression there, we may have further problems identifying more serious issues.

About proxy generation: that happens EVERY time in dev environments. Generate them once and disable it afterwards.

Marco Pivetta OK I can disable the auto generation then (I'll have to remember to regenerate them when I edit the model).

Thanks for the feedback

Is that possible to make those proxies generate only if the entity file has been modified since the last generation? (only asking if can and should be done, I can look for implementing it myself if that's the case)

Matthieu Napoli that would be very obnoxious when changing entities often. I wouldn't do that (generating only if not already available)

Marco Pivetta Yes but for now they are regenerated at every request when in dev mode (at least with the default configuration http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/configuration.html#obtaining-an-entitymanager)

Which one is worse: generating every proxy class at every request, or generate only those which changed (in dev environment of course, not prod)?

If neither of these options are good (i.e. auto generation should be disabled), I don't understand why the docs say to enable auto generation when in dev environment.

Matthieu Napoli that's because in dev environments you shouldn't care about that one exception (usually happens when you got concurrent requests).

It is worse to generate only on changes: that's a lot of additional checks, variables to keep in memory and additional logic that is not needed.

Let's keep it as it is (generating at each request) for dev environments: works fine

Another (eventual) solution for dev environments would be not to write the proxy file, but to eval it.

eval it would be a good solution IMO, no more "woops the directory is not writable" and it's more neutral for the user filesystem (but not as easy to debug). But OK, I see what you mean, it works let's keep it that way.

Actually the problem on my setup is that PHP errors are turned into exceptions, so on an (poorly designed) AJAX treeview (lots of nodes to load => lots of requests), I end up with some nodes not loaded because of the exception. And it feels weird to either silently log all PHP warnings or silently ignore the specific warning for the rename.

Matthieu Napoli I'd go with `eval` then. Needs refactoring of the abstract proxy factory and of the proxy generator (proxy generator should no longer write files).

Re-opening: the proxy factory could directly `eval()` the produced proxy code. The ProxyGenerator should no longer write the generated files to disk automatically.

I've opened a PR: https://github.com/doctrine/common/pull/269

Tested both Mac OS X and Ubuntu