[DDC-3212] Remove ArrayHydrator logic Created: 14/Jul/14  Updated: 13/Oct/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.5
Fix Version/s: 3.0
Security Level: All

Type: Improvement Priority: Critical
Reporter: Marco Pivetta Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: hydration, hydrator


 Description   

The Doctrine\ORM\Internal\Hydration\ArrayHydrator ( https://github.com/doctrine/doctrine2/blob/85fbf684363b932a4ebaf543ef059f9ee1e512b0/lib/Doctrine/ORM/Internal/Hydration/ArrayHydrator.php ) is currently very messy and complicated due to the lack of actual Doctrine\ORM\UnitOfWork references when working with it, since we are not dealing with objects.

In order to reduce the amount of bugs and code duplication when working with array hydration, I would simply suggest making use of the Doctrine\ORM\Internal\Hydration\ObjectHydrator ( https://github.com/doctrine/doctrine2/blob/85fbf684363b932a4ebaf543ef059f9ee1e512b0/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php ) and its siblings, and then extracting data coming from the results in it as an array.

This could be a simple reflection-based extraction (pseudo):

class ArrayHydrator
{
    public function hydrateAllData()
    {
        foreach ($this->objectHydrator->hydrateAllData() as $object) {
            yield $this->extractData($object);
        }
    }
}

The point here is that array-based hydration is not the primary focus of the ORM, and users should probably rely on SQL only when they want array hydration.



 Comments   
Comment by Marco Pivetta [ 14/Jul/14 ]

Note: performance is one of our biggest requirements, and array hydration is meant to achieve that. As I stated above, plain SQL is probably better for this sort of operation, so the issue is more about deprecating the ArrayHydrator completely, providing utilities to manipulate SQL resultsets instead.





[DDC-2560] Schema tool invalid DDL syntax for default values Created: 19/Jul/13  Updated: 19/Jul/13

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: 2.3.4
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Ben Davies Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: schematool
Environment:

Postgresql



 Description   

Not sure if this is Postgresql specific or not, but here are the reproduction steps:

Have a table with a column setup like so:

ALTER TABLE usr ADD COLUMN contact_count integer;
ALTER TABLE usr ALTER COLUMN contact_count SET NOT NULL;
ALTER TABLE usr ALTER COLUMN contact_count SET DEFAULT 0;

Have a Entity Column Definition like so:

    /**
     * @var integer $contactCount
     *
     * @ORM\Column(name="contact_count", type="integer", nullable=false)
     */
    private $contactCount = 0;

orm:schema-tool:update will generate the following SQL:

ALTER TABLE usr ALTER contact_count SET ;

adding

options={"default":0}

to the column definition results in no sql being generated (correctly I assume)



 Comments   
Comment by Marco Pivetta [ 19/Jul/13 ]

The ORM doesn't support default column values.
That's fine-tuning you can do on the column definitions.

Your "workaround" is actually the correct way of dealing with this kind of DDL change.

Comment by Ben Davies [ 19/Jul/13 ]

I realise the ORM doesn't support default values, but the point is that the schematool picks up a diff in the columns, tries to generate a diff, and generates an invalid sql statement. Surely that is a bug?

Comment by Marco Pivetta [ 19/Jul/13 ]

Ben Davies the DBAL is responsible for generating this wrong DDL... Are you able to reproduce this in DBAL only?

Comment by Ben Davies [ 19/Jul/13 ]

Happy to try.

Comment by Ben Davies [ 19/Jul/13 ]

here you go
https://github.com/bendavies/dbal/blob/DBAL-561/tests/Doctrine/Tests/DBAL/Functional/Ticket/DBAL561Test.php





[DDC-2452] Additional `WITH` condition in joins between JTI roots cause invalid SQL to be produced Created: 16/May/13  Updated: 27/Mar/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: DQL, ORM
Affects Version/s: Git Master
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Marco Pivetta Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: dql, sql-walker
Environment:

irrelevant



 Description   

Given a simple Joined Table Inheritance like following:

/**
 * @Entity @Table(name="foo") @InheritanceType("JOINED")
 * @DiscriminatorColumn(name="discr", type="string")
 * @DiscriminatorMap({"foo" = "DDC2452Foo", "bar" = "DDC2452Bar"})
 */
class DDC2452Foo
{
    /** @Id @Column(type="integer") @GeneratedValue */
    public $id;
}

/** @Entity @Table(name="bar") */
class DDC2452Bar extends DDC2452Foo
{
}

Following DQL

SELECT foo1 FROM DDC2452Foo foo1 JOIN DDC2452Foo foo2 WITH 1=1

Will produce broken SQL:

SELECT
    f0_.id AS id0, f0_.discr AS discr1 
FROM 
    foo f0_ 
LEFT JOIN bar b1_ 
    ON f0_.id = b1_.id 
LEFT JOIN foo f2_ 
LEFT JOIN bar b3_ 
    ON f2_.id = b3_.id 
    ON (1 = 1)

(please note the duplicate `ON` in the SQL)

That is caused because of the SQL walker producing the JTI filter with already the `ON` clause in it.

That happens because the JTI join conditions are added in https://github.com/doctrine/doctrine2/blob/2.4.0-BETA2/lib/Doctrine/ORM/Query/SqlWalker.php#L823-L825 (`walkRangeVariableDeclaration`), while the additional defined `WITH` conditions are considered in `walkJoinAssociationDeclaration` later on.

Added a test case and fix at https://github.com/doctrine/doctrine2/pull/668






[DDC-2147] Custom annotation in MappedSuperclass Created: 15/Nov/12  Updated: 07/May/13

Status: Awaiting Feedback
Project: Doctrine 2 - ORM
Component/s: Mapping Drivers, ORM
Affects Version/s: 2.2.1
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: kluk Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Linux 3.6.6-1.fc17.x86_64


Attachments: Text File error.log    

 Description   

When you try use custom annotation in mappedsuperclass like here http://pastebin.com/YMxKvcLk and then i try get metadata for class i get this error
Undefined index: fieldName
ClassMetadataInfo.php function addInheritedFieldMapping
Problem is that custom annotation doesnt have fieldName.
Quick fix is add condition to test if fieldName isset.



 Comments   
Comment by kluk [ 15/Nov/12 ]

error log from orm:validate-schema

Comment by Marco Pivetta [ 23/Jan/13 ]

Copying from pastebin:

use \Doctrine\ORM\Mapping as ORM;
use \xxx\Doctrine\Annotation\Entity as re;
use \xxx\Doctrine\Annotation\Forms as rf;
use \Doctrine\Common\Collections;
 
/**
 * @ORM\Entity
 */
class EventPicture extends \Picture
{
 
    /**
     * @ORM\ManyToOne(targetEntity="Event", inversedBy="eventPicture")
     * @ORM\JoinColumn(name="FK_Event", referencedColumnName="id")
     */
    protected $event;
 
}
use \Doctrine\ORM\Mapping as ORM;
use \xxx\Doctrine\Annotation\Entity as re;
use \xxx\Doctrine\Annotation\Forms as rf;
use \Doctrine\Common\Collections;
 
/** @ORM\MappedSuperclass */
class Picture extends \xxx\Doctrine\Entity\BaseEntity
{
 
    /**
     * @ORM\Id
     * @ORM\Column(type="integer")
     * @ORM\GeneratedValue(strategy="IDENTITY")
     * @var type
     */
    protected $id;
 
    /**
     * @ORM\Column(type="string",unique=true, nullable=false)
     *  @rf\FileUpload(fileSize="php",uploadType="local",fieldName="link",formControl="FileUploadField",image=true)
     *
     */
    protected $link;
 
}

kluk does this happen also with any other simple custom annotation? For example following:

/**
 * @Annotation 
 * @Target({"PROPERTY","ANNOTATION"})
 */
final class Entity implements Annotation
{
    /**
     * @var string
     */
    public $value;
}
Comment by kluk [ 30/Jan/13 ]

the same error when using simple annotation.

 
<?php

use \Doctrine\ORM\Mapping as ORM;
use \xxx\Doctrine\Annotation\Entity as re;
use \xxx\Doctrine\Annotation\Forms as rf;
use \Doctrine\Common\Collections;

/** @ORM\MappedSuperclass */
class Picture extends \xxx\Doctrine\Entity\BaseEntity {

    /**
     * @ORM\Id
     * @ORM\Column(type="integer")
     * @ORM\GeneratedValue(strategy="IDENTITY")
     * @var type
     */
    protected $id;

   
    /**
     * @ORM\Column(type="integer")
     * @rf\SetClass({"class","hide"})
     */
    public $value;

    /**
     * @ORM\Column(type="string",unique=true, nullable=true)
     * @rf\FileUpload(fileSize="php",uploadType="local",fieldName="link",formControl="FileUploadField",image=true)
     *
     */
    protected $link;

}

When i remove $value , $picture from class everything goes ok.
Easy fix for me is change ClassMetadataInfo.

    /**
     * INTERNAL:
     * Adds a field mapping without completing/validating it.
     * This is mainly used to add inherited field mappings to derived classes.
     *
     * @param array $fieldMapping
     *
     * @return void
     */
    public function addInheritedFieldMapping(array $fieldMapping)
    {
        if(isset($fieldMapping['fieldName'])){
        $this->fieldMappings[$fieldMapping['fieldName']] = $fieldMapping;
        $this->columnNames[$fieldMapping['fieldName']] = $fieldMapping['columnName'];
        $this->fieldNames[$fieldMapping['columnName']] = $fieldMapping['fieldName'];
        }
    }

But i dont know if this fix can break another part of doctrine.

Comment by Benjamin Eberlei [ 04/May/13 ]

Can you put the code of your annotations online? I can't seem to understand why this happens.

Comment by kluk [ 07/May/13 ]
Unable to find source-code formatter for language: php. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
 
namespace libs\Doctrine\Annotation\Entity;
use Doctrine\Common\Annotations\Annotation;

/** @Annotation */
class CustomMapping extends Annotation
{
    /**
     *
     * @var string
     */
    public $className;
    /**
     * 
     * 
     * @var IQueryable| string
     */
    public $dataSource;
}




[DDC-1963] Remove by-ref access to changeset in lifecycle event args Created: 31/Jul/12  Updated: 16/Apr/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: None
Fix Version/s: 2.x, 2.5
Security Level: All

Type: Improvement Priority: Major
Reporter: Marco Pivetta Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None


 Description   

UoW currently passes computed changesets to lifecycle event args byref. This has to be changed to force users to use UoW public API to modify changesets instead.






[DDC-1947] Update EBNF with arbitrary joins Created: 26/Jul/12  Updated: 26/Jul/12

Status: Open
Project: Doctrine 2 - ORM
Component/s: Documentation
Affects Version/s: 2.3, Git Master
Fix Version/s: None
Security Level: All

Type: Documentation Priority: Major
Reporter: Marco Pivetta Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Arbitrary joins need to be documented in EBNF






[DDC-3115] UnitOfWok can't access proxies protected property Created: 04/May/14  Updated: 04/May/14

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: Git Master
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Machete Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: orm
Environment:

php5.5.11-1~dotdeb.1
debian 7.5 64bit
apache2
doctrine-orm-module 0.8.0


Attachments: PNG File doctrine-error.PNG    

 Description   

Running the code on different environments for a while now, I have never had any problems. Recently I installed the code onto a different server and I've been experiencing the following issue:

> Fatal error: Cannot access protected property DoctrineORMModule\Proxy_CG_\Page\Entity\PageRevision::$date in src/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php on line 2596

I am absolutely clueless, why this happens. It seems to be affecting only specific data rows, as you might reproduce here: http://de.serlo.org/pages

The field is defined as followed:

    /**
     * @ORM\Column(type="datetime", options={"default"="CURRENT_TIMESTAMP"})
     */
    protected $date;

However, the exact same database snapshot works fine here: http://ptr.serlo.org/pages

Attached you will find a screenshot of the datarows. The yellow ones mark each one working and one not working row.

I have no idea what the problem is here.



 Comments   
Comment by Marco Pivetta [ 04/May/14 ]

Discussed this on IRC - very tricky, seems like one of the servers has the reflection property not set to "accessible"

Comment by Machete [ 04/May/14 ]

Adding `$class->reflFields[$field]->setAccessible(true)` before line 2596 resolved this issue. Even after removing it, everything worked fine.

note to self:
> [18:23] <@ocramius> machete_: can you find out where that particular property is instantiated?
> [18:23] <@ocramius> should be in `ClassMetadataInfo::wakeUpReflection()` or such
> [18:25] <@ocramius> if you find that again, consider looking at locations in the code where `new ReflectionClass()` is instantiated





[DDC-3063] Unexpected behavior with 'WHERE NOT IN' and empty array Created: 01/Apr/14  Updated: 06/May/14

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: DQL
Affects Version/s: 2.4
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Tim Stamp Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None
Environment:

CentOS, PHP 5.4, mysql



 Description   

I tried to set version to 2.4.0 but was prevented.

Assume the set 'n' contains 10 records, all with id > 0.

->andWhere('n.id NOT IN (:ids)')->setParameter('ids', [])
returns 0 records.

->andWhere('n.id NOT IN (:ids)')->setParameter('ids', [0])
returns 10 records.


 Comments   
Comment by Marco Pivetta [ 01/Apr/14 ]

This is a simple misunderstanding of how SQL IN() works

Comment by Tim Stamp [ 01/Apr/14 ]

You're right that its invalid SQL, but this is DQL. In SQL this would raise an error, in DQL it says nothing and returns nothing.

Isnt there a way to check if a statement has caused this error?

Comment by Marco Pivetta [ 01/Apr/14 ]

Tim Stamp yeah, when using IN() and NOT IN() you should actually check if the parameters are empty or not before adding the clause.

Comment by Steve Müller [ 01/Apr/14 ]

It's still weird that no error is raised by that. Tim Stamp can you check what SQL is generated by your query?

Comment by Steve Müller [ 01/Apr/14 ]

Hehe I think I got the problem. It might be DBAL related. See:

https://github.com/doctrine/dbal/blob/master/lib/Doctrine/DBAL/SQLParserUtils.php#L140
https://github.com/doctrine/dbal/blob/master/lib/Doctrine/DBAL/SQLParserUtils.php#L169

Looks like the implementing person did that by intention but don't ask me why.

Comment by Marco Pivetta [ 01/Apr/14 ]

Thats to avoid having an SQL statement like

SELECT * FROM foo WHERE bar IN()

, which is invalid.

Indeed, the NOT IN() case is not contemplated by that.

Comment by Steve Müller [ 01/Apr/14 ]

Just wondering if that kind of implementation isn't changing expectations because it looks like IN(NULL) will select all NULL rows then, even though this is not explicitly requested by the user. I don't care really TBH but this behaviour is not very transparent to the user. Personally I would like to see an error instead... Then at least I know what's going on and can fix that by additional checks instead.

Comment by Marco Pivetta [ 01/Apr/14 ]

Makes sense. Re-opening.

Comment by Steve Müller [ 01/Apr/14 ]

The question is if we can change this behaviour without breaking applications that rely on the current one. Because changing the code to throw an error breaks applications that insist on returning 0 rows for an empty array. I don't know what rules apply here concerning BC. What do you think Marco Pivetta?

Comment by Marco Pivetta [ 01/Apr/14 ]

Steve Müller existing apps should do following anyway:

if ($productIds) {
    $qb->andWhere('p.id IN (:productIds)')->setParameter('productIds', $productIds);
}

If they are not, that's most likely a bug in their codebase.

Comment by Steve Müller [ 01/Apr/14 ]

Agree. Was just wondering about what we can expect and what we can't expect.

Comment by Guilherme Blanco [ 21/Apr/14 ]

Steve Müller feel free to patch DBAL.
It's a bug, not a supported feature. If we have 2 tickets:
1- If I pass an empty array, I get nullable rows and I can't fix this
2- If I pass an empty array, it used to return nullable rows, now it returns nothing
Which one would you fix?

Comment by Tim Stamp [ 06/May/14 ]

Guilherme Blanco shouldn't both these use cases throw an exception, as both cases in MySQL would return an error?





[DDC-1698] Inconsistent proxy file name & namespace result in __PHP_Incomplete_Class when unserializing entities Created: 13/Mar/12  Updated: 25/Sep/13

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.2, 2.2.1, 2.3, 2.4
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: Benjamin Morel Assignee: Marco Pivetta
Resolution: Unresolved Votes: 2
Labels: proxy


 Description   

Starting with Doctrine 2.2, the Proxy classes have inconsistent naming with their file name, which raises problems with class autoloading.
For example, a class named Application\Model\User creates the following proxy class:

Application\Proxy\__CG__\Application\Model\User

This class is located in the following file:

Application/Proxy/__CG__ApplicationModelUser.php

But whe we serialize such an entity, then unserialize it in another session, the framework autoloader expects the class to be located in:

Application/Proxy/__CG__/Application/Model/User.php

But it is not.
As a result, a __PHP_Incomplete_Class is created instead of the expected proxy class.

I'm not sure whether this is an intended behavior, but I would assume this is a bug.



 Comments   
Comment by Benjamin Morel [ 13/Mar/12 ]

It looks like there is an even broader problem with the new _CG_ prefix; the PSR-0 standard for autoloading states that the underscores should be handled this way:

\namespace\package\Class_Name => {...}/namespace/package/Class/Name.php

Which means that in the above example, it could even expect the file to be located in:

Application/Proxy///CG///Application/Model/User.php

... which is far away from the actual location.
Upgrade to 2.2 broke this code, for us.

Comment by Benjamin Eberlei [ 14/Mar/12 ]

Proxy classes do not follow PSR-0. For the case unserializing objects we should provide an extra autoloader i guess.

See here how symfony does it https://github.com/doctrine/DoctrineBundle/blob/master/DoctrineBundle.php#L57

Comment by Benjamin Eberlei [ 14/Mar/12 ]

See https://github.com/doctrine/doctrine2/commit/9b4d60897dfc7e9b165712428539e694ec596c80 and https://github.com/doctrine/orm-documentation/commit/01381fae1ff3d4944086c7cfe46721925bf6ca15

Comment by Benjamin Morel [ 14/Mar/12 ]

Thanks for the quick fix, Benjamin.
However, I have to admit that I'm not fully happy with the fix, as we (and probably many others) are not using the Doctrine autoloader.
I supposed that the purpose of PSR-0 was precisely not to be tied to a particular autoloader implementation, and this benefit is lost with this version of Doctrine.

You mentioned in the doc that the proxies are not PSR-0 compliant "for implementation reasons"; as this was working fine before 2.2, could you please explain what requirement prevents Doctrine from keeping the previous naming convention?

Comment by Benjamin Eberlei [ 29/Mar/12 ]

In 2.1 the proxies are not PSR-0 compatible themselves, however their class naming is simpler.

In 2.2 we changed proxy names so that you can derive the original name of the proxy by searching for the _CG_ flag. This flag obviously contains the __ chars that some PSR autoloaders detect as directory seperators. I agree this is an unfortunate decision, but it was done this way.

I do think however that we can automatically register the proxy atuoloader (if not yet done) in EntityManager#create(). This would hide this fact from developers automatically.

Comment by Benjamin Morel [ 29/Oct/12 ]

@Benjamin Eberlei
In 2.3 we still have to manually call Autoloader::register() before unserializing entities that may contain proxies.
So EntityManager::create() still doesn't register it. Is there a plan to add this feature?

Comment by Benjamin Eberlei [ 06/Jan/13 ]

Benjamin Morel Not at the moment, seems too dangerous for me since it might produce race conditions. This should really be done in the bootstrap of the system.

We need to document this though.

Comment by Benjamin Morel [ 06/Jan/13 ]

Ok, thanks for your answer!

Comment by Matthieu Napoli [ 09/Sep/13 ]

Sorry to be a pain, but can the description of the task be updated:

  • Affects Version/s: should show all versions (or none) but not just 2.2, 2.2.1
  • Fix Version/s: should be blank

This is to make it clear that this bug is not resolved (since we are way past v2.2.2).

Thanks

Comment by Chris Hawkins [ 25/Sep/13 ]

Some additional documentation about this would be useful. I just spent some considerable time diagnosing an error related to restoring an entity from a cache. I wasn't able to find a mention of the need to call Autoloader::register until I found this page. It was resolved fairly quickly after I found this. For reference, I added this to my bootstrap:

ini_set('unserialize_callback_func', 'proxymissing');
function proxymissing( $name ) {
	if ( preg_match( '/^DoctrineProxies/', $name ) ) {
		\Doctrine\ORM\Proxy\Autoloader::register( PROXYPATH, 'DoctrineProxies' );
	}
}




[DDC-1695] SQLs for PostgreSQL case sensite tables/fields are wrongly generated Created: 09/Mar/12  Updated: 06/Aug/14

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: 2.1.6, 2.4.2
Fix Version/s: 2.1.7, 2.2.2
Security Level: All

Type: Bug Priority: Major
Reporter: Ignacio Larranaga Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None
Environment:

PostgreSQL 9.x, Symfony 2


Attachments: Text File doctrine-2.1.6.patch     Text File SqlWalker.patch    

 Description   

The SQLs for case sensitive schemas in postgreSQL are wronly generated.

Example:
Schema:

CREATE TABLE "News" (
  "IdNews" serial NOT NULL,
  "IdUser" bigint NOT NULL,
  "IdLanguage" integer NOT NULL,
  "IdCondition" integer,
  "IdHealthProvider" integer,
  "IdSpeciality" integer,
  "IdMedicineType" integer,
  "IdTreatment" integer,
  "Title" character varying,
  "SmallText" character varying,
  "LongText" character varying,
  "PublishDate" timestamp with time zone,
  "IdxNews" tsvector,
  "Highlight" boolean NOT NULL DEFAULT false,
  "Order" integer NOT NULL DEFAULT 0,
  "Deleted" boolean NOT NULL DEFAULT false,
  "Active" boolean NOT NULL DEFAULT false,
  "UpdateToHighlighted" boolean DEFAULT false,
  CONSTRAINT "News_pkey" PRIMARY KEY ("IdNews" ));

Object (I added quotes trying to generate the SQLs quoted.:

<?php

namespace GlobalTreatments\ApplicationBundle\Entity;

use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\Table(name="""News""")
 * @ORM\Entity
 */
class News
{
    /**
     * @var integer $idNews
     *
     * @ORM\Column(name="""IdNews""", type="integer", nullable=false)
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="SEQUENCE")
     * @ORM\SequenceGenerator(sequenceName="""News_IdNews_seq""", allocationSize="1", initialValue="1")
     */
    private $idNews;

    /**
     * @var bigint $iduser
     *
     * @ORM\Column(name="""IdUser""", type="bigint", nullable=false)
     */
    private $idUser;

    /**
     * @var integer $idLanguage
     *
     * @ORM\Column(name="""IdLanguage""", type="integer", nullable=false)
     */
    private $idLanguage;

    /**
     * @var integer $idCondition
     *
     * @ORM\Column(name="""IdCondition""", type="integer", nullable=true)
     */
    private $idCondition;

    /**
     * @var integer $idHealthProvider
     *
     * @ORM\Column(name="""IdHealthProvider""", type="integer", nullable=true)
     */
    private $idHealthProvider;

    /**
     * @var integer $idSpeciality
     *
     * @ORM\Column(name="""IdSpeciality""", type="integer", nullable=true)
     */
    private $idSpeciality;

    /**
     * @var integer $idMedicineType
     *
     * @ORM\Column(name="""IdMedicineType""", type="integer", nullable=true)
     */
    private $idMedicineType;

    /**
     * @var integer $idTreatment
     *
     * @ORM\Column(name="""IdTreatment""", type="integer", nullable=true)
     */
    private $idTreatment;

    /**
     * @var string $title
     *
     * @ORM\Column(name="""Title""", type="string", nullable=true)
     */
    private $title;

    /**
     * @var string $smallText
     *
     * @ORM\Column(name="""SmallText""", type="string", nullable=true)
     */
    private $smallText;

    /**
     * @var string $longText
     *
     * @ORM\Column(name="""LongText""", type="string", nullable=true)
     */
    private $longText;

    /**
     * @var datetimetz $publishDate
     *
     * @ORM\Column(name="""PublishDate""", type="datetimetz", nullable=true)
     */
    private $publishDate;

    /**
     * @var tsvector $idxNews
     *
     * @ORM\Column(name="""IdxNews""", type="tsvector", nullable=true)
     */
    private $idxNews;

    /**
     * @var boolean $highlight
     *
     * @ORM\Column(name="""Highlight""", type="boolean", nullable=false)
     */
    private $highlight;

    /**
     * @var integer $order
     *
     * @ORM\Column(name="""Order""", type="integer", nullable=false)
     */
    private $order;

    /**
     * @var boolean $deleted
     *
     * @ORM\Column(name="""Deleted""", type="boolean", nullable=false)
     */
    private $deleted;

    /**
     * @var boolean $active
     *
     * @ORM\Column(name="""Active""", type="boolean", nullable=false)
     */
    private $active;

    /**
     * @var boolean $updateToHighlighted
     *
     * @ORM\Column(name="""UpdateToHighlighted""", type="boolean", nullable=true)
     */
    private $updateToHighlighted;

    /**
     * @var condition
     *
     * @ORM\ManyToOne(targetEntity="Condition")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="""IdCondition""", referencedColumnName="""IdCondition""")
     * })
     */
    private $condition;

    /**
     * @var healthProvider
     *
     * @ORM\ManyToOne(targetEntity="HealthProvider")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="""IdHealthProvider""", referencedColumnName="""IdHealthProvider""")
     * })
     */
    private $healthProvider;

    /**
     * @var language
     *
     * @ORM\ManyToOne(targetEntity="Language")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="""IdLanguage""", referencedColumnName="""IdLanguage""")
     * })
     */
    private $language;

    /**
     * @var medicineType
     *
     * @ORM\ManyToOne(targetEntity="MedicineType")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="""IdMedicineType""", referencedColumnName="""IdMedicineType""")
     * })
     */
    private $medicineType;

    /**
     * @var speciality
     *
     * @ORM\ManyToOne(targetEntity="Speciality")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="""IdSpeciality""", referencedColumnName="""IdSpeciality""")
     * })
     */
    private $speciality;

    /**
     * @var treatment
     *
     * @ORM\ManyToOne(targetEntity="Treatment")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="""IdTreatment""", referencedColumnName="""IdTreatment""")
     * })
     */
    private $treatment;

    /**
     * @var user
     *
     * @ORM\ManyToOne(targetEntity="User")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="""IdUser""", referencedColumnName="""IdUser""")
     * })
     */
    private $user;

    ....

}

DQL:

'SELECT n.smallText, n.publishDate ' .
'FROM News n ' .
	'INNER JOIN n.language la ' .
'WHERE la.languageCode = :languageCode ' .
'ORDER BY n.publishDate DESC'

Generated SQL:

SELECT "0_."SmallText" AS "SmallText"0, "0_."PublishDate" AS "PublishDate"1 FROM "News" "0_ INNER JOIN "Language" "1_ ON "0_."IdLanguage" = "1_."IdLanguage" WHERE "1_."LanguageCode" = ? ORDER BY "0_."PublishDate" DESC LIMIT 6

Notice there are unmattched " in the SQL.



 Comments   
Comment by Ignacio Larranaga [ 09/Mar/12 ]

If there is another approach to specify the table/column names are case sensitive in PGSQL please let me know.

Comment by Ignacio Larranaga [ 09/Mar/12 ]

Just to comment. I also tried the normal quoting.

Example: @ORM\Column(name="`IdNews`", type="integer", nullable=false)

And does not work too because of the same reason.

Comment by Ignacio Larranaga [ 09/Mar/12 ]

Hi, I generate this patch and seems to be working for me (at least what I'm testing right now).

I used ´ to quote tables and single attributes (not relationships) and the SQLs are correctly generated.

Comment by Ignacio Larranaga [ 09/Mar/12 ]

Adding a new patch for another files I need to change.

Comment by Benjamin Eberlei [ 11/Mar/12 ]

Formatted code

Comment by Benjamin Eberlei [ 11/Mar/12 ]

Fixed and merged into 2.1.x and 2.2 branches

Comment by Julian Aicardo [ 05/Aug/14 ]

This bug still unresolved for version 2.3.6-DEV included in Symfony 2.3.18.

The patches does not work with this version.

Generated query:

SELECT "0_.id AS id0, "0_."verFirmware" AS verfirmware1, "0_."idEstacion" AS idestacion2, "0_."fechaHora" AS fechahora3, "0_.global AS global4, "0_.directa AS directa5, "0_.difusa AS difusa6, "0_."tempSensDirecta" AS tempsensdirecta7, "0_.vbat1 AS vbat18, "0_.vbat2 AS vbat29, "0_.flags AS flags10, "0_."Estacion" AS estacion11 FROM "RegistroRS" "0_ WHERE "0_."idEstacion" = '1' AND "0_."fechaHora" >= '2014-02-01 03:00:00' AND "0_."fechaHora" <= '2014-08-01 03:00:00'

Notice there are unmattched " in the SQL.

Comment by Marco Pivetta [ 05/Aug/14 ]

Julian Aicardo why are you using 2.3.6-DEV? Shouldn't you use a stable version? Is the bug reproducible also with later versions?

Comment by Julian Aicardo [ 06/Aug/14 ]

After changing to version 2.4.2 the bug still there. The same SQL query was generated.

Comment by Julian Aicardo [ 06/Aug/14 ]

The error occurs when try to get an alias based on case sensitive table name, because the alias is generated with the first character of table's name which is ".
Ex:
Table: car Alias: c0_
Table: "Car" Alias: "0_

Is in file SqlWalker.php line 295 in Doctrine 2.4.4:

SqlWalker.php
public function getSQLTableAlias($tableName, $dqlAlias = '')
{
    $tableName .= ($dqlAlias) ? '@[' . $dqlAlias . ']' : '';
        
    if ( ! isset($this->tableAliasMap[$tableName])) {
        $this->tableAliasMap[$tableName] = strtolower(substr($tableName, 0, 1)) . $this->tableAliasCounter++ . '_';
    }

    return $this->tableAliasMap[$tableName];
}




[DDC-2838] Leaky abstraction when applying Criteria to hydrated/non-hydrated PersistentCollection Created: 03/Dec/13  Updated: 20/Aug/14

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.4.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: brian ridley Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None


 Description   

When applying a Criteria to a PersistentCollection that has been hydrated the field names must be camel case, if the collection has not yet been hydrated the field names must be underscore separated.

The github repo linked here contains a simplified testcase for the matrix of hydrated/non-hydrated entities and camel case/underscore separated fields.

https://github.com/ptlis/DoctrineTestcase



 Comments   
Comment by Marco Pivetta [ 18/Aug/14 ]

We can't check out an entire project just to test a bug.

Please write an actual functional test case that can be integrated into the Doctrine ORM test suite.

Comment by brian ridley [ 20/Aug/14 ]

Hi,

i'm happy to do so - i'll take a look at this over the weekend.





[DDC-2995] Joined Inheritance Mapping and Filters Created: 22/Feb/14  Updated: 14/Oct/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.4.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Bojidar Hristov Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: orm


 Description   

Is there any reason why Inheritance Mapping IS LEFT JOINED not INNER JOINED?

When there is a filter and it's left joined it happens a record might not have parent table record. For example

Class B extends Class A.
Class A have column is_active | and filter activated with is_active = 1 condition.

Final query: LEFT JOIN parent_table s1_ ON p2_.id = s1_.id AND (s1_.is_active = '1')

Record 1 have is_active = false, so result set looks like this:

table_b | table_a
id | id is_active discriminatorColumn
1 | null null null

and occurs exception: The discriminator column ... is missing for ....



 Comments   
Comment by Marco Pivetta [ 22/Feb/14 ]

JTI MUST be left-joined, since not every subclass of the root of the inheritance causes inserts on all of the inheritance tables.

What is the DQL query that is causing the problem?

Comment by Bojidar Hristov [ 22/Feb/14 ]

Hello Marco, I don't query from parent table, but from child. No sense to be LEFT JOINED.

DQL is something like that:

SELECT class1, class2 FROM Class1 class1 JOIN class1.class2 class2;

(Class2 extends Class3)
Class3 have is_active column.

And following filter:

public function addFilterConstraint(ClassMetadata $targetEntity, $targetTableAlias) {
if (!$targetEntity->reflClass->implementsInterface('ActivateableInterface'))

{ return ""; }

return $targetTableAlias . '.is_active = ' . $this->getParameter('isActive');
}

Filter condition is added in LEFT JOINED parent_table. Full example SQL:
SELECT ....
FROM class1 c1
INNER JOIN class2 c2 ON c1.c2_id = c2.id
LEFT JOIN table3 c3 ON c2.id = c3.id AND (c3.is_active = '1')

Comment by Marco Pivetta [ 26/Feb/14 ]

There are two possible solutions here:

  • verify if the selected entity is a leaf of a JTI, and use INNER JOIN on that if possible (probably not optimal, since the selection may be part of a LEFT JOIN itself)
  • add filtering on the discriminator column

Bojidar Hristov can you provide a failing test case to make it easier to work on this?

Comment by Bojidar Hristov [ 26/Feb/14 ]

Yes. Here is it sample failing test.

http://pastebin.com/xi8uUNX5 - Parent Class
http://pastebin.com/XRZLbyGX - Child Class
http://pastebin.com/HuPFb98u - Assoc Class
http://pastebin.com/hGqRk74e - TestCase

Here it should found 0 records because ot JOIN and is_active = true filter.

Comment by Benjamin Eberlei [ 23/Mar/14 ]

Since filters are only available on the root, i guess inner join is really necessary here.

Comment by Bojidar Hristov [ 14/Oct/14 ]

Do you need something more? The bug is status Feedback?





[DDC-3005] Events::postLoad fires without filled associations Created: 02/Mar/14  Updated: 19/Oct/14

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Artur Eshenbrener Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Dependency
depends on DDC-3070 [GH-1001] [DDC-3005] Defer invoking o... Open

 Description   

When we load entities throw one dql query like this:

SELECT e, joined_link FROM Entity LEFT JOIN e.link joined_link

In event subscriber, subscribed to postLoad event for instances of Entity property "link" of entity does not contains fetched in same query joined entity, and does not contains proxy object.

Tell me if failing test case needed.



 Comments   
Comment by Marco Pivetta [ 02/Mar/14 ]

The postLoad event is fired without warranty that association entities/proxies will be set:

http://docs.doctrine-project.org/en/latest/reference/events.html#lifecycle-events

Comment by Artur Eshenbrener [ 03/Mar/14 ]

But why? This shounds like involuntary restriction, and I think it can be fixed. Will my PR with fix accepted, or collaborators don't want to change this behaviour?

Comment by Marco Pivetta [ 03/Mar/14 ]

Artur Eshenbrener triggering `postLoad` in the correct moment in time (when all dependencies are loaded) requires a lot of additional complexity to be inserted in various locations of the ORM.

Since `postLoad` is supposed to be used like `__wakeup` and in general for simple tasks, this kind of refactoring/rewrite would be an overkill.

Comment by Artur Eshenbrener [ 04/Mar/14 ]

> requires a lot of additional complexity to be inserted in various locations of the ORM.
Sounds like "It is very hard to implement, and no one want to do this"

> and in general for simple tasks, this kind of refactoring/rewrite would be an overkill.
Disagree with this. Why only simple? I need to deal with associations in this event. Without this I should inject whole EntityManager to my entity, but I dont want to do this.

And I forced to repeat the question: will PR with fix accepted or not?

Comment by Marco Pivetta [ 04/Mar/14 ]

Sounds like "It is very hard to implement, and no one want to do this"

Not really, the main problem here is performance, since hydration would have to be completely redesigned.
Yes, "too hard" is a good reason for something that is an edge case.

And I forced to repeat the question: will PR with fix accepted or not?

Sure thing! Just needs to avoid a massive rewrite though.

Comment by Marco Pivetta [ 04/Mar/14 ]

To give you some hints, Doctrine\ORM\Events::postLoad is triggered in https://github.com/doctrine/doctrine2/blob/15432fc55f83c2d6ce8d9b86fd3139dd2a4fc328/lib/Doctrine/ORM/UnitOfWork.php#L2788, and Doctrine\ORM\UnitOfWork#createEntity() is used by all the various hydrators internally.

To ensure that Doctrine\ORM\Events::postLoad is triggered after an entity is fully loaded, the various hydrators must trigger the events after being sure that all data has been set, and that the entity is "ready" for use. That's some copy-paste work :\

Comment by Artur Eshenbrener [ 04/Mar/14 ]

I think the entry point is here: https://github.com/doctrine/doctrine2/blob/15432fc55f83c2d6ce8d9b86fd3139dd2a4fc328/lib/Doctrine/ORM/Internal/Hydration/AbstractHydrator.php#L140,
so, copy-paste work is not needed )

Comment by Artur Eshenbrener [ 05/Apr/14 ]

PR is ready: https://github.com/doctrine/doctrine2/pull/1001

Comment by Doctrine Bot [ 19/Oct/14 ]

A related Github Pull-Request [GH-1001] was assigned:
https://github.com/doctrine/doctrine2/pull/1001

Comment by Doctrine Bot [ 19/Oct/14 ]

A related Github Pull-Request [GH-1001] was unassigned:
https://github.com/doctrine/doctrine2/pull/1001





[DDC-3361] Doctrine error while trying to execute a DQL query on PostgreSQL 9.2.x Created: 23/Oct/14  Updated: 23/Oct/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: DQL
Affects Version/s: 2.4.6
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Reynier Perez Mira Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: postgresql, typecast
Environment:

CentOS 6.5, PostgreSQL 9.2, Symfony 2.5.5, Doctrine 2.4.6



 Description   

I have this DQL on a repository class:

    public function filterNorm($codigo = null, $anno = null, $term = null, $comite_tecnico = null)
    {
        $qb = $this->getEntityManager()->createQueryBuilder();
        $qb
                ->select('n')
                ->from("AppBundle:Norma", "n");

        if ($codigo != NULL) {
            $qb->where(
                    $qb->expr()->like('n.numero', $qb->expr()->literal('%'.$codigo.'%'))
            );
        }

        if ($anno != NULL) {
            $qb->orWhere(
                    $qb->expr()->like('n.anno', $qb->expr()->literal('%'.(int) $anno.'%'))
            );
        }

        if ($term != NULL) {
            $qb->orWhere(
                    $qb->expr()->like('n.nombre', $qb->expr()->literal('%'.$term.'%'))
            );
        }

        if ($comite_tecnico != NULL) {
            $qb->orWhere(
                    $qb->expr()->like('n.comite_tecnico', $qb->expr()->literal('%'.(int) $comite_tecnico.'%'))
            );
        }

        return $qb->getQuery()->getResult();
    }

And when I try to execute it, leaving the first paramter null, I got this error:

An exception occurred while executing 'SELECT n0_.numero AS numero0, n0_.anno AS anno1, n0_.id AS id2, n0_.nombre AS nombre3, n0_.activo AS activo4, n0_.comite_tecnico_id AS comite_tecnico_id5 FROM nomencladores.norma n0_ WHERE n0_.anno LIKE '%34%' OR n0_.nombre LIKE '%sad%'':

SQLSTATE[42883]: Undefined function: 7 ERROR: operator does not exist: integer ~~ unknown
LINE 1: ...o_id5 FROM nomencladores.norma n0_ WHERE n0_.anno LIKE '%34%...
^
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts. 

I think in somewhere Doctrine is failing or not, not so sure, if not what will be the solution around this issue I'm having?



 Comments   
Comment by Marco Pivetta [ 23/Oct/14 ]

First of all: do not hardcode parameters in your DQL, as that represents a possible DQL injection vector.

Is your schema in sync with the ORM mappings? Postresql seems to complain about a missing operator between incompatible types.

Comment by Reynier Perez Mira [ 23/Oct/14 ]

What you mean with "do not hardcode parameters in your DQL"? How should I do this? And yes schema and ORM mappings are good:

Symfony > doctrine:schema:validate
[Mapping] OK - The mapping files are correct.
[Database] OK - The database schema is in sync with the mapping files.

Comment by Marco Pivetta [ 23/Oct/14 ]

What you mean with "do not hardcode parameters in your DQL"? How should I do this?

You wrote:

$qb->where($qb->expr()->like('n.numero', $qb->expr()->literal('%'.$codigo.'%')));

Should be:

$qb->where($qb->expr()->like('n.numero', ':codigo'));
$qb->setParameter('codigo', '%' . $codigo . '%');




[DDC-3036] [doctrine:generate:entities] Created: 17/Mar/14  Updated: 17/Mar/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM, Tools
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Miguel Simões Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: orm


 Description   

When using an Entity that is extended by some other class (that may not be a Doctrine Entity), the command doctrine:generate:entities should not create getters/setters that are already defined in the parent class.






[DDC-2556] Proxy getId() different code generated when using Trait Created: 16/Jul/13  Updated: 16/Jul/13

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Minor
Reporter: Eduardo Oliveira Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None


 Description   
class Timezone {
    /**
     * @var integer
     *
     * @ORM\Column(name="id", type="smallint", nullable=false, options={"unsigned": true})
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    protected $id;

    /**
     * Get id
     *
     * @return int
     */
    public function getId()
    {
        return $this->id;
    }
}

If I replace that code with a Trait with equal code and use it on the entity the proxy will be generated different.

Without trait:

    /**
     * {@inheritDoc}
     */
    public function getId()
    {
       	if ($this->__isInitialized__ === false) {
            return (int)  parent::getId();
        }


       	$this->__initializer__ && $this->__initializer__->__invoke($this, 'getId', array());

       	return parent::getId();
    }

With trait:

    /**
     * {@inheritDoc}
     */
    public function getId()
    {

        $this->__initializer__ && $this->__initializer__->__invoke($this, 'getId', array());

        return parent::getId();
    }

And in this code:

$timezone = $this->timezoneRepository->findReadOnly(
    $campaign->getTimezone()->getId()
);

I get:
Doctrine\ORM\ORMException: The identifier id is missing for a query of EB\Core\KernelBundle\Entity\Common\Timezone

$campaign->getTimezone()->getId()
Is returning null.

About the findReadOnly is just a wrapper that will fetch from cache if not there fetch from DB, anyway if I change it to find() exact same problem

My versions of doctrine:

$ php composer.phar show -i | grep doctrine
doctrine/annotations                   v1.1.2             Docblock Annotations Parser
doctrine/cache                         v1.0               Caching library offering an object-oriented API for many cache backends
doctrine/collections                   v1.1               Collections Abstraction library
doctrine/common                        2.4.0-RC4          Common Library for Doctrine projects
doctrine/data-fixtures                 dev-master 6924952 Data Fixtures for all Doctrine Object Managers
doctrine/dbal                          2.4.0-RC2          Database Abstraction Layer
doctrine/doctrine-bundle               v1.2.0             Symfony DoctrineBundle
doctrine/doctrine-fixtures-bundle      dev-master 512fc0f Symfony DoctrineFixturesBundle
doctrine/doctrine-migrations-bundle    dev-master 5fc1167 Symfony DoctrineMigrationsBundle
doctrine/inflector                     v1.0               Common String Manipulations with regard to casing and singular/plural rules.
doctrine/lexer                         v1.0               Base library for a lexer that can be used in Top-Down, Recursive Descent Parsers.
doctrine/migrations                    dev-master ced3b41 Database Schema migrations using Doctrine DBAL
doctrine/orm                           2.4.0-RC2          Object-Relational-Mapper for PHP
gedmo/doctrine-extensions              v2.3.6             Doctrine2 behavioral extensions
stof/doctrine-extensions-bundle        dev-master 6577f23 Integration of the gedmo/doctrine-extensions with Symfony2


 Comments   
Comment by Marco Pivetta [ 16/Jul/13 ]

Is the problem here the repository or just the trait method?

Can you please clean up the issue to insulate only the affected part? What is `getId` returning on that kind of object?

Comment by Eduardo Oliveira [ 16/Jul/13 ]

Marco I will try to do better, I will try to do make a test case that proves this.

But i can leave here already more info:

  • There is an entity Campaign that have a Timezone (ManyToOne in Campaign side);
  • This entity Campaign is fetch (with lazy in all associations) and serialized and put in cache;
  • This entity Campaign don't have timezone itself, but $campaign->getTimezone()->getId() hit the return (int) parent::getId(); on proxy not making any query to get the timezone, in the case with trait the proxy code is different and it returns null
Comment by Marco Pivetta [ 16/Jul/13 ]

Eduardo Oliveira is the proxy in a detached state when you do this? What happens if (with an existing identifier) you do following?

var_dump($entityManager->getReference('Timezone', 123)->getId());
Comment by Eduardo Oliveira [ 16/Jul/13 ]
var_dump ( $this->timezoneRepository->getEM()->getReference('EBCoreKernelBundle:Common\Timezone', 1)->getId() ); // 1

Marco yes is on detached mode, and I know that, I'm relying on the proxy to get the ID, because I don't want any queries being "issued", so I'm doing like

$campaign = $campaignRepository->findReadOnly(10); // hit cache
$timezoneRepository->findReadOnly($campaign->getTimezone()->getId() /* rely on proxy */); // hit memcache

This just work in the last versions of Doctrine that have lazy getId(), but it looks that for traits something wrong happens on the generation of Proxy.

If on Entity timezone I use the trait but override the getId() everything works perfect again.

Comment by Marco Pivetta [ 16/Jul/13 ]

Eduardo Oliveira does this also happen with 2.3.x? (asking because the proxy generation logic was rewritten for 2.4)

Comment by Eduardo Oliveira [ 16/Jul/13 ]

2.3 the proxy is quite different but the essence of the problem remains:

Proxy with trait (doesn't work, because as far as I know will try to make a query in detach mode):

    public function getId()
    {
        $this->__load();
        return parent::getId();
    }

Proxy without trait (works well):

    public function getId()
    {
        if ($this->__isInitialized__ === false) {
            return (int) $this->_identifier["id"];
        }
        $this->__load();
        return parent::getId();
    }
Comment by Marco Pivetta [ 16/Jul/13 ]

Ok, so at least I now know it's not an issue with the upgrade, but it was also borked before. Thanks for following along till here: I'll work on a patch as soon as I have time.





[DDC-2227] Add details about developer being responsible of inverse side of an association Created: 09/Jan/13  Updated: 09/Jan/13

Status: Open
Project: Doctrine 2 - ORM
Component/s: Documentation, ORM
Affects Version/s: None
Fix Version/s: None

Type: Documentation Priority: Minor
Reporter: Marco Pivetta Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None


 Description   

As far as I know, docs don't explain that it is up to the developer to keep the object graph consistent instead of relying on Doctrine ORM for everything.

For example, for many to many, examples like following may be used:
https://gist.github.com/3121916






[DDC-3087] Entity code generation skip setters Created: 15/Apr/14  Updated: 07/Jun/14

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Flip Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None


 Description   

When executing `orm:generate-entities` to generated methods, do not generate setters when entity is readOnly `@ORM\Entity(readOnly=true)`



 Comments   
Comment by Christopher Stea [ 06/Jun/14 ]

I would like to see this feature but at the field level: Maybe my entity has a timestamp property that is auto-generated (uses Timestampable). I want to be able to have a getter but not a setter. Right now if I doctrine:generate and remove the setter function, it is re-created any time i re-run doctrine:generate.

I would suggest having @ORM/readonly as an available setting for properties.

Comment by Marco Pivetta [ 06/Jun/14 ]

I think this use case is simply a customization that should be applied by coding, and not via the generator.

Comment by Christopher Stea [ 07/Jun/14 ]

I have made the required edits and created a pull request:

https://github.com/doctrine/doctrine2/pull/1052
http://www.doctrine-project.org/jira/browse/DDC-3157

In the case of the original request, users can use readonly on every property in the table if they chose to make the entire entity read-only.

Comment by Doctrine Bot [ 07/Jun/14 ]

A related Github Pull-Request [GH-1052] was closed:
https://github.com/doctrine/doctrine2/pull/1052

Comment by Marco Pivetta [ 07/Jun/14 ]

See comments on DDC-3157

Comment by Flip [ 07/Jun/14 ]

The PR by Christopher Stea adds a readonly annotation for properties. This ticket is about using the readonly annotation on the class (ORM\Entity) which is already in Doctrine. And just modify the code generator a little bit to skip the setters. Please reopen!!!

Comment by Marco Pivetta [ 07/Jun/14 ]

The PR introduces metadata used solely for the purpose of code generation.

Comment by Flip [ 07/Jun/14 ]

Yes that's not what THIS ticket is about, so please reopen because the PR is related, but not at all a solution to this ticket.

Comment by Marco Pivetta [ 07/Jun/14 ]

Hmm, makes sense. Re-opening then.





[DDC-2381] Pagination query can be simplified when simple joins are applied Created: 31/Mar/13  Updated: 29/Nov/13

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.3, 2.4
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Sergey Gerdel Assignee: Marco Pivetta
Resolution: Unresolved Votes: 3
Labels: paginator

Attachments: HTML File EXPLAIN.htmL     HTML File EXPLAIN1.html     HTML File EXPLAIN2.html     HTML File EXPLAIN3.htm     HTML File EXPLAIN4.htm    

 Description   

Hi.
In mysql db table i have > 200,000 items.
I use native doctrine pagination for paging the items list.
But generated query that gets ids for items list in paging works more then 150 sec on my workstation

SELECT DISTINCT id0 FROM (SELECT m0_.id AS id0, m0_.title AS title1, m0_.text AS text2, m0_.price AS price3, m0_.originalPrice AS originalPrice4, m0_.condition_type AS condition_type5, m0_.image_1 AS image_16, m0_.image_2 AS image_27, m0_.image_3 AS image_38, m0_.image_4 AS image_49, m0_.image_5 AS image_510, m0_.video AS video11, m0_.contact_email AS contact_email12, m0_.contact_name AS contact_name13, m0_.contact_phone AS contact_phone14, m0_.contact_type AS contact_type15, m0_.published AS published16, m0_.type AS type17, m0_.status AS status18, m0_.highlight AS highlight19, m0_.urgent AS urgent20, m0_.topads AS topads21, m0_.period AS period22, m0_.hits AS hits23, m0_.ip AS ip24, m0_.created_at AS created_at25, m0_.updated_at AS updated_at26 FROM milla_message m0_ INNER JOIN milla_currency m1_ ON m0_.currency_id = m1_.id INNER JOIN milla_category m2_ ON m0_.category_id = m2_.id INNER JOIN milla_region m3_ ON m0_.region_id = m3_.id INNER JOIN milla_city m4_ ON m0_.city_id = m4_.id WHERE m0_.status = 1 ORDER BY m0_.published DESC) dctrn_result LIMIT 20 OFFSET 0

source code
https://github.com/doctrine/doctrine2/blob/master/lib/Doctrine/ORM/Tools/Pagination/LimitSubqueryOutputWalker.php#L141

why SELECT DISTINCT %s FROM (%s) dctrn_result ???
why not SELECT DISTINCT m0_.id AS id0 FROM milla_message m0_ WHERE m0_.status = 1 ORDER BY m0_.published DESC LIMIT 20 OFFSET 0



 Comments   
Comment by Marco Pivetta [ 31/Mar/13 ]

Not a blocker

Comment by Marco Pivetta [ 31/Mar/13 ]

What's the result of `EXPLAIN` on a query without the subquery?

Comment by Sergey Gerdel [ 31/Mar/13 ]

explain without the subquery

Comment by Marco Pivetta [ 31/Mar/13 ]

Sergey Gerdel that's not the same query.

Comment by Marco Pivetta [ 31/Mar/13 ]

Sergey Gerdel this is still using

Using index; Using temporary; Using filesort

Check your indexes

Comment by Sergey Gerdel [ 31/Mar/13 ]

Not in the index problem

SELECT DISTINCT id0 FROM (SELECT m0_.id AS id0, m0_.title AS title1, m0_.text AS text2, m0_.price AS price3, m0_.originalPrice AS originalPrice4, m0_.condition_type AS condition_type5, m0_.image_1 AS image_16, m0_.image_2 AS image_27, m0_.image_3 AS image_38, m0_.image_4 AS image_49, m0_.image_5 AS image_510, m0_.video AS video11, m0_.contact_email AS contact_email12, m0_.contact_name AS contact_name13, m0_.contact_phone AS contact_phone14, m0_.contact_type AS contact_type15, m0_.published AS published16, m0_.type AS type17, m0_.status AS status18, m0_.highlight AS highlight19, m0_.urgent AS urgent20, m0_.topads AS topads21, m0_.period AS period22, m0_.hits AS hits23, m0_.ip AS ip24, m0_.created_at AS created_at25, m0_.updated_at AS updated_at26 FROM milla_message m0_ WHERE m0_.status = 1 ORDER BY m0_.published DESC) dctrn_result LIMIT 20 OFFSET 0

Time: 104.614s explain 3

SELECT DISTINCT m0_.id AS id0 FROM milla_message m0_ WHERE m0_.status = 1 ORDER BY m0_.published DESC LIMIT 20 OFFSET 0;

Time: 0.001s explain 4

Comment by Marco Pivetta [ 01/Apr/13 ]

Sergey Gerdel the ORM cannot simplify a complex query that way. There may be a conditional on one of the joined results, or generally usage of one of the joined results.

Things that could be optimized here are:

  • Removal of the `ORDER BY` clause when grouping (check ORM master, I think somebody already did that)
  • Trying to simplify the query by doing some serious hacking on the AST.

The problem I see here is that the chance to spawn random bugs because of the optimization is very high, and you'd have to rewrite `walkSelectStatement`

Comment by Marco Pivetta [ 01/Apr/13 ]

Marking as improvement

Comment by Sergey Gerdel [ 07/Apr/13 ]

Minor?
i have 100 sec for this query.
200k items are selected for temporary table. wtf?

OK. Programmers may be mistaken in parser
expect ORDER BY m0_.published DESC LIMIT 20 OFFSET 0) dctrn_result
Time: 0.001s

reality ORDER BY m0_.published DESC) dctrn_result LIMIT 20 OFFSET 0

Comment by Marco Pivetta [ 07/Apr/13 ]

Sergey Gerdel this problem does not introduce security issues and can be worked around by you while using your own pagination logic. It does not stop you from doing anything, that's why it's minor.

Comment by Sergey Gerdel [ 08/Apr/13 ]

ok)
i have already created my own paginator.
at last
please see how to fix this problem
https://github.com/Sergic/doctrine2/commit/2733c815387273d3bd199a68acb717e0cbc8ccfe

Comment by Tom Pryor [ 11/Oct/13 ]

I've also run into this problem which makes Doctrine's Paginator useless for large datasets. The actual query takes 0.002ms but the SELECT DISTINCT query doctrine executes takes over 30s because MySQL creates a temporary table with 200k+ records.

You don't need to remove the joined tables from the paginator query (I have conditions on the joined tables anyway), this has a negligible impact performance, but rather it is caused by SELECT DISTINCT and ORDER BY which no index configuration can solve. Rather, perhaps a flag could be added to the paginator to indicate my query does not fetch join any has many collections (i.e each row returned will be unique) negating the need for the SELECT DISTINCT. The pagination would only then need to perform the original query with LIMIT and OFFSET applied along with a separate COUNT query on the primary key, both of which are very fast as they'd use the indexes setup for the the original query.

Comment by Christophe Coevoet [ 11/Oct/13 ]

This flag already exists for the select query. See the second argument of the constructor.

For the count query, you should call $paginator->setUseOutputWalkers(false) to make it use a DQL AST walker instead of the SQL Output walker (the AST walker does not support counting on queries using HAVING which is why it is not selected by default)

Comment by Robert (Jamie) Munro [ 29/Nov/13 ]

I think this is more important than "minor", as I've experienced this when upgrading from 2.2. My site became unusably slow.

I can't easily work around it because I am using Symfony bundles that use this, I am not using this directly. None of the workarounds mentioned so far seem to have helped.





Generated at Sat Oct 25 01:00:00 UTC 2014 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.