[DDC-3212] Remove ArrayHydrator logic Created: 14/Jul/14  Updated: 13/Oct/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.5
Fix Version/s: 3.0
Security Level: All

Type: Improvement Priority: Critical
Reporter: Marco Pivetta Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: hydration, hydrator


 Description   

The Doctrine\ORM\Internal\Hydration\ArrayHydrator ( https://github.com/doctrine/doctrine2/blob/85fbf684363b932a4ebaf543ef059f9ee1e512b0/lib/Doctrine/ORM/Internal/Hydration/ArrayHydrator.php ) is currently very messy and complicated due to the lack of actual Doctrine\ORM\UnitOfWork references when working with it, since we are not dealing with objects.

In order to reduce the amount of bugs and code duplication when working with array hydration, I would simply suggest making use of the Doctrine\ORM\Internal\Hydration\ObjectHydrator ( https://github.com/doctrine/doctrine2/blob/85fbf684363b932a4ebaf543ef059f9ee1e512b0/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php ) and its siblings, and then extracting data coming from the results in it as an array.

This could be a simple reflection-based extraction (pseudo):

class ArrayHydrator
{
    public function hydrateAllData()
    {
        foreach ($this->objectHydrator->hydrateAllData() as $object) {
            yield $this->extractData($object);
        }
    }
}

The point here is that array-based hydration is not the primary focus of the ORM, and users should probably rely on SQL only when they want array hydration.



 Comments   
Comment by Marco Pivetta [ 14/Jul/14 ]

Note: performance is one of our biggest requirements, and array hydration is meant to achieve that. As I stated above, plain SQL is probably better for this sort of operation, so the issue is more about deprecating the ArrayHydrator completely, providing utilities to manipulate SQL resultsets instead.





[DDC-3441] Unidirectional ManyToOne Not Lazy Loading Created: 09/Dec/14  Updated: 11/Dec/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: Git Master, 2.4.6
Fix Version/s: None
Security Level: All

Type: Bug Priority: Critical
Reporter: Marcus Fulbright Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: lazy-loading, proxy, public-properties, reflection

Issue Links:
Dependency
depends on DDC-3442 [GH-1217] @DDC3441 failing test cases... Open

 Description   

The Unidirectional ManyToOne association described in the docs does not lazy load correctly. The appropriate SQL will get executed, and the returned Proxy does pass type hinting for the correct class. However, the lazy loaded object always has the following properties:

  • _initializer_
  • _cloner_
  • _isInitialized_
  • lazyPropertiesDefaults

Any properties from the class definition do not show up. This is problematic when trying to get reflected properties and their values. Methods are correctly reflected.

Pull request for failing test case



 Comments   
Comment by Marcus Fulbright [ 09/Dec/14 ]

Let me know if anything else is needed.

Comment by Doctrine Bot [ 11/Dec/14 ]

A related Github Pull-Request [GH-1217] was assigned:
https://github.com/doctrine/doctrine2/pull/1217

Comment by Marco Pivetta [ 11/Dec/14 ]

Looks like a current ORM limitation (private properties lazy loading).

Related:





[DDC-2560] Schema tool invalid DDL syntax for default values Created: 19/Jul/13  Updated: 19/Jul/13

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: 2.3.4
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Ben Davies Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: schematool
Environment:

Postgresql



 Description   

Not sure if this is Postgresql specific or not, but here are the reproduction steps:

Have a table with a column setup like so:

ALTER TABLE usr ADD COLUMN contact_count integer;
ALTER TABLE usr ALTER COLUMN contact_count SET NOT NULL;
ALTER TABLE usr ALTER COLUMN contact_count SET DEFAULT 0;

Have a Entity Column Definition like so:

    /**
     * @var integer $contactCount
     *
     * @ORM\Column(name="contact_count", type="integer", nullable=false)
     */
    private $contactCount = 0;

orm:schema-tool:update will generate the following SQL:

ALTER TABLE usr ALTER contact_count SET ;

adding

options={"default":0}

to the column definition results in no sql being generated (correctly I assume)



 Comments   
Comment by Marco Pivetta [ 19/Jul/13 ]

The ORM doesn't support default column values.
That's fine-tuning you can do on the column definitions.

Your "workaround" is actually the correct way of dealing with this kind of DDL change.

Comment by Ben Davies [ 19/Jul/13 ]

I realise the ORM doesn't support default values, but the point is that the schematool picks up a diff in the columns, tries to generate a diff, and generates an invalid sql statement. Surely that is a bug?

Comment by Marco Pivetta [ 19/Jul/13 ]

Ben Davies the DBAL is responsible for generating this wrong DDL... Are you able to reproduce this in DBAL only?

Comment by Ben Davies [ 19/Jul/13 ]

Happy to try.

Comment by Ben Davies [ 19/Jul/13 ]

here you go
https://github.com/bendavies/dbal/blob/DBAL-561/tests/Doctrine/Tests/DBAL/Functional/Ticket/DBAL561Test.php





[DDC-2452] Additional `WITH` condition in joins between JTI roots cause invalid SQL to be produced Created: 16/May/13  Updated: 27/Mar/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: DQL, ORM
Affects Version/s: Git Master
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Marco Pivetta Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: dql, sql-walker
Environment:

irrelevant



 Description   

Given a simple Joined Table Inheritance like following:

/**
 * @Entity @Table(name="foo") @InheritanceType("JOINED")
 * @DiscriminatorColumn(name="discr", type="string")
 * @DiscriminatorMap({"foo" = "DDC2452Foo", "bar" = "DDC2452Bar"})
 */
class DDC2452Foo
{
    /** @Id @Column(type="integer") @GeneratedValue */
    public $id;
}

/** @Entity @Table(name="bar") */
class DDC2452Bar extends DDC2452Foo
{
}

Following DQL

SELECT foo1 FROM DDC2452Foo foo1 JOIN DDC2452Foo foo2 WITH 1=1

Will produce broken SQL:

SELECT
    f0_.id AS id0, f0_.discr AS discr1 
FROM 
    foo f0_ 
LEFT JOIN bar b1_ 
    ON f0_.id = b1_.id 
LEFT JOIN foo f2_ 
LEFT JOIN bar b3_ 
    ON f2_.id = b3_.id 
    ON (1 = 1)

(please note the duplicate `ON` in the SQL)

That is caused because of the SQL walker producing the JTI filter with already the `ON` clause in it.

That happens because the JTI join conditions are added in https://github.com/doctrine/doctrine2/blob/2.4.0-BETA2/lib/Doctrine/ORM/Query/SqlWalker.php#L823-L825 (`walkRangeVariableDeclaration`), while the additional defined `WITH` conditions are considered in `walkJoinAssociationDeclaration` later on.

Added a test case and fix at https://github.com/doctrine/doctrine2/pull/668






[DDC-2147] Custom annotation in MappedSuperclass Created: 15/Nov/12  Updated: 07/May/13

Status: Awaiting Feedback
Project: Doctrine 2 - ORM
Component/s: Mapping Drivers, ORM
Affects Version/s: 2.2.1
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: kluk Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Linux 3.6.6-1.fc17.x86_64


Attachments: Text File error.log    

 Description   

When you try use custom annotation in mappedsuperclass like here http://pastebin.com/YMxKvcLk and then i try get metadata for class i get this error
Undefined index: fieldName
ClassMetadataInfo.php function addInheritedFieldMapping
Problem is that custom annotation doesnt have fieldName.
Quick fix is add condition to test if fieldName isset.



 Comments   
Comment by kluk [ 15/Nov/12 ]

error log from orm:validate-schema

Comment by Marco Pivetta [ 23/Jan/13 ]

Copying from pastebin:

use \Doctrine\ORM\Mapping as ORM;
use \xxx\Doctrine\Annotation\Entity as re;
use \xxx\Doctrine\Annotation\Forms as rf;
use \Doctrine\Common\Collections;
 
/**
 * @ORM\Entity
 */
class EventPicture extends \Picture
{
 
    /**
     * @ORM\ManyToOne(targetEntity="Event", inversedBy="eventPicture")
     * @ORM\JoinColumn(name="FK_Event", referencedColumnName="id")
     */
    protected $event;
 
}
use \Doctrine\ORM\Mapping as ORM;
use \xxx\Doctrine\Annotation\Entity as re;
use \xxx\Doctrine\Annotation\Forms as rf;
use \Doctrine\Common\Collections;
 
/** @ORM\MappedSuperclass */
class Picture extends \xxx\Doctrine\Entity\BaseEntity
{
 
    /**
     * @ORM\Id
     * @ORM\Column(type="integer")
     * @ORM\GeneratedValue(strategy="IDENTITY")
     * @var type
     */
    protected $id;
 
    /**
     * @ORM\Column(type="string",unique=true, nullable=false)
     *  @rf\FileUpload(fileSize="php",uploadType="local",fieldName="link",formControl="FileUploadField",image=true)
     *
     */
    protected $link;
 
}

kluk does this happen also with any other simple custom annotation? For example following:

/**
 * @Annotation 
 * @Target({"PROPERTY","ANNOTATION"})
 */
final class Entity implements Annotation
{
    /**
     * @var string
     */
    public $value;
}
Comment by kluk [ 30/Jan/13 ]

the same error when using simple annotation.

 
<?php

use \Doctrine\ORM\Mapping as ORM;
use \xxx\Doctrine\Annotation\Entity as re;
use \xxx\Doctrine\Annotation\Forms as rf;
use \Doctrine\Common\Collections;

/** @ORM\MappedSuperclass */
class Picture extends \xxx\Doctrine\Entity\BaseEntity {

    /**
     * @ORM\Id
     * @ORM\Column(type="integer")
     * @ORM\GeneratedValue(strategy="IDENTITY")
     * @var type
     */
    protected $id;

   
    /**
     * @ORM\Column(type="integer")
     * @rf\SetClass({"class","hide"})
     */
    public $value;

    /**
     * @ORM\Column(type="string",unique=true, nullable=true)
     * @rf\FileUpload(fileSize="php",uploadType="local",fieldName="link",formControl="FileUploadField",image=true)
     *
     */
    protected $link;

}

When i remove $value , $picture from class everything goes ok.
Easy fix for me is change ClassMetadataInfo.

    /**
     * INTERNAL:
     * Adds a field mapping without completing/validating it.
     * This is mainly used to add inherited field mappings to derived classes.
     *
     * @param array $fieldMapping
     *
     * @return void
     */
    public function addInheritedFieldMapping(array $fieldMapping)
    {
        if(isset($fieldMapping['fieldName'])){
        $this->fieldMappings[$fieldMapping['fieldName']] = $fieldMapping;
        $this->columnNames[$fieldMapping['fieldName']] = $fieldMapping['columnName'];
        $this->fieldNames[$fieldMapping['columnName']] = $fieldMapping['fieldName'];
        }
    }

But i dont know if this fix can break another part of doctrine.

Comment by Benjamin Eberlei [ 04/May/13 ]

Can you put the code of your annotations online? I can't seem to understand why this happens.

Comment by kluk [ 07/May/13 ]
Unable to find source-code formatter for language: php. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
 
namespace libs\Doctrine\Annotation\Entity;
use Doctrine\Common\Annotations\Annotation;

/** @Annotation */
class CustomMapping extends Annotation
{
    /**
     *
     * @var string
     */
    public $className;
    /**
     * 
     * 
     * @var IQueryable| string
     */
    public $dataSource;
}




[DDC-1963] Remove by-ref access to changeset in lifecycle event args Created: 31/Jul/12  Updated: 16/Apr/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: None
Fix Version/s: 2.x, 2.5
Security Level: All

Type: Improvement Priority: Major
Reporter: Marco Pivetta Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None


 Description   

UoW currently passes computed changesets to lifecycle event args byref. This has to be changed to force users to use UoW public API to modify changesets instead.






[DDC-1947] Update EBNF with arbitrary joins Created: 26/Jul/12  Updated: 26/Jul/12

Status: Open
Project: Doctrine 2 - ORM
Component/s: Documentation
Affects Version/s: 2.3, Git Master
Fix Version/s: None
Security Level: All

Type: Documentation Priority: Major
Reporter: Marco Pivetta Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Arbitrary joins need to be documented in EBNF






[DDC-3115] UnitOfWok can't access proxies protected property Created: 04/May/14  Updated: 04/May/14

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: Git Master
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Machete Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: orm
Environment:

php5.5.11-1~dotdeb.1
debian 7.5 64bit
apache2
doctrine-orm-module 0.8.0


Attachments: PNG File doctrine-error.PNG    

 Description   

Running the code on different environments for a while now, I have never had any problems. Recently I installed the code onto a different server and I've been experiencing the following issue:

> Fatal error: Cannot access protected property DoctrineORMModule\Proxy_CG_\Page\Entity\PageRevision::$date in src/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php on line 2596

I am absolutely clueless, why this happens. It seems to be affecting only specific data rows, as you might reproduce here: http://de.serlo.org/pages

The field is defined as followed:

    /**
     * @ORM\Column(type="datetime", options={"default"="CURRENT_TIMESTAMP"})
     */
    protected $date;

However, the exact same database snapshot works fine here: http://ptr.serlo.org/pages

Attached you will find a screenshot of the datarows. The yellow ones mark each one working and one not working row.

I have no idea what the problem is here.



 Comments   
Comment by Marco Pivetta [ 04/May/14 ]

Discussed this on IRC - very tricky, seems like one of the servers has the reflection property not set to "accessible"

Comment by Machete [ 04/May/14 ]

Adding `$class->reflFields[$field]->setAccessible(true)` before line 2596 resolved this issue. Even after removing it, everything worked fine.

note to self:
> [18:23] <@ocramius> machete_: can you find out where that particular property is instantiated?
> [18:23] <@ocramius> should be in `ClassMetadataInfo::wakeUpReflection()` or such
> [18:25] <@ocramius> if you find that again, consider looking at locations in the code where `new ReflectionClass()` is instantiated





[DDC-3063] Unexpected behavior with 'WHERE NOT IN' and empty array Created: 01/Apr/14  Updated: 06/May/14

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: DQL
Affects Version/s: 2.4
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Tim Stamp Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None
Environment:

CentOS, PHP 5.4, mysql



 Description   

I tried to set version to 2.4.0 but was prevented.

Assume the set 'n' contains 10 records, all with id > 0.

->andWhere('n.id NOT IN (:ids)')->setParameter('ids', [])
returns 0 records.

->andWhere('n.id NOT IN (:ids)')->setParameter('ids', [0])
returns 10 records.


 Comments   
Comment by Marco Pivetta [ 01/Apr/14 ]

This is a simple misunderstanding of how SQL IN() works

Comment by Tim Stamp [ 01/Apr/14 ]

You're right that its invalid SQL, but this is DQL. In SQL this would raise an error, in DQL it says nothing and returns nothing.

Isnt there a way to check if a statement has caused this error?

Comment by Marco Pivetta [ 01/Apr/14 ]

Tim Stamp yeah, when using IN() and NOT IN() you should actually check if the parameters are empty or not before adding the clause.

Comment by Steve Müller [ 01/Apr/14 ]

It's still weird that no error is raised by that. Tim Stamp can you check what SQL is generated by your query?

Comment by Steve Müller [ 01/Apr/14 ]

Hehe I think I got the problem. It might be DBAL related. See:

https://github.com/doctrine/dbal/blob/master/lib/Doctrine/DBAL/SQLParserUtils.php#L140
https://github.com/doctrine/dbal/blob/master/lib/Doctrine/DBAL/SQLParserUtils.php#L169

Looks like the implementing person did that by intention but don't ask me why.

Comment by Marco Pivetta [ 01/Apr/14 ]

Thats to avoid having an SQL statement like

SELECT * FROM foo WHERE bar IN()

, which is invalid.

Indeed, the NOT IN() case is not contemplated by that.

Comment by Steve Müller [ 01/Apr/14 ]

Just wondering if that kind of implementation isn't changing expectations because it looks like IN(NULL) will select all NULL rows then, even though this is not explicitly requested by the user. I don't care really TBH but this behaviour is not very transparent to the user. Personally I would like to see an error instead... Then at least I know what's going on and can fix that by additional checks instead.

Comment by Marco Pivetta [ 01/Apr/14 ]

Makes sense. Re-opening.

Comment by Steve Müller [ 01/Apr/14 ]

The question is if we can change this behaviour without breaking applications that rely on the current one. Because changing the code to throw an error breaks applications that insist on returning 0 rows for an empty array. I don't know what rules apply here concerning BC. What do you think Marco Pivetta?

Comment by Marco Pivetta [ 01/Apr/14 ]

Steve Müller existing apps should do following anyway:

if ($productIds) {
    $qb->andWhere('p.id IN (:productIds)')->setParameter('productIds', $productIds);
}

If they are not, that's most likely a bug in their codebase.

Comment by Steve Müller [ 01/Apr/14 ]

Agree. Was just wondering about what we can expect and what we can't expect.

Comment by Guilherme Blanco [ 21/Apr/14 ]

Steve Müller feel free to patch DBAL.
It's a bug, not a supported feature. If we have 2 tickets:
1- If I pass an empty array, I get nullable rows and I can't fix this
2- If I pass an empty array, it used to return nullable rows, now it returns nothing
Which one would you fix?

Comment by Tim Stamp [ 06/May/14 ]

Guilherme Blanco shouldn't both these use cases throw an exception, as both cases in MySQL would return an error?





[DDC-1698] Inconsistent proxy file name & namespace result in __PHP_Incomplete_Class when unserializing entities Created: 13/Mar/12  Updated: 25/Sep/13

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.2, 2.2.1, 2.3, 2.4
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: Benjamin Morel Assignee: Marco Pivetta
Resolution: Unresolved Votes: 2
Labels: proxy


 Description   

Starting with Doctrine 2.2, the Proxy classes have inconsistent naming with their file name, which raises problems with class autoloading.
For example, a class named Application\Model\User creates the following proxy class:

Application\Proxy\__CG__\Application\Model\User

This class is located in the following file:

Application/Proxy/__CG__ApplicationModelUser.php

But whe we serialize such an entity, then unserialize it in another session, the framework autoloader expects the class to be located in:

Application/Proxy/__CG__/Application/Model/User.php

But it is not.
As a result, a __PHP_Incomplete_Class is created instead of the expected proxy class.

I'm not sure whether this is an intended behavior, but I would assume this is a bug.



 Comments   
Comment by Benjamin Morel [ 13/Mar/12 ]

It looks like there is an even broader problem with the new _CG_ prefix; the PSR-0 standard for autoloading states that the underscores should be handled this way:

\namespace\package\Class_Name => {...}/namespace/package/Class/Name.php

Which means that in the above example, it could even expect the file to be located in:

Application/Proxy///CG///Application/Model/User.php

... which is far away from the actual location.
Upgrade to 2.2 broke this code, for us.

Comment by Benjamin Eberlei [ 14/Mar/12 ]

Proxy classes do not follow PSR-0. For the case unserializing objects we should provide an extra autoloader i guess.

See here how symfony does it https://github.com/doctrine/DoctrineBundle/blob/master/DoctrineBundle.php#L57

Comment by Benjamin Eberlei [ 14/Mar/12 ]

See https://github.com/doctrine/doctrine2/commit/9b4d60897dfc7e9b165712428539e694ec596c80 and https://github.com/doctrine/orm-documentation/commit/01381fae1ff3d4944086c7cfe46721925bf6ca15

Comment by Benjamin Morel [ 14/Mar/12 ]

Thanks for the quick fix, Benjamin.
However, I have to admit that I'm not fully happy with the fix, as we (and probably many others) are not using the Doctrine autoloader.
I supposed that the purpose of PSR-0 was precisely not to be tied to a particular autoloader implementation, and this benefit is lost with this version of Doctrine.

You mentioned in the doc that the proxies are not PSR-0 compliant "for implementation reasons"; as this was working fine before 2.2, could you please explain what requirement prevents Doctrine from keeping the previous naming convention?

Comment by Benjamin Eberlei [ 29/Mar/12 ]

In 2.1 the proxies are not PSR-0 compatible themselves, however their class naming is simpler.

In 2.2 we changed proxy names so that you can derive the original name of the proxy by searching for the _CG_ flag. This flag obviously contains the __ chars that some PSR autoloaders detect as directory seperators. I agree this is an unfortunate decision, but it was done this way.

I do think however that we can automatically register the proxy atuoloader (if not yet done) in EntityManager#create(). This would hide this fact from developers automatically.

Comment by Benjamin Morel [ 29/Oct/12 ]

@Benjamin Eberlei
In 2.3 we still have to manually call Autoloader::register() before unserializing entities that may contain proxies.
So EntityManager::create() still doesn't register it. Is there a plan to add this feature?

Comment by Benjamin Eberlei [ 06/Jan/13 ]

Benjamin Morel Not at the moment, seems too dangerous for me since it might produce race conditions. This should really be done in the bootstrap of the system.

We need to document this though.

Comment by Benjamin Morel [ 06/Jan/13 ]

Ok, thanks for your answer!

Comment by Matthieu Napoli [ 09/Sep/13 ]

Sorry to be a pain, but can the description of the task be updated:

  • Affects Version/s: should show all versions (or none) but not just 2.2, 2.2.1
  • Fix Version/s: should be blank

This is to make it clear that this bug is not resolved (since we are way past v2.2.2).

Thanks

Comment by Chris Hawkins [ 25/Sep/13 ]

Some additional documentation about this would be useful. I just spent some considerable time diagnosing an error related to restoring an entity from a cache. I wasn't able to find a mention of the need to call Autoloader::register until I found this page. It was resolved fairly quickly after I found this. For reference, I added this to my bootstrap:

ini_set('unserialize_callback_func', 'proxymissing');
function proxymissing( $name ) {
	if ( preg_match( '/^DoctrineProxies/', $name ) ) {
		\Doctrine\ORM\Proxy\Autoloader::register( PROXYPATH, 'DoctrineProxies' );
	}
}




[DDC-1695] SQLs for PostgreSQL case sensite tables/fields are wrongly generated Created: 09/Mar/12  Updated: 06/Aug/14

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: 2.1.6, 2.4.2
Fix Version/s: 2.1.7, 2.2.2
Security Level: All

Type: Bug Priority: Major
Reporter: Ignacio Larranaga Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None
Environment:

PostgreSQL 9.x, Symfony 2


Attachments: Text File doctrine-2.1.6.patch     Text File SqlWalker.patch    

 Description   

The SQLs for case sensitive schemas in postgreSQL are wronly generated.

Example:
Schema:

CREATE TABLE "News" (
  "IdNews" serial NOT NULL,
  "IdUser" bigint NOT NULL,
  "IdLanguage" integer NOT NULL,
  "IdCondition" integer,
  "IdHealthProvider" integer,
  "IdSpeciality" integer,
  "IdMedicineType" integer,
  "IdTreatment" integer,
  "Title" character varying,
  "SmallText" character varying,
  "LongText" character varying,
  "PublishDate" timestamp with time zone,
  "IdxNews" tsvector,
  "Highlight" boolean NOT NULL DEFAULT false,
  "Order" integer NOT NULL DEFAULT 0,
  "Deleted" boolean NOT NULL DEFAULT false,
  "Active" boolean NOT NULL DEFAULT false,
  "UpdateToHighlighted" boolean DEFAULT false,
  CONSTRAINT "News_pkey" PRIMARY KEY ("IdNews" ));

Object (I added quotes trying to generate the SQLs quoted.:

<?php

namespace GlobalTreatments\ApplicationBundle\Entity;

use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\Table(name="""News""")
 * @ORM\Entity
 */
class News
{
    /**
     * @var integer $idNews
     *
     * @ORM\Column(name="""IdNews""", type="integer", nullable=false)
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="SEQUENCE")
     * @ORM\SequenceGenerator(sequenceName="""News_IdNews_seq""", allocationSize="1", initialValue="1")
     */
    private $idNews;

    /**
     * @var bigint $iduser
     *
     * @ORM\Column(name="""IdUser""", type="bigint", nullable=false)
     */
    private $idUser;

    /**
     * @var integer $idLanguage
     *
     * @ORM\Column(name="""IdLanguage""", type="integer", nullable=false)
     */
    private $idLanguage;

    /**
     * @var integer $idCondition
     *
     * @ORM\Column(name="""IdCondition""", type="integer", nullable=true)
     */
    private $idCondition;

    /**
     * @var integer $idHealthProvider
     *
     * @ORM\Column(name="""IdHealthProvider""", type="integer", nullable=true)
     */
    private $idHealthProvider;

    /**
     * @var integer $idSpeciality
     *
     * @ORM\Column(name="""IdSpeciality""", type="integer", nullable=true)
     */
    private $idSpeciality;

    /**
     * @var integer $idMedicineType
     *
     * @ORM\Column(name="""IdMedicineType""", type="integer", nullable=true)
     */
    private $idMedicineType;

    /**
     * @var integer $idTreatment
     *
     * @ORM\Column(name="""IdTreatment""", type="integer", nullable=true)
     */
    private $idTreatment;

    /**
     * @var string $title
     *
     * @ORM\Column(name="""Title""", type="string", nullable=true)
     */
    private $title;

    /**
     * @var string $smallText
     *
     * @ORM\Column(name="""SmallText""", type="string", nullable=true)
     */
    private $smallText;

    /**
     * @var string $longText
     *
     * @ORM\Column(name="""LongText""", type="string", nullable=true)
     */
    private $longText;

    /**
     * @var datetimetz $publishDate
     *
     * @ORM\Column(name="""PublishDate""", type="datetimetz", nullable=true)
     */
    private $publishDate;

    /**
     * @var tsvector $idxNews
     *
     * @ORM\Column(name="""IdxNews""", type="tsvector", nullable=true)
     */
    private $idxNews;

    /**
     * @var boolean $highlight
     *
     * @ORM\Column(name="""Highlight""", type="boolean", nullable=false)
     */
    private $highlight;

    /**
     * @var integer $order
     *
     * @ORM\Column(name="""Order""", type="integer", nullable=false)
     */
    private $order;

    /**
     * @var boolean $deleted
     *
     * @ORM\Column(name="""Deleted""", type="boolean", nullable=false)
     */
    private $deleted;

    /**
     * @var boolean $active
     *
     * @ORM\Column(name="""Active""", type="boolean", nullable=false)
     */
    private $active;

    /**
     * @var boolean $updateToHighlighted
     *
     * @ORM\Column(name="""UpdateToHighlighted""", type="boolean", nullable=true)
     */
    private $updateToHighlighted;

    /**
     * @var condition
     *
     * @ORM\ManyToOne(targetEntity="Condition")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="""IdCondition""", referencedColumnName="""IdCondition""")
     * })
     */
    private $condition;

    /**
     * @var healthProvider
     *
     * @ORM\ManyToOne(targetEntity="HealthProvider")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="""IdHealthProvider""", referencedColumnName="""IdHealthProvider""")
     * })
     */
    private $healthProvider;

    /**
     * @var language
     *
     * @ORM\ManyToOne(targetEntity="Language")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="""IdLanguage""", referencedColumnName="""IdLanguage""")
     * })
     */
    private $language;

    /**
     * @var medicineType
     *
     * @ORM\ManyToOne(targetEntity="MedicineType")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="""IdMedicineType""", referencedColumnName="""IdMedicineType""")
     * })
     */
    private $medicineType;

    /**
     * @var speciality
     *
     * @ORM\ManyToOne(targetEntity="Speciality")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="""IdSpeciality""", referencedColumnName="""IdSpeciality""")
     * })
     */
    private $speciality;

    /**
     * @var treatment
     *
     * @ORM\ManyToOne(targetEntity="Treatment")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="""IdTreatment""", referencedColumnName="""IdTreatment""")
     * })
     */
    private $treatment;

    /**
     * @var user
     *
     * @ORM\ManyToOne(targetEntity="User")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="""IdUser""", referencedColumnName="""IdUser""")
     * })
     */
    private $user;

    ....

}

DQL:

'SELECT n.smallText, n.publishDate ' .
'FROM News n ' .
	'INNER JOIN n.language la ' .
'WHERE la.languageCode = :languageCode ' .
'ORDER BY n.publishDate DESC'

Generated SQL:

SELECT "0_."SmallText" AS "SmallText"0, "0_."PublishDate" AS "PublishDate"1 FROM "News" "0_ INNER JOIN "Language" "1_ ON "0_."IdLanguage" = "1_."IdLanguage" WHERE "1_."LanguageCode" = ? ORDER BY "0_."PublishDate" DESC LIMIT 6

Notice there are unmattched " in the SQL.



 Comments   
Comment by Ignacio Larranaga [ 09/Mar/12 ]

If there is another approach to specify the table/column names are case sensitive in PGSQL please let me know.

Comment by Ignacio Larranaga [ 09/Mar/12 ]

Just to comment. I also tried the normal quoting.

Example: @ORM\Column(name="`IdNews`", type="integer", nullable=false)

And does not work too because of the same reason.

Comment by Ignacio Larranaga [ 09/Mar/12 ]

Hi, I generate this patch and seems to be working for me (at least what I'm testing right now).

I used ´ to quote tables and single attributes (not relationships) and the SQLs are correctly generated.

Comment by Ignacio Larranaga [ 09/Mar/12 ]

Adding a new patch for another files I need to change.

Comment by Benjamin Eberlei [ 11/Mar/12 ]

Formatted code

Comment by Benjamin Eberlei [ 11/Mar/12 ]

Fixed and merged into 2.1.x and 2.2 branches

Comment by Julian Aicardo [ 05/Aug/14 ]

This bug still unresolved for version 2.3.6-DEV included in Symfony 2.3.18.

The patches does not work with this version.

Generated query:

SELECT "0_.id AS id0, "0_."verFirmware" AS verfirmware1, "0_."idEstacion" AS idestacion2, "0_."fechaHora" AS fechahora3, "0_.global AS global4, "0_.directa AS directa5, "0_.difusa AS difusa6, "0_."tempSensDirecta" AS tempsensdirecta7, "0_.vbat1 AS vbat18, "0_.vbat2 AS vbat29, "0_.flags AS flags10, "0_."Estacion" AS estacion11 FROM "RegistroRS" "0_ WHERE "0_."idEstacion" = '1' AND "0_."fechaHora" >= '2014-02-01 03:00:00' AND "0_."fechaHora" <= '2014-08-01 03:00:00'

Notice there are unmattched " in the SQL.

Comment by Marco Pivetta [ 05/Aug/14 ]

Julian Aicardo why are you using 2.3.6-DEV? Shouldn't you use a stable version? Is the bug reproducible also with later versions?

Comment by Julian Aicardo [ 06/Aug/14 ]

After changing to version 2.4.2 the bug still there. The same SQL query was generated.

Comment by Julian Aicardo [ 06/Aug/14 ]

The error occurs when try to get an alias based on case sensitive table name, because the alias is generated with the first character of table's name which is ".
Ex:
Table: car Alias: c0_
Table: "Car" Alias: "0_

Is in file SqlWalker.php line 295 in Doctrine 2.4.4:

SqlWalker.php
public function getSQLTableAlias($tableName, $dqlAlias = '')
{
    $tableName .= ($dqlAlias) ? '@[' . $dqlAlias . ']' : '';
        
    if ( ! isset($this->tableAliasMap[$tableName])) {
        $this->tableAliasMap[$tableName] = strtolower(substr($tableName, 0, 1)) . $this->tableAliasCounter++ . '_';
    }

    return $this->tableAliasMap[$tableName];
}




[DDC-2995] Joined Inheritance Mapping and Filters Created: 22/Feb/14  Updated: 14/Oct/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.4.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Bojidar Hristov Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: orm


 Description   

Is there any reason why Inheritance Mapping IS LEFT JOINED not INNER JOINED?

When there is a filter and it's left joined it happens a record might not have parent table record. For example

Class B extends Class A.
Class A have column is_active | and filter activated with is_active = 1 condition.

Final query: LEFT JOIN parent_table s1_ ON p2_.id = s1_.id AND (s1_.is_active = '1')

Record 1 have is_active = false, so result set looks like this:

table_b | table_a
id | id is_active discriminatorColumn
1 | null null null

and occurs exception: The discriminator column ... is missing for ....



 Comments   
Comment by Marco Pivetta [ 22/Feb/14 ]

JTI MUST be left-joined, since not every subclass of the root of the inheritance causes inserts on all of the inheritance tables.

What is the DQL query that is causing the problem?

Comment by Bojidar Hristov [ 22/Feb/14 ]

Hello Marco, I don't query from parent table, but from child. No sense to be LEFT JOINED.

DQL is something like that:

SELECT class1, class2 FROM Class1 class1 JOIN class1.class2 class2;

(Class2 extends Class3)
Class3 have is_active column.

And following filter:

public function addFilterConstraint(ClassMetadata $targetEntity, $targetTableAlias) {
if (!$targetEntity->reflClass->implementsInterface('ActivateableInterface'))

{ return ""; }

return $targetTableAlias . '.is_active = ' . $this->getParameter('isActive');
}

Filter condition is added in LEFT JOINED parent_table. Full example SQL:
SELECT ....
FROM class1 c1
INNER JOIN class2 c2 ON c1.c2_id = c2.id
LEFT JOIN table3 c3 ON c2.id = c3.id AND (c3.is_active = '1')

Comment by Marco Pivetta [ 26/Feb/14 ]

There are two possible solutions here:

  • verify if the selected entity is a leaf of a JTI, and use INNER JOIN on that if possible (probably not optimal, since the selection may be part of a LEFT JOIN itself)
  • add filtering on the discriminator column

Bojidar Hristov can you provide a failing test case to make it easier to work on this?

Comment by Bojidar Hristov [ 26/Feb/14 ]

Yes. Here is it sample failing test.

http://pastebin.com/xi8uUNX5 - Parent Class
http://pastebin.com/XRZLbyGX - Child Class
http://pastebin.com/HuPFb98u - Assoc Class
http://pastebin.com/hGqRk74e - TestCase

Here it should found 0 records because ot JOIN and is_active = true filter.

Comment by Benjamin Eberlei [ 23/Mar/14 ]

Since filters are only available on the root, i guess inner join is really necessary here.

Comment by Bojidar Hristov [ 14/Oct/14 ]

Do you need something more? The bug is status Feedback?





[DDC-3005] Events::postLoad fires without filled associations Created: 02/Mar/14  Updated: 19/Oct/14

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Artur Eshenbrener Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Dependency
depends on DDC-3070 [GH-1001] [DDC-3005] Defer invoking o... Open

 Description   

When we load entities throw one dql query like this:

SELECT e, joined_link FROM Entity LEFT JOIN e.link joined_link

In event subscriber, subscribed to postLoad event for instances of Entity property "link" of entity does not contains fetched in same query joined entity, and does not contains proxy object.

Tell me if failing test case needed.



 Comments   
Comment by Marco Pivetta [ 02/Mar/14 ]

The postLoad event is fired without warranty that association entities/proxies will be set:

http://docs.doctrine-project.org/en/latest/reference/events.html#lifecycle-events

Comment by Artur Eshenbrener [ 03/Mar/14 ]

But why? This shounds like involuntary restriction, and I think it can be fixed. Will my PR with fix accepted, or collaborators don't want to change this behaviour?

Comment by Marco Pivetta [ 03/Mar/14 ]

Artur Eshenbrener triggering `postLoad` in the correct moment in time (when all dependencies are loaded) requires a lot of additional complexity to be inserted in various locations of the ORM.

Since `postLoad` is supposed to be used like `__wakeup` and in general for simple tasks, this kind of refactoring/rewrite would be an overkill.

Comment by Artur Eshenbrener [ 04/Mar/14 ]

> requires a lot of additional complexity to be inserted in various locations of the ORM.
Sounds like "It is very hard to implement, and no one want to do this"

> and in general for simple tasks, this kind of refactoring/rewrite would be an overkill.
Disagree with this. Why only simple? I need to deal with associations in this event. Without this I should inject whole EntityManager to my entity, but I dont want to do this.

And I forced to repeat the question: will PR with fix accepted or not?

Comment by Marco Pivetta [ 04/Mar/14 ]

Sounds like "It is very hard to implement, and no one want to do this"

Not really, the main problem here is performance, since hydration would have to be completely redesigned.
Yes, "too hard" is a good reason for something that is an edge case.

And I forced to repeat the question: will PR with fix accepted or not?

Sure thing! Just needs to avoid a massive rewrite though.

Comment by Marco Pivetta [ 04/Mar/14 ]

To give you some hints, Doctrine\ORM\Events::postLoad is triggered in https://github.com/doctrine/doctrine2/blob/15432fc55f83c2d6ce8d9b86fd3139dd2a4fc328/lib/Doctrine/ORM/UnitOfWork.php#L2788, and Doctrine\ORM\UnitOfWork#createEntity() is used by all the various hydrators internally.

To ensure that Doctrine\ORM\Events::postLoad is triggered after an entity is fully loaded, the various hydrators must trigger the events after being sure that all data has been set, and that the entity is "ready" for use. That's some copy-paste work :\

Comment by Artur Eshenbrener [ 04/Mar/14 ]

I think the entry point is here: https://github.com/doctrine/doctrine2/blob/15432fc55f83c2d6ce8d9b86fd3139dd2a4fc328/lib/Doctrine/ORM/Internal/Hydration/AbstractHydrator.php#L140,
so, copy-paste work is not needed )

Comment by Artur Eshenbrener [ 05/Apr/14 ]

PR is ready: https://github.com/doctrine/doctrine2/pull/1001

Comment by Doctrine Bot [ 19/Oct/14 ]

A related Github Pull-Request [GH-1001] was assigned:
https://github.com/doctrine/doctrine2/pull/1001

Comment by Doctrine Bot [ 19/Oct/14 ]

A related Github Pull-Request [GH-1001] was unassigned:
https://github.com/doctrine/doctrine2/pull/1001





[DDC-3376] Only one row is returned Created: 06/Nov/14  Updated: 07/Nov/14

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.4.6
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Patryyyck Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: inheritance, lazy-loading, n+1-queries, sti
Environment:

Linux Mint 17
PHP 5.5.9
Apache 2.4.7
Symfony 2.4.10
Doctrine ORM 2.4.6
Doctrine DBAL 2.4.3
Sql Server database



 Description   

Hi,

I don't know if you are the appropriate person.
I have an issue with Doctrine ORM or DBAL.
By the way, i'm not sure the issue is in one of these bundles.

I'm trying to get all the rows of on an entity A which is a SINGLE_TABLE with 3 discriminators.
This entity has a property "parent" which is a self referencing property (ManyToOne on entity A).

The getResult of my query only return the first row of my table.
When i replace getResult by getArrayResult, i have all my table.
But i want the objects

After hours of debugging, i saw that the Doctrine ORM ObjectHydrator.php loops on the result of my query (which have the good number of results):

while ($row = $this->_stmt->fetch(PDO::FETCH_ASSOC)) {
    $this->hydrateRowData($row, $cache, $result);
}

But, the hydrateRowData executes another query to get the parent of each row (Doctrine DBAL Connection.php function executeQuery).
This function prepares a query to get the parent with a param (parent id) and i think the prepare function return the same statement (stmt) as the first query (the list of all the entity A).
So, the prepared query is executed but when the Doctrine ORM ObjectHydrator.php want to fetch the second row, he considers that there's no more rows to fetch and return the first rows.

I don't know if i am clear enough. Tell me if you want more details.

Oh. I forgot. If i try the same request with the same datas but in a mySQL database (pdo_mysql), i have a good result. I don't know if that helps.

Thanks

Regards



 Comments   
Comment by Marco Pivetta [ 07/Nov/14 ]

The issue you described is very well known and it is a limitation of the ORM. What happens is following:

1. you have an inheritance A -> B | C | D (A being the root of the inheritance)
2. A has a self-referencing association to A (typically like a hierarchy of parent-child)
3. the ORM tries to load an A instance that has a reference to a parent of type A. In order to do so, the ORM has to resolve the actual type of the referenced parent, and therefore cannot lazily load it via a proxy (a query has to be executed in order to find out the value of the discriminator column)

This means that for performance reasons you should NEVER reference the root of an inheritance, but only leafs (where the ORM does not need to do this sort of resolution).

See also http://doctrine-orm.readthedocs.org/en/latest/reference/inheritance-mapping.html#id4 (Performance impact of inheritance mapping) for more details.

Comment by Patryyyck [ 07/Nov/14 ]

Thanks Marco for your quick answer.

I understand the ORM limitation.
But i still don't understand why this limitation doesn't exist when i use a MySQL database.





[DDC-3361] Doctrine error while trying to execute a DQL query on PostgreSQL 9.2.x Created: 23/Oct/14  Updated: 08/Nov/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: DQL
Affects Version/s: 2.4.6
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Reynier Perez Mira Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: postgresql, typecast
Environment:

CentOS 6.5, PostgreSQL 9.2, Symfony 2.5.5, Doctrine 2.4.6



 Description   

I have this DQL on a repository class:

    public function filterNorm($codigo = null, $anno = null, $term = null, $comite_tecnico = null)
    {
        $qb = $this->getEntityManager()->createQueryBuilder();
        $qb
                ->select('n')
                ->from("AppBundle:Norma", "n");

        if ($codigo != NULL) {
            $qb->where(
                    $qb->expr()->like('n.numero', $qb->expr()->literal('%'.$codigo.'%'))
            );
        }

        if ($anno != NULL) {
            $qb->orWhere(
                    $qb->expr()->like('n.anno', $qb->expr()->literal('%'.(int) $anno.'%'))
            );
        }

        if ($term != NULL) {
            $qb->orWhere(
                    $qb->expr()->like('n.nombre', $qb->expr()->literal('%'.$term.'%'))
            );
        }

        if ($comite_tecnico != NULL) {
            $qb->orWhere(
                    $qb->expr()->like('n.comite_tecnico', $qb->expr()->literal('%'.(int) $comite_tecnico.'%'))
            );
        }

        return $qb->getQuery()->getResult();
    }

And when I try to execute it, leaving the first paramter null, I got this error:

An exception occurred while executing 'SELECT n0_.numero AS numero0, n0_.anno AS anno1, n0_.id AS id2, n0_.nombre AS nombre3, n0_.activo AS activo4, n0_.comite_tecnico_id AS comite_tecnico_id5 FROM nomencladores.norma n0_ WHERE n0_.anno LIKE '%34%' OR n0_.nombre LIKE '%sad%'':

SQLSTATE[42883]: Undefined function: 7 ERROR: operator does not exist: integer ~~ unknown
LINE 1: ...o_id5 FROM nomencladores.norma n0_ WHERE n0_.anno LIKE '%34%...
^
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts. 

I think in somewhere Doctrine is failing or not, not so sure, if not what will be the solution around this issue I'm having?



 Comments   
Comment by Marco Pivetta [ 23/Oct/14 ]

First of all: do not hardcode parameters in your DQL, as that represents a possible DQL injection vector.

Is your schema in sync with the ORM mappings? Postresql seems to complain about a missing operator between incompatible types.

Comment by Reynier Perez Mira [ 23/Oct/14 ]

What you mean with "do not hardcode parameters in your DQL"? How should I do this? And yes schema and ORM mappings are good:

Symfony > doctrine:schema:validate
[Mapping] OK - The mapping files are correct.
[Database] OK - The database schema is in sync with the mapping files.

Comment by Marco Pivetta [ 23/Oct/14 ]

What you mean with "do not hardcode parameters in your DQL"? How should I do this?

You wrote:

$qb->where($qb->expr()->like('n.numero', $qb->expr()->literal('%'.$codigo.'%')));

Should be:

$qb->where($qb->expr()->like('n.numero', ':codigo'));
$qb->setParameter('codigo', '%' . $codigo . '%');
Comment by Reynier Perez Mira [ 08/Nov/14 ]

Hi there @ocramius any advice around this issue? Any tip to get ride of this? I have similar queries all over the application I'm working on and still don't know how to deal with this, thanks

Comment by Marco Pivetta [ 08/Nov/14 ]

Seems to be a db-side type mismatch in a comparison, not a D2 bug.

Comment by Reynier Perez Mira [ 08/Nov/14 ]

@ocramius and how I should deal with this? Any idea?





[DDC-3385] [GH-1181] Support fetching entities by aliased name Created: 11/Nov/14  Updated: 11/Nov/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: Mapping Drivers, ORM
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: New Feature Priority: Major
Reporter: Doctrine Bot Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: class-alias, metadata, resolve-target-entities

Issue Links:
Dependency
depends on DCOM-257 [GH-342] Class metadata loading fallb... Open

 Description   

This issue is created automatically through a Github pull request on behalf of Ocramius:

Url: https://github.com/doctrine/doctrine2/pull/1181

Message:

See #385 (this PR supersedes it)

The final aim is allowing something like:

```php
$user = $em->find(UserInterface::class, 123);
```

This PR depends also on doctrine/common#342






[DDC-3387] [GH-1182] #1086 identifier type in proxies Created: 11/Nov/14  Updated: 17/Nov/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: DQL, ORM
Affects Version/s: 2.4.6
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: hydration, jti, persister, sti

Issue Links:
Dependency
is required for DDC-3223 Failing test (get id return string type) Open
Reference
is referenced by DDC-3394 UOW CreateEntity failure with zerofil... Resolved

 Description   

This issue is created automatically through a Github pull request on behalf of Ocramius:

Url: https://github.com/doctrine/doctrine2/pull/1182

Message:

See #1086. This fix simply ensures that the column type for identifiers of proxies of STI/JTI are kept consistent with the DBAL types they are mapped to.

May be related/colliding with #1178

Ping @jaspernbrouwer






[DDC-3400] Wrong result using php-cli Created: 19/Nov/14  Updated: 20/Nov/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.4.6
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Damir Abdijevic Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: cache, cli, dql, environment, orm
Environment:

Windows 7, Debian GNU/Linux 7 PHP 5.5.15



 Description   

Same query produces different results. With apache module everything works like expected. With php-cli the join condition to i18n table is ignored and calling getCountries() returns all and not only that entity that is matched by join condition.

        $qb = $this->_em->createQueryBuilder()
                        ->select('t', 'i18n')
                        ->from($this->_entityName, 't')
                        ->innerJoin('t.countries', 'i18n')
                        ->where('i18n.locale = :localeId')
                        ->setParameter('localeId', $localeId);

Country Entity:

namespace MyApp\Model\Entity;

use Doctrine\ORM\Mapping as ORM;
use Doctrine\Common\Collections\ArrayCollection;

/**
 * Country entity
 *
 * @ORM\Entity(repositoryClass="MyApp\Model\Repository\Country")
 * @ORM\Table(name="country")
 *
 */
class Country
{
    /**
     * @var int
     * @ORM\Id
     * @ORM\Column(type="integer", name="id")
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    protected $id;

    /**
     * @var string
     * @ORM\Column(type="string", name="name", length=32, unique=true)
     */
    protected $name;

    /**
     * @var int
     * @ORM\Column(type="integer", name="sort", unique=true)
     */
    protected $sort;

    /**
     * @var ArrayCollection
     * @ORM\OneToMany(targetEntity="ProductDescription\Model\Entity\CountryI18n", mappedBy="country", cascade={"all"})
     */
    protected $countries;


    /**
     * Constructor
     *
     * @return Country
     */
    public function __construct()
    {
        $this->countries = new ArrayCollection();
    }

    /**
     * Getter for $this->id
     *
     * @return int
     */
    public function getId()
    {
        return $this->id;
    }

    /**
     * Setter for $this->id
     *
     * @param int $id entity id
     *
     * @return void
     */
    public function setId($id)
    {
        $this->id = (int) $id;
    }

    /**
     * Getter for $this->sort
     *
     * @return int
     */
    public function getSort()
    {
        return $this->sort;
    }

    /**
     * Setter for $this->sort
     *
     * @param int $sort sort order
     *
     * @return void
     */
    public function setSort($sort)
    {
        $this->sort = (int) $sort;
    }

    /**
     * Getter for $this->name
     *
     * @return string
     */
    public function getName()
    {
        return $this->name;
    }

    /**
     * Setter for $this->name
     *
     * @param string $name language name in german
     *
     * @return void
     */
    public function setName($name)
    {
        $this->name = (string) $name;
    }

   /**
    * Get collection from i18n table
    *
    * @return ArrayCollection
    */
    public function getCountries()
    {
        return $this->countries;
    }

    /**
     * Proxy method. So we have working with all entities same method
     * to getting i18n data.
     *
     * @return ArrayCollection
     */
    public function getI18n()
    {
        return $this->getCountries();
    }

CountryI18nEntity:

namespace MyApp\Model\Entity;

use Doctrine\ORM\Mapping as ORM;

/**
 * CountryI18n entity
 *
 * @ORM\Entity
 * @ORM\Table(name="country_i18n", uniqueConstraints={@ORM\UniqueConstraint(name="idx_UNIQUE_country_id_locale_id", columns={"country_id", "locale_id"})})
 */
class CountryI18n
{
    /**
     * @var int
     * @ORM\Id
     * @ORM\Column(type="integer", name="id")
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    protected $id;

    /**
     * @ORM\ManyToOne(targetEntity="ProductDescription\Model\Entity\Country", inversedBy="countries")
     * @ORM\JoinColumn(name="country_id", referencedColumnName="id")
     */
    protected $country;

    /**
     * @ORM\ManyToOne(targetEntity="ProductDescription\Model\Entity\Language", inversedBy="countries")
     * @ORM\JoinColumn(name="locale_id", referencedColumnName="id")
     */
    protected $locale;

    /**
     * @var string
     * @ORM\Column(type="string", name="name", length=255)
     */
    protected $name;


    /**
     * Getter for $this->id
     *
     * @return int
     */
    public function getId()
    {
        return $this->id;
    }

    /**
     * Setter for $this->id
     *
     * @param int $id id of row primary key
     *
     * @return void
     */
    public function setId($id)
    {
        $this->id = (int) $id;
    }

    /**
     * Getter for $this->country
     *
     * @return Country
     */
    public function getCountry()
    {
        return $this->country;
    }

    /**
     * Setter for $this->country
     *
     * @param Country $country country entity to set
     *
     * @return void
     */
    public function setCountry(Country $country)
    {
        $this->country = $country;
    }

    /**
     * Getter for $this->locale
     *
     * @return Language
     */
    public function getLocale()
    {
        return $this->locale;
    }

    /**
     * Setter for $this->locale
     *
     * @param Language $locale language entity to set as locale
     *
     * @return void
     */
    public function setLocale(Language $locale)
    {
        $this->locale = $locale;
    }

    /**
     * Getter for $this->name
     *
     * @return string
     */
    public function getName()
    {
        return $this->name;
    }

    /**
     * Setter for $this->name
     *
     * @param string $name translation name
     *
     * @return void
     */
    public function setName($name)
    {
        $this->name = (string) $name;
    }

}


 Comments   
Comment by Marco Pivetta [ 19/Nov/14 ]

Looks like a caching issue. The amount of information provided is insufficient as it is. I'd suggest verifying if the generated SQL is the same, and checking that all caches were cleared both in CLI and in WEB sapis.

Comment by Damir Abdijevic [ 20/Nov/14 ]

O.k. here some further information.

No caching like Xcache or APC is enabled. PHP 5.5 integrated opcache ist not enabled. All Doctrine caches are set to Array. The sql queries are in both cases exactly the same:

sql
SELECT c0_.id AS id0, c0_.name AS name1, c0_.sort AS sort2, c1_.id AS id3, c1_.name AS name4, c1_.country_id AS country_id5, c1_.locale_id AS locale_id6 FROM country c0_ INNER JOIN country_i18n c1_ ON c0_.id = c1_.country_id WHERE c1_.locale_id = ?

/* locale_id = 2 */

The sql result is correct

Running the console script a ZF2 Initializer runs before that performs this query builder query:

QueryBuilder

$qb = $this->_em->createQueryBuilder()
                            ->select('t', 'i18n', 'l')
                            ->from($this->_entityName, 't')
                            ->innerJoin('t.countries', 'i18n')
                            ->innerJoin('i18n.locale', 'l');

That results in following SQL:

sql
SELECT c0_.id AS id0, c0_.name AS name1, c0_.sort AS sort2, c1_.id AS id3, c1_.name AS name4, l2_.id AS id5, l2_.name AS name6, l2_.full_name AS full_name7, l2_.locale AS locale8, c1_.country_id AS country_id9, c1_.locale_id AS locale_id10, l2_.country_id AS country_id11 FROM country c0_ INNER JOIN country_i18n c1_ ON c0_.id = c1_.country_id INNER JOIN language l2_ ON c1_.locale_id = l2_.id
Comment by Marco Pivetta [ 20/Nov/14 ]

What happens if you run those SQL statements via CLI (dbal:run-sql) or WEB? Same results?

Comment by Damir Abdijevic [ 20/Nov/14 ]

The sql result is correct. Can I attach it to the ticket? I have exported it to csv.

Comment by Marco Pivetta [ 20/Nov/14 ]

If the same results are produced on CLI and WEB APIs then I suggest trying to insulate the issue in a functional test to be run in both context. You probably have a different ORM bootstrap for CLI and WEB.

Attaching a CSV for same results makes no real difference here.

Comment by Damir Abdijevic [ 20/Nov/14 ]

No, I didn't want to attach two times the same result. Wanted to attach it one time to show that those entitites that are wrong in the result doesn't appear in the sql result at all. I don't have different bootstraps. After a few short tests I think it is an error in th ArrayCache mechanism. The difference was that using Apache one initializer was not called. Calling this initializer in both cases leads now to wrong results in CLI and WEB API.

When the second query is not executed everything is fine. But when the second longer query runs and selects all i18n entities and after it the first query runs then the issue appears.





[DDC-3424] Class Table Inheritance - wrong table order on insert with more than one level of inheritance Created: 02/Dec/14  Updated: 04/Dec/14

Status: Awaiting Feedback
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.3.3
Fix Version/s: 2.2.3
Security Level: All

Type: Bug Priority: Major
Reporter: mohammed Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: ORM
Environment:

Mysql



 Description   

when i use class table inheritance with multiple levels in doctrine :
Account<=User<=Dealer
(Dealer inherits from User, User from Account)
Account has the discriminator column and mapping.
when i persist a new Dealer entity I get a foreign key error because there is no row in the User table.
So the order in which the insert statements are executed:
1) Insert into Account
2) Insert into Dealer (which causes the error)
3) insert into User
Can someone help me thanks in advance.



 Comments   
Comment by Marco Pivetta [ 02/Dec/14 ]

Is this also valid for 2.4.x and master? 2.3.3 is a very old version.

Additionally, could you come up with a small reproduction test case? You can check the ones at https://github.com/doctrine/doctrine2/tree/v2.4.6/tests/Doctrine/Tests/ORM/Functional/Ticket for reference

Comment by mohammed [ 03/Dec/14 ]

This is also valid for 2.4.x and master
this is the test case https://github.com/rogerghost/doctrine2/blob/patch-1/tests/Doctrine/Tests/ORM/Persisters/DDC3424Test.php





[DDC-3436] [GH-1212] [DDC-3108] Fix regression where join aliases were no longer accessible in Criteria expressions Created: 06/Dec/14  Updated: 08/Dec/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None


 Description   

This issue is created automatically through a Github pull request on behalf of kieljohn:

Url: https://github.com/doctrine/doctrine2/pull/1212

Message:

The solution to issue DDC-2764 was to blanket prefix all the fields with the root alias. The result was then impossible to use fields from multiple FROM tables or a JOIN.

This patch injects all available aliases into the comparison, if a field uses one of the aliases it will pass without modification. If it is an unknown alias it will prefix the first rootAlias listed using the same functionality as the deprecated getRootAlias() function.

Also improves the ORDER BY on Criteria expressions, again introduced by DDC-2764 using the above functionality, this was out of scope of bug DDC-3108.

Added unit tests to test new functionality.



 Comments   
Comment by Doctrine Bot [ 08/Dec/14 ]

A related Github Pull-Request [GH-1212] was assigned:
https://github.com/doctrine/doctrine2/pull/1212





[DDC-2838] Leaky abstraction when applying Criteria to hydrated/non-hydrated PersistentCollection Created: 03/Dec/13  Updated: 17/Dec/14

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.4.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: brian ridley Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None

Attachments: File DDC2838Test.php    
Issue Links:
Reference
is referenced by DCOM-249 Criteria are unable to locate getters... Open

 Description   

When applying a Criteria to a PersistentCollection that has been hydrated the field names must be camel case, if the collection has not yet been hydrated the field names must be underscore separated.

The github repo linked here contains a simplified testcase for the matrix of hydrated/non-hydrated entities and camel case/underscore separated fields.

https://github.com/ptlis/DoctrineTestcase



 Comments   
Comment by Marco Pivetta [ 18/Aug/14 ]

We can't check out an entire project just to test a bug.

Please write an actual functional test case that can be integrated into the Doctrine ORM test suite.

Comment by brian ridley [ 20/Aug/14 ]

Hi,

i'm happy to do so - i'll take a look at this over the weekend.

Comment by Simon Paridon [ 12/Dec/14 ]

brian ridley: Any progress on this? This is currently an issue for us as well and hope to get fixed. I could look into converting it to a test for integration with the test suite if you don't have the time... but it might take a while since I have no experience with the requirements that should be met. (Plus, I am not sure how tightly coupled it is with your project)

Comment by Simon Paridon [ 17/Dec/14 ]

Hi Marco Pivetta, brian ridley,

I attached a functional Test that integrates with the test suite. Please let me know if I should issue a PR, and I'll do that this evening.





[DDC-3036] [doctrine:generate:entities] Created: 17/Mar/14  Updated: 17/Mar/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM, Tools
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Miguel Simões Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: orm


 Description   

When using an Entity that is extended by some other class (that may not be a Doctrine Entity), the command doctrine:generate:entities should not create getters/setters that are already defined in the parent class.






[DDC-2556] Proxy getId() different code generated when using Trait Created: 16/Jul/13  Updated: 16/Jul/13

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Minor
Reporter: Eduardo Oliveira Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None


 Description   
class Timezone {
    /**
     * @var integer
     *
     * @ORM\Column(name="id", type="smallint", nullable=false, options={"unsigned": true})
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    protected $id;

    /**
     * Get id
     *
     * @return int
     */
    public function getId()
    {
        return $this->id;
    }
}

If I replace that code with a Trait with equal code and use it on the entity the proxy will be generated different.

Without trait:

    /**
     * {@inheritDoc}
     */
    public function getId()
    {
       	if ($this->__isInitialized__ === false) {
            return (int)  parent::getId();
        }


       	$this->__initializer__ && $this->__initializer__->__invoke($this, 'getId', array());

       	return parent::getId();
    }

With trait:

    /**
     * {@inheritDoc}
     */
    public function getId()
    {

        $this->__initializer__ && $this->__initializer__->__invoke($this, 'getId', array());

        return parent::getId();
    }

And in this code:

$timezone = $this->timezoneRepository->findReadOnly(
    $campaign->getTimezone()->getId()
);

I get:
Doctrine\ORM\ORMException: The identifier id is missing for a query of EB\Core\KernelBundle\Entity\Common\Timezone

$campaign->getTimezone()->getId()
Is returning null.

About the findReadOnly is just a wrapper that will fetch from cache if not there fetch from DB, anyway if I change it to find() exact same problem

My versions of doctrine:

$ php composer.phar show -i | grep doctrine
doctrine/annotations                   v1.1.2             Docblock Annotations Parser
doctrine/cache                         v1.0               Caching library offering an object-oriented API for many cache backends
doctrine/collections                   v1.1               Collections Abstraction library
doctrine/common                        2.4.0-RC4          Common Library for Doctrine projects
doctrine/data-fixtures                 dev-master 6924952 Data Fixtures for all Doctrine Object Managers
doctrine/dbal                          2.4.0-RC2          Database Abstraction Layer
doctrine/doctrine-bundle               v1.2.0             Symfony DoctrineBundle
doctrine/doctrine-fixtures-bundle      dev-master 512fc0f Symfony DoctrineFixturesBundle
doctrine/doctrine-migrations-bundle    dev-master 5fc1167 Symfony DoctrineMigrationsBundle
doctrine/inflector                     v1.0               Common String Manipulations with regard to casing and singular/plural rules.
doctrine/lexer                         v1.0               Base library for a lexer that can be used in Top-Down, Recursive Descent Parsers.
doctrine/migrations                    dev-master ced3b41 Database Schema migrations using Doctrine DBAL
doctrine/orm                           2.4.0-RC2          Object-Relational-Mapper for PHP
gedmo/doctrine-extensions              v2.3.6             Doctrine2 behavioral extensions
stof/doctrine-extensions-bundle        dev-master 6577f23 Integration of the gedmo/doctrine-extensions with Symfony2


 Comments   
Comment by Marco Pivetta [ 16/Jul/13 ]

Is the problem here the repository or just the trait method?

Can you please clean up the issue to insulate only the affected part? What is `getId` returning on that kind of object?

Comment by Eduardo Oliveira [ 16/Jul/13 ]

Marco I will try to do better, I will try to do make a test case that proves this.

But i can leave here already more info:

  • There is an entity Campaign that have a Timezone (ManyToOne in Campaign side);
  • This entity Campaign is fetch (with lazy in all associations) and serialized and put in cache;
  • This entity Campaign don't have timezone itself, but $campaign->getTimezone()->getId() hit the return (int) parent::getId(); on proxy not making any query to get the timezone, in the case with trait the proxy code is different and it returns null
Comment by Marco Pivetta [ 16/Jul/13 ]

Eduardo Oliveira is the proxy in a detached state when you do this? What happens if (with an existing identifier) you do following?

var_dump($entityManager->getReference('Timezone', 123)->getId());
Comment by Eduardo Oliveira [ 16/Jul/13 ]
var_dump ( $this->timezoneRepository->getEM()->getReference('EBCoreKernelBundle:Common\Timezone', 1)->getId() ); // 1

Marco yes is on detached mode, and I know that, I'm relying on the proxy to get the ID, because I don't want any queries being "issued", so I'm doing like

$campaign = $campaignRepository->findReadOnly(10); // hit cache
$timezoneRepository->findReadOnly($campaign->getTimezone()->getId() /* rely on proxy */); // hit memcache

This just work in the last versions of Doctrine that have lazy getId(), but it looks that for traits something wrong happens on the generation of Proxy.

If on Entity timezone I use the trait but override the getId() everything works perfect again.

Comment by Marco Pivetta [ 16/Jul/13 ]

Eduardo Oliveira does this also happen with 2.3.x? (asking because the proxy generation logic was rewritten for 2.4)

Comment by Eduardo Oliveira [ 16/Jul/13 ]

2.3 the proxy is quite different but the essence of the problem remains:

Proxy with trait (doesn't work, because as far as I know will try to make a query in detach mode):

    public function getId()
    {
        $this->__load();
        return parent::getId();
    }

Proxy without trait (works well):

    public function getId()
    {
        if ($this->__isInitialized__ === false) {
            return (int) $this->_identifier["id"];
        }
        $this->__load();
        return parent::getId();
    }
Comment by Marco Pivetta [ 16/Jul/13 ]

Ok, so at least I now know it's not an issue with the upgrade, but it was also borked before. Thanks for following along till here: I'll work on a patch as soon as I have time.





[DDC-2227] Add details about developer being responsible of inverse side of an association Created: 09/Jan/13  Updated: 09/Jan/13

Status: Open
Project: Doctrine 2 - ORM
Component/s: Documentation, ORM
Affects Version/s: None
Fix Version/s: None

Type: Documentation Priority: Minor
Reporter: Marco Pivetta Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None


 Description   

As far as I know, docs don't explain that it is up to the developer to keep the object graph consistent instead of relying on Doctrine ORM for everything.

For example, for many to many, examples like following may be used:
https://gist.github.com/3121916






[DDC-3087] Entity code generation skip setters Created: 15/Apr/14  Updated: 07/Jun/14

Status: Reopened
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Flip Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None


 Description   

When executing `orm:generate-entities` to generated methods, do not generate setters when entity is readOnly `@ORM\Entity(readOnly=true)`



 Comments   
Comment by Christopher Stea [ 06/Jun/14 ]

I would like to see this feature but at the field level: Maybe my entity has a timestamp property that is auto-generated (uses Timestampable). I want to be able to have a getter but not a setter. Right now if I doctrine:generate and remove the setter function, it is re-created any time i re-run doctrine:generate.

I would suggest having @ORM/readonly as an available setting for properties.

Comment by Marco Pivetta [ 06/Jun/14 ]

I think this use case is simply a customization that should be applied by coding, and not via the generator.

Comment by Christopher Stea [ 07/Jun/14 ]

I have made the required edits and created a pull request:

https://github.com/doctrine/doctrine2/pull/1052
http://www.doctrine-project.org/jira/browse/DDC-3157

In the case of the original request, users can use readonly on every property in the table if they chose to make the entire entity read-only.

Comment by Doctrine Bot [ 07/Jun/14 ]

A related Github Pull-Request [GH-1052] was closed:
https://github.com/doctrine/doctrine2/pull/1052

Comment by Marco Pivetta [ 07/Jun/14 ]

See comments on DDC-3157

Comment by Flip [ 07/Jun/14 ]

The PR by Christopher Stea adds a readonly annotation for properties. This ticket is about using the readonly annotation on the class (ORM\Entity) which is already in Doctrine. And just modify the code generator a little bit to skip the setters. Please reopen!!!

Comment by Marco Pivetta [ 07/Jun/14 ]

The PR introduces metadata used solely for the purpose of code generation.

Comment by Flip [ 07/Jun/14 ]

Yes that's not what THIS ticket is about, so please reopen because the PR is related, but not at all a solution to this ticket.

Comment by Marco Pivetta [ 07/Jun/14 ]

Hmm, makes sense. Re-opening then.





[DDC-2381] Pagination query can be simplified when simple joins are applied Created: 31/Mar/13  Updated: 29/Nov/13

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.3, 2.4
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Sergey Gerdel Assignee: Marco Pivetta
Resolution: Unresolved Votes: 3
Labels: paginator

Attachments: HTML File EXPLAIN.htmL     HTML File EXPLAIN1.html     HTML File EXPLAIN2.html     HTML File EXPLAIN3.htm     HTML File EXPLAIN4.htm    

 Description   

Hi.
In mysql db table i have > 200,000 items.
I use native doctrine pagination for paging the items list.
But generated query that gets ids for items list in paging works more then 150 sec on my workstation

SELECT DISTINCT id0 FROM (SELECT m0_.id AS id0, m0_.title AS title1, m0_.text AS text2, m0_.price AS price3, m0_.originalPrice AS originalPrice4, m0_.condition_type AS condition_type5, m0_.image_1 AS image_16, m0_.image_2 AS image_27, m0_.image_3 AS image_38, m0_.image_4 AS image_49, m0_.image_5 AS image_510, m0_.video AS video11, m0_.contact_email AS contact_email12, m0_.contact_name AS contact_name13, m0_.contact_phone AS contact_phone14, m0_.contact_type AS contact_type15, m0_.published AS published16, m0_.type AS type17, m0_.status AS status18, m0_.highlight AS highlight19, m0_.urgent AS urgent20, m0_.topads AS topads21, m0_.period AS period22, m0_.hits AS hits23, m0_.ip AS ip24, m0_.created_at AS created_at25, m0_.updated_at AS updated_at26 FROM milla_message m0_ INNER JOIN milla_currency m1_ ON m0_.currency_id = m1_.id INNER JOIN milla_category m2_ ON m0_.category_id = m2_.id INNER JOIN milla_region m3_ ON m0_.region_id = m3_.id INNER JOIN milla_city m4_ ON m0_.city_id = m4_.id WHERE m0_.status = 1 ORDER BY m0_.published DESC) dctrn_result LIMIT 20 OFFSET 0

source code
https://github.com/doctrine/doctrine2/blob/master/lib/Doctrine/ORM/Tools/Pagination/LimitSubqueryOutputWalker.php#L141

why SELECT DISTINCT %s FROM (%s) dctrn_result ???
why not SELECT DISTINCT m0_.id AS id0 FROM milla_message m0_ WHERE m0_.status = 1 ORDER BY m0_.published DESC LIMIT 20 OFFSET 0



 Comments   
Comment by Marco Pivetta [ 31/Mar/13 ]

Not a blocker

Comment by Marco Pivetta [ 31/Mar/13 ]

What's the result of `EXPLAIN` on a query without the subquery?

Comment by Sergey Gerdel [ 31/Mar/13 ]

explain without the subquery

Comment by Marco Pivetta [ 31/Mar/13 ]

Sergey Gerdel that's not the same query.

Comment by Marco Pivetta [ 31/Mar/13 ]

Sergey Gerdel this is still using

Using index; Using temporary; Using filesort

Check your indexes

Comment by Sergey Gerdel [ 31/Mar/13 ]

Not in the index problem

SELECT DISTINCT id0 FROM (SELECT m0_.id AS id0, m0_.title AS title1, m0_.text AS text2, m0_.price AS price3, m0_.originalPrice AS originalPrice4, m0_.condition_type AS condition_type5, m0_.image_1 AS image_16, m0_.image_2 AS image_27, m0_.image_3 AS image_38, m0_.image_4 AS image_49, m0_.image_5 AS image_510, m0_.video AS video11, m0_.contact_email AS contact_email12, m0_.contact_name AS contact_name13, m0_.contact_phone AS contact_phone14, m0_.contact_type AS contact_type15, m0_.published AS published16, m0_.type AS type17, m0_.status AS status18, m0_.highlight AS highlight19, m0_.urgent AS urgent20, m0_.topads AS topads21, m0_.period AS period22, m0_.hits AS hits23, m0_.ip AS ip24, m0_.created_at AS created_at25, m0_.updated_at AS updated_at26 FROM milla_message m0_ WHERE m0_.status = 1 ORDER BY m0_.published DESC) dctrn_result LIMIT 20 OFFSET 0

Time: 104.614s explain 3

SELECT DISTINCT m0_.id AS id0 FROM milla_message m0_ WHERE m0_.status = 1 ORDER BY m0_.published DESC LIMIT 20 OFFSET 0;

Time: 0.001s explain 4

Comment by Marco Pivetta [ 01/Apr/13 ]

Sergey Gerdel the ORM cannot simplify a complex query that way. There may be a conditional on one of the joined results, or generally usage of one of the joined results.

Things that could be optimized here are:

  • Removal of the `ORDER BY` clause when grouping (check ORM master, I think somebody already did that)
  • Trying to simplify the query by doing some serious hacking on the AST.

The problem I see here is that the chance to spawn random bugs because of the optimization is very high, and you'd have to rewrite `walkSelectStatement`

Comment by Marco Pivetta [ 01/Apr/13 ]

Marking as improvement

Comment by Sergey Gerdel [ 07/Apr/13 ]

Minor?
i have 100 sec for this query.
200k items are selected for temporary table. wtf?

OK. Programmers may be mistaken in parser
expect ORDER BY m0_.published DESC LIMIT 20 OFFSET 0) dctrn_result
Time: 0.001s

reality ORDER BY m0_.published DESC) dctrn_result LIMIT 20 OFFSET 0

Comment by Marco Pivetta [ 07/Apr/13 ]

Sergey Gerdel this problem does not introduce security issues and can be worked around by you while using your own pagination logic. It does not stop you from doing anything, that's why it's minor.

Comment by Sergey Gerdel [ 08/Apr/13 ]

ok)
i have already created my own paginator.
at last
please see how to fix this problem
https://github.com/Sergic/doctrine2/commit/2733c815387273d3bd199a68acb717e0cbc8ccfe

Comment by Tom Pryor [ 11/Oct/13 ]

I've also run into this problem which makes Doctrine's Paginator useless for large datasets. The actual query takes 0.002ms but the SELECT DISTINCT query doctrine executes takes over 30s because MySQL creates a temporary table with 200k+ records.

You don't need to remove the joined tables from the paginator query (I have conditions on the joined tables anyway), this has a negligible impact performance, but rather it is caused by SELECT DISTINCT and ORDER BY which no index configuration can solve. Rather, perhaps a flag could be added to the paginator to indicate my query does not fetch join any has many collections (i.e each row returned will be unique) negating the need for the SELECT DISTINCT. The pagination would only then need to perform the original query with LIMIT and OFFSET applied along with a separate COUNT query on the primary key, both of which are very fast as they'd use the indexes setup for the the original query.

Comment by Christophe Coevoet [ 11/Oct/13 ]

This flag already exists for the select query. See the second argument of the constructor.

For the count query, you should call $paginator->setUseOutputWalkers(false) to make it use a DQL AST walker instead of the SQL Output walker (the AST walker does not support counting on queries using HAVING which is why it is not selected by default)

Comment by Robert (Jamie) Munro [ 29/Nov/13 ]

I think this is more important than "minor", as I've experienced this when upgrading from 2.2. My site became unusably slow.

I can't easily work around it because I am using Symfony bundles that use this, I am not using this directly. None of the workarounds mentioned so far seem to have helped.





[DDC-3223] Failing test (get id return string type) Created: 22/Jul/14  Updated: 11/Nov/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: Git Master
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Thomas Lallement Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Linux Ubuntu 13.10 / PHP 5.5.3 / Mysql 5.5.x


Issue Links:
Dependency
depends on DDC-3387 [GH-1182] #1086 identifier type in pr... Open

 Description   

I found an issue in a specific case, when you find a child entity (JOINED) which is linked to another child entity.

If I clone the linked child entity and call getId(), it gives the value as 'string' rather than 'integer'. If I set the property id as public, there is no problem.

If I call getId on the child entity rather than the clone, it gives 'integer' as expected. So it's weird...

See failing test here: https://github.com/doctrine/doctrine2/pull/1086



 Comments   
Comment by Doctrine Bot [ 11/Nov/14 ]

A related Github Pull-Request [GH-1086] was closed:
https://github.com/doctrine/doctrine2/pull/1086





[DDC-3391] RFC Allow adding extra metadata attributes Created: 13/Nov/14  Updated: 19/Nov/14

Status: Awaiting Feedback
Project: Doctrine 2 - ORM
Component/s: Mapping Drivers
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Gonzalo Vilaseca Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: drivers, metadata


 Description   

What I'd like to propose is a way of having custom metadata in ClassMetadata.

So the current problem is this: I want to add custom tags to doctrine metadata files, and then get this tags on a loadClassMetadata listener in order to modify the mapping.
Currently the only workaround is parsing this custom tags in the loadClassMetadata listener, going through all the files again.

I was thinking of having a new 'extra' array field in ClassMetadata, every time a driver parses a configuration, throw a new event with the read configuration (being xml, yaml...etc.). A custom listener will parse it looking for the custom tags and return an array that will be added to the 'extra' array field in ClassMetadata.

Then, on the loadClassMetadata listener we retrieve this 'extra' configuration information, and modify the mapping as we want.

Does this make sense?



 Comments   
Comment by Gonzalo Vilaseca [ 14/Nov/14 ]

Ideally instead of the 'extra' array field, it would be nice to easily extend Metadata class so that the new values are filled in the new extended Metadata class.

Comment by Christophe Coevoet [ 14/Nov/14 ]

This would be even worse. If the recommended way for extensions is to extend the Doctrine Metadata object to add their field, it means you can only use 1 extension at a time (because you cannot use the extended class of both extensions at the same time for the same object).
This is why it is much better for other libraries to store their own metadata in their own object instead of trying to put it inside the Doctrine ones.

I'm not even sure putting custom tags inside the Doctrine mapping file is the best solution. It may be better to use a separate metadata file for the mapping of the other library (just like you use a different mapping file for the Symfony validation mapping even if it applies to the same class than the Doctrine mapping for instance)

Comment by Gonzalo Vilaseca [ 14/Nov/14 ]

You're right regarding the metadata.

As for the separate files, validation in Symfony is not doctrine specific, that's why it's in a separate file. If you have a look at some doctrine extensions like ``Prezent translable`` or ``Doctrine2 behavioral extensions``, the natural location for the custom tags are the Doctrine mapping files as they are specific to doctrine, by looking at just one file you see the whole picture.

Yes, you could have your own mapping files, but then you would need to do some Symfony magic to be able to load them, and this is what would be nice to avoid.

I think of it as a way to easily extend doctrine mapping capabilities, in a 'plugin' way.

I'm currently working on a i18n bundle for Symfony, the tags I add in doctrine mapping files create associations between entities and their translations: I've had to create quite a few compiler passes for my current project to work as desired, and I see no way of abstracting this in a general way, it will need to be application specific. If I could hook into the Doctrine workflow and get those tags to populate my metadata class, that would be great, simple and reusable.

Comment by Gonzalo Vilaseca [ 19/Nov/14 ]

I've come out with another use case:
I need some custom metadata when the repository is instantiated, AFAIK there is no way of doing this right now, or is there?

Comment by Marco Pivetta [ 19/Nov/14 ]

You'd use a different metadata factory for that, specific to your use-case.
Mixing ORM mappings with the rest will just cause more coupling between the ORM and the userland use-case.





Generated at Fri Dec 19 00:38:25 UTC 2014 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.