[DDC-3224] getResult(HYDRATE_OBJECT) with joined query is returning reduced number of rows Created: 23/Jul/14  Updated: 23/Jul/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.4.2
Fix Version/s: None
Security Level: All

Type: Bug Priority: Critical
Reporter: gondo Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: orm
Environment:

osx, PHP-FPM 5.5.13, nginx/1.6.0, mysql 5.6.19 Homebrew



 Description   

given that i have these 2 entities (pseodocode):

/**
 * @ORM\Table(name="entity1", options={"collate"="utf8_unicode_ci", "charset"="utf8"})
 * @ORM\Entity(repositoryClass="Entity1Repository")
 */
class Entity1 {
    /**
     * @var integer
     * @ORM\Column(name="id", type="integer")
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    protected $id;

    /**
     * @var string
     * @ORM\Column(name="name", type="string")
     */
    protected $name;

    /**
     * @var Entity2[]
     * @ORM\OneToMany(targetEntity="Entity2", mappedBy="entity1")
     */
    protected $entity2;
}

/**
 * @ORM\Table(name="entity2", options={"collate"="utf8_unicode_ci", "charset"="utf8"})
 * @ORM\Entity()
 */
class Entity2 {
    /**
     * @var integer
     * @ORM\Column(name="id", type="integer")
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    protected $id;

    /**
     * @var \DateTime
     * @ORM\Column(name="date", type="datetime")
     */
    protected $date;

    /**
     * @var Entity1
     * @ORM\ManyToOne(targetEntity="Entity1", inversedBy="entity2", fetch="EAGER")
     */
    protected $entity1;
}

tables and data

entity1:

id name
1 Jhon
2 Clare

entity2:

id date entity1_id
1 2011-01-01 00:00:01 1
2 2012-02-02 00:00:02 1
3 2013-03-03 00:00:03 2
4 2014-04-04 00:00:04 2

my query builder

use Doctrine\ORM\EntityRepository;

class Entity1Repository extends EntityRepository
{
    public function getData()
    {
        $qb = $this
            ->createQueryBuilder('Entity1')
            ->select('Entity1, Entity2.date')
            ->join('Entity1.entity2', 'Entity2', Join::WITH, 'Entity2.date > :date')
            ->setParameter('date', '2000-01-01 00:00:01')
        ;
        $result1 = $qb->getQuery()->getArrayResult(); // HYDRATE_ARRAY
        $result = $qb->getQuery()->getResult(); // HYDRATE_OBJECT
        
//        return $result1;
//        return $result2;
    }
}

proper result is this:

id name date
1 Jhon 2011-01-01 00:00:01
1 Jhon 2012-02-02 00:00:02
2 Clare 2013-03-03 00:00:03
2 Clare 2014-04-04 00:00:04

what is happening

$result1 = $qb->getQuery()->getArrayResult(); // HYDRATE_ARRAY

is really returning proper number of rows

BUT and here comes the BUG finally:

$result2 = $qb->getQuery()->getResult(); // HYDRATE_OBJECT

is returning just 2 rows:

id name date
1 Jhon 2011-01-01 00:00:01
2 Clare 2013-03-03 00:00:03

this is because somehow entities are made unique.

my workaround

as a workaround, what i have to do is, to exectute 2 queries. 1st to get just Entity1.ids joined with Entity2.dates by using `getArrayResult()`
and second query to get Entity1 objeces by unique ids from 1st query.
and than manualy join those results in php.



 Comments   
Comment by Marco Pivetta [ 23/Jul/14 ]

I see that you are using Join::WITH, but not providing a conditional in the example. Are you filtering a fetch-joined association?

Comment by gondo [ 23/Jul/14 ]

sorry i've tried to simplify my structure as much as it was possible, there are actually real conditions, one of them is date condition (among many others). i've updated my code

Comment by Marco Pivetta [ 23/Jul/14 ]

There are still some inconsistencies in the issue - where is that query parameter used, for example?

Comment by gondo [ 23/Jul/14 ]

im using it in EntityRepository (sorry, didnt know thats important) i ll update my code
and the whole code is in Symfony2 project (web and command line applications)

Comment by Marco Pivetta [ 23/Jul/14 ]

What I mean is that in

->setParameter('date', new \DateTime('last month'))

, parameter :date does not exist in the DQL.

Comment by gondo [ 23/Jul/14 ]

i see, sorry its part of JOIN condition, i've updated the code





[DDC-3222] PostUpdate event destroying collectionUpdates Created: 21/Jul/14  Updated: 21/Jul/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.4.2
Fix Version/s: None
Security Level: All

Type: Bug Priority: Critical
Reporter: Jacob Spizziri Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Mac OSX, AMP, php 5.3.27



 Description   

I have an entity that contains a Many-To-Many Unidirectional association. When the association is updated and the entity is flushed the changes are not being persisted to the database.

This is because the postUpdate event is being fired on executeUpdates, which is being called before the collection updates are being processed here:

            // Collection updates (deleteRows, updateRows, insertRows)
            foreach ($this->collectionUpdates as $collectionToUpdate) {
                $this->getCollectionPersister($collectionToUpdate->getMapping())->update($collectionToUpdate);
            }

This is occurring in UnitOfWork->commit() lines 333 through 366.

Apparently the postUpdate listener I wrote was causing the collectionUpdates property to be erased.



 Comments   
Comment by Marco Pivetta [ 21/Jul/14 ]

What does the listener actually do? Doesn't look like a bug to me...

Comment by Jacob Spizziri [ 21/Jul/14 ]

I'm using FOSElasticaBundle/ElasticSearch. On the postUpdate event I'm calling fos:elastica:populate on the entity indexes. Heres the code

<?php

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */

namespace SRC\Bundle\SearchBundle\EventListener;

use Symfony\Component\Console\Input\ArrayInput;
use Symfony\Component\Console\Output\NullOutput;
use Doctrine\ORM\Event\LifecycleEventArgs;
use SRC\Bundle\ProductBundle\Entity\Product;
use SRC\Bundle\UserBundle\Entity\User;

/**
 * Description of SearchIndexer
 *
 * @author jacobspizziri
 */
class SearchIndexer
{	
	protected $container;
	protected $logger;
	
	public function __construct($container, $logger) {
		$this->container = $container;
		$this->logger = $logger;
	}
	
	
	public function postUpdate(LifecycleEventArgs $args){
		$this->index($args);
	}
	
	protected function index($args){
		$entity = $args->getEntity();

		$arguments = false;
		$command = new \FOS\ElasticaBundle\Command\PopulateCommand();
		$command->setContainer($this->container);
		
		//TODO: probably need to specify the --env=prod/dev
		// update Elastica on Product update
		if ($entity instanceof Product) {
			$this->logger->info('Updating Product Search Indexes');
			
			$arguments = array(
					'--index'=>'website',
					'--type' => 'product'
					);
		} else if ($entity instanceof User) {
			$this->logger->info('Updating User Search Indexes');

			$arguments = array(
					'--index'=>'website',
					'--type' => 'user'
					);
		}
		
		
		if($command && $arguments){
			$input = new ArrayInput($arguments);
			$output = new NullOutput();
			
			$result = $command->run($input, $output);
			$this->logger->info('Finished Updating Search Indexes with result: '. strval($result));
		}
	}
}
?>

Am I using this event incorrectly? Should I only be using postPersist?





[DDC-3212] Remove ArrayHydrator logic Created: 14/Jul/14  Updated: 14/Jul/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.5
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Critical
Reporter: Marco Pivetta Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: hydration, hydrator


 Description   

The Doctrine\ORM\Internal\Hydration\ArrayHydrator ( https://github.com/doctrine/doctrine2/blob/85fbf684363b932a4ebaf543ef059f9ee1e512b0/lib/Doctrine/ORM/Internal/Hydration/ArrayHydrator.php ) is currently very messy and complicated due to the lack of actual Doctrine\ORM\UnitOfWork references when working with it, since we are not dealing with objects.

In order to reduce the amount of bugs and code duplication when working with array hydration, I would simply suggest making use of the Doctrine\ORM\Internal\Hydration\ObjectHydrator ( https://github.com/doctrine/doctrine2/blob/85fbf684363b932a4ebaf543ef059f9ee1e512b0/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php ) and its siblings, and then extracting data coming from the results in it as an array.

This could be a simple reflection-based extraction (pseudo):

class ArrayHydrator
{
    public function hydrateAllData()
    {
        foreach ($this->objectHydrator->hydrateAllData() as $object) {
            yield $this->extractData($object);
        }
    }
}

The point here is that array-based hydration is not the primary focus of the ORM, and users should probably rely on SQL only when they want array hydration.



 Comments   
Comment by Marco Pivetta [ 14/Jul/14 ]

Note: performance is one of our biggest requirements, and array hydration is meant to achieve that. As I stated above, plain SQL is probably better for this sort of operation, so the issue is more about deprecating the ArrayHydrator completely, providing utilities to manipulate SQL resultsets instead.





[DDC-3062] [GH-997] [FIX] Allow to use ManyToMany with all operators Created: 31/Mar/14  Updated: 08/Jul/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: 2.5
Security Level: All

Type: Bug Priority: Critical
Reporter: Doctrine Bot Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

This issue is created automatically through a Github pull request on behalf of bakura10:

Url: https://github.com/doctrine/doctrine2/pull/997

Message:

Hi,

ping @guillhermoblanco : I think this may be blocking for 2.5

I introduced not so long ago support for ManyToMany for Criteria. However, I realized my implementation was really incomplete, because I hard-coded the "=" operator (https://github.com/doctrine/doctrine2/pull/885/files#diff-982b7374bbe9d5f4b6b71f4869a446eaR575). This means that it fails in a lot of cases when you use something different than "eq" for Criteria.

This PR fixes that, however it's a bit hacky. The SqlExpressionVisitor was made by type hinting for a BasicEntityPersister, preventing us from using us for a collection persister. Therefore I added a new interface to keep BC.

There is also a lot of code duplication (the whole "getSelectConditionSQL" was copied from the BasicEntityPersister), but without trait or BC, I have no idea about how to solve it.

All tests pass, test were added for other operators.






[DDC-2953] ArrayHydrator: Not all items hydrated while orderBy Created: 05/Feb/14  Updated: 14/Jul/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.3.4, 2.4.1
Fix Version/s: None
Security Level: All

Type: Bug Priority: Critical
Reporter: Mariusz Jaskółka Assignee: Guilherme Blanco
Resolution: Unresolved Votes: 0
Labels: array, hydration
Environment:

Linux and Windows, PHP5


Attachments: File ArrayHydrator.php     File Array_Hydrator_4.2.php     File Array_hydrator_4.2_bugfix.php    

 Description   

I will explain the problem using example and pseudo-code:

I have query like that:
SELECT (...) FROM order LEFT JOIN person LEFT JOIN identifier (...) order by (...)

The rows returned by query are following (the order is very important):
order_id|person_id|identifier_id|
12 |21 |33 |
12 |21 |34 |
11 |21 |35 |
11 |21 |33 |
11 |21 |34 |
12 |21 |35 |

After hydration the result is like:
result[0][person][identifier][0][id]=33
result[0][person][identifier][0][id]=34
result[1][person][identifier][0][id]=35
result[1][person][identifier][0][id]=34

But it should be:
result[0][person][identifier][0][id]=33
result[0][person][identifier][0][id]=34
result[0][person][identifier][0][id]=35
result[1][person][identifier][0][id]=35
result[1][person][identifier][0][id]=34
result[1][person][identifier][0][id]=33

The reason is that ArrayHydrator::_identifierMap contains only object id and parents object id. In may example there is difference in parents parent (grandparent) id.



 Comments   
Comment by Marco Pivetta [ 05/Feb/14 ]

I've started work on this at https://github.com/doctrine/doctrine2/pull/933

Comment by Mariusz Jaskółka [ 06/Feb/14 ]

This is how I temporary solved the issue, maybe it can help (attachment).

Comment by Marco Pivetta [ 06/Feb/14 ]

Mariusz Jaskółka can you provide a diff with current master? This seems to be based off 2.3 or previous versions...

Comment by Mariusz Jaskółka [ 06/Feb/14 ]

Version 2.4.2 oryginal file and bugfix

Comment by Mariusz Jaskółka [ 06/Feb/14 ]

Oh sorry it is version 2.4.1 (composer downloaded that version for me).
I hope it will be ok.

Comment by Marco Pivetta [ 06/Feb/14 ]

Mariusz Jaskółka thanks! I'm trying it right now

Comment by Mariusz Jaskółka [ 06/Feb/14 ]

I added one dimention to $this->_identifierMap to be sure that all object's children's keys will be mapped separately.

Comment by Marco Pivetta [ 06/Feb/14 ]

Mariusz Jaskółka, I've applied your hotfix at https://github.com/doctrine/doctrine2/commit/c05921032ff6947daca2d7275031e5cde4700634 and the tests seem to pass on my system and on travis. You may want to check it out and see if it works for your use case.

Comment by Mariusz Jaskółka [ 06/Feb/14 ]

It works for my use case correctly. You apparently forgot to remove/comment line 82 which is not necessary now.

Comment by Marco Pivetta [ 06/Feb/14 ]

Mariusz Jaskółka fixed, thanks!

Comment by Marco Pivetta [ 14/Jul/14 ]

I don't think we'll have a fix for this issue for 2.5, as it will probably require a complete rewrite of the array hydrator

Comment by Guilherme Blanco [ 14/Jul/14 ]

... and introducing BC breaks.





Generated at Thu Jul 31 17:49:13 UTC 2014 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.