[DDC-1783] Combination of Query::iterate() and ObjectHydrator results in continued memory growth after clearing the entity manager Created: 17/Apr/12  Updated: 27/May/12  Resolved: 27/May/12

Status: Resolved
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.2.2
Fix Version/s: 2.2.3
Security Level: All

Type: Bug Priority: Major
Reporter: Erik Bernhardson Assignee: Benjamin Eberlei
Resolution: Fixed Votes: 0
Labels: None
Environment:

ubuntu 11.04, php 5.3.6-13ubuntu3.6


Attachments: File sandbox.tgz    

 Description   

To reproduce:

Start with the doctrine sandbox. Remove address relation from user. append the following to index.php

[code]
for($i=0;$i<100000;++$i) {
$user = new User;
$user->setName('foo' . $i);
$em->persist($user);
}
$em->flush();
$em->clear();

$query = $em->getRepository('Entities\User')>createQueryBuilder('u')>getQuery();
$i = 0;
echo "Running test:" . PHP_EOL . "start: " . memory_get_usage() . PHP_EOL;
foreach($query->iterate() as $row) {
$em->detach($row[0]);
if (++$i === 100000)

{ echo "end: " . memory_get_usage() . PHP_EOL; }

}

echo PHP_EOL . "done" . PHP_EOL;
[/code]

The result i get is

[code]
Running test:
start: 7658928
end: 32601776
[/code]

Adding my own custom hydrator which simply extends and resets the ObjectHydrator::_identifierMap (may cause bugs, i dont know) i get

[code]
Running test:
start: 7658768
end: 10724984
[/code]

This originally came up while working with an process to import an existing doctrine table of ~10M rows into elastic search. I realize going through the ORM will never be the most efficient, but there is room for more efficiency. With larger objects and larger result sets the memory growth is much more pronounced.



 Comments   
Comment by Erik Bernhardson [ 17/Apr/12 ]

code paste above didn't work out so well. Attached is a .tgz of the sandbox used above.

Comment by Benjamin Eberlei [ 27/May/12 ]

Fixed

Generated at Wed Oct 22 14:06:32 UTC 2014 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.