Doctrine 2 - ORM
  1. Doctrine 2 - ORM
  2. DDC-1783

Combination of Query::iterate() and ObjectHydrator results in continued memory growth after clearing the entity manager

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.2
    • Fix Version/s: 2.2.3
    • Component/s: ORM
    • Security Level: All
    • Labels:
      None
    • Environment:
      ubuntu 11.04, php 5.3.6-13ubuntu3.6

      Description

      To reproduce:

      Start with the doctrine sandbox. Remove address relation from user. append the following to index.php

      [code]
      for($i=0;$i<100000;++$i) {
      $user = new User;
      $user->setName('foo' . $i);
      $em->persist($user);
      }
      $em->flush();
      $em->clear();

      $query = $em->getRepository('Entities\User')>createQueryBuilder('u')>getQuery();
      $i = 0;
      echo "Running test:" . PHP_EOL . "start: " . memory_get_usage() . PHP_EOL;
      foreach($query->iterate() as $row) {
      $em->detach($row[0]);
      if (++$i === 100000)

      { echo "end: " . memory_get_usage() . PHP_EOL; }

      }

      echo PHP_EOL . "done" . PHP_EOL;
      [/code]

      The result i get is

      [code]
      Running test:
      start: 7658928
      end: 32601776
      [/code]

      Adding my own custom hydrator which simply extends and resets the ObjectHydrator::_identifierMap (may cause bugs, i dont know) i get

      [code]
      Running test:
      start: 7658768
      end: 10724984
      [/code]

      This originally came up while working with an process to import an existing doctrine table of ~10M rows into elastic search. I realize going through the ORM will never be the most efficient, but there is room for more efficiency. With larger objects and larger result sets the memory growth is much more pronounced.

      1. sandbox.tgz
        384 kB
        Erik Bernhardson

        Activity

        Erik Bernhardson created issue -
        Hide
        Erik Bernhardson added a comment -

        code paste above didn't work out so well. Attached is a .tgz of the sandbox used above.

        Show
        Erik Bernhardson added a comment - code paste above didn't work out so well. Attached is a .tgz of the sandbox used above.
        Erik Bernhardson made changes -
        Field Original Value New Value
        Attachment sandbox.tgz [ 11184 ]
        Hide
        Benjamin Eberlei added a comment -

        Fixed

        Show
        Benjamin Eberlei added a comment - Fixed
        Benjamin Eberlei made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s 2.2.3 [ 10196 ]
        Resolution Fixed [ 1 ]
        Benjamin Eberlei made changes -
        Workflow jira [ 13640 ] jira-feedback [ 15323 ]
        Benjamin Eberlei made changes -
        Workflow jira-feedback [ 15323 ] jira-feedback2 [ 17187 ]
        Benjamin Eberlei made changes -
        Workflow jira-feedback2 [ 17187 ] jira-feedback3 [ 19442 ]

        This list may be incomplete, as errors occurred whilst retrieving source from linked applications:

        • Request to http://www.doctrine-project.org/fisheye/ failed: Error in remote call to 'FishEye 0 (http://www.doctrine-project.org/fisheye/)' (http://www.doctrine-project.org/fisheye) [AbstractRestCommand{path='/rest-service-fe/search-v1/crossRepositoryQuery', params={query=DDC-1783, expand=changesets[0:20].revisions[0:29],reviews}, methodType=GET}] : Received status code 503 (Service Temporarily Unavailable)

          People

          • Assignee:
            Benjamin Eberlei
            Reporter:
            Erik Bernhardson
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: