Uploaded image for project: 'Doctrine 2 - ORM'
  1. Doctrine 2 - ORM
  2. DDC-1783

Combination of Query::iterate() and ObjectHydrator results in continued memory growth after clearing the entity manager

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.2
    • Fix Version/s: 2.2.3
    • Component/s: ORM
    • Security Level: All
    • Labels:
      None
    • Environment:
      ubuntu 11.04, php 5.3.6-13ubuntu3.6

      Description

      To reproduce:

      Start with the doctrine sandbox. Remove address relation from user. append the following to index.php

      [code]
      for($i=0;$i<100000;++$i) {
      $user = new User;
      $user->setName('foo' . $i);
      $em->persist($user);
      }
      $em->flush();
      $em->clear();

      $query = $em->getRepository('Entities\User')>createQueryBuilder('u')>getQuery();
      $i = 0;
      echo "Running test:" . PHP_EOL . "start: " . memory_get_usage() . PHP_EOL;
      foreach($query->iterate() as $row) {
      $em->detach($row[0]);
      if (++$i === 100000)

      { echo "end: " . memory_get_usage() . PHP_EOL; }

      }

      echo PHP_EOL . "done" . PHP_EOL;
      [/code]

      The result i get is

      [code]
      Running test:
      start: 7658928
      end: 32601776
      [/code]

      Adding my own custom hydrator which simply extends and resets the ObjectHydrator::_identifierMap (may cause bugs, i dont know) i get

      [code]
      Running test:
      start: 7658768
      end: 10724984
      [/code]

      This originally came up while working with an process to import an existing doctrine table of ~10M rows into elastic search. I realize going through the ORM will never be the most efficient, but there is room for more efficiency. With larger objects and larger result sets the memory growth is much more pronounced.

      1. sandbox.tgz
        384 kB
        Erik Bernhardson

        Activity

        ebernhardson Erik Bernhardson created issue -
        Hide
        ebernhardson Erik Bernhardson added a comment -

        code paste above didn't work out so well. Attached is a .tgz of the sandbox used above.

        Show
        ebernhardson Erik Bernhardson added a comment - code paste above didn't work out so well. Attached is a .tgz of the sandbox used above.
        ebernhardson Erik Bernhardson made changes -
        Field Original Value New Value
        Attachment sandbox.tgz [ 11184 ]
        Hide
        beberlei Benjamin Eberlei added a comment -

        Fixed

        Show
        beberlei Benjamin Eberlei added a comment - Fixed
        beberlei Benjamin Eberlei made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s 2.2.3 [ 10196 ]
        Resolution Fixed [ 1 ]
        beberlei Benjamin Eberlei made changes -
        Workflow jira [ 13640 ] jira-feedback [ 15323 ]
        beberlei Benjamin Eberlei made changes -
        Workflow jira-feedback [ 15323 ] jira-feedback2 [ 17187 ]
        beberlei Benjamin Eberlei made changes -
        Workflow jira-feedback2 [ 17187 ] jira-feedback3 [ 19442 ]

        This list may be incomplete, as errors occurred whilst retrieving source from linked applications:

        • Request to http://www.doctrine-project.org/fisheye/ failed: Error in remote call to 'FishEye 0 (http://www.doctrine-project.org/fisheye/)' (http://www.doctrine-project.org/fisheye) [AbstractRestCommand{path='/rest-service-fe/search-v1/crossRepositoryQuery', params={expand=changesets[0:20].revisions[0:29],reviews, query=DDC-1783}, methodType=GET}] : Received status code 503 (Service Temporarily Unavailable)

          People

          • Assignee:
            beberlei Benjamin Eberlei
            Reporter:
            ebernhardson Erik Bernhardson
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: