Doctrine 2 - ORM
  1. Doctrine 2 - ORM
  2. DDC-546

New fetch mode EXTRA_LAZY for collections

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0-BETA1
    • Fix Version/s: 2.1
    • Component/s: ORM
    • Security Level: All
    • Labels:
      None

      Description

      A new fetch mode EXTRA_LAZY for one-to-many and many-to-many associations could have the following effects on PersistentCollection methods:

      • count() : Does not initialize the collection but issues a straight SQL count query.
      • remove($key) : Does not initialize the collection but issues straight SQL instead.
      • removeElement($element) : Does not initialize the collection but issues straight SQL instead.
      • contains($element) : Does not initialize the collection but issues straight SQL instead.
      • containsKey($key) : Does not initialize the collection but issues straight SQL instead.

      This mode would usually be useful for (potentially) large collections.

      We need to work out concrete use-case examples and implementation proposals before implementation.

      The semantics of the mentioned methods with EXTRA_LAZY need to be carefully worked out, i.e. what happens to already managed instances in case of the remove operations and stuff like that.

        Issue Links

          Activity

          Roman S. Borschel created issue -
          Roman S. Borschel made changes -
          Field Original Value New Value
          Description A new fetch mode EXTRA_LAZY for one-to-many and many-to-many associations could have the following effects on PersistentCollection methods:

           * count() : Does not initialize the collection but issues a straight SQL count query.
           * remove($key) : Does not initialize the collection but issues straight SQL instead.
           * removeElement($element) : Does not initialize the collection but issues straight SQL instead.
           * contains($element) : Does not initialize the collection but issues straight SQL instead.
           * containsKey($key) : Does not initialize the collection but issues straight SQL instead.

          This mode would usually be useful for (potentially) large collections.

          We need to work out concrete use-case examples and implementation proposals before implementation.
          A new fetch mode EXTRA_LAZY for one-to-many and many-to-many associations could have the following effects on PersistentCollection methods:

           * count() : Does not initialize the collection but issues a straight SQL count query.
           * remove($key) : Does not initialize the collection but issues straight SQL instead.
           * removeElement($element) : Does not initialize the collection but issues straight SQL instead.
           * contains($element) : Does not initialize the collection but issues straight SQL instead.
           * containsKey($key) : Does not initialize the collection but issues straight SQL instead.

          This mode would usually be useful for (potentially) large collections.

          We need to work out concrete use-case examples and implementation proposals before implementation.

          The semantics of the mentioned methods with EXTRA_LAZY need to be carefully worked out, i.e. what happens to already managed instances in case of the remove operations and stuff like that.
          Hide
          Marc Hodgins added a comment -

          Wow, this is an excellent idea. I was just thinking how it is unfortunate that there aren't ways to manipulate large collections without resorting to DQL.

          If nothing else, having access to a count() would be particularly great.

          Show
          Marc Hodgins added a comment - Wow, this is an excellent idea. I was just thinking how it is unfortunate that there aren't ways to manipulate large collections without resorting to DQL. If nothing else, having access to a count() would be particularly great.
          Hide
          Roman S. Borschel added a comment -

          @Marc: You might then also be interested in a related ticket: http://www.doctrine-project.org/jira/browse/DDC-547

          This would basically allow you to craft special implementations that inherit from PersistentCollection that are "optimized" in some way for a specific association (i.e. write your own custom SQL query when count() is invoked). Your wrapper class is then used by Doctrine instead of the default one, thats the idea. Of course implementing such a custom collection and overriding methods needs to be done carefully to fully preserve the public API contract (PersistentCollection wrappers are supposed to be "invisible" to the user after all). So this would be an advanced feature. EXTRA_LAZY I think is a pretty easy feature, accessible for all users with no further knowledge required. It just means that the collection delays initialization even more, whereever possible, which is a good thing for large collections which you normally don't really want to load, yet it allows you to use the normal OO API of your domain model without resorting to special (DQL) queries.

          Show
          Roman S. Borschel added a comment - @Marc: You might then also be interested in a related ticket: http://www.doctrine-project.org/jira/browse/DDC-547 This would basically allow you to craft special implementations that inherit from PersistentCollection that are "optimized" in some way for a specific association (i.e. write your own custom SQL query when count() is invoked). Your wrapper class is then used by Doctrine instead of the default one, thats the idea. Of course implementing such a custom collection and overriding methods needs to be done carefully to fully preserve the public API contract (PersistentCollection wrappers are supposed to be "invisible" to the user after all). So this would be an advanced feature. EXTRA_LAZY I think is a pretty easy feature, accessible for all users with no further knowledge required. It just means that the collection delays initialization even more, whereever possible, which is a good thing for large collections which you normally don't really want to load, yet it allows you to use the normal OO API of your domain model without resorting to special (DQL) queries.
          Hide
          Roman S. Borschel added a comment -

          Oh, I just saw you're watching that one already sorry for the noise

          Show
          Roman S. Borschel added a comment - Oh, I just saw you're watching that one already sorry for the noise
          Hide
          Roman S. Borschel added a comment -

          Rescheduling for beta3 as we're running out of time.

          Show
          Roman S. Borschel added a comment - Rescheduling for beta3 as we're running out of time.
          Roman S. Borschel made changes -
          Fix Version/s 2.0-BETA3 [ 10060 ]
          Fix Version/s 2.0-BETA2 [ 10050 ]
          Hide
          Benjamin Eberlei added a comment -

          Additional features that have been requested by users:

          • Fetch entries in this collection in batches, i.e. only in 20 steps, issue a limit 0,21 and check if there is more

          I don't know about the remove() and removeElement() operations, would they register with the UoW or directly execute the DELETE or UPDATE stements themselves?

          Show
          Benjamin Eberlei added a comment - Additional features that have been requested by users: Fetch entries in this collection in batches, i.e. only in 20 steps, issue a limit 0,21 and check if there is more I don't know about the remove() and removeElement() operations, would they register with the UoW or directly execute the DELETE or UPDATE stements themselves?
          Benjamin Eberlei made changes -
          Link This issue is referenced by DDC-80 [ DDC-80 ]
          Hide
          Roman S. Borschel added a comment -

          Pushing back to beta4.

          Show
          Roman S. Borschel added a comment - Pushing back to beta4.
          Roman S. Borschel made changes -
          Fix Version/s 2.0-BETA4 [ 10072 ]
          Fix Version/s 2.0-BETA3 [ 10060 ]
          Hide
          Roman S. Borschel added a comment -

          Moved to 2.1 due to lack of time for larger new stuff for 2.0.

          Show
          Roman S. Borschel added a comment - Moved to 2.1 due to lack of time for larger new stuff for 2.0.
          Roman S. Borschel made changes -
          Fix Version/s 2.1 [ 10022 ]
          Fix Version/s 2.0-BETA4 [ 10072 ]
          Hide
          Benjamin Eberlei added a comment -

          https://github.com/doctrine/doctrine2/tree/DDC-546 first prototype implementation with tests.

          Missing:

          • XML and YAML Mapping support.
          • Contains() support.
          • RemoveElement() support

          Necessary for later scheduling (only make sense when persisting keys):

          • ContainsKey() support
          • RemoveKey() support
          Show
          Benjamin Eberlei added a comment - https://github.com/doctrine/doctrine2/tree/DDC-546 first prototype implementation with tests. Missing: XML and YAML Mapping support. Contains() support. RemoveElement() support Necessary for later scheduling (only make sense when persisting keys): ContainsKey() support RemoveKey() support
          Benjamin Eberlei made changes -
          Assignee Roman S. Borschel [ romanb ] Benjamin Eberlei [ beberlei ]
          Benjamin Eberlei made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Hide
          Benjamin Eberlei added a comment -

          I updated the branch to include XML, YAML support, refactored a little bit and added contains() support.

          The RemoveElement() support should be put into its own ticket that relates to the EntityManager#link() / EntityManager#unlink() functionality.

          Show
          Benjamin Eberlei added a comment - I updated the branch to include XML, YAML support, refactored a little bit and added contains() support. The RemoveElement() support should be put into its own ticket that relates to the EntityManager#link() / EntityManager#unlink() functionality.
          Benjamin Eberlei made changes -
          Link This issue relates to DDC-128 [ DDC-128 ]
          Hide
          Benjamin Eberlei added a comment -

          Implemented

          Show
          Benjamin Eberlei added a comment - Implemented
          Benjamin Eberlei made changes -
          Status In Progress [ 3 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Benjamin Eberlei added a comment -

          Usage would be inside a @OneToMany or @ManyToMany definition set

          Annotations/XML:

          fetch="EXTRA_LAZY"
          

          YAML:

          fetch: EXTRA_LAZY
          
          Show
          Benjamin Eberlei added a comment - Usage would be inside a @OneToMany or @ManyToMany definition set Annotations/XML: fetch= "EXTRA_LAZY" YAML: fetch: EXTRA_LAZY
          Benjamin Eberlei made changes -
          Workflow jira [ 11277 ] jira-feedback [ 14403 ]
          Benjamin Eberlei made changes -
          Workflow jira-feedback [ 14403 ] jira-feedback2 [ 16267 ]
          Benjamin Eberlei made changes -
          Workflow jira-feedback2 [ 16267 ] jira-feedback3 [ 18520 ]

          This list may be incomplete, as errors occurred whilst retrieving source from linked applications:

          • Request to http://www.doctrine-project.org/fisheye/ failed: Error in remote call to 'FishEye 0 (http://www.doctrine-project.org/fisheye/)' (http://www.doctrine-project.org/fisheye) [AbstractRestCommand{path='/rest-service-fe/search-v1/crossRepositoryQuery', params={query=DDC-546, expand=changesets[0:20].revisions[0:29],reviews}, methodType=GET}] : Received status code 503 (Service Temporarily Unavailable)

            People

            • Assignee:
              Benjamin Eberlei
              Reporter:
              Roman S. Borschel
            • Votes:
              3 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: