[DDC-546] New fetch mode EXTRA_LAZY for collections Created: 27/Apr/10  Updated: 02/Jan/11  Resolved: 02/Jan/11

Status: Resolved
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.0-BETA1
Fix Version/s: 2.1
Security Level: All

Type: New Feature Priority: Major
Reporter: Roman S. Borschel Assignee: Benjamin Eberlei
Resolution: Fixed Votes: 3
Labels: None

Issue Links:
Reference
relates to DDC-128 Consider adding EntityManager#link/un... Open
is referenced by DDC-80 Allow custom collections Resolved

 Description   

A new fetch mode EXTRA_LAZY for one-to-many and many-to-many associations could have the following effects on PersistentCollection methods:

  • count() : Does not initialize the collection but issues a straight SQL count query.
  • remove($key) : Does not initialize the collection but issues straight SQL instead.
  • removeElement($element) : Does not initialize the collection but issues straight SQL instead.
  • contains($element) : Does not initialize the collection but issues straight SQL instead.
  • containsKey($key) : Does not initialize the collection but issues straight SQL instead.

This mode would usually be useful for (potentially) large collections.

We need to work out concrete use-case examples and implementation proposals before implementation.

The semantics of the mentioned methods with EXTRA_LAZY need to be carefully worked out, i.e. what happens to already managed instances in case of the remove operations and stuff like that.

 Comments   
Comment by Marc Hodgins [ 07/May/10 ]

Wow, this is an excellent idea. I was just thinking how it is unfortunate that there aren't ways to manipulate large collections without resorting to DQL.

If nothing else, having access to a count() would be particularly great.

Comment by Roman S. Borschel [ 07/May/10 ]

@Marc: You might then also be interested in a related ticket: http://www.doctrine-project.org/jira/browse/DDC-547

This would basically allow you to craft special implementations that inherit from PersistentCollection that are "optimized" in some way for a specific association (i.e. write your own custom SQL query when count() is invoked). Your wrapper class is then used by Doctrine instead of the default one, thats the idea. Of course implementing such a custom collection and overriding methods needs to be done carefully to fully preserve the public API contract (PersistentCollection wrappers are supposed to be "invisible" to the user after all). So this would be an advanced feature. EXTRA_LAZY I think is a pretty easy feature, accessible for all users with no further knowledge required. It just means that the collection delays initialization even more, whereever possible, which is a good thing for large collections which you normally don't really want to load, yet it allows you to use the normal OO API of your domain model without resorting to special (DQL) queries.

Comment by Roman S. Borschel [ 07/May/10 ]

Oh, I just saw you're watching that one already sorry for the noise

Comment by Roman S. Borschel [ 19/May/10 ]

Rescheduling for beta3 as we're running out of time.

Comment by Benjamin Eberlei [ 30/Jun/10 ]

Additional features that have been requested by users:

  • Fetch entries in this collection in batches, i.e. only in 20 steps, issue a limit 0,21 and check if there is more

I don't know about the remove() and removeElement() operations, would they register with the UoW or directly execute the DELETE or UPDATE stements themselves?

Comment by Roman S. Borschel [ 07/Jul/10 ]

Pushing back to beta4.

Comment by Roman S. Borschel [ 12/Jul/10 ]

Moved to 2.1 due to lack of time for larger new stuff for 2.0.

Comment by Benjamin Eberlei [ 04/Dec/10 ]

https://github.com/doctrine/doctrine2/tree/DDC-546 first prototype implementation with tests.

Missing:

  • XML and YAML Mapping support.
  • Contains() support.
  • RemoveElement() support

Necessary for later scheduling (only make sense when persisting keys):

  • ContainsKey() support
  • RemoveKey() support
Comment by Benjamin Eberlei [ 29/Dec/10 ]

I updated the branch to include XML, YAML support, refactored a little bit and added contains() support.

The RemoveElement() support should be put into its own ticket that relates to the EntityManager#link() / EntityManager#unlink() functionality.

Comment by Benjamin Eberlei [ 02/Jan/11 ]

Implemented

Comment by Benjamin Eberlei [ 02/Jan/11 ]

Usage would be inside a @OneToMany or @ManyToMany definition set

Annotations/XML:

fetch="EXTRA_LAZY"

YAML:

fetch: EXTRA_LAZY
Generated at Wed May 22 08:02:03 UTC 2013 using JIRA 5.2.7#850-sha1:b2af0c8dc8537b36121c6a579fabbdf79fc919e5.