Doctrine 2 - ORM

New fetch mode EXTRA_LAZY for collections

Details

  • Type: New Feature New Feature
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 2.0-BETA1
  • Fix Version/s: 2.1
  • Component/s: ORM
  • Security Level: All
  • Labels:
    None

Description

A new fetch mode EXTRA_LAZY for one-to-many and many-to-many associations could have the following effects on PersistentCollection methods:

  • count() : Does not initialize the collection but issues a straight SQL count query.
  • remove($key) : Does not initialize the collection but issues straight SQL instead.
  • removeElement($element) : Does not initialize the collection but issues straight SQL instead.
  • contains($element) : Does not initialize the collection but issues straight SQL instead.
  • containsKey($key) : Does not initialize the collection but issues straight SQL instead.

This mode would usually be useful for (potentially) large collections.

We need to work out concrete use-case examples and implementation proposals before implementation.

The semantics of the mentioned methods with EXTRA_LAZY need to be carefully worked out, i.e. what happens to already managed instances in case of the remove operations and stuff like that.

Issue Links

is referenced by

Improvement - An improvement or enhancement to an existing feature or task. DDC-80 Allow custom collections

  • Minor - Minor loss of function, or other problem where easy workaround is present.
  • Resolved - A resolution has been taken, and it is awaiting verification by reporter. From here issues are either reopened, or are closed.
relates to

New Feature - A new feature of the product, which has yet to be developed. DDC-128 Consider adding EntityManager#link/unlink methods for direct association manipulation

  • Major - Major loss of function.
  • Open - The issue is open and ready for the assignee to start work on it.

Activity

Descending order - Click to sort in ascending order
Hide
Permalink
Benjamin Eberlei added a comment -

Usage would be inside a @OneToMany or @ManyToMany definition set

Annotations/XML:

fetch="EXTRA_LAZY"

YAML:

fetch: EXTRA_LAZY
Show
Benjamin Eberlei added a comment - Usage would be inside a @OneToMany or @ManyToMany definition set Annotations/XML: fetch= "EXTRA_LAZY" YAML: fetch: EXTRA_LAZY
Hide
Permalink
Benjamin Eberlei added a comment -

Implemented

Show
Benjamin Eberlei added a comment - Implemented
Hide
Permalink
Benjamin Eberlei added a comment -

I updated the branch to include XML, YAML support, refactored a little bit and added contains() support.

The RemoveElement() support should be put into its own ticket that relates to the EntityManager#link() / EntityManager#unlink() functionality.

Show
Benjamin Eberlei added a comment - I updated the branch to include XML, YAML support, refactored a little bit and added contains() support. The RemoveElement() support should be put into its own ticket that relates to the EntityManager#link() / EntityManager#unlink() functionality.
Hide
Permalink
Benjamin Eberlei added a comment -

https://github.com/doctrine/doctrine2/tree/DDC-546 first prototype implementation with tests.

Missing:

  • XML and YAML Mapping support.
  • Contains() support.
  • RemoveElement() support

Necessary for later scheduling (only make sense when persisting keys):

  • ContainsKey() support
  • RemoveKey() support
Show
Benjamin Eberlei added a comment - https://github.com/doctrine/doctrine2/tree/DDC-546 first prototype implementation with tests. Missing: XML and YAML Mapping support. Contains() support. RemoveElement() support Necessary for later scheduling (only make sense when persisting keys): ContainsKey() support RemoveKey() support
Hide
Permalink
Roman S. Borschel added a comment -

Moved to 2.1 due to lack of time for larger new stuff for 2.0.

Show
Roman S. Borschel added a comment - Moved to 2.1 due to lack of time for larger new stuff for 2.0.
Hide
Permalink
Roman S. Borschel added a comment -

Pushing back to beta4.

Show
Roman S. Borschel added a comment - Pushing back to beta4.
Hide
Permalink
Benjamin Eberlei added a comment -

Additional features that have been requested by users:

  • Fetch entries in this collection in batches, i.e. only in 20 steps, issue a limit 0,21 and check if there is more

I don't know about the remove() and removeElement() operations, would they register with the UoW or directly execute the DELETE or UPDATE stements themselves?

Show
Benjamin Eberlei added a comment - Additional features that have been requested by users: Fetch entries in this collection in batches, i.e. only in 20 steps, issue a limit 0,21 and check if there is more I don't know about the remove() and removeElement() operations, would they register with the UoW or directly execute the DELETE or UPDATE stements themselves?
Hide
Permalink
Roman S. Borschel added a comment -

Rescheduling for beta3 as we're running out of time.

Show
Roman S. Borschel added a comment - Rescheduling for beta3 as we're running out of time.
Hide
Permalink
Roman S. Borschel added a comment -

Oh, I just saw you're watching that one already sorry for the noise

Show
Roman S. Borschel added a comment - Oh, I just saw you're watching that one already sorry for the noise
Hide
Permalink
Roman S. Borschel added a comment -

@Marc: You might then also be interested in a related ticket: http://www.doctrine-project.org/jira/browse/DDC-547

This would basically allow you to craft special implementations that inherit from PersistentCollection that are "optimized" in some way for a specific association (i.e. write your own custom SQL query when count() is invoked). Your wrapper class is then used by Doctrine instead of the default one, thats the idea. Of course implementing such a custom collection and overriding methods needs to be done carefully to fully preserve the public API contract (PersistentCollection wrappers are supposed to be "invisible" to the user after all). So this would be an advanced feature. EXTRA_LAZY I think is a pretty easy feature, accessible for all users with no further knowledge required. It just means that the collection delays initialization even more, whereever possible, which is a good thing for large collections which you normally don't really want to load, yet it allows you to use the normal OO API of your domain model without resorting to special (DQL) queries.

Show
Roman S. Borschel added a comment - @Marc: You might then also be interested in a related ticket: http://www.doctrine-project.org/jira/browse/DDC-547 This would basically allow you to craft special implementations that inherit from PersistentCollection that are "optimized" in some way for a specific association (i.e. write your own custom SQL query when count() is invoked). Your wrapper class is then used by Doctrine instead of the default one, thats the idea. Of course implementing such a custom collection and overriding methods needs to be done carefully to fully preserve the public API contract (PersistentCollection wrappers are supposed to be "invisible" to the user after all). So this would be an advanced feature. EXTRA_LAZY I think is a pretty easy feature, accessible for all users with no further knowledge required. It just means that the collection delays initialization even more, whereever possible, which is a good thing for large collections which you normally don't really want to load, yet it allows you to use the normal OO API of your domain model without resorting to special (DQL) queries.
Hide
Permalink
Marc Hodgins added a comment -

Wow, this is an excellent idea. I was just thinking how it is unfortunate that there aren't ways to manipulate large collections without resorting to DQL.

If nothing else, having access to a count() would be particularly great.

Show
Marc Hodgins added a comment - Wow, this is an excellent idea. I was just thinking how it is unfortunate that there aren't ways to manipulate large collections without resorting to DQL. If nothing else, having access to a count() would be particularly great.

People

  • Assignee:
    Benjamin Eberlei
    Reporter:
    Roman S. Borschel
Vote (3)
Watch (6)

Dates

  • Created:
    Updated:
    Resolved: