Linux, PHP 5.3.10, Symfony 2.2

"many-to-many TO a mapped superclass"? I don't think relations TO mapped superclasses are supported in any way... A mapped superclass should never appear in a `targetEntity` mapping.

Could you please re-formulate the description of the issue? There is no mapped superclass in your examples

You are of course right. Edited the issue description/title.

It is a blocker for us, btw., not minor - being able to use a many-to-many relation there is rather central to the code of our app. I'm not seeing a workaround, easy or otherwise.

Moritz Kraft yes, but it is not a blocker for the next release Will restore prio.

Ah right, yeah, that makes sense. Thank you!

Fixed in 1b5f051 - thanks Benjamin!

Fixed in 2.3 branch, backported to 2.2 as well

PHP 5.3.6

Can you profile where exactly the extra queries are executed using xdebug_start_trace? Directly during hydration or later in your code?

xdebug trace start just before I query the database

Hi Pascal, sorry but this is not enough. I need this query until all the other entities (or at least one) are n+1 joined.

Here the complete trace, started just before the first query. I'm sorry but I can't make heads or tails with that... I hope it can help you pinpoint the problem.

Please also note that I have update Doctrine ORM to 2.1.5 and there was a change in the number of "superfluous" queries done. I am now only left with additional queries to get the relations to "parent", cf the model ( http://pastie.textmate.org/private/przxzfimsfyua02cxqcv9a ).

Is the trace from before upgrade to 2.1.5 or after?

The "complete" trace, i.e. the one that is 5.94 mb big is from after the upgrade to 2.1.5.

Can you disable the nested set extension? the other ticket uses it too and i want to rule out that its the extensions fault.

Alright. I deactivated the Tree extension, but there are no changes in the number of queries.

I found the issue.

Fixed

Merged into master

A related Github Pull-Request [GH-169] was opened
https://github.com/doctrine/dbal/pull/169

A related Github Pull-Request [GH-169] was closed
https://github.com/doctrine/dbal/pull/169

Merged

A related Github Pull-Request [GH-160] was opened
https://github.com/doctrine/dbal/pull/160

A related Github Pull-Request [GH-160] was closed
https://github.com/doctrine/dbal/pull/160

implemented

A related Github Pull-Request [GH-158] was opened
https://github.com/doctrine/dbal/pull/158

I agree it seems very complex to implement for the Object and ArrayHydrator, however it should be very easy for the getScalarResult() hydrator.

Even more I think there is a general bug with INDEX BY and scalars in Object and ArrayHydrator that has to be evaluated.

This PR will fix this issue https://github.com/doctrine/doctrine2/pull/151, it is scheduled for 2.2

This is fixed in master

This behavior has not changed in the latest 2.2.1. getArrayResult returns an indexed result. getScalarResult returns zero based result. Do I need to create a separate ticket for that issue?

You can only fetch entities using the normalized value of any field, that means you have to find a common transformation that is saved in your entities and then converted on the fly.

There is a default converting method from database to PHP and back you could use, however this is only in one direction. You could define a class for the Point and convert from and to this Call in your Type. This way you could encapsualte all the converting logic in an object and your entities only use the object.

You can extend DQL to have additional queries, however when you select additional fields in DQL and apply a function to them, they are retrieved as scalar and not inside your entity. How to do this is detailed in the manual and the cookbook.

Closed, there are already extension points that help with this matter-

Hi Benjamin.

Thanks for your response.

I don't fully understand what you mean. As far as I can tell this is a specific use case that is not supported by doctrine.

What I have tried is creating an entity that contains a field named $point. This is of my custom type PointType which extends Doctrine\Dbal\Types\Type

In this method I intended to create my own Geometry\Point object by parsing a human readable value returned from the database and seamlessly converting to and from this value in convertToDatabaseValue($value, AbstractPlatform $platform) and convertToPHPValue($value, AbstractPlatform $platform)

The problem is that the column is in binary format and I need the database to decode it for me.

I also understand about extending DQL to use custom functions. If I went down this route, every time I fetch my entities, I would have to do an extra step to parse the scalar. Suddenly my domain logic is not encapsulated within my entity and I have a feeling this could end up being messy.

There is also another possibility as far as I can tell, using a NativeQuery, however this would mean that every time I fetch an entity I would have to do field mapping, and any time I didn't map the field to a function my entity would fail to decode the point and break, this route would also mean a lot of extra coding and care.

I am happy to accept that this use case may not be useful to a large portion of doctrine users and therefore may not be a justified addition to doctrine. I do however believe that this is a feature that would be nice to have, and others will miss it in the future, particularly in postgis integration.

Doctrine already has custom DQL and custom types. As a new user my opinion is that there is definitely some room for some advanced functionality in this area. However, you guys are the core devs, you understand the project and its internals so if this is a feature that should be left out then so be it. (I may just have to look into my own hack).

Kind regards

Re-opened, although this might possibly be a better ORM than DBAL extension. We have to re-evaluate this for 2.1

DBAL-53 highlightes a possible problem, how to handle foreign keys in this regard?

Idea: For each type, wrap the column in a possible conversion snippet:

class Type
{
    public function convertToDatabaseValueSQL($sqlExpr, $platform)
    {
        return $sqlExpr;
    }

    public function convertToPHPValueSQL($sqlExpr, $platform)
    {
        return $sqlExpr;
    }
}

Example:

class IPAddress extends Type
{
    public function convertToDatabaseValueSQL($sqlExpr, $platform)
    {
        return "INET_ATON($sqlExpr)";
    }

    public function convertToPHPValueSQL($sqlExpr, $platform)
    {
        return "INET_PTON($sqlExpr)";
    }
}

Conversion inside the ORM would take place at the following locations:

1. When generating SELECT SQL inside the persister, use the conversion in the SELECT clause.
2. When generating SELECT SQL from a DQL, use conversion in the SELECT clause.
3. When updating an entity inside the persister use this in the SET clause and WHERE clause for the ID.
4. When inserting an entity inside the persister use this in the VALUES clause.

Open questions:

1. Should conversion also be done for occurances in any other clauses other than SELECT?
2. Can we bear the performance impact of this on SQL generation.

Acttually thinking about it. It has to be everywhere when 3 is necessary (and DELETE WHERE also needs this) then of course the values should be converted for SELECT clauses WHERE also.

One Exception might be JOIN clauses WHERE its better to have:

a.foo = b.foo

vs

FUNC(a.foo) = FUNC(b.foo)

I think we can optimize the performance of this considerably in the ORM by adding a field option "requiresSQLConversion" that is only set if a boolean method "requiresSQLConversion()" on the type returns true. This method gets a default implementation in abstract type of:

public function requiresSQLConversion()
{
  return false;
}

Every complex type would need to overwrite this method.

Then inside ORM we would only need to check if (isset($field['requiresSQLConversion'])) to determine if we have to modify the SQL or not. Also caching structure would only grow by this option if the variable is actually true.

As DBAL-53 has been closed as a duplicate of this one, I'm attaching our patches to make type conversion work

Thanks Lars, this is something else what i would implement though.

This ticket gets high priority for 2.2, i will implement and check in directly after 2.1 will be released end of next month.

Added a missing type hint in DBAL\Types\Type.php

Will this feature be implemented soon? convertToDatabaseValueSQL() and convertToPHPValueSQL() are not used anywhere yet...

Implemented.

What about the missing type hint on convertToPHPValueSQL() ?

Is there a reason for the inconsistency in these method declarations:

public function convertToDatabaseValueSQL($sqlExpr, AbstractPlatform $platform)
public function convertToPHPValueSQL($sqlExpr, $platform)

I think this issue might be resolved in 2.2 since the cache layer seems to have been rewritten.

To address this issue we require the rewrite that was done in 2.2.
Closing as fixed.

Fixed.

MAC OSX 10.7

This issue is referenced in Github Pull-Request GH-230
https://github.com/doctrine/doctrine2/pull/230

Related Pull Request was closed: https://github.com/doctrine/doctrine2/pull/230

This issue is referenced in Github Pull-Request GH-231
https://github.com/doctrine/doctrine2/pull/231

Related Pull Request was closed: https://github.com/doctrine/doctrine2/pull/231

Fixed

I do miss this patch in the recently checked out 2.2 branch. Is there any reason, it was skipped, even if the fix-version here is still 2.2?
I really miss this feature and was unable to get it to work by applying it manually. The parser seems to have troubles applying my parameters :\

Hi Markus

The bit comparisons was been added as DQL functions,
Can you explain your problem ?

Hi Fabio, thank you for the fast reply and the great Doctrine anyway!

My problem is the following:
I have an @Entity called Picture with some columns and the following

	/** @Column(type = "integer", nullable = false)
	 * @var int */
	private $visibility;

Now I try to fetch entities from the database that satisfy some special bit mask in this visibility attribute by doing the following:

	/** @var $query \Doctrine\ORM\Query */
	$query = $this->entityManager->createQuery('SELECT e FROM Classes\Models\Event e JOIN e.pictures p WHERE p.visibility & :visibility = :visibility ORDER BY p.uploadTime DESC');
	$query->setParameter('visibility', \Classes\Session::instance()->getActualUsersRights());
	return $query->getResult();

The error reported is:

Fatal error:  Uncaught exception 'Doctrine\ORM\Query\QueryException' with message '[Syntax Error] line 0, col 78: Error: Expected end of string, got '='' in ...\Doctrine\ORM\Query\QueryException.php:42
Stack trace:
#0 ...\Doctrine\ORM\Query\Parser.php(380): Doctrine\ORM\Query\QueryException::syntaxError('line 0, col 78:...')
#1 ...\Doctrine\ORM\Query\Parser.php(745): Doctrine\ORM\Query\Parser->syntaxError('end of string')
#2 ...\Doctrine\ORM\Query\Parser.php(213): Doctrine\ORM\Query\Parser->QueryLanguage()
#3 ...\Doctrine\ORM\Query\Parser.php(288): Doctrine\ORM\Query\Parser->getAST()
#4 ...\Doctrine\ORM\Query.php(231): Doctrine\ORM\Query\Parser->parse()
#5 ...\Doctrine\ORM\Query.php(242): Doctrine\ORM\Query->_parse()
#6 in <b>...\Doctrine\ORM\Query\QueryException.php</b> on line <b>42</b><br />

Please note: I applied the above patch by hand to the Doctrine 2.2 code. Before that I got an "expected = got '&'" exception in the query.

Am I wrong, if I look through the code and try to find the suggested lines from Ronald there? They aren't in the downloadable package and even not in the SVN, I checked out this morning from http://svn.github.com/doctrine/doctrine2.git

Hi Markus

The "&" and "|" is not available for database compatibilities
You should use the DQL function BIT_AND or BIT_OR

WHERE BIT_AND( p.visibility, :visibility )

Thanks a lot Fabio,

stupid me, sorry.

For completeness, I now use:

SELECT e FROM Classes\Models\Event e JOIN e.pictures p WHERE BIT_AND(p.visibility, :visibility) = :visibility ORDER BY p.uploadTime DESC

and seem to get past the critical part.

Now I can go on trying to get the project back to work with the new Doctrine 2.2
I know its worth the work

Fixed

fixed

Hm github is still saying that this is not in 2.2 branch....

Cherrypick

ah no, now its up. thanks

Fixed

Fixed

Merged in https://github.com/doctrine/doctrine2/commit/c1012f79706491e9dbf6c8741225b62833c6d2ce

done

This error depends on the caching layer. Without APC/Memcache this test works for me, not with cache. But it only happens on PGSQL not on MySQL or SQLITE

It specifically depends on the Memcache driver, test works with APC

Fixed

Fixed

Here the PR for the ORM SchemaTool : https://github.com/doctrine/doctrine2/pull/261
Here the PR for the DBAL Table : https://github.com/doctrine/dbal/pull/94

Merged https://github.com/doctrine/doctrine2/commit/2bb511584e5d37ddad6c669a19d8e6b4a20f7b2b

DBAL merged back into 2.2 and 2.1.x

and Merged into 2.2 and 2.1.x ORM

Implemented, using the KNP Code, the PAgerfanta tests and putting a set of functional tests on top.

Merged back into 2.2 aswell as the code can be considered stable and does not affect any other area of the code.

Fixed

Fixed in https://github.com/doctrine/doctrine2/commit/d39760ba49418f42f235e9befe223896f11de4bd

Merged back to 2.2

Can you explain why you think this is necessary?

You are right an id method with logic could exist in 4 lines of code, but for what purpose?

First of all it is a question of concept. Doctrine shouldn't assume anything about entities outside of their fields. It already introduces a level of complication when we explain how to clone/serialize objects, which was very confusing.

Asking for the purpose of an identifier field getter method is in my opinion again an attempt of making assumptions over user's code...

What if the user wrote something like:

public function getIdentifierField1()
{
return $this->identifierField1? $this->_myInitializationStuff() : null;
}

private function _myInitializationStuff()
{
//open files, check stuff, make things difficult for the D2 team
}

For instance, opening file handlers, sockets, whatever... This is a case that would break the entity because of optimization introduced by the ProxyFactory. (not to mention when getIdentifierField1 does some conversion, like return $this->identifierField1 + self::OFFSET_REQUIRED_BY_WEBSERVICE;

I'm not arguing about the validity of this optimization, but that the checks are too lazy.

I've read something about moving the ProxyFactory to common and using code scanner tools, and the check should be about applying the optimization only when the form is

return $this->identifierField1;

That's why I put the example of the regex. That would probably not be as safe as using some reflection-based tool, but surely more than just checking if the code is <= 4 lines...

Also don't know what stuff like EAccelerator (known in this Jira as of it's fantastic idea about stripping comments) would make the check of the 4 lines like.

This argument is void i just seen

A method:

public function getIdentifierField1()
{
   return $this->identifierField1? $this->_myInitializationStuff() : null;
} 

Will only used when the id is not set anyways.

In any other case Ids are Immutable and changing them in your code would break a lot more than just this smart proxy method generation.

Nope, this code actually works only if the ID is set
I'm not talking about changing IDs, it's just that getters and setters don't always reflect the class attributes...

As of IRC there's 3 ways (for now) to get this solved:

• Some code scanner/stronger checks (difficult/problems with private methods/slow)
• Regex (as of description)
• Adding configuration (per-entity or per-method. Probably too messy)
• Documenting it as "magic" of proxies and let the users be aware of it

Windows 7, Apache, PHP 5.3.8, Zend Framework

Fixed and merged into 2.2 and 2.1

Duplicated

This is not a duplicate, its a runtime issue in the code when setting the wrong target entity type.

Implemented an instanceof check.

Fixed

WindowsXP/Windows2008 R2 / PHP 5.3 / MsSQL-Server 2005 / MsSQL-PDO_PHP-Driver

Assigned to juozas, but i think the issue here is that you have to use Datetime2, or add your own type replacing the shipped one to support the old datetime.

Ok, I had another problem with the datetime, but this does not regard the problem of this issue ( at least not totally).
The problem with the MsSQL-Server before 2008 is that there is no data type named 'datetime2', just 'datetime'.
The next problem is that every date conversion for a query is done in the language set upon conection time.
Thus leads to a problem, when it is not possible to set the connection language.

So the problem is that the MsSQL-Server relies on the settings above.
In my case the datetime conversion failed, as the server always thought that the datetime-string would come in
the following format: Y-d-m . This is not true, as the default format string is: Y-m-d . So every insert/update query fails.
To solve the problem I did that: $entityManager->getConnection()->exec('SET DATEFORMAT ymd'); .
This way I ensured that the dateformat string works fine, except the issue problem.

To solve the problem in general, it would be helpful to subclass the MsSqlPlatform into a class named MsSql2005Platform or something like this and just override the getDateTimeFormatString and upon connection setting the format for the queries
as mentioned before.

Hope this helps out.
Besides, here a link to the datetime problem (in german): http://www.insidesql.org/blogs/frankkalis/2010/08/19/der-ultimative-guide-fuer-die-datetime-datentypen

I have somehow manage to miss the fact that datetime2 wasn't around in datetime... What I'm thinking now is there a need for datetime2 in Doctrine at all. If only thing it brings is additional accuracy for microseconds, maybe best idea would be to use datetime for 2008 installs too if used from Doctrine. However datetime is now a standard and Microsoft recommends to use it for new installs. What I can do is I can always insert 3 fractional points to datetime column as both datetime2 and datetime would accept it as valid date string.

We can have separate platforms for 2008 and 2005 servers, but that would be quite resource intensive. Let me see what is the best way to fix it.

Regards to Dateformat, I guess the solution would be to set format on connection, how you suggested. How about you create a separate ticket for this and assign it to me.

Oracle also has an Session Init Listener that handles the date format things, i guess we can take this as example. However I think having Mssql2005Platform sounds goods also, it would be only one method to override.

To solve this issue, at least for MsSQL-Server datetime data types, change the following TypeClass of Doctrine by adding
this check before converting to PHP\DateTime in 'convertToPHPValue()' :

if( strlen($value) == 24 && $platform->getDateTimeFormatString() == 'Y-m-d H:i:s.u')
$value = $value.'000';

I know this is propably very specific, but I do not know, how other DBs handle microseconds in datetime strings.

I fixed this bug with some changes in the DateTimeType class. As there is no Explicit MSSQL2005 Plattform this change would also affect datetime2 type in the SQLServer 2008 plattform, which is the data type that has 6 microseconds.
So either populate a MSSQLServer2005 Plattform, or introduce a new DateTimeType for the 2005 platform.

<?php
/*
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
 * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 *
 * This software consists of voluntary contributions made by many individuals
 * and is licensed under the LGPL. For more information, see
 * <http://www.doctrine-project.org>.
 */

namespace Doctrine\DBAL\Types;

use Doctrine\DBAL\Platforms\AbstractPlatform;

/**
 * Type that maps an SQL DATETIME/TIMESTAMP to a PHP DateTime object.
 *
 * @since 2.0
 */
class DateTimeType extends Type
{
    public function getName()
    {
        return Type::DATETIME;
    }

    public function getSQLDeclaration(array $fieldDeclaration, AbstractPlatform $platform)
    {
        return $platform->getDateTimeTypeDeclarationSQL($fieldDeclaration);
    }

    public function convertToDatabaseValue($value, AbstractPlatform $platform)
    {
		if( $value === null)
			return null;

		$value = $value->format($platform->getDateTimeFormatString());

		if( strlen($value) == 26 &&
			$platform->getDateTimeFormatString() == 'Y-m-d H:i:s.u' &&
			$platform instanceof \Doctrine\DBAL\Platforms\MsSqlPlatform )
			$value = substr($value, 0, \strlen($value)-3);

		return $value;

    }

    public function convertToPHPValue($value, AbstractPlatform $platform)
    {
        if ($value === null) {
            return null;
        }

		if( strlen($value) == 24 && $platform->getDateTimeFormatString() == 'Y-m-d H:i:s.u')
			$value = $value.'000';

        $val = \DateTime::createFromFormat($platform->getDateTimeFormatString(),$value);
        if (!$val) {
            throw ConversionException::conversionFailedFormat($value, $this->getName(), $platform->getDateTimeFormatString());
        }
        return $val;
    }
}

Added SQLServer2005 platform that uses DATETIME and the .000 format as per instructions of Martin.

Weird, this code looks very strange. I will check and fix that before the 2.2 RC release.

The default result cache is now passed to any new query cache profile. The following API is now working as expected:

$query->useResultCache(true);
$cache = $query->getQueryCacheProfile()->getResultCacheDriver();

That's perfect! Thank you!

Fixed

Linux, PostgreSQL

By now, it's currently impossible to do cross database joins in an easy way.
We have planned it through the schema support in @Table.

I'll be working on this support for 2.2, but until there, my only recommendation is to not use a keyword as a table name.

Cheers,

It's not cross database; it's just schema.table in PostgreSQL. It works when not using reserved words, but fails with them.

How much of this ticket does your commit from some weeks ago cover guilherme?

Fixed for 2.2

It still doesn't work. Tried on master.

what kind of error do you get?

could reproduce the problem with table names and sequence names.

Fixed.

I've just got fresh clones of doctrine-doctrine2, doctrine-dbal and doctrine-common — all on their master branches.

My entity has this annotation:

@ORM\Table(name="`user`.`user`")

I'm getting the following exception:

SQLSTATE[42P01]: Undefined table: 7 ERROR: relation "user.user" 
does not exist LINE 1: ... is_private9,
u0_.created_at AS created_at10 FROM "user.user... 

So, it's translating `user`.`user` to "user.user", while it should be "user"."user".

What changed is that earlier `user`.`user` was translated to "user`.`user", while now it's translated to "user.user", but should be to "user"."user".

I saw the commit related to this ticket (https://github.com/doctrine/doctrine2/commit/959a68694e5827a74ad5b8c8216996ffff6196ba) and I want to note that I'm getting this error not when creating a table, but when querying it. I don't use Doctrine for creating tables anyway — I do it via plain SQL.

I keep trying until i fixed this, now with a commit into DBAL: https://github.com/doctrine/dbal/commit/4cb8a99b65fb08e01fbc02cf9c0e07255e3f8463

Will be included in Beta 2

Now `user`.user, `user`.`user` and `user.user` get escaped properly, ending up as "user"."user". Interestingly enough, even only the beginning tick like in `user.user ends up as "user"."user".

user.`user`, though, results in user.`user` — that is, nothing gets escaped. I don't think this is a big deal, because if I can escape both schema and table name parts, then I won't run into problems. I'm pointing this out in case it was intended to affect the escaping too.

Good job, thanks.

Would be great to get this functionality

Assigned to asm89

Scheduled for 2.2

I've been working on this ticket over here:
https://github.com/asm89/doctrine2/tree/DDC-551

Latest thing I added was the state of the collection of filters, because this is needed for parsing (and sometimes not parsing) the queries to generate SQL. I'd like some feedback about the state keeping. More information at the commit:
https://github.com/asm89/doctrine2/commit/2653d735e230d19b0fcb703800008f3abdb7a3ba

At this point the EntityManager keeps track of this state, but maybe it would be nice to have a separate FilterCollection keep track of the state/hashes etc?

This issue is referenced in Github Pull-Request GH-224
https://github.com/doctrine/doctrine2/pull/224

Related Pull Request was closed: https://github.com/doctrine/doctrine2/pull/210

Related Pull Request was closed: https://github.com/doctrine/doctrine2/pull/224

Implemented

This issue is referenced in Github Pull-Request GH-225
https://github.com/doctrine/doctrine2/pull/225

Related Pull Request was closed: https://github.com/doctrine/doctrine2/pull/225

This issue is referenced in Github Pull-Request GH-227
https://github.com/doctrine/doctrine2/pull/227

Related Pull Request was closed: https://github.com/doctrine/doctrine2/pull/227

Alex mentioned on IRC that filters do not provide the functionality that the OP requires, so this issue should really re-opened, unless I'm missing something?

This issue is referenced in Github Pull-Request GH-237
https://github.com/doctrine/doctrine2/pull/237

This issue is referenced in Github Pull-Request GH-233
https://github.com/doctrine/doctrine2/pull/233

Fixed in https://github.com/doctrine/doctrine2/commit/4bc014c6961be2e24b131d06d25c885fdf07bc3b

Related Pull Request was closed: https://github.com/doctrine/doctrine2/pull/233

ubuntu php 5.3.8

I have created a test.

the bug may be in \Doctrine\ORM\Internal\Hydration\AbstractHydrator at line 246.

i think...

remove
if ( ! isset($rowData[$dqlAlias][$cache[$key]['fieldName']]) || $value !== null) {

and replace with :

if ( ! isset($rowData[$dqlAlias][$cache[$key]['fieldName']]) && $value !== null) {

this will fix the bug

Fixed since this commit: https://github.com/doctrine/doctrine2/commit/68663fac4b5619643b8a140d6bbe7031e4fa2ad5

Thanks a lot for the report and patch! =D

Update fix version, this was merged into 2.1.5

See here for the test case: https://github.com/bschussek/doctrine-orm/tree/DDC-1545

Increasing priority

Issue fixed since this commit:

https://github.com/doctrine/doctrine2/commit/a8478d5766e2cc4185612af680b7f6bcd83af61e

Thanks a lot for the bug reporting. It was the second time someone reported me this issue and I could not reproduce! =)

This issue is referenced in Github Pull-Request GH-229
https://github.com/doctrine/doctrine2/pull/229

Related Pull Request was closed: https://github.com/doctrine/doctrine2/pull/229

Fixed properly now. Will be merged into 2.1.x

Update fix version, this was merged into 2.1.5

It also avoids unnecessary "proxy not initialized, can't serialize" exceptions.

If we make this change, the caching won't be "as" affective right since it will still have to go through the hydration process?

Correct, but its almost impossible to result cache entities because they are not allowed to have a single proxy entity at any depth of the object graph.

Maybe we should introduce another notion of SqlResultCache? The ResultCache is good and effective for all Array and Scalar hydration modes, however the SQL Result caching is more useful for Object Hydration.

Exactly. Yes it means hydration is always run but you really cant speak of "effective" when caching large object result sets because serialization & especially unserialization which then happens on every request will be pretty slow.

As Benjamin said, result caching is perfect for anything but objects so this ticket really only affects queries that use object hydration. Storing the raw sql result for these has the advantage that the objects retrieved from the cache are not detached and thus need not be merged and also you dont have issues with uninitialized proxies preventing serialization.

Is this also the cause of this issue? http://www.doctrine-project.org/jira/browse/DDC-446

Since we're unserializing some objects, does Doctrine know about them?

Yes, as already said, objects coming from the result cache are currently DETACHED. Thus the EM does not care about them. This is fine if you just want to display them but if you want to modify them and persist new changes, you need to merge() the ones you want to become MANAGED again into the EM.

It should also be noticed that if we make this change and the cached objects are managed after grabbing from the cache this can have a negative effect on flushing performance, if you only want the cached objects for read-only/display purposes. Maybe we should still provide both options, like Benjamin said.

Moved to 2.1 for now.

I would like to see the ability to cache pre-hydration as I have already run into this issue.

Maybe the interface could be along similar syntaxes as turning on result caches

$query->useResultCache(true)
      ->setResultCacheLifetime(3600)
      ->setResultCachePreHydration(true); //Result cache should work pre-hydration, false caches 

However from my examinations the hydrators will have to be rewritten to accept an array instead of a pdo statement.

That would be my vote. If you don't have the resources to work on the ticket I can tackle it.

rewriting to use an array is not an option. this would require to call ->fetchAll() which is not wanted for the Iterate Result feature.

There has to be a better way to redefine the method signatures to allow both use-cases.

Ah btw, any help is appreciated. Just Fork doctrine2 over at Github and develop this feature in a new branch (git checkout -b DDC-217) and link it here or issue a pull request on Github.

Forked here: https://github.com/dcousineau/doctrine2 though probably should pull it out into a local branch.

I went ahead and went down that route of altering hydrators to accept an array as well as a statement. This way uncached queries will work with the statement as it used to, but can also hydrate from an array.

As for the iterable hydrator you have a point but in this case you really can't cache it period, be it pre or post hydration (since to cache we have to serialize ALL the information).

What I have up is working(ish with the caveat that I backported the changes to an earlier tagged release for our application and tested using that), take a look and see if we like or don't like how I did it.

You'll notice I did split up _doExecute() for the query objects, because the DQL query object does so much analysis in the _doExecute() method which produces and attaches information (namely the result set mapping) that is depended on by the hydration process, I split it up into a "doPreExecute" function so that it can be called separately without actually executing the query.

Yes i know that the serialize thing won't work with the Iterator. My point was that the Iterator USES the Hydrator code and it should still support that use-case.

See the IterableResult class and make sure that its still working after your refactoring.

I don't like the instanceof check then iterate vs the forach over the array. I would rather find a solutio nthat works for both, but i cant find one now that doesnt involve findAll() (which is bad).

I'll double check against the IterableResult and make sure it's compatible.

Beyond that, I don't know what I could other than the instance of check. I could extend the statement object and create a statement that takes a flat array so all the code remains the same (i'm passing a statement).

Assigning Guilherme

I've made my own implementation for this:
https://github.com/sobstel/doctrine2/commit/f583625f791da22dec77dd7d6c83b813f0bcbdaa

There are no changes in specific hydrators and there is no fetchAll(). I've added 2 wrappers: one for cached result array, and one for uncached pdo statement. The former (CachedResultStatement) is actually ArrayIterator which just implements used methods (fetch & closeCursor => also specified in ResultStatementInterface). The latter is decorator for stmt, which just collects results on each call of fetch() (results are later available by calling getResult), and apart from this all calls are delegated to actual stmt object.

I've used (copied) following changes from Daniel https://github.com/dcousineau/doctrine2/commit/c23b5a902da79ffc472c0769b9258775baf3189e

Btw, what is your reasoning for not using fetchAll() anywhere?

maybe a dumb question but is it normal that result cache fetched entities associations are set to blank objects?

I've sent a pull request for the fix(https://github.com/bogdanalbei/doctrine2/commit/00d1aa2192b86e51872fbc833b2436354f6fe162) . The changes were tested under live conditions sice v2.0.

Workaround for this issue: https://gist.github.com/57e6b4deea566baac053

Also: This will be fixed in 2.2.

First step of this hit DBAL today https://github.com/doctrine/dbal/pull/69

DBAL code was heavily refactored and merged, the ORM PR is now open for discussion:

https://github.com/doctrine/doctrine2/pull/172

Merged into master

Related Pull Request was closed: https://github.com/doctrine/doctrine2/pull/87

Linux Debian Squeeze, MySQL

Keep it coming, very valuable

Fixed, but referenced with wrong ticket in master right now.

This issue is referenced in Github Pull-Request GH-219
https://github.com/doctrine/doctrine2/pull/219

Related Pull Request was closed: https://github.com/doctrine/doctrine2/pull/219

Fixed in https://github.com/doctrine/doctrine2/commit/f6d9344d892e5e959b4474437258b02777b41eb1

Oracle

This is not allowed currently, there is a PR for custom id generators, but so long i am closing this as invalid.

Also i added an error exception which makes this kind of setup fail.

Added a part on setiing up the CLi into the ORM docs.

Linux, Debian Lenny, PHP 5.3.8

Fixed

This issue is referenced in Github Pull-Request GH-222
https://github.com/doctrine/doctrine2/pull/222

Related Pull Request was closed: https://github.com/doctrine/doctrine2/pull/222

This issue is referenced in Github Pull-Request GH-223
https://github.com/doctrine/doctrine2/pull/223

Related Pull Request was closed: https://github.com/doctrine/doctrine2/pull/223

Fixed.

Why dont you define the named queries as array? just @NamedQuery(

) ?

Also please consider attaching a diff of your patch, not the whole changed class.

Added better validations.

This includes a BC break.
Marking as won't fix.

Merged

Fixed in https://github.com/doctrine/orm-documentation/commit/373090f2232a59b7c4cb35212a1c1d5843bc8be7

MySQL - Ubuntu

Could you please include your Model mappings too?

Hi,

This special situation is not really possible.
You are not allowed to create a cross-reference Entity without defining an own @Id.
The situation you exposed here is due to it. Issue was left open but now after some tests it can be closed. In 2.2-DEV we added a better explanation for your problem, I hope the message is clear enough for you:

Entity of type Doctrine\Tests\ORM\Functional\Ticket\DDC979AgentFile has identity through a foreign entity Doctrine\Tests\ORM\Functional\Ticket\DDC979Agent, however this entity has no identity itself. 
You have to call EntityManager#persist() on the related entity and make sure that an identifier was generated before trying to persist 'Doctrine\Tests\ORM\Functional\Ticket\DDC979AgentFile'. 
In case of Post Insert ID Generation (such as MySQL Auto-Increment or PostgreSQL SERIAL) this means you have to call EntityManager#flush() between both persist operations.

Marking the issue as won't fix.

Added the testcase I have used.

Done with current refactorings

Also the documentation at http://www.doctrine-project.org/docs/orm/2.1/en/ only mentions COALESCE in the EBNF but not in the real documentation part. At first I thought that Doctrine doesn't support coalesce at all.

COALESCE does not exist on 2.1.
COALESCE is a 2.2-DEV, so use latest master.

Marking as invalid, since we won't change 2.1 to support this.

Don't all the private variables have getter and setter methods to modify them?

Btw, can you explain your use-case a little bit more? This would greatly help to find solutions to the issue.

They do have getter and setter methods, but privates don't show up at all in a class that extends the base class. SqlWalker works great if it's not meant to be extended.

Sure, I was using HINT_CUSTOM_OUTPUT_WALKER to point to an extended version of SqlWalker to strip some joins off of a query (in the SqlWalker::walkFromClause() method) that I cloned from the base query I was using. Something like...
SELECT id FROM table
LEFT JOIN table1 table1.table_id = table.id
ORDER BY field1
I wanted just the ids without the joins for the cloned query so the easiest thing I could think of was to override SqlWalker with the HINT_CUSTOM_OUTPUT_WALKER and strip the joins in ::walkFromClause(). I was also thinking about removing the order by but it didn't seem to have much impact on performance, but if I had, it would have been the same way, although I think that one would be removable via a HINT_CUSTOM_TREE_WALKER? I didn't really look into that part. I know my use-case is pretty specific and won't apply to 99.9% of people out there.

There's a good chance I misunderstood how you guys wanted the HINT_CUSTOM_OUTPUT_WALKER to work, but it seemed to me like it would be easiest to use it to extend SqlWalker and that worked great once I changed the privates to protected

@"... and that worked great once I changed the privates to protected"

Until the next refactoring This is heavily "moving" code and I don't think we want to take care of the backwards compatibility issues with making them protected. You were right in assuming that currently SQLWalker is not meant to be extended, implementing TreeWalker directly with a custom walker. I do see that it can be a valid use-case but the problem is that this will cause quite some burden on backwards compatibility because this code tends to change a lot.

Sounds good to me. As always, thanks!!

As previously defined, SqlWalker is not meant to be extended.

If user wants to create a customized output, he must provide his own customOutputWalker entirely implemented.
SqlWalker is a place of heavy implementation, one of the core functionalities ans it tends to restrict a lot our possible fixes because internal API is always changing.

Cheers,

Fixed since https://github.com/doctrine/doctrine2/commit/0febf0611439e05abbb1bf4b4e9668f7ad125c67

Marked as bug

This issue was already fixed, maybe docs should be regenerated.
Marking as fixed for now.

LINUX doctrine2.git