[DDC-1800] Paginator results is wrong if your query use order by clause Created: 27/Apr/12  Updated: 09/Apr/13  Resolved: 29/Aug/12

Status: Resolved
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.2.2
Fix Version/s: 2.2.4, 2.3
Security Level: All

Type: Bug Priority: Major
Reporter: Marc Drolet Assignee: Benjamin Eberlei
Resolution: Fixed Votes: 0
Labels: None
Environment:

linux oracle



 Description   

NOTE: I didn't try this on other database, I'm using Oracle.

if my original fetchJoin query use an order by clause, the results is not keeping the provided order by clause and re-order them by id.

here is my generated query to get the distinct records that get generated:

SELECT distinct ID0
FROM
(
SELECT f0_.id AS ID0, f0_.deal_type_id AS DEAL_TYPE_ID1, f0_.title AS TITLE2, f0_.deal_date AS DEAL_DATE3, f0_.amount AS AMOUNT4,
f0_.abstract AS ABSTRACT5, f0_.created_date AS CREATED_DATE6, f0_.last_updated_date AS LAST_UPDATED_DATE7,
f0_.object_status_id AS OBJECT_STATUS_ID8, f0_.published_date AS PUBLISHED_DATE9, f0_.publishing_status_id AS PUBLISHING_STATUS_ID10,
f1_.id AS ID11, f1_.role_id AS ROLE_ID12, f1_.role_type_id AS ROLE_TYPE_ID13, f2_.id AS ID14, f3_.id AS ID15, f4_.id AS ID16, f5_.id AS ID17,
c6_.id AS ID18, d7_.id AS ID19
FROM fo_deal f0_
INNER JOIN fo_deal_role f1_ ON f0_.id = f1_.deal_id
INNER JOIN fo_people f2_ ON f1_.people_id = f2_.id
INNER JOIN fo_property_deal f3_ ON f0_.id = f3_.deal_id
INNER JOIN fo_property f4_ ON f3_.property_id = f4_.id
LEFT JOIN fo_property_asset f5_ ON f4_.id = f5_.property_id
LEFT JOIN co_asset c6_ ON f5_.asset_id = c6_.id
LEFT JOIN ds_record d7_ ON c6_.ds_id = d7_.id
WHERE f1_.people_id = 2
AND f0_.object_status_id <> 3
AND f0_.publishing_status_id = 2
ORDER BY f0_.deal_date DESC, f0_.published_date DESC
)

running this query I get the id 30, 44 when the inner query return 44, 30

here is the query that should get generated to take care of the order by clause:
SELECT distinct ID0, rownum+
FROM
(
SELECT f0_.id AS ID0, f0_.deal_type_id AS DEAL_TYPE_ID1, f0_.title AS TITLE2, f0_.deal_date AS DEAL_DATE3, f0_.amount AS AMOUNT4,
f0_.abstract AS ABSTRACT5, f0_.created_date AS CREATED_DATE6, f0_.last_updated_date AS LAST_UPDATED_DATE7,
f0_.object_status_id AS OBJECT_STATUS_ID8, f0_.published_date AS PUBLISHED_DATE9, f0_.publishing_status_id AS PUBLISHING_STATUS_ID10,
f1_.id AS ID11, f1_.role_id AS ROLE_ID12, f1_.role_type_id AS ROLE_TYPE_ID13, f2_.id AS ID14, f3_.id AS ID15, f4_.id AS ID16, f5_.id AS ID17,
c6_.id AS ID18, d7_.id AS ID19
FROM fo_deal f0_
INNER JOIN fo_deal_role f1_ ON f0_.id = f1_.deal_id
INNER JOIN fo_people f2_ ON f1_.people_id = f2_.id
INNER JOIN fo_property_deal f3_ ON f0_.id = f3_.deal_id
INNER JOIN fo_property f4_ ON f3_.property_id = f4_.id
LEFT JOIN fo_property_asset f5_ ON f4_.id = f5_.property_id
LEFT JOIN co_asset c6_ ON f5_.asset_id = c6_.id
LEFT JOIN ds_record d7_ ON c6_.ds_id = d7_.id
WHERE f1_.people_id = 2
AND f0_.object_status_id <> 3
AND f0_.publishing_status_id = 2
ORDER BY f0_.deal_date DESC, f0_.published_date DESC
) ORDER BY rownum ASC

To fix this on the Paginator code:

file: ORM/Tools/Pagination/LimitSubqueryOutputWalker.php
method: walkSelectStatement

change:
$sql = sprintf('SELECT b.*, rownum as rn FROM (SELECT DISTINCT %s FROM (%s)) b', // AS _dctrn_result',
implode(', ', $sqlIdentifier), $sql);

for:
$sql = sprintf('SELECT b.*, rownum as rn FROM (SELECT DISTINCT %s, numrow FROM (%s) ORDER BY numrow ASC) b', // AS _dctrn_result',
implode(', ', $sqlIdentifier), $sql);



 Comments   
Comment by Marc Drolet [ 14/May/12 ]

rownum instead of numrow. sorry.

$sql = sprintf('SELECT b.*, rownum as rn FROM (SELECT DISTINCT %s, rownum FROM (%s) ORDER BY rownum ASC) b',
implode(', ', $sqlIdentifier), $sql);

Comment by Benjamin Eberlei [ 07/Jul/12 ]

Doctrine 2.2.2 doesnt have the LimitSubqueryoutputWalker and Doctrine 2.3-dev does not have the line in the code. Can you make a more explicit statement of where the change is necessary?

Comment by Marc Drolet [ 09/Jul/12 ]

It's in the Pagination of version 2.2.2. ORM/Tools/Pagination/LimitSubqueryOutputWalker.php

Comment by Benjamin Eberlei [ 09/Jul/12 ]

This is the 2.2 branch, https://github.com/doctrine/doctrine2/tree/2.2/lib/Doctrine/ORM/Tools/Pagination and https://github.com/doctrine/doctrine2/tree/2.2.2/lib/Doctrine/ORM/Tools/Pagination is the 2.2.2 tag.

no LimitSubqueryOutputWalker.php in there.

Comment by Benjamin Eberlei [ 29/Aug/12 ]

Fixed

Comment by Raymond Kolbe [ 09/Apr/13 ]

This issue is popping it's head up again!

Benjamin, your tests don't test for the ordering problem unless those tests are happening somewhere else?

https://github.com/doctrine/doctrine2/commit/f55b5411c8b1f75bf2b5cf5ffe4bc50034fb91cb

I am performing a query as complex as Marc's and I experience the same exact issue. I have checked out today's latest master branch as well as the 2.3 tag with no change.

Please advise.

Comment by Raymond Kolbe [ 09/Apr/13 ]

I have a PR in https://github.com/doctrine/doctrine2/pull/645





[DDC-2068] [GH-474] Fixed bug with comment option not being added to column. Created: 11/Oct/12  Updated: 12/Oct/12  Resolved: 12/Oct/12

Status: Resolved
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: 2.2.4, 2.3.1
Security Level: All

Type: Bug Priority: Major
Reporter: Benjamin Eberlei Assignee: Benjamin Eberlei
Resolution: Fixed Votes: 0
Labels: None


 Description   

This issue is created automatically through a Github pull request on behalf of okovalov:

Url: https://github.com/doctrine/doctrine2/pull/474

Message:

Title explains everything... =)



 Comments   
Comment by Benjamin Eberlei [ 12/Oct/12 ]

A related Github Pull-Request [GH-474] was closed
https://github.com/doctrine/doctrine2/pull/474





[DDC-2059] Property perceived as dumplicate in composite foreign key Created: 04/Oct/12  Updated: 05/Oct/12  Resolved: 05/Oct/12

Status: Resolved
Project: Doctrine 2 - ORM
Component/s: Mapping Drivers
Affects Version/s: 2.1.3
Fix Version/s: 2.2.4, 2.3.1
Security Level: All

Type: Bug Priority: Major
Reporter: Dimitris Bozelos Assignee: Benjamin Eberlei
Resolution: Fixed Votes: 0
Labels: None


 Description   

I have the following schema:

CREATE TABLE `user` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  PRIMARY KEY (`id`)
);

CREATE TABLE `project` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `user_id` int(10) unsigned NOT NULL,
  PRIMARY KEY (`id`),
);

CREATE TABLE `project_conversation` (
  `project_id` int(10) unsigned NOT NULL,
  `user_id` int(10) unsigned NOT NULL,
  PRIMARY KEY (`project_id`,`user_id`)
)

I have ommitted the foreign key definitions for better readability. When I execute doctrine:mapping:convert (in Symfony2, but it seems it's a Doctrine2 issue), I get the following error:

[Doctrine\ORM\Mapping\MappingException]                                               
Property "user" in "Project" was already declared, but it must be declared only once

I have tracked down the issue to be caused by the existence of `user_id` in the project table. So basically, because `project_conversation` references `project` which in turn references `user`, `project_conversation` reference to `user` is perceived as duplicate.

I don't think that this should be the expected behavior though. user_id in `project` references the creator of the project while user_id in `project_conversation` references the creator of the conversation and thus I think the schema is valid.



 Comments   
Comment by Benjamin Eberlei [ 05/Oct/12 ]

Will be fixed in 2.2.4 and 2.3.1





[DDC-2012] Inserting a new entity with a custom mapping type does not call convertToDatabaseValueSQL() when using InheritanceType("JOINED") Created: 04/Sep/12  Updated: 03/Oct/12  Resolved: 03/Oct/12

Status: Resolved
Project: Doctrine 2 - ORM
Component/s: Mapping Drivers
Affects Version/s: 2.2.3
Fix Version/s: 2.2.4, 2.3.1
Security Level: All

Type: Bug Priority: Critical
Reporter: Kaspars Sproģis Assignee: Fabio B. Silva
Resolution: Fixed Votes: 0
Labels: None
Environment:

PHP


Attachments: Text File DDC2012Test.php     File DDC2012Test.php    

 Description   

When using class type inheritance - @InheritanceType("JOINED") and inserting new entity with a custom mapping type, custom type method convertToDatabaseValueSQL() is never called.

Here is sample class mapping:

Unable to find source-code formatter for language: php. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
 
/**
 * @Table(name="item")
 * @Entity
 * @InheritanceType("JOINED")
 * @DiscriminatorColumn(name="type_id", type="smallint")
 * @DiscriminatorMap({1 = "ItemPerson"})
 */
class Item {

	/**
	 * @Column(name="tsv", type="tsvector", nullable=true)
	 */
	protected $tsv;
}

/**
 * @Table(name="item_person")
 * @Entity
 */
class ItemPerson extends Item
{
}

I am using the same custom TsvectorType with simple entities and even Mapped Superclasses and it works perfectly, however on InheritanceType("JOINED") method convertToDatabaseValueSQL() is never called :/
Hope someone knows how to fix this.
Thank you.



 Comments   
Comment by Fabio B. Silva [ 24/Sep/12 ]

Hi Kaspars,

I can't reproduce,
Could you change the added testcase and try to make it fails ?

Thanks

Comment by Kaspars Sproģis [ 26/Sep/12 ]

@Fabio thanks for looking into my problem
I attached test where you can detect the problem.

It was quite strange, all i did was changed column that uses custom type to array and some minimal convertToDatabaseValue and convertToDatabaseValueSQL logic and convertToDatabaseValueSQL was never called.

One more thing i noticed, this bug only appears on persist and not on merge.

Thanks

Comment by Fabio B. Silva [ 29/Sep/12 ]

Thanks Kaspars

But sorry, I dont get your use case.

Notice that convertToDatabaseValueSQL is called just when using queries to find a object by a especific columns which is your mapping type.

http://docs.doctrine-project.org/en/2.1/cookbook/advanced-field-value-conversion-using-custom-mapping-types.html#the-mapping-type

Comment by Kaspars Sproģis [ 29/Sep/12 ]

I am using PostgreSQL tsvector data type for full text search.

Here is my tsvector custom data type class:
https://gist.github.com/3129096

The only way to update this field in postgresql is to use postgresql function to_tsvector('some text').
And everything works fine, if i persist simple entity, this method transforms insert query as needed:

public function convertToDatabaseValueSQL($sqlExpr, AbstractPlatform $platform)
{
	return sprintf('to_tsvector(%s)', $sqlExpr);
}

But when i use inheritance, then by some reason convertToDatabaseValueSQL method is not called and tsv field is updated with simple text as returned by convertToDatabaseValue() method.
I modified the Ticket Test so that you can see exact moment of when it is not called, which is exactly my problem.

Here is the result after persisting (for persist it failed)

$person = new ItemPerson();
$person->setName('some words for test');
$em->persist($person);
$em->flush();

DB Result:
Name                | Tsv
--------------------|------------------------------------
some words for test | 'for' 'some' 'test' 'words'

Here is the result after second time update (now by tsv format you can see it worked):

$person->setName('some more words for test');
$em->flush();

DB Result:
Name                | Tsv
--------------------|------------------------------------
some words for test | 'test':5 'word':3
Comment by Fabio B. Silva [ 30/Sep/12 ]

Thanks Kaspars

Now i saw the problem

Writing a patch ...

Comment by Kaspars Sproģis [ 03/Oct/12 ]

Just tested fixed version and everything works perfectly now.
Thank you!

Comment by Fabio B. Silva [ 03/Oct/12 ]

Fixed by : https://github.com/doctrine/doctrine2/commit/91caff1d8965c20b72d5fdd04ffadf3ab063c1ba





[DDC-2050] [GH-459] Fix DDC-2012 Created: 30/Sep/12  Updated: 03/Oct/12  Resolved: 03/Oct/12

Status: Resolved
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: None
Fix Version/s: 2.2.4, 2.3.1
Security Level: All

Type: Bug Priority: Major
Reporter: Benjamin Eberlei Assignee: Benjamin Eberlei
Resolution: Fixed Votes: 0
Labels: None


 Description   

This issue is created automatically through a Github pull request on behalf of FabioBatSilva:

Url: https://github.com/doctrine/doctrine2/pull/459

Message:

http://www.doctrine-project.org/jira/browse/DDC-2012



 Comments   
Comment by Benjamin Eberlei [ 03/Oct/12 ]

A related Github Pull-Request [GH-459] was closed
https://github.com/doctrine/doctrine2/pull/459





[DDC-1977] Undefined index in ParameterTypeInferer Created: 10/Aug/12  Updated: 29/Aug/12  Resolved: 25/Aug/12

Status: Resolved
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.2.3
Fix Version/s: 2.2.4, 2.3
Security Level: All

Type: Bug Priority: Minor
Reporter: Matt Button Assignee: Fabio B. Silva
Resolution: Fixed Votes: 0
Labels: None
Environment:

PHP 5.3.6-13ubuntu3.8 with Suhosin-Patch (cli) (built: Jun 13 2012 17:19:54)

{ "package": "doctrine/common", "version": "2.2.2" }

,

{ "package": "doctrine/dbal", "version": "2.2.x-dev", "source-reference": "b961a3fce6bf220f1dca47d7d747b9074bea4730", "commit-date": "1341779435" }

,

{ "package": "doctrine/doctrine-bundle", "version": "dev-master", "source-reference": "62134e6a8dd3f330131ee6a970f0cee1d7760c1d", "commit-date": "1343203511" }

,

{ "package": "doctrine/orm", "version": "2.2.x-dev", "source-reference": "5d2a3bcb3b467f41ee58575764f3ba84937f76e4", "commit-date": "1341676080" }

,



 Description   

Trying to bind an empty array as a parameter to a raw SQL query results in an undefined index error on line 59.

https://github.com/doctrine/doctrine2/blob/master/lib/Doctrine/ORM/Query/ParameterTypeInferer.php#L59



 Comments   
Comment by Matt Button [ 10/Aug/12 ]

One way to work around this is to specify the type of the array as the third parameter to addParameter

Comment by Fabio B. Silva [ 25/Aug/12 ]

Fixed by : https://github.com/doctrine/doctrine2/commit/ece6a005bcecc4a9e4a154d9379cfbe141370415





Generated at Mon May 20 07:31:40 UTC 2013 using JIRA 5.2.7#850-sha1:b2af0c8dc8537b36121c6a579fabbdf79fc919e5.