[DC-329] Problem saving Self Referencing (Nest Relations) Created: 05/Dec/09  Updated: 03/Jun/11

Status: Open
Project: Doctrine 1
Component/s: Relations
Affects Version/s: 1.2.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Jaime Suez Assignee: Jonathan H. Wage
Resolution: Unresolved Votes: 14
Labels: None
Environment:

Mac OSX 10.5, Symfony 1.4 and MAMP


Attachments: Zip Archive h2aEqualable.zip     PNG File saving process.png     File schema.yml     File testEqualNestRelationsTest.php     File testNestedEqualRelationWithObjectsTest.php    

 Description   

There is a problem when you save Many to Many Self Referencing (Nest Relations).

I used the same example that it's used on Doctrine 1.2 Self Referencing (Nest Relations) Documentation:

My Schema:

User:
columns:
name:

Unknown macro: { type}

relations:
Parents:
class: User
local: child_id
foreign: parent_id
refClass: UserReference
foreignAlias: Children

UserReference:
columns:
parent_id:
type: integer
primary: true
child_id:
type: integer
primary: true

Fixtures:

User:
james:
name: James
alexander:
name: Alexander
david:
name: David

The problem happens with the Children. The first time I assigned children and saved the User, there is no problem; but when I saved the User again (without any change on him) the values in the UserReference? table changes: the "parent_id" takes the value of the "child_id".

I test it with Sf 1.4 and with Sf 1.3 stable versions, and I've got the same error.

If it not clear to understand what I'm saying, I put a "very ilustrative" image about the form and how change the values of the UserReference? table. Please see it, because it would be very usefull for understanding (explains better than my terrible English)

Thanks for all your work!

P. D.

I forgot to to say that watching the stack trace I think that the problem is in:

symfony/lib/plugins/sfDoctrinePlugin/lib/vendor/doctrine/Doctrine/Connection/UnitOfWork.php

In the line 429:

public function saveAssociations(Doctrine_Record $record)

I would like to help more, but I couldn't figure out how this function works.



 Comments   
Comment by Jaime Suez [ 07/Dec/09 ]

This picture explains the error that it's done when you do an insert and then you save it again.

Comment by Jonathan H. Wage [ 07/Dec/09 ]

Can you provide a failing Doctrine test case? This may be a problem with Symfony and not Doctrine.

Comment by Jaime Suez [ 08/Dec/09 ]

First I published this bug on Symfony, because I thought that was a problem with Symfony, but you set it as invalid because it's belong to Doctrine.

This is the ticket on symfony:
http://trac.symfony-project.org/ticket/7690

I could do the Doctrine test case, but in one more mounth, because tomorrow I go out for vacations.
When I come back I'll do them.

Comment by psylosss [ 20/Dec/09 ]

Yes, I have got the same problem. During hours of debugging and exploring Doctrine's code, I found that problem localized in Hydrator component. I cannot digg deeper, so I ask you to fix it very much

Comment by Filippo De Santis [ 26/Dec/09 ]

Hi,
I had this problem in the 1.0 version of doctrine as well.
Using symfony, I upgraded to symfony 1.3 and doctrine 1.2 and I've found the same problem.

Attached to this comment you can find the files containing the schema I used (it is the same as that describe on this ticket but with a different class naming), and 2 unit tests. One using the symfony form and one using directly the doctrine objects. They report the bug discussed on this ticket and show another bug saving objects that already have one ore more relations.

As fixtures I used a fixtures.yml file containing:

Issue:
<?php for ($i = 1; $i <= 30; $i++): ?>
Issue_<?php echo $i; ?>:
title: "new issue <?php echo $i ?>"
<?php endfor; ?>

The following are the tests results:

– testEqualNestRelationsTest.php (using the forms) –

> First form for first issue: creation of the relation "issue 1 -> issue 2"
ok 1 - the edit form for issue 1 is valid
ok 2 - form1 saved
ok 3 - issue 1 is related with another issue
ok 4 - issue 1 is really related with issue 2

> Second form for first issue: saving the relation between "issue 1" and "issue 2" from "issue 2"'s form
ok 5 - the edit form for issue 2 is valid
ok 6 - form2 saved
not ok 7 - issue 2 is related with another issue ******** this is the problem reported on this ticket ********

  1. Failed test (./test/unit/testEqualNestRelationsTest.php at line 57)
  2. got: 0
  3. expected: 1
    not ok 8 - issue 2 is really related with issue 1
  4. Failed test (./test/unit/testEqualNestRelationsTest.php at line 58)
  5. got: NULL
  6. expected: 1

ok 9 - issue 1 is related with another issue
not ok 10 - issue 1 is really related with issue 2 ******** this is the problem reported on this ticket ********

  1. Failed test (./test/unit/testEqualNestRelationsTest.php at line 63)
  2. got: '1'
  3. expected: 2
    > Checking the relation table IssueReference ...
    ok 11 - There is only one relation saved on IssueReference
    ok 12 - 1 is the issue1 field of the relation
    not ok 13 - 2 is the issue2 field of the relation ******** this is the problem reported on this ticket ********
  4. Failed test (./test/unit/testEqualNestRelationsTest.php at line 71)
  5. got: '1'
  6. expected: '2'

> multi selection proof
ok 14 - the edit form for issue 1 is valid
ok 15 - form1 saved
ok 16 - the edit form for issue 2 is valid

                • this is the other bug I've found ********
                  not ok 17 - form2 exception catched : SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1-1' for key 1
  1. Failed test (./test/unit/testEqualNestRelationsTest.php at line 114)

– testNestedEqualRelationWithObjectsTest.php (test using the "doctrine" objects ) –

> Saving relation directly from the objects and not using the forms...
ok 1 - issue1 saved
> checking issue1 relations
ok 2 - issue 1 is related with another issue
ok 3 - issue 1 is really related with issue 2

> checking "reverse" relation on issue2
ok 4 - issue 2 is related with another issue
ok 5 - issue 2 is really related with issue 1

> adding a relation issue2->issue1 directly
ok 6 - issue2 saved

> checking issue2 relations
not ok 7 - issue 2 is related with another issue ******** this is the problem reported on this ticket ********

  1. Failed test (./test/unit/testNestedEqualRelationWithObjectsTest.php at line 56)
  2. got: 0
  3. expected: 1
    not ok 8 - issue 2 is really related with issue 1
  4. Failed test (./test/unit/testNestedEqualRelationWithObjectsTest.php at line 57)
  5. got: NULL
  6. expected: 1

> checking issue1 relations
ok 9 - issue 1 is related with another issue
not ok 10 - issue 1 is really related with issue 2 ******** this is the problem reported on this ticket ********

  1. Failed test (./test/unit/testNestedEqualRelationWithObjectsTest.php at line 63)
  2. got: '1'
  3. expected: 2

> Checking the relation table IssueReference ...
ok 11 - There is only one relation saved on IssueReference
ok 12 - 1 is the issue1 field of the relation
not ok 13 - 2 is the issue2 field of the relation

  1. Failed test (./test/unit/testNestedEqualRelationWithObjectsTest.php at line 71)
  2. got: '1'
  3. expected: '2'

> object multi adding relation proof
ok 14 - issue1 saved

                • this is the other bug I've found ********
                  not ok 15 - issue2 exception catched : SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1-1' for key 1
  1. Failed test (./test/unit/testNestedEqualRelationWithObjectsTest.php at line 112)
Comment by Jaime Suez [ 13/Jan/10 ]

With the test provided by Filippo De Santis is any other thing needed??

I still on vacations untill 3 of febrary, so then I could do any other thing needed.

Comment by Jaime Suez [ 01/Feb/10 ]

The necessary test have to be how are illustrated in this link???

http://www.doctrine-project.org/documentation/manual/1_2/en/unit-testing

Comment by Alexander Sweis [ 09/Feb/10 ]

What has happen with this bug?? I also have the same problem...

Comment by Jonathan H. Wage [ 02/Mar/10 ]

Hi, yes. We still need a unit test case like described here: http://www.doctrine-project.org/documentation/manual/1_2/en/unit-testing

So we can run it with all the other tests and reproduce the issue in Doctrine by itself. Not in Symfony.

Thanks, Jon

Comment by psylosss [ 02/Mar/10 ]

Jon, here is the test for this bug: http://gist.github.com/319856

Please, fix it.. my work has been stopped cause of it

Comment by Jaime Suez [ 15/Mar/10 ]

Yes... please fix it... I'm also stuck in my work since a lot of time that the bug it's open.

Thanks

Comment by Nil Null [ 15/Mar/10 ]

I'm affected with this bug as well, fortunately, I found a way to get over it, it's ugly and innefficient, but at least my work isn't stopped due to this bug. I'll try to explain what I did commenting here later. Anyway, I really need this bug fixed too before I can start betatesting my app.
Thanks!

P.D: I'm using Doctrine 1.2.1.

Comment by Jonathan H. Wage [ 15/Mar/10 ]

I am trying to understand the issue but it just doesn't make much sense to me yet. The test case is wrong because you are dealing with instances of objects that have previously loaded references, then you try and re-fetch the same object instance. If you do $user1->free(true); then re-fetch the object the test passes as expected. So, I don't really have a proper failing test case still that actually exposes the problem. I'll keep playing around see if I can understand. let me know if anyone else has more information or a patch to test.

Thanks, Jon

Comment by Filippo De Santis [ 16/Mar/10 ]

Hi Jonathan,
This is the "translated" version of my unit test for symfony in "doctrine unit test"

http://github.com/ideato/phpcollab3/blob/phpcollabsf13/test/doctrine_unit_test/DC329TestCase.php

The first test is to show the error "SQLSTATE[23000]: Integrity constraint violation: 19 columns issue1, issue2 are not unique".
The second one is to show the fact that the relation are saved with wrong data.

Sorry for the late translation.

Filippo

Comment by Hendri Kurniawan [ 16/Mar/10 ]

Hi Jonathan,

It seems that the problem is caused by Doctrine_Collection::add() method line 456 (I could be totally wrong).

if (isset($this->referenceField)) {
    $value = $this->reference->get($this->relation->getLocalFieldName());
    if ($value !== null) {
        $record->set($this->referenceField, $value, false);
    } else {
        $record->set($this->referenceField, $this->reference, false);
    }
...

What is trying to do is to set the reference field value to the reference value that it's linked to.

This only happens when you try to fetch record from the "children".

Consider the following:

Employees:
  tableName: employees
  columns:
    id:
      type: integer(4)
      primary: true
      autoincrement: true
    name:
      type: string(100)
  relations:
    managing:
      class: Employees
      type: many
      local: manager_id
      foreign: employee_id
      foreignAlias: managedBy
      refClass: EmployeeManager

EmployeeManager:
...

Code:

// ----- The following will not give you any issues
  $manager = Doctrine::getTable('Issue')->findOneBy('id', '1'); // This is a manager
  foreach ($manager->managing as $employee) {
    ....
  }

  // This will not save anything
  $manager->save();
// -----

// ----- The following will
  $employee = Doctrine::getTable('Issue')->findOneBy('id', '2'); // This is an employee
  foreach ($employee->managedBy as $manager) {
    ...
  }

  // This will save the wrong data to the "EmployeeManager" table
  // because the code from Doctrine_Collection is overwriting "EmployeeManager" records
  // When they were being added to the collection by the Hydrator.
  $employee->save();
// -----

Debug backtrace when this happens:

string(100) "File: /Data/app/onlinevw-drupalvividwireless/libs/external-doctrine1/Doctrine/Record.php @ line 1432"
string(103) "File: /Data/app/onlinevw-drupalvividwireless/libs/external-doctrine1/Doctrine/Collection.php @ line 461"
string(99) "File: /Data/app/onlinevw-drupalvividwireless/libs/external-doctrine1/Doctrine/Access.php @ line 131"
string(107) "File: /Data/app/onlinevw-drupalvividwireless/libs/external-doctrine1/Doctrine/Hydrator/Graph.php @ line 229"
string(101) "File: /Data/app/onlinevw-drupalvividwireless/libs/external-doctrine1/Doctrine/Hydrator.php @ line 137"
string(108) "File: /Data/app/onlinevw-drupalvividwireless/libs/external-doctrine1/Doctrine/Query/Abstract.php @ line 1036"
string(105) "File: /Data/app/onlinevw-drupalvividwireless/libs/external-doctrine1/Doctrine/Relation/Nest.php @ line 84"
string(100) "File: /Data/app/onlinevw-drupalvividwireless/libs/external-doctrine1/Doctrine/Record.php @ line 1362"
string(100) "File: /Data/app/onlinevw-drupalvividwireless/libs/external-doctrine1/Doctrine/Record.php @ line 1333"

Comment by Pierrot Evrard [ 19/Mar/10 ]

Hi all,

For people who are looking for a workaround, I've found one that work for me.

Get the doctrine extension h2aEqualable (h2aEqualable.zip), register it as a doctrine extension (see http://www.doctrine-project.org/documentation/manual/1_2/0_11/extensions) and make your refClass of every equal nest relation acting as h2aEqualable.

In your PHP script :
{{
Doctrine::setExtensionsPath( 'path/to/unzip/place' );
Doctrine_Manager::getInstance()->registerExtension( 'h2aEqualable' );
}}

In your YAML schema :
{{
MyEqualNestRelationRefClassModel:
actAs:
h2aEqualable:
fields: [id_1,id_2]
}}

As I've said before, it works for me, so let me know if it solve your issues too...

Comment by Pierre Guéguin [ 18/Jun/10 ]

I have the very same issue with non equal nest relations.
It is stopping my project

SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '87-87' for key 'PRIMARY'

execute : UPDATE cop_job_stream_successor SET job_stream_destination_id = ?, updated_at = ? WHERE job_stream_source_id = ? AND job_stream_destination_id = ? - (87, 2010-06-18 18:55:32, 87, 94)

CopJobStream:
columns:
name:
type: string(255)
notnull: false
....
relations:
SuccessorJobStreams:
class: CopJobStream
local: job_stream_source_id
foreign: job_stream_destination_id
refClass: CopJobStreamSuccessor
foreignAlias: PredecessorJobStreams
onDelete: CASCADE
onUpdate: CASCADE

CopJobStreamSuccessor:
columns:
job_stream_source_id:
type: integer
primary: true
job_stream_destination_id:
type: integer
primary: true
relations:
CopJobStream:
local: job_stream_source_id
foreignAlias: CopJobStreamSuccessors
onDelete: CASCADE
onUpdate: CASCADE

Comment by Pierre Guéguin [ 18/Jun/10 ]

It works anyway with above h2aEqualable extension

Comment by Daniel Reiche [ 26/Jan/11 ]

after stumbling into the same pit as everyone, i created a fresh report as DC-958. Later I found out that it has been reported before, so please mark DC-958 as duplicate of this. (or at least related, as I was using a Non-Equal Nested Set) also DC-952 refers to the same problem, but also with Non-Equal nestet sets.

My Case: Non-Equal-Nested-Set. Problems occur on saving Objects for 2nd or more time. Doctrine deletes all Child-Relations of the Object to be saved and updates Relations of the Childs to other Objects (other parents of the children objects) with references to the child itself!

the proposed h2aEqualable extension does prevent the wrong update statements but not the initial delete of the child-relations.

What exactly is missing on this bug to be fixed? It is now a year old, and clearly preventing people from using Self-Referencing relations with doctrine.

How can we help? More sample data, schema definitions, test cases?

Comment by Francesco Levorato [ 15/Feb/11 ]

Here is a better and simpler test case for the equal relation: https://gist.github.com/827414

There is clearly a memory problem here. I'll leave it up to Jonathan to decide whether it's a scenario which can be avoided in Doctrine itself or must be handled by Symfony (http://trac.symfony-project.org/ticket/9398).

In the Symfony context calls to refresh() might not be an option when modifying an object (aka saving a form).

Comment by Yitzchak Schaffer [ 17/Feb/11 ]

h2aEqualable seems to work for me.

Comment by Pavel Campr [ 02/Jun/11 ]

h2aEqualable works only partly for me (same as Daniel Reiche wrote).

I tried to identify the problem.

My usecase was:
1) I have an Article record in $article
2) I called $article->RelatedArticles->getPrimaryKeys() , where RelatedArticles is a name of my self reference
3) $article->save();

The problem originates from step 2), during hydratation process for related articles, which are added to a collection of related articles, in Collection->add() method (line 465) here:

if ($value !== null)

{ $record->set($this->referenceField, $value, false); }

else {

but, $this->referenceField contains wrong name of reference field. Self referenced model (Article) has two relations to the same class (one from "owning" side, second from "referencing" side). $this->referenceField in the collection is somewhere filled with the wrong relation name (not the"referencing" one, but the "owning" one).

When I save the $article, all referenced records are saved too and this is the problem, why the Doctrine tries to save corrupted data:

In my example, I have this data:

in Doctrine_Collection->add on line 457: $record) is a RelatedArticle (
article1 = string(1) "4"
article2 = string(1) "9"
)
in Doctrine_Collection->add on line 458: $this->referenceField) = string(19) "article1"
in Doctrine_Collection->add on line 459: $value) = string(1) "9"

BUT expected data is:

in Doctrine_Collection->add on line 458: $this->referenceField) = string(19) "article2"

This wrong behavior modified my RelatedArticle into this: RelatedArticle (
article1 = string(1) "9"
article2 = string(1) "9"
)

I have no idea how to fix it, but maybe someone can, I hope

(
my schema is:

Article
relations:
RelatedArticles:
class: Article
local: article1
foreign: article2
refClass: RelatedArticle
)

Maybe, when "refClassRelationAlias" option was invented to specifically point to a relation, instead of using "refClass", which can be used more times for one record and thus is not unambiguous, something for self referenced relations should be added as well. My "refClass: RelatedArticle" creates (probably) a relation from the foreign side too and Article contains two relations, which uses same refClass. Doctrine then doesn't know, which one should be used.

Comment by Pavel Campr [ 03/Jun/11 ]

Non-equal nest relations work properly, when refClassRelationAlias undocumented option is used, e.g.

Article:
...
relations:
RelatedArticles:
class: Article
local: article1
foreign: article2
refClass: RelatedArticle
refClassRelationAlias: RelatedArticleAlias

Anything can be used as a value for refClassRelationAlias.

Unfortunately, this doesn't fix equal nest relations.

Generated at Thu Jul 31 11:37:23 UTC 2014 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.