Doctrine 1
  1. Doctrine 1
  2. DC-41

cache problem with joined tables and external data change

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: 1.1.4
    • Fix Version/s: None
    • Component/s: Caching
    • Labels:
      None

      Description

      Doctrine doesn't invalidate cache properly. Queries like the one below has the second table taken from cache and it shouldn't.

      $comp = Doctrine_Query::create()
      			->select('a.*, s.*')
      			->from('AttachedComponents a, a.SubscriptionComponents s')
      			->fetchOne();
      

      In bug DC-24 the advice is to select id of second table, I selected all fields in second table but doctrine doesn't seem to notice foreign key has changed.

      1. my2.diff
        1 kB
        Ionut E
      2. my2TestCase.php
        3 kB
        Ionut E

        Issue Links

          Activity

          Hide
          Roman S. Borschel added a comment -

          This has nothing to do with DC-24 and nothing to do with caching. This merely looks like a strange hydration issue.

          Show
          Roman S. Borschel added a comment - This has nothing to do with DC-24 and nothing to do with caching. This merely looks like a strange hydration issue.
          Hide
          Ionut E added a comment -

          Made a patch.

          Show
          Ionut E added a comment - Made a patch.
          Hide
          Jonathan H. Wage added a comment -

          I understand this is a weird problem, but this is actually expected behavior to keep what is in memory and not refresh/overwrite local stuff. Because $comp is previously instantiated and in the identity map, it already has that relationship loaded and referenced. So when you run the query again, it doesn't override the data that is already inside that object. To fix it in your project you will just need to manually refresh the object or remove the old reference and be aware of these things in the future.

          While your patch technically "fixes" the test case, it is still not valid. The patch opens up all kinds of other problems.

          Show
          Jonathan H. Wage added a comment - I understand this is a weird problem, but this is actually expected behavior to keep what is in memory and not refresh/overwrite local stuff. Because $comp is previously instantiated and in the identity map, it already has that relationship loaded and referenced. So when you run the query again, it doesn't override the data that is already inside that object. To fix it in your project you will just need to manually refresh the object or remove the old reference and be aware of these things in the future. While your patch technically "fixes" the test case, it is still not valid. The patch opens up all kinds of other problems.
          Hide
          Roman S. Borschel added a comment -

          Sorry for reopening Jon but I think there might be a valid issue here still. While we agree that local changes in objects are generally considered newer and not overriden by queries from the database this here is something different I think. I'll try to make it clear:

          When any object is hydrated, we pick the PK and first look in the identity map ("session cache"). If it is there, we take that and put it in the query result.

          OK. But this issue seems to demonstrate that this is not happening, at least on joined single-valued associations. The second query returns the associated object with ID 2. So Doctrine should try to look up that object with ID 2 in the identity map and if its not there construct it and put it there. Instead Doctrine currently seems to just retain anything that is set in the association.

          So I think that needs to be fixed by not blindly retaining what is there but instead by picking the PK of the joined record/entity that was returned by the query and then looking that up as usual in the identity map. If it is found there, we take that object and if not we construct it and put it there.

          This might also affect Doctrine 2 so I will copy this issue there for further investigation, too.

          Show
          Roman S. Borschel added a comment - Sorry for reopening Jon but I think there might be a valid issue here still. While we agree that local changes in objects are generally considered newer and not overriden by queries from the database this here is something different I think. I'll try to make it clear: When any object is hydrated, we pick the PK and first look in the identity map ("session cache"). If it is there, we take that and put it in the query result. OK. But this issue seems to demonstrate that this is not happening, at least on joined single-valued associations. The second query returns the associated object with ID 2. So Doctrine should try to look up that object with ID 2 in the identity map and if its not there construct it and put it there. Instead Doctrine currently seems to just retain anything that is set in the association. So I think that needs to be fixed by not blindly retaining what is there but instead by picking the PK of the joined record/entity that was returned by the query and then looking that up as usual in the identity map. If it is found there, we take that object and if not we construct it and put it there. This might also affect Doctrine 2 so I will copy this issue there for further investigation, too.
          Hide
          Roman S. Borschel added a comment -

          Relaxing priority a bit as this is definitely not a blocker as there are many ways around this issue.

          Show
          Roman S. Borschel added a comment - Relaxing priority a bit as this is definitely not a blocker as there are many ways around this issue.
          Hide
          Roman S. Borschel added a comment -

          This is a conceptually difficult issue I think. I already have a patch ready for Doctrine 2 but I am still unsure as to what the ideal behavior should be.

          I reproduced it in Doctrine 2 with the following scenario:

                  $address = new CmsAddress;
                  $address->country = 'de';
                  $address->zip = '12345';
                  $address->city = 'Berlin';
                  
                  $user1 = new CmsUser;
                  $user1->status = 'dev';
                  $user1->username = 'romanb';
                  $user1->name = 'Roman B.';
          
                  $user2 = new CmsUser;
                  $user2->status = 'dev';
                  $user2->username = 'gblanco';
                  $user2->name = 'Guilherme Blanco';
                  
                  $address->setUser($user1);
                  
                  $this->_em->persist($address);
                  $this->_em->persist($user1);
                  $this->_em->persist($user2);
                  $this->_em->flush();
                  
                  
                  $this->assertSame($user1, $address->user);
                  
                  //external update to CmsAddress
                  $this->_em->getConnection()->executeUpdate('update cms_addresses set user_id = ?', array($user2->getId()));
                  
                  //select
                  $q = $this->_em->createQuery('select a, u from Doctrine\Tests\Models\CMS\CmsAddress a join a.user u');
                  $address2 = $q->getSingleResult();
                  
                  $this->assertSame($address, $address2);
                  $this->assertSame($user2, $address2->user);
                  $this->assertSame($user2->address, $address);
                  
                  // $user1 still points to $address. $user2 now as well.
                  // The object model is now corrupt.
          

          Assuming the external update can happen concurrently in another request, that means after a query you could suddenly have a broken object model in-memory and I imagine this being pretty hard to debug. It might even be possible to reset $user1->address = null during hydration but would that be correct then?

          I will probably need to reproduce this issue with other ORMs to see how they react. If anyone else has a well-founded opinion on this matter, please shoot.

          Show
          Roman S. Borschel added a comment - This is a conceptually difficult issue I think. I already have a patch ready for Doctrine 2 but I am still unsure as to what the ideal behavior should be. I reproduced it in Doctrine 2 with the following scenario: $address = new CmsAddress; $address->country = 'de'; $address->zip = '12345'; $address->city = 'Berlin'; $user1 = new CmsUser; $user1->status = 'dev'; $user1->username = 'romanb'; $user1->name = 'Roman B.'; $user2 = new CmsUser; $user2->status = 'dev'; $user2->username = 'gblanco'; $user2->name = 'Guilherme Blanco'; $address->setUser($user1); $ this ->_em->persist($address); $ this ->_em->persist($user1); $ this ->_em->persist($user2); $ this ->_em->flush(); $ this ->assertSame($user1, $address->user); //external update to CmsAddress $ this ->_em->getConnection()->executeUpdate('update cms_addresses set user_id = ?', array($user2->getId())); //select $q = $ this ->_em->createQuery('select a, u from Doctrine\Tests\Models\CMS\CmsAddress a join a.user u'); $address2 = $q->getSingleResult(); $ this ->assertSame($address, $address2); $ this ->assertSame($user2, $address2->user); $ this ->assertSame($user2->address, $address); // $user1 still points to $address. $user2 now as well. // The object model is now corrupt. Assuming the external update can happen concurrently in another request, that means after a query you could suddenly have a broken object model in-memory and I imagine this being pretty hard to debug. It might even be possible to reset $user1->address = null during hydration but would that be correct then? I will probably need to reproduce this issue with other ORMs to see how they react. If anyone else has a well-founded opinion on this matter, please shoot.
          Hide
          Roman S. Borschel added a comment -

          After some more testing and further discussion this is desired behavior. See DDC-21 and DDC-22 for things that need to be done to have this behavior implemented consistently in Doctrine 2. Its probably too late to make such large changes to Doctrine 1 in order to fix the inconsistencies.

          Show
          Roman S. Borschel added a comment - After some more testing and further discussion this is desired behavior. See DDC-21 and DDC-22 for things that need to be done to have this behavior implemented consistently in Doctrine 2. Its probably too late to make such large changes to Doctrine 1 in order to fix the inconsistencies.

            People

            • Assignee:
              Jonathan H. Wage
              Reporter:
              Ionut E
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: