[DDC-3400] Wrong result using php-cli Created: 19/Nov/14  Updated: 20/Nov/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.4.6
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Damir Abdijevic Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: cache, cli, dql, environment, orm
Environment:

Windows 7, Debian GNU/Linux 7 PHP 5.5.15



 Description   

Same query produces different results. With apache module everything works like expected. With php-cli the join condition to i18n table is ignored and calling getCountries() returns all and not only that entity that is matched by join condition.

        $qb = $this->_em->createQueryBuilder()
                        ->select('t', 'i18n')
                        ->from($this->_entityName, 't')
                        ->innerJoin('t.countries', 'i18n')
                        ->where('i18n.locale = :localeId')
                        ->setParameter('localeId', $localeId);

Country Entity:

namespace MyApp\Model\Entity;

use Doctrine\ORM\Mapping as ORM;
use Doctrine\Common\Collections\ArrayCollection;

/**
 * Country entity
 *
 * @ORM\Entity(repositoryClass="MyApp\Model\Repository\Country")
 * @ORM\Table(name="country")
 *
 */
class Country
{
    /**
     * @var int
     * @ORM\Id
     * @ORM\Column(type="integer", name="id")
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    protected $id;

    /**
     * @var string
     * @ORM\Column(type="string", name="name", length=32, unique=true)
     */
    protected $name;

    /**
     * @var int
     * @ORM\Column(type="integer", name="sort", unique=true)
     */
    protected $sort;

    /**
     * @var ArrayCollection
     * @ORM\OneToMany(targetEntity="ProductDescription\Model\Entity\CountryI18n", mappedBy="country", cascade={"all"})
     */
    protected $countries;


    /**
     * Constructor
     *
     * @return Country
     */
    public function __construct()
    {
        $this->countries = new ArrayCollection();
    }

    /**
     * Getter for $this->id
     *
     * @return int
     */
    public function getId()
    {
        return $this->id;
    }

    /**
     * Setter for $this->id
     *
     * @param int $id entity id
     *
     * @return void
     */
    public function setId($id)
    {
        $this->id = (int) $id;
    }

    /**
     * Getter for $this->sort
     *
     * @return int
     */
    public function getSort()
    {
        return $this->sort;
    }

    /**
     * Setter for $this->sort
     *
     * @param int $sort sort order
     *
     * @return void
     */
    public function setSort($sort)
    {
        $this->sort = (int) $sort;
    }

    /**
     * Getter for $this->name
     *
     * @return string
     */
    public function getName()
    {
        return $this->name;
    }

    /**
     * Setter for $this->name
     *
     * @param string $name language name in german
     *
     * @return void
     */
    public function setName($name)
    {
        $this->name = (string) $name;
    }

   /**
    * Get collection from i18n table
    *
    * @return ArrayCollection
    */
    public function getCountries()
    {
        return $this->countries;
    }

    /**
     * Proxy method. So we have working with all entities same method
     * to getting i18n data.
     *
     * @return ArrayCollection
     */
    public function getI18n()
    {
        return $this->getCountries();
    }

CountryI18nEntity:

namespace MyApp\Model\Entity;

use Doctrine\ORM\Mapping as ORM;

/**
 * CountryI18n entity
 *
 * @ORM\Entity
 * @ORM\Table(name="country_i18n", uniqueConstraints={@ORM\UniqueConstraint(name="idx_UNIQUE_country_id_locale_id", columns={"country_id", "locale_id"})})
 */
class CountryI18n
{
    /**
     * @var int
     * @ORM\Id
     * @ORM\Column(type="integer", name="id")
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    protected $id;

    /**
     * @ORM\ManyToOne(targetEntity="ProductDescription\Model\Entity\Country", inversedBy="countries")
     * @ORM\JoinColumn(name="country_id", referencedColumnName="id")
     */
    protected $country;

    /**
     * @ORM\ManyToOne(targetEntity="ProductDescription\Model\Entity\Language", inversedBy="countries")
     * @ORM\JoinColumn(name="locale_id", referencedColumnName="id")
     */
    protected $locale;

    /**
     * @var string
     * @ORM\Column(type="string", name="name", length=255)
     */
    protected $name;


    /**
     * Getter for $this->id
     *
     * @return int
     */
    public function getId()
    {
        return $this->id;
    }

    /**
     * Setter for $this->id
     *
     * @param int $id id of row primary key
     *
     * @return void
     */
    public function setId($id)
    {
        $this->id = (int) $id;
    }

    /**
     * Getter for $this->country
     *
     * @return Country
     */
    public function getCountry()
    {
        return $this->country;
    }

    /**
     * Setter for $this->country
     *
     * @param Country $country country entity to set
     *
     * @return void
     */
    public function setCountry(Country $country)
    {
        $this->country = $country;
    }

    /**
     * Getter for $this->locale
     *
     * @return Language
     */
    public function getLocale()
    {
        return $this->locale;
    }

    /**
     * Setter for $this->locale
     *
     * @param Language $locale language entity to set as locale
     *
     * @return void
     */
    public function setLocale(Language $locale)
    {
        $this->locale = $locale;
    }

    /**
     * Getter for $this->name
     *
     * @return string
     */
    public function getName()
    {
        return $this->name;
    }

    /**
     * Setter for $this->name
     *
     * @param string $name translation name
     *
     * @return void
     */
    public function setName($name)
    {
        $this->name = (string) $name;
    }

}


 Comments   
Comment by Marco Pivetta [ 19/Nov/14 ]

Looks like a caching issue. The amount of information provided is insufficient as it is. I'd suggest verifying if the generated SQL is the same, and checking that all caches were cleared both in CLI and in WEB sapis.

Comment by Damir Abdijevic [ 20/Nov/14 ]

O.k. here some further information.

No caching like Xcache or APC is enabled. PHP 5.5 integrated opcache ist not enabled. All Doctrine caches are set to Array. The sql queries are in both cases exactly the same:

sql
SELECT c0_.id AS id0, c0_.name AS name1, c0_.sort AS sort2, c1_.id AS id3, c1_.name AS name4, c1_.country_id AS country_id5, c1_.locale_id AS locale_id6 FROM country c0_ INNER JOIN country_i18n c1_ ON c0_.id = c1_.country_id WHERE c1_.locale_id = ?

/* locale_id = 2 */

The sql result is correct

Running the console script a ZF2 Initializer runs before that performs this query builder query:

QueryBuilder

$qb = $this->_em->createQueryBuilder()
                            ->select('t', 'i18n', 'l')
                            ->from($this->_entityName, 't')
                            ->innerJoin('t.countries', 'i18n')
                            ->innerJoin('i18n.locale', 'l');

That results in following SQL:

sql
SELECT c0_.id AS id0, c0_.name AS name1, c0_.sort AS sort2, c1_.id AS id3, c1_.name AS name4, l2_.id AS id5, l2_.name AS name6, l2_.full_name AS full_name7, l2_.locale AS locale8, c1_.country_id AS country_id9, c1_.locale_id AS locale_id10, l2_.country_id AS country_id11 FROM country c0_ INNER JOIN country_i18n c1_ ON c0_.id = c1_.country_id INNER JOIN language l2_ ON c1_.locale_id = l2_.id
Comment by Marco Pivetta [ 20/Nov/14 ]

What happens if you run those SQL statements via CLI (dbal:run-sql) or WEB? Same results?

Comment by Damir Abdijevic [ 20/Nov/14 ]

The sql result is correct. Can I attach it to the ticket? I have exported it to csv.

Comment by Marco Pivetta [ 20/Nov/14 ]

If the same results are produced on CLI and WEB APIs then I suggest trying to insulate the issue in a functional test to be run in both context. You probably have a different ORM bootstrap for CLI and WEB.

Attaching a CSV for same results makes no real difference here.

Comment by Damir Abdijevic [ 20/Nov/14 ]

No, I didn't want to attach two times the same result. Wanted to attach it one time to show that those entitites that are wrong in the result doesn't appear in the sql result at all. I don't have different bootstraps. After a few short tests I think it is an error in th ArrayCache mechanism. The difference was that using Apache one initializer was not called. Calling this initializer in both cases leads now to wrong results in CLI and WEB API.

When the second query is not executed everything is fine. But when the second longer query runs and selects all i18n entities and after it the first query runs then the issue appears.





[DDC-3379] [GH-1177] Ensure metadata cache is not ArrayCache in production Created: 08/Nov/14  Updated: 11/Nov/14  Resolved: 11/Nov/14

Status: Resolved
Project: Doctrine 2 - ORM
Component/s: Mapping Drivers, Tools
Affects Version/s: 2.4.6
Fix Version/s: 2.5
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Marco Pivetta
Resolution: Fixed Votes: 0
Labels: array-cache, cache, configuration, production-settings


 Description   

This issue is created automatically through a Github pull request on behalf of c960657:

Url: https://github.com/doctrine/doctrine2/pull/1177

Message:

This PR adds a new check to Configuration::ensureProductionSettings(). It ensures that the metadata cache does not use ArrayCache in production.

The ArrayCache forgets everything at the end of each request, while the other bundled cache engines (Memcache, MongoDB, FileCache etc.) may keep entries for much longer. The metadata only changes when the source files are changed, so in production the metadata cache only needs to be cleared when new files are deployed, just like the generation of proxy classes.

It is possible to specify/modify metadata dynamically at runtime e.g. using the loadClassMetadata event so that the metadata changes without any changes to the source files, but I assume this is not a supported use-case (at least not without clearing the metadata cache manually). Agree?

The ArrayCache is the default cache engine added by e.g. Doctrine Bundle (for Symfony) and Doctrine ORM Service Provider (for Silex), so most developers are probably not aware of it. However, accessing the metadata at run-time has a significant performance impact (for some anecdotal evidence, see https://github.com/doctrine/common/pull/319#issuecomment-60945429), so a warning in ensureProductionSettings() is relevant.

The PR also fixes the units for ensureProductionSettings() that were previously not being run due to a missing "test" prefix on the ConfigurationTest::ensureProductionSettings method name.



 Comments   
Comment by Doctrine Bot [ 10/Nov/14 ]

A related Github Pull-Request [GH-1177] was assigned:
https://github.com/doctrine/doctrine2/pull/1177





[DDC-3174] Query Cache not correct working when using SQLFilter Created: 17/Jun/14  Updated: 17/Jun/14

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.4.1
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Benno Eggnauer Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: cache, sqlfilter


 Description   

We have an SQLFilter to filter on entities with a specific Trait implemented. The filter is very easy:

$res = $targetTableAlias . '.agency_id = ' . $this->getCurrentAgencyId();

On our system we have the query cache enabled, this works as long the "AgencyId" doesn't change. When the ID changes, the query cache seems to return the wrong (old cache) query.



 Comments   
Comment by Marco Pivetta [ 17/Jun/14 ]

I'm not sure if this case should be contemplated by the ORM. Filters are low-level and supposed to be stateless (services).

Comment by Benno Eggnauer [ 17/Jun/14 ]

OK, we can disable the query cache for this case. But then should at least the documentation be updated, which explicitly mentions to use filter for locales, which are also not stateless: http://doctrine-orm.readthedocs.org/en/latest/reference/filters.html#example-filter-class

Also in the query cache chapter: http://doctrine-orm.readthedocs.org/en/latest/reference/caching.html#query-cache

It is highly recommended that in a production environment you cache the transformation of a DQL query to its SQL counterpart. It doesn’t make sense to do this parsing multiple times as it doesn’t change unless you alter the DQL query.

Comment by Marco Pivetta [ 17/Jun/14 ]

Benno Eggnauer can you eventually provide a pull request?





[DDC-3078] Doctrine\ORM\Cache::__construct is in an interface Created: 10/Apr/14  Updated: 17/Apr/14  Resolved: 17/Apr/14

Status: Resolved
Project: Doctrine 2 - ORM
Component/s: None
Affects Version/s: Git Master
Fix Version/s: 2.5
Security Level: All

Type: Bug Priority: Blocker
Reporter: Marco Pivetta Assignee: Marco Pivetta
Resolution: Fixed Votes: 0
Labels: cache, config, second-level-cache

Issue Links:
Dependency
depends on DDC-3080 [GH-1008] DDC-3078 SLC Cache interfac... Resolved

 Description   

CTOR in the interface is a huge problem. This absolutely needs to be fixed before 2.5 is released, or we will have trouble in future.

I'm writing a PoC patch right now.



 Comments   
Comment by Fabio B. Silva [ 17/Apr/14 ]

Fixed by https://github.com/doctrine/doctrine2/commit/6af3236ba6f285fe14763866b20ddc1085e6ea39





[DDC-2570] Doctrine CLI Tools - Clear All Cache Created: 24/Jul/13  Updated: 08/Feb/14

Status: Awaiting Feedback
Project: Doctrine 2 - ORM
Component/s: ORM, Tools
Affects Version/s: 2.3.4
Fix Version/s: 2.x, 2.5

Type: Improvement Priority: Minor
Reporter: Frederick Marcoux Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: Cli, cache, orm


 Description   

It would be nice to be able to clear all cache one shot instead of clearing them one after one...

Like this:

root$ ./doctrine orm:clear-cache:all

Instead of:

root$ ./doctrine orm:clear-cache:metadata
root$ ./doctrine orm:clear-cache:result
root$ ./doctrine orm:clear-cache:query






[DDC-2164] Extend the cache support to eAccelerator Created: 23/Nov/12  Updated: 26/Nov/12

Status: Open
Project: Doctrine 2 - ORM
Component/s: ORM
Affects Version/s: 2.4, 3.0
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Enea Bette Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: cache, drivers


 Description   

It would be nice if the Doctrine caching drivers would support the eAccelerator library.



 Comments   
Comment by Marco Pivetta [ 23/Nov/12 ]

Enea Bette eAccelerator is known for being stripping comments from cached source (making it impossible to use annotations)... Do you happen to know if this is fixed? Supporting it as cache driver is fine btw, I just wonder how many users will start thinking of using eAccelerator and then will be facing this huge limitation.

Comment by Enea Bette [ 26/Nov/12 ]

I know that eAccelerator has this issue. It would be nice if we could utilize it with XML, YML and PHP based mapping though.
Do you know if the same problem would appear with these kinds of mapping strategies?

To give response to your question (eAccelerator and annotations incompatibility), there is a pull request on github, https://github.com/eaccelerator/eaccelerator/issues/19 .

It seems that in the future these could be resolved, and at that time it would be very nice to have that supported with doctrine (symfony2 already has support for this library).

"I just wonder how many users will start thinking of using eAccelerator and then will be facing this huge limitation". Sometimes users just does not have a choice. Imagine the case when you have a hosted site that requires caching functionalities and the only available cache library is eAccelerator (as just in my case). You would be fried as a chicken hehe





[DDC-2155] problem with DQL and cache Created: 18/Nov/12  Updated: 25/Nov/12  Resolved: 25/Nov/12

Status: Resolved
Project: Doctrine 2 - ORM
Component/s: DQL, ORM
Affects Version/s: 2.3
Fix Version/s: None
Security Level: All

Type: Bug Priority: Critical
Reporter: gabriel sancho Assignee: Benjamin Eberlei
Resolution: Invalid Votes: 0
Labels: cache, dql
Environment:

linux , php 5.4.8, mysql 5.5.28



 Description   

I have a problem when I get database records through a query DQL
and then they are changed by another application
If I repeat the query, Doctrine always return the first value, not the current value of the base

<?php

// bootstrap

$cache = new \Doctrine\Common\Cache\ArrayCache();
$config = new Doctrine\ORM\Configuration();
$config->setQueryCacheImpl($cache);
$conn = array(
				'dbname' => $database_name,
				'user' => $cnx_user,
				'password' => $cnx_pass,
				'host' => $cnx_host,
				'driver' => $cnx_type,
				'charset' => 'utf8',
				'driverOptions' => array( 1002 => "SET NAMES 'utf8'" )
				);

			$em = Doctrine\ORM\EntityManager::create($conn, $config);




while(true){
   $dql = "SELECT s from Register s WHERE s.id = 1";
   $query = $em->createQuery($dql);
// the next line is optional, produces same result
   $query->useResultCache(false);
   $res = $query->getResult();
   $orm = reset($res);
   	
   echo " regiter id :".$orm->getId()."  field "$orm->getText()."\n";

}

I run this code in a terminal, and then edit the registry (field text), but the terminal still shows the same result



 Comments   
Comment by Benjamin Eberlei [ 25/Nov/12 ]

Doctrine uses an IdentityMap pattern which leads to this issue.

You need to call "EntityManager#clear()" to clean the in memory cache of Doctrine and fetch records from the database again. or call "EntityManager#refresh($entity)"





Generated at Sun Nov 23 02:23:05 UTC 2014 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.