Doctrine Project

[DC-963] Doctrine cache - Salt dissociation

Mon, 18 Apr 2011 10:13:28 +0000

Doctrine Cache store data into a persistence storage.

Regarding APC, Doctrine use a share storage.
Doctrine is able to cache SQL from DQL, and DOM from SQL.
To do this, Doctrine create a DQL hash, and store the SQL result refer to the hash.

I'm using a server to host two Doctrine project, a preproduction & production Website. In some case, DQL is the same on both project, but the data model definition isn't.

Preproduction convert DQL to SQL using data model definition, and store the SQL result into APC cache refer to the DQL hash.
Production create a DQL hash, this is the same hash as preproduction.. So production instance use the SQL refer to the preproduction..

I'm not sure about the quality of this explanation... But I can add some information is needed.

The solution of this problem is easy. Just add a SALT to any cache id's. It's a Doctrine_Cache problem, not only a Doctrine_Cache_APC problem..
For Query cache it's not really important because this cache is optional, but for result cache.. It's more critical.
The SALT can be define when instantiate the Doctrine_Cache object, it's just an option..

$cacheDriver = new doctrine_Cache_Apc ();
$cacheDriver->setOption ("salt", "domain.tld");

[DC-644] _getCacheKeys() exhausts memory

Wed, 6 Jul 2011 08:17:14 +0000

My scripts have excessive memory consumption and I've often saw in my logs:

PHP Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 2097152 bytes) in /proj/lib/vendor/symfony/lib/plugins/sfDoctrinePlugin/lib/vendor/doctrine/Doctrine/Cache/Apc.php on line 111

Looking into the code I've found which function to blame:

protected function _getCacheKeys()
{
$ci = apc_cache_info('user');
$keys = array();

foreach ($ci['cache_list'] as $entry)

Unknown macro: { $keys[] = $entry['info']; ######### THIS IS THE LINE }

return $keys;
}

My server extensively uses APC caching and it's normal to have many cache keys.
Obviously retrieving ALL of them is time and memory consuming.
As I'm not well versed with Doctrine's code, I didn't want to dive further in.

Is there another way to avoid this pitfall?

[DC-902] Xcache Cache Driver is not documented

Tue, 26 Oct 2010 14:38:59 +0000

Xcache Cache Driver is not documented at all. Is it working? Is it stable? Can we use it?

[DC-999] Query cache key can be incorrectly generated

Mon, 27 Jun 2011 18:20:20 +0000

1. We have two versions of the application on the same server.
2. Second application has an updated database. New field is added to one of the models.
3. When the second app is hit first, query is stored in APC.
4. First app finds cached query and tries to call it. Exception is thrown as it doesn't know anything about the new field yet.

Situation often happens on shared development machine when one developer adds a field but others don't have in their models yet. It also happens on staging server if it's shared with production.

I suspect it only affects queries without explicitly listed fields.

To quickly fix the issue in my symfony project I extended Doctrine_Cache_Apc to implement namespaces (https://gist.github.com/944524). More appropriate place to fix it would be Doctrine_Query_Abstract::calculateQueryCacheHash().

[DC-390] doctrine_cache_keys bigger than 1mb

Reference — Wed, 27 Jan 2010 23:33:01 +0000

When multiple projects using doctrine use the same memcached server the doctrine_cache_keys can get bigger then 1mb and php will be throwing warnings telling that the data is too big to be stored by memcached.

I am currently using a workaround, having 1 doctrine_cache_keys per application, on the file Cache/Driver.php i added the following line to the constructor

$this->_cacheKeyIndexKey = 'doctrine_cache_keys_' . md5(Doctrine_Manager::getInstance()->getCurrentConnection()->getOption('dsn'));

but if an application uses more than 1mb en keys then the notice will rise again.

[DC-389] query cache doesn't cache _isLimitSubqueryUsed

Mon, 27 Jun 2011 18:06:25 +0000

The problem is that _isLimitSubqueryUsed is not cached with query cache.
It gets calculated when building query, but when the query is coming
from cache it's not.
Because of this, line 1087 of Query/Abstract.php is never executed,
when coming from cache:
if ($this->isLimitSubqueryUsed() &&
$this->_conn->getAttribute(Doctrine::ATTR_DRIVER_NAME) !==
'mysql')

{ $params = array_merge((array) $params, (array) $params); } Maybe it is on purpose, but I didn't get any answer on the google-groups. Here is a diff I use now: Index: trunk/gui/doctrine-library/Doctrine/Query/Abstract.php =================================================================== --- a/trunk/gui/doctrine-library/Doctrine/Query/Abstract.php +++ b/trunk/gui/doctrine-library/Doctrine/Query/Abstract.php @@ -1286,4 +1286,5 @@ $cached = unserialize($cached); $this->_tableAliasMap = $cached[2]; + $this->_isLimitSubqueryUsed = $cached[3]; $customComponent = $cached[0]; @@ -1346,5 +1347,5 @@ }

return serialize(array($customComponent, $componentInfo,
$this->getTableAliasMap()));
+ return serialize(array($customComponent, $componentInfo,
$this->getTableAliasMap(), $this->isLimitSubqueryUsed()));
}

[DC-829] Hydrator/RecordDriver/setLastElement And APC useResultCache

Mon, 16 Aug 2010 17:02:25 +0000

1) I have a Query with APC useResultCache with leftJoin between User and Avatar. (With no avatar for this User).
2) I have au Query without APC useResultCache with leftJoin between Gallery, User and Avatar (Same user than 1))

On the first show of my page => OK
On second show of my page (With APC Cache) error when i call user->getGalleries() :

In Hydrator/RecordDriver.php setLastElement()

$coll is instanceOf Doctrine_Null and count($coll) return 1, so => Crash because ->getLast doesn't exist in Doctrine_Null class

If you add in setLastElement() (Before if (count($coll) > 0) ....

if ($coll instanceOf Doctrine_Null)

{ return; }

It's works.

[DC-812] Postgresql and query cache

Fri, 6 Aug 2010 17:14:47 +0000

After enabling:

$manager->setAttribute(Doctrine_Core::ATTR_QUERY_CACHE, $cacheDriver);
(both Memcache and APC)

I get:

Doctrine_Connection_Pgsql_Exception: SQLSTATE[08P01]: <>: 7 ERROR: bind message supplies 8 parameters, but prepared statement "pdo_stmt_00000008" requires 16 in /home/taat/www/library/Doctrine/1.2.2/lib/Doctrine/Connection.php on line 1082

Without the cache everything works OK.
RESULT_CACHE works OK too.

Maybe this may help somehow:
http://bugs.php.net/33886

[DC-1001] Doctrine Caching page does not mention the "prefix" option

Mon, 27 Jun 2011 18:14:37 +0000

http://www.doctrine-project.org/documentation/manual/1_1/en/caching

When not using a prefix with multiple / yet similay projects a mixup of query caching will occur (for example with sfDoctrineGuard queries). This is very easialy fixed with a prefix - but I only realized after two months - that this feature actually existed. There is no mention of this in the documention.

A mention of the feature

$q = Doctrine_Query::create()
->useResultCache(new Doctrine_Cache_Apc(array('prefix'=>'myproject_')));

Would have helped me and I think it will also help others.