[DC-363] Multiple connections and i18n Created: 16/Dec/09  Updated: 28/Mar/12  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: Connection, I18n
Affects Version/s: 1.0.14, 1.2.2, 1.2.3
Fix Version/s: 1.2.3

Type: Bug Priority: Blocker
Reporter: Xav. Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 1
Labels: None
Environment:

MySQL 5.1.37
PHP 5.2.11
symfony 1.2.11 DEV

symfony 1.4.13
PHP 5.3.6
Postgres 8.4.8



 Description   

I used to work with a single database named "doctrine". The query was working properly.

I then decided to use 2 databases so I got my schema like this:

connection: doctrine

Category:
actAs:
I18n:
actAs:
Sluggable:
fields: [name]
Timestampable: ~
fields: [name, description]
columns:
id: ~
name:
type: string(255)
notnull: true
description: string

User:
connection: second
columns:
id: ~
name:
type: string(255)
notnull: true

I did setup my connections in config/databases.yml this way:

all:
doctrine:
// ....
second:
// ....

build-model, build-forms, build-filters and cc got ran. But now, I got an exception saying the "Translation" relation doesn't exist. The Base Models include correctly the bindComponent line:

Doctrine_Manager::getInstance()->bindComponent('Category', 'doctrine');

For now, I managed to kind of fixing it with simply swapping the databases order in my config/databases.yml and it's now working again perfectly.

I forgot to mention that in the CategoryTable when i call $this->getConnection()->getName(), it outputs "second"



 Comments   
Comment by Colin Darie [ 04/Feb/10 ]

I'm experiencing the same issue with 4 connections. The I18n behavior is almost unusable for models whose connection is not the last one defined. I searched for an acceptable solution but I haven't found one (IMHO the setting to the right connection before each query is not acceptable in large projects). I tried to do like in Search behavior, but it didn't work, I doesn't know enough doctrine internals to understand why.

Comment by Jonathan H. Wage [ 01/Mar/10 ]

Can you test this in Doctrine 1.2? I believe it is fixed.

Comment by Georg [ 17/Feb/11 ]

I am using the latest symfony 1.4 which is doctrine 1.2.3 afaik, and this bug still exists.
Try

actAs:
I18n:
fields: [name]
generateFiles: true
generatePath: /project/lib/model/doctrine/i18n

and the resulting model base class is not bound to the correct connection.

Comment by Joe Siponen [ 27/May/11 ]

I've just now battled with the very same problem in Doctrine 1.2 (the version bundled with symfony 1.4) and the problem seems to be caused by the fact that Doctrine_Record_Generator simply isn't written such that it is able to reinitialize generators for unloaded table instances after a connection is closed. This problem also manifests itself after a table has been loaded in a connection and one tries retrieve a table again after Doctrine_Connection->evictTables() has been called. This makes it impossible to to open more than one connection at a time in a request/script when using behaviors that dynamically modify table instances (such as the i18n behavior). The issue states that this has been fixed but I looked at the latest code and the problem still seems to be very much the same.

Doctrine_Record_Generator determines if it needs to run its initialization methods simply by checking if the to-be generated class, as defined by the className option, exists using a class_exists call. This means that the first time this method is called the initialization happens but for every subsequent call no initialization is made. Now, in the i18m behavior, the important initialization happens in its setTableDefinition method in which it removes any of the translated fields from the table instance that is been setup and redefines them as relations on the to-be-created Translation class. It then finishes off by dynamically declaring the new class for the translation record using its generateClassFromTable method.

Thus, the first time everything goes smoothly and the i18n generator's setTableDefinition is called and the table instance is properly initialized. Everything will now work as expected while the current connection is open since the connection instance keeps the i18n modified table instances alive and well for callers.

But, when the current connection is closed the i18n modified table instances it holds are also removed (goes out of scope). Then, when a new connection is opened, this new connection will start without having any table instances. This means that the next time one asks the new connection for a table instance of the same class with the i18n behavior the i18n behaviors will fail to initialize because the generator at this time believes its class has actually been initialized which, in turn, means that the table using the i18n behavior isn't properly initialized. No initialization means that this table will now include the non-existant i18n fields in the select part of its queries (those are in the translation table) causing those queries to fail miserably.

I believe this could be fixed by adding a static attribute to Doctrine_Record_Generator that tracks the spl_object_hash of the underlying dbh instance variable of the doctrine connection of the table parameter. If the hash is the same the next time that the initialize method is called the generator can decide not to reinitialize itself but if it detects that the hash of the current connection is different then that is definitely a clue to the generator that it needs to reinitialize itself (i.e. run all of the initialization methods but generateClassFromTable which should't be called more than once).

Maybe do it like this perhaps:

 
abstract class Doctrine_Record_Generator extends Doctrine_Record_Abstract
{
  public function initialize(Doctrine_Table $table)
  {
    /* ... */ 
  
    $currentConnectionHash = spl_object_hash($table->getConnection()->getDbh());
    
    //Next part is called if this is the first connection made or if this is a new open connection with new table instances
    if ($currentConnectionHash != self::$lastConnectionHash)
    {
      self::$lastConnectionHash = $currentConnectionHash;
      
      $this->buildTable();

      $fk = $this->buildForeignKeys($this->_options['table']);

      $this->_table->setColumns($fk);

      $this->buildRelation();

      $this->setTableDefinition();
      $this->setUp();
      
      if ($this->_options['generateFiles'] === false && class_exists($this->_options['className'])) {
        $this->generateClassFromTable($this->_table); //Don't generate the class more than once ever
      }
      
      $this->buildChildDefinitions();

      $this->_table->initIdentifier();
    }
  }
}
Comment by James Bell [ 23/Aug/11 ]

I'm also experiencing this issue, using the stable version of Symfony 1.4.13. If I define multiple database connections, the i18n Translation relations fail with this call: Doctrine_Relation_Parser->getRelation('Translation', )

'Unknown relation alias Translation'

dev:
mysql1:
class: sfDoctrineDatabase
param:
dsn: 'mysql:host=localhost;dbname=microscooters'
username: microuser_uk
password: sailing

mysql2:
class: sfDoctrineDatabase
param:
dsn: 'mysql:host=localhost;dbname=microscooters_ie'
username: microuser_ie
password: windy

postgresql:
class: sfDoctrineDatabase
param:
dsn: 'pgsql:host=localhost;dbname=mses6'
username: postgres
password: postgres

In this case, the primary connection is the postgresql one, and that is where the i18n behaviour is defined:

Category:
actAs:
Timestampable: ~
Auditable: ~
NestedSet: ~
I18n:
fields: [picture_id, sort_type, name, handle, subheading, breadcrumb, description, enabled, meta_title, meta_keywords, meta_description]
i18nField: culture
length: 5
actAs:
Timestampable: ~
Auditable: ~
...

I tried to implement the suggest above (ie adding a static hash of the database handle to the Doctrine_Record_Generator class file, which does clear out the connections. However, I then have difficulty with Doctrine recognizing CategoryTranslation as a class:

"( ! ) Fatal error: Class 'CategoryTranslation' not found in /sitename/lib/vendor/symfony/lib/plugins/sfDoctrinePlugin/lib/vendor/doctrine/Doctrine/Table.php on line 545"

Is there anything else I can to do test/fix this?

Comment by Joe Siponen [ 23/Aug/11 ]

This is a duplicate of http://www.doctrine-project.org/jira/browse/DC-373. I've also added a failing test case to that issue that should reproduce the issue as described here.

Comment by Andy.L [ 28/Mar/12 ]

still exists in 1.2.4





[DC-203] MsSQL TIMESTAMPS Fail Created: 09/Nov/09  Updated: 23/Nov/09  Resolved: 23/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: Record
Affects Version/s: 1.2.0-BETA1
Fix Version/s: None

Type: Bug Priority: Blocker
Reporter: Trevor Lanyon Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None
Environment:

Doctrine Version: 1.2.0-BETA1 on Symfony 1.3 - BETA
pdo_dblib / MsSQL
Linux



 Description   

sfDoctrineRecord.class.php incorrectly attempts to parse dates from the MsSQL datetime field.

MsSQL dates return in the following format:

Oct 12 2009 12:16:22:000AM



 Comments   
Comment by Jonathan H. Wage [ 16/Nov/09 ]

I don't understand your issue. Can you provide some more information?

Comment by Trevor Lanyon [ 16/Nov/09 ]

After reviewing this might be a Symfony issue and not a doctrine issue as the problem is with this class. I'll explain this better in case:

lib/plugins/sfDoctrinePlugin/lib/record/sfDoctrineRecord.class.php

It has this method:

public function getDateTimeObject($dateFieldName)
{
$type = $this->getTable()->getTypeOf($dateFieldName);
if ($type == 'date' || $type == 'timestamp')

{ return new DateTime($this->get($dateFieldName)); }

else

{ throw new sfException('Cannot call getDateTimeObject() on a field that is not of type date or timestamp.'); }

}

Microsoft SQL server returns TIMESTAMPS in a ridiculous format like this "Nov 16 2009 12:08:44:000AM" so this method breaks. Easy fix but a bug nonetheless.

Again, this doesn't belong here does it, it's a symfony thing?

Comment by Jonathan H. Wage [ 16/Nov/09 ]

I don't think this is something that can be "fixed". the DateTime object uses strtotime() and is mssql provides a date it doesn't understand, we can't do much about it. I think you need to configure your mssql server to return a date format that php can parse properly.

Comment by Trevor Lanyon [ 17/Nov/09 ]

I don't want to be working with Mssql (who would?). I can't change the database or the configuration of the database, other systems rely on the stuff.

There are some serious problems with trying to use Doctrine with MsSQL. PDO for mssql (pdo_dblib) doesn't let you indicate columns more then 30 characters. With the aliasing of columns scheme some columns in my tables go over that and Doctrine thinks it needs to hydrate all of the objects I return because it sees a null (pragmatic: I didn't trace the code to find that, only trial and error). And now it looks like I'll have to keep the modification I made to the code to work with dates (which means I will without exception run into further problems) .

I suppose I'll have to crawl back to my propel code base. With the no real composite key support and no interest in supporting MsSQL I'll have a hard time explaining the choice to anyone else in my group.

Awesome tool if I'm building something out of the box and I can design the environment for it's needs. Too many problems if you have to wrap it around something existing. Good luck! I look forward to future releases. Thank you for your responses!

Comment by Jonathan H. Wage [ 17/Nov/09 ]

I'm open to any suggestions to fix the issue. Before we can really say mssql works well in Doctrine, we'll really need to thoroughly test it against a mssql server. Currently none of us have one.

Comment by Trevor Lanyon [ 23/Nov/09 ]

In the newest version of Symfony, Doctrine (and php 5.3.x) this is no longer a problem.

There is a new problem now (http://trac.symfony-project.org/ticket/7676) but I thought i was probably more a symfony problem.

I'm not ready to throw the towel in just yet.

In response to your need for a mssql server: I have a clustered environment that I have to work with. If I can do any testing please let me know. I would very much like to contribute to Doctrine's success.

Thank you for your help.





[DC-83] Doctrine_Record::fromArray() calls protected methods with wrong arguments when dealing with relations. Created: 06/Oct/09  Updated: 07/Oct/09  Resolved: 07/Oct/09

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Blocker
Reporter: Marc Weistroff Assignee: Jonathan H. Wage
Resolution: Cannot Reproduce Votes: 0
Labels: None
Environment:

php 5.3 / symfony 1.3


Attachments: File DC83TestCase.php    

 Description   

Hi there.

This a bug related with Symfony but I think the problem comes from Doctrine so I post it here.
When trying to update an object with this array:

{{
var_dump($values)
array
'id' => string '14' (length=2)
'foo' => null
'bar_relation' =>
array
'id' => string '15' (length=2)
'title' => string '' (length=0)
'bar_other_relation' =>
array
'id' => string '16' (length=2)
'title' => string '' (length=0)
}}

When calling
{{
$object->fromArray($values):
}}

I'l have this error message:
{{
Call to a member function getTable() on a non-object in [...]/Doctrine/Record.php on line 1511.
}}
And the stack trace
{{
sfFormDoctrine->doSave( ) ../sfFormDoctrine.class.php:218
sfFormDoctrine->updateObject( ) ../sfFormDoctrine.class.php:415
sfFormDoctrine->doUpdateObject( ) ../sfFormDoctrine.class.php:249
Doctrine_Record->fromArray( ) ../sfFormDoctrine.class.php:267
MyModel->setBarRelation( ) ../Record.php:1923
sfDoctrineRecord->__call( ) ../sfDoctrineRecord.class.php:0
call_user_func_array ( ) ../sfDoctrineRecord.class.php:204
Doctrine_Record->set( ) ../sfDoctrineRecord.class.php:0
Doctrine_Record->_set( ) ../Record.php:1382
Doctrine_Record->coreSetRelated( ) ../Record.php:1426
}}

Doctrine_Record->coreSetRelated is called with an array instead of a proper Doctrine_Record, and Doctrine try to call ->getTable on this array, causing the error.

Is this a symfony problem or a doctrine one?



 Comments   
Comment by Jonathan H. Wage [ 06/Oct/09 ]

When I try and reproduce this in a test case with your example it doesn't behave that way. Can you try and make a failing test case for us to look at?

Comment by Marc Weistroff [ 06/Oct/09 ]

Thx for the prompt reply Jon. i'll do it asap

Comment by Marc Weistroff [ 07/Oct/09 ]

I can't reproduce the case and I found a workaround by changing the name of my embed forms in symfony.

Comment by Marc Weistroff [ 07/Oct/09 ]

I attach the test case I tried to work on even if it DOES NOT reproduce the problem.





[DC-594] When using a combination of: a group by field referencing a table in a relation, a join to a different table via a many type relation and a limit clause, doctrine creates a broken query then throws an exception Created: 22/Mar/10  Updated: 08/Jun/10  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: Query, Relations
Affects Version/s: 1.2.2
Fix Version/s: 1.2.3

Type: Bug Priority: Blocker
Reporter: will ferrer Assignee: Guilherme Blanco
Resolution: Fixed Votes: 0
Labels: None
Environment:

XP Xamp Current



 Description   

Hi All

I have run into a very problematic Doctrine 1.2.2 bug which puts me in quite a bit of danger of having to rewrite my whole app in Doctrine 2 (hopefully if this is a bug it isn't present in Doctrine 2).

The problem I am running into seems like something that probably would have been found and rectified however so hopefully there is something wrong in my execution.

The problem I am running into is caused when I have a combination of the following 3 things in my query:
1) A group by field referencing a table in a relation
2) A join to a different table via a many type relation
3) A limit clause

This combination causes Doctrine to create a broken SQL query which it then throws an exception about when I try to execute the query or call getSqlQuery() on it.

Using some sample data I put together some very simple examples to illustrate the problem:

$q = Doctrine_Query::create(); 
$q->from('Customer Customer'); 
$q->leftJoin('Customer.Order Order'); 
$q->leftJoin('Customer.Zip Zip'); 
$q->addGroupBy('Zip.city');
$q->addSelect('Zip.city as city');
$q->addSelect('Customer.customer_id'); 
$q->addSelect('Order.order_id');
$q->offset(0);
$q->limit(5);

This creates the following dql:

SELECT Zip.city as city, Customer.customer_id, Order.order_id FROM Customer Customer LEFT JOIN Customer.Order Order LEFT JOIN Customer.Zip Zip GROUP BY Zip.city LIMIT 5 OFFSET 0

However when I attempt to run $q->getSqlQuery() an exception is thrown:
{"type":"exception","tid":3,"exception":{},"message":"SQLSTATE[42S22]: Column not found: 1054 Unknown column 'z2.city' in 'group statement'. Failing Query: \"SELECT DISTINCT c2.customer_id FROM customers c2 GROUP BY z2.city LIMIT 5\"","where":"#0 C:\\htdocs\\php_library\\Doctrine-1.2.1\\lib\\Doctrine
Connection.php(1025): Doctrine_Connection->rethrowException(Object(PDOException), Object(Doctrine_Connection_Mysql), 'SELECT DISTINCT...')\n#1 C:\\htdocs\\php_library\\Doctrine-1.2.1\\lib\\Doctrine
Query.php(1263): Doctrine_Connection->execute('SELECT DISTINCT...', Array)\n#2 C:\\htdocs\\php_library\\Doctrine-1.2.1\\lib\\Doctrine
Query.php(1122): Doctrine_Query->buildSqlQuery(true)\n#3............

As you can see from the SQL in the exception Doctrine is trying to create a query that groups by a field from the Zip table with out first joining to it (SELECT DISTINCT c2.customer_id FROM customers c2 GROUP BY z2.city LIMIT 5).

I know Doctrine works some magic to get limit statements to work with joins and I suspect that something with in that magic may be broken, but hopefully its something I am doing wrong.

Take out any one of the 3 things I mentioned I above and everything works fine – the following all work:

Remove the limit:

$q = Doctrine_Query::create(); 
$q->from('Customer Customer'); 
$q->leftJoin('Customer.Order Order'); 
$q->leftJoin('Customer.Zip Zip'); 
$q->addGroupBy('Zip.city'); 
$q->addSelect('Zip.city as city');
$q->addSelect('Order.order_id');
$q->addSelect('Customer.customer_id'); 

Remove the additional join:

$q = Doctrine_Query::create(); 
$q->from('Customer Customer'); 
$q->leftJoin('Customer.Zip Zip'); 
$q->addGroupBy('Zip.city'); 
$q->addSelect('Zip.city as city');
$q->addSelect('Customer.customer_id');
$q->offset(0);
$q->limit(5);

Remove the group by:

$q = Doctrine_Query::create(); 
$q->from('Customer Customer'); 
$q->leftJoin('Customer.Order Order'); 
$q->leftJoin('Customer.Zip Zip'); 
$q->addSelect('Zip.city as city');
$q->addSelect('Customer.customer_id'); 
$q->addSelect('Order.order_id');
$q->offset(0);
$q->limit(5);

Its also worth noting that the following changes also stop the problem from happening:
Changing the relation to the Order table to be a one relation instead of a many relation.
Changing the group by to a field located in the "from" table (such as: $q->addGroupBy('Customer.customer_id')

Here are the relevant parts of my sample data schema:

detect_relations: false
package: Example
options:
  type: INNODB
  charset: utf8
Order:
  tableName: orders
  columns:
    order_id:
      type: integer(4)
      primary: true
      notnull: true
    customer_id:
      type: integer(4)
    order_date: timestamp
  relations:
    OrderItem:
      type: many
      local: order_id
      foreign: order_id
    Customer:
      type: one
      local: customer_id
      foreign: customer_id
  options:
    type: InnoDB
Customer:
  tableName: customers
  columns:
    customer_id:
      type: integer(4)
      primary: true
      notnull: true
      autoincrement: true
    firstname:
      type: string(45)
    lastname:
      type: string(45)
    streetaddress:
      type: string(45)
    city:
      type: string(45)
    state:
      type: string(45)
    postalcode:
      type: string(45)
  relations:
    Order:
      type: many
      local: customer_id
      foreign: customer_id
    Zip:
      type: one
      local: postalcode
      foreign: postalcode
  options:
    type: InnoDB
Zip:
  connection: default_schema
  tableName: zips
  columns:
    postalcode:
      type: varchar(30)
      primary: true
    latitude: 'float(10,6)'
    longitude: 'float(10,6)'
    city: string(50)
    state: string(50)
    country: string(50)
    type: string(50)
  relations:
    Customer:
      type: many
      local: postalcode
      foreign: postalcode

Thank for any advice or information you can give me on this.

Best regards

Will Ferrer



 Comments   
Comment by will ferrer [ 24/Mar/10 ]

I realized another crucial aspect of the problem. I am using a left join to my Zip table instead of using an inner join. When I change my code to do an inner join it works again:

$q = Doctrine_Query::create(); 
$q->from('Customer Customer'); 
$q->leftJoin('Customer.Order Order'); 
$q->innerJoin('Customer.Zip Zip'); 
$q->addGroupBy('Zip.city');
$q->addSelect('Zip.city as city');
$q->addSelect('Customer.customer_id'); 
$q->addSelect('Order.order_id');
$q->offset(0);
$q->limit(5);

Thanks much in advance.

Will Ferrer

Comment by will ferrer [ 03/Apr/10 ]

I took a look at the doctrine 1.2.2 code to try to track down what was causing this bug and I think I have found and fixed it in my copy of the code base.

The problem is on line 1459 of Doctrine_Query and looks like it was just an oversight. The code was checking if it should preserve left joins while generating the subquery based on whether or not there were any orderBys, wheres, or havings added to the query. I changed the code to also watch for groupBys and it seems to have resolved this issue.

The code was:

 if (empty($this->_sqlParts['orderby']) && empty($this->_sqlParts['where']) && empty($this->_sqlParts['having'])) {

I changed it to:

if (empty($this->_sqlParts['orderby']) && empty($this->_sqlParts['where']) && empty($this->_sqlParts['having']) && empty($this->_sqlParts['groupby'])) {

Please let me know if I am over looking anything. If I am not then this change should probably be added to the next revision of doctrine.

Sincerely

Will Ferrer





[DC-578] Severe documentation omission Created: 16/Mar/10  Updated: 29/Mar/10  Resolved: 29/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Documentation
Affects Version/s: 1.2.0
Fix Version/s: None

Type: Improvement Priority: Blocker
Reporter: Danny Kopping Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None


 Description   

I wanted to give you guys a heads-up about a fairly critical missing line of code from the "Getting Started" docs... From what i can tell, the

spl_autoload_register(array('Doctrine_Core', 'modelsAutoload'));

statement is not mentioned anywhere in the docs and for beginners (like myself) who start to experiment with creating tables from YAML schemas, this is a dead-end since the models do not load and hence no tables are generated (without a warning i might add).

After some fierce Googling, i managed to find that critical line of code, popped it in and Bob's my father's brother... For newbies i fear that this might be a dealbreaker.
Thanks



 Comments   
Comment by Jonathan H. Wage [ 29/Mar/10 ]

This is mentioned in the Introduction to Models chapter now.





[DC-483] Indentifiers are not quoted in queries build for Nested Relations (patch included) Created: 08/Feb/10  Updated: 01/Mar/10  Resolved: 01/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Native SQL, Query, Relations
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Blocker
Reporter: Elnur Abdurrakhimov Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None
Environment:

Ubuntu, PHP, MySQL


Attachments: File patch.diff    

 Description   

It was OK using reserved words as column names before I started using Nested Relations: queries started to fail because Doctrine_Core::ATTR_QUOTE_IDENTIFIER is ignored by query builder for Nested Relations.

I solved this problem by wrapping identifiers with Doctrine_Formatter::quoteIdentifier() in several places. I'm providing a patch for your to review and commit.

Thanks.






[DC-717] Default value for columns is generated wrong. THe default value becomes the result value in the schema files. Created: 05/Jun/10  Updated: 06/Jun/10  Resolved: 06/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: Schema Files
Affects Version/s: 1.2.2
Fix Version/s: 1.2.2

Type: Bug Priority: Blocker
Reporter: angelo ayres camargo Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None
Environment:

Apache 2.2, PostgresSQL 8.4, Windows XP



 Description   

When generating models from db with doctrine 1.2.2, default with functions in database get evaluated and the value goes to the generated schema files thus cousing error on insert/update.

Example:
Table:
CREATE TABLE Usuario (
id SERIAL NOT NULL,
apelido CHARACTER VARYING(50) NOT NULL,
email CHARACTER VARYING(255) NOT NULL,
dtRegistro TIMESTAMP DEFAULT 'now()' NOT NULL,
idUsuarioSituacao INTEGER,
senha CHARACTER(32),
CONSTRAINT PK_Usuario PRIMARY KEY (id),
CONSTRAINT TUC_Usuario_1 UNIQUE (email)
)

Code to generate files:
$dManager = Doctrine_Manager::getInstance();
$dManager->setAttribute(Doctrine::ATTR_MODEL_LOADING,Doctrine::MODEL_LOADING_CONSERVATIVE);
$dManager->setAttribute(Doctrine_Core::ATTR_MODEL_LOADING, Doctrine_Core::MODEL_LOADING_CONSERVATIVE);

$dManager->setAttribute(Doctrine::ATTR_PORTABILITY, Doctrine::PORTABILITY_NONE);
$dManager->setAttribute(Doctrine::ATTR_VALIDATE, Doctrine::VALIDATE_NONE);
$dManager->setAttribute(Doctrine::ATTR_AUTO_ACCESSOR_OVERRIDE, true);
$dManager->setAttribute(Doctrine::ATTR_AUTOLOAD_TABLE_CLASSES, true);
$dManager->setAttribute(Doctrine::ATTR_QUERY_LIMIT, Doctrine::LIMIT_ROWS);

Doctrine_Core::generateModelsFromDb('c:\temp', array('dbcon'), array('generateTableClasses' => true, 'classPrefix' => 'D'));

Resulting file:
abstract class DBaseUsuario extends Doctrine_Record
{
public function setTableDefinition()

{ $this->setTableName('usuario'); $this->hasColumn('id', 'integer', 4, array( 'type' => 'integer', 'length' => 4, 'fixed' => false, 'unsigned' => false, 'primary' => true, 'sequence' => 'usuario_id', )); $this->hasColumn('apelido', 'string', null, array( 'type' => 'string', 'fixed' => false, 'unsigned' => false, 'notnull' => true, 'primary' => false, )); $this->hasColumn('email', 'string', null, array( 'type' => 'string', 'fixed' => false, 'unsigned' => false, 'notnull' => true, 'primary' => false, )); $this->hasColumn('dtregistro', 'timestamp', null, array( 'type' => 'timestamp', 'fixed' => false, 'unsigned' => false, 'notnull' => true, 'default' => '\'2010-06-04 11:10:12.078\'::timestamp without time zone', 'primary' => false, )); $this->hasColumn('idusuariosituacao', 'integer', 4, array( 'type' => 'integer', 'length' => 4, 'fixed' => false, 'unsigned' => false, 'notnull' => false, 'primary' => false, )); $this->hasColumn('senha', 'string', null, array( 'type' => 'string', 'fixed' => true, 'unsigned' => false, 'notnull' => false, 'primary' => false, )); }

}

The function now() for default value gets replaced by 'default' => '\'2010-06-04 11:10:12.078\'::timestamp without time zone',



 Comments   
Comment by angelo ayres camargo [ 06/Jun/10 ]

as of now i dont know if this is a error from case modeling tool or doctrine, will close and do further testing.





[DC-706] Subqueries do not work in select Created: 27/May/10  Updated: 08/Jun/10  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: 1.2.2
Fix Version/s: 1.2.3

Type: Bug Priority: Blocker
Reporter: will ferrer Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None
Environment:

XP, Xamp


Attachments: Text File Query.php.patch     Text File ticketDC706.patch    

 Description   

Hi All

I have found another bug in Doctrine 1.2.2.

Subqueries are not working in select statements at the moment.

The reason for this is that this line (found in the parseSelect function of the class Doctrine_Query) :

$componentAlias = $this->getExpressionOwner($expression);

Returns an unusable value when passed an $expression which is a subquery.

To fix this problem I have patched my version of the code to use an existing "$componentAlias" instead of the value that would be returned by this function when a subquery is encountered.

My code now reads:

if ($pos !== false && substr($term[0], 0, 1) !== "'" && substr($term[0], 0, $pos) == '') {
	$_queryComponents = $this->_queryComponents;
	reset($components);
	$componentAlias = key($_queryComponents);
} else {
        $componentAlias = $this->getExpressionOwner($expression);
}

I have not rigorously tested this patch yet but it has been working for me in what tests I have done so far.

I have posted several very large bugs into jira over the past few months and haven't heard back regarding them. This leads me to believe that the Doctrine team has moved on from Doctrine 1.2.2 and is focusing only on Doctrine 2 issues at this point. I still love version 1 and haven't had the heart (or the time) to migrate my code over to 2 yet. If this is the case then the job of patching bugs like the ones I have reported in 1.2.2 is probably up to us users at this point. As such I will post my patched version of Doctrine_Query in a comment to this bug (My patched version also fixes another bug I reported: DC-594) .

Best Regards

Will Ferrer



 Comments   
Comment by will ferrer [ 27/May/10 ]

Here is the whole of my Doctrine_Query posted for any one who wants to use the bug patches I put into the file (they are both marked with a comment that says: "Patched By Will Ferrer"

<?php
/*
 *  $Id: Query.php 7490 2010-03-29 19:53:27Z jwage $
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
 * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 *
 * This software consists of voluntary contributions made by many individuals
 * and is licensed under the LGPL. For more information, see
 * <http://www.doctrine-project.org>.
 */

/**
 * Doctrine_Query
 * A Doctrine_Query object represents a DQL query. It is used to query databases for
 * data in an object-oriented fashion. A DQL query understands relations and inheritance
 * and is dbms independant.
 *
 * @package     Doctrine
 * @subpackage  Query
 * @license     http://www.opensource.org/licenses/lgpl-license.php LGPL
 * @link        www.doctrine-project.org
 * @since       1.0
 * @version     $Revision: 7490 $
 * @author      Konsta Vesterinen <kvesteri@cc.hut.fi>
 * @todo        Proposal: This class does far too much. It should have only 1 task: Collecting
 *              the DQL query parts and the query parameters (the query state and caching options/methods
 *              can remain here, too).
 *              The actual SQL construction could be done by a separate object (Doctrine_Query_SqlBuilder?)
 *              whose task it is to convert DQL into SQL.
 *              Furthermore the SqlBuilder? can then use other objects (Doctrine_Query_Tokenizer?),
 *              (Doctrine_Query_Parser(s)?) to accomplish his work. Doctrine_Query does not need
 *              to know the tokenizer/parsers. There could be extending
 *              implementations of SqlBuilder? that cover the specific SQL dialects.
 *              This would release Doctrine_Connection and the Doctrine_Connection_xxx classes
 *              from this tedious task.
 *              This would also largely reduce the currently huge interface of Doctrine_Query(_Abstract)
 *              and better hide all these transformation internals from the public Query API.
 *
 * @internal    The lifecycle of a Query object is the following:
 *              After construction the query object is empty. Through using the fluent
 *              query interface the user fills the query object with DQL parts and query parameters.
 *              These get collected in {@link $_dqlParts} and {@link $_params}, respectively.
 *              When the query is executed the first time, or when {@link getSqlQuery()}
 *              is called the first time, the collected DQL parts get parsed and the resulting
 *              connection-driver specific SQL is generated. The generated SQL parts are
 *              stored in {@link $_sqlParts} and the final resulting SQL query is stored in
 *              {@link $_sql}.
 */
class Doctrine_Query extends Doctrine_Query_Abstract implements Countable
{
    /**
     * @var array  The DQL keywords.
     */
    protected static $_keywords  = array('ALL',
                                         'AND',
                                         'ANY',
                                         'AS',
                                         'ASC',
                                         'AVG',
                                         'BETWEEN',
                                         'BIT_LENGTH',
                                         'BY',
                                         'CHARACTER_LENGTH',
                                         'CHAR_LENGTH',
                                         'CURRENT_DATE',
                                         'CURRENT_TIME',
                                         'CURRENT_TIMESTAMP',
                                         'DELETE',
                                         'DESC',
                                         'DISTINCT',
                                         'EMPTY',
                                         'EXISTS',
                                         'FALSE',
                                         'FETCH',
                                         'FROM',
                                         'GROUP',
                                         'HAVING',
                                         'IN',
                                         'INDEXBY',
                                         'INNER',
                                         'IS',
                                         'JOIN',
                                         'LEFT',
                                         'LIKE',
                                         'LOWER',
                                         'MEMBER',
                                         'MOD',
                                         'NEW',
                                         'NOT',
                                         'NULL',
                                         'OBJECT',
                                         'OF',
                                         'OR',
                                         'ORDER',
                                         'OUTER',
                                         'POSITION',
                                         'SELECT',
                                         'SOME',
                                         'TRIM',
                                         'TRUE',
                                         'UNKNOWN',
                                         'UPDATE',
                                         'WHERE');

    /**
     * @var array
     */
    protected $_subqueryAliases = array();

    /**
     * @var array $_aggregateAliasMap       an array containing all aggregate aliases, keys as dql aliases
     *                                      and values as sql aliases
     */
    protected $_aggregateAliasMap      = array();

    /**
     * @var array
     */
    protected $_pendingAggregates = array();

    /**
     * @param boolean $needsSubquery
     */
    protected $_needsSubquery = false;

    /**
     * @param boolean $isSubquery           whether or not this query object is a subquery of another
     *                                      query object
     */
    protected $_isSubquery;

    /**
     * @var array $_neededTables            an array containing the needed table aliases
     */
    protected $_neededTables = array();

    /**
     * @var array $pendingSubqueries        SELECT part subqueries, these are called pending subqueries since
     *                                      they cannot be parsed directly (some queries might be correlated)
     */
    protected $_pendingSubqueries = array();

    /**
     * @var array $_pendingFields           an array of pending fields (fields waiting to be parsed)
     */
    protected $_pendingFields = array();

    /**
     * @var array $_parsers                 an array of parser objects, each DQL query part has its own parser
     */
    protected $_parsers = array();

    /**
     * @var array $_pendingJoinConditions    an array containing pending joins
     */
    protected $_pendingJoinConditions = array();

    /**
     * @var array
     */
    protected $_expressionMap = array();

    /**
     * @var string $_sql            cached SQL query
     */
    protected $_sql;

    /**
     * create
     * returns a new Doctrine_Query object
     *
     * @param Doctrine_Connection $conn  optional connection parameter
     * @param string $class              Query class to instantiate
     * @return Doctrine_Query
     */
    public static function create($conn = null, $class = null)
    {
        if ( ! $class) {
            $class = Doctrine_Manager::getInstance()
                ->getAttribute(Doctrine_Core::ATTR_QUERY_CLASS);
        }
        return new $class($conn);
    }

    /**
     * Clears all the sql parts.
     */
    protected function clear()
    {
        $this->_preQueried = false;
        $this->_pendingJoinConditions = array();
        $this->_state = self::STATE_DIRTY;
    }

    /**
     * Resets the query to the state just after it has been instantiated.
     */
    public function reset()
    {
        $this->_subqueryAliases = array();
        $this->_aggregateAliasMap = array();
        $this->_pendingAggregates = array();
        $this->_pendingSubqueries = array();
        $this->_pendingFields = array();
        $this->_neededTables = array();
        $this->_expressionMap = array();
        $this->_subqueryAliases = array();
        $this->_needsSubquery = false;
        $this->_isLimitSubqueryUsed = false;
    }

    /**
     * createSubquery
     * creates a subquery
     *
     * @return Doctrine_Hydrate
     */
    public function createSubquery()
    {
        $class = get_class($this);
        $obj   = new $class();

        // copy the aliases to the subquery
        $obj->copySubqueryInfo($this);

        // this prevents the 'id' being selected, re ticket #307
        $obj->isSubquery(true);

        return $obj;
    }

    /**
     * addPendingJoinCondition
     *
     * @param string $componentAlias    component alias
     * @param string $joinCondition     dql join condition
     * @return Doctrine_Query           this object
     */
    public function addPendingJoinCondition($componentAlias, $joinCondition)
    {
        if ( ! isset($this->_pendingJoinConditions[$componentAlias])) {
            $this->_pendingJoinConditions[$componentAlias] = array();
        }

        $this->_pendingJoinConditions[$componentAlias][] = $joinCondition;
    }

    /**
     * fetchArray
     * Convenience method to execute using array fetching as hydration mode.
     *
     * @param string $params
     * @return array
     */
    public function fetchArray($params = array())
    {
        return $this->execute($params, Doctrine_Core::HYDRATE_ARRAY);
    }

    /**
     * fetchOne
     * Convenience method to execute the query and return the first item
     * of the collection.
     *
     * @param string $params        Query parameters
     * @param int $hydrationMode    Hydration mode: see Doctrine_Core::HYDRATE_* constants
     * @return mixed                Array or Doctrine_Collection, depending on hydration mode. False if no result.
     */
    public function fetchOne($params = array(), $hydrationMode = null)
    {
        $collection = $this->execute($params, $hydrationMode);

        if (is_scalar($collection)) {
            return $collection;
        }

        if (count($collection) === 0) {
            return false;
        }

        if ($collection instanceof Doctrine_Collection) {
            return $collection->getFirst();
        } else if (is_array($collection)) {
            return array_shift($collection);
        }

        return false;
    }

    /**
     * isSubquery
     * if $bool parameter is set this method sets the value of
     * Doctrine_Query::$isSubquery. If this value is set to true
     * the query object will not load the primary key fields of the selected
     * components.
     *
     * If null is given as the first parameter this method retrieves the current
     * value of Doctrine_Query::$isSubquery.
     *
     * @param boolean $bool     whether or not this query acts as a subquery
     * @return Doctrine_Query|bool
     */
    public function isSubquery($bool = null)
    {
        if ($bool === null) {
            return $this->_isSubquery;
        }

        $this->_isSubquery = (bool) $bool;
        return $this;
    }

    /**
     * getSqlAggregateAlias
     *
     * @param string $dqlAlias      the dql alias of an aggregate value
     * @return string
     */
    public function getSqlAggregateAlias($dqlAlias)
    {
        if (isset($this->_aggregateAliasMap[$dqlAlias])) {
            // mark the expression as used
            $this->_expressionMap[$dqlAlias][1] = true;

            return $this->_aggregateAliasMap[$dqlAlias];
        } else if ( ! empty($this->_pendingAggregates)) {
            $this->processPendingAggregates();

            return $this->getSqlAggregateAlias($dqlAlias);
        } else if( ! ($this->_conn->getAttribute(Doctrine_Core::ATTR_PORTABILITY) & Doctrine_Core::PORTABILITY_EXPR)){
            return $dqlAlias;
        } else {
            throw new Doctrine_Query_Exception('Unknown aggregate alias: ' . $dqlAlias);
        }
    }

    /**
     * Check if a dql alias has a sql aggregate alias
     *
     * @param string $dqlAlias 
     * @return boolean
     */
    public function hasSqlAggregateAlias($dqlAlias)
    {
        try {
            $this->getSqlAggregateAlias($dqlAlias);
            return true;
        } catch (Exception $e) {
            return false;
        }
    }

    /**
     * Adjust the processed param index for "foo.bar IN ?" support
     *
     */
    public function adjustProcessedParam($index)
    {
        // Retrieve all params
        $params = $this->getInternalParams();

        // Retrieve already processed values
        $first = array_slice($params, 0, $index);
        $last = array_slice($params, $index, count($params) - $index);

        // Include array as values splicing the params array
        array_splice($last, 0, 1, $last[0]);

        // Put all param values into a single index
        $this->_execParams = array_merge($first, $last);
    }

    /**
     * Retrieves a specific DQL query part.
     *
     * @see Doctrine_Query_Abstract::$_dqlParts
     * <code>
     * var_dump($q->getDqlPart('where'));
     * // array(2) { [0] => string(8) 'name = ?' [1] => string(8) 'date > ?' }
     * </code>
     * @param string $queryPart     the name of the query part; can be:
     *     array from, containing strings;
     *     array select, containg string;
     *     boolean forUpdate;
     *     array set;
     *     array join;
     *     array where;
     *     array groupby;
     *     array having;
     *     array orderby, containing strings such as 'id ASC';
     *     array limit, containing numerics;
     *     array offset, containing numerics;
     * @return array
     */
    public function getDqlPart($queryPart)
    {
        if ( ! isset($this->_dqlParts[$queryPart])) {
           throw new Doctrine_Query_Exception('Unknown query part ' . $queryPart);
        }

        return $this->_dqlParts[$queryPart];
    }

    /**
     * contains
     *
     * Method to check if a arbitrary piece of dql exists
     *
     * @param string $dql Arbitrary piece of dql to check for
     * @return boolean
     */
    public function contains($dql)
    {
      return stripos($this->getDql(), $dql) === false ? false : true;
    }

    /**
     * processPendingFields
     * the fields in SELECT clause cannot be parsed until the components
     * in FROM clause are parsed, hence this method is called everytime a
     * specific component is being parsed. For instance, the wildcard '*'
     * is expanded in the list of columns.
     *
     * @throws Doctrine_Query_Exception     if unknown component alias has been given
     * @param string $componentAlias        the alias of the component
     * @return string SQL code
     * @todo Description: What is a 'pending field' (and are there non-pending fields, too)?
     *       What is 'processed'? (Meaning: What information is gathered & stored away)
     */
    public function processPendingFields($componentAlias)
    {
        $tableAlias = $this->getSqlTableAlias($componentAlias);
        $table = $this->_queryComponents[$componentAlias]['table'];

        if ( ! isset($this->_pendingFields[$componentAlias])) {
            if ($this->_hydrator->getHydrationMode() != Doctrine_Core::HYDRATE_NONE) {
                if ( ! $this->_isSubquery && $componentAlias == $this->getRootAlias()) {
                    throw new Doctrine_Query_Exception("The root class of the query (alias $componentAlias) "
                            . " must have at least one field selected.");
                }
            }
            return;
        }

        // At this point we know the component is FETCHED (either it's the base class of
        // the query (FROM xyz) or its a "fetch join").

        // Check that the parent join (if there is one), is a "fetch join", too.
        if ( ! $this->isSubquery() && isset($this->_queryComponents[$componentAlias]['parent'])) {
            $parentAlias = $this->_queryComponents[$componentAlias]['parent'];
            if (is_string($parentAlias) && ! isset($this->_pendingFields[$parentAlias])
                    && $this->_hydrator->getHydrationMode() != Doctrine_Core::HYDRATE_NONE
                    && $this->_hydrator->getHydrationMode() != Doctrine_Core::HYDRATE_SCALAR
                    && $this->_hydrator->getHydrationMode() != Doctrine_Core::HYDRATE_SINGLE_SCALAR) {
                throw new Doctrine_Query_Exception("The left side of the join between "
                        . "the aliases '$parentAlias' and '$componentAlias' must have at least"
                        . " the primary key field(s) selected.");
            }
        }

        $fields = $this->_pendingFields[$componentAlias];

        // check for wildcards
        if (in_array('*', $fields)) {
            $fields = $table->getFieldNames();
        } else {
            $driverClassName = $this->_hydrator->getHydratorDriverClassName();
            // only auto-add the primary key fields if this query object is not
            // a subquery of another query object or we're using a child of the Object Graph
            // hydrator
            if ( ! $this->_isSubquery && is_subclass_of($driverClassName, 'Doctrine_Hydrator_Graph')) {
                $fields = array_unique(array_merge((array) $table->getIdentifier(), $fields));
            }
        }

        $sql = array();
        foreach ($fields as $fieldName) {
            $columnName = $table->getColumnName($fieldName);
            if (($owner = $table->getColumnOwner($columnName)) !== null &&
                    $owner !== $table->getComponentName()) {

                $parent = $this->_conn->getTable($owner);
                $columnName = $parent->getColumnName($fieldName);
                $parentAlias = $this->getSqlTableAlias($componentAlias . '.' . $parent->getComponentName());
                $sql[] = $this->_conn->quoteIdentifier($parentAlias) . '.' . $this->_conn->quoteIdentifier($columnName)
                       . ' AS '
                       . $this->_conn->quoteIdentifier($tableAlias . '__' . $columnName);
            } else {
                $columnName = $table->getColumnName($fieldName);
                $sql[] = $this->_conn->quoteIdentifier($tableAlias) . '.' . $this->_conn->quoteIdentifier($columnName)
                       . ' AS '
                       . $this->_conn->quoteIdentifier($tableAlias . '__' . $columnName);
            }
        }

        $this->_neededTables[] = $tableAlias;

        return implode(', ', $sql);
    }

    /**
     * Parses a nested field
     * <code>
     * $q->parseSelectField('u.Phonenumber.value');
     * </code>
     *
     * @param string $field
     * @throws Doctrine_Query_Exception     if unknown component alias has been given
     * @return string   SQL fragment
     * @todo Description: Explain what this method does. Is there a relation to parseSelect()?
     *       This method is not used from any class or testcase in the Doctrine package.
     *
     */
    public function parseSelectField($field)
    {
        $terms = explode('.', $field);

        if (isset($terms[1])) {
            $componentAlias = $terms[0];
            $field = $terms[1];
        } else {
            reset($this->_queryComponents);
            $componentAlias = key($this->_queryComponents);
            $fields = $terms[0];
        }

        $tableAlias = $this->getSqlTableAlias($componentAlias);
        $table      = $this->_queryComponents[$componentAlias]['table'];


        // check for wildcards
        if ($field === '*') {
            $sql = array();

            foreach ($table->getColumnNames() as $field) {
                $sql[] = $this->parseSelectField($componentAlias . '.' . $field);
            }

            return implode(', ', $sql);
        } else {
            $name = $table->getColumnName($field);

            $this->_neededTables[] = $tableAlias;

            return $this->_conn->quoteIdentifier($tableAlias . '.' . $name)
                   . ' AS '
                   . $this->_conn->quoteIdentifier($tableAlias . '__' . $name);
        }
    }

    /**
     * getExpressionOwner
     * returns the component alias for owner of given expression
     *
     * @param string $expr      expression from which to get to owner from
     * @return string           the component alias
     * @todo Description: What does it mean if a component is an 'owner' of an expression?
     *       What kind of 'expression' are we talking about here?
     */
    public function getExpressionOwner($expr)
    {
        if (strtoupper(substr(trim($expr, '( '), 0, 6)) !== 'SELECT') {
            preg_match_all("/[a-z_][a-z0-9_]*\.[a-z_][a-z0-9_]*[\.[a-z0-9]+]*/i", $expr, $matches);

            $match = current($matches);

            if (isset($match[0])) {
                $terms = explode('.', $match[0]);

                return $terms[0];
            }
        }
        return $this->getRootAlias();

    }

    /**
     * parseSelect
     * parses the query select part and
     * adds selected fields to pendingFields array
     *
     * @param string $dql
     * @todo Description: What information is extracted (and then stored)?
     */
    public function parseSelect($dql)
    {
        $refs = $this->_tokenizer->sqlExplode($dql, ',');

        $pos   = strpos(trim($refs[0]), ' ');
        $first = substr($refs[0], 0, $pos);

        // check for DISTINCT keyword
        if ($first === 'DISTINCT') {
            $this->_sqlParts['distinct'] = true;

            $refs[0] = substr($refs[0], ++$pos);
        }

        $parsedComponents = array();

        foreach ($refs as $reference) {
            $reference = trim($reference);

            if (empty($reference)) {
                continue;
            }

            $terms = $this->_tokenizer->sqlExplode($reference, ' ');
            $pos   = strpos($terms[0], '(');

            if (count($terms) > 1 || $pos !== false) {
                $expression = array_shift($terms);
                $alias = array_pop($terms);

                if ( ! $alias) {
                    $alias = substr($expression, 0, $pos);
                }
				
			//Patched By Will Ferrer to detect when a subquery was encountered and use an existing componentAlias if the expression is a subquery. (not well tested) Fixes Bug -- DC-706
				if ($pos !== false && substr($term[0], 0, 1) !== "'" && substr($term[0], 0, $pos) == '') {
					$_queryComponents = $this->_queryComponents;
					reset($components);
					$componentAlias = key($_queryComponents);
				} else {
               		$componentAlias = $this->getExpressionOwner($expression);
				}
			//End Patch
			
				$expression = $this->parseClause($expression);

                $tableAlias = $this->getSqlTableAlias($componentAlias);
				
                $index    = count($this->_aggregateAliasMap);

                $sqlAlias = $this->_conn->quoteIdentifier($tableAlias . '__' . $index);

                $this->_sqlParts['select'][] = $expression . ' AS ' . $sqlAlias;

                $this->_aggregateAliasMap[$alias] = $sqlAlias;
                $this->_expressionMap[$alias][0] = $expression;

                $this->_queryComponents[$componentAlias]['agg'][$index] = $alias;
				
                $this->_neededTables[] = $tableAlias;
            } else {
                $e = explode('.', $terms[0]);

                if (isset($e[1])) {
                    $componentAlias = $e[0];
                    $field = $e[1];
                } else {
                    reset($this->_queryComponents);
                    $componentAlias = key($this->_queryComponents);
                    $field = $e[0];
                }

                $this->_pendingFields[$componentAlias][] = $field;
            }
        }
    }

    /**
     * parseClause
     * parses given DQL clause
     *
     * this method handles five tasks:
     *
     * 1. Converts all DQL functions to their native SQL equivalents
     * 2. Converts all component references to their table alias equivalents
     * 3. Converts all field names to actual column names
     * 4. Quotes all identifiers
     * 5. Parses nested clauses and subqueries recursively
     *
     * @return string   SQL string
     * @todo Description: What is a 'dql clause' (and what not)?
     *       Refactor: Too long & nesting level
     */
    public function parseClause($clause)
    {
        $clause = $this->_conn->dataDict->parseBoolean(trim($clause));

        if (is_numeric($clause)) {
           return $clause;
        }

        $terms = $this->_tokenizer->clauseExplode($clause, array(' ', '+', '-', '*', '/', '<', '>', '=', '>=', '<=', '&', '|'));
        $str = '';

        foreach ($terms as $term) {
            $pos = strpos($term[0], '(');

            if ($pos !== false && substr($term[0], 0, 1) !== "'") {
                $name = substr($term[0], 0, $pos);

                $term[0] = $this->parseFunctionExpression($term[0]);
            } else {
                if (substr($term[0], 0, 1) !== "'" && substr($term[0], -1) !== "'") {
                    if (strpos($term[0], '.') !== false) {
                        if ( ! is_numeric($term[0])) {
                            $e = explode('.', $term[0]);

                            $field = array_pop($e);

                            if ($this->getType() === Doctrine_Query::SELECT) {
                                $componentAlias = implode('.', $e);

                                if (empty($componentAlias)) {
                                    $componentAlias = $this->getRootAlias();
                                }

                                $this->load($componentAlias);

                                // check the existence of the component alias
                                if ( ! isset($this->_queryComponents[$componentAlias])) {
                                    throw new Doctrine_Query_Exception('Unknown component alias ' . $componentAlias);
                                }

                                $table = $this->_queryComponents[$componentAlias]['table'];

                                $def = $table->getDefinitionOf($field);

                                // get the actual field name from alias
                                $field = $table->getColumnName($field);

                                // check column existence
                                if ( ! $def) {
                                    throw new Doctrine_Query_Exception('Unknown column ' . $field);
                                }

                                if (isset($def['owner'])) {
                                    $componentAlias = $componentAlias . '.' . $def['owner'];
                                }

                                $tableAlias = $this->getSqlTableAlias($componentAlias);

                                // build sql expression
                                $term[0] = $this->_conn->quoteIdentifier($tableAlias)
                                         . '.'
                                         . $this->_conn->quoteIdentifier($field);
                            } else {
                                // build sql expression
                                $field = $this->getRoot()->getColumnName($field);
                                $term[0] = $this->_conn->quoteIdentifier($field);
                            }
                        }
                    } else {
                        if ( ! empty($term[0]) && ! in_array(strtoupper($term[0]), self::$_keywords) &&
                             ! is_numeric($term[0]) && $term[0] !== '?' && substr($term[0], 0, 1) !== ':') {

                            $componentAlias = $this->getRootAlias();

                            $found = false;

                            if ($componentAlias !== false && $componentAlias !== null) {
                                $table = $this->_queryComponents[$componentAlias]['table'];

                                // check column existence
                                if ($table->hasField($term[0])) {
                                    $found = true;

                                    $def = $table->getDefinitionOf($term[0]);

                                    // get the actual column name from field name
                                    $term[0] = $table->getColumnName($term[0]);


                                    if (isset($def['owner'])) {
                                        $componentAlias = $componentAlias . '.' . $def['owner'];
                                    }

                                    $tableAlias = $this->getSqlTableAlias($componentAlias);

                                    if ($this->getType() === Doctrine_Query::SELECT) {
                                        // build sql expression
                                        $term[0] = $this->_conn->quoteIdentifier($tableAlias)
                                                 . '.'
                                                 . $this->_conn->quoteIdentifier($term[0]);
                                    } else {
                                        // build sql expression
                                        $term[0] = $this->_conn->quoteIdentifier($term[0]);
                                    }
                                } else {
                                    $found = false;
                                }
                            }

                            if ( ! $found) {
                                $term[0] = $this->getSqlAggregateAlias($term[0]);
                            }
                        }
                    }
                }
            }

            $str .= $term[0] . $term[1];
        }
        return $str;
    }

    public function parseIdentifierReference($expr)
    {

    }

    public function parseFunctionExpression($expr)
    {
        $pos = strpos($expr, '(');
        $name = substr($expr, 0, $pos);

        if ($name === '') {
            return $this->parseSubquery($expr);
        }

        $argStr = substr($expr, ($pos + 1), -1);
        $args   = array();
        // parse args

        foreach ($this->_tokenizer->sqlExplode($argStr, ',') as $arg) {
           $args[] = $this->parseClause($arg);
        }

        // convert DQL function to its RDBMS specific equivalent
        try {
            $expr = call_user_func_array(array($this->_conn->expression, $name), $args);
        } catch (Doctrine_Expression_Exception $e) {
            throw new Doctrine_Query_Exception('Unknown function ' . $name . '.');
        }

        return $expr;
    }


    public function parseSubquery($subquery)
    {
        $trimmed = trim($this->_tokenizer->bracketTrim($subquery));

        // check for possible subqueries
        if (substr($trimmed, 0, 4) == 'FROM' || substr($trimmed, 0, 6) == 'SELECT') {
            // parse subquery
            $q = $this->createSubquery()->parseDqlQuery($trimmed);
            $trimmed = $q->getSqlQuery();
            $q->free();
        } else if (substr($trimmed, 0, 4) == 'SQL:') {
            $trimmed = substr($trimmed, 4);
        } else {
            $e = $this->_tokenizer->sqlExplode($trimmed, ',');

            $value = array();
            $index = false;

            foreach ($e as $part) {
                $value[] = $this->parseClause($part);
            }

            $trimmed = implode(', ', $value);
        }

        return '(' . $trimmed . ')';
    }


    /**
     * processPendingSubqueries
     * processes pending subqueries
     *
     * subqueries can only be processed when the query is fully constructed
     * since some subqueries may be correlated
     *
     * @return void
     * @todo Better description. i.e. What is a 'pending subquery'? What does 'processed' mean?
     *       (parsed? sql is constructed? some information is gathered?)
     */
    public function processPendingSubqueries()
    {
        foreach ($this->_pendingSubqueries as $value) {
            list($dql, $alias) = $value;

            $subquery = $this->createSubquery();

            $sql = $subquery->parseDqlQuery($dql, false)->getQuery();
            $subquery->free();

            reset($this->_queryComponents);
            $componentAlias = key($this->_queryComponents);
            $tableAlias = $this->getSqlTableAlias($componentAlias);

            $sqlAlias = $tableAlias . '__' . count($this->_aggregateAliasMap);

            $this->_sqlParts['select'][] = '(' . $sql . ') AS ' . $this->_conn->quoteIdentifier($sqlAlias);

            $this->_aggregateAliasMap[$alias] = $sqlAlias;
            $this->_queryComponents[$componentAlias]['agg'][] = $alias;
        }
        $this->_pendingSubqueries = array();
    }

    /**
     * processPendingAggregates
     * processes pending aggregate values for given component alias
     *
     * @return void
     * @todo Better description. i.e. What is a 'pending aggregate'? What does 'processed' mean?
     */
    public function processPendingAggregates()
    {
        // iterate trhough all aggregates
        foreach ($this->_pendingAggregates as $aggregate) {
            list ($expression, $components, $alias) = $aggregate;

            $tableAliases = array();

            // iterate through the component references within the aggregate function
            if ( ! empty ($components)) {
                foreach ($components as $component) {

                    if (is_numeric($component)) {
                        continue;
                    }

                    $e = explode('.', $component);

                    $field = array_pop($e);
                    $componentAlias = implode('.', $e);

                    // check the existence of the component alias
                    if ( ! isset($this->_queryComponents[$componentAlias])) {
                        throw new Doctrine_Query_Exception('Unknown component alias ' . $componentAlias);
                    }

                    $table = $this->_queryComponents[$componentAlias]['table'];

                    $field = $table->getColumnName($field);

                    // check column existence
                    if ( ! $table->hasColumn($field)) {
                        throw new Doctrine_Query_Exception('Unknown column ' . $field);
                    }

                    $sqlTableAlias = $this->getSqlTableAlias($componentAlias);

                    $tableAliases[$sqlTableAlias] = true;

                    // build sql expression

                    $identifier = $this->_conn->quoteIdentifier($sqlTableAlias . '.' . $field);
                    $expression = str_replace($component, $identifier, $expression);
                }
            }

            if (count($tableAliases) !== 1) {
                $componentAlias = reset($this->_tableAliasMap);
                $tableAlias = key($this->_tableAliasMap);
            }

            $index    = count($this->_aggregateAliasMap);
            $sqlAlias = $this->_conn->quoteIdentifier($tableAlias . '__' . $index);

            $this->_sqlParts['select'][] = $expression . ' AS ' . $sqlAlias;

            $this->_aggregateAliasMap[$alias] = $sqlAlias;
            $this->_expressionMap[$alias][0] = $expression;

            $this->_queryComponents[$componentAlias]['agg'][$index] = $alias;

            $this->_neededTables[] = $tableAlias;
        }
        // reset the state
        $this->_pendingAggregates = array();
    }

    /**
     * _buildSqlQueryBase
     * returns the base of the generated sql query
     * On mysql driver special strategy has to be used for DELETE statements
     * (where is this special strategy??)
     *
     * @return string       the base of the generated sql query
     */
    protected function _buildSqlQueryBase()
    {
        switch ($this->_type) {
            case self::DELETE:
                $q = 'DELETE FROM ';
            break;
            case self::UPDATE:
                $q = 'UPDATE ';
            break;
            case self::SELECT:
                $distinct = ($this->_sqlParts['distinct']) ? 'DISTINCT ' : '';
                $q = 'SELECT ' . $distinct . implode(', ', $this->_sqlParts['select']) . ' FROM ';
            break;
        }
        return $q;
    }

    /**
     * _buildSqlFromPart
     * builds the from part of the query and returns it
     *
     * @return string   the query sql from part
     */
    protected function _buildSqlFromPart($ignorePending = false)
    {
        $q = '';

        foreach ($this->_sqlParts['from'] as $k => $part) {
            $e = explode(' ', $part);

            if ($k === 0) {
                if ( ! $ignorePending && $this->_type == self::SELECT) {
                    // We may still have pending conditions
                    $alias = count($e) > 1
                        ? $this->getComponentAlias($e[1])
                        : null;
                    $where = $this->_processPendingJoinConditions($alias);

                    // apply inheritance to WHERE part
                    if ( ! empty($where)) {
                        if (count($this->_sqlParts['where']) > 0) {
                            $this->_sqlParts['where'][] = 'AND';
                        }

                        if (substr($where, 0, 1) === '(' && substr($where, -1) === ')') {
                            $this->_sqlParts['where'][] = $where;
                        } else {
                            $this->_sqlParts['where'][] = '(' . $where . ')';
                        }
                    }
                }

                $q .= $part;

                continue;
            }

            // preserve LEFT JOINs only if needed
            // Check if it's JOIN, if not add a comma separator instead of space
            if ( ! preg_match('/\bJOIN\b/i', $part) && ! isset($this->_pendingJoinConditions[$k])) {
                $q .= ', ' . $part;
            } else {
                if (substr($part, 0, 9) === 'LEFT JOIN') {
                    $aliases = array_merge($this->_subqueryAliases,
                                array_keys($this->_neededTables));

                    if ( ! in_array($e[3], $aliases) && ! in_array($e[2], $aliases) && ! empty($this->_pendingFields)) {
                        continue;
                    }

                }

                if ( ! $ignorePending && isset($this->_pendingJoinConditions[$k])) {
                    if (strpos($part, ' ON ') !== false) {
                        $part .= ' AND ';
                    } else {
                        $part .= ' ON ';
                    }

                    $part .= $this->_processPendingJoinConditions($k);
                }

                $componentAlias = $this->getComponentAlias($e[3]);
                $string = $this->getInheritanceCondition($componentAlias);

                if ($string) {
                    $part = $part . ' AND ' . $string;
                }
                $q .= ' ' . $part;
            }

            $this->_sqlParts['from'][$k] = $part;
        }
        return $q;
    }

    /**
     * Processes the pending join conditions, used for dynamically add conditions
     * to root component/joined components without interfering in the main dql
     * handling.
     *
     * @param string $alias Component Alias
     * @return Processed pending conditions
     */
    protected function _processPendingJoinConditions($alias)
    {
        $parts = array();

        if ($alias !== null && isset($this->_pendingJoinConditions[$alias])) {
            $parser = new Doctrine_Query_JoinCondition($this, $this->_tokenizer);

            foreach ($this->_pendingJoinConditions[$alias] as $joinCondition) {
                $parts[] = $parser->parse($joinCondition);
            }

            // FIX #1860 and #1876: Cannot unset them, otherwise query cannot be reused later
            //unset($this->_pendingJoinConditions[$alias]);
        }

        return (count($parts) > 0 ? '(' . implode(') AND (', $parts) . ')' : '');
    }

    /**
     * builds the sql query from the given parameters and applies things such as
     * column aggregation inheritance and limit subqueries if needed
     *
     * @param array $params             an array of prepared statement params (needed only in mysql driver
     *                                  when limit subquery algorithm is used)
     * @param bool $limitSubquery Whether or not to try and apply the limit subquery algorithm
     * @return string                   the built sql query
     */
    public function getSqlQuery($params = array(), $limitSubquery = true)
    {
        // Assign building/execution specific params
        $this->_params['exec'] = $params;

        // Initialize prepared parameters array
        $this->_execParams = $this->getFlattenedParams();

        if ($this->_state !== self::STATE_DIRTY) {
            $this->fixArrayParameterValues($this->getInternalParams());

            // Return compiled SQL
            return $this->_sql;
        }
        return $this->buildSqlQuery($limitSubquery);
    }

    /**
     * Build the SQL query from the DQL
     *
     * @param bool $limitSubquery Whether or not to try and apply the limit subquery algorithm
     * @return string $sql The generated SQL string
     */
    public function buildSqlQuery($limitSubquery = true)
    {
        // reset the state
        if ( ! $this->isSubquery()) {
            $this->_queryComponents = array();
            $this->_pendingAggregates = array();
            $this->_aggregateAliasMap = array();
        }

        $this->reset();

        // invoke the preQuery hook
        $this->_preQuery();

        // process the DQL parts => generate the SQL parts.
        // this will also populate the $_queryComponents.
        foreach ($this->_dqlParts as $queryPartName => $queryParts) {
            // If we are parsing FROM clause, we'll need to diff the queryComponents later
            if ($queryPartName == 'from') {
                // Pick queryComponents before processing
                $queryComponentsBefore = $this->getQueryComponents();
            }

            // FIX #1667: _sqlParts are cleaned inside _processDqlQueryPart.
            if ($queryPartName != 'forUpdate') {
                $this->_processDqlQueryPart($queryPartName, $queryParts);
            }

            // We need to define the root alias
            if ($queryPartName == 'from') {
                // Pick queryComponents aftr processing
                $queryComponentsAfter = $this->getQueryComponents();

                // Root alias is the key of difference of query components
                $diffQueryComponents = array_diff_key($queryComponentsAfter, $queryComponentsBefore);
                $this->_rootAlias = key($diffQueryComponents);
            }
        }
        $this->_state = self::STATE_CLEAN;

        // Proceed with the generated SQL
        if (empty($this->_sqlParts['from'])) {
            return false;
        }

        $needsSubQuery = false;
        $subquery = '';
        $map = $this->getRootDeclaration();
        $table = $map['table'];
        $rootAlias = $this->getRootAlias();

        if ( ! empty($this->_sqlParts['limit']) && $this->_needsSubquery &&
                $table->getAttribute(Doctrine_Core::ATTR_QUERY_LIMIT) == Doctrine_Core::LIMIT_RECORDS) {
            // We do not need a limit-subquery if DISTINCT is used
            // and the selected fields are either from the root component or from a localKey relation (hasOne)
            // (i.e. DQL: SELECT DISTINCT u.id FROM User u LEFT JOIN u.phonenumbers LIMIT 5).
            if(!$this->_sqlParts['distinct']) {
                $this->_isLimitSubqueryUsed = true;
                $needsSubQuery = true;
            } else {
                foreach( array_keys($this->_pendingFields) as $alias){
                    //no subquery for root fields
                    if($alias == $this->getRootAlias()){
                        continue;
                    }

                    //no subquery for ONE relations
                    if(isset($this->_queryComponents[$alias]['relation']) &&
                        $this->_queryComponents[$alias]['relation']->getType() == Doctrine_Relation::ONE){
                        continue;
                    }

                    $this->_isLimitSubqueryUsed = true;
                    $needsSubQuery = true;
                }
            }
        }

        $sql = array();

        if ( ! empty($this->_pendingFields)) {
            foreach ($this->_queryComponents as $alias => $map) {
                $fieldSql = $this->processPendingFields($alias);
                if ( ! empty($fieldSql)) {
                    $sql[] = $fieldSql;
                }
            }
        }

        if ( ! empty($sql)) {
            array_unshift($this->_sqlParts['select'], implode(', ', $sql));
        }

        $this->_pendingFields = array();

        // build the basic query
        $q  = $this->_buildSqlQueryBase();
        $q .= $this->_buildSqlFromPart();

        if ( ! empty($this->_sqlParts['set'])) {
            $q .= ' SET ' . implode(', ', $this->_sqlParts['set']);
        }

        $string = $this->getInheritanceCondition($this->getRootAlias());

        // apply inheritance to WHERE part
        if ( ! empty($string)) {
            if (count($this->_sqlParts['where']) > 0) {
                $this->_sqlParts['where'][] = 'AND';
            }

            if (substr($string, 0, 1) === '(' && substr($string, -1) === ')') {
                $this->_sqlParts['where'][] = $string;
            } else {
                $this->_sqlParts['where'][] = '(' . $string . ')';
            }
        }

        $modifyLimit = true;
        $limitSubquerySql = '';

        if ( ( ! empty($this->_sqlParts['limit']) || ! empty($this->_sqlParts['offset'])) && $needsSubQuery && $limitSubquery) {
            $subquery = $this->getLimitSubquery();

            // what about composite keys?
            $idColumnName = $table->getColumnName($table->getIdentifier());

            switch (strtolower($this->_conn->getDriverName())) {
                case 'mysql':
                    $this->useQueryCache(false);

                    // mysql doesn't support LIMIT in subqueries
                    $list = $this->_conn->execute($subquery, $this->_execParams)->fetchAll(Doctrine_Core::FETCH_COLUMN);
                    $subquery = implode(', ', array_map(array($this->_conn, 'quote'), $list));

                    break;

                case 'pgsql':
                    $subqueryAlias = $this->_conn->quoteIdentifier('doctrine_subquery_alias');

                    // pgsql needs special nested LIMIT subquery
                    $subquery = 'SELECT ' . $subqueryAlias . '.' . $this->_conn->quoteIdentifier($idColumnName)
                            . ' FROM (' . $subquery . ') AS ' . $subqueryAlias;

                    break;
            }

            $field = $this->getSqlTableAlias($rootAlias) . '.' . $idColumnName;

            // FIX #1868: If not ID under MySQL is found to be restricted, restrict pk column for null
            //            (which will lead to a return of 0 items)
            $limitSubquerySql = $this->_conn->quoteIdentifier($field)
                              . (( ! empty($subquery)) ? ' IN (' . $subquery . ')' : ' IS NULL')
                              . ((count($this->_sqlParts['where']) > 0) ? ' AND ' : '');

            $modifyLimit = false;
        }

        // FIX #DC-26: Include limitSubquerySql as major relevance in conditions
        $emptyWhere = empty($this->_sqlParts['where']);

        if ( ! ($emptyWhere && $limitSubquerySql == '')) {
            $where = implode(' ', $this->_sqlParts['where']);
            $where = ($where == '' || (substr($where, 0, 1) === '(' && substr($where, -1) === ')'))
                ? $where : '(' . $where . ')';

            $q .= ' WHERE ' . $limitSubquerySql . $where;
            //   .  (($limitSubquerySql == '' && count($this->_sqlParts['where']) == 1) ? substr($where, 1, -1) : $where);
        }

        // Fix the orderbys so we only have one orderby per value
        foreach ($this->_sqlParts['orderby'] as $k => $orderBy) {
            $e = explode(', ', $orderBy);
            unset($this->_sqlParts['orderby'][$k]);
            foreach ($e as $v) {
                $this->_sqlParts['orderby'][] = $v;
            }
        }

        // Add the default orderBy statements defined in the relationships and table classes
        // Only do this for SELECT queries
        if ($this->_type === self::SELECT) {
            foreach ($this->_queryComponents as $alias => $map) {
                $sqlAlias = $this->getSqlTableAlias($alias);
                if (isset($map['relation'])) {
                    $orderBy = $map['relation']->getOrderByStatement($sqlAlias, true);
                    if ($orderBy == $map['relation']['orderBy']) {
                        if (isset($map['ref'])) {
                            $orderBy = $map['relation']['refTable']->processOrderBy($sqlAlias, $map['relation']['orderBy'], true);
                        } else {
                            $orderBy = null;
                        }
                    }
				} else {
					$orderBy = $map['table']->getOrderByStatement($sqlAlias, true);
				}

                if ($orderBy) {
                    $e = explode(',', $orderBy);
                    $e = array_map('trim', $e);
                    foreach ($e as $v) {
                        if ( ! in_array($v, $this->_sqlParts['orderby'])) {
                            $this->_sqlParts['orderby'][] = $v;
                        }
                    }
                }
            }
        }

        $q .= ( ! empty($this->_sqlParts['groupby'])) ? ' GROUP BY ' . implode(', ', $this->_sqlParts['groupby'])  : '';
        $q .= ( ! empty($this->_sqlParts['having'])) ?  ' HAVING '   . implode(' AND ', $this->_sqlParts['having']): '';
        $q .= ( ! empty($this->_sqlParts['orderby'])) ? ' ORDER BY ' . implode(', ', $this->_sqlParts['orderby'])  : '';

        if ($modifyLimit) {
            $q = $this->_conn->modifyLimitQuery($q, $this->_sqlParts['limit'], $this->_sqlParts['offset']);
        }

        $q .= $this->_sqlParts['forUpdate'] === true ? ' FOR UPDATE ' : '';

        $this->_sql = $q;

        $this->clear();

        return $q;
    }

    /**
     * getLimitSubquery
     * this is method is used by the record limit algorithm
     *
     * when fetching one-to-many, many-to-many associated data with LIMIT clause
     * an additional subquery is needed for limiting the number of returned records instead
     * of limiting the number of sql result set rows
     *
     * @return string       the limit subquery
     * @todo A little refactor to make the method easier to understand & maybe shorter?
     */
    public function getLimitSubquery()
    {
        $map = reset($this->_queryComponents);
        $table = $map['table'];
        $componentAlias = key($this->_queryComponents);

        // get short alias
        $alias = $this->getSqlTableAlias($componentAlias);
        // what about composite keys?
        $primaryKey = $alias . '.' . $table->getColumnName($table->getIdentifier());

        $driverName = $this->_conn->getAttribute(Doctrine_Core::ATTR_DRIVER_NAME);

        // initialize the base of the subquery
        if (($driverName == 'oracle' || $driverName == 'oci') && $this->_isOrderedByJoinedColumn()) {
            $subquery = 'SELECT ';
        } else {
            $subquery = 'SELECT DISTINCT ';
        }
        $subquery .= $this->_conn->quoteIdentifier($primaryKey);

        // pgsql & oracle need the order by fields to be preserved in select clause
        if ($driverName == 'pgsql' || $driverName == 'oracle' || $driverName == 'oci' || $driverName == 'mssql' || $driverName == 'odbc') {
            foreach ($this->_sqlParts['orderby'] as $part) {
                // Remove identifier quoting if it exists
                $e = $this->_tokenizer->bracketExplode($part, ' ');
                foreach ($e as $f) {
                    if ($f == 0 || $f % 2 == 0) {
                        $partOriginal = str_replace(',', '', trim($f));
                        $callback = create_function('$e', 'return trim($e, \'[]`"\');');
                        $part = trim(implode('.', array_map($callback, explode('.', $partOriginal))));
                
                        if (strpos($part, '.') === false) {
                            continue;
                        }
                
                        // don't add functions
                        if (strpos($part, '(') !== false) {
                            continue;
                        }
                
                        // don't add primarykey column (its already in the select clause)
                        if ($part !== $primaryKey) {
                            $subquery .= ', ' . $partOriginal;
                        }
                    }
                }
            }
        }

        $orderby = $this->_sqlParts['orderby'];
        $having = $this->_sqlParts['having'];
        if ($driverName == 'mysql' || $driverName == 'pgsql') {
            foreach ($this->_expressionMap as $dqlAlias => $expr) {
                if (isset($expr[1])) {
                    $subquery .= ', ' . $expr[0] . ' AS ' . $this->_aggregateAliasMap[$dqlAlias];
                }
            }
        } else {
            foreach ($this->_expressionMap as $dqlAlias => $expr) {
                if (isset($expr[1])) {
                    foreach ($having as $k => $v) {
                        $having[$k] = str_replace($this->_aggregateAliasMap[$dqlAlias], $expr[0], $v);
                    }
                    foreach ($orderby as $k => $v) {
                        $e = explode(' ', $v);
                        if ($e[0] == $this->_aggregateAliasMap[$dqlAlias]) {
                            $orderby[$k] = $expr[0];
                        }
                    }
                }
            }
        }

        // Add having fields that got stripped out of select
        preg_match_all('/`[a-z0-9_]+`\.`[a-z0-9_]+`/i', implode(' ', $having), $matches, PREG_PATTERN_ORDER);
        if (count($matches[0]) > 0) {
            $subquery .= ', ' . implode(', ', array_unique($matches[0]));
        }

        $subquery .= ' FROM';

        foreach ($this->_sqlParts['from'] as $part) {
            // preserve LEFT JOINs only if needed
            if (substr($part, 0, 9) === 'LEFT JOIN') {
                $e = explode(' ', $part);
			//Patched by Will Ferrer to also check for groupBys. Fixes Bug -- DC-594
                if (empty($this->_sqlParts['orderby']) && empty($this->_sqlParts['where']) && empty($this->_sqlParts['having']) && empty($this->_sqlParts['groupby'])) {
             //End Patch
				   continue;
                }
            }

            $subquery .= ' ' . $part;
        }

        // all conditions must be preserved in subquery
        $subquery .= ( ! empty($this->_sqlParts['where']))?   ' WHERE '    . implode(' ', $this->_sqlParts['where'])  : '';
        $subquery .= ( ! empty($this->_sqlParts['groupby']))? ' GROUP BY ' . implode(', ', $this->_sqlParts['groupby'])   : '';
        $subquery .= ( ! empty($having))?  ' HAVING '   . implode(' AND ', $having) : '';
        $subquery .= ( ! empty($orderby))? ' ORDER BY ' . implode(', ', $orderby)  : '';

        if (($driverName == 'oracle' || $driverName == 'oci') && $this->_isOrderedByJoinedColumn()) {
            // When using "ORDER BY x.foo" where x.foo is a column of a joined table,
            // we may get duplicate primary keys because all columns in ORDER BY must appear
            // in the SELECT list when using DISTINCT. Hence we need to filter out the
            // primary keys with an additional DISTINCT subquery.
            // #1038
            $quotedIdentifierColumnName = $this->_conn->quoteIdentifier($table->getColumnName($table->getIdentifier()));
            $subquery = 'SELECT doctrine_subquery_alias.' . $quotedIdentifierColumnName
                    . ' FROM (' . $subquery . ') doctrine_subquery_alias'
                    . ' GROUP BY doctrine_subquery_alias.' . $quotedIdentifierColumnName
                    . ' ORDER BY MIN(ROWNUM)';
        }

        // add driver specific limit clause
        $subquery = $this->_conn->modifyLimitSubquery($table, $subquery, $this->_sqlParts['limit'], $this->_sqlParts['offset']);

        $parts = $this->_tokenizer->quoteExplode($subquery, ' ', "'", "'");

        foreach ($parts as $k => $part) {
            if (strpos($part, ' ') !== false) {
                continue;
            }

            $part = str_replace(array('"', "'", '`'), "", $part);

            if ($this->hasSqlTableAlias($part)) {
                $parts[$k] = $this->_conn->quoteIdentifier($this->generateNewSqlTableAlias($part));
                continue;
            }

            if (strpos($part, '.') === false) {
                continue;
            }

            preg_match_all("/[a-zA-Z0-9_]+\.[a-z0-9_]+/i", $part, $m);

            foreach ($m[0] as $match) {
                $e = explode('.', $match);

                // Rebuild the original part without the newly generate alias and with quoting reapplied
                $e2 = array();
                foreach ($e as $k2 => $v2) {
                  $e2[$k2] = $this->_conn->quoteIdentifier($v2);
                }
                $match = implode('.', $e2);

                // Generate new table alias
                $e[0] = $this->generateNewSqlTableAlias($e[0]);

                // Requote the part with the newly generated alias
                foreach ($e as $k2 => $v2) {
                  $e[$k2] = $this->_conn->quoteIdentifier($v2);
                }

                $replace = implode('.' , $e);

                // Replace the original part with the new part with new sql table alias
                $parts[$k] = str_replace($match, $replace, $parts[$k]);
            }
        }

        if ($driverName == 'mysql' || $driverName == 'pgsql') {
            foreach ($parts as $k => $part) {
                if (strpos($part, "'") !== false) {
                    continue;
                }
                if (strpos($part, '__') == false) {
                    continue;
                }

                preg_match_all("/[a-zA-Z0-9_]+\_\_[a-z0-9_]+/i", $part, $m);

                foreach ($m[0] as $match) {
                    $e = explode('__', $match);
                    $e[0] = $this->generateNewSqlTableAlias($e[0]);

                    $parts[$k] = str_replace($match, implode('__', $e), $parts[$k]);
                }
            }
        }

        $subquery = implode(' ', $parts);
        return $subquery;
    }

    /**
     * Checks whether the query has an ORDER BY on a column of a joined table.
     * This information is needed in special scenarios like the limit-offset when its
     * used with an Oracle database.
     *
     * @return boolean  TRUE if the query is ordered by a joined column, FALSE otherwise.
     */
    private function _isOrderedByJoinedColumn() {
        if ( ! $this->_queryComponents) {
            throw new Doctrine_Query_Exception("The query is in an invalid state for this "
                    . "operation. It must have been fully parsed first.");
        }
        $componentAlias = key($this->_queryComponents);
        $mainTableAlias = $this->getSqlTableAlias($componentAlias);
        foreach ($this->_sqlParts['orderby'] as $part) {
            $part = trim($part);
            $e = $this->_tokenizer->bracketExplode($part, ' ');
            $part = trim($e[0]);
            if (strpos($part, '.') === false) {
                continue;
            }
            list($tableAlias, $columnName) = explode('.', $part);
            if ($tableAlias != $mainTableAlias) {
                return true;
            }
        }
        return false;
    }

    /**
     * DQL PARSER
     * parses a DQL query
     * first splits the query in parts and then uses individual
     * parsers for each part
     *
     * @param string $query                 DQL query
     * @param boolean $clear                whether or not to clear the aliases
     * @throws Doctrine_Query_Exception     if some generic parsing error occurs
     * @return Doctrine_Query
     */
    public function parseDqlQuery($query, $clear = true)
    {
        if ($clear) {
            $this->clear();
        }

        $query = trim($query);
        $query = str_replace("\r", "\n", str_replace("\r\n", "\n", $query));
        $query = str_replace("\n", ' ', $query);

        $parts = $this->_tokenizer->tokenizeQuery($query);

        foreach ($parts as $partName => $subParts) {
            $subParts = trim($subParts);
            $partName = strtolower($partName);
            switch ($partName) {
                case 'create':
                    $this->_type = self::CREATE;
                break;
                case 'insert':
                    $this->_type = self::INSERT;
                break;
                case 'delete':
                    $this->_type = self::DELETE;
                break;
                case 'select':
                    $this->_type = self::SELECT;
                    $this->_addDqlQueryPart($partName, $subParts);
                break;
                case 'update':
                    $this->_type = self::UPDATE;
                    $partName = 'from';
                case 'from':
                    $this->_addDqlQueryPart($partName, $subParts);
                break;
                case 'set':
                    $this->_addDqlQueryPart($partName, $subParts, true);
                break;
                case 'group':
                case 'order':
                    $partName .= 'by';
                case 'where':
                case 'having':
                case 'limit':
                case 'offset':
                    $this->_addDqlQueryPart($partName, $subParts);
                break;
            }
        }

        return $this;
    }

    /**
     * @todo Describe & refactor... too long and nested.
     * @param string $path          component alias
     * @param boolean $loadFields
     */
    public function load($path, $loadFields = true)
    {
        if (isset($this->_queryComponents[$path])) {
            return $this->_queryComponents[$path];
        }

        $e = $this->_tokenizer->quoteExplode($path, ' INDEXBY ');

        $mapWith = null;
        if (count($e) > 1) {
            $mapWith = trim($e[1]);

            $path = $e[0];
        }

        // parse custom join conditions
        $e = explode(' ON ', str_ireplace(' on ', ' ON ', $path));

        $joinCondition = '';

        if (count($e) > 1) {
            $joinCondition = substr($path, strlen($e[0]) + 4, strlen($e[1]));
            $path = substr($path, 0, strlen($e[0]));

            $overrideJoin = true;
        } else {
            $e = explode(' WITH ', str_ireplace(' with ', ' WITH ', $path));

            if (count($e) > 1) {
                $joinCondition = substr($path, strlen($e[0]) + 6, strlen($e[1]));
                $path = substr($path, 0, strlen($e[0]));
            }

            $overrideJoin = false;
        }

        $tmp            = explode(' ', $path);
        $componentAlias = $originalAlias = (count($tmp) > 1) ? end($tmp) : null;

        $e = preg_split("/[.:]/", $tmp[0], -1);

        $fullPath = $tmp[0];
        $prevPath = '';
        $fullLength = strlen($fullPath);

        if (isset($this->_queryComponents[$e[0]])) {
            $table = $this->_queryComponents[$e[0]]['table'];
            $componentAlias = $e[0];

            $prevPath = $parent = array_shift($e);
        }

        foreach ($e as $key => $name) {
            // get length of the previous path
            $length = strlen($prevPath);

            // build the current component path
            $prevPath = ($prevPath) ? $prevPath . '.' . $name : $name;

            $delimeter = substr($fullPath, $length, 1);

            // if an alias is not given use the current path as an alias identifier
            if (strlen($prevPath) === $fullLength && isset($originalAlias)) {
                $componentAlias = $originalAlias;
            } else {
                $componentAlias = $prevPath;
            }

            // if the current alias already exists, skip it
            if (isset($this->_queryComponents[$componentAlias])) {
                throw new Doctrine_Query_Exception("Duplicate alias '$componentAlias' in query.");
            }

            if ( ! isset($table)) {
                // process the root of the path

                $table = $this->loadRoot($name, $componentAlias);
            } else {
                $join = ($delimeter == ':') ? 'INNER JOIN ' : 'LEFT JOIN ';

                $relation = $table->getRelation($name);
                $localTable = $table;

                $table = $relation->getTable();
                $this->_queryComponents[$componentAlias] = array('table' => $table,
                                                                 'parent'   => $parent,
                                                                 'relation' => $relation,
                                                                 'map'      => null);
                if ( ! $relation->isOneToOne()) {
                   $this->_needsSubquery = true;
                }

                $localAlias   = $this->getSqlTableAlias($parent, $localTable->getTableName());
                $foreignAlias = $this->getSqlTableAlias($componentAlias, $relation->getTable()->getTableName());

                $foreignSql   = $this->_conn->quoteIdentifier($relation->getTable()->getTableName())
                              . ' '
                              . $this->_conn->quoteIdentifier($foreignAlias);

                $map = $relation->getTable()->inheritanceMap;

                if ( ! $loadFields || ! empty($map) || $joinCondition) {
                    $this->_subqueryAliases[] = $foreignAlias;
                }

                if ($relation instanceof Doctrine_Relation_Association) {
                    $asf = $relation->getAssociationTable();

                    $assocTableName = $asf->getTableName();

                    if ( ! $loadFields || ! empty($map) || $joinCondition) {
                        $this->_subqueryAliases[] = $assocTableName;
                    }

                    $assocPath = $prevPath . '.' . $asf->getComponentName() . ' ' . $componentAlias;

                    $this->_queryComponents[$assocPath] = array(
                        'parent' => $prevPath,
                        'relation' => $relation,
                        'table' => $asf,
                        'ref' => true);

                    $assocAlias = $this->getSqlTableAlias($assocPath, $asf->getTableName());

                    $queryPart = $join
                            . $this->_conn->quoteIdentifier($assocTableName)
                            . ' '
                            . $this->_conn->quoteIdentifier($assocAlias);

                    $queryPart .= ' ON (' . $this->_conn->quoteIdentifier($localAlias
                                . '.'
                                . $localTable->getColumnName($localTable->getIdentifier())) // what about composite keys?
                                . ' = '
                                . $this->_conn->quoteIdentifier($assocAlias . '.' . $relation->getLocalRefColumnName());

                    if ($relation->isEqual()) {
                        // equal nest relation needs additional condition
                        $queryPart .= ' OR '
                                    . $this->_conn->quoteIdentifier($localAlias
                                    . '.'
                                    . $table->getColumnName($table->getIdentifier()))
                                    . ' = '
                                    . $this->_conn->quoteIdentifier($assocAlias . '.' . $relation->getForeignRefColumnName());
                    }

                    $queryPart .= ')';

                    $this->_sqlParts['from'][] = $queryPart;

                    $queryPart = $join . $foreignSql;

                    if ( ! $overrideJoin) {
                        $queryPart .= $this->buildAssociativeRelationSql($relation, $assocAlias, $foreignAlias, $localAlias);
                    }
                } else {
                    $queryPart = $this->buildSimpleRelationSql($relation, $foreignAlias, $localAlias, $overrideJoin, $join);
                }

                $queryPart .= $this->buildInheritanceJoinSql($table->getComponentName(), $componentAlias);
                $this->_sqlParts['from'][$componentAlias] = $queryPart;

                if ( ! empty($joinCondition)) {
                    $this->addPendingJoinCondition($componentAlias, $joinCondition);
                }
            }

            if ($loadFields) {
                $restoreState = false;

                // load fields if necessary
                if ($loadFields && empty($this->_dqlParts['select'])) {
                    $this->_pendingFields[$componentAlias] = array('*');
                }
            }

            $parent = $prevPath;
        }

        $table = $this->_queryComponents[$componentAlias]['table'];

        return $this->buildIndexBy($componentAlias, $mapWith);
    }

    protected function buildSimpleRelationSql(Doctrine_Relation $relation, $foreignAlias, $localAlias, $overrideJoin, $join)
    {
        $queryPart = $join . $this->_conn->quoteIdentifier($relation->getTable()->getTableName())
                           . ' '
                           . $this->_conn->quoteIdentifier($foreignAlias);

        if ( ! $overrideJoin) {
            $queryPart .= ' ON '
                       . $this->_conn->quoteIdentifier($localAlias . '.' . $relation->getLocalColumnName())
                       . ' = '
                       . $this->_conn->quoteIdentifier($foreignAlias . '.' . $relation->getForeignColumnName());
        }

        return $queryPart;
    }

    protected function buildIndexBy($componentAlias, $mapWith = null)
    {
        $table = $this->_queryComponents[$componentAlias]['table'];

        $indexBy = null;
        $column = false;

        if (isset($mapWith)) {
            $terms = explode('.', $mapWith);

            if (count($terms) == 1) {
                $indexBy = $terms[0];
            } else if (count($terms) == 2) {
                $column = true;
                $indexBy = $terms[1];
            }
        } else if ($table->getBoundQueryPart('indexBy') !== null) {
            $indexBy = $table->getBoundQueryPart('indexBy');
        }

        if ($indexBy !== null) {
            if ( $column && ! $table->hasColumn($table->getColumnName($indexBy))) {
                throw new Doctrine_Query_Exception("Couldn't use key mapping. Column " . $indexBy . " does not exist.");
            }

            $this->_queryComponents[$componentAlias]['map'] = $indexBy;
        }

        return $this->_queryComponents[$componentAlias];
    }


    protected function buildAssociativeRelationSql(Doctrine_Relation $relation, $assocAlias, $foreignAlias, $localAlias)
    {
        $table = $relation->getTable();

        $queryPart = ' ON ';

        if ($relation->isEqual()) {
            $queryPart .= '(';
        }

        $localIdentifier = $table->getColumnName($table->getIdentifier());

        $queryPart .= $this->_conn->quoteIdentifier($foreignAlias . '.' . $localIdentifier)
                    . ' = '
                    . $this->_conn->quoteIdentifier($assocAlias . '.' . $relation->getForeignRefColumnName());

        if ($relation->isEqual()) {
            $queryPart .= ' OR '
                        . $this->_conn->quoteIdentifier($foreignAlias . '.' . $localIdentifier)
                        . ' = '
                        . $this->_conn->quoteIdentifier($assocAlias . '.' . $relation->getLocalRefColumnName())
                        . ') AND '
                        . $this->_conn->quoteIdentifier($foreignAlias . '.' . $localIdentifier)
                        . ' != '
                        . $this->_conn->quoteIdentifier($localAlias . '.' . $localIdentifier);
        }

        return $queryPart;
    }

    /**
     * loadRoot
     *
     * @param string $name
     * @param string $componentAlias
     * @return Doctrine_Table
     * @todo DESCRIBE ME!
     * @todo this method is called only in Doctrine_Query class. Shouldn't be private or protected?
     */
    public function loadRoot($name, $componentAlias)
    {
        // get the connection for the component
        $manager = Doctrine_Manager::getInstance();
        if ( ! $this->_passedConn && $manager->hasConnectionForComponent($name)) {
            $this->_conn = $manager->getConnectionForComponent($name);
        }

        $table = $this->_conn->getTable($name);
        $tableName = $table->getTableName();

        // get the short alias for this table
        $tableAlias = $this->getSqlTableAlias($componentAlias, $tableName);
        // quote table name
        $queryPart = $this->_conn->quoteIdentifier($tableName);

        if ($this->_type === self::SELECT) {
            $queryPart .= ' ' . $this->_conn->quoteIdentifier($tableAlias);
        }

        $this->_tableAliasMap[$tableAlias] = $componentAlias;

        $queryPart .= $this->buildInheritanceJoinSql($name, $componentAlias);

        $this->_sqlParts['from'][] = $queryPart;

        $this->_queryComponents[$componentAlias] = array('table' => $table, 'map' => null);

        return $table;
    }

    /**
     * @todo DESCRIBE ME!
     * @param string $name              component class name
     * @param string $componentAlias    alias of the component in the dql
     * @return string                   query part
     */
    public function buildInheritanceJoinSql($name, $componentAlias)
    {
        // get the connection for the component
        $manager = Doctrine_Manager::getInstance();
        if ( ! $this->_passedConn && $manager->hasConnectionForComponent($name)) {
            $this->_conn = $manager->getConnectionForComponent($name);
        }

        $table = $this->_conn->getTable($name);
        $tableName = $table->getTableName();

        // get the short alias for this table
        $tableAlias = $this->getSqlTableAlias($componentAlias, $tableName);

        $queryPart = '';

        foreach ($table->getOption('joinedParents') as $parent) {
            $parentTable = $this->_conn->getTable($parent);

            $parentAlias = $componentAlias . '.' . $parent;

            // get the short alias for the parent table
            $parentTableAlias = $this->getSqlTableAlias($parentAlias, $parentTable->getTableName());

            $queryPart .= ' LEFT JOIN ' . $this->_conn->quoteIdentifier($parentTable->getTableName())
                        . ' ' . $this->_conn->quoteIdentifier($parentTableAlias) . ' ON ';

            //Doctrine_Core::dump($table->getIdentifier());
            foreach ((array) $table->getIdentifier() as $identifier) {
                $column = $table->getColumnName($identifier);

                $queryPart .= $this->_conn->quoteIdentifier($tableAlias)
                            . '.' . $this->_conn->quoteIdentifier($column)
                            . ' = ' . $this->_conn->quoteIdentifier($parentTableAlias)
                            . '.' . $this->_conn->quoteIdentifier($column);
            }
        }

        return $queryPart;
    }

    /**
     * Get count sql query for this Doctrine_Query instance.
     *
     * This method is used in Doctrine_Query::count() for returning an integer
     * for the number of records which will be returned when executed.
     *
     * @return string $q
     */
    public function getCountSqlQuery()
    {
        // triggers dql parsing/processing
        $this->getSqlQuery(array(), false); // this is ugly

        // initialize temporary variables
        $where   = $this->_sqlParts['where'];
        $having  = $this->_sqlParts['having'];
        $groupby = $this->_sqlParts['groupby'];

        $rootAlias = $this->getRootAlias();
        $tableAlias = $this->getSqlTableAlias($rootAlias);

        // Build the query base
        $q = 'SELECT COUNT(*) AS ' . $this->_conn->quoteIdentifier('num_results') . ' FROM ';

        // Build the from clause
        $from = $this->_buildSqlFromPart(true);

        // Build the where clause
        $where = ( ! empty($where)) ? ' WHERE ' . implode(' ', $where) : '';

        // Build the group by clause
        $groupby = ( ! empty($groupby)) ? ' GROUP BY ' . implode(', ', $groupby) : '';

        // Build the having clause
        $having = ( ! empty($having)) ? ' HAVING ' . implode(' AND ', $having) : '';

        // Building the from clause and finishing query
        if (count($this->_queryComponents) == 1 && empty($having)) {
            $q .= $from . $where . $groupby . $having;
        } else {
            // Subselect fields will contain only the pk of root entity
            $ta = $this->_conn->quoteIdentifier($tableAlias);

            $map = $this->getRootDeclaration();
            $idColumnNames = $map['table']->getIdentifierColumnNames();

            $pkFields = $ta . '.' . implode(', ' . $ta . '.', $this->_conn->quoteMultipleIdentifier($idColumnNames));

            // We need to do some magic in select fields if the query contain anything in having clause
            $selectFields = $pkFields;

            if ( ! empty($having)) {
                // For each field defined in select clause
                foreach ($this->_sqlParts['select'] as $field) {
                    // We only include aggregate expressions to count query
                    // This is needed because HAVING clause will use field aliases
                    if (strpos($field, '(') !== false) {
                        $selectFields .= ', ' . $field;
                    }
                }
                // Add having fields that got stripped out of select
                preg_match_all('/`[a-z0-9_]+`\.`[a-z0-9_]+`/i', $having, $matches, PREG_PATTERN_ORDER);
                if (count($matches[0]) > 0) {
                    $selectFields .= ', ' . implode(', ', array_unique($matches[0]));
                }
            }

            // If we do not have a custom group by, apply the default one
            if (empty($groupby)) {
                $groupby = ' GROUP BY ' . $pkFields;
            }

            $q .= '(SELECT ' . $selectFields . ' FROM ' . $from . $where . $groupby . $having . ') '
                . $this->_conn->quoteIdentifier('dctrn_count_query');
        }

        return $q;
    }

    /**
     * Fetches the count of the query.
     *
     * This method executes the main query without all the
     * selected fields, ORDER BY part, LIMIT part and OFFSET part.
     *
     * Example:
     * Main query:
     *      SELECT u.*, p.phonenumber FROM User u
     *          LEFT JOIN u.Phonenumber p
     *          WHERE p.phonenumber = '123 123' LIMIT 10
     *
     * The modified DQL query:
     *      SELECT COUNT(DISTINCT u.id) FROM User u
     *          LEFT JOIN u.Phonenumber p
     *          WHERE p.phonenumber = '123 123'
     *
     * @param array $params        an array of prepared statement parameters
     * @return integer             the count of this query
     */
    public function count($params = array())
    {
        $q = $this->getCountSqlQuery();
        $params = $this->getCountQueryParams($params);
        $params = $this->_conn->convertBooleans($params);

        if ($this->_resultCache) {
            $conn = $this->getConnection(); 
            $cacheDriver = $this->getResultCacheDriver();
            $hash = $this->getResultCacheHash($params).'_count';
            $cached = ($this->_expireResultCache) ? false : $cacheDriver->fetch($hash);

            if ($cached === false) {
                // cache miss
                $results = $this->getConnection()->fetchAll($q, $params);
                $cacheDriver->save($hash, serialize($results), $this->getResultCacheLifeSpan());
            } else {
                $results = unserialize($cached);
            }
        } else {
            $results = $this->getConnection()->fetchAll($q, $params);
        }

        if (count($results) > 1) {
            $count = count($results);
        } else {
            if (isset($results[0])) {
                $results[0] = array_change_key_case($results[0], CASE_LOWER);
                $count = $results[0]['num_results'];
            } else {
                $count = 0;
            }
        }

        return (int) $count;
    }

    /**
     * Queries the database with DQL (Doctrine Query Language).
     *
     * This methods parses a Dql query and builds the query parts.
     *
     * @param string $query      Dql query
     * @param array $params      prepared statement parameters
     * @param int $hydrationMode Doctrine_Core::HYDRATE_ARRAY or Doctrine_Core::HYDRATE_RECORD
     * @see Doctrine_Core::FETCH_* constants
     * @return mixed
     */
    public function query($query, $params = array(), $hydrationMode = null)
    {
        $this->parseDqlQuery($query);
        return $this->execute($params, $hydrationMode);
    }

    /**
     * Copies a Doctrine_Query object.
     *
     * @return Doctrine_Query  Copy of the Doctrine_Query instance.
     */
    public function copy(Doctrine_Query $query = null)
    {
        if ( ! $query) {
            $query = $this;
        }

        $new = clone $query;

        return $new;
    }

    /**
     * Magic method called after cloning process.
     *
     * @return void
     */
    public function __clone()
    {
        $this->_parsers = array();
        $this->_hydrator = clone $this->_hydrator;

        // Subqueries share some information from the parent so it can intermingle
        // with the dql of the main query. So when a subquery is cloned we need to
        // kill those references or it causes problems
        if ($this->isSubquery()) {
            $this->_killReference('_params');
            $this->_killReference('_tableAliasMap');
            $this->_killReference('_queryComponents');
        }
    }

    /**
     * Kill the reference for the passed class property.
     * This method simply copies the value to a temporary variable and then unsets
     * the reference and re-assigns the old value but not by reference
     *
     * @param string $key
     */
    protected function _killReference($key)
    {
        $tmp = $this->$key;
        unset($this->$key);
        $this->$key = $tmp;
    }

    /**
     * Frees the resources used by the query object. It especially breaks a
     * cyclic reference between the query object and it's parsers. This enables
     * PHP's current GC to reclaim the memory.
     * This method can therefore be used to reduce memory usage when creating
     * a lot of query objects during a request.
     *
     * @return Doctrine_Query   this object
     */
    public function free()
    {
        $this->reset();
        $this->_parsers = array();
        $this->_dqlParts = array();
    }
}
Comment by Jonathan H. Wage [ 27/May/10 ]

Do you test your changes against our test suite? We will still be releasing bug fix releases, we're just not monitoring and fixing bugs on a day to day basis. As I don't have any help on Doctrine 1 I have to spend a week every 1-2 months just going through issues and closing as many as possible.

Comment by will ferrer [ 27/May/10 ]

Hi Jonathan

Thanks for the write back.

I hadn't been using the test cases but I just tried and this latest patch was causing many of them to fail (I have resolved the issues however – see below).

The current problem is that with out my patch the following dql:

SELECT Charity.id, Charity.id as charity_id, (SQL:SELECT p.id AS p__0 FROM product_customers p LIMIT 1) as custom_subQuery FROM Charity Charity LIMIT 20

Produces the following exception:

{"type":"exception","tid":3,"exception":{},"message":"Couldn't get short alias for p","where":"#0 C:\\htdocs\\php_library\\Doctrine-1.2.2\\lib\\Doctrine\\Query.php(641): Doctrine_Query_Abstract->getSqlTableAlias('p')\n#1 C:\\htdocs\\php_library\\Doctrine-1.2.2\\lib\\Doctrine\\Query\\Select.php(37): Doctrine_Query->parseSelect('(SQL:SELECT p.i...')\n#2 C:\\htdocs\\php_library\\Doctrine-1.2.2\\lib\\Doctrine\\Query\\Abstract.php(2078): Doctrine_Query_Select->parse('(SQL:SELECT p.i...')\n#3 C:\\htdocs\\php_library\\Doctrine-1.2.2\\lib\\Doctrine\\Query.php(1168): Doctrine_Query_Abstract->_processDqlQueryPart('select', Array)\n#4 C:\\htdocs\\php_library\\Doctrine-1.2.2\\lib\\Doctrine\\Query.php(1134): Doctrine_Query->buildSqlQuery(true)\n#5 C:\\htdocs\\Root\\modules\\default\\util\\AnalyticsGrid.php(166): Doctrine_Query->getSqlQuery()\n#6 C:\\htdocs\\Root\\modules\\default\\models\\FileInFolders.php(166): Util_AnalyticsGrid->readAnalyticsGrid('{\"select\":{\"chi...', 0, 20, Array, Array, Array, Array, Array)\n#7 [internal function]: Models_FileInFolders->readAnalyticsTableProxy(37, 0, 20, Array, Array, Array, Array, Array)\n#8 C:\\htdocs\\php_library\\Doctrine-1.2.2\\lib\\Doctrine\\Table.php(2808): call_user_func_array(Array, Array)\n#9 [internal function]: Doctrine_Table->__call('readAnalytics', Array)\n#10 C:\\htdocs\\Root\\modules\\event\\controllers\\AnalyticsController.php(96): Doctrine_Table->readAnalytics(37, 0, 20, Array, Array, Array, Array, Array)\n#11 [internal function]: Event_AnalyticsController->readAnalyticsAction(Object(stdClass))\n#12 C:\\htdocs\\Root\\modules\\default\\util\\helper\\ZendDirectRouter.php(194): call_user_func_array(Array, Array)\n#13 C:\\htdocs\\Root\\modules\\default\\util\\helper\\ZendDirectRouter.php(68): Util_Helper_ZendDirectRouter->rpc(Object(stdClass))\n#14 C:\\htdocs\\Root\\modules\\default\\base\\ActionController.php(80): Util_Helper_ZendDirectRouter->dispatch()\n#15 C:\\htdocs\\php_library\\zendframework\\library\\Zend\\Controller\\Action.php(513): Base_ActionController->directAction()\n#16 C:\\htdocs\\php_library\\zendframework\\library\\Zend\\Controller\\Dispatcher\\Standard.php(289): Zend_Controller_Action->dispatch('directAction')\n#17 C:\\htdocs\\php_library\\zendframework\\library\\Zend\\Controller\\Front.php(955): Zend_Controller_Dispatcher_Standard->dispatch(Object(Zend_Controller_Request_Http), Object(Zend_Controller_Response_Http))\n#18 C:\\htdocs\\Root\\public\\index.php(78): Zend_Controller_Front->dispatch()\n#19 {main}"}

I looked at my code again and saw some very glaring errors in it (I was rushing through bugs and as soon as I thought I had fixed this one I didn't take the time to read it over – in the future will be sure to use the tests in order to avoid any such oversights).

At any rate I have changed my patch to the code and this is both working for my needs and passing all the tests again. Here is the newest version of my patched code:

<?php
/*
 *  $Id: Query.php 7490 2010-03-29 19:53:27Z jwage $
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
 * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 *
 * This software consists of voluntary contributions made by many individuals
 * and is licensed under the LGPL. For more information, see
 * <http://www.doctrine-project.org>.
 */

/**
 * Doctrine_Query
 * A Doctrine_Query object represents a DQL query. It is used to query databases for
 * data in an object-oriented fashion. A DQL query understands relations and inheritance
 * and is dbms independant.
 *
 * @package     Doctrine
 * @subpackage  Query
 * @license     http://www.opensource.org/licenses/lgpl-license.php LGPL
 * @link        www.doctrine-project.org
 * @since       1.0
 * @version     $Revision: 7490 $
 * @author      Konsta Vesterinen <kvesteri@cc.hut.fi>
 * @todo        Proposal: This class does far too much. It should have only 1 task: Collecting
 *              the DQL query parts and the query parameters (the query state and caching options/methods
 *              can remain here, too).
 *              The actual SQL construction could be done by a separate object (Doctrine_Query_SqlBuilder?)
 *              whose task it is to convert DQL into SQL.
 *              Furthermore the SqlBuilder? can then use other objects (Doctrine_Query_Tokenizer?),
 *              (Doctrine_Query_Parser(s)?) to accomplish his work. Doctrine_Query does not need
 *              to know the tokenizer/parsers. There could be extending
 *              implementations of SqlBuilder? that cover the specific SQL dialects.
 *              This would release Doctrine_Connection and the Doctrine_Connection_xxx classes
 *              from this tedious task.
 *              This would also largely reduce the currently huge interface of Doctrine_Query(_Abstract)
 *              and better hide all these transformation internals from the public Query API.
 *
 * @internal    The lifecycle of a Query object is the following:
 *              After construction the query object is empty. Through using the fluent
 *              query interface the user fills the query object with DQL parts and query parameters.
 *              These get collected in {@link $_dqlParts} and {@link $_params}, respectively.
 *              When the query is executed the first time, or when {@link getSqlQuery()}
 *              is called the first time, the collected DQL parts get parsed and the resulting
 *              connection-driver specific SQL is generated. The generated SQL parts are
 *              stored in {@link $_sqlParts} and the final resulting SQL query is stored in
 *              {@link $_sql}.
 */
class Doctrine_Query extends Doctrine_Query_Abstract implements Countable
{
    /**
     * @var array  The DQL keywords.
     */
    protected static $_keywords  = array('ALL',
                                         'AND',
                                         'ANY',
                                         'AS',
                                         'ASC',
                                         'AVG',
                                         'BETWEEN',
                                         'BIT_LENGTH',
                                         'BY',
                                         'CHARACTER_LENGTH',
                                         'CHAR_LENGTH',
                                         'CURRENT_DATE',
                                         'CURRENT_TIME',
                                         'CURRENT_TIMESTAMP',
                                         'DELETE',
                                         'DESC',
                                         'DISTINCT',
                                         'EMPTY',
                                         'EXISTS',
                                         'FALSE',
                                         'FETCH',
                                         'FROM',
                                         'GROUP',
                                         'HAVING',
                                         'IN',
                                         'INDEXBY',
                                         'INNER',
                                         'IS',
                                         'JOIN',
                                         'LEFT',
                                         'LIKE',
                                         'LOWER',
                                         'MEMBER',
                                         'MOD',
                                         'NEW',
                                         'NOT',
                                         'NULL',
                                         'OBJECT',
                                         'OF',
                                         'OR',
                                         'ORDER',
                                         'OUTER',
                                         'POSITION',
                                         'SELECT',
                                         'SOME',
                                         'TRIM',
                                         'TRUE',
                                         'UNKNOWN',
                                         'UPDATE',
                                         'WHERE');

    /**
     * @var array
     */
    protected $_subqueryAliases = array();

    /**
     * @var array $_aggregateAliasMap       an array containing all aggregate aliases, keys as dql aliases
     *                                      and values as sql aliases
     */
    protected $_aggregateAliasMap      = array();

    /**
     * @var array
     */
    protected $_pendingAggregates = array();

    /**
     * @param boolean $needsSubquery
     */
    protected $_needsSubquery = false;

    /**
     * @param boolean $isSubquery           whether or not this query object is a subquery of another
     *                                      query object
     */
    protected $_isSubquery;

    /**
     * @var array $_neededTables            an array containing the needed table aliases
     */
    protected $_neededTables = array();

    /**
     * @var array $pendingSubqueries        SELECT part subqueries, these are called pending subqueries since
     *                                      they cannot be parsed directly (some queries might be correlated)
     */
    protected $_pendingSubqueries = array();

    /**
     * @var array $_pendingFields           an array of pending fields (fields waiting to be parsed)
     */
    protected $_pendingFields = array();

    /**
     * @var array $_parsers                 an array of parser objects, each DQL query part has its own parser
     */
    protected $_parsers = array();

    /**
     * @var array $_pendingJoinConditions    an array containing pending joins
     */
    protected $_pendingJoinConditions = array();

    /**
     * @var array
     */
    protected $_expressionMap = array();

    /**
     * @var string $_sql            cached SQL query
     */
    protected $_sql;

    /**
     * create
     * returns a new Doctrine_Query object
     *
     * @param Doctrine_Connection $conn  optional connection parameter
     * @param string $class              Query class to instantiate
     * @return Doctrine_Query
     */
    public static function create($conn = null, $class = null)
    {
        if ( ! $class) {
            $class = Doctrine_Manager::getInstance()
                ->getAttribute(Doctrine_Core::ATTR_QUERY_CLASS);
        }
        return new $class($conn);
    }

    /**
     * Clears all the sql parts.
     */
    protected function clear()
    {
        $this->_preQueried = false;
        $this->_pendingJoinConditions = array();
        $this->_state = self::STATE_DIRTY;
    }

    /**
     * Resets the query to the state just after it has been instantiated.
     */
    public function reset()
    {
        $this->_subqueryAliases = array();
        $this->_aggregateAliasMap = array();
        $this->_pendingAggregates = array();
        $this->_pendingSubqueries = array();
        $this->_pendingFields = array();
        $this->_neededTables = array();
        $this->_expressionMap = array();
        $this->_subqueryAliases = array();
        $this->_needsSubquery = false;
        $this->_isLimitSubqueryUsed = false;
    }

    /**
     * createSubquery
     * creates a subquery
     *
     * @return Doctrine_Hydrate
     */
    public function createSubquery()
    {
        $class = get_class($this);
        $obj   = new $class();

        // copy the aliases to the subquery
        $obj->copySubqueryInfo($this);

        // this prevents the 'id' being selected, re ticket #307
        $obj->isSubquery(true);

        return $obj;
    }

    /**
     * addPendingJoinCondition
     *
     * @param string $componentAlias    component alias
     * @param string $joinCondition     dql join condition
     * @return Doctrine_Query           this object
     */
    public function addPendingJoinCondition($componentAlias, $joinCondition)
    {
        if ( ! isset($this->_pendingJoinConditions[$componentAlias])) {
            $this->_pendingJoinConditions[$componentAlias] = array();
        }

        $this->_pendingJoinConditions[$componentAlias][] = $joinCondition;
    }

    /**
     * fetchArray
     * Convenience method to execute using array fetching as hydration mode.
     *
     * @param string $params
     * @return array
     */
    public function fetchArray($params = array())
    {
        return $this->execute($params, Doctrine_Core::HYDRATE_ARRAY);
    }

    /**
     * fetchOne
     * Convenience method to execute the query and return the first item
     * of the collection.
     *
     * @param string $params        Query parameters
     * @param int $hydrationMode    Hydration mode: see Doctrine_Core::HYDRATE_* constants
     * @return mixed                Array or Doctrine_Collection, depending on hydration mode. False if no result.
     */
    public function fetchOne($params = array(), $hydrationMode = null)
    {
        $collection = $this->execute($params, $hydrationMode);

        if (is_scalar($collection)) {
            return $collection;
        }

        if (count($collection) === 0) {
            return false;
        }

        if ($collection instanceof Doctrine_Collection) {
            return $collection->getFirst();
        } else if (is_array($collection)) {
            return array_shift($collection);
        }

        return false;
    }

    /**
     * isSubquery
     * if $bool parameter is set this method sets the value of
     * Doctrine_Query::$isSubquery. If this value is set to true
     * the query object will not load the primary key fields of the selected
     * components.
     *
     * If null is given as the first parameter this method retrieves the current
     * value of Doctrine_Query::$isSubquery.
     *
     * @param boolean $bool     whether or not this query acts as a subquery
     * @return Doctrine_Query|bool
     */
    public function isSubquery($bool = null)
    {
        if ($bool === null) {
            return $this->_isSubquery;
        }

        $this->_isSubquery = (bool) $bool;
        return $this;
    }

    /**
     * getSqlAggregateAlias
     *
     * @param string $dqlAlias      the dql alias of an aggregate value
     * @return string
     */
    public function getSqlAggregateAlias($dqlAlias)
    {
        if (isset($this->_aggregateAliasMap[$dqlAlias])) {
            // mark the expression as used
            $this->_expressionMap[$dqlAlias][1] = true;

            return $this->_aggregateAliasMap[$dqlAlias];
        } else if ( ! empty($this->_pendingAggregates)) {
            $this->processPendingAggregates();

            return $this->getSqlAggregateAlias($dqlAlias);
        } else if( ! ($this->_conn->getAttribute(Doctrine_Core::ATTR_PORTABILITY) & Doctrine_Core::PORTABILITY_EXPR)){
            return $dqlAlias;
        } else {
            throw new Doctrine_Query_Exception('Unknown aggregate alias: ' . $dqlAlias);
        }
    }

    /**
     * Check if a dql alias has a sql aggregate alias
     *
     * @param string $dqlAlias 
     * @return boolean
     */
    public function hasSqlAggregateAlias($dqlAlias)
    {
        try {
            $this->getSqlAggregateAlias($dqlAlias);
            return true;
        } catch (Exception $e) {
            return false;
        }
    }

    /**
     * Adjust the processed param index for "foo.bar IN ?" support
     *
     */
    public function adjustProcessedParam($index)
    {
        // Retrieve all params
        $params = $this->getInternalParams();

        // Retrieve already processed values
        $first = array_slice($params, 0, $index);
        $last = array_slice($params, $index, count($params) - $index);

        // Include array as values splicing the params array
        array_splice($last, 0, 1, $last[0]);

        // Put all param values into a single index
        $this->_execParams = array_merge($first, $last);
    }

    /**
     * Retrieves a specific DQL query part.
     *
     * @see Doctrine_Query_Abstract::$_dqlParts
     * <code>
     * var_dump($q->getDqlPart('where'));
     * // array(2) { [0] => string(8) 'name = ?' [1] => string(8) 'date > ?' }
     * </code>
     * @param string $queryPart     the name of the query part; can be:
     *     array from, containing strings;
     *     array select, containg string;
     *     boolean forUpdate;
     *     array set;
     *     array join;
     *     array where;
     *     array groupby;
     *     array having;
     *     array orderby, containing strings such as 'id ASC';
     *     array limit, containing numerics;
     *     array offset, containing numerics;
     * @return array
     */
    public function getDqlPart($queryPart)
    {
        if ( ! isset($this->_dqlParts[$queryPart])) {
           throw new Doctrine_Query_Exception('Unknown query part ' . $queryPart);
        }

        return $this->_dqlParts[$queryPart];
    }

    /**
     * contains
     *
     * Method to check if a arbitrary piece of dql exists
     *
     * @param string $dql Arbitrary piece of dql to check for
     * @return boolean
     */
    public function contains($dql)
    {
      return stripos($this->getDql(), $dql) === false ? false : true;
    }

    /**
     * processPendingFields
     * the fields in SELECT clause cannot be parsed until the components
     * in FROM clause are parsed, hence this method is called everytime a
     * specific component is being parsed. For instance, the wildcard '*'
     * is expanded in the list of columns.
     *
     * @throws Doctrine_Query_Exception     if unknown component alias has been given
     * @param string $componentAlias        the alias of the component
     * @return string SQL code
     * @todo Description: What is a 'pending field' (and are there non-pending fields, too)?
     *       What is 'processed'? (Meaning: What information is gathered & stored away)
     */
    public function processPendingFields($componentAlias)
    {
        $tableAlias = $this->getSqlTableAlias($componentAlias);
        $table = $this->_queryComponents[$componentAlias]['table'];

        if ( ! isset($this->_pendingFields[$componentAlias])) {
            if ($this->_hydrator->getHydrationMode() != Doctrine_Core::HYDRATE_NONE) {
                if ( ! $this->_isSubquery && $componentAlias == $this->getRootAlias()) {
                    throw new Doctrine_Query_Exception("The root class of the query (alias $componentAlias) "
                            . " must have at least one field selected.");
                }
            }
            return;
        }

        // At this point we know the component is FETCHED (either it's the base class of
        // the query (FROM xyz) or its a "fetch join").

        // Check that the parent join (if there is one), is a "fetch join", too.
        if ( ! $this->isSubquery() && isset($this->_queryComponents[$componentAlias]['parent'])) {
            $parentAlias = $this->_queryComponents[$componentAlias]['parent'];
            if (is_string($parentAlias) && ! isset($this->_pendingFields[$parentAlias])
                    && $this->_hydrator->getHydrationMode() != Doctrine_Core::HYDRATE_NONE
                    && $this->_hydrator->getHydrationMode() != Doctrine_Core::HYDRATE_SCALAR
                    && $this->_hydrator->getHydrationMode() != Doctrine_Core::HYDRATE_SINGLE_SCALAR) {
                throw new Doctrine_Query_Exception("The left side of the join between "
                        . "the aliases '$parentAlias' and '$componentAlias' must have at least"
                        . " the primary key field(s) selected.");
            }
        }

        $fields = $this->_pendingFields[$componentAlias];

        // check for wildcards
        if (in_array('*', $fields)) {
            $fields = $table->getFieldNames();
        } else {
            $driverClassName = $this->_hydrator->getHydratorDriverClassName();
            // only auto-add the primary key fields if this query object is not
            // a subquery of another query object or we're using a child of the Object Graph
            // hydrator
            if ( ! $this->_isSubquery && is_subclass_of($driverClassName, 'Doctrine_Hydrator_Graph')) {
                $fields = array_unique(array_merge((array) $table->getIdentifier(), $fields));
            }
        }

        $sql = array();
        foreach ($fields as $fieldName) {
            $columnName = $table->getColumnName($fieldName);
            if (($owner = $table->getColumnOwner($columnName)) !== null &&
                    $owner !== $table->getComponentName()) {

                $parent = $this->_conn->getTable($owner);
                $columnName = $parent->getColumnName($fieldName);
                $parentAlias = $this->getSqlTableAlias($componentAlias . '.' . $parent->getComponentName());
                $sql[] = $this->_conn->quoteIdentifier($parentAlias) . '.' . $this->_conn->quoteIdentifier($columnName)
                       . ' AS '
                       . $this->_conn->quoteIdentifier($tableAlias . '__' . $columnName);
            } else {
                $columnName = $table->getColumnName($fieldName);
                $sql[] = $this->_conn->quoteIdentifier($tableAlias) . '.' . $this->_conn->quoteIdentifier($columnName)
                       . ' AS '
                       . $this->_conn->quoteIdentifier($tableAlias . '__' . $columnName);
            }
        }

        $this->_neededTables[] = $tableAlias;

        return implode(', ', $sql);
    }

    /**
     * Parses a nested field
     * <code>
     * $q->parseSelectField('u.Phonenumber.value');
     * </code>
     *
     * @param string $field
     * @throws Doctrine_Query_Exception     if unknown component alias has been given
     * @return string   SQL fragment
     * @todo Description: Explain what this method does. Is there a relation to parseSelect()?
     *       This method is not used from any class or testcase in the Doctrine package.
     *
     */
    public function parseSelectField($field)
    {
        $terms = explode('.', $field);

        if (isset($terms[1])) {
            $componentAlias = $terms[0];
            $field = $terms[1];
        } else {
            reset($this->_queryComponents);
            $componentAlias = key($this->_queryComponents);
            $fields = $terms[0];
        }

        $tableAlias = $this->getSqlTableAlias($componentAlias);
        $table      = $this->_queryComponents[$componentAlias]['table'];


        // check for wildcards
        if ($field === '*') {
            $sql = array();

            foreach ($table->getColumnNames() as $field) {
                $sql[] = $this->parseSelectField($componentAlias . '.' . $field);
            }

            return implode(', ', $sql);
        } else {
            $name = $table->getColumnName($field);

            $this->_neededTables[] = $tableAlias;

            return $this->_conn->quoteIdentifier($tableAlias . '.' . $name)
                   . ' AS '
                   . $this->_conn->quoteIdentifier($tableAlias . '__' . $name);
        }
    }

    /**
     * getExpressionOwner
     * returns the component alias for owner of given expression
     *
     * @param string $expr      expression from which to get to owner from
     * @return string           the component alias
     * @todo Description: What does it mean if a component is an 'owner' of an expression?
     *       What kind of 'expression' are we talking about here?
     */
    public function getExpressionOwner($expr)
    {
        if (strtoupper(substr(trim($expr, '( '), 0, 6)) !== 'SELECT') {
            preg_match_all("/[a-z_][a-z0-9_]*\.[a-z_][a-z0-9_]*[\.[a-z0-9]+]*/i", $expr, $matches);

            $match = current($matches);

            if (isset($match[0])) {
                $terms = explode('.', $match[0]);

                return $terms[0];
            }
        }
        return $this->getRootAlias();

    }

    /**
     * parseSelect
     * parses the query select part and
     * adds selected fields to pendingFields array
     *
     * @param string $dql
     * @todo Description: What information is extracted (and then stored)?
     */
    public function parseSelect($dql)
    {
        $refs = $this->_tokenizer->sqlExplode($dql, ',');

        $pos   = strpos(trim($refs[0]), ' ');
        $first = substr($refs[0], 0, $pos);

        // check for DISTINCT keyword
        if ($first === 'DISTINCT') {
            $this->_sqlParts['distinct'] = true;

            $refs[0] = substr($refs[0], ++$pos);
        }

        $parsedComponents = array();

        foreach ($refs as $reference) {
            $reference = trim($reference);

            if (empty($reference)) {
                continue;
            }

            $terms = $this->_tokenizer->sqlExplode($reference, ' ');
            $pos   = strpos($terms[0], '(');

            if (count($terms) > 1 || $pos !== false) {
                $expression = array_shift($terms);
                $alias = array_pop($terms);

                if ( ! $alias) {
                    $alias = substr($expression, 0, $pos);
                }
				
				
			//Patched By Will Ferrer to detect when a subquery was encountered and use an existing componentAlias if the expression is a subquery. (not well tested) Fixes Bug -- DC-706
				if ($pos !== false && substr($expression, 0, 1) !== "'" && substr($expression, 0, $pos) == '') {
					$_queryComponents = $this->_queryComponents;
					reset($_queryComponents);
					$componentAlias = key($_queryComponents);
				} else {
               		$componentAlias = $this->getExpressionOwner($expression);
				}
			//End Patch
			
				$expression = $this->parseClause($expression);

                $tableAlias = $this->getSqlTableAlias($componentAlias);
				
                $index    = count($this->_aggregateAliasMap);

                $sqlAlias = $this->_conn->quoteIdentifier($tableAlias . '__' . $index);

                $this->_sqlParts['select'][] = $expression . ' AS ' . $sqlAlias;

                $this->_aggregateAliasMap[$alias] = $sqlAlias;
                $this->_expressionMap[$alias][0] = $expression;

                $this->_queryComponents[$componentAlias]['agg'][$index] = $alias;
				
                $this->_neededTables[] = $tableAlias;
            } else {
                $e = explode('.', $terms[0]);

                if (isset($e[1])) {
                    $componentAlias = $e[0];
                    $field = $e[1];
                } else {
                    reset($this->_queryComponents);
                    $componentAlias = key($this->_queryComponents);
                    $field = $e[0];
                }

                $this->_pendingFields[$componentAlias][] = $field;
            }
        }
    }

    /**
     * parseClause
     * parses given DQL clause
     *
     * this method handles five tasks:
     *
     * 1. Converts all DQL functions to their native SQL equivalents
     * 2. Converts all component references to their table alias equivalents
     * 3. Converts all field names to actual column names
     * 4. Quotes all identifiers
     * 5. Parses nested clauses and subqueries recursively
     *
     * @return string   SQL string
     * @todo Description: What is a 'dql clause' (and what not)?
     *       Refactor: Too long & nesting level
     */
    public function parseClause($clause)
    {
        $clause = $this->_conn->dataDict->parseBoolean(trim($clause));

        if (is_numeric($clause)) {
           return $clause;
        }

        $terms = $this->_tokenizer->clauseExplode($clause, array(' ', '+', '-', '*', '/', '<', '>', '=', '>=', '<=', '&', '|'));
        $str = '';

        foreach ($terms as $term) {
            $pos = strpos($term[0], '(');

            if ($pos !== false && substr($term[0], 0, 1) !== "'") {
                $name = substr($term[0], 0, $pos);

                $term[0] = $this->parseFunctionExpression($term[0]);
            } else {
                if (substr($term[0], 0, 1) !== "'" && substr($term[0], -1) !== "'") {
                    if (strpos($term[0], '.') !== false) {
                        if ( ! is_numeric($term[0])) {
                            $e = explode('.', $term[0]);

                            $field = array_pop($e);

                            if ($this->getType() === Doctrine_Query::SELECT) {
                                $componentAlias = implode('.', $e);

                                if (empty($componentAlias)) {
                                    $componentAlias = $this->getRootAlias();
                                }

                                $this->load($componentAlias);

                                // check the existence of the component alias
                                if ( ! isset($this->_queryComponents[$componentAlias])) {
                                    throw new Doctrine_Query_Exception('Unknown component alias ' . $componentAlias);
                                }

                                $table = $this->_queryComponents[$componentAlias]['table'];

                                $def = $table->getDefinitionOf($field);

                                // get the actual field name from alias
                                $field = $table->getColumnName($field);

                                // check column existence
                                if ( ! $def) {
                                    throw new Doctrine_Query_Exception('Unknown column ' . $field);
                                }

                                if (isset($def['owner'])) {
                                    $componentAlias = $componentAlias . '.' . $def['owner'];
                                }

                                $tableAlias = $this->getSqlTableAlias($componentAlias);

                                // build sql expression
                                $term[0] = $this->_conn->quoteIdentifier($tableAlias)
                                         . '.'
                                         . $this->_conn->quoteIdentifier($field);
                            } else {
                                // build sql expression
                                $field = $this->getRoot()->getColumnName($field);
                                $term[0] = $this->_conn->quoteIdentifier($field);
                            }
                        }
                    } else {
                        if ( ! empty($term[0]) && ! in_array(strtoupper($term[0]), self::$_keywords) &&
                             ! is_numeric($term[0]) && $term[0] !== '?' && substr($term[0], 0, 1) !== ':') {

                            $componentAlias = $this->getRootAlias();

                            $found = false;

                            if ($componentAlias !== false && $componentAlias !== null) {
                                $table = $this->_queryComponents[$componentAlias]['table'];

                                // check column existence
                                if ($table->hasField($term[0])) {
                                    $found = true;

                                    $def = $table->getDefinitionOf($term[0]);

                                    // get the actual column name from field name
                                    $term[0] = $table->getColumnName($term[0]);


                                    if (isset($def['owner'])) {
                                        $componentAlias = $componentAlias . '.' . $def['owner'];
                                    }

                                    $tableAlias = $this->getSqlTableAlias($componentAlias);

                                    if ($this->getType() === Doctrine_Query::SELECT) {
                                        // build sql expression
                                        $term[0] = $this->_conn->quoteIdentifier($tableAlias)
                                                 . '.'
                                                 . $this->_conn->quoteIdentifier($term[0]);
                                    } else {
                                        // build sql expression
                                        $term[0] = $this->_conn->quoteIdentifier($term[0]);
                                    }
                                } else {
                                    $found = false;
                                }
                            }

                            if ( ! $found) {
                                $term[0] = $this->getSqlAggregateAlias($term[0]);
                            }
                        }
                    }
                }
            }

            $str .= $term[0] . $term[1];
        }
        return $str;
    }

    public function parseIdentifierReference($expr)
    {

    }

    public function parseFunctionExpression($expr)
    {
        $pos = strpos($expr, '(');
        $name = substr($expr, 0, $pos);

        if ($name === '') {
            return $this->parseSubquery($expr);
        }

        $argStr = substr($expr, ($pos + 1), -1);
        $args   = array();
        // parse args

        foreach ($this->_tokenizer->sqlExplode($argStr, ',') as $arg) {
           $args[] = $this->parseClause($arg);
        }

        // convert DQL function to its RDBMS specific equivalent
        try {
            $expr = call_user_func_array(array($this->_conn->expression, $name), $args);
        } catch (Doctrine_Expression_Exception $e) {
            throw new Doctrine_Query_Exception('Unknown function ' . $name . '.');
        }

        return $expr;
    }


    public function parseSubquery($subquery)
    {
        $trimmed = trim($this->_tokenizer->bracketTrim($subquery));

        // check for possible subqueries
        if (substr($trimmed, 0, 4) == 'FROM' || substr($trimmed, 0, 6) == 'SELECT') {
            // parse subquery
            $q = $this->createSubquery()->parseDqlQuery($trimmed);
            $trimmed = $q->getSqlQuery();
            $q->free();
        } else if (substr($trimmed, 0, 4) == 'SQL:') {
            $trimmed = substr($trimmed, 4);
        } else {
            $e = $this->_tokenizer->sqlExplode($trimmed, ',');

            $value = array();
            $index = false;

            foreach ($e as $part) {
                $value[] = $this->parseClause($part);
            }

            $trimmed = implode(', ', $value);
        }

        return '(' . $trimmed . ')';
    }


    /**
     * processPendingSubqueries
     * processes pending subqueries
     *
     * subqueries can only be processed when the query is fully constructed
     * since some subqueries may be correlated
     *
     * @return void
     * @todo Better description. i.e. What is a 'pending subquery'? What does 'processed' mean?
     *       (parsed? sql is constructed? some information is gathered?)
     */
    public function processPendingSubqueries()
    {
        foreach ($this->_pendingSubqueries as $value) {
            list($dql, $alias) = $value;

            $subquery = $this->createSubquery();

            $sql = $subquery->parseDqlQuery($dql, false)->getQuery();
            $subquery->free();

            reset($this->_queryComponents);
            $componentAlias = key($this->_queryComponents);
            $tableAlias = $this->getSqlTableAlias($componentAlias);

            $sqlAlias = $tableAlias . '__' . count($this->_aggregateAliasMap);

            $this->_sqlParts['select'][] = '(' . $sql . ') AS ' . $this->_conn->quoteIdentifier($sqlAlias);

            $this->_aggregateAliasMap[$alias] = $sqlAlias;
            $this->_queryComponents[$componentAlias]['agg'][] = $alias;
        }
        $this->_pendingSubqueries = array();
    }

    /**
     * processPendingAggregates
     * processes pending aggregate values for given component alias
     *
     * @return void
     * @todo Better description. i.e. What is a 'pending aggregate'? What does 'processed' mean?
     */
    public function processPendingAggregates()
    {
        // iterate trhough all aggregates
        foreach ($this->_pendingAggregates as $aggregate) {
            list ($expression, $components, $alias) = $aggregate;

            $tableAliases = array();

            // iterate through the component references within the aggregate function
            if ( ! empty ($components)) {
                foreach ($components as $component) {

                    if (is_numeric($component)) {
                        continue;
                    }

                    $e = explode('.', $component);

                    $field = array_pop($e);
                    $componentAlias = implode('.', $e);

                    // check the existence of the component alias
                    if ( ! isset($this->_queryComponents[$componentAlias])) {
                        throw new Doctrine_Query_Exception('Unknown component alias ' . $componentAlias);
                    }

                    $table = $this->_queryComponents[$componentAlias]['table'];

                    $field = $table->getColumnName($field);

                    // check column existence
                    if ( ! $table->hasColumn($field)) {
                        throw new Doctrine_Query_Exception('Unknown column ' . $field);
                    }

                    $sqlTableAlias = $this->getSqlTableAlias($componentAlias);

                    $tableAliases[$sqlTableAlias] = true;

                    // build sql expression

                    $identifier = $this->_conn->quoteIdentifier($sqlTableAlias . '.' . $field);
                    $expression = str_replace($component, $identifier, $expression);
                }
            }

            if (count($tableAliases) !== 1) {
                $componentAlias = reset($this->_tableAliasMap);
                $tableAlias = key($this->_tableAliasMap);
            }

            $index    = count($this->_aggregateAliasMap);
            $sqlAlias = $this->_conn->quoteIdentifier($tableAlias . '__' . $index);

            $this->_sqlParts['select'][] = $expression . ' AS ' . $sqlAlias;

            $this->_aggregateAliasMap[$alias] = $sqlAlias;
            $this->_expressionMap[$alias][0] = $expression;

            $this->_queryComponents[$componentAlias]['agg'][$index] = $alias;

            $this->_neededTables[] = $tableAlias;
        }
        // reset the state
        $this->_pendingAggregates = array();
    }

    /**
     * _buildSqlQueryBase
     * returns the base of the generated sql query
     * On mysql driver special strategy has to be used for DELETE statements
     * (where is this special strategy??)
     *
     * @return string       the base of the generated sql query
     */
    protected function _buildSqlQueryBase()
    {
        switch ($this->_type) {
            case self::DELETE:
                $q = 'DELETE FROM ';
            break;
            case self::UPDATE:
                $q = 'UPDATE ';
            break;
            case self::SELECT:
                $distinct = ($this->_sqlParts['distinct']) ? 'DISTINCT ' : '';
                $q = 'SELECT ' . $distinct . implode(', ', $this->_sqlParts['select']) . ' FROM ';
            break;
        }
        return $q;
    }

    /**
     * _buildSqlFromPart
     * builds the from part of the query and returns it
     *
     * @return string   the query sql from part
     */
    protected function _buildSqlFromPart($ignorePending = false)
    {
        $q = '';

        foreach ($this->_sqlParts['from'] as $k => $part) {
            $e = explode(' ', $part);

            if ($k === 0) {
                if ( ! $ignorePending && $this->_type == self::SELECT) {
                    // We may still have pending conditions
                    $alias = count($e) > 1
                        ? $this->getComponentAlias($e[1])
                        : null;
                    $where = $this->_processPendingJoinConditions($alias);

                    // apply inheritance to WHERE part
                    if ( ! empty($where)) {
                        if (count($this->_sqlParts['where']) > 0) {
                            $this->_sqlParts['where'][] = 'AND';
                        }

                        if (substr($where, 0, 1) === '(' && substr($where, -1) === ')') {
                            $this->_sqlParts['where'][] = $where;
                        } else {
                            $this->_sqlParts['where'][] = '(' . $where . ')';
                        }
                    }
                }

                $q .= $part;

                continue;
            }

            // preserve LEFT JOINs only if needed
            // Check if it's JOIN, if not add a comma separator instead of space
            if ( ! preg_match('/\bJOIN\b/i', $part) && ! isset($this->_pendingJoinConditions[$k])) {
                $q .= ', ' . $part;
            } else {
                if (substr($part, 0, 9) === 'LEFT JOIN') {
                    $aliases = array_merge($this->_subqueryAliases,
                                array_keys($this->_neededTables));

                    if ( ! in_array($e[3], $aliases) && ! in_array($e[2], $aliases) && ! empty($this->_pendingFields)) {
                        continue;
                    }

                }

                if ( ! $ignorePending && isset($this->_pendingJoinConditions[$k])) {
                    if (strpos($part, ' ON ') !== false) {
                        $part .= ' AND ';
                    } else {
                        $part .= ' ON ';
                    }

                    $part .= $this->_processPendingJoinConditions($k);
                }

                $componentAlias = $this->getComponentAlias($e[3]);
                $string = $this->getInheritanceCondition($componentAlias);

                if ($string) {
                    $part = $part . ' AND ' . $string;
                }
                $q .= ' ' . $part;
            }

            $this->_sqlParts['from'][$k] = $part;
        }
        return $q;
    }

    /**
     * Processes the pending join conditions, used for dynamically add conditions
     * to root component/joined components without interfering in the main dql
     * handling.
     *
     * @param string $alias Component Alias
     * @return Processed pending conditions
     */
    protected function _processPendingJoinConditions($alias)
    {
        $parts = array();

        if ($alias !== null && isset($this->_pendingJoinConditions[$alias])) {
            $parser = new Doctrine_Query_JoinCondition($this, $this->_tokenizer);

            foreach ($this->_pendingJoinConditions[$alias] as $joinCondition) {
                $parts[] = $parser->parse($joinCondition);
            }

            // FIX #1860 and #1876: Cannot unset them, otherwise query cannot be reused later
            //unset($this->_pendingJoinConditions[$alias]);
        }

        return (count($parts) > 0 ? '(' . implode(') AND (', $parts) . ')' : '');
    }

    /**
     * builds the sql query from the given parameters and applies things such as
     * column aggregation inheritance and limit subqueries if needed
     *
     * @param array $params             an array of prepared statement params (needed only in mysql driver
     *                                  when limit subquery algorithm is used)
     * @param bool $limitSubquery Whether or not to try and apply the limit subquery algorithm
     * @return string                   the built sql query
     */
    public function getSqlQuery($params = array(), $limitSubquery = true)
    {
        // Assign building/execution specific params
        $this->_params['exec'] = $params;

        // Initialize prepared parameters array
        $this->_execParams = $this->getFlattenedParams();

        if ($this->_state !== self::STATE_DIRTY) {
            $this->fixArrayParameterValues($this->getInternalParams());

            // Return compiled SQL
            return $this->_sql;
        }
        return $this->buildSqlQuery($limitSubquery);
    }

    /**
     * Build the SQL query from the DQL
     *
     * @param bool $limitSubquery Whether or not to try and apply the limit subquery algorithm
     * @return string $sql The generated SQL string
     */
    public function buildSqlQuery($limitSubquery = true)
    {
        // reset the state
        if ( ! $this->isSubquery()) {
            $this->_queryComponents = array();
            $this->_pendingAggregates = array();
            $this->_aggregateAliasMap = array();
        }

        $this->reset();

        // invoke the preQuery hook
        $this->_preQuery();

        // process the DQL parts => generate the SQL parts.
        // this will also populate the $_queryComponents.
        foreach ($this->_dqlParts as $queryPartName => $queryParts) {
            // If we are parsing FROM clause, we'll need to diff the queryComponents later
            if ($queryPartName == 'from') {
                // Pick queryComponents before processing
                $queryComponentsBefore = $this->getQueryComponents();
            }

            // FIX #1667: _sqlParts are cleaned inside _processDqlQueryPart.
            if ($queryPartName != 'forUpdate') {
                $this->_processDqlQueryPart($queryPartName, $queryParts);
            }

            // We need to define the root alias
            if ($queryPartName == 'from') {
                // Pick queryComponents aftr processing
                $queryComponentsAfter = $this->getQueryComponents();

                // Root alias is the key of difference of query components
                $diffQueryComponents = array_diff_key($queryComponentsAfter, $queryComponentsBefore);
                $this->_rootAlias = key($diffQueryComponents);
            }
        }
        $this->_state = self::STATE_CLEAN;

        // Proceed with the generated SQL
        if (empty($this->_sqlParts['from'])) {
            return false;
        }

        $needsSubQuery = false;
        $subquery = '';
        $map = $this->getRootDeclaration();
        $table = $map['table'];
        $rootAlias = $this->getRootAlias();

        if ( ! empty($this->_sqlParts['limit']) && $this->_needsSubquery &&
                $table->getAttribute(Doctrine_Core::ATTR_QUERY_LIMIT) == Doctrine_Core::LIMIT_RECORDS) {
            // We do not need a limit-subquery if DISTINCT is used
            // and the selected fields are either from the root component or from a localKey relation (hasOne)
            // (i.e. DQL: SELECT DISTINCT u.id FROM User u LEFT JOIN u.phonenumbers LIMIT 5).
            if(!$this->_sqlParts['distinct']) {
                $this->_isLimitSubqueryUsed = true;
                $needsSubQuery = true;
            } else {
                foreach( array_keys($this->_pendingFields) as $alias){
                    //no subquery for root fields
                    if($alias == $this->getRootAlias()){
                        continue;
                    }

                    //no subquery for ONE relations
                    if(isset($this->_queryComponents[$alias]['relation']) &&
                        $this->_queryComponents[$alias]['relation']->getType() == Doctrine_Relation::ONE){
                        continue;
                    }

                    $this->_isLimitSubqueryUsed = true;
                    $needsSubQuery = true;
                }
            }
        }

        $sql = array();

        if ( ! empty($this->_pendingFields)) {
            foreach ($this->_queryComponents as $alias => $map) {
                $fieldSql = $this->processPendingFields($alias);
                if ( ! empty($fieldSql)) {
                    $sql[] = $fieldSql;
                }
            }
        }

        if ( ! empty($sql)) {
            array_unshift($this->_sqlParts['select'], implode(', ', $sql));
        }

        $this->_pendingFields = array();

        // build the basic query
        $q  = $this->_buildSqlQueryBase();
        $q .= $this->_buildSqlFromPart();

        if ( ! empty($this->_sqlParts['set'])) {
            $q .= ' SET ' . implode(', ', $this->_sqlParts['set']);
        }

        $string = $this->getInheritanceCondition($this->getRootAlias());

        // apply inheritance to WHERE part
        if ( ! empty($string)) {
            if (count($this->_sqlParts['where']) > 0) {
                $this->_sqlParts['where'][] = 'AND';
            }

            if (substr($string, 0, 1) === '(' && substr($string, -1) === ')') {
                $this->_sqlParts['where'][] = $string;
            } else {
                $this->_sqlParts['where'][] = '(' . $string . ')';
            }
        }

        $modifyLimit = true;
        $limitSubquerySql = '';

        if ( ( ! empty($this->_sqlParts['limit']) || ! empty($this->_sqlParts['offset'])) && $needsSubQuery && $limitSubquery) {
            $subquery = $this->getLimitSubquery();

            // what about composite keys?
            $idColumnName = $table->getColumnName($table->getIdentifier());

            switch (strtolower($this->_conn->getDriverName())) {
                case 'mysql':
                    $this->useQueryCache(false);

                    // mysql doesn't support LIMIT in subqueries
                    $list = $this->_conn->execute($subquery, $this->_execParams)->fetchAll(Doctrine_Core::FETCH_COLUMN);
                    $subquery = implode(', ', array_map(array($this->_conn, 'quote'), $list));

                    break;

                case 'pgsql':
                    $subqueryAlias = $this->_conn->quoteIdentifier('doctrine_subquery_alias');

                    // pgsql needs special nested LIMIT subquery
                    $subquery = 'SELECT ' . $subqueryAlias . '.' . $this->_conn->quoteIdentifier($idColumnName)
                            . ' FROM (' . $subquery . ') AS ' . $subqueryAlias;

                    break;
            }

            $field = $this->getSqlTableAlias($rootAlias) . '.' . $idColumnName;

            // FIX #1868: If not ID under MySQL is found to be restricted, restrict pk column for null
            //            (which will lead to a return of 0 items)
            $limitSubquerySql = $this->_conn->quoteIdentifier($field)
                              . (( ! empty($subquery)) ? ' IN (' . $subquery . ')' : ' IS NULL')
                              . ((count($this->_sqlParts['where']) > 0) ? ' AND ' : '');

            $modifyLimit = false;
        }

        // FIX #DC-26: Include limitSubquerySql as major relevance in conditions
        $emptyWhere = empty($this->_sqlParts['where']);

        if ( ! ($emptyWhere && $limitSubquerySql == '')) {
            $where = implode(' ', $this->_sqlParts['where']);
            $where = ($where == '' || (substr($where, 0, 1) === '(' && substr($where, -1) === ')'))
                ? $where : '(' . $where . ')';

            $q .= ' WHERE ' . $limitSubquerySql . $where;
            //   .  (($limitSubquerySql == '' && count($this->_sqlParts['where']) == 1) ? substr($where, 1, -1) : $where);
        }

        // Fix the orderbys so we only have one orderby per value
        foreach ($this->_sqlParts['orderby'] as $k => $orderBy) {
            $e = explode(', ', $orderBy);
            unset($this->_sqlParts['orderby'][$k]);
            foreach ($e as $v) {
                $this->_sqlParts['orderby'][] = $v;
            }
        }

        // Add the default orderBy statements defined in the relationships and table classes
        // Only do this for SELECT queries
        if ($this->_type === self::SELECT) {
            foreach ($this->_queryComponents as $alias => $map) {
                $sqlAlias = $this->getSqlTableAlias($alias);
                if (isset($map['relation'])) {
                    $orderBy = $map['relation']->getOrderByStatement($sqlAlias, true);
                    if ($orderBy == $map['relation']['orderBy']) {
                        if (isset($map['ref'])) {
                            $orderBy = $map['relation']['refTable']->processOrderBy($sqlAlias, $map['relation']['orderBy'], true);
                        } else {
                            $orderBy = null;
                        }
                    }
				} else {
					$orderBy = $map['table']->getOrderByStatement($sqlAlias, true);
				}

                if ($orderBy) {
                    $e = explode(',', $orderBy);
                    $e = array_map('trim', $e);
                    foreach ($e as $v) {
                        if ( ! in_array($v, $this->_sqlParts['orderby'])) {
                            $this->_sqlParts['orderby'][] = $v;
                        }
                    }
                }
            }
        }

        $q .= ( ! empty($this->_sqlParts['groupby'])) ? ' GROUP BY ' . implode(', ', $this->_sqlParts['groupby'])  : '';
        $q .= ( ! empty($this->_sqlParts['having'])) ?  ' HAVING '   . implode(' AND ', $this->_sqlParts['having']): '';
        $q .= ( ! empty($this->_sqlParts['orderby'])) ? ' ORDER BY ' . implode(', ', $this->_sqlParts['orderby'])  : '';

        if ($modifyLimit) {
            $q = $this->_conn->modifyLimitQuery($q, $this->_sqlParts['limit'], $this->_sqlParts['offset']);
        }

        $q .= $this->_sqlParts['forUpdate'] === true ? ' FOR UPDATE ' : '';

        $this->_sql = $q;

        $this->clear();

        return $q;
    }

    /**
     * getLimitSubquery
     * this is method is used by the record limit algorithm
     *
     * when fetching one-to-many, many-to-many associated data with LIMIT clause
     * an additional subquery is needed for limiting the number of returned records instead
     * of limiting the number of sql result set rows
     *
     * @return string       the limit subquery
     * @todo A little refactor to make the method easier to understand & maybe shorter?
     */
    public function getLimitSubquery()
    {
        $map = reset($this->_queryComponents);
        $table = $map['table'];
        $componentAlias = key($this->_queryComponents);

        // get short alias
        $alias = $this->getSqlTableAlias($componentAlias);
        // what about composite keys?
        $primaryKey = $alias . '.' . $table->getColumnName($table->getIdentifier());

        $driverName = $this->_conn->getAttribute(Doctrine_Core::ATTR_DRIVER_NAME);

        // initialize the base of the subquery
        if (($driverName == 'oracle' || $driverName == 'oci') && $this->_isOrderedByJoinedColumn()) {
            $subquery = 'SELECT ';
        } else {
            $subquery = 'SELECT DISTINCT ';
        }
        $subquery .= $this->_conn->quoteIdentifier($primaryKey);

        // pgsql & oracle need the order by fields to be preserved in select clause
        if ($driverName == 'pgsql' || $driverName == 'oracle' || $driverName == 'oci' || $driverName == 'mssql' || $driverName == 'odbc') {
            foreach ($this->_sqlParts['orderby'] as $part) {
                // Remove identifier quoting if it exists
                $e = $this->_tokenizer->bracketExplode($part, ' ');
                foreach ($e as $f) {
                    if ($f == 0 || $f % 2 == 0) {
                        $partOriginal = str_replace(',', '', trim($f));
                        $callback = create_function('$e', 'return trim($e, \'[]`"\');');
                        $part = trim(implode('.', array_map($callback, explode('.', $partOriginal))));
                
                        if (strpos($part, '.') === false) {
                            continue;
                        }
                
                        // don't add functions
                        if (strpos($part, '(') !== false) {
                            continue;
                        }
                
                        // don't add primarykey column (its already in the select clause)
                        if ($part !== $primaryKey) {
                            $subquery .= ', ' . $partOriginal;
                        }
                    }
                }
            }
        }

        $orderby = $this->_sqlParts['orderby'];
        $having = $this->_sqlParts['having'];
        if ($driverName == 'mysql' || $driverName == 'pgsql') {
            foreach ($this->_expressionMap as $dqlAlias => $expr) {
                if (isset($expr[1])) {
                    $subquery .= ', ' . $expr[0] . ' AS ' . $this->_aggregateAliasMap[$dqlAlias];
                }
            }
        } else {
            foreach ($this->_expressionMap as $dqlAlias => $expr) {
                if (isset($expr[1])) {
                    foreach ($having as $k => $v) {
                        $having[$k] = str_replace($this->_aggregateAliasMap[$dqlAlias], $expr[0], $v);
                    }
                    foreach ($orderby as $k => $v) {
                        $e = explode(' ', $v);
                        if ($e[0] == $this->_aggregateAliasMap[$dqlAlias]) {
                            $orderby[$k] = $expr[0];
                        }
                    }
                }
            }
        }

        // Add having fields that got stripped out of select
        preg_match_all('/`[a-z0-9_]+`\.`[a-z0-9_]+`/i', implode(' ', $having), $matches, PREG_PATTERN_ORDER);
        if (count($matches[0]) > 0) {
            $subquery .= ', ' . implode(', ', array_unique($matches[0]));
        }

        $subquery .= ' FROM';

        foreach ($this->_sqlParts['from'] as $part) {
            // preserve LEFT JOINs only if needed
            if (substr($part, 0, 9) === 'LEFT JOIN') {
                $e = explode(' ', $part);
			//Patched by Will Ferrer to also check for groupBys. Fixes Bug -- DC-594
                if (empty($this->_sqlParts['orderby']) && empty($this->_sqlParts['where']) && empty($this->_sqlParts['having']) && empty($this->_sqlParts['groupby'])) {
             //End Patch
				   continue;
                }
            }

            $subquery .= ' ' . $part;
        }

        // all conditions must be preserved in subquery
        $subquery .= ( ! empty($this->_sqlParts['where']))?   ' WHERE '    . implode(' ', $this->_sqlParts['where'])  : '';
        $subquery .= ( ! empty($this->_sqlParts['groupby']))? ' GROUP BY ' . implode(', ', $this->_sqlParts['groupby'])   : '';
        $subquery .= ( ! empty($having))?  ' HAVING '   . implode(' AND ', $having) : '';
        $subquery .= ( ! empty($orderby))? ' ORDER BY ' . implode(', ', $orderby)  : '';

        if (($driverName == 'oracle' || $driverName == 'oci') && $this->_isOrderedByJoinedColumn()) {
            // When using "ORDER BY x.foo" where x.foo is a column of a joined table,
            // we may get duplicate primary keys because all columns in ORDER BY must appear
            // in the SELECT list when using DISTINCT. Hence we need to filter out the
            // primary keys with an additional DISTINCT subquery.
            // #1038
            $quotedIdentifierColumnName = $this->_conn->quoteIdentifier($table->getColumnName($table->getIdentifier()));
            $subquery = 'SELECT doctrine_subquery_alias.' . $quotedIdentifierColumnName
                    . ' FROM (' . $subquery . ') doctrine_subquery_alias'
                    . ' GROUP BY doctrine_subquery_alias.' . $quotedIdentifierColumnName
                    . ' ORDER BY MIN(ROWNUM)';
        }

        // add driver specific limit clause
        $subquery = $this->_conn->modifyLimitSubquery($table, $subquery, $this->_sqlParts['limit'], $this->_sqlParts['offset']);

        $parts = $this->_tokenizer->quoteExplode($subquery, ' ', "'", "'");

        foreach ($parts as $k => $part) {
            if (strpos($part, ' ') !== false) {
                continue;
            }

            $part = str_replace(array('"', "'", '`'), "", $part);

            if ($this->hasSqlTableAlias($part)) {
                $parts[$k] = $this->_conn->quoteIdentifier($this->generateNewSqlTableAlias($part));
                continue;
            }

            if (strpos($part, '.') === false) {
                continue;
            }

            preg_match_all("/[a-zA-Z0-9_]+\.[a-z0-9_]+/i", $part, $m);

            foreach ($m[0] as $match) {
                $e = explode('.', $match);

                // Rebuild the original part without the newly generate alias and with quoting reapplied
                $e2 = array();
                foreach ($e as $k2 => $v2) {
                  $e2[$k2] = $this->_conn->quoteIdentifier($v2);
                }
                $match = implode('.', $e2);

                // Generate new table alias
                $e[0] = $this->generateNewSqlTableAlias($e[0]);

                // Requote the part with the newly generated alias
                foreach ($e as $k2 => $v2) {
                  $e[$k2] = $this->_conn->quoteIdentifier($v2);
                }

                $replace = implode('.' , $e);

                // Replace the original part with the new part with new sql table alias
                $parts[$k] = str_replace($match, $replace, $parts[$k]);
            }
        }

        if ($driverName == 'mysql' || $driverName == 'pgsql') {
            foreach ($parts as $k => $part) {
                if (strpos($part, "'") !== false) {
                    continue;
                }
                if (strpos($part, '__') == false) {
                    continue;
                }

                preg_match_all("/[a-zA-Z0-9_]+\_\_[a-z0-9_]+/i", $part, $m);

                foreach ($m[0] as $match) {
                    $e = explode('__', $match);
                    $e[0] = $this->generateNewSqlTableAlias($e[0]);

                    $parts[$k] = str_replace($match, implode('__', $e), $parts[$k]);
                }
            }
        }

        $subquery = implode(' ', $parts);
        return $subquery;
    }

    /**
     * Checks whether the query has an ORDER BY on a column of a joined table.
     * This information is needed in special scenarios like the limit-offset when its
     * used with an Oracle database.
     *
     * @return boolean  TRUE if the query is ordered by a joined column, FALSE otherwise.
     */
    private function _isOrderedByJoinedColumn() {
        if ( ! $this->_queryComponents) {
            throw new Doctrine_Query_Exception("The query is in an invalid state for this "
                    . "operation. It must have been fully parsed first.");
        }
        $componentAlias = key($this->_queryComponents);
        $mainTableAlias = $this->getSqlTableAlias($componentAlias);
        foreach ($this->_sqlParts['orderby'] as $part) {
            $part = trim($part);
            $e = $this->_tokenizer->bracketExplode($part, ' ');
            $part = trim($e[0]);
            if (strpos($part, '.') === false) {
                continue;
            }
            list($tableAlias, $columnName) = explode('.', $part);
            if ($tableAlias != $mainTableAlias) {
                return true;
            }
        }
        return false;
    }

    /**
     * DQL PARSER
     * parses a DQL query
     * first splits the query in parts and then uses individual
     * parsers for each part
     *
     * @param string $query                 DQL query
     * @param boolean $clear                whether or not to clear the aliases
     * @throws Doctrine_Query_Exception     if some generic parsing error occurs
     * @return Doctrine_Query
     */
    public function parseDqlQuery($query, $clear = true)
    {
        if ($clear) {
            $this->clear();
        }

        $query = trim($query);
        $query = str_replace("\r", "\n", str_replace("\r\n", "\n", $query));
        $query = str_replace("\n", ' ', $query);

        $parts = $this->_tokenizer->tokenizeQuery($query);

        foreach ($parts as $partName => $subParts) {
            $subParts = trim($subParts);
            $partName = strtolower($partName);
            switch ($partName) {
                case 'create':
                    $this->_type = self::CREATE;
                break;
                case 'insert':
                    $this->_type = self::INSERT;
                break;
                case 'delete':
                    $this->_type = self::DELETE;
                break;
                case 'select':
                    $this->_type = self::SELECT;
                    $this->_addDqlQueryPart($partName, $subParts);
                break;
                case 'update':
                    $this->_type = self::UPDATE;
                    $partName = 'from';
                case 'from':
                    $this->_addDqlQueryPart($partName, $subParts);
                break;
                case 'set':
                    $this->_addDqlQueryPart($partName, $subParts, true);
                break;
                case 'group':
                case 'order':
                    $partName .= 'by';
                case 'where':
                case 'having':
                case 'limit':
                case 'offset':
                    $this->_addDqlQueryPart($partName, $subParts);
                break;
            }
        }

        return $this;
    }

    /**
     * @todo Describe & refactor... too long and nested.
     * @param string $path          component alias
     * @param boolean $loadFields
     */
    public function load($path, $loadFields = true)
    {
        if (isset($this->_queryComponents[$path])) {
            return $this->_queryComponents[$path];
        }

        $e = $this->_tokenizer->quoteExplode($path, ' INDEXBY ');

        $mapWith = null;
        if (count($e) > 1) {
            $mapWith = trim($e[1]);

            $path = $e[0];
        }

        // parse custom join conditions
        $e = explode(' ON ', str_ireplace(' on ', ' ON ', $path));

        $joinCondition = '';

        if (count($e) > 1) {
            $joinCondition = substr($path, strlen($e[0]) + 4, strlen($e[1]));
            $path = substr($path, 0, strlen($e[0]));

            $overrideJoin = true;
        } else {
            $e = explode(' WITH ', str_ireplace(' with ', ' WITH ', $path));

            if (count($e) > 1) {
                $joinCondition = substr($path, strlen($e[0]) + 6, strlen($e[1]));
                $path = substr($path, 0, strlen($e[0]));
            }

            $overrideJoin = false;
        }

        $tmp            = explode(' ', $path);
        $componentAlias = $originalAlias = (count($tmp) > 1) ? end($tmp) : null;

        $e = preg_split("/[.:]/", $tmp[0], -1);

        $fullPath = $tmp[0];
        $prevPath = '';
        $fullLength = strlen($fullPath);

        if (isset($this->_queryComponents[$e[0]])) {
            $table = $this->_queryComponents[$e[0]]['table'];
            $componentAlias = $e[0];

            $prevPath = $parent = array_shift($e);
        }

        foreach ($e as $key => $name) {
            // get length of the previous path
            $length = strlen($prevPath);

            // build the current component path
            $prevPath = ($prevPath) ? $prevPath . '.' . $name : $name;

            $delimeter = substr($fullPath, $length, 1);

            // if an alias is not given use the current path as an alias identifier
            if (strlen($prevPath) === $fullLength && isset($originalAlias)) {
                $componentAlias = $originalAlias;
            } else {
                $componentAlias = $prevPath;
            }

            // if the current alias already exists, skip it
            if (isset($this->_queryComponents[$componentAlias])) {
                throw new Doctrine_Query_Exception("Duplicate alias '$componentAlias' in query.");
            }

            if ( ! isset($table)) {
                // process the root of the path

                $table = $this->loadRoot($name, $componentAlias);
            } else {
                $join = ($delimeter == ':') ? 'INNER JOIN ' : 'LEFT JOIN ';

                $relation = $table->getRelation($name);
                $localTable = $table;

                $table = $relation->getTable();
                $this->_queryComponents[$componentAlias] = array('table' => $table,
                                                                 'parent'   => $parent,
                                                                 'relation' => $relation,
                                                                 'map'      => null);
                if ( ! $relation->isOneToOne()) {
                   $this->_needsSubquery = true;
                }

                $localAlias   = $this->getSqlTableAlias($parent, $localTable->getTableName());
                $foreignAlias = $this->getSqlTableAlias($componentAlias, $relation->getTable()->getTableName());

                $foreignSql   = $this->_conn->quoteIdentifier($relation->getTable()->getTableName())
                              . ' '
                              . $this->_conn->quoteIdentifier($foreignAlias);

                $map = $relation->getTable()->inheritanceMap;

                if ( ! $loadFields || ! empty($map) || $joinCondition) {
                    $this->_subqueryAliases[] = $foreignAlias;
                }

                if ($relation instanceof Doctrine_Relation_Association) {
                    $asf = $relation->getAssociationTable();

                    $assocTableName = $asf->getTableName();

                    if ( ! $loadFields || ! empty($map) || $joinCondition) {
                        $this->_subqueryAliases[] = $assocTableName;
                    }

                    $assocPath = $prevPath . '.' . $asf->getComponentName() . ' ' . $componentAlias;

                    $this->_queryComponents[$assocPath] = array(
                        'parent' => $prevPath,
                        'relation' => $relation,
                        'table' => $asf,
                        'ref' => true);

                    $assocAlias = $this->getSqlTableAlias($assocPath, $asf->getTableName());

                    $queryPart = $join
                            . $this->_conn->quoteIdentifier($assocTableName)
                            . ' '
                            . $this->_conn->quoteIdentifier($assocAlias);

                    $queryPart .= ' ON (' . $this->_conn->quoteIdentifier($localAlias
                                . '.'
                                . $localTable->getColumnName($localTable->getIdentifier())) // what about composite keys?
                                . ' = '
                                . $this->_conn->quoteIdentifier($assocAlias . '.' . $relation->getLocalRefColumnName());

                    if ($relation->isEqual()) {
                        // equal nest relation needs additional condition
                        $queryPart .= ' OR '
                                    . $this->_conn->quoteIdentifier($localAlias
                                    . '.'
                                    . $table->getColumnName($table->getIdentifier()))
                                    . ' = '
                                    . $this->_conn->quoteIdentifier($assocAlias . '.' . $relation->getForeignRefColumnName());
                    }

                    $queryPart .= ')';

                    $this->_sqlParts['from'][] = $queryPart;

                    $queryPart = $join . $foreignSql;

                    if ( ! $overrideJoin) {
                        $queryPart .= $this->buildAssociativeRelationSql($relation, $assocAlias, $foreignAlias, $localAlias);
                    }
                } else {
                    $queryPart = $this->buildSimpleRelationSql($relation, $foreignAlias, $localAlias, $overrideJoin, $join);
                }

                $queryPart .= $this->buildInheritanceJoinSql($table->getComponentName(), $componentAlias);
                $this->_sqlParts['from'][$componentAlias] = $queryPart;

                if ( ! empty($joinCondition)) {
                    $this->addPendingJoinCondition($componentAlias, $joinCondition);
                }
            }

            if ($loadFields) {
                $restoreState = false;

                // load fields if necessary
                if ($loadFields && empty($this->_dqlParts['select'])) {
                    $this->_pendingFields[$componentAlias] = array('*');
                }
            }

            $parent = $prevPath;
        }

        $table = $this->_queryComponents[$componentAlias]['table'];

        return $this->buildIndexBy($componentAlias, $mapWith);
    }

    protected function buildSimpleRelationSql(Doctrine_Relation $relation, $foreignAlias, $localAlias, $overrideJoin, $join)
    {
        $queryPart = $join . $this->_conn->quoteIdentifier($relation->getTable()->getTableName())
                           . ' '
                           . $this->_conn->quoteIdentifier($foreignAlias);

        if ( ! $overrideJoin) {
            $queryPart .= ' ON '
                       . $this->_conn->quoteIdentifier($localAlias . '.' . $relation->getLocalColumnName())
                       . ' = '
                       . $this->_conn->quoteIdentifier($foreignAlias . '.' . $relation->getForeignColumnName());
        }

        return $queryPart;
    }

    protected function buildIndexBy($componentAlias, $mapWith = null)
    {
        $table = $this->_queryComponents[$componentAlias]['table'];

        $indexBy = null;
        $column = false;

        if (isset($mapWith)) {
            $terms = explode('.', $mapWith);

            if (count($terms) == 1) {
                $indexBy = $terms[0];
            } else if (count($terms) == 2) {
                $column = true;
                $indexBy = $terms[1];
            }
        } else if ($table->getBoundQueryPart('indexBy') !== null) {
            $indexBy = $table->getBoundQueryPart('indexBy');
        }

        if ($indexBy !== null) {
            if ( $column && ! $table->hasColumn($table->getColumnName($indexBy))) {
                throw new Doctrine_Query_Exception("Couldn't use key mapping. Column " . $indexBy . " does not exist.");
            }

            $this->_queryComponents[$componentAlias]['map'] = $indexBy;
        }

        return $this->_queryComponents[$componentAlias];
    }


    protected function buildAssociativeRelationSql(Doctrine_Relation $relation, $assocAlias, $foreignAlias, $localAlias)
    {
        $table = $relation->getTable();

        $queryPart = ' ON ';

        if ($relation->isEqual()) {
            $queryPart .= '(';
        }

        $localIdentifier = $table->getColumnName($table->getIdentifier());

        $queryPart .= $this->_conn->quoteIdentifier($foreignAlias . '.' . $localIdentifier)
                    . ' = '
                    . $this->_conn->quoteIdentifier($assocAlias . '.' . $relation->getForeignRefColumnName());

        if ($relation->isEqual()) {
            $queryPart .= ' OR '
                        . $this->_conn->quoteIdentifier($foreignAlias . '.' . $localIdentifier)
                        . ' = '
                        . $this->_conn->quoteIdentifier($assocAlias . '.' . $relation->getLocalRefColumnName())
                        . ') AND '
                        . $this->_conn->quoteIdentifier($foreignAlias . '.' . $localIdentifier)
                        . ' != '
                        . $this->_conn->quoteIdentifier($localAlias . '.' . $localIdentifier);
        }

        return $queryPart;
    }

    /**
     * loadRoot
     *
     * @param string $name
     * @param string $componentAlias
     * @return Doctrine_Table
     * @todo DESCRIBE ME!
     * @todo this method is called only in Doctrine_Query class. Shouldn't be private or protected?
     */
    public function loadRoot($name, $componentAlias)
    {
        // get the connection for the component
        $manager = Doctrine_Manager::getInstance();
        if ( ! $this->_passedConn && $manager->hasConnectionForComponent($name)) {
            $this->_conn = $manager->getConnectionForComponent($name);
        }

        $table = $this->_conn->getTable($name);
        $tableName = $table->getTableName();

        // get the short alias for this table
        $tableAlias = $this->getSqlTableAlias($componentAlias, $tableName);
        // quote table name
        $queryPart = $this->_conn->quoteIdentifier($tableName);

        if ($this->_type === self::SELECT) {
            $queryPart .= ' ' . $this->_conn->quoteIdentifier($tableAlias);
        }

        $this->_tableAliasMap[$tableAlias] = $componentAlias;

        $queryPart .= $this->buildInheritanceJoinSql($name, $componentAlias);

        $this->_sqlParts['from'][] = $queryPart;

        $this->_queryComponents[$componentAlias] = array('table' => $table, 'map' => null);

        return $table;
    }

    /**
     * @todo DESCRIBE ME!
     * @param string $name              component class name
     * @param string $componentAlias    alias of the component in the dql
     * @return string                   query part
     */
    public function buildInheritanceJoinSql($name, $componentAlias)
    {
        // get the connection for the component
        $manager = Doctrine_Manager::getInstance();
        if ( ! $this->_passedConn && $manager->hasConnectionForComponent($name)) {
            $this->_conn = $manager->getConnectionForComponent($name);
        }

        $table = $this->_conn->getTable($name);
        $tableName = $table->getTableName();

        // get the short alias for this table
        $tableAlias = $this->getSqlTableAlias($componentAlias, $tableName);

        $queryPart = '';

        foreach ($table->getOption('joinedParents') as $parent) {
            $parentTable = $this->_conn->getTable($parent);

            $parentAlias = $componentAlias . '.' . $parent;

            // get the short alias for the parent table
            $parentTableAlias = $this->getSqlTableAlias($parentAlias, $parentTable->getTableName());

            $queryPart .= ' LEFT JOIN ' . $this->_conn->quoteIdentifier($parentTable->getTableName())
                        . ' ' . $this->_conn->quoteIdentifier($parentTableAlias) . ' ON ';

            //Doctrine_Core::dump($table->getIdentifier());
            foreach ((array) $table->getIdentifier() as $identifier) {
                $column = $table->getColumnName($identifier);

                $queryPart .= $this->_conn->quoteIdentifier($tableAlias)
                            . '.' . $this->_conn->quoteIdentifier($column)
                            . ' = ' . $this->_conn->quoteIdentifier($parentTableAlias)
                            . '.' . $this->_conn->quoteIdentifier($column);
            }
        }

        return $queryPart;
    }

    /**
     * Get count sql query for this Doctrine_Query instance.
     *
     * This method is used in Doctrine_Query::count() for returning an integer
     * for the number of records which will be returned when executed.
     *
     * @return string $q
     */
    public function getCountSqlQuery()
    {
        // triggers dql parsing/processing
        $this->getSqlQuery(array(), false); // this is ugly

        // initialize temporary variables
        $where   = $this->_sqlParts['where'];
        $having  = $this->_sqlParts['having'];
        $groupby = $this->_sqlParts['groupby'];

        $rootAlias = $this->getRootAlias();
        $tableAlias = $this->getSqlTableAlias($rootAlias);

        // Build the query base
        $q = 'SELECT COUNT(*) AS ' . $this->_conn->quoteIdentifier('num_results') . ' FROM ';

        // Build the from clause
        $from = $this->_buildSqlFromPart(true);

        // Build the where clause
        $where = ( ! empty($where)) ? ' WHERE ' . implode(' ', $where) : '';

        // Build the group by clause
        $groupby = ( ! empty($groupby)) ? ' GROUP BY ' . implode(', ', $groupby) : '';

        // Build the having clause
        $having = ( ! empty($having)) ? ' HAVING ' . implode(' AND ', $having) : '';

        // Building the from clause and finishing query
        if (count($this->_queryComponents) == 1 && empty($having)) {
            $q .= $from . $where . $groupby . $having;
        } else {
            // Subselect fields will contain only the pk of root entity
            $ta = $this->_conn->quoteIdentifier($tableAlias);

            $map = $this->getRootDeclaration();
            $idColumnNames = $map['table']->getIdentifierColumnNames();

            $pkFields = $ta . '.' . implode(', ' . $ta . '.', $this->_conn->quoteMultipleIdentifier($idColumnNames));

            // We need to do some magic in select fields if the query contain anything in having clause
            $selectFields = $pkFields;

            if ( ! empty($having)) {
                // For each field defined in select clause
                foreach ($this->_sqlParts['select'] as $field) {
                    // We only include aggregate expressions to count query
                    // This is needed because HAVING clause will use field aliases
                    if (strpos($field, '(') !== false) {
                        $selectFields .= ', ' . $field;
                    }
                }
                // Add having fields that got stripped out of select
                preg_match_all('/`[a-z0-9_]+`\.`[a-z0-9_]+`/i', $having, $matches, PREG_PATTERN_ORDER);
                if (count($matches[0]) > 0) {
                    $selectFields .= ', ' . implode(', ', array_unique($matches[0]));
                }
            }

            // If we do not have a custom group by, apply the default one
            if (empty($groupby)) {
                $groupby = ' GROUP BY ' . $pkFields;
            }

            $q .= '(SELECT ' . $selectFields . ' FROM ' . $from . $where . $groupby . $having . ') '
                . $this->_conn->quoteIdentifier('dctrn_count_query');
        }

        return $q;
    }

    /**
     * Fetches the count of the query.
     *
     * This method executes the main query without all the
     * selected fields, ORDER BY part, LIMIT part and OFFSET part.
     *
     * Example:
     * Main query:
     *      SELECT u.*, p.phonenumber FROM User u
     *          LEFT JOIN u.Phonenumber p
     *          WHERE p.phonenumber = '123 123' LIMIT 10
     *
     * The modified DQL query:
     *      SELECT COUNT(DISTINCT u.id) FROM User u
     *          LEFT JOIN u.Phonenumber p
     *          WHERE p.phonenumber = '123 123'
     *
     * @param array $params        an array of prepared statement parameters
     * @return integer             the count of this query
     */
    public function count($params = array())
    {
        $q = $this->getCountSqlQuery();
        $params = $this->getCountQueryParams($params);
        $params = $this->_conn->convertBooleans($params);

        if ($this->_resultCache) {
            $conn = $this->getConnection(); 
            $cacheDriver = $this->getResultCacheDriver();
            $hash = $this->getResultCacheHash($params).'_count';
            $cached = ($this->_expireResultCache) ? false : $cacheDriver->fetch($hash);

            if ($cached === false) {
                // cache miss
                $results = $this->getConnection()->fetchAll($q, $params);
                $cacheDriver->save($hash, serialize($results), $this->getResultCacheLifeSpan());
            } else {
                $results = unserialize($cached);
            }
        } else {
            $results = $this->getConnection()->fetchAll($q, $params);
        }

        if (count($results) > 1) {
            $count = count($results);
        } else {
            if (isset($results[0])) {
                $results[0] = array_change_key_case($results[0], CASE_LOWER);
                $count = $results[0]['num_results'];
            } else {
                $count = 0;
            }
        }

        return (int) $count;
    }

    /**
     * Queries the database with DQL (Doctrine Query Language).
     *
     * This methods parses a Dql query and builds the query parts.
     *
     * @param string $query      Dql query
     * @param array $params      prepared statement parameters
     * @param int $hydrationMode Doctrine_Core::HYDRATE_ARRAY or Doctrine_Core::HYDRATE_RECORD
     * @see Doctrine_Core::FETCH_* constants
     * @return mixed
     */
    public function query($query, $params = array(), $hydrationMode = null)
    {
        $this->parseDqlQuery($query);
        return $this->execute($params, $hydrationMode);
    }

    /**
     * Copies a Doctrine_Query object.
     *
     * @return Doctrine_Query  Copy of the Doctrine_Query instance.
     */
    public function copy(Doctrine_Query $query = null)
    {
        if ( ! $query) {
            $query = $this;
        }

        $new = clone $query;

        return $new;
    }

    /**
     * Magic method called after cloning process.
     *
     * @return void
     */
    public function __clone()
    {
        $this->_parsers = array();
        $this->_hydrator = clone $this->_hydrator;

        // Subqueries share some information from the parent so it can intermingle
        // with the dql of the main query. So when a subquery is cloned we need to
        // kill those references or it causes problems
        if ($this->isSubquery()) {
            $this->_killReference('_params');
            $this->_killReference('_tableAliasMap');
            $this->_killReference('_queryComponents');
        }
    }

    /**
     * Kill the reference for the passed class property.
     * This method simply copies the value to a temporary variable and then unsets
     * the reference and re-assigns the old value but not by reference
     *
     * @param string $key
     */
    protected function _killReference($key)
    {
        $tmp = $this->$key;
        unset($this->$key);
        $this->$key = $tmp;
    }

    /**
     * Frees the resources used by the query object. It especially breaks a
     * cyclic reference between the query object and it's parsers. This enables
     * PHP's current GC to reclaim the memory.
     * This method can therefore be used to reduce memory usage when creating
     * a lot of query objects during a request.
     *
     * @return Doctrine_Query   this object
     */
    public function free()
    {
        $this->reset();
        $this->_parsers = array();
        $this->_dqlParts = array();
    }
}

I have loved working with Doctrine and would be happy to contribute back to the code base any patches I resolve for my project in the future.

Best regards

Will Ferrer

Comment by Jonathan H. Wage [ 27/May/10 ]

Hi, in order to apply your changes we need a patch and not the whole file. Thanks, Jon

Comment by will ferrer [ 02/Jun/10 ]

Hi Jonathan

I tried using winmerge to make a patch file but it seems contain both the entire before and after files in it. I am not sure if I am doing something wrong with the software or if this is just what a patch file looks like.

At any rate I have attached the patch file to this issue.

Please let me know if this patch file is correct.

Best Regards

Will Ferrer

Comment by Jonathan H. Wage [ 08/Jun/10 ]

Hi, the patch is not correct. Just go into your svn checkout where the changes are and run the command svn diff. Output that to a file and attach it here. Thanks, Jon

Comment by Jonathan H. Wage [ 08/Jun/10 ]

I was able to generate a patch. It had some errors in our test suite but I fixed them. Since we don't have a test case for it I am not sure if the changes I made affected anything for you. Can you test the patch or provide a test case?

Comment by will ferrer [ 08/Jun/10 ]

Hi Jon

I checked out the svn branch 1.2.2 and noticed that you had my original/broken patch in the code (that one failed some tests for me so I fixed it and tried to upload it in the last patch file I attached to this thread – the patch file that didn't work).

Using the technique you described I made a working patch to put the correct version of my code into the 1.2.2 branch.

I also added a test case for this fix in my patch.

Please see the new patch I have attached to this thread.

Thanks for all your help.

Will Ferrer

Comment by will ferrer [ 08/Jun/10 ]

Here is the correct patch for the bug fix, along with the test case for it.

Comment by will ferrer [ 08/Jun/10 ]

I reopened the issue to call attention to the fixed patch/test case I added to the thread.

Comment by Jonathan H. Wage [ 08/Jun/10 ]

That one works and looks better. Just had a few tabs vs spaces problems that I fixed. Thanks for your work on this. It is much appreciated!!!!!

Comment by will ferrer [ 08/Jun/10 ]

Hi Jon

No problem – I am very glad to contribute back to the project . Doctrine has really been invaluable to me in my development, thanks for building it.

Hope you are well.

Will Ferrer





[DC-22] Record instances are shared; local data is overwritten. Created: 15/Sep/09  Updated: 15/Sep/09  Resolved: 15/Sep/09

Status: Resolved
Project: Doctrine 1
Component/s: Record
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Blocker
Reporter: Craig Marvelley Assignee: Roman S. Borschel
Resolution: Fixed Votes: 0
Labels: None
Environment:

LAMP, PHP 5.2.9 & 5.3.0


Attachments: File 2486TestCase.php     File index.php    

 Description   

If I make changes to the properties of an instance of a record but don't persist them (so the record's dirty), then in a separate process hydrate another record instance, the original record instance's properties are altered.

I'm not aware of all the instances in which this happens, but it definitely occurs when finding records through a DQL query or loading related records.

This is particularly troublesome since we're experiencing the loading of one record affecting another instance thousands of lines of code away. It seems like the records are shared somehow. This occurs without any caching of any kind.

I'd really appreciate someone taking a look at this. Example and test attached.



 Comments   
Comment by Roman S. Borschel [ 15/Sep/09 ]

This is unfortunately the default behavior of Doctrine 1.x. Because of backwards compatibility this default behavior can not be easily changed. However, some time ago a new attribute was introduced: Doctrine::ATTR_HYDRATE_OVERWRITE. You can set this to FALSE which does what you want I think. Doctrine 2 already behaves this way by default.

Comment by Craig Marvelley [ 15/Sep/09 ]

Ah, fab. Sounds like what I need! Missed it in the docs, sorry. Is there a list somewhere of all the settings and what they do? That would be handy if not, since they seem to be dotted around the manual at the moment so they're easy to miss. Are there any other side effects I should be expecting if I turn that setting on? Thanks!

Comment by Roman S. Borschel [ 15/Sep/09 ]

I dont think any such list exists. As you said all of the attributes are spread throughout the documentation. You can create an issue as an enhancement request for the documentation for that.

Setting that attribute to false should have no other side effects. It really only affects hydration and controls whether the values from the database or the local values should be considered "newer" and therefore kept around.

I will close this issue. If the attribute does not solve your issue, please let us know.





[DC-24] "AS" clause in DQL crashes the further update of the row in one-many relationship Created: 15/Sep/09  Updated: 19/Sep/09  Resolved: 17/Sep/09

Status: Resolved
Project: Doctrine 1
Component/s: Query
Affects Version/s: 1.1.4
Fix Version/s: 1.2.0-ALPHA1

Type: Bug Priority: Blocker
Reporter: Jacek Jędrzejewski Assignee: Jonathan H. Wage
Resolution: Won't Fix Votes: 1
Labels: None

Attachments: File 9999bTestCase.php     File 9999cTestCase.php     File 9999TestCase.php     File select_alias.diff    

 Description   

moving from trac, issue #2410 by zyxist

This ticket is related to the MySQL database and InnoDB engine with exporting all the information about the models (including foreign keys).

I have two tables connected with one-to-many relationship. First, I select a row from a "master" table with attached a field from the related "servant" row:

$master = Doctrine_Query::create()
	->select('m.*, s.bar AS joe')
	->from('Ticket_9999_Master m')
	->innerJoin('m.Ticket_9999_Servant s')
	->where('m.id = 1')
	->fetchOne();

Next, I modify one of the columns in the master row and save everything:

$master->foo = 5;
$master->save();

Expected result: the master row is updated.

Actual result: MySQL error:

Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails (`doctrine`.`ticket_9999_master`, CONSTRAINT `ticket_9999_master_servant_id_ticket_9999_servant_id` FOREIGN KEY (`servant_id`) REFERENCES `ticket_9999_servant` (`id`))

If we remove "AS joe" part from the query, it gives the correct result. I noticed that with this alias, Doctrine forgets to retrieve the primary key of the servant row and this produces an invalid UPDATE query. There are two possible ways to retrieve this:

1. Use the data from the "Master" row, as we fetch everything, including the ID of the servant row. Doctrine knows about the relationship and potentially can use it.

2. Simply select the servant id automatically with the "s.bar" column.

Tested on software:
PHP 5.2.10, PHP 5.3.0
MySQL 5.1.36
Doctrine 1.1-DEV (rev. 6217)



 Comments   
Comment by Jacek Jędrzejewski [ 15/Sep/09 ]

Two words from me:

Well, the problem is not the saving process but the thing that selecting a relation only with aliased field does not retrieve the ID. I mean that this query:

SELECT m.*, s.bar FROM Master m, m.Servant S

is also selecting s.id field, because Doctrine always needs it, right? Adding an alias "s.bar AS something" makes the query not select "s.id", so in save() Doctrine thinks Servant is a new record. And that's why mysql throws "Integrity constraint violation".

Comment by Ionut E [ 17/Sep/09 ]

Patch for the 1.1 branch. Fixes the alias problem but messes a lot of other tests which are query related.

My fix forces all fields of the table with the column alias to be inserted in sql as well.

Comment by Jonathan H. Wage [ 17/Sep/09 ]

You just need to select the id.

$master = Doctrine_Query::create()
	->select('m.*, s.id, s.bar AS joe')
	->from('Ticket_9999_Master m')
	->innerJoin('m.Ticket_9999_Servant s')
	->where('m.id = 1')
	->fetchOne();

If you have a valid suggestion for how to fix this or make it "smarter" we're open to suggestions.

Comment by Jacek Jędrzejewski [ 17/Sep/09 ]

lol, I know that selecting ID solves this issue. But it is illogical, because a query on field without alias select ID automatically while aliased - no.

Comment by Jonathan H. Wage [ 17/Sep/09 ]

It doesn't automatically select the id unless you also select some real property. In your case you selected and aliased so the object has no real properties loaded.

Comment by Roman S. Borschel [ 17/Sep/09 ]

It actually is logical but I absolutely understand why it looks illogical to you.

Doctrine 1.x auto-adds the primary key only if it is needed. It is needed only if the hydration mode is record/array and any non-scalar value is selected. If neither the hydration mode requires the primary key, nor the selected result expressions contains object fields (not object fields selected as scalar values, i.e. what AS .. does) then it is not added.

"select x.foo as something" only selects a scalar value. The fact that makes this so confusing is that scalar values are put into Record objects in Doctrine 1.x, so to you it looks like a full object was hydrated when in fact it was just a scalar value.

in Doctrine 1.x this is unfortunate but expected behavior. In Doctrine 2 this confusion will not arise as there is a clear separation and scalar values are never put into objects.

Comment by Ionut E [ 18/Sep/09 ]

Jacek, could you please show me an result of the select from the test? On my computer foreign key gets deleted and I have a really hard time beliving this is logical and expected behaviour.

Comment by Jacek Jędrzejewski [ 18/Sep/09 ]

Roman S. Borschel - so if it is not a "real" record but only a "plain" record with a scalar value, why doctrine tries to use it for save? I'm not sure about the internals of this process but something must be wrong there

lonut: this is from toArray(true)

Array
(
    [id] => 1
    [foo] => 6
    [servant_id] =>
    [Ticket_9999_Servant] => Array
        (
            [id] =>
            [bar] =>
            [joe] => 6
        )

    [joe] => 6
)

OKOK, so now try new testcase (attachment 9999b):
I have checked it and this issue is not only related to MySQL and InnoDB. This testcase fails with this error:
Doctrine_Ticket_9999b_TestCase : method testTest failed on line 72 SQLSTATE[23000]: Integrity constraint violation: 19 ticket_9999b__master.servant_id may not be NULL

So now let's try this without notnull constraint on "servant_id" (testcase 9999c):
It works ok, no error on saving. But now let's retrieve that record from DB:

$master2 = Doctrine_Query::create()->select('m.*')->from('Ticket_9999c_Master m')->where('m.id = 1')->fetchOne(array(), Doctrine::HYDRATE_ARRAY);

"servant_id" is empty! Please don't tell me it is a correct behavior!

Comment by Ionut E [ 18/Sep/09 ]

Caching seems to be badly implemented. See DC-41

Comment by Roman S. Borschel [ 18/Sep/09 ]

Jacek,

because it is broken by design. Whoever made the decision to put scalar values into record objects (I think it was Konsta) did not think that through. Doctrine itself does not make a distinction internally between "real" records and "records with only scalars". Hence why it pukes on save(). I just dont see a good way to fix this without significant work and breaking existing code. If you have an idea I am all ears. If we start to add the primary key even if only scalar values are selected of a class we start to make all kinds of queries impossible that compute aggregate values.

Ionut,

this has nothing to do with DC-41, as far as I can see.

Comment by Roman S. Borschel [ 18/Sep/09 ]

Of course, if anyone has a decent idea on how to fix this inconsistency, ideally without breaking backwards-compatibility, that would be great.

Comment by Jonathan H. Wage [ 18/Sep/09 ]

In 1.2 I fixed this by removing the hydration of aggregate values in to the relationship record. This was something that was built by design in 1.0, we realized it was not good so in 1.1 I made it so the aggregate values ALSO went in to the root component, and I deprecated the use of the value from the relationship. So now, in 1.2 we can remove it. I committed your test case and it passes now with this changeset.

Thanks, Jon

Comment by Jacek Jędrzejewski [ 19/Sep/09 ]

Thanks Jon! Hell, I want doctrine 1.2 and symfony 1.3!

BTW. it can be now marked as "fixed" xd





[DC-16] postHydrate listeners should be called after relationships are loaded. Created: 15/Sep/09  Updated: 14/Apr/10  Resolved: 17/Sep/09

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: 1.2.0-ALPHA1, 1.2.2
Fix Version/s: None

Type: Improvement Priority: Blocker
Reporter: Roman S. Borschel Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None

Attachments: File Doctrine-real-postHydrate(2).diff     File Doctrine-real-postHydrate.diff    

 Description   

Doctrine 1.1.3 calls post hydration listeners before relationships are loaded.

This O patch caches the information needed to call the listeners and calls the listeners in reverse order after all the data has been loaded. The net effect is that postHydrate listeners are called for all an object's children before being called for a parent object.

Memory cost should be minor (O @ two pointers per record), and the (additional) processing cost should be flat and almost zero (n function calls to postHydrate have simply been moved outside the loop to a second loop). There is an additional call to $table->getComponentName() in the second loop that could be eliminated by storing $componentName with the other event initializers in the new stack).



 Comments   
Comment by Jonathan H. Wage [ 17/Sep/09 ]

This patch no longer applies as the hydration code is re factored a lot in 1.2

Comment by Exception e [ 14/Apr/10 ]

It seems that even after the rewrite postHydrate() doesn't work correctly.

class Member extends BaseMember 
       private $roles = array();

	function postHydrate($event)	{
		if ($this->relatedExists('Artist'))	{
			array_push($this->roles, 'artist');
		}
		
	}
}

throws exception «Unknown record property / related component "member_id" on "Member"», even if I have joined the related object Artist. member_id is the PK of Member.

Trying isset in

function postHydrate($event) {
if (isset($this->Artist)))

{ array_push($this->roles, 'artist'); }

}

will consistently return false.

This is a blocker since there is no workaround. postHydrate is the last event on record loading afaik.





[DC-933] Results from Doctrine_Query::execute inconsistent with results from Doctrine_Query::getSqlQuery() Created: 19/Nov/10  Updated: 28/Feb/11  Resolved: 28/Feb/11

Status: Resolved
Project: Doctrine 1
Component/s: Query
Affects Version/s: 1.2.3
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Roger Webb Assignee: Guilherme Blanco
Resolution: Invalid Votes: 0
Labels: None
Environment:

Redhat Linux, Apache, PHP 5,



 Description   

The DQL Query:

$query = Doctrine_Query::create()
->select("
b.borrowers_date as borrowers_date,
b.borrower_id as borrower_id,
borrower_contact.first_name as borrower_first_name,
borrower_contact.last_name as borrower_last_name,
lo_contact.first_name as lo_first_name,
lo_contact.last_name as lo_last_name,
realtor_contact.first_name as realtor_first_name,
realtor_contact.last_name as realtor_last_name,
lo_company_contact.first_name as lender,
b.current_status as current_status
")
->from("Borrowers b")
>innerJoin("b.UserBorrowerAssigned ubass WITH ubass.user_id = " . $this>user_id)
->innerJoin("b.ContactInfo borrower_contact")
->innerJoin("b.LoBorrowerAssigned lobass")
->innerJoin("lobass.LoanOfficers lo")
->innerJoin("lo.ContactInfo lo_contact")
->innerJoin("lo.Companies lo_company")
->innerJoin("lo_company.ContactInfo lo_company_contact")
->leftJoin("b.RealtorBorrowerAssigned realbass")
->leftJoin("realbass.Realtors realtor")
->leftJoin("realtor.ContactInfo realtor_contact");
...
$query->where("b.current_status != 'finialized'")
->andWhere("b.current_status != 'ignored'")
->andWhere("b.current_status != 'dead'");

$query->execute Returns only 1 row

--------------------------------------------------------------------------------

$query->getSqlQuery() returns:

SELECT b.borrowers_date AS b_0, b.borrower_id AS b1, c.first_name AS c2, c.last_name AS c3, c2.first_name AS c24, c2.last_name AS c25, c5.first_name AS c56, c5.last_name AS c57, c4.first_name AS c48, b.current_status AS b_9 FROM borrowers b INNER JOIN user_borrower_assigned u ON b.borrower_id = u.borrower_id AND (u.user_id = 129) INNER JOIN contact_info c ON b.contact_info_id = c.contact_info_id INNER JOIN lo_borrower_assigned l ON b.borrower_id = l.borrower_id INNER JOIN loan_officers l2 ON l.loan_officer_id = l2.loan_officer_id INNER JOIN contact_info c2 ON l2.contact_info_id = c2.contact_info_id INNER JOIN companies c3 ON l2.company_id = c3.company_id INNER JOIN contact_info c4 ON c3.contact_info_id = c4.contact_info_id INNER JOIN realtor_borrower_assigned r ON b.borrower_id = r.borrower_id INNER JOIN realtors r2 ON r.realtor_id = r2.realtor_id INNER JOIN contact_info c5 ON r2.contact_info_id = c5.contact_info_id WHERE (b.current_status != 'finialized' AND b.current_status != 'ignored' AND b.current_status != 'dead')

Running this query in PhpMyAdmin returns 1,095 rows.

-----------------------------------------------------------------------------------

Results Inconsistent



 Comments   
Comment by Roger Webb [ 19/Nov/10 ]

I have stripped the query down to this:

->select("
b.borrowers_date as borrowers_date,
b.borrower_id as borrower_id,
borrower_contact.first_name as borrower_first_name,
borrower_contact.last_name as borrower_last_name,
b.current_status as current_status
")
->from("Borrowers b")
->innerJoin("b.ContactInfo borrower_contact")
->innerJoin("b.UserBorrowerAssigned ubass");

$query->where("b.current_status != 'finialized'")
->andWhere("b.current_status != 'ignored'")
->andWhere("b.current_status != 'dead'")
>andWhere("ubass.user_id = ?", $this>user_id);

Results are still consistent with but report above.

Classes:

class Borrowers extends BaseBorrowers
{

function setUp()

{ parent::setUp(); $this->hasOne('ContactInfo', array('local' => 'contact_info_id', 'foreign' => 'contact_info_id')); $this->hasOne('LoBorrowerAssigned', array('local' => 'borrower_id', 'foreign' => 'borrower_id')); $this->hasOne('RealtorBorrowerAssigned', array('local' => 'borrower_id', 'foreign' => 'borrower_id')); $this->hasOne('UserBorrowerAssigned', array('local' => 'borrower_id', 'foreign' => 'borrower_id')); $this->unshiftFilter(new Doctrine_Record_Filter_Compound(array('ContactInfo'))); }

}

class UserBorrowerAssigned extends BaseUserBorrowerAssigned {

function setUp()

{ parent::setUp(); $this->hasOne("Borrowers", array("local" => "borrower_id", "foreign" => "borrower_id")); $this->hasOne("Users", array("local" => "user_id", "foreign" => "user_id")); }

}

class Users extends BaseUsers {

function setUp()

{ parent::setUp(); $this->hasOne("ContactInfo", array("local" => "contact_info_id", "foreign" => "contact_info_id")); $this->unshiftFilter(new Doctrine_Record_Filter_Compound(array('ContactInfo'))); }

}

The users class also contains the function that issues the query in question.

Comment by Roger Webb [ 28/Feb/11 ]

Non issue. Wasn't familiar with use of Doctrine.





[DC-895] [I18n] Defining languages with locality (eg. en_GB) breaks functionality with SQL Integrity error - fix included Created: 20/Oct/10  Updated: 20/Oct/10  Resolved: 20/Oct/10

Status: Resolved
Project: Doctrine 1
Component/s: I18n
Affects Version/s: 1.2.0, 1.2.1, 1.2.2, 1.2.3
Fix Version/s: 1.2.0, 1.2.1, 1.2.2, 1.2.3

Type: Bug Priority: Critical
Reporter: Erik Van Kelst Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

all



 Description   

When defining languages as language_COUNTRY codes (supported by symfony by default), the functionality to work with I18n records breaks, resulting in "SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry" errors.

The reason is very simple: Doctrine's I18n language column is defined as a CHAR(2), thus shortening eg. "en_GB" value to "en", thus causing the above SQL error when a "en" translation for a record already exists.

The solution is even simpler: change the column's length to 7 in the Doctrine_I18n class's options: I've tested this and all runs great: the correct SQL is being generated, the models behave correct, ...



 Comments   
Comment by Erik Van Kelst [ 20/Oct/10 ]

Length of the i18n column is configurable...





[DC-444] Reimporting exported data-fixtures fails if they contain <? or <?php Created: 23/Jan/10  Updated: 11/May/10  Resolved: 01/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Data Fixtures, File Parser
Affects Version/s: 1.0.14, 1.1.4, 1.2.1
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Benjamin Steininger Assignee: Jonathan H. Wage
Resolution: Won't Fix Votes: 0
Labels: None


 Description   

In Doctrine_Parser::doLoad() the fixtures are included which will try to execute the file with the php-interpreter resulting in all unescapted code which starts with <? (if shorttags are on) or <?php being executed (as long as it is in one line and ends with ?>).
As long as the php-code is multi-lined the break after <? or <?php gets converted to \r\n which will create a parse-error and make it impossible to import the fixture:

build-all-reload - Are you sure you wish to drop your databases? (y/n)
y
build-all-reload - Successfully dropped database for connection named 'development'
build-all-reload - Generated models successfully from YAML schema
build-all-reload - Successfully created database for connection named 'development'
build-all-reload - Created tables successfully


Warning: Unexpected character in input:  '\' (ASCII=92) state=1 in /..../doctrine/data/fixtures/data.yml on line 9724

Call Stack:
    0.0003      72344   1. {main}() /.../scripts/doctrine-cli:0
    0.7356   14656192   2. Doctrine_Cli->run() /.../scripts/doctrine-cli:25
    0.7356   14656256   3. Doctrine_Cli->_run() /.../library/doctrine/1.2.1/lib/Doctrine/Cli.php:452
    0.7367   14666244   4. Doctrine_Cli->executeTask() /.../library/doctrine/1.2.1/lib/Doctrine/Cli.php:498
    0.7367   14666388   5. Doctrine_Task_BuildAllReload->execute() /.../library/doctrine/1.2.1/lib/Doctrine/Cli.php:516
    4.2097   22318416   6. Doctrine_Task_LoadData->execute() /.../library/doctrine/1.2.1/lib/Doctrine/Task/BuildAllReload.php:56
    4.2176   22318488   7. Doctrine_Core::loadData() /.../library/doctrine/1.2.1/lib/Doctrine/Task/LoadData.php:43
    4.2185   22357660   8. Doctrine_Data->importData() /.../library/doctrine/1.2.1/lib/Doctrine/Core.php:996
    4.2202   22472372   9. Doctrine_Data_Import->doImport() /.../library/doctrine/1.2.1/lib/Doctrine/Data.php:222
    4.2202   22472440  10. Doctrine_Data_Import->doParsing() /.../library/doctrine/1.2.1/lib/Doctrine/Data/Import.php:112
    4.2204   22488760  11. Doctrine_Parser::load() /.../library/doctrine/1.2.1/lib/Doctrine/Data/Import.php:95
    4.2204   22489152  12. Doctrine_Parser_Yml->loadData() /.../library/doctrine/1.2.1/lib/Doctrine/Parser.php:89
    4.2204   22489216  13. Doctrine_Parser->doLoad() /.../library/doctrine/1.2.1/lib/Doctrine/Parser/Yml.php:78

Parse error: syntax error, unexpected T_STRING in /.../data/fixtures/data.yml on line 9724

Call Stack:
    0.0003      72344   1. {main}() /.../scripts/doctrine-cli:0
    0.7356   14656192   2. Doctrine_Cli->run() /.../scripts/doctrine-cli:25
    0.7356   14656256   3. Doctrine_Cli->_run() /.../library/doctrine/1.2.1/lib/Doctrine/Cli.php:452
    0.7367   14666244   4. Doctrine_Cli->executeTask() /.../library/doctrine/1.2.1/lib/Doctrine/Cli.php:498
    0.7367   14666388   5. Doctrine_Task_BuildAllReload->execute() /.../library/doctrine/1.2.1/lib/Doctrine/Cli.php:516
    4.2097   22318416   6. Doctrine_Task_LoadData->execute() /.../library/doctrine/1.2.1/lib/Doctrine/Task/BuildAllReload.php:56
    4.2176   22318488   7. Doctrine_Core::loadData() /.../library/doctrine/1.2.1/lib/Doctrine/Task/LoadData.php:43
    4.2185   22357660   8. Doctrine_Data->importData() /.../library/doctrine/1.2.1/lib/Doctrine/Core.php:996
    4.2202   22472372   9. Doctrine_Data_Import->doImport() /.../library/doctrine/1.2.1/lib/Doctrine/Data.php:222
    4.2202   22472440  10. Doctrine_Data_Import->doParsing() /.../library/doctrine/1.2.1/lib/Doctrine/Data/Import.php:112
    4.2204   22488760  11. Doctrine_Parser::load() /.../library/doctrine/1.2.1/lib/Doctrine/Data/Import.php:95
    4.2204   22489152  12. Doctrine_Parser_Yml->loadData() /.../library/doctrine/1.2.1/lib/Doctrine/Parser.php:89
    4.2204   22489216  13. Doctrine_Parser->doLoad() /.../library/doctrine/1.2.1/lib/Doctrine/Parser/Yml.php:78

That can happen with unescapted <?xml too if short-tags are active and result in parse-error.

A way to load fixtures without running it through the php-interpreter would be nice or the <? needs to be escaped on export and unescaped on import.

Checked with version 1.2.1, 1.1.4 and 1.0.14, all versions share this code.

This "bug" can probably get a bit ugly in the raw case that a persons is using that dump-feature with something like

<?php system('rm -rf /'); ?>

anywhere in the database.
As long as the code is in one line there aren't any \r\n added and it will get executed without.



 Comments   
Comment by Jonathan H. Wage [ 01/Mar/10 ]

I think this is somewhat expected. As I have stressed in the past. Dumping data fixtures from the database can never work 100%. The idea is that you dump to get started, then modify the dumped fixtures so that they will re-import properly. If you have a suggested fix/patch please feel free to share and re-open.

Comment by Jannis Mosshammer [ 11/May/10 ]

I modified the doLoad() function to not use include but file_get_contents and the import of some critical fields (which contained xml data as well as raw php data) worked fine.

But honestly, I've got the feeling that I haven't really understood why the yaml load is done via including the file and saving the output buffer - so maybe I'm missing the big picture.
My altered version of the doLoad function looks like this:

Unable to find source-code formatter for language: php. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
abstract class Doctrine_Parser
{
    ...
    /**
     * doLoad
     *
     * Get contents whether it is the path to a file file or a string of txt.
     * Either should allow php code in it.
     *
     * @param string $path 
     * @return void
     */
    public function doLoad($path)
    {
        //ob_start();
        if ( ! file_exists($path)) {
            $contents = $path;
            $path = sys_get_temp_dir() . DIRECTORY_SEPARATOR . 'dparser_' . microtime();

            file_put_contents($path, $contents);
        }

	    $contents = file_get_contents($path);
        /*include($path);
        // Fix #1569. Need to check if it's still all valid
        $contents = ob_get_clean(); //iconv("UTF-8", "UTF-8", ob_get_clean());
		*/
        return $contents;
    }
    ...
}





[DC-433] UnitOfWork saveRelatedLocalKeys() creating erroneous child/parent rows Created: 15/Jan/10  Updated: 15/Mar/10  Resolved: 15/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Record, Relations
Affects Version/s: 1.1.6
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Justin Mazzi Assignee: Jonathan H. Wage
Resolution: Incomplete Votes: 0
Labels: None

Attachments: Text File unitofwork.patch    

 Description   

This issue is a lot like the bug with refreshRelated() in ticket #DC-93. Calling save() on a record will cause hasOne rows to be generated in error (with all NULL values). It looks the saveRelatedLocalKeys function checks for foreign keys, calls isModified, which returns true, and calls save on that related object. isModified is returning true for a row that doesn't exist.

schema.yml
 
---
Staff:
  columns:
    username: string(50)
    password: string(255)
    active:
      type: boolean
      default: true
  indexes:
    user_pass_active_idx:
      fields: [username, password, active]


Tickets:
  columns:
    mask:
      unique: true
      type: string(10)
    staff_id: integer(9)
  relations:
    Staff:
      local: staff_id
      foreign: id
test.php
 
<?php
$ticket = new Tickets();
$ticket->staff = null;
// This save generates the NULL record in the staff table
$ticket->save();


 Comments   
Comment by Jonathan H. Wage [ 01/Mar/10 ]

Hi, this patch is not valid. We can't only save records that already exist. That means it stops all new records from being saved. If you want to help fix this, first we'll need a reproducible test case. Thanks, Jon





[DC-367] generate-models-db doesn't fetch cascade delete constraint Created: 18/Dec/09  Updated: 01/Mar/10  Resolved: 01/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Cli
Affects Version/s: 1.2.0
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Francesco Montefoschi Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

Windows / PostgreSQL



 Description   

Running
doctrine-cli.php generate-models-db

everything works fine, I get in setUp() all the relations, by the way no one of them has the 'cascade' => array('delete') option, even if I setup all the constraints correctly.



 Comments   
Comment by Benjamin Eberlei [ 27/Jan/10 ]

however, this information cant be detected on mysql 5.0 versions and came in late in other versions.

Comment by Francesco Montefoschi [ 28/Jan/10 ]

Why can't this be detected on mysql 5?

Comment by Jonathan H. Wage [ 01/Mar/10 ]

I just tested this with the most recent mysql 5 version and it works as expected.

Comment by Francesco Montefoschi [ 01/Mar/10 ]

By the way, I reported this bug against Postgres





[DC-352] Doctrine_RawSql/Caching fatal error Created: 10/Dec/09  Updated: 01/Mar/10  Resolved: 01/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Caching, Native SQL
Affects Version/s: 1.2.0, 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Critical
Reporter: Jonathan Shelly Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 1
Labels: None
Environment:

Solaris



 Description   

Doctrine_RawSql::calculateResultCacheHash tries to call $this->getSql() - an undefined method - on line 444 of RawSql.php.



 Comments   
Comment by Witold Wasiczko [ 24/Jan/10 ]

I get the same error during:
$q = new Doctrine_RawSql();
$q->useResultCache(true);

I fixed it, by change from:
$sql = $this->getSql();
to:
$sql = $this->getDql();

In Doctrine_RawSql 444 line.





[DC-324] Quote identifier breaks multi-column index sql generation Created: 03/Dec/09  Updated: 01/Mar/10  Resolved: 01/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Connection
Affects Version/s: 1.2.0
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Juozas Kaziukenas Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None

Attachments: Text File connection2.patch    

 Description   

By enabling quote identifier:

Doctrine_Manager::getInstance()->setAttribute(
Doctrine::ATTR_QUOTE_IDENTIFIER,
true
);

If table has multi-column index, results are:

CREATE INDEX name ON table ("column1, column2")

expected result

CREATE INDEX name ON table ("column1", "column2")

I created a patch for this bug, which extends quoteIdentifer() method in connection to detect comma separated fields and escape them separately. It also supports any amount of fields (not just two). It's done in this method, and not in getIndexFieldDeclarationList() in export classes, because it would require a major refactoring for a lot of classes and methods inside different classes. Please review this issue, so I can commit.



 Comments   
Comment by Jonathan H. Wage [ 07/Dec/09 ]

I don't think it should be done this way. It should be done specifically where it is needed, in getIndexFieldDeclarationList()

Comment by Jonathan H. Wage [ 07/Dec/09 ]

Hmm. After looking at this, it doesn't seem right. The method getIndexFieldDeclarationList() is already implemented this way and each field is quoted individually. Let me see your schema where you define the index..

Comment by Juozas Kaziukenas [ 07/Dec/09 ]

Ok, it might have been not right, but I made it like this, because this data was tunelling from definition to getIndexFieldDeclarationList() and at that point fields are already as list (comma separated), so I guess my fix worked more like a hack.

Model looks something like this:

<?php

abstract class Model extends Doctrine_Record
{
public function setTableDefinition()
{
$this->setTableName('table');
$this->hasColumn('userid', 'integer', 4, array(
'type' => 'integer',
'length' => 4,
'fixed' => false,
'unsigned' => false,
'notnull' => true,
'primary' => false,
));
$this->hasColumn('moduleid', 'integer', 4, array(
'type' => 'integer',
'length' => 4,
'fixed' => false,
'unsigned' => false,
'notnull' => true,
'primary' => false,
));
$this->hasColumn('privilegeid', 'string', null, array(
'type' => 'string',
'fixed' => true,
'unsigned' => false,
'notnull' => true,
'primary' => false,
));
}

public function setUp()
{
parent::setUp();

$this->hasOne('Model_Privileges as Privileges', array(
'local' => 'privilegeid, moduleid',
'foreign' => 'id, moduleid'));
}
}

It is generated by doctrine task generate-models-db, so it is a generator issue then?

Comment by Jonathan H. Wage [ 07/Dec/09 ]

What database type?

$this->hasOne('Model_Privileges as Privileges', array(
'local' => 'privilegeid, moduleid',
'foreign' => 'id, moduleid'));
}

This part is wrong. the comma separated list of fields is not supported and Doctrine doesn't support composite foreign keys. So it is a problem with the importer for the database you're using. Is it mssql?

Comment by Juozas Kaziukenas [ 07/Dec/09 ]

I was not aware that comma separated fields were invalid. My patch is invalid them and I need to fix generator.

I'm working with pgsql, and will review the problem, propose a pach.

Comment by Jonathan H. Wage [ 07/Dec/09 ]

If composite foreign keys were fully supported in Doctrine 1 it would be with an array:

$this->hasOne('Model_Privileges as Privileges', array(
'local' => array('privilegeid', 'moduleid'),
'foreign' => array('id', 'moduleid')));
}
Comment by Jonathan H. Wage [ 07/Dec/09 ]

If you define it that way, then everything is generated fine.





[DC-162] Doctrine_Migration_Builder fail to buildChangeColumn() Created: 30/Oct/09  Updated: 02/Nov/09  Resolved: 02/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: Migrations
Affects Version/s: 1.2.0-ALPHA3
Fix Version/s: 1.2.0-BETA1

Type: Bug Priority: Critical
Reporter: thibault duplessis Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None
Environment:

Symfony 1.3 daily svn



 Description   

Doctrine_Migration_Builder->buildChangeColumn contains a simple error.

return "        \$this->changeColumn('" . $tableName . "', '" . $columnName. "', '" . $length . "', '" . $type . "', " . $this->varExport($column) . ");";

When I look Doctrine_Migration_Base->changeColumn declaration, i see

public function changeColumn($tableName, $columnName, $type = null, $length = null, array $options = array())

The $length and $type parameters are inverted. It makes all column changes fail.

Thank you for your work, Doctrine is great !



 Comments   
Comment by Jonathan H. Wage [ 02/Nov/09 ]

This has already been fixed in another issue/commit.





[DC-160] Search doesn't use the Search Analyzer to escape the query Created: 30/Oct/09  Updated: 13/Jul/12  Resolved: 02/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: Searchable
Affects Version/s: 1.0.12, 1.1.4, 1.2.0-ALPHA1, 1.2.0-ALPHA2, 1.2.0-ALPHA3
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Markus Lanthaler Assignee: Jonathan H. Wage
Resolution: Can't Fix Votes: 0
Labels: None


 Description   

When you use the Doctrine_Search_Analyzer_Standard all special characters like "ü" are removed or converted to e.g. "ue". So far so good.. The problem arises when a user performs a search.
Since Doctrine is using the plain query with the special char "ü" instead of converting it also there no results are returned.

Using the UTF8 analyzer is no option because often the normalization is a desired feature. It allows for example a user to formulate the query either as "Muenchen" or "München" and he still receives a relevant result.



 Comments   
Comment by Jonathan H. Wage [ 30/Oct/09 ]

The only option I can see is to do this in Doctrine_Search_Query::query()

$text = Doctrine_Inflector::unaccent($text);

I am not sure about doing this though. What do you think?

Comment by Markus Lanthaler [ 30/Oct/09 ]

I don't think that that's a good idea since it breaks the UTF8 analyzer. I would refactor the analyzers to include a method like normalize(). Those methods could then be called in the analyzers analyze() method.

Doctrine_Search_Analyzer_Standard::normalize($text, $encoding = $null) would look as follows:

public function normalize($text, $encoding = null) 
{ 
  $text = preg_replace('/[\'`�"]/', '', $text); 
  $text = Doctrine_Inflector::unaccent($text); 
  $text = preg_replace('/[^A-Za-z0-9]/', ' ', $text); 
  $text = str_replace('  ', ' ', $text); 

  return strtolower(trim($text));
}

Doctrine_Search_Analyzer_Utf8::normalize($text, $encoding = $null) would look as follows:

public function normalize($text, $encoding = null) 
{ 
  if (is_null($encoding)) { 
    $encoding = isset($this->_options['encoding']) ? $this->_options['encoding']:'utf-8'; 
  } 

  // check that $text encoding is utf-8, if not convert it 
  if (strcasecmp($encoding, 'utf-8') != 0 && strcasecmp($encoding, 'utf8') != 0) { 
    $text = iconv($encoding, 'UTF-8', $text); 
  } 

  $text = preg_replace('/[^\p{L}\p{N}]+/u', ' ', $text); 
  $text = str_replace('  ', ' ', $text); 

  return mb_strtolower(trim($text), 'UTF-8');
}

This would then also allow to remove some code duplication in the analyze() method. It could be changed to the following code in Doctrine_Search_Analyzer_Standard and could be completely removed in Doctrine_Search_Analyzer_Utf8:

public function analyze($text, $encoding = null)
{
  $text = $this->normalize($text, $encoding);

  $terms = explode(' ', $text);

  $ret = array();
  if ( ! empty($terms)) {
      foreach ($terms as $i => $term) {
          if (empty($term)) {
              continue;
          }

          if (in_array($lower, self::$_stopwords)) {
              continue;
          }

          $ret[$i] = $lower;
      }
  }
  return $ret;
}

Finally the normalize() method is called in Doctrine_Search_Query::query(). Unfortunately I have no idea how to call it there!?
What do you think?

Comment by Jonathan H. Wage [ 02/Nov/09 ]

At first this seems like a good solution but I realized it will break things even more. We allow wildcards and certain keywords in the query string. *, OR, AND, etc. If we were to run the normalize() method on the query text it would break all that functionality.

Comment by Markus Lanthaler [ 02/Nov/09 ]

Well.. that's nowhere documented.. the only thing I found was

Doctrine_Search provides a query language similar to Apache Lucene. The Doctrine_Search_Query converts human readable, easy-to-construct search queries to their complex DQL equivalents which are then converted to SQL like normal.

So I would rather break those special things than to have the search missing existing items. But maybe there's a better place to call that normalize() - perhaps where the query is analyzed and converted to a DQL statement. It should be possible there to run normalize on every search term.

Comment by Jonathan H. Wage [ 02/Nov/09 ]

That's what this means:

Doctrine_Search provides a query language similar to Apache Lucene

You can do things like

$query->query('some text* OR some more test*');

If we normalized each term/word it will still remove those wildcards. That is what query language similar to Apache lucene means.

Comment by João Veríssimo [ 13/Jul/12 ]

What do you think about this solution?
class ErpSearchAnalizer extends Doctrine_Search_Analyzer_Standard {
public function analyze($text, $encoding = null) {
$text = preg_replace('/[\'`�"]/', '', $text);
$text = Doctrine_Inflector::unaccent($text);
// for * search
//$text = preg_replace('/[^A-Za-z0-9]/', ' ', $text);
$text = str_replace(' ', ' ', $text);
$terms = explode(' ', $text);

$ret = array();
if (!empty($terms)) {
foreach ($terms as $i => $term) {
if (empty($term))

{ continue; }
if($term == 'OR'){ $ret[$i] = $term; continue; }
$lower = strtolower(trim($term));

if (in_array($lower, parent::$_stopwords)) { continue; }

$ret[$i] = $lower;
}
}
return $ret;
}
}
to make a search like this
$analize = new ErpSearchAnalizer();
$val_user = $analizar->analyze($val_user);
$tempresult = $table->search(implode(" ", $val_user));

will it work?





[DC-693] Last_insert_id is not save in properties "id" after save() (using sqlite) Created: 20/May/10  Updated: 08/Jun/10  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: Connection
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Valery Sizov Assignee: Jonathan H. Wage
Resolution: Can't Fix Votes: 1
Labels: None
Environment:

php- 5.2.10, mysql 5.1,sqlite 3.0



 Description   

using mysql
--------------------
$user = new User;
$user->name = 'Job';
$user->save();

echo $user->id; //1 (for example);

it`s ok!
-------------------------------------------------
using sqlite
-//-
echo $user->id; //0 (always and any where );



 Comments   
Comment by Dennis Gearon [ 28/May/10 ]

I have same problem on a postgresql table, 1 out of 10 in the schema. The table has 6 primary foreign keys and a single autoincrementing primary integer key, 'id', column. 'id' is unavailable after a save(). I'm getting around this with a search for the inserted record, but it's a slow PITA.

Comment by Dennis Gearon [ 28/May/10 ]

sorry, ubuntu 9.10, postgres-8.4, symfoy 1.4.4

Comment by Jonathan H. Wage [ 08/Jun/10 ]

I think this is an unfortunate problem that cannot be fixed. You cannot have a auto increment primary key with some other composite primary keys.





[DC-671] Record could be saved, even tough a NotNull column is null Created: 09/May/10  Updated: 10/May/10  Resolved: 09/May/10

Status: Resolved
Project: Doctrine 1
Component/s: Validators
Affects Version/s: 1.2.2
Fix Version/s: 1.2.3

Type: Bug Priority: Critical
Reporter: Fabian Spillner Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

OS X 10.6.3 - php 5.3.1


Attachments: File DC671TestCase.php    

 Description   

Strange behavior: The record could be saved without getting validation exception even tough a NotNull column is empty.

If you use Sqlite as mock driver you get validation exception. Test case is added into ticket!



 Comments   
Comment by Fabian Spillner [ 09/May/10 ]

there the test case (About filename: I wonder why there is same file (empty test) on the doctrine ticket tests - please rename this test case if I call it wrong)

Comment by Jonathan H. Wage [ 09/May/10 ]

Do you have validation enabled? It is off by default.

Comment by Fabian Spillner [ 09/May/10 ]

Ah! It works now! Thank you! It's new default behavior since 1.2?

Comment by Fabian Spillner [ 09/May/10 ]

I forget to enable the attribute Doctrine_Core::ATTR_VALIDATE.

Comment by Jonathan H. Wage [ 09/May/10 ]

No it has always been off by default.

Comment by Fabian Spillner [ 10/May/10 ]

The reason of my confusion:

On Symfony 1.2 the validation is enabled by default:

http://trac.symfony-project.org/browser/branches/1.2/lib/plugins/sfDoctrinePlugin/config/config.php

And nothing is talked about it:
http://www.symfony-project.org/tutorial/1_4/en/whats-new#chapter_a2fae23c9403b0e9ec99806fccf6b53e_doctrine_integration





[DC-672] Adding DISTINCT when not needed hurts performance badly Created: 10/May/10  Updated: 10/Jun/10  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: Query
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Amir W Assignee: Guilherme Blanco
Resolution: Can't Fix Votes: 0
Labels: None
Environment:

LAMP - other details seem irrelevant



 Description   

Doctrine seems to be adding DISTINCT to SQL queries without any good reason (to the table's primary key!) and thus SERIOUSLY HURTS performance. It even seems like Doctrine adds a whole new DISTINCT query without a good reason. I believe the extra query is the result of the different implementation of LIMIT in Doctrine vs. SQL. However, when a unique index is involved, this extra query seems redundant.

In the following example, execution time went from ~5sec down to ~1sec when I simply removed the DISTINCT added by Doctrine.

Why is the extra query created anyway?
Is there any way to prevent Doctrine to add the DISTINCT part when there's a unique index on the column?

Here's the DQL:

SELECT i.id, ip.item_id, pic.id, pic.width, pic.height, i.producer_id, i.series_id, i.issue_date, t_i.name, FROM ZpcPhonecard i INNER JOIN i.Picture pic WITH pic.width > 125 AND pic.width >= pic.height LEFT JOIN i.Translation t_i WITH t_i.lang = 0 INDEXBY t_i.lang WHERE i.front_picture_id > 0 AND i.state = ? AND i.zpc_system_id = 4 ORDER BY i.id DESC LIMIT 20

And the resulting SQLs:

SELECT DISTINCT z3.id FROM zpc_phonecard z3 INNER JOIN picture p2 ON z3.front_picture_id = p2.id AND ((p2.width > 125 AND p2.width >= p2.height)) LEFT JOIN zpc_phonecard_translation z4 ON z3.id = z4.id AND (z4.lang = 0) WHERE (z3.front_picture_id > 0 AND z3.state = 'Active') AND z3.zpc_system_id = 4 ORDER BY z3.id DESC LIMIT 20;

SELECT z.id AS z_id, z.producer_id AS zproducer_id, z.series_id AS zseries_id, z.issue_date AS zissue_date, p.id AS pid, p.width AS pwidth, p.height AS pheight, z2.id AS z2id, z2.lang AS z2lang, z2.name AS z2_name FROM zpc_phonecard z INNER JOIN picture p ON z.front_picture_id = p.id AND ((p.width > 125 AND p.width >= p.height)) LEFT JOIN zpc_phonecard_translation z2 ON z.id = z2.id AND (z2.lang = 0) WHERE z.id IN ('231871', '231870', '231869', '231868', '231865', '231864', '231863', '231862', '231861', '231860', '231859', '231858', '231857', '231856', '231855', '231853', '231852', '231851', '231850', '231849') AND ((z.front_picture_id > 0 AND z.state = 'Active') AND z.zpc_system_id = 4) ORDER BY z.id DESC

Here's my own translation from DQL to A MUCH FASTER SINGLE SQL statement:
SELECT i.id, pic.id, pic.width, pic.height, i.producer_id, i.series_id, i.issue_date, t_i.name FROM Zpc_Phonecard i INNER JOIN Picture pic ON i.front_picture_id=pic.id AND pic.width > 125 AND pic.width >= pic.height LEFT JOIN Zpc_Phonecard_translation t_i ON i.id = t_i.id AND t_i.lang = 0 WHERE i.front_picture_id > 0 AND i.state = 'Active' AND i.zpc_system_id = 4 ORDER BY i.id DESC LIMIT 20



 Comments   
Comment by Amir W [ 10/May/10 ]

Here's a workaround to avoid Doctrine's handling of LIMIT queries. Is this there recommended way or is there another?

Before the $q->execute() part simply add these lines:

$q->buildSqlQuery(false);
$map = $q->getRootDeclaration();
$table = $map['table'];
$table->setAttribute(Doctrine_Core::ATTR_QUERY_LIMIT, Doctrine_Core::LIMIT_ROWS);

Now the DQL is automatically translated to the following SQL:

SELECT z.id AS z_id, z.producer_id AS zproducer_id, z.series_id AS zseries_id, z.issue_date AS zissue_date, p.id AS pid, p.width AS pwidth, p.height AS pheight, z2.id AS z2id, z2.lang AS z2lang, z2.name AS z2_name FROM zpc_phonecard z INNER JOIN picture p ON z.front_picture_id = p.id AND ((p.width > 125 AND p.width >= p.height)) LEFT JOIN zpc_phonecard_translation z2 ON z.id = z2.id AND (z2.lang = 0) WHERE ((z.front_picture_id > 0 AND z.state = 'Active') AND z.zpc_system_id = 4) ORDER BY z.id DESC LIMIT 20

Comment by Jonathan H. Wage [ 08/Jun/10 ]

This is the dreaded limit subquery algorithm at work. I don't think we can do anything to patch/fix this in Doctrine 1.

Comment by Amir W [ 10/Jun/10 ]

As there is a workaround (as I've suggested above) you can obviously add a flag (via a function or any other way) allowing the user NOT to use the "dreaded limit subquery"...





[DC-645] Query with a leftJoin() + where(NOT IN) + limit() generate wrong SQL alias in the NOT IN part Created: 23/Apr/10  Updated: 02/Sep/10  Resolved: 02/Sep/10

Status: Resolved
Project: Doctrine 1
Component/s: Query
Affects Version/s: 1.2.2
Fix Version/s: 1.2.4

Type: Bug Priority: Critical
Reporter: David Jeanmonod Assignee: Guilherme Blanco
Resolution: Fixed Votes: 1
Labels: None
Environment:

PHP 5.3.1 (cli) (built: Feb 11 2010 02:32:22)
mysql Ver 14.14 Distrib 5.1.41, for apple-darwin9.5.0 (i386) using readline 5.1
Doctrine version 1.2.2 from SVN: http://doctrine.mirror.svn.symfony-project.com/tags/1.2.2/lib/Doctrine.php


Attachments: File NotIn_LeftJoin_Limit_TestCase.php    

 Description   

I have a simple case with 3 Classes. Contact, Phone and Email. A Contact can many many phones, but only one email.

When doing this simple query:
$query = Doctrine_Query::create()->from('Contact c');
$query->leftJoin('c.Phones p ON c.id = p.contact_id');
$query->where('c.id NOT IN (SELECT Email.contact_id FROM Email)');
$query->execute();
Every thing went fine.

But when adding a limit() condition like:
$query->limit(20);
$query->execute();

The SQL generated query is not valid. There is a problem with the alias used in the NOT IN subquery. This query is generated like this:
WHERE c2.id NOT IN (SELECT e2.contact_id AS e2__contact_id FROM email e)
There is a mix between alias e2 and e

I have been trying to debug, but I didn't understand what was going wrong. The problem seems to happend in the class Doctrine_Query between line 1486 and 1554, but this part is obscur to me.

I attach to this ticket a valid TestCase

Thanks for support



 Comments   
Comment by David Jeanmonod [ 23/Apr/10 ]

TEXT VERSION OF THE TEST CASE

<?php

require_once('doctrine/lib/Doctrine.php');
spl_autoload_register(array('Doctrine', 'autoload'));
$manager = Doctrine_Manager::getInstance();
$manager->setAttribute(Doctrine::ATTR_EXPORT, Doctrine::EXPORT_ALL);
$conn = Doctrine_Manager::connection('mysql://root:@localhost/test_doctrine');
echo "Connection is set up\n";

class Contact extends Doctrine_Record {
public function setTableDefinition()

{ $this->setTableName('contact'); $this->hasColumn('name', 'string', 300, array('type' => 'string', 'length' => 300)); }

public function setUp()

{ $this->hasMany('Phone as Phones', array('local' => 'id', 'foreign' => 'contact_id')); $this->hasOne('Email as Email', array('local' => 'id', 'foreign' => 'contact_id')); }

}
class Phone extends Doctrine_Record {
public function setTableDefinition()

{ $this->setTableName('phone'); $this->hasColumn('contact_id', 'integer', null, array('type' => 'integer')); }

public function setUp()

{ $this->hasOne('Contact', array('local' => 'contact_id', 'foreign' => 'id', 'onDelete' => 'CASCADE')); }

}
class Email extends Doctrine_Record {
public function setTableDefinition()

{ $this->setTableName('email'); $this->hasColumn('contact_id', 'integer', null, array('type' => 'integer')); }

public function setUp()

{ $this->hasOne('Contact', array('local' => 'contact_id', 'foreign' => 'id')); }

}
echo "Classes Contact, Phone and Email are defines\n";

try

{Doctrine::dropDatabases();}

catch(Exception $e){} // Drop if exist
Doctrine::createDatabases();
Doctrine::createTablesFromArray(array('Contact', 'Phone', 'Email'));
echo "Databases tables are create\n";

$query = Doctrine_Query::create()->from('Contact c');
$query->leftJoin('c.Phones p ON c.id = p.contact_id');
$query->where('c.id NOT IN (SELECT Email.contact_id FROM Email)');
$query->limit(20);
try {
$query->execute();
echo "TEST: Doctrine LEFTJOIN + NOT IN + LIMIT is OK\n";
}
catch (Exception $e){
echo "TEST: Doctrine LEFTJOIN + NOTIN + LIMIT is NOT working\n Alias in the NOT IN part are wrong, we get e2 and e\n\nDetail of the error:\n ", $e->getMessage(), "\n";
}

Comment by will ferrer [ 30/Jun/10 ]

Hi David

I had a problem with subqueries in the which I worked around by including real sql in the subquery with a prefix of SQL:.

My bug occurred trying to run this code:

$q = Doctrine_Query::create(); 
$q->from('Customer Customer'); 
$q->addWhere(' Customer.id in (SELECT Customer.id as customer_id FROM Customer Customer)'); 
$q->addSelect('Customer.id'); 
$q->addSelect('Customer.id as customer_id');
$q->limit(20);

However this code works fine for me now (though make sure you have the latest svn build of doctrine because there were some patches that helped this work):

$q = Doctrine_Query::create(); 
$q->from('Customer Customer'); 
$q->addWhere(' Customer.id in (SQL:SELECT p.id AS p__0 FROM product_customers p)'); 
$q->addSelect('Customer.id'); 
$q->addSelect('Customer.id as customer_id');

Notice the use of SQL: in the subquery.

Here is the bug:

http://www.doctrine-project.org/jira/browse/DC-692

Also worth noting that I am getting that sql by building another query and then running the getSqlQuery method to return the sql I am then using in the other query.

Hope that helps.

Will Ferrer





[DC-633] Charset problem Created: 15/Apr/10  Updated: 08/Jun/10  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Oguzhan Cetin Assignee: Jonathan H. Wage
Resolution: Cannot Reproduce Votes: 0
Labels: None
Environment:

windows 7 , bitnami wappstack



 Description   

Hi,

I'm trying to insert query . But i have a charset problem.

I'm poor speak english, sorry.

my bootstrap code quote:

$conn = Doctrine_Manager::connection($db);
$conn->setCollate('utf8_unicode_ci');
$conn->setCharset('utf8');

controller code:
$user = new YpUser();
$user -> business_name = 'Turcom';
$user -> complete_adress = 'Maslak/Itü';
$user -> contact_person = 'Oguzhan Cetin';
$user -> contact_number = '05065000000';
$user -> save();

///////////////////
Error:

Fatal error: Uncaught exception 'Doctrine_Connection_Pgsql_Exception' with message 'SQLSTATE[22021]: Character not in repertoire: 7 HATA: "UTF8" dil kodlaması için geçersiz bayt dizisi: 0xfc HINT: Bu hata ayrıca bayt sırasının sunucunun beklediÄ� kodlamada olmadıÄ�± zaman meydana gelmektedir. İstemci dil kodlaması "client_encoding" seçeneÄ� ile ayarlanmaktadır.' in C:\LOCAL\stack\apache2\htdocs\pbmp\libraries\doctrine\Doctrine\Connection.php:1082 Stack trace: #0 C:\LOCAL\stack\apache2\htdocs\pbmp\libraries\doctrine\Doctrine\Connection\Statement.php(269): Doctrine_Connection->rethrowException(Object(PDOException), Object(Doctrine_Connection_Statement)) #1 C:\LOCAL\stack\apache2\htdocs\pbmp\libraries\doctrine\Doctrine\Connection.php(1042): Doctrine_Connection_Statement->execute(Array) #2 C:\LOCAL\stack\apache2\htdocs\pbmp\libraries\doctrine\Doctrine\Connection\Pgsql.php(244): Doctrine_Connection->exec('INSERT INTO yp_...', Array) #3 C:\LOCAL\stack\apache2\htdocs\pbmp\libraries\doctrine\Doctrine\Connection\U in C:\LOCAL\stack\apache2\htdocs\pbmp\libraries\doctrine\Doctrine\Connection.php on line 1082

Thanks For help.






[DC-620] Unserialize does not add entity to the table entitymap Created: 06/Apr/10  Updated: 08/Jun/10  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: Record
Affects Version/s: 1.2.2
Fix Version/s: 1.2.3

Type: Bug Priority: Critical
Reporter: Maurice Makaay Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None

Attachments: Text File Doctrine_Record.unserialize.patch     Text File Doctrine_Record.unserialize.patch    

 Description   

Hello,

Considering this test script:

if (!file_exists("/tmp/serialize.doctrine"))
{
  $a = Address::getById(1);
  $b = Address::getById(2);
  $c = Address::getById(3);
  $d = Address::getById(4);
  print "Serialize using oid=" . $d->getOid() . "\n";
  file_put_contents("/tmp/serialize.doctrine", serialize($d));
}
else
{
  $d = unserialize(file_get_contents("/tmp/serialize.doctrine"));
  print "unserialized oid = " . $d->getOid() . "\n";

  $d2 = Address::getById(4);
  print "new object oid = " . $d2->getOid() . "\n";

  $d->housenumber ++;
  print "{$d->housenumber} versus {$d2->housenumber}\n";
}

Things are going wrong in the unserialization. In this test script, I manage to create two objects holding the same oid, but pointing at different object instances (but for the same database id). The output of this script after running it twice shows me:

$ php test.php
Serialize using oid=5
$ php test.php
unserialized oid = 5
new object oid = 5
24 versus 23

Looking at the unserialize() code in Doctrine_Record, the problem seems to be coming from two issues in there:

  • $this->_oid is generated before the data unserialization. Because of this, the generated $this->_oid is overwritten with whatever was in the serialized data (this is why in above example, the oid became 5 for the unserialized record)
  • The unserialized record is not added to the table's identitymap, making it kind of invisible for the table's getRecord() code, resulting in two different objects that both represent the entity with id = 4 (as you can see in the "24 versus 23" output).

See the attached patch for a fix that I did on our code tree to make the test work.

Note that I added some extra code in the unserialize method to cleanup any existing entitymap and table repository entry for the unserialized object. This is of course not the best route, but it helps with some unit testing code where a lot of serialize/unserialize handling is going on and objects got mixed up when not doing this cleanup.
IMO, better would be to add an unserialization method to the Doctrine_Table, which would, like find(), act as a factory for turning unserialized record data into a Doctrine_Record object, possibly making use of already loaded entities that are available in the table's internal caches. With unserialize being handled directly from the Doctrine_Record, there is no way AFAICS to keep up the rule of always having exactly one object in a scope that represents a certain object in the database.

When running above script with this patch applied and with removing the $a, $b and $c assignments (just for making the oid's different between the two script runs), we get the following output:

$ php test.php
Serialize using oid=2
$ php test.php
unserialized oid = 5
new object oid = 5
24 versus 24

So here, unserializing an object and then reloading the object through the table object gives use two times the same object, representing db object with id = 4;

When doing things in a different order, we still can force an issue, but this is due to the things mentioned above: to cleanly handle this, unserialization should be handled from a factory method on Table. This script shows the behavior:

if (!file_exists("/tmp/serialize.doctrine"))
{
  $d = Address::getById(4);
  print "Serialize using oid=" . $d->getOid() . "\n";
  file_put_contents("/tmp/serialize.doctrine", serialize($d));
}
else
{
  $d2 = Address::getById(4);
  print "new object oid = " . $d2->getOid() . "\n";

  $d = unserialize(file_get_contents("/tmp/serialize.doctrine"));
  print "unserialized oid = " . $d->getOid() . "\n";

  $d3 = Address::getById(4);
  print "new object oid = " . $d3->getOid() . "\n";

  $d->housenumber ++;
  print "{$d->housenumber} versus {$d2->housenumber} versus {$d3->housenumber}\n";
}

The output of the second run being:

$php test.php
new object oid = 2
unserialized oid = 5
new object oid = 5
24 versus 23 versus 24

The first object that was created is still different from the unserialized object, but at least the third object that is created, is the object that was unserialized.

I hope that this makes the issue clear and that my input helps in fixing things. This issue provided us with some really unexpected behavior and we're glad that we were able to track it down to here.



 Comments   
Comment by Maurice Makaay [ 07/Apr/10 ]

FYI: we removed the "Remove existing record from the repository and table entity map" code from our patch. We had some issues with unit testing without this bit, but we updated the unit tests to work without this bit of code. The "Add the unserialized record to repository and entity map' part stays of course. I think you can ignore the cleanup code in the patch. As long as scripts unserialize data before working with it, things should be fine without it.

Comment by Maurice Makaay [ 07/Apr/10 ]

New version of the patch, without the cleanup code in it.

Comment by Maurice Makaay [ 07/Apr/10 ]

Trying to always use the same object for the same entity in the database, we came up with a factory method that we put on our record class as a static public. Maybe this is an idea that you want to include in Doctrine as well?

  static public function fromSerialized($serialized)
  {
    $entity = unserialize($serialized);

    if (!($entity instanceof Doctrine_Record)) throw new Exception(
      __METHOD__ . ': serialized object is not a Doctrine_Record object'
    );

    // If the unserialized object is a persisted entity, then we must
    // check if there is already an object for that entity available in
    // Doctrine's table repository.
    if ($entity->exists())
    {
      // Retrieve the entity through the table repository.
      $table = $entity->getTable();
      $repository_entity = $table->find($entity->id);

      // If a different object was returned than our unserialized
      // object, then there was an object loaded before unserialization.
      // We will merge the data from the unserialized object with
      // the existing object and return the existing object to the caller.
      if ($entity->getOid() !== $repository_entity->getOid()) {
        $repository_entity->merge($entity);
        $entity = $repository_entity;
      }
    }

    return $entity;
  }

For us, it's working wonders in combination with above Doctrine_Record::unserialize() patch. When letting this code handle our unserialization, the same object (checked by its oid) is used for referencing the same entity throughout the code. We call the code like this:

$object = Doctrine_Record::fromSerialized($serialized_data);




[DC-603] Doctrine model loading broken on Centos/RHEL 5 Created: 26/Mar/10  Updated: 29/Mar/10  Resolved: 29/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Dennis Jacobfeuerborn Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

Doctrine 1.1.5 on Centos 5.4



 Description   

The loading of models fails on Centos 5 systems because loadModels() relies on RecursiveDirectoryIterator which seems to return filenames in different orders on different operating systems (see this older doctrine bug: http://trac.doctrine-project.org/ticket/1688 ).

The result is that Doctrine tries to load the model class before it's base class which fails because the base-class is not yet known at that time.

Example:

I setup the auto-loading like this:

spl_autoload_register( array('Doctrine', 'autoload') );
Doctrine::loadModels('/data/models');

and "/data" looks like this:
/data/models/ftp/FtpTransfers.php
/data/models/ftp/FtpTransfersTable.php
/data/models/ftp/generated/BaseFtpTransfers.php

However when I run a test app I get the following error:
PHP Fatal error: Class 'BaseFtpTransfers' not found in /data/models/
ftp/FtpTransfers.php on line 13

Executing the same code works fine on Fedora.

This is the order in which loadModels() tries to load the files on both systems:

Centos 5:
/data/models/ftp/ftp_logs/FtpTransfers.php
/data/models/ftp/ftp_logs/FtpStatsHour.php
/data/models/ftp/ftp_logs/FtpStatsDayTable.php
/data/models/ftp/ftp_logs/FtpTransfersTable.php
/data/models/ftp/ftp_logs/generated/BaseFtpStatsHour.php
/data/models/ftp/ftp_logs/generated/BaseFtpTransfers.php
/data/models/ftp/ftp_logs/generated/BaseFtpStatsDay.php
/data/models/ftp/ftp_logs/FtpStatsHourTable.php
/data/models/ftp/ftp_logs/FtpStatsDay.php

Fedora 11:
/data/models/ftp/ftp_logs/generated/BaseFtpStatsHour.php
/data/models/ftp/ftp_logs/generated/BaseFtpTransfers.php
/data/models/ftp/ftp_logs/generated/BaseFtpStatsDay.php
/data/models/ftp/ftp_logs/FtpStatsHourTable.php
/data/models/ftp/ftp_logs/FtpStatsDayTable.php
/data/models/ftp/ftp_logs/FtpTransfers.php
/data/models/ftp/ftp_logs/FtpStatsHour.php
/data/models/ftp/ftp_logs/FtpStatsDay.php
/data/models/ftp/ftp_logs/FtpTransfersTable.php



 Comments   
Comment by Jonathan H. Wage [ 29/Mar/10 ]

The aggressive model loading can only be used if you handle the dependencies between classes yourself or you don't have any dependencies and it is no problem to aggressively require all files it finds in a directory. It sounds like you need to use conservative or pear style model loading.





[DC-582] DataDict entry missing for datetime type for MySQL causes migrations to fail due to sql error Created: 18/Mar/10  Updated: 10/Jan/12  Resolved: 29/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Migrations
Affects Version/s: 1.2.0, 1.2.1
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Rich Birch Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

LAMP



 Description   

I discovered this whilst trying out migrations via symfony. I added a datetime field to my schema.yml and generated the migrations, but upon running the migration I got the following error:

SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '()' at line 1. Failing Query: "ALTER TABLE order_item ADD purchased_at datetime()"

The following code causes the failure in the actual migration:

$this->addColumn('order_item', 'purchased_at', 'datetime', '', array());

because it generates the following sql:

ALTER TABLE order_item ADD purchased_at datetime()

The diff from my patched version which fixes the issue is as follows:

Index: Doctrine/DataDict/Mysql.php
===================================================================
— Doctrine/DataDict/Mysql.php (revision 7415)
+++ Doctrine/DataDict/Mysql.php (working copy)
@@ -227,6 +227,7 @@
return 'DATE';
case 'time':
return 'TIME';
+ case 'datetime':
case 'timestamp':
return 'DATETIME';
case 'float':

It's against the following repository file:

http://doctrine.mirror.svn.symfony-project.com/branches/1.2/lib/Doctrine/DataDict/Mysql.php

I hope this is useful and gets committed



 Comments   
Comment by Rich Birch [ 22/Mar/10 ]

I've just discovered that the same issue exists for fields of type 'text':

SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ') NOT NULL, session_time DATETIME NOT NULL, INDEX session_id_index_idx (session_' at line 1. Failing Query: "CREATE TABLE session (id BIGINT AUTO_INCREMENT, session_id VARCHAR(64) NOT NULL, session_data text() NOT NULL, session_time DATETIME NOT NULL, INDEX session_id_index_idx (session_id), PRIMARY KEY(id)) ENGINE = INNODB"

from the following schema:

Session:
columns:
session_id:

{ type: string(64), notnull: true }

session_data:

{ type: text, notnull: true }

session_time:

{ type: timestamp, notnull: true }

indexes:
session_id_index:
fields: [ session_id ]
unique: true

I guess there may be other field entries missing too. Is there a comprehensive list of doctrine field types somewhere?

Comment by Rich Birch [ 22/Mar/10 ]

Ok, I might have been being dumb here. I've just checked the doctrine documentation for defining the schema (http://www.doctrine-project.org/documentation/manual/1_2/en/defining-models) and there's no mention of a datetime or text field (I've just realised that I should have used string instead of text anyway), but datetime still works as a column type so shouldn't it be documented?

I guess either datetime should be fully removed or fully supported

Comment by Jonathan H. Wage [ 29/Mar/10 ]

You should be using the Doctrine portable types. So you would use date or timestamp I believe and Doctrine will convert it to the appropriate type for your dbms.

Comment by ToleaN [ 10/Jan/12 ]

not correctly generated migration, in my case generated:

$this->addColumn('tree', 'published_at', 'datetime', '', array(
));

but if change the fourth parameters on null, all ok





[DC-562] Broken OneToOne Relationship on two Primary Keys Created: 11/Mar/10  Updated: 27/Aug/10  Resolved: 15/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Data Fixtures
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Ulf Thomas Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

Doctrine 1.2.1
MySQL 5.1.37



 Description   

Hi,
if you have a One-To-One Relationship which uses Primary Keys on both ends Doctrine screws up the relationship if you import the data from a data fixture.

Example:

DB contents:

User:
id|username|password
1|eins|eins
2|zwei|zwei
3||drei|drei
4|vier|vier

Contact:
id|name
1|eins
3|drei

Contact:
id|name
2|zwei
4|vier

{{
//schema.yml
User:
columns:
id:
primary: true
autoincrement: true
type: integer(4)
username: string(255)
password: string(255)
relations:
Contact:
local: id
foreign: id
foreignType: one

Contact:
columns:
id:
type: integer(4)
primary: true
autoincrement: true
name:
type: string(255)

Contact2:
columns:
id:
type: integer(4)
primary: true
autoincrement: true
name:
type: string(255)
}}

./doctrine dump-data

{{
//data/fixtures/data.yml
Contact:
Contact_1:
name: eins
Contact_3:
name: drei
Contact2:
Contact2_2:
name: zwei
Contact2_4:
name: vier
User:
User_1:
username: eins
password: eins
User_2:
username: zwei
password: zwei
User_3:
username: drei
password: drei
User_4:
username: vier
password: vier
}}

./doctrine build-all-reload

DB contents:

User:
id|username|password
1|eins|eins
2|zwei|zwei
3||drei|drei
4|vier|vier

Contact:
id|name
1|eins
2|drei

Contact:
id|name
1|zwei
2|vier

Now the relations are broken.



 Comments   
Comment by Jonathan H. Wage [ 15/Mar/10 ]

Your ticket is hard to read, just for future reference it would be wise to make sure the issue is readable for us. Anyways, I had a look and your schema is incorrect. You have a foreign key but it is also set as primary and auto increment.

Comment by Ulf Thomas [ 16/Mar/10 ]

Sorry for the unreadable ticket, I tryed to make it understadable as much as I can.

The thing with the foreign key and primary key is that we have a existing database and want to port it to Doctrine.
In the database the structure is like that and in the documentation I didn't see anything that say that this won't work. Can you point me to some aditional information?

Comment by Shirley Chan [ 27/Aug/10 ]

I'm working on an open source application, Sahana Agasti (http://sahanafoundation.org/), using Symfony 1.4 as the framework and Doctrine as the ORM. This project stumble into a similar issue with establishing a direct 1:1 relationships on primary keys. Below is an example of a person and person's date of birth table. In the yml file, Person has an id field as the primary key. This field is referenced by the primary key field person_id in PersonDateOfBirth. However, Doctrine only creates the tables and not the relationship between the two tables correctly.

schema.yml
agPerson:
columns:
id:
primary: true
type: integer(5)
autoincrement: true
notnull: true
relations:
agPersonDateOfBirth:
type: one
local: id
foreign: person_id
agPersonDateOfBirth:
columns:
person_id:
primary: true
type: integer(5)
notnull: true
date_of_birth:
type: date
notnull: true
relations:
agPerson:
foreignType: one
type: one
local: person_id
foreign: id

schema.sql
CREATE TABLE ag_person (id BIGINT AUTO_INCREMENT, PRIMARY KEY(id)) ENGINE = INNODB;
CREATE TABLE ag_person_date_of_birth (person_id BIGINT, date_of_birth DATE NOT NULL, PRIMARY KEY(person_id)) ENGINE = INNODB;





[DC-534] Couldn't hydrate. Found non-unique key mapping named 'lang' Created: 02/Mar/10  Updated: 28/Dec/11  Resolved: 15/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: I18n
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Adamczewski Assignee: Jonathan H. Wage
Resolution: Incomplete Votes: 0
Labels: None
Environment:

PHP 5.2.9 Symfony 1.4



 Description   

For the following query

return $this->createQuery('a')
->innerJoin('a.Translation t WITH t.lang = ?', $lang)
->innerJoin('a.UserAttributes ua')
->innerJoin('ua.AttrOptions o WITH o.attribute_id = a.id')
->innerJoin('o.Translation ot WITH ot.lang = ?', $lang)
->addWhere('ua.user_id = ?', $userId)
->addWhere('a.tab = ?', $tab)
->addOrderBy('a.ord asc')
->execute();

Unknown macro: {/quote}

i get

Couldn't hydrate. Found non-unique key mapping named 'lang'.

{/quote}

error, so i cannot join more than one translation table in one query because it cause error.



 Comments   
Comment by Jonathan H. Wage [ 02/Mar/10 ]

Can you create a test case for this? You can find information about Doctrine unit testing here: http://www.doctrine-project.org/documentation/manual/1_2/en/unit-testing

Comment by Mikhail Menshinskiy [ 28/Dec/11 ]

I have the same error.
How to solve this issue?





[DC-523] Aggregating values in select(), always joins a record, even if there is no relation Created: 25/Feb/10  Updated: 08/Jun/10  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: pbijl Assignee: Jonathan H. Wage
Resolution: Can't Fix Votes: 0
Labels: None
Environment:

mysql5, unix


Attachments: File AggregateFunctionalExpressiosTestCase.php    

 Description   

Example:

I want to aggregate a expression, so i need to select everything:

->select('a., d., CONCAT_WS(" ", d.initials, d.surname) as formalName))
->from('Account a')
->leftJoin('a.Details d')

Result will be an empty record of Details if there is no relation, removing the aggregated value will fix this and not join a record.



 Comments   
Comment by Jonathan H. Wage [ 01/Mar/10 ]

Sorry, I don't quite understand with such minimal information. Can you provide a failing test case?

Comment by pbijl [ 02/Mar/10 ]
 

// logical example
// result: working as expected
$q = Doctrine_Query::create()
->from('Accounts a')
->leftJoin('a.Data d')
->execute()
;

// this time the same query but with a ambigious select clause
// result: working as expected, 
$q = Doctrine_Query::create()
->select('a.*, d.*')
->from('Accounts a')
->leftJoin('a.Data d')
->execute()
;

// same query, but there should not be a Data row because of the bogus WITH condition
// result: working as expected
$q = Doctrine_Query::create()
->select('a.*, d.*')
->from('Accounts a')
->leftJoin('a.Data d with d.type = "blabla"')
->execute()
;

// same query, but with an aggregated column
// result: working as expected, again, no Data row
$q = Doctrine_Query::create()
->select('a.*, d.*, d.surname as test')
->from('Accounts a')
->leftJoin('a.Data d with d.type = "blabla"')
->execute()
;

// same query, but with an aggregated functional expression
// result: FAILS. `test` is joined as an empty column in the Accounts record, AND the Data record, which results in an empty Data record 
// you can imagine saving the Accounts object to save an empty Data record in return
$q = Doctrine_Query::create()
->select('a.*, d.*, concat_ws(" ", d.firstname, d.surname) as test')
->from('Accounts a')
->leftJoin('a.Data d with d.type = "blabla"')
->execute()
;

// a expression that doesnt concatenates doesnt join an empty Data record, but does aggregate a empty `test` column on Accounts
$q = Doctrine_Query::create()
->select('a.*, d.*, trim(d.firstname) as test')
->from('Accounts a')
->leftJoin('a.Data d with d.type = "blabla"')
->execute()
;

Comment by Jonathan H. Wage [ 02/Mar/10 ]

Hi, you can find information about how to create a valid Doctrine unit test case here: http://www.doctrine-project.org/documentation/manual/1_2/en/unit-testing

Comment by pbijl [ 10/Mar/10 ]
<?php

class Doctrine_AggregateFunctionalExpressios_TestCase extends Doctrine_UnitTestCase 
{
	private function baseQueryForAllMethods() {
		return Doctrine_Query::create()
		->from('User u')
		->where('u.id = ?', 4)
		;
	}
    public function testAggregateColumnInParentModel()
    {
		$res = $this->baseQueryForAllMethods()
		->select('u.*, concat(u.name, u.loginname) as aggregatedColumn, p.*')
		->leftJoin('u.Phonenumber p')		
		->fetchOne()
		;
		$this->assertTrue(isset($res->aggregatedColumn));
		$this->assertFalse(isset($res->Phonenumber->aggregatedColumn));
	}	
    public function testAggregateColumnInJoinedModel()
    {
		$res = $this->baseQueryForAllMethods()
		->select('u.*, p.*, concat(p.phonenumber, p.entity_id) as aggregatedColumn')
		->leftJoin('u.Phonenumber p')		
		->fetchOne()
		;
		$this->assertTrue(isset($res->Phonenumber->aggregatedColumn));
		$this->assertFalse(isset($res->aggregatedColumn));
	}
    public function testAggregateColumnInJoinedModelWithFailingWhereClause()
    {
		$res = $this->baseQueryForAllMethods()
		->select('u.*, p.*, concat(p.phonenumber, p.entity_id) as aggregatedColumn')
		->leftJoin('u.Phonenumber p WITH p.id = ?', 100)		
		->fetchOne()
		;
		$this->assertFalse(isset($res->Phonenumber->aggregatedColumn));		
		$this->assertFalse(isset($res->aggregatedColumn));				
	}
}
Comment by pbijl [ 10/Mar/10 ]

testcase attached~

Comment by Jonathan H. Wage [ 08/Jun/10 ]

Your test case has some errors in it. The ->Phonenumber relationship is a many so it is a Doctrine_Collection. Also, this is all expected behavior for aggregates to be located in the root. They only exist in the relationship as well for BC reasons. The whole functionality is flawed a bit and can't be patched without breaking backwards compatibility. All these issues have been thought through and addressed in Doctrine 2





[DC-514] cascading behaviors Created: 22/Feb/10  Updated: 15/Mar/10  Resolved: 15/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Behaviors, Record
Affects Version/s: 1.1.4, 1.1.5, 1.1.6, 1.2.0, 1.2.1
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Andreas Wissl Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None


 Description   

If you want to add a behavior to a behavior, you (depending on your template) can get an error message telling you, that the _table property is not an object.

This happens, as actAs() method of Record_Abstract does first the $tpl->setUp() call and then the $tpl->setTableDefnition(), as for the cascading templates - which are using Record_Generator - the $child->setTableDefinition() is called before $child->setUp() in method buildChildDefinitions(). Switching those two lines fixes the error and has, as far we can see (having it in our production system for quite a while now) no side effects besides fixing the error.



 Comments   
Comment by Jonathan H. Wage [ 01/Mar/10 ]

Hi, when providing feedback and you have a change, it is best to provide a patch. Can you generate a patch for your changes and attach it to the ticket?

However, if I understand your verbal description of the change correctly, it is not right. We can't call setUp() first, because setTableDefinition() is supposed to be called first. This is how it is supposed to work.





[DC-832] PostgreSQL - lastInsertId fails because sequence name on table with column alias on primary key does not work [+patch] Created: 18/Aug/10  Updated: 24/Aug/10  Resolved: 24/Aug/10

Status: Resolved
Project: Doctrine 1
Component/s: Query
Affects Version/s: 1.2.2
Fix Version/s: 1.2.3

Type: Bug Priority: Critical
Reporter: Enrico Stahn Assignee: Guilherme Blanco
Resolution: Fixed Votes: 0
Labels: None


 Description   

PostgreSQL - lastInsertId fails because sequence name on table with column alias on primary key does not work.

DDL created sequence name: <table>_<column name>
Last Insert ID sequence name: <table>_<identifier> (identifer is the field name which can be different to the column name)



 Comments   
Comment by Enrico Stahn [ 18/Aug/10 ]

Patch:
http://github.com/estahn/doctrine1/tree/DC-832
http://github.com/estahn/doctrine1/commit/5921da9fe159844e354fd280dca8cc156a8680a7

Comment by Jonathan H. Wage [ 24/Aug/10 ]

Thanks, fixed by http://trac.doctrine-project.org/changeset/7684





[DC-800] PostgreSQL does not have LOCATE expressions Created: 28/Jul/10  Updated: 24/Aug/10  Resolved: 24/Aug/10

Status: Resolved
Project: Doctrine 1
Component/s: Native SQL, Query
Affects Version/s: None
Fix Version/s: 1.2.3

Type: Bug Priority: Critical
Reporter: Ilya Sabelnikov Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None
Environment:

PHP: v5.2.13 (cli) (built: May 6 2010 01:51:58) Zend Engine v2.2.0, Xdebug v2.0.5

OS: FreeBSD x 8.0-RELEASE-p2 FreeBSD 8.0-RELEASE-p2 #0: Thu May 6 03:37:19 EEST 2010 x@y.z:/usr/obj/usr/src/sys/CUSTOM_8_0 amd64

Database: postgres (PostgreSQL) 8.4.3

Symfony: 1.4.7-DEV (/web/vendor/symfony/1.4-svn/lib)

Web-server: nginx/0.7.65



 Description   

Introduction

As it's described in Doctrine documentation: http://www.doctrine-project.org/projects/orm/1.2/docs/manual/dql-doctrine-query-language/en#functional-expressions. In case I have correctly understood this documentation, I can use registered expressions (CONCAT,TRIM,LOCATE etc.) with supported database drivers.

Issue

The problem is with PostgreSQL, - it does not have the string function "LOCATE" since v7.4 (i have no info about previous version)

Here is my example:

  $q = PortalTable::getInstance()
      ->createQuery()
      ->addSelect(
        '(1 <= LOCATE(host, ?) as is_host_matched)',
        array('.google.com')
  );

  $q->execute();

And this code dies with an error:

Doctrine_Connection_Pgsql_Exception: SQLSTATE[42601]: Syntax error: 7 ERROR: syntax error at or near "as" LINE 1: SELECT (1 <= LOCATE("p"."host", $1) as is_host_matched) AS "... ^. Failing Query: "SELECT (1 <= LOCATE("p"."host", ?) as is_host_matched) AS "p__0" FROM "portal" "p"" in /web/vendor/symfony/1.4-svn/lib/plugins/sfDoctrinePlugin/lib/vendor/doctrine/Doctrine/Connection.php on line 1082

In PgSQL you can use POSITION for this needs (I want to mention, MySQL has this function too as an alias for LOCATE)

btw, POSITION is SQL-92 standard http://owen.sj.ca.us/~rk/howto/sql92.html - maybe it is better to rename LOCATE with POSITION?

Patch

Index: Doctrine/Expression/Pgsql.php
===================================================================
--- Doctrine/Expression/Pgsql.php	(revision 7678)
+++ Doctrine/Expression/Pgsql.php	(working copy)
@@ -230,4 +230,31 @@
     	$translate = 'TRANSLATE(' . $string . ', ' . $from . ', ' . $to . ')';
     	return $translate;
     }
-}
\ No newline at end of file
+
+    /**
+     * transform locate to position
+     *
+     * @param string $substr string to find
+     * @param string $str to find where
+     * @return string
+     */
+    public function locate($substr, $str)
+    {
+        return $this->position($substr, $str);
+    }
+
+    /**
+     * position
+     *
+     * @param string $substr string to find
+     * @param string $str to find where
+     * @return string
+     */
+    public function position($substr, $str)
+    {
+        $substr = $this->getIdentifier($substr);
+        $str = $this->getIdentifier($str);
+        
+        return sprintf('POSITION(%s IN %s)', $substr, $str);
+    }
+}

Solution without patch:

  # will work with PgSQL and MySQL (tested)
  $exp = new Doctrine_Expression('POSITION(host IN ?)');

  $q = PortalTable::getInstance()
      ->createQuery()
      ->addSelect(
        "(1 <= {$exp} as is_host_matched)",
        array('.google.com')
  );

  $q->execute();


 Comments   
Comment by Jonathan H. Wage [ 24/Aug/10 ]

Fixed in http://trac.doctrine-project.org/changeset/7685

Thanks, Jon





[DC-720] Add primary key for count / groupby query Created: 08/Jun/10  Updated: 08/Jun/10  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: 1.2.2
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Brice Maron Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

linux symfony 1.4.5



 Description   

with table

Maintenance:
columns:
id:

Unknown macro: { type}

record_id:

Unknown macro: { type}

,
referenced_relation:

Unknown macro: { type}

people_ref:

Unknown macro: { type}

,
category:

Unknown macro: { type}

i'm trying to count number of maintenances by record_id

and doctrine construct my dql correctly :

SELECT COUNT(m.id) AS cnt, m.record_id FROM Maintenance m WHERE m.record_id IN GROUP BY m.record_id

But after this, it translate to a wrong sql query :

SELECT c.id AS c_id, c.record_id AS crecord_id, COUNT(c.id) AS c_0 FROM collection_maintenance c WHERE (c.record_id IN ) GROUP BY c.record_id

And postgresql says that c.id must be in the group by... but i don't want it!

Please help me



 Comments   
Comment by Jonathan H. Wage [ 08/Jun/10 ]

Hi, if you execute object record hydration it will always include the identifier. You must bypass the hydration.

For future reference, the Jira system is for reporting bugs. You can ask questions using the mailing lists here: http://www.doctrine-project.org/community





[DC-237] HYDRATE_ARRAY_HIERARCHY: Output is not an array Created: 16/Nov/09  Updated: 16/Nov/09  Resolved: 16/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: Nested Set
Affects Version/s: 1.2.0-BETA2
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Nate Assignee: Jonathan H. Wage
Resolution: Cannot Reproduce Votes: 0
Labels: None


 Description   

When HYDRATE_ARRAY_HIERARCHY to get descendants, the output is not an array as expected. It is a Doctrine_Collection instead. In 1.2 Beta1 it worked as expected. This bug only occurs in Beta2.

Example Code: http://pastebin.ca/1673773
The class used for $file (if you need it): http://pastebin.ca/1673706



 Comments   
Comment by Jonathan H. Wage [ 16/Nov/09 ]

This is working for me as expected with the latest version of Doctrine 1.2 from SVN.





[DC-913] A way to auto detect what connection should be used to run a query based on what table was used in the from Created: 01/Nov/10  Updated: 01/Nov/10  Resolved: 01/Nov/10

Status: Resolved
Project: Doctrine 1
Component/s: Query
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major
Reporter: will ferrer Assignee: Guilherme Blanco
Resolution: Fixed Votes: 0
Labels: None
Environment:

XP Xamp



 Description   

I needed a way to allow my queries to figure out what connection to use based on what table was used in the from. In other words I needed my queries to be able to run on the same connection that the table in the from used, but the way my code is structured I didn't have a convenient way to do this outside of doctrine.

I built a feature into doctrine for this I called: autodetectConnection.

I will post the patch for this after I build a test case for this ticket.

Will Ferrer



 Comments   
Comment by will ferrer [ 01/Nov/10 ]

Upon trying to make the test cases for this I looked into the feature I had added further and found that it seems as though doctrine has this ability already. I suspect my addition of this feature was in response to a bug I had which I have since fixed.





[DC-848] Validator Timestamp does not validate "YYYY-MM-DD hh:mm:ss"-Timestamps Created: 31/Aug/10  Updated: 01/Sep/10  Resolved: 01/Sep/10

Status: Resolved
Project: Doctrine 1
Component/s: Validators
Affects Version/s: 1.2.2, 1.2.3
Fix Version/s: 1.2.4

Type: Bug Priority: Major
Reporter: Steffen Zeidler Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None

Attachments: File DC848TestCase.php     Text File Timestamp.patch    

 Comments   
Comment by Steffen Zeidler [ 31/Aug/10 ]

patch & test

Comment by Steffen Zeidler [ 31/Aug/10 ]

patch

Comment by Jonathan H. Wage [ 01/Sep/10 ]

Thanks for the issue and patches!

Fixed here http://github.com/doctrine/doctrine1/commit/680b4ba489d15a4c7fba73ec6a832ca142877b7b





[DC-452] Doctrine_RawSql->parseDqlQueryPart() using $this->_sqlParts Created: 25/Jan/10  Updated: 15/Mar/10  Resolved: 15/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Query
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Christian Blanquera Assignee: Jonathan H. Wage
Resolution: Incomplete Votes: 0
Labels: None
Environment:

Server is CentOS, running PHP5.2.x, Symfony 1.4, Doctrine 1.2



 Description   

Current definition of Doctrine_RawSql->parseDqlQueryPart() and use of Doctrine_Query_Abstract->orWhere() results in "AND WHERE" in query.

DQL Example:
$q->select('distinct p.*')
->from('Post p
LEFT JOIN
Post_Project pp ON (p.id=pp.post_id)
LEFT JOIN
Project_User pu USING (project_id)
LEFT JOIN
Post_User puu ON (p.id=puu.post_id)')
->addComponent('p', 'Post p')
->where('p.created_by = ?', $userId)
->orWhere('pu.user_id = ?', $userId)
->orWhere('puu.user_id = ?', $userId)
->orderBy('p.updated_at DESC');

Results in:
SELECT p.id AS p__id FROM Post p LEFT JOIN Post_Project pp ON (p.id=pp.post_id) LEFT JOIN Project_User pu USING (project_id) LEFT JOIN Post_User puu ON (p.id=puu.post_id) WHERE p.created_by = '4' AND pu.user_id = '4' AND puu.user_id = '4' ORDER BY p.updated_at DESC

I drilled down to the source of the problem and:

Doctrine_RawSql->parseDqlQueryPart()

is using:

$this->_sqlParts

should be using:

$this->_dqlParts

or displaying a note that orWhere() officially does not work with Doctrine_RawSql.



 Comments   
Comment by Jonathan H. Wage [ 15/Mar/10 ]

I tried the change you suggested and as I expected it breaks the test suite. Can you re-open if you have more information or a test case for us to play with? Thanks, Jon





[DC-448] Default Values for option "generatePath" Created: 25/Jan/10  Updated: 15/Mar/10  Resolved: 15/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Import/Export, Record
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Hans-Peter Oeri Assignee: Jonathan H. Wage
Resolution: Incomplete Votes: 0
Labels: None

Attachments: File diff    

 Description   

As of now, each and every record generation has to set "generatePath" individually. a) setting it everywhere and b) changing it afterwards is quite a pain.

I propose a default value mechanism in Doctrine_Core.



 Comments   
Comment by Jonathan H. Wage [ 15/Mar/10 ]

I like the idea but the patch is not really good. What do you think?





[DC-445] SQLite3 *does* support foreign keys Created: 25/Jan/10  Updated: 16/Mar/10  Resolved: 16/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Connection, Relations
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Hans-Peter Oeri Assignee: Jonathan H. Wage
Resolution: Won't Fix Votes: 0
Labels: None
Environment:

SQLite 3


Attachments: File diff    

 Description   

SQLite3 from 3.6.19 does natively support foreign keys. Older versions do read and ignore foreign key declarations - and tools exist to read the schema from a sqlite3 db and create "foreign key triggers"

Not including foreign key declarations in table DDL seems rather odd, if not a bug



 Comments   
Comment by Jonathan H. Wage [ 02/Mar/10 ]

I don't think we can add this to 1.2.2 because instantly turning on this support for people already using sqlite might break existing code. Moving it to 1.3 milestone.

Comment by Jonathan H. Wage [ 16/Mar/10 ]

The version 3.6.19 is pretty new, we should avoid version specific features. Maybe in the future once this version has established itself we can consider adding support for this. Anyways, sqlite is not really a recommended production rdbms.





[DC-439] Import of table with (silly) name "index" Created: 20/Jan/10  Updated: 24/Mar/11  Resolved: 09/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: Import/Export
Affects Version/s: 1.2.1
Fix Version/s: 1.2.3

Type: Bug Priority: Major
Reporter: Jochen Bayer Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None
Environment:

MS-SQL-Server 2005
php 5.2.11
openSUSE 11.1



 Description   

In Doctrine/Import/Mssql.php are this procedure calls:

92: $sql = 'EXEC sp_primary_keys_rowset @table_name = ' . $this->conn->quoteIdentifier($table, true);
99: $sql = 'EXEC sp_columns @table_name = ' . $this->conn->quoteIdentifier($table, true);
219: $query = 'EXEC sp_statistics @table_name = ' . $table;
222: $query = 'EXEC sp_pkeys @table_name = ' . $table;

which fail with sql "keyword" table names.

Using ...['. $table . ' ]' around the table name worked for me.
I don't know anything about the backend of doctrine.
Should this be done by quoteIdentifier, or is this fix okay?
.. @table_name = [' . $this->conn->quoteIdentifier($table, true) . ']';

Many thanx for all!
Jochen Bayer



 Comments   
Comment by Jonathan H. Wage [ 08/Jun/10 ]

It is hard to understand what you changed exactly. Can you provide a diff?

Comment by Jochen Bayer [ 09/Jun/10 ]

Index: Mssql.php
===================================================================
— Mssql.php (Revision 49)
+++ Mssql.php (Arbeitskopie)
@@ -89,14 +89,14 @@
*/
public function listTableColumns($table)
{

  • $sql = 'EXEC sp_primary_keys_rowset @table_name = ' . $this->conn->quoteIdentifier($table, true);
    + $sql = 'EXEC sp_primary_keys_rowset @table_name = [' . $this->conn->quoteIdentifier($table, true) . ']';
    $result = $this->conn->fetchAssoc($sql);
    $primary = array();
    foreach ($result as $key => $val) { $primary[] = $val['COLUMN_NAME']; }
  • $sql = 'EXEC sp_columns @table_name = ' . $this->conn->quoteIdentifier($table, true);
    + $sql = 'EXEC sp_columns @table_name = [' . $this->conn->quoteIdentifier($table, true) . ']';
    $result = $this->conn->fetchAssoc($sql);
    $columns = array();

@@ -216,10 +216,10 @@
}
}
$table = $this->conn->quote($table, 'text');

  • $query = 'EXEC sp_statistics @table_name = ' . $table;
    + $query = 'EXEC sp_statistics @table_name = [' . $table . ']';
    $indexes = $this->conn->fetchColumn($query, $keyName);
  • $query = 'EXEC sp_pkeys @table_name = ' . $table;
    + $query = 'EXEC sp_pkeys @table_name = [' . $table. ']';
    $pkAll = $this->conn->fetchColumn($query, $pkName);

$result = array();

Comment by Jonathan H. Wage [ 09/Jun/10 ]

You need to turn on identifier quoting. The quoteIdentifier() method takes care of this for you.

http://www.doctrine-project.org/documentation/manual/1_2/en/configuration:identifier-quoting

Comment by Jonathan H. Wage [ 09/Jun/10 ]

Fixed missed calls to quoteIdentifier(). Turn on identifier quoting for identifiers to be wrapped with []

http://www.doctrine-project.org/documentation/manual/1_2/en/configuration:identifier-quoting

Comment by T. B. [ 24/Mar/11 ]

I have a similar problem after executing the symfony (1.4.10) build-schema task:

SQLSTATE[42000]: Syntax error or access violation: 2812 [Microsoft][SQL Server Native Client 10.0][SQL Server]Could not find stored procedure 'sp_primary_keys_rowset'. (SQLExecute[2812] at ext\pdo_odbc\odbc_stmt.c:254). Failing Query: "EXEC sp_primary_keys_rowset @table_name = Appointment"

I hoped ATTR_QUOTE_IDENTIFIER will also solve my problem but I get the sama error message:

SQLSTATE[42000]: Syntax error or access violation: 2812 [Microsoft][SQL Server Native Client 10.0][SQL Server]Could not find stored procedure 'sp_primary_keys_rowset'. (SQLExecute[2812] at ext\pdo_odbc\odbc_stmt.c:254). Failing Query: "EXEC sp_primary_keys_rowset @table_name = [Appointment]"





[DC-435] Doctrine_Table::__constructor() executes initIdentifier() before setUp() Created: 17/Jan/10  Updated: 08/Jun/10  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: Record
Affects Version/s: 1.2.0, 1.2.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Eugene Janusov Assignee: Jonathan H. Wage
Resolution: Can't Fix Votes: 0
Labels: None

Attachments: Text File DC-435.patch     File DC435TestCase.php    

 Description   

Doctrine_Table::__constructor() executes $this->initIdentifier() first and $this->record->setUp() after.
This leads to the incorrect identifier initialization in cases when identifier is defined in a behavioural template.

In the same time, Doctrine_Record_Generator::initialize() does these operations in reverse order:

176         $this->setTableDefinition();
177         $this->setUp();
...
183         $this->_table->initIdentifier();

For me personally, it results in the following message:

Warning: Illegal offset type in /net/eugene.dev/data/ethel/apps/external-doctrine1/1.2/lib/Doctrine/Record.php on line 1892



 Comments   
Comment by Eugene Janusov [ 17/Jan/10 ]

Attached test case.

Comment by Eugene Janusov [ 17/Jan/10 ]

Attached proposed patch.

Comment by Eugene Janusov [ 20/Jan/10 ]

Oh, I forgot to run all the test after applying the patch. It seems my changes terribly broke something else.

Then I don't have a clue how to fix this problem.

Comment by Jonathan H. Wage [ 08/Jun/10 ]

This is an unfortunate limitation. A behavior cannot alter the primary key without problems





[DC-431] findBy magic method doesn't work with many fields specified in the method name Created: 15/Jan/10  Updated: 01/Mar/10  Resolved: 01/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Jerome Calais Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

Using php 5.2.9 on wamp / Windows XP


Attachments: File Table.php    

 Description   

When trying to use 'findBy' magic method this way :

Doctrine::getTable('Article')->findByIsPublishedAndIsModerate(Array(true,true),DOCTRINE::HYDRATE_ARRAY);

It does not work because there is a bug in the __call magic method whereas calling

Doctrine::getTable('Article')->findBy("IsPublishedAndIsModerate",Array(true,true),DOCTRINE::HYDRATE_ARRAY);

works fine

Hydration mode is not set to specified value and the arguments are encapsulated into a non necessary array.

Please find the Table.php attached with the correction to make it work.

line 2771 : '>' operator should be replaced with a '>=' operator
line 2789 : '$arguments' should be replaced by '$arguments[0]' on the line 'return $this->$method($by, $arguments[0], $hydrationMode);'

Thank you for including this patch in the next release.



 Comments   
Comment by Jonathan H. Wage [ 01/Mar/10 ]

The syntax should be:

Doctrine::getTable('Article')->findByIsPublishedAndIsModerate(true,true, DOCTRINE::HYDRATE_ARRAY);




[DC-429] on delete -> files Created: 15/Jan/10  Updated: 08/Jun/10  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major
Reporter: Leandro Vidal Assignee: Jonathan H. Wage
Resolution: Won't Fix Votes: 0
Labels: None
Environment:

all



 Description   

Hello. I'd like doctrine could map a column with a directory in order to put in that column a name of a file and when that row were deleted, the file going to be deleted. For example, if I have a table with the names of images, and a customer with a foreign key to an image, when I delete the costumer with cascade, the image row will be deleted. It would be a great idea if a file named like the name in that table were deleted too. So, we don't need to delete the file manually and we don't need to store the binary file into the database.

I hope my explanation were clear. Sorry for my English






[DC-421] Doctrine_Table::createQuery() can use wrong connection Created: 12/Jan/10  Updated: 07/Dec/10  Resolved: 07/Dec/10

Status: Resolved
Project: Doctrine 1
Component/s: Connection, Query
Affects Version/s: 1.2.0, 1.2.1
Fix Version/s: 1.2.4

Type: Bug Priority: Major
Reporter: Eugene Janusov Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None

Attachments: Text File DC-421.patch     File DC421TestCase.php    

 Description   

r6799 introduced 3 changes to the Doctrine_Table: in findByDql(), getQueryObject(), and createQuery() methods.
The first two are pretty obvious, but the last one is a mistake in my opinion:

--- lib/Doctrine/Table.php	(revision 6798)
+++ lib/Doctrine/Table.php	(revision 6799)
@@ -1033,7 +1033,7 @@
 
         $class = $this->getAttribute(Doctrine_Core::ATTR_QUERY_CLASS);
 
-        return Doctrine_Query::create($this->_conn, $class)
+        return Doctrine_Query::create(null, $class)
             ->from($this->getComponentName() . $alias);
     }

An instance of Doctrine_Table always has a reference to a particular Doctrine_Connection, so why createQuery() method, intended to "create a query on this table" should use different (i.e. default one) connection instead of already specified one?



 Comments   
Comment by Eugene Janusov [ 20/Jan/10 ]

Attached test case.

Comment by Eugene Janusov [ 20/Jan/10 ]

While preparing the test case, I looked more closely to the sources. It turns out that if you do something like this:

Doctrine::getTable('ComponentName')->createQuery()->execute()

then eventually the preferred connection will be used.

But createQuery() method creates a new Doctrine_Query and doesn't pass a connection, thus Doctrine_Query_Abstract's constructor tries to get a default connection, which is unacceptable if you'd like to work without a default connection.

A possible solution that I see is to not set $this>_conn inside Doctrine_Query as long as it will not be really required.-

Comment by Eugene Janusov [ 20/Jan/10 ]

I found another solution. As I said above, the problem is really relevant only if you don't use a default connection. But in this case you must bind all your components to a particular connection(s). So, we can just check for hasConnectionForComponent() inside Doctrine_Table::createQuery().

Comment by Jonathan H. Wage [ 08/Jun/10 ]

Can you provide the fix you found as a patch so I can clearly see it and test it? Thanks, Jon

Comment by Paul Kamer [ 17/Jul/10 ]

Jonathan, I can confirm that this patch solves this issue.
It also fixes this ticked in Symfony 1.X: http://trac.symfony-project.org/ticket/7689

Comment by Jochen Bayer [ 07/Aug/10 ]

DC-421.patch works for me too.

Comment by Jonathan H. Wage [ 07/Dec/10 ]

I've reverted that one change and everything seems better now: https://github.com/doctrine/doctrine1/commit/b4dc8e66a89a7e17cd195c489b18005e19ca9ea5





[DC-416] Issue finding INDEX key name when using 'name: KEY as ALIAS' in schema.yml Created: 10/Jan/10  Updated: 10/Jan/10  Resolved: 10/Jan/10

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Roger Mangraviti Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None
Environment:

Linux Debian, php 5.2.12



 Description   

When trying to create sql database with the below schema

User:
tableName: users
columns:
acctno:
type: integer(4)
primary: true
unsigned: true
notnull: true
autoincrement: true

results in a failed table creation, due to the name found for the INDEX.:

INDEX user_id_idx (user_id)



 Comments   
Comment by Roger Mangraviti [ 10/Jan/10 ]

redundant schema/yml file in schema dir.





[DC-415] HYDRATE_ARRAY_HIERARCHY does not return array (returns Doctrine_Collection) Created: 10/Jan/10  Updated: 14/Jan/10  Resolved: 14/Jan/10

Status: Resolved
Project: Doctrine 1
Component/s: Nested Set
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Anush Ramani Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

Vista Ultimate / Apache 2.2.8 / PHP5.2.4 / Symfony 1.4


Attachments: File 010-category.yml    

 Description   

When hydration type is specified as HYDRATE_ARRAY_HIERARCHY, the result is a Doctrine_Collection rather than an array.

Unable to find source-code formatter for language: php. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
// The Category table is a typical NestedSet of shopping categories (definition below)
$categories = Doctrine_Core::getTable('Category')
  ->findByDql('name LIKE ? OR name LIKE ?', array("$q%", "% $q%"), Doctrine_Core::HYDRATE_RECORD_HIERARCHY);

// This should return bool(true), but returns bool(false)
echo var_dump(is_array($categories));

// This should return null, but returns Doctrine_Collection
echo get_class($categories);
Unable to find source-code formatter for language: yaml. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
# the category table definition (see attachment for data)
Category:
  actAs:
    NestedSet: { hasManyRoots: true, rootColumnName: root_id }
  columns:
    category_id: { type: integer, unsigned: true, primary: true, autoincrement: true }
    name: { type: string(100), notnull: true }
    description: clob

Note that this issue was opened for 1.2.0-BETA2 (DC-237) but was resolved as unable to reproduce. However, I can confirm that this issue still exists.



 Comments   
Comment by Anush Ramani [ 14/Jan/10 ]

I feel a bit silly... HYDRATE_ARRAY_HIERARCHY works as expected. I was using HYDRATE_RECORD_HIERARCHY.





[DC-406] more robust handling for "actAs: [Timstampable]" handling Created: 07/Jan/10  Updated: 12/Jan/10  Resolved: 12/Jan/10

Status: Resolved
Project: Doctrine 1
Component/s: Behaviors
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Major
Reporter: Lukas Kahwe Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None

Attachments: File Builder.php.diff    

 Description   

see: http://trac.symfony-project.org/ticket/8044

This patch ensures that any numeric keys are flipped individually instead of simply peaking into the first element and assuming the rest of the array has the same structure.






[DC-410] Email-validator doesn't work Created: 08/Jan/10  Updated: 02/Mar/10  Resolved: 02/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Validators
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Major
Reporter: Sander Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

PostgreSQL 8.4, PHP 5.2.11, Apache2


Attachments: File BaseAppUser.php     File schema.yml    

 Description   

I'm generating my models from YAML. The email validator is correctly parsed into the model, but it doesn't work. If I try to insert 'a' or something that is not valid, I don't get any error at all and the row is inserted. See attachments for models.

$user = new AppUser();
$user->name = 'Testuser';
$user->password = sha1('lala');
$user->email = 'a';
$user->role = 'ADMIN';
$user->save();



 Comments   
Comment by Benjamin Steininger [ 17/Jan/10 ]

Is Validation activated for your Connection/Manager via the attributes ? It's off by default.

From the manual:
http://www.doctrine-project.org/documentation/manual/1_2/en/configuration#naming-convention-attributes:validation-mode-constants

Validation by default is turned off so if you wish for your data to be validated you will need to enable it. Some examples of how to change this configuration are provided below.

Comment by Sander [ 17/Jan/10 ]

Ah crap, I feel so stupid. My bad......





[DC-395] sfYaml library included twice Created: 05/Jan/10  Updated: 08/Jun/10  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Major
Reporter: Karma Dordrak (Drak) Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None


 Description   

I don't know if this is important but it seems sfYaml is included twice in Doctrine 1.2.1. Once in vendor/sfYaml and once again in Doctrine/Parser/sfYaml



 Comments   
Comment by Benjamin Eberlei [ 16/Feb/10 ]

This should be fixed asap, it makes running the test-suite (with coverage) impossible.





[DC-391] Connection rollback exception hides real exception Created: 01/Jan/10  Updated: 08/Jun/10  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: Connection
Affects Version/s: 1.1.6
Fix Version/s: 1.2.3

Type: Bug Priority: Major
Reporter: Alessandro Vermeulen Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None


 Description   

Same issue as http://www.doctrine-project.org/jira/browse/DC-207 but also for the 1.1 branch.

This is not only an issue when trying to talk to non-existing databases but for any exception thrown by the connection.

UnitOfWork.php:133

        } catch (Exception $e) {
            // Make sure we roll back our internal transaction
            //$record->state($state);
            $conn->rollback();
            throw $e;
        }


 Comments   
Comment by Jonathan H. Wage [ 08/Jun/10 ]

This is already fixed in Doctrine 1.2





[DC-369] Problems hydrating I18n data (while processing ebedded forms in symfony) Created: 18/Dec/09  Updated: 15/Mar/10  Resolved: 15/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: I18n
Affects Version/s: 1.0.12, 1.0.13, 1.0.14
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Christian Seaman Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

Symfony-1.2.7, Doctrine-1.0.9



 Description   

OK, I've spent several hours trying to track down what's happening.... I hope you can understand the problem as I define it below!

The situation: We recently upgraded a project from Symfony-1.2.7 to 1.2.10. It seemed that all was working, however our tests didn't check the I18n embedded forms in the backend (something we're working on changing, but that's another story).

The problem: We have some I18n forms embedded in a parent form. With symfony-1.2.7 and Doctrine-1.0.8 everything works perfectly (as far as we can tell). However, when we submit the form now we get the error:

Couldn't hydrate. Found non-unique key mapping named 'lang'.

After much searching and trying different symfony and Doctrine versions, we finally found that the cause of the problem is the changes made to /lib/Doctrine/Hydrator.php in r5629 (relating to the old doctrine trac ticket #1991). The specific lines look like this in the new version (as of Doctrine-1.0.9):

if ( ! isset($element[$field])) {
  throw new Doctrine_Hydrator_Exception("Couldn't hydrate. Found a non-existent key named '$field'.");
} else if (isset($result[$element[$field]])) {
  throw new Doctrine_Hydrator_Exception("Couldn't hydrate. Found non-unique key mapping named '$field'.");
}

And like this in the old version (as in Doctrine-1.0.8):

if (isset($result[$field])) {
  throw new Doctrine_Hydrator_Exception("Couldn't hydrate. Found non-unique key mapping named '$field'.");
} else if ( ! isset($element[$field])) {
  throw new Doctrine_Hydrator_Exception("Couldn't hydrate. Found a non-existent key named '$field'.");
}

Note that the difference (other than the order of the checks) is that the new code checks $result[$element[$field]] whereas the old code checks $result[$field] (i.e. checks for $result['en_GB'] instead of $result['lang'].

On further inspection we find that the state of things just before the error looks like this:

$field => 'lang';

$element->toArray() => Array 
(
  [id] => 2
  [filename] => file2.gif
  [lang] => en_GB
)

$result->toArray() => Array 
(
  [en_GB] => Array
    (
      [id] => 1
      [filename] => file1.jpg
      [lang] => en_GB
    )
)

Now this is where I get a little lost, since I'm not completely au fait with the internal workings of Doctrine - why is this check done? What is the implication if it fails? In this case, lang is not a unique key for the model (it should be id+lang) so either the key-generator or the check-logic would seem to be incorrect.

However, I do know that changing this code back to what it was before both (a) allows these forms to submit and (b) direct inspection of the database before and after such a submit shows that things were saved as expected with no side-effects.

So - over to you. Somebody who knows the reason for these checks and understands the implications of their failure needs to decide (a) if the changes made in r5629 are correct, (b) if the checks here are too onerous (since they are failing a procedure that otherwise seems to work in our case) and (c) if any other changes are needed.

Other than raising this issue up to the visibility of those far wiser than myself, I also humbly submit a tiny fix - the error message in the new version has not been updated along with the code. I think it should be:

throw new Doctrine_Hydrator_Exception("Couldn't hydrate. Found non-unique key mapping named '{$element[$field]}'.");

Instead of:

throw new Doctrine_Hydrator_Exception("Couldn't hydrate. Found non-unique key mapping named '$field]'.");

I look forward to your response and, as ever, am willing to help in any way that you think I can.

Yours,

Christian



 Comments   
Comment by Jonathan H. Wage [ 02/Mar/10 ]

Can you provide your changes as a patch?

Comment by Jonathan H. Wage [ 15/Mar/10 ]

I committed a fix for the error message. As for your other questions, the changes you are questioning are correct and necessary. You can't have a result set with a nonunique value for the key mapping. Otherwise it would just override the previously value and you would never know that it is happening.





[DC-360] setColumnOption[s] doesn't actually set column options Created: 15/Dec/09  Updated: 08/Jun/10  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: Record
Affects Version/s: 1.2.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Travis Kroh Assignee: Jonathan H. Wage
Resolution: Cannot Reproduce Votes: 0
Labels: None
Environment:

PHP 5.2.6



 Description   

class User extends BaseUser
{
/* Based on example from Doctrine_Record_Abstract line 295-302:

  • public function setTableDefinition()
  • { * parent::setTableDefinition(); * $this->setColumnOptions('username', array( * 'unique' => true * )); * }

    */
    public function setTableDefinition()

    { parent::setTableDefinition(); $this->setColumnOptions('emailAddress', array('email' => true)); /* Neither of these enable the validation! * $this->setColumnOption('emailAddress', 'email', true); * $this->setColumnOptions('emailAddress', array('email' => true)); */ }

    }

$george = Doctrine_Core::getTable('User')->find('974');

// Does not result in error.
$george->setEmailAddress('This aint an email address');



 Comments   
Comment by Jonathan H. Wage [ 08/Jun/10 ]

This is working as expected. Do you have the validation attribute enabled?





[DC-349] Issue with quoting of booleans in Oracle Created: 10/Dec/09  Updated: 08/Jun/10  Resolved: 08/Jun/10

Status: Resolved
Project: Doctrine 1
Component/s: Connection
Affects Version/s: 1.0.14
Fix Version/s: 1.2.3

Type: Bug Priority: Major
Reporter: David Brewer Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None
Environment:

Oracle Driver



 Description   

I've run into a problem where I'm unable to set a default value for a boolean field when creating Oracle tables using Doctrine. The summary of this issue is that when Doctrine quotes a '0' or false value for a boolean field, it comes back blank... which doesn't work as an option for the DEFAULT parameter of field definitions in table creation.

Here are the hairy details. I have a schema which looks something like this (simplified for the example):

User:

  tableName: users

  columns:

    id: {type: integer, primary: true, autoincrement: true}

    username: {type: string, length: 255, unique: true, notnull: true}

    disabled: {type: boolean, default: false}

When I export the table, it's generating invalid DDL like this:

CREATE TABLE users (id NUMBER(8), username VARCHAR2(255) NOT NULL UNIQUE, disabled NUMBER(1) DEFAULT , PRIMARY KEY(id))

The important part of this is the DEFAULT parameter on the 'disabled' field definition – it's missing the value. I tracked all this down to an issue in Doctrine_Formatter->quote(), which gets called when the DDL is being created. When passed '0' as its input with a type of boolean, it apparently returns the empty string. This may ultimately be a bug in the PDO_OCI driver, because the quote method is relying on the quote method of the database handle.

I've hacked around this by overriding the behaviour of the quote method in Doctrine_Connection_Oracle. I did this by adding the following method:

    /**
     * Override quote behaviour for boolean to fix issues with quoting of
     * boolean values.
     */
    public function quote($input, $type = null)
    {
        if ($type === 'boolean') {
            if ($input === null) {
                return null;
            } else {
                return (($input) ? 1 : 0);    
            }
      } else {
        return parent::quote($input, $type);  
      }
    }

It seems to work for me, but I'd appreciate it if wiser heads than me would take a look and see if this is a sign of some larger issue, and if the way I've fixed it seems appropriate. I also am unsure of how/if this needs to propogate to newer versions.






[DC-345] delete reference between OneToOne-Relation with synchronizeWithArray(), patch available Created: 09/Dec/09  Updated: 11/Mar/10  Resolved: 11/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Record, Relations
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Improvement Priority: Major
Reporter: Marcus Häußler Assignee: Jonathan H. Wage
Resolution: Incomplete Votes: 0
Labels: None


 Description   

There is no possibility to delete a reference on a OneToOne-Relation with synchronizeWithArray().

If User and Group would be OneToOne related, i would expect that I could do something like the following (like on ManyToMany-Relations), to delete the relation between both.

$user->synchronizeWithArray(array(
    'Group' => array() // no error, but the relation is not deleted
));

or

$user->synchronizeWithArray(array(
    'Group' => array(null) // error, cause doctrine expects an existing ID
));

This fixes the problem:

Index: library/Doctrine/Record.php
===================================================================
--- library/Doctrine/Record.php	(revision 6903)
+++ library/Doctrine/Record.php	(working copy)
@@ -2012,6 +2012,8 @@
                     if (isset($value[0]) && ! is_array($value[0])) {
                         $this->unlink($key, array(), false);
                         $this->link($key, $value, false);
+                    } else if (empty($value) || (null === current($value))) {
+                    	$this->unlink($key, array(), false);
                     } else {
                         $this->$key->synchronizeWithArray($value);
                     }


 Comments   
Comment by Jonathan H. Wage [ 01/Mar/10 ]

Hmm. The second syntax is weird. I think the first syntax makes more sense, no?





[DC-346] synchronize records based on the primary keys, patch available Created: 09/Dec/09  Updated: 11/Mar/10  Resolved: 11/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Record, Relations
Affects Version/s: 1.2.1
Fix Version/s: 1.2.2

Type: Improvement Priority: Major
Reporter: Marcus Häußler Assignee: Jonathan H. Wage
Resolution: Incomplete Votes: 0
Labels: None


 Description   

If I do a synchronizeWithArray(), the order of the elements in the array has to be the same as they are saved in the database.
Of course this can not be assumed in any situation, where you have reordered form-elements etc.

I would expect, if I add the IDs to the elements in the array, that the update-process is based on them and not on the element-order.

Please take a look at this old ticket: http://trac.doctrine-project.org/ticket/2097

The attached patch works really nice on the actual revision too.



 Comments   
Comment by Jonathan H. Wage [ 01/Mar/10 ]

Hi, have you tested this patch against Doctrine 1.2 and ran the test suite? When I give it a try, it wouldn't quite apply cleanly and it seems to break lots of tests. I'll keep looking at it, let me know if you have any more information or if you have another version of the patch. Thanks, Jon





[DC-341] Doctrine_Cache_Driver::save($i, $d, $l, $s) expects $d to be a string, Doctrine_Cache_Driver::_saveKey($key), Doctrine_Cache_Driver::_deleteKey($key) do not serialize before saving Created: 08/Dec/09  Updated: 08/Dec/09  Resolved: 08/Dec/09

Status: Resolved
Project: Doctrine 1
Component/s: Caching
Affects Version/s: 1.2.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Andreas Möller Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

WAMP (Windows XP, Apache 2.2.11, MySQL 5.0.51a, PHP 5.2.8)



 Description   

I've sub-classed

Doctrine_Cache_Driver

in order to attach it to a

Zend_Cache_Backend_File

for the time being, but I'm a bit confused as in

Doctrine_Cache_Driver::_saveKey($key)
Doctrine_Cache_Driver::_deleteKey($key)

with either of them attempting to call

Doctrine_Cache_Driver::save($this->_cacheKeyIndexKey, $keys, 
null, false);

the variable

$keys

is passed as an array, although

Doctrine_Cache_Driver::save($id, $data, $lifeTime, $saveKey)

expects

$data

i.e.,

$keys

to be a string (and thus, as Zend_Cache_Frontend has been instantiated
with

automatic_serialization

turned off, an exception is being thrown because it is not.



 Comments   
Comment by Andreas Möller [ 08/Dec/09 ]

Overriding the methods

Doctrine_Cache_Driver::_saveKey($key)
Doctrine_Cache_Driver::_deleteKey($key)

as following fixed it.

    /**
     * Save a cache key in the index of cache keys
     *
     * @param string $key
     * @return boolean True if successful and false if something went wrong.
     */
    protected function _saveKey($key)
    {
        $keys = unserialize($this->fetch($this->_cacheKeyIndexKey));
        $keys[] = $key;
        $keys = serialize($keys);

        return $this->save($this->_cacheKeyIndexKey, $keys, null, false);
    }

    /**
     * Delete a cache key from the index of cache keys
     *
     * @param string $key
     * @return boolean True if successful and false if something went wrong.
     */
    public function _deleteKey($key)
    {
        $keys = unserialize($this->fetch($this->_cacheKeyIndexKey));
        $key = array_search($key, $keys);
        if ($key !== false) {
            unset($keys[$key]);
			$keys = serialize($keys);
            return $this->save($this->_cacheKeyIndexKey, $keys, null, false);
        }

        return false;
    }    
Comment by Jonathan H. Wage [ 08/Dec/09 ]

This is expected. The driver you implement is responsible for storing the PHP data properly. Whether that be you serialize it or or do whatever to store it.





[DC-336] Regression: Query Fails in Doctrine 1.1.6, Works in 1.1.1 Created: 08/Dec/09  Updated: 08/Dec/09  Resolved: 08/Dec/09

Status: Resolved
Project: Doctrine 1
Component/s: Query
Affects Version/s: 1.1.6
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Josh Boyd Assignee: Guilherme Blanco
Resolution: Fixed Votes: 0
Labels: None
Environment:

Doctrine 1.1.6
mysql 5.0.75
PHP 5.2.6
Ubuntu 9.04



 Description   

Recently upgraded project to 1.1.6 from 1.1.1. I get the following error with my somewhat complex query, which worked fine in 1.1.1:

! ) Fatal error: Maximum function nesting level of '100' reached, aborting! in /home/josh/src/openfisma/trunk/library/Doctrine/Query/Tokenizer.php on line 878
Call Stack
#	Time	Memory	Function	Location
1	0.0005	67088	{main}( )	../index.php:0
2	0.0646	1843400	Fisma::dispatch( )	../index.php:31
3	0.1153	6508224	Zend_Controller_Front->dispatch( )	../Fisma.php:349
4	0.2990	14344680	Zend_Controller_Dispatcher_Standard->dispatch( )	../Front.php:946
5	0.3220	14861596	Zend_Controller_Action->dispatch( )	../Standard.php:289
6	0.3223	14866776	ReportController->overdueAction( )	../Action.php:513
7	0.3756	15978796	Doctrine_Query_Abstract->execute( )	../ReportController.php:275
8	0.3756	15978796	Doctrine_Query_Abstract->_preQuery( )	../Abstract.php:1119
9	0.3757	15978876	Doctrine_Query_Abstract->_getDqlCallbackComponents( )	../Abstract.php:1217
10	0.3758	15981824	Doctrine_Query->getSqlQuery( )	../Abstract.php:1255
11	0.3865	16461108	Doctrine_Query_Abstract->_processDqlQueryPart( )	../Query.php:1180
12	0.3903	16613300	Doctrine_Query_Select->parse( )	../Abstract.php:2269
13	0.3903	16613300	Doctrine_Query->parseSelect( )	../Select.php:37
14	0.3937	16626436	Doctrine_Query->parseClause( )	../Query.php:668
15	0.3954	16644676	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
16	0.3956	16645928	Doctrine_Query->parseClause( )	../Query.php:862
17	0.3968	16646852	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
18	0.3968	16646852	Doctrine_Query->parseSubquery( )	../Query.php:854
19	0.3968	16647084	Doctrine_Query->parseClause( )	../Query.php:888
20	0.3977	16648700	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
21	0.3977	16648700	Doctrine_Query->parseSubquery( )	../Query.php:854
22	0.3978	16648768	Doctrine_Query->parseClause( )	../Query.php:888
23	0.3979	16649844	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
24	0.3979	16649844	Doctrine_Query->parseSubquery( )	../Query.php:854
25	0.3979	16649912	Doctrine_Query->parseClause( )	../Query.php:888
26	0.3980	16650804	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
27	0.3980	16650804	Doctrine_Query->parseSubquery( )	../Query.php:854
28	0.3980	16650872	Doctrine_Query->parseClause( )	../Query.php:888
29	0.3981	16651764	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
30	0.3981	16651764	Doctrine_Query->parseSubquery( )	../Query.php:854
31	0.3981	16651832	Doctrine_Query->parseClause( )	../Query.php:888
32	0.3982	16652668	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
33	0.3982	16652668	Doctrine_Query->parseSubquery( )	../Query.php:854
34	0.3983	16652736	Doctrine_Query->parseClause( )	../Query.php:888
35	0.3984	16653660	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
36	0.3984	16653660	Doctrine_Query->parseSubquery( )	../Query.php:854
37	0.3984	16653784	Doctrine_Query->parseClause( )	../Query.php:888
38	0.3985	16655012	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
39	0.3985	16655012	Doctrine_Query->parseSubquery( )	../Query.php:854
40	0.3985	16655136	Doctrine_Query->parseClause( )	../Query.php:888
41	0.3986	16656364	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
42	0.3986	16656364	Doctrine_Query->parseSubquery( )	../Query.php:854
43	0.3986	16656488	Doctrine_Query->parseClause( )	../Query.php:888
44	0.3987	16658264	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
45	0.3987	16658264	Doctrine_Query->parseSubquery( )	../Query.php:854
46	0.3988	16658388	Doctrine_Query->parseClause( )	../Query.php:888
47	0.3990	16659616	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
48	0.3990	16659616	Doctrine_Query->parseSubquery( )	../Query.php:854
49	0.3990	16659740	Doctrine_Query->parseClause( )	../Query.php:888
50	0.3991	16660968	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
51	0.3991	16660968	Doctrine_Query->parseSubquery( )	../Query.php:854
52	0.3992	16661092	Doctrine_Query->parseClause( )	../Query.php:888
53	0.3993	16662320	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
54	0.3993	16662320	Doctrine_Query->parseSubquery( )	../Query.php:854
55	0.3993	16662444	Doctrine_Query->parseClause( )	../Query.php:888
56	0.3995	16663672	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
57	0.3995	16663672	Doctrine_Query->parseSubquery( )	../Query.php:854
58	0.3995	16663796	Doctrine_Query->parseClause( )	../Query.php:888
59	0.3996	16665024	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
60	0.3996	16665024	Doctrine_Query->parseSubquery( )	../Query.php:854
61	0.3996	16665148	Doctrine_Query->parseClause( )	../Query.php:888
62	0.3997	16666376	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
63	0.3997	16666376	Doctrine_Query->parseSubquery( )	../Query.php:854
64	0.3997	16666500	Doctrine_Query->parseClause( )	../Query.php:888
65	0.3998	16667728	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
66	0.3999	16667728	Doctrine_Query->parseSubquery( )	../Query.php:854
67	0.3999	16667852	Doctrine_Query->parseClause( )	../Query.php:888
68	0.4000	16669080	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
69	0.4000	16669080	Doctrine_Query->parseSubquery( )	../Query.php:854
70	0.4000	16669204	Doctrine_Query->parseClause( )	../Query.php:888
71	0.4001	16670432	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
72	0.4001	16670432	Doctrine_Query->parseSubquery( )	../Query.php:854
73	0.4001	16670556	Doctrine_Query->parseClause( )	../Query.php:888
74	0.4007	16672224	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
75	0.4007	16672224	Doctrine_Query->parseSubquery( )	../Query.php:854
76	0.4007	16672348	Doctrine_Query->parseClause( )	../Query.php:888
77	0.4008	16674016	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
78	0.4008	16674016	Doctrine_Query->parseSubquery( )	../Query.php:854
79	0.4008	16674140	Doctrine_Query->parseClause( )	../Query.php:888
80	0.4009	16675808	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
81	0.4010	16675808	Doctrine_Query->parseSubquery( )	../Query.php:854
82	0.4010	16675932	Doctrine_Query->parseClause( )	../Query.php:888
83	0.4011	16677600	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
84	0.4011	16677600	Doctrine_Query->parseSubquery( )	../Query.php:854
85	0.4011	16677724	Doctrine_Query->parseClause( )	../Query.php:888
86	0.4012	16680440	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
87	0.4012	16680440	Doctrine_Query->parseSubquery( )	../Query.php:854
88	0.4012	16680564	Doctrine_Query->parseClause( )	../Query.php:888
89	0.4013	16683324	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
90	0.4013	16683324	Doctrine_Query->parseSubquery( )	../Query.php:854
91	0.4014	16683448	Doctrine_Query->parseClause( )	../Query.php:888
92	0.4015	16685156	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
93	0.4015	16685156	Doctrine_Query->parseSubquery( )	../Query.php:854
94	0.4015	16685280	Doctrine_Query->parseClause( )	../Query.php:888
95	0.4016	16686948	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
96	0.4016	16686948	Doctrine_Query->parseSubquery( )	../Query.php:854
97	0.4016	16687072	Doctrine_Query->parseClause( )	../Query.php:888
98	0.4017	16688740	Doctrine_Query->parseFunctionExpression( )	../Query.php:734
99	0.4017	16688740	Doctrine_Query->parseSubquery( )	../Query.php:854

Here's my query:

        if ('search' == $req->getParam('s') || isset($isExport)) {
            $q = Doctrine_Query::create()
                 ->select('f.id') // unused, but Doctrine requires a field to be selected from the parent object
                 ->addSelect("CONCAT_WS(' - ', o.nickname, o.name) orgSystemName")
                 ->addSelect(
                     "QUOTE(
                         IF (f.status IN ('NEW', 'DRAFT', 'MSA'), 
                             'Mitigation Strategy', 
                             IF (f.status IN ('EN', 'EA'), 
                                'Corrective Action', 
                                NULL
                             )
                         )
                      ) actionType"
                 )
                 ->addSelect('SUM(IF(DATEDIFF(NOW(), f.nextduedate) BETWEEN 0 AND 29, 1, 0)) lessThan30')
                 ->addSelect('SUM(IF(DATEDIFF(NOW(), f.nextduedate) BETWEEN 30 AND 59, 1, 0)) moreThan30')
                 ->addSelect('SUM(IF(DATEDIFF(NOW(), f.nextduedate) BETWEEN 60 AND 89, 1, 0)) moreThan60')
                 ->addSelect('SUM(IF(DATEDIFF(NOW(), f.nextduedate) BETWEEN 90 AND 119, 1, 0)) moreThan90')
                 ->addSelect('SUM(IF(DATEDIFF(NOW(), f.nextduedate) >= 120, 1, 0)) moreThan120')
                 ->addSelect('COUNT(f.id) total')
                 ->addSelect('ROUND(AVG(DATEDIFF(NOW(), f.nextduedate))) average')
                 ->addSelect('MAX(DATEDIFF(NOW(), f.nextduedate)) max')
                 ->from('Finding f')
                 ->leftJoin('f.ResponsibleOrganization o')
                 ->where('f.nextduedate < NOW()');

            if (!empty($params['orgSystemId'])) {
                $q->andWhere('f.responsibleOrganizationId = ?', $params['orgSystemId']);
            }
            if (!empty($params['sourceId'])) {
                $q->andWhere('f.sourceId = ?', $params['sourceId']);
            }
            if ($params['overdueType'] == 'sso') {
                $q->whereIn('f.status', array('NEW', 'DRAFT', 'MSA'));
            } elseif ($params['overdueType'] == 'action') {
                $q->whereIn('f.status', array('EN', 'EA'));
            } else {
                $q->whereIn('f.status', array('NEW', 'DRAFT', 'MSA', 'EN', 'EA'));
            }

            $q->groupBy('orgSystemName, actionType');
            $q->setHydrationMode(Doctrine::HYDRATE_ARRAY);
            $list = $q->execute();


 Comments   
Comment by Josh Boyd [ 08/Dec/09 ]

This seemed to be a spacing issue that was causing the tokenizer to freak out...

--- trunk/application/controllers/ReportController.php  Tue Dec  8 17:46:25 2009        (r2702)
+++ trunk/application/controllers/ReportController.php  Tue Dec  8 18:22:30 2009        (r2703)
@@ -234,15 +234,8 @@
                 ->select('f.id') // unused, but Doctrine requires a field to be selected from the parent object
                 ->addSelect("CONCAT_WS(' - ', o.nickname, o.name) orgSystemName")
                 ->addSelect(
-                     "QUOTE(
-                         IF (f.status IN ('NEW', 'DRAFT', 'MSA'),
-                             'Mitigation Strategy',
-                             IF (f.status IN ('EN', 'EA'),
-                                'Corrective Action',
-                                NULL
-                             )
-                         )
-                      ) actionType"
+                     "QUOTE(IF(f.status IN ('NEW', 'DRAFT', 'MSA'), 'Mitigation Strategy', IF(f.status IN ('EN', 'EA'),
+                     'Corrective Action', NULL))) actionType"
                 )
                 ->addSelect('SUM(IF(DATEDIFF(NOW(), f.nextduedate) BETWEEN 0 AND 29, 1, 0)) lessThan30')
                 ->addSelect('SUM(IF(DATEDIFF(NOW(), f.nextduedate) BETWEEN 30 AND 59, 1, 0)) moreThan30')




[DC-327] CLONE -Doctrine_Migration_Diff breaks on inherited class Created: 04/Dec/09  Updated: 04/Dec/09  Resolved: 04/Dec/09

Status: Resolved
Project: Doctrine 1
Component/s: Migrations
Affects Version/s: 1.2.0-ALPHA1, 1.2.0-ALPHA2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Tom Boutell Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

php 5.3 / symfony 1.3



 Description   

This ticket is a duplicate of this one http://trac.symfony-project.org/ticket/7272 but it's more a Doctrine bug than a symfony one.

When trying to make a diff on schema with inheritance, the task breaks when trying to load a model inherited from an other one. I think this is because the inherited model might be loaded before his ancestor.






[DC-319] getModified() returns object where ids are expected Created: 03/Dec/09  Updated: 02/Mar/10  Resolved: 02/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: Record, Relations
Affects Version/s: 1.1.7, 1.2.1
Fix Version/s: 1.2.2

Type: Bug Priority: Major
Reporter: Emil Vladev Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

PHP 5.2.10 on Ubuntu 9.10



 Description   

Assigning a new unsaved relation to an entity confuses getModified() to return records for values such as userId, that are expected to be ints.

Here is a full example to reproduce:

<?php
require_once '_loader.php';

$mgr = Doctrine_Manager::getInstance();
$mgr->openConnection('....');

class User extends Doctrine_Record
{
    public function setTableDefinition()
    {
        $this->setTableName('t_users');

        $this->hasColumn('name', 'string', 32);
    }
}

class Post extends Doctrine_Record
{
    public function setTableDefinition()
    {
        $this->setTableName('t_posts');

        $this->hasColumn('body', 'string', 256);
        $this->hasColumn('userId', 'integer');
    }

    public function setUp()
    {
        $this->hasOne('User', array(
            'local'   => 'userId',
            'foreign' => 'id'
        ));
    }
}

Doctrine::dropDatabases();
Doctrine::createDatabases();
Doctrine::createTablesFromArray(array('User', 'Post'));

$joe = new User();
$joe->name = 'Joe';
$joe->save();

$post = new Post();
$post->body = 'Hi';
$post->User = $joe;
$post->save();

$bill = new User();
$bill->name = 'Bill';
//$bill->save(); // If this is uncommented everything works fine

$post->User = $bill;

var_dump(' --- New --- ');
var_dump($post->getModified());

And instead of a id for userId I get this

string(13) " --- New --- "
array(1) {
  ["userId"]=>
  object(User)#37 (18) {
    ["_node:protected"]=>
    NULL
    ["_id:protected"]=>
    array(0) {
    }
    ["_data:protected"]=>
    array(2) {
      ["id"]=>
      object(Doctrine_Null)#4 (0) {
      }
      ["name"]=>
      string(4) "Bill"
    }
    ["_values:protected"]=>
    array(0) {
    }
    ["_state:protected"]=>
    int(2)
    ["_lastModified:protected"]=>
    array(0) {
    }
    ["_modified:protected"]=>
    array(1) {
      [0]=>
      string(4) "name"
    }
    ["_oldValues:protected"]=>
    array(1) {
      ["name"]=>
      NULL
    }
    ... trimmed ...

expected

string(13) " --- New --- "
array(1) {
  ["userId"]=>
  string(1) "2"
}

Tested on 1.1 and 1.2 branches from svn. (same result)



 Comments   
Comment by Jonathan H. Wage [ 02/Mar/10 ]

This is the expected behavior. Because you haven't saved $bill yet, it is temporarily storing the reference to the object in the foreign key field.

Comment by Emil Vladev [ 02/Mar/10 ]

But sholnd't it be NULL. userId is expected to be an int - not an object - doesn't feel right?!?

Comment by Jonathan H. Wage [ 02/Mar/10 ]

Hmm. I guess it could go either way. If you haven't saved yet, then the property has changed, we just don't know the id yet. I could imagine someone wanting to know that an object was assigned to that foreign key but not saved yet. Either way, I don't think we can change the behavior in Doctrine 1.2 if people might be relying on it already.





[DC-311] Doctrine_Data do not properly detect relations Created: 02/Dec/09  Updated: 15/Mar/10  Resolved: 15/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: 1.1.6
Fix Version/s: 1.1.7

Type: Bug Priority: Major
Reporter: G. Perréal Assignee: Jonathan H. Wage
Resolution: Incomplete Votes: 0
Labels: None
Environment:

Debian Lenny


Attachments: Zip Archive dtest.zip    

 Description   

When using lowerCamelCase for column names, Doctrine::dumpData fails to detect relation fields, and thus outputs raw identifier values instead of reference list in YAML format. This cause the loading to fails for any table using autoincrement, as the new identifier will rarely matchs the dumped one.

This seems related to the way relations are testing in Doctrine_Data_Export#prepareData and Doctrine_Data#isRelation: the former calls the latter passing record and field name (as returned by $record->toArray(false)). Doctrine_Data#isRelation looks for the field name in relation definitions, which contain column names.

The attachment contains a sample YAML schema and a PHP script to test the behavior.



 Comments   
Comment by Jonathan H. Wage [ 07/Dec/09 ]

Can you prepare this as a Doctrine test case? Thanks.





[DC-310] unlink() memory leak Created: 02/Dec/09  Updated: 04/Dec/09  Resolved: 04/Dec/09

Status: Resolved
Project: Doctrine 1
Component/s: Record
Affects Version/s: 1.2.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Nicholas Clark Assignee: Jonathan H. Wage
Resolution: Won't Fix Votes: 0
Labels: None
Environment:

PHP 5.2.11, Doctrine 1.2



 Description   

Calling unlink() and then save() does not appear to completely free the unlinked object or leaves some artifact behind that does not get garbage collected. When performing this task thousands of times, I eventually run out of memory. If I simply do a delete(), there is no memory leak.

For example, this appears to cause a memory leak:

 

large loop {
    // Simple one to many relationship
    $user->unlink('Groups', array(<id>));
    ...
    $user-save();
    $user->free(true);
}

This does not cause a memory leak, but it's not the desired way to code this operation:

 

large loop {
    $user->Groups[<index>]->delete();
    ...
    $user->free(true);
}

I did my best to track the probably down, but I wasn't able to find the cause. I thought it might be something wrong with UnitOfWork->_executeDeletions() but it appears to remove the back links (I think). I was getting in over my head, so I thought I would submit a bug report. If you need any other information, please let me know. I can easily reproduce the problem with my current models and data.

-Nick

UPDATE: I said there was no memory leak when I call $obj->delete(), but that doesn't appear to be true. It still eats up memory but at a much slower pace. Here is an example:

 

$q = Doctrine_Query::create()->from('User');
$result = $q->execute(array(), Doctrine_Core::HYDRATE_ON_DEMAND);
foreach ($result as $user) {
    $user->delete();
    $user->free(true);
}

If this record has a simple one-to-many relationship to, say, UserGroup and I run this loop for thousands of records, I run out of memory.



 Comments   
Comment by Nicholas Clark [ 03/Dec/09 ]

Adding more information.

Comment by Juozas Kaziukenas [ 03/Dec/09 ]

This can be caused by 5.2, because on my 5.3 servers memory growth ratio is very low.

You can also try to run Doctrine_Manager::connection->clear(), since all objects you fetch from DB are stored in identity map and calling free() doesn't remove them.

Comment by Nicholas Clark [ 04/Dec/09 ]

Calling Doctrine_Manager::connection()->clear() does help keep memory usage under control in most situations, so I'll close this ticket. It sounds like upgrading to PHP 5.3 is my best bet at keeping memory usage under control.





[DC-303] Inflector tableize and classify inconsistencies Created: 01/Dec/09  Updated: 02/Dec/09  Resolved: 01/Dec/09

Status: Resolved
Project: Doctrine 1
Component/s: Import/Export
Affects Version/s: 1.1.6
Fix Version/s: None

Type: Bug Priority: Major
Reporter: G. Perréal Assignee: Jonathan H. Wage
Resolution: Won't Fix Votes: 0
Labels: None
Environment:

linux & windows



 Description   

While tracking down a bug where dumpData does not correctly output relations, it appeared it is related to the way Doctrine computes the attributes/field names.

It seems that the Doctrine_Inflector::classify methods give incorrect results with some strings, that could appear in schemas. I ran them with simple strings to test it:

Doctrine_Inflector::tableize("table_name") => "table_name" OK
Doctrine_Inflector::tableize("TableName") => "table_name" OK
Doctrine_Inflector::tableize("table_name_id") => "table_name_id" OK
Doctrine_Inflector::tableize("TableName_id") => "table_name_id" ??? (not sure what the expected result should be).

Doctrine_Inflector::classify("table_name") => "TableName" OK
Doctrine_Inflector::classify("TableName") => "Tablename" FAIL
Doctrine_Inflector::classify("table_name_id") => "TableNameId" OK
Doctrine_Inflector::classify("TableName_id") => "TablenameId" FAIL

Now, if you combine them:

Doctrine_Inflector::classify(Doctrine_Inflector::tableize("table_name")) => "TableName" OK
Doctrine_Inflector::classify(Doctrine_Inflector::tableize("TableName")) => "TableName" OK
Doctrine_Inflector::classify(Doctrine_Inflector::tableize("table_name_id")) => "TableNameId" OK
Doctrine_Inflector::classify(Doctrine_Inflector::tableize("TableName_id")) => "TableNameId" ???

Doctrine_Inflector::tableize(Doctrine_Inflector::classify("table_name")) => "table_name" OK
Doctrine_Inflector::tableize(Doctrine_Inflector::classify("TableName")) => "tablename" FAIL
Doctrine_Inflector::tableize(Doctrine_Inflector::classify("table_name_id")) => "table_name_id" OK
Doctrine_Inflector::tableize(Doctrine_Inflector::classify("TableName_id")) => "tablename_id" ???

I'm not sure this are really bugs. But if tableize and classify are called in turn in various situations, it could lead to some funny behaviors, considering all that name matching is case sensitive.

If this was the intended behavior, a word of warning about table and field names would be welcome in the documentation.



 Comments   
Comment by Jonathan H. Wage [ 01/Dec/09 ]

Fixing the irregularities actually break Doctrine 1..It is a bit weird and changing it is not BC. I am not sure what you want me to do for this ticket? Can you explain the bug you ran into so we can fix it specifically instead. We can write our own functions instead of using classify/tableize.

Comment by G. Perréal [ 01/Dec/09 ]

I figured this out. Actually I was thinking the schema name case didn't matter, e.g. I ended up with this kind of yaml schema:

MyTable:
columns:
someValue; string
otherEntitityId: integer
relations:
OtherEntity:
local: otherEntitityId
foreign: id

Firstly, auto-detect doesn't work but that isn't really an issue.

Secondly, with a real world case, it seems that relation definitions have sometimes 'otherEntitityId' stored in the 'local' field, and other time I found "otherentitityid". This causes issue when trying to dump the data, because the Data_Export doesn't find otherEntitityId to be the local field of any relation.

I thought it was primarily caused by these inconsistencies. Now maybe this is caused at another place, where column name is used instead of field name.

Comment by Jonathan H. Wage [ 01/Dec/09 ]

The tableize and classify are meant to work with "under_score" format and "ThisFormat", so if you make your column names weird casing it can break it. To get the best results you should use this format.

ThisFormat == this_format
this_format == ThisFormat

Comment by Jonathan H. Wage [ 01/Dec/09 ]

This is also a problem because some databases return the column names in all lower case. And in most databases the casing doesn't matter, so they force lower or upper. You will see best results if your column names are formatted like "column_name". We can't do anything to "fix" this or make any code changes that I am aware of so I am closing this ticket as won't fix.

Comment by G. Perréal [ 01/Dec/09 ]

Though I understand your point, I'll quote the documentation : "In short: You can name your fields however you want, using under_scores, camelCase or whatever you prefer." (http://www.doctrine-project.org/documentation/manual/1_1/en/defining-models) Something needs to be changed.

Comment by Jonathan H. Wage [ 01/Dec/09 ]

I'd be happy to fix whatever bug you find. Can you describe the bug? We can fix it to not use classify() and tableize() but instead some custom methods. But in order to do this, I need you to explain the bug with a test case

Comment by G. Perréal [ 02/Dec/09 ]

I'll try to narrow down the issue and write a test cast. I'll open a new ticket.





[DC-301] Doctrine_Core::generateTablesFromModels() does not work on v1.2 but OK on v1.1 Created: 30/Nov/09  Updated: 30/Nov/09  Resolved: 30/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: Import/Export
Affects Version/s: 1.2.0-BETA2, 1.2.0-RC1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: KONDRATEK Nicolas Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

WAMP (Windows XP) and MAMP (Mac Os X)


Attachments: Zip Archive Test_Doctrine.zip    

 Description   

When generating models using Doctrine_Core::createTablesFromModels($modelPath) on Doctrine 1.2, the database isn't generate, but it's work on Doctrine 1.1.

I've send an attachment of my small projet where this problem append.



 Comments   
Comment by Jonathan H. Wage [ 30/Nov/09 ]

Just taking a quick look at this, you are using some weird options together. You need to have classPrefixFiles = true, and the packages in your schema aren't really necessary. Also you aren't doing:

spl_autoload_register(array('Doctrine', 'modelsAutoload'));

So the generated models could never be loaded. Nothing is broken in 1.2 that didn't work in 1.0 or 1.1, you're setup just needs a little work.





[DC-294] Problem with TIME and Oracle [#2340] Created: 25/Nov/09  Updated: 11/Mar/10  Resolved: 11/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: 1.2.0-RC1
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Benjamin Eberlei Assignee: Jonathan H. Wage
Resolution: Incomplete Votes: 0
Labels: None

Attachments: File dc294.diff    

 Description   

The Doctrine Column Types TIME, DATE and TIMESTAMP map with Oracles Column Type DATE, because Oracle doesn't have such specific Types.

Storing DATEs and TIMESTAMPs works with Doctrine, but you have to use String Literals. Is ist planned to support PHP's DateTime?-Objects, too?

More important is a problem I have with Doctrine's TIME Type: Neither in a Yaml Fixture, nor in PHP, I'm able to set a single Time Value like '01:01:01', bevause it results in a DATE-Value '0001-01-01 00:00:00'. Surely I don't want to set a Date-Value for a Column I defined as TIME.

Regards, Michael Galka
Change History
follow-up: ↓ 2 Changed 4 months ago by jwage ¶

    * status changed from new to closed
    * resolution set to fixed
    * milestone changed from New to 2.0.0

In Doctrine 2.0 all dates are DateTime? objects to the user. This is a known problem/limitation of 1.x unfortunately.
in reply to: ↑ 1 Changed 4 months ago by mgalka ¶

    * status changed from closed to reopened
    * resolution fixed deleted

Replying to jwage:

In Doctrine 2.0 all dates are DateTime? objects to the user. This is a known problem/limitation of 1.x unfortunately.

Sorry for having to re-open this ticket. But to be able to use Doctrine with Oracle, I need a hint how to deal with Time fields.

I have several Time-Fields in my Doctrine schema. As described, they are mapped to Oracle DATE columns.

Therefore, Oracle needs a Date-Part for every Time stored in a DATE Field. I do not want to specify a Date, but okay, let's always use a dummy date like '0001-01-01'.

My Problem: If I assign a value like '0001-01-01 08:15:30' to a Doctrine Time Field, Doctrine Validation fails for that value. However, if I disable Doctrine Validation, it works fine, but I really want to use Doctrine Validation.

Is there any way to just use the Time values? This would work with Doctrine Validation, and Doctrine has just to internally add a dummy date part when connected to Oracle, respectively remove that date part when loading DATE-columns into a Time-field.

In Short:

$myObj->timeValue = '08:15:30';
$myObj->isValid(); // true
$myObj->save(); // Will store '0001-01-01 08:15:30' in Oracle. - Object Loading works vice versa.

I hope this is possible. And it would be great if you can tell me where I have to patch Doctrine (for the definition of a dummy date part, e.g.)

Thanks in advance.

Michael
Changed 3 months ago by mgalka ¶

  • priority changed from major to critical

I do not mean to be a pain in the neck, but I'd appreciate a hint for my problem described above.

Collegues keep complaining about the disabled Doctrine Validation in our project, and as described, I'm not able to turn it on.

Regards, Michael
Changed 3 months ago by jwage ¶

  • status changed from reopened to closed
  • resolution set to invalid

Use a valid date for the dummy date instead of an invalid one. That would work.
Changed 3 months ago by mgalka ¶

  • status changed from closed to reopened
  • resolution invalid deleted

Okay, that wasn't helpful. I'll give it another try.

0001-01-01 is a valid date, why shouldn't it be? It's accepted by Oracle, but it's invalid for a Unix Timestamp which has 1970-01-01 as earliest Date. But really, this is just an example Date, a Dummy, which might be replaced by any other Date in the Universe. It's not the source of my problem.

I just want to have an Option to just get and set Time values, if I deal with Doctrine Time Fields. I don't want a Date Part, however, I don't care if internally a Date Part is used, because Oracle needs it. It just has to be used transparent. I outlined my desired solution in my last Post.

Really, I don't want to complain, but I have to justify my decision pro Doctrine. If i can't make myself clear about my problem, don't hestiate to drop me a mail. No offence...

Regards, Michael
Changed 3 months ago by jwage ¶

I understand the problem but I can't quite see a clear way to fix this. Do you have some suggestions for a patch that might fix your problem?
Changed 3 months ago by mgalka ¶

I'm not so familiar with Doctrine's Internals. But assuming that Doctrine Validation is turned on:

$obj->timeValue = '10:10:10';
$obj->isValid(); // true, without further patching, correct?
$obj->save(); // here, Doctrine has to consider that it is connected to Oracle (which it is surely capable of). It has therefore to modify the Value (all Time Field Values) somewhere after the internal validation, adding a dummy date part.
$obj = Doctrine::getTable('Objects')->find(1);
echo $obj->timeValue; // '10:10:10', because Doctrine considered being connected to Oracle and therefore removed the dummy date part somewhere after the load-routine.

With this workaround, Developers with Oracle DB don't have to worry about a Date Part in Time Fields. They use Time Fields just like Developers with MySQL DB. Do you agree?

Thanks for your help.
Changed 7 weeks ago by mgalka ¶

Hi,

any News to this Ticket? Please drop me a note if this bug isn't planned to be fixed in Version 1.x.

This Ticket was created 3 Months ago...

Reards, Michael
Changed 6 weeks ago by mgalka ¶

No Reaction, too bad...

Doctrine can't and won't be used for any other of my projects then. Anyway, thanks for the Support so far.

Regards, Michael



 Comments   
Comment by Benjamin Eberlei [ 25/Nov/09 ]

Patch that, being on Oracle, converts datetime database fields to time values and back.

Comment by Benjamin Eberlei [ 25/Nov/09 ]

This heavily relies on a dateformat that looks like "<date> <time>", which is configurable for each connection.

Comment by Benjamin Eberlei [ 25/Nov/09 ]

in regards to the format, we could add another attribute ORACLE_DATEFORMAT, which could help the cause of this.

Comment by Jonathan H. Wage [ 11/Mar/10 ]

Hi, I would like to fix this but the patch is not acceptable. I would prefer to avoid having these conditionals for driver specific code.

Comment by Jonathan H. Wage [ 11/Mar/10 ]

Benjamin, if you are comfortable with this or another patch, feel free to commit it.





[DC-297] Inheritance : doctrine should not create constraint on shared foreign key Created: 26/Nov/09  Updated: 30/Nov/09  Resolved: 30/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: Inheritance
Affects Version/s: 1.2.0-RC1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Colin Darie Assignee: Roman S. Borschel
Resolution: Invalid Votes: 0
Labels: None
Environment:

MySQL / InnoDB



 Description   

Doctrine should not generate the SQL query to add the constraint if the foreign key is in the table used for a column aggregation inheritance.
Consider the (very simplified) schema.yml

Phonenumber:
  columns:
    id:
      type: integer(4)
      autoincrement: true
      primary: true
    number: string(15)
    type_phonenumber_id: integer(1)
    owner_id: integer(4)
  relations:
    TypePhonenumber:
      local: type_phonenumber_id
      foreign: id
      type: one

Place:
  columns:
    id:
      type: integer(4)
      autoincrement: true
      primary: true
    #some columns
  relations:
    Phonenumbers:
      class: PlacePhonenumber
      type: many
      local: id
      foreign: owner_id
      foreignType: one

PlacePhonenumber:
  inheritance:
    extends: Phonenumber
    type: column_aggregation
    keyField: owner_type
    keyValue: 'place'

User:
  columns:
    id:
      type: integer(4)
      autoincrement: true
      primary: true
    #some columns
  relations:
    Phonenumbers:
      class: UserPhonenumber
      local: id
      foreign: owner_id
      type: many
      foreignType: one

UserPhonenumber:
  inheritance:
    extends: Phonenumber
    type: column_aggregation
    keyField: owner_type
    keyValue: 'user'

TypePhonenumber:
  columns:
   id:
     type: integer(1)
     autoincrement: true
     primary: true
   name: string(255) 

Doctrine generates the following SQL :

[...]
ALTER TABLE phonenumber ADD CONSTRAINT phonenumber_type_phonenumber_id_type_phonenumber_id FOREIGN KEY (type_phonenumber_id) REFERENCES type_phonenumber(id);
ALTER TABLE phonenumber ADD CONSTRAINT phonenumber_owner_id_place_id FOREIGN KEY (owner_id) REFERENCES place(id);
ALTER TABLE phonenumber ADD CONSTRAINT phonenumber_owner_id_user_id FOREIGN KEY (owner_id) REFERENCES user(id);

But the 2 last constraints are wrong because owner_id can reference 2 differents tables. With theses constrains, the tables are unusable.

IMHO, the simpliest and more useful way to solve this issue is to provide an option in relations to disable the constraint creation, but I didn't find any option like this (sorry if I missed something !) :

User
  columns:
  [...]
  relations:
    Phonenumbers:
      class: UserPhonenumber
      type: many
      local: id
      foreign: owner_id
      foreignType: one
      addConstraint: false


 Comments   
Comment by Jonathan H. Wage [ 30/Nov/09 ]

You can disable the exporting of foreign keys for specific models:

User:
  attributes:
    export: tables
  columns:
# ....

or with php:

$userTable->setAttribute(Doctrine::ATTR_EXPORT, Doctrine::EXPORT_TABLES);

Now it will only export the table definition and none of the foreign keys. You can use: none, tables, constraints, plugins, or all.

Comment by Colin Darie [ 30/Nov/09 ]

Thanks for the tip, I completely missed the export options. This will help.

However in this case, the constraint for a "real" foreign key will not be exported (type_phonenumber_id in the Phonenumber table), but this is an acceptable loss.





[DC-295] Export_Schema does not properly account for classes that have a prefix. Created: 26/Nov/09  Updated: 30/Nov/09  Resolved: 30/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: 1.2.0-BETA3, 1.2.0-RC1
Fix Version/s: 1.2.0

Type: Bug Priority: Major
Reporter: Matthew Miller Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None


 Description   

When trying to run doctrine-cli generate-yaml-models (I think other tasks are affected too), I get a yaml file consisting of { }.
I think this is because on line 48 of Doctrine/Export/Schema.php $classPrefix is not passed to Doctrine_Core::loadModels($directory) even if it is set in the configuration array.
This is causing doctrine to think there are no models to generate the yaml for if I am using the $classPrefix.

I'm just learning Doctrine so it's entirely possibly something else is to blame for my issue, but this is where I was able to trace it to.



 Comments   
Comment by Jonathan H. Wage [ 28/Nov/09 ]

I fixed an issue similar to this last week. Make sure you're all fully up-to-date from the 1.2 branch and let me know if the problem still persists there.

Thanks, Jon

Comment by Matthew Miller [ 29/Nov/09 ]

I did an svn checkout from here http://svn.doctrine-project.org/branches/1.2/ and the behavior appears to be the same.

Comment by Jonathan H. Wage [ 30/Nov/09 ]

I added an attribute for setting the class prefix:

$manager->setAttribute(Doctrine_Core::ATTR_MODEL_CLASS_PREFIX, 'MyPrefix_');

Later we can make use of this attribute in more places possibly.

Comment by Matthew Miller [ 30/Nov/09 ]

I'll do some more testing to get a better idea of what's going on but here's what I've got so far.

line 627 of Core.php in the loadModelsfunction you have:

$classPrefix = $classPrefix === null ? $manager->getAttribute(Doctrine_Core::ATTR_MODEL_CLASS_PREFIX) : null;

Shouldn't the final null be $classPrefix? Otherwise I think you're overwriting a prefix that was passed as a parameter to the function.

When generating the models from the DB I pass options to the Doctrine_Cli that look something like:

$options['generate_models_options'] = array(
    'pearStyle' => true,
    'generateTableClasses' => true,
    'classPrefix' => 'Model_',
    'baseClassPrefix' => 'Base_',
    'baseClassesDirectory' => 'Base',
    'classPrefixFiles' => false,
    'generateAccessors' => false,
);

Would it make sense to use these or similar for generating the yaml from the models?

My initial test got me a little further but now it's having trouble loading the Base classes. It looks like this has to do with the baseClassPrefix and baseClassesDirectory but I'll test it more later.

Comment by Jonathan H. Wage [ 30/Nov/09 ]

Yes you are right, the line was wrong and the null should have been $classPrefix. As for the options, I am not sure. You will just need to play with different combinations until it works. Make sure that the class name and path to the file with the class in it are the same. That is how it is able to autoload the model classes.





[DC-296] Error when more subqueries in where condition Created: 26/Nov/09  Updated: 30/Nov/09  Resolved: 30/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: klemen nagode Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None


 Description   

This couse an error in doctrine but it should work:

->where(" (SELECT COUNT(*) FROM Magazine)  = (SELECT COUNT(*) FROM Magazine) " );


 Comments   
Comment by Jonathan H. Wage [ 30/Nov/09 ]

Sorry but this is not enough information. Please re-open with more information and a valid test case.





[DC-291] prefix on use of inheritance class Created: 25/Nov/09  Updated: 07/Dec/09  Resolved: 07/Dec/09

Status: Resolved
Project: Doctrine 1
Component/s: Migrations
Affects Version/s: 1.2.0-RC1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Christian Jaentsch Assignee: Jonathan H. Wage
Resolution: Cannot Reproduce Votes: 0
Labels: None


 Description   

When setting inheritance in a model schema, e.g.

File:
tableName: files
inheritance:
extends: Eeecore_Record_File

...there is a problem building the migrations. The migration diff builder tries to always put a prefix in front of the inheritance (base-)class, so it tries to point to "toprfx_Eeecore_Record_File" which obviously does not exist. We fixed the problem by changing the following line...

Doctrine_Import_Builder (line 1076)

$definition['inheritance']['extends'] = $definition['inheritance']['extends']; // TODO: FIX THIS (no $prefix for inheritance class)



 Comments   
Comment by Jonathan H. Wage [ 30/Nov/09 ]

Hmm. This is the expected behavior. Currently when you diff some models it has to copy them and prefix everything in order to properly perform the diff. Can you give some more information? a test case or something? The change you made obviously isn't right





[DC-286] Trailing DIRECTORY_SEPARATOR will cause Doctrine_Core::loadModels in PEAR mode to fail Created: 25/Nov/09  Updated: 30/Nov/09  Resolved: 30/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: 1.2.0-RC1
Fix Version/s: 1.2.0

Type: Bug Priority: Major
Reporter: Pete Hatton Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None
Environment:

Running Debian Lenny under Xen-64bit 1gig memory.

PHP 5.2.6-1+lenny3 with Suhosin-Patch 0.9.6.2 (cli) (built: Apr 26 2009 20:09:03)
Copyright (c) 1997-2008 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies
with Xdebug v2.0.3, Copyright (c) 2002-2007, by Derick Rethans

Apache - not relevant - running from the command line.

mysql Ver 14.12 Distrib 5.0.51a, for debian-linux-gnu (x86_64) using readlin.2


Attachments: File DC-286-Patch.diff    

 Description   

If I'm using the PEAR model loading in Doctrine, and I try to create the tables from the models as follows.

Doctrine::createTablesFromModels('/path/to/my/models/');

Doctrine doesn't recognise the models and can't create any tables as a result.
However if I remove the trailing slash.

Doctrine::createTablesFromModels('/path/to/my/models');

It works fine. This wasn't a problem is 1.2BETA-3. The problem is caused by changeset 6796, and the code that extracts the class name from the path of the filename not taking into account the trailing DIRECTORY_SEPARATOR.



 Comments   
Comment by Pete Hatton [ 25/Nov/09 ]

Patch for the issue.





[DC-288] Doctrine_Parser_Yml dependany on sfYaml causing issues for non symfony users Created: 25/Nov/09  Updated: 18/Feb/10  Resolved: 30/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: File Parser
Affects Version/s: 1.2.0
Fix Version/s: 1.2.0

Type: Bug Priority: Major
Reporter: Matt Cockayne Assignee: Jonathan H. Wage
Resolution: Fixed Votes: 0
Labels: None


 Description   

I am currently undergoing an integration with the Zend Framework. Unfortunatly we do not use the Symfony framework and have no scope available to include it.

The requirement for sfyaml in its rawest form has caused me to have to rewrite part of your parser to ensure it includes the crrect files which I have now included in our working version of Doctrine.

As Symfony appears to be distributed under an open license surely it would make sense for you to actually include this (or a modified version of this) into your library.

This would make the code much more portable and reusable for people who do not use symfony

My "Quick Fix" just involved dropping all the sfYaml files into the parser folder and doing a straight include untill I can find a more elegant solution



 Comments   
Comment by Ashley Broadley [ 27/Nov/09 ]

I could not get your 'quick fix' to work no matter whether i dumped the files in the parser dir, on my include path or anywhere.

Would it not be a good idea to reformat the sfYaml class names into the Pear/Zend/Doctrine scheme and integrate them into doctrine it's self?

for example:
File name:
sfYaml.php -> Doctrine/Parser/Yaml.php
Class name:
sfYaml -> Doctrine_Parser_Yaml

File name:
sfYamlDumper.php -> Doctrine/Parser/Yaml/Dumper.php
Class name:
sfYamlDumper -> Doctrine_Parser_Yaml_Dumper

etc...

If these files are going to be used as standard i would think it to be a good idea.

Comment by Matt Cockayne [ 27/Nov/09 ]

Hi Ashley

This link will show you how I added the files to create my quick fix from the files stored in my subversion repo.

http://subversion.zucchi.co.uk/listing.php?repname=titan&path=%2Ftrunk%2Flibrary%2FDoctrine%2FParser%2F#path_trunk_library_Doctrine_Parser_

As well as puutting the files in the parser dir I also had to change the requires at the top of the yaml.ph file.

Changeing the names of the classes would be a good idea for a more permanent fix but I will leave that to the good people at Doctrine to decide to put in place

Comment by Jonathan H. Wage [ 28/Nov/09 ]

I will fix the issue first thing this week. @Ashley, this is the way it used to be before this change. The issue is then we don't get bug fixes from sfYaml. It is better if we use it as a vendor library.

Comment by Jonathan H. Wage [ 30/Nov/09 ]

I moved the sfYaml external folder to the Parser folder. This should work better for everyone now.

Comment by Andreas Möller [ 03/Dec/09 ]

It's nice that one can now skip to add

/library/Doctrine

to the included paths, but it's not so nice that now the sfYaml files can not be found.

Comment by Exception e [ 06/Dec/09 ]

This ticket is not fixed. The doctrine autoloader should handle all it's dependencies, including those from symphony. These kind of things should not leak outside doctrine.

It took me quite some time to figure out. It is a severe bug.

Comment by Jonathan H. Wage [ 06/Dec/09 ]

hmm, What is the problem? The Doctrine::autoload() function does handle the autoloading of sfYaml now. Can you explain more detail about the problem you are encountering?

Comment by Matt Cockayne [ 06/Dec/09 ]

I had a problem too in it nit being handled by the autoloader. It may be down to how I have integrated Doctrine into my Zend Framework app, so I cant really comment on that one

the problem is easily fixed with a simple set of requires in the parser file. not ideal but it works for the moment

Comment by Jonathan H. Wage [ 06/Dec/09 ]

What was the problem? What was the error you got? I'd like to try and help fix something in Doctrine to make this handled better so any other more specific information anyone can provide would be much appreciated.

Comment by Peter Petermann [ 07/Dec/09 ]

In an Application using Zend Framework 1.9.6 and Doctrine 1.2,

$ doctrine-cli generate-models-yaml
PHP Fatal error: Class 'sfYaml' not found in /path/to//library/Doctrine/Doctrine/Parser/Yml.php on line 80
Fatal error: Class 'sfYaml' not found in /path/to/library/Doctrine/Doctrine/Parser/Yml.php on line 80

further investigation shows that the method in Core.php, which was modified with the new Path is never called.

the thing is, usually one sets up Zend Framework, so it uses the Doctrine autoloader
whenever a class is prefixed with Doctrine (by registering that namespace)
however, since sfYaml doesnt fall under that rule, the Zend Autoloader does not know that it needs to forward this, therefor the Doctrine one is never called.

A workarround is to add
require_once 'Doctrine.php';
$autoloader = Zend_Loader_Autoloader::getInstance();
$autoloader->registerNamespace('sfYaml')->pushAutoloader(array('Doctrine', 'autoload'), 'sfYaml');
to the Zend Booststrap class - like in your _initDoctrine method.

This, tbfh, is still an ugly workarround, but so is the sfYaml loading in the doctrine autoloader itself. Personally i agree with the OP, Doctrine should come as a complete package, not using symfony externals.

Comment by Lukas Kahwe [ 07/Dec/09 ]

Why is that "workaround" ugly? The fact of the matter is that not all code that is useful follows the same naming conventions. Having to fork or write wrappers around code with different naming conventions is a maintenance nightmare. Doctrine's autoloader implementation is a reference implementation but I assume most people use their own (by way of their chosen framework) and that one better be flexible enough to deal with different naming conventions. It seems ZF has these capabilities, so why is it ugly using them?

Comment by Jonathan H. Wage [ 07/Dec/09 ]

I see, so the problem is when you're not using Doctrine::autoload(). The thing is we seem to have problems no matter how we integrate sfYaml. Originally we had it setup as Doctrine_Parser_Yaml_SfYaml so it was just like a normal Doctrine class, but then it got out of date and we don't get the latest bug fixes. I changed it to be in lib/vendor/sfYaml and required it manually in the Doctrine code that used it. This causes problems if the project has already loaded sfYaml by other means. For example in a Symfony project sfYaml exists sometimes or if the user happens to use the sfYaml component in his project. So, it needs to be handled by the autoloader for sure I think.

Comment by Jeremy Hicks [ 16/Dec/09 ]

Edit: Looks like having this:

$manager->setAttribute(
Doctrine::ATTR_MODEL_LOADING,
Doctrine::MODEL_LOADING_CONSERVATIVE
);

in an Application Resource plug-in along with the CLI call stops the tables from getting created.

Autoloading the sfYaml stuff works for me now, but I'm having what appears to be a related problem. The Doctrine cli task "build-all-reload" is supposed to both recreate the database and generate the model files. It is generating the model files fine now. However, the database gets dropped and re-added, but none of the tables get re-added. The output says, "Created tables successfully" but after checking the DB they are not there. I'm using an application resource class, not a bootstrap init function, if that matters.

Comment by Michael Ekoka [ 18/Feb/10 ]

Currently using ZF 1.10 and Doctrine 1.2.1. To resolve this issue you need to ask Zend's Autoloader to register Doctrine's Autoloader for Doctrine and sfYaml namespaces:

$loader = Zend_Loader_Autoloader::getInstance();
$loader->pushAutoloader(array('Doctrine_Core','autoload'), array('Doctrine', 'sfYaml'));





[DC-284] Handle errors on ROLLBACK during migrations Created: 24/Nov/09  Updated: 24/Nov/09  Resolved: 24/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: 1.2.0-RC1
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Daniel Cousineau Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

Windows XP, SQL Server 2005, PHP 5.2.9-2


Attachments: Text File doctrine-1.2-handle-rollback-errors.patch    

 Description   

While I haven't tracked down the real problem, a problem when running migrations against SQL Server 2005 (other versions I would assume as well) is that I'm assuming ROLLBACKS are sent twice, the second time resulting in a "corresponding TRANSACTION not found" exception that is uncaught and causes migrations to fail out without giving you the real error message (e.g. "key name already exists" etc.).

I've attached a patch that just wraps the rollbackTransaction calls in a try/catch block and adds the caught exception to the list of errors to be printed at the end of the migration task. The patch is against the most current SVN checkout of the Doctrine 1.2 branch (r6798)



 Comments   
Comment by Jonathan H. Wage [ 24/Nov/09 ]

Sorry, the patch isn't valid and the ticket doesn't really have enough information. We need to know the real problem if any. Can you re-open with some more information.





[DC-208] Make it possible to define an expression/constant for temporal columns via yaml Created: 10/Nov/09  Updated: 05/Apr/10  Resolved: 16/Mar/10

Status: Resolved
Project: Doctrine 1
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Lukas Kahwe Assignee: Jonathan H. Wage
Resolution: Can't Fix Votes: 1
Labels: None


 Description   

some_date:
type: timestamp
notnull: true
default: CURRENT_TIMESTAMP

http://trac.doctrine-project.org/ticket/1740

In general (maybe 1.3?) it might make sense to enable people to somehow say that the default should not be quoted. Maybe an additional "flag" like the "fixed: true" we have for string type definitions?

However for 1.2 I propose that we look if we can change so that default values are never quoted. Rather they are expected to be an expression/constant. This is somewhat unclean, but should address the most common uses cases with minimal invasion into the yaml syntax. Furthermore even if one does need the string to be quoted (for example when setting a default date "2009-03-03", then one could hard quotes into the default "'2009-03-03'" and it should work on most RDBMS (of course only if they support the ISO date format .. which should be the case for even the most obsure locale settings).



 Comments   
Comment by Lukas Kahwe [ 10/Nov/09 ]

i guess for mysql this feature would actually need to work through a trigger, which makes this change even less appealing. maybe we should instead create a behavior that can handle either setting the CURRENT_* constant or setup a trigger.

In general I think there is some merit to also making it optionally possible to have defaults set inside the model before the data is send to the server (especially for time it can be good to use only the date/time settings from a single server). But this again isnt anything for doctrine 1.2 .. maybe this ticket should be cleaned up into a 2.0 ticket and/or a ticket for a behavior?

Comment by Jonathan H. Wage [ 10/Nov/09 ]

The issue is not present in 2.0. Database default values are not supported(I'm 99% sure). A default value just consists of simple oop. private $field = 'value'; in the entity class definition.

Comment by Whitefawn [ 05/Apr/10 ]

I am using "default now()" quite often in other project. It feels like Doctrine restricting me here and not really helping.

Is it possible to implement this somehow?





[DC-206] The date type cannot be read in the YYYY-MM-DD format in fixtures Created: 10/Nov/09  Updated: 10/Nov/09  Resolved: 10/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: Data Fixtures
Affects Version/s: 1.2.0-BETA2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Jérémy Subtil Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

PostgreSQL 8.3.7



 Description   

Hi there!

Consider the following relation:

EyPayment:
  actAs:
    Timestampable:
    SoftDelete:
  columns:
    id:
      type:               integer(6)
      primary:            true
      autoincrement:      true
    comment:
      type:               string(255)
      notnull:            false
    requested_at:
      type:               date
      notnull:            false
    received_at:
      type:               date
      notnull:            false
    sent_at:
      type:               date
      notnull:            false
    signed_at:
      type:               date
      notnull:            false
    validated_at:
      type:               date
      notnull:            false

Consider the following fixtures:

EyPayment:
  ey_payment_1:
    comment:              bla bla
    requested_at:         2009-01-11
    received_at:          2009-01-14
    created_at:           2009-01-15 19:00

Only the created_at field is filled in the database, whereas the requested_at and received_at are not. However, as specified in the Doctrine 1.0 book, a date field can be filled with the YYYY-MM-DD format.

Note that I'm using PostgreSQL.

Thanks,
Jérémy Subtil, Sensio Labs



 Comments   
Comment by Jonathan H. Wage [ 10/Nov/09 ]

The YAML specification reads dates into a unix timestamp, so strtotime() is used. If you want to force it to be a string date, then you need to wrap it in single quotes.





[DC-200] replace() does not set created_at and updated_at when using Timestampable behavior Created: 08/Nov/09  Updated: 10/Nov/09  Resolved: 10/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: Behaviors, Timestampable
Affects Version/s: 1.0.13
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Tom Boutell Assignee: Jonathan H. Wage
Resolution: Duplicate Votes: 0
Labels: None
Environment:

MacOS, Symfony 1.2.9DEV



 Description   

The Timestampable behavior does not set created_at or updated_at on a call to replace() (if it doesn't exist yet it ought to do both).

I see some old discussion on the subject of possibly exterminating replace() rather than fixing it to work with things like behaviors, but I also see that a similar ticket was opened for Doctrine 1.2 and the issue has apparently been addressed there.



 Comments   
Comment by Jonathan H. Wage [ 10/Nov/09 ]

We did have a ticket for this already and it was fixed in Doctrine 1.2. It's not something we can include in 1.0.





[DC-196] Saving related records from relations doen't work Created: 06/Nov/09  Updated: 19/Jan/10  Resolved: 10/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: Connection
Affects Version/s: 1.1.4
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Dominik Winter Assignee: Jonathan H. Wage
Resolution: Won't Fix Votes: 0
Labels: None
Environment:

Debian 5.0.3, PHP 5.2.11-0.dotdeb.1 with Suhosin-Patch 0.9.7


Attachments: Text File UnitOfWork.patch    

 Description   

Hello guys,

at first my ERM:

[Table1] --1---*--> [Table1Sub] --1---1--> [Table2] --1---*--> [Table2Sub]

I want to add datasets in Table1, Table1Sub and Table2Sub. Table2 already has a dataset. So I try

<?php
$table1 = new Table1;
$table1->fromArray($someDataForTable1);

$table1sub = new Table1Sub;
$table1sub->fromArray($someDataForTable1Sub); // table2_id is set, so everything is right

$table2sub = new Table2Sub;
$table2sub->fromArray($someDataForTable2Sub);

$table1sub->table2->table2sub[] = $table2sub;
$table1->table1sub[] = $table1sub;

$table1->save();
?>

Doctrine saves all datasets but Table2Sub.

I reversed engineer a while, and found in Doctrine_Connection_UnitOfWork::saveRelatedLocalKeys() the if condition isModified().
So, Table2Sub is not saved, because Table2 is not modified. When a changed it to isModified(true), Doctine saves everything.

Is it possible to add the attached patch?

BTW: Can somebody add a 'return $this' in Doctrine_Record::fromArray()?

Greets,
Dominik



 Comments   
Comment by Jonathan H. Wage [ 10/Nov/09 ]

It is not this way due to performance reasons. Checking if any related records are dirty would be too slow. If you want to force it to save you can change the state of the object manually.

$obj->state('DIRTY');
Comment by Dominik Winter [ 11/Nov/09 ]

Our workaround is to save $table2 manually, because this is more readable in my point of view.

$table2->save();

Manipulating the state of record looks like a hack.

Maybe you can implement a saveDeep() method in Doctrine_Record?

Comment by Benoît Guchet [ 18/Jan/10 ]

I strongly agree with Dominik.

Often I construct hierarchies of new objects, possibly with only the "leaf" ones having "real properties" modified.
It's very hard to deal with that...





[DC-193] Importing fixtures from YAML in PostgreSQL can fail with column aggregation since sfYaml is an svn external Created: 06/Nov/09  Updated: 10/Nov/09  Resolved: 10/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: Data Fixtures, Inheritance
Affects Version/s: 1.2.0-BETA2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Jérémy Subtil Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:

PostgreSQL 8.3.7, PHP 5.2.11



 Description   

Hi,

While I'm reloading my fixtures, I get this exception:

SQLSTATE[42883]: Undefined function: 7 ERROR:  operator does not exist: character varying = integer
  LINE 1: ...ET deleted_at = $1 WHERE (deleted_at IS NULL AND (type = 1))
                                                                    ^
  HINT:  No operator matches the given name and argument type(s). You might need to add explicit type casts.

I get this issue since revision 6623, when sfYaml has been made an svn external.

The interesting point is that I can reload my fixtures properly under MySQL, but not under PostgreSQL. This bug is probably related to the new sfYaml type checking (DC-165).

I did some investigation and in reality the fixture loading fails when a relation implementing column aggregation inheritance uses an integer as a key value.

For instance, this piece of YAML should fail at reloading when some EyContactAuthor fixtures are added :

EyContact:
  actAs:
    Timestampable:
    SoftDelete:
  columns:
    id:
      type:              integer(6)
      primary:           true
      autoincrement:     true
    title:
      type:              string(4)
      notnull:           true
      default:           M
    first_name:
      type:              string(150)
      notnull:           true
    last_name:
      type:              string(150)
      notnull:           true

EyContactAuthor:
  actAs:
    Timestampable:
    SoftDelete:
  inheritance:
    extends:             EyContact
    type:                column_aggregation
    keyField:            type
    keyValue:            1
  columns:
    published_first_name:
      type:              string(150)
      notnull:           false
    published_last_name:
      type:              string(150)
      notnull:           false
    biography:
      type:              string(5000)
      notnull:           false

Conversely, the following piece of YAML is fine:

EyContact:
  actAs:
    Timestampable:
    SoftDelete:
  columns:
    id:
      type:              integer(6)
      primary:           true
      autoincrement:     true
    title:
      type:              string(4)
      notnull:           true
      default:           M
    first_name:
      type:              string(150)
      notnull:           true
    last_name:
      type:              string(150)
      notnull:           true

EyContactAuthor:
  actAs:
    Timestampable:
    SoftDelete:
  inheritance:
    extends:             EyContact
    type:                column_aggregation
    keyField:            type
    keyValue:            author
  columns:
    published_first_name:
      type:              string(150)
      notnull:           false
    published_last_name:
      type:              string(150)
      notnull:           false
    biography:
      type:              string(5000)
      notnull:           false

So there are two solutions: either the key value should be compatible again with the integer type under PostgreSQL, or the compulsory use of a string should be documented.

Thanks for your time,
Jérémy Subtil, Sensio Labs



 Comments   
Comment by Jonathan H. Wage [ 10/Nov/09 ]

The type column that is added is a string I believe. So in your YAML when you use 1 as the value, in PHP it ends up being a integer 1, which that param is then bound to the type column. So in pgsql you get the error. You can fix this by doing '1' forcing the yaml parser to read it in as a string, or change the type column on the top most parent model to be a string.

Comment by Jonathan H. Wage [ 10/Nov/09 ]

No code fix is necessary for this ticket.





[DC-177] UnitOfWork orders the $flushTree incorrectly for 1:1 relations Created: 04/Nov/09  Updated: 04/Nov/09  Resolved: 04/Nov/09

Status: Resolved
Project: Doctrine 1
Component/s: Import/Export
Affects Version/s: 1.1.5, 1.2.0-BETA1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: David Abdemoulaie Assignee: Jonathan H. Wage
Resolution: Invalid Votes: 0
Labels: None
Environment:
  • Doctrine 1.2.0-BETA1 Sandbox
  • Doctrine 1.1.5 Sandbox
  • MySQL DSN


 Description   

To Reproduce:

  • Use these models & fixtures for the sandbox: http://gist.github.com/225647
  • Run the following commands
  • ./doctrine drop-db
  • ./doctrine create-db
  • ./doctrine create-tables
  • ./doctrine load-data

The last command will result in the following error:

SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '0' for key 'PRIMARY'

Viewing the MySQL query log shows the following statements executed as a result of the load-data attempt:

DELETE FROM tickets_mm
DELETE FROM tickets
START TRANSACTION
INSERT INTO tickets_mm (external_identifier, delivery_requested) VALUES ('', '0')
INSERT INTO tickets (details, id) VALUES ('Stuff', NULL)
COMMIT
START TRANSACTION
INSERT INTO tickets_mm (external_identifier, delivery_requested) VALUES ('', '0')
ROLLBACK	

The table containing the foreign key should not