Doctrine Project

[DC-279] Code Corrections For MsSql modifyLimitQuery

Mon, 23 Nov 2009 23:25:38 +0000

Below are some code corrections for the modifyLimitQuery. These fixes should also fix ~~DC-37~~

Code details:

Fixed case on sorting orders to match all other syntax.
Added additional explode to ensure we only get fieldname and not tablename.fieldname.
Added code to determine primary key for use in select statement. There may be a better way of doing this, but not with the input sent to the function I don't think.
Removed the outer table (not sure how this could have ever worked) and switched inner table to return only primary key and not all fields.
Removed outer table order by logic.

Diff - Old To New:

160c160
<                     $sorts[$i] = (stripos($orders[$i], ' desc') !== false) ? 'desc' : 'asc';
---
>                     $sorts[$i] = (stripos($orders[$i], ' desc') !== false) ? 'DESC' : 'ASC';
171a172
>                     $aux2 = explode('.', end($aux2));
185a187,191
>             $fields_string = substr($query, strlen($selectReplace), strpos($query, ' FROM ') - strlen($selectReplace));
>             $field_array = explode(',', $fields_string);
>             $aux2 = explode('.', $field_array[0]);
>             $key_field = trim(end($aux2));
> 
187c193
<             $query = 'SELECT * FROM (SELECT TOP ' . $count . ' * FROM (' . $query . ') AS ' . $this->quoteIdentifier('inner_tbl');
---
>             $query = 'SELECT TOP ' . $count . ' ' . $this->quoteIdentifier('inner_tbl') . '.' . $key_field . ' FROM (' . $query . ') AS ' . $this->quoteIdentifier('inner_tbl');
199,212d204
<                 }
<             }
< 
<             $query .= ') AS ' . $this->quoteIdentifier('outer_tbl');
< 
<             if ($orderby !== false) {
<                 $query .= ' ORDER BY '; 
< 
<                 for ($i = 0, $l = count($orders); $i < $l; $i++) { 
<                     if ($i > 0) { // not first order clause 
<                         $query .= ', '; 
<                     } 
< 
<                     $query .= $this->quoteIdentifier('outer_tbl') . '.' . $aliases[$i] . ' ' . $sorts[$i];

Entire New Function:

    public function modifyLimitQuery($query, $limit = false, $offset = false, $isManip = false)
    {
        if ($limit > 0) {
            $count = intval($limit);
            $offset = intval($offset);

            if ($offset < 0) {
                throw new Doctrine_Connection_Exception("LIMIT argument offset=$offset is not valid");
            }

            $orderby = stristr($query, 'ORDER BY');

            if ($orderby !== false) {
                // Ticket #1835: Fix for ORDER BY alias
                // Ticket #2050: Fix for multiple ORDER BY clause
                $order = str_ireplace('ORDER BY', '', $orderby);
                $orders = explode(',', $order);

                for ($i = 0; $i < count($orders); $i++) {
                    $sorts[$i] = (stripos($orders[$i], ' desc') !== false) ? 'DESC' : 'ASC';
                    $orders[$i] = trim(preg_replace('/\s+(ASC|DESC)$/i', '', $orders[$i]));

                    // find alias in query string
                    $helper_string = stristr($query, $orders[$i]);

                    $from_clause_pos = strpos($helper_string, ' FROM ');
                    $fields_string = substr($helper_string, 0, $from_clause_pos + 1);

                    $field_array = explode(',', $fields_string);
                    $field_array = array_shift($field_array);
                    $aux2 = spliti(' as ', $field_array);
                    $aux2 = explode('.', end($aux2));

                    $aliases[$i] = trim(end($aux2));
                }
            }

            // Ticket #1259: Fix for limit-subquery in MSSQL
            $selectRegExp = 'SELECT\s+';
            $selectReplace = 'SELECT ';

            if (preg_match('/^SELECT(\s+)DISTINCT/i', $query)) {
                $selectRegExp .= 'DISTINCT\s+';
                $selectReplace .= 'DISTINCT ';
            }

            $fields_string = substr($query, strlen($selectReplace), strpos($query, ' FROM ') - strlen($selectReplace));
            $field_array = explode(',', $fields_string);
            $aux2 = explode('.', $field_array[0]);
            $key_field = trim(end($aux2));

            $query = preg_replace('/^'.$selectRegExp.'/i', $selectReplace . 'TOP ' . ($count + $offset) . ' ', $query);
            $query = 'SELECT TOP ' . $count . ' ' . $this->quoteIdentifier('inner_tbl') . '.' . $key_field . ' FROM (' . $query . ') AS ' . $this->quoteIdentifier('inner_tbl');

            if ($orderby !== false) {
                $query .= ' ORDER BY '; 

                for ($i = 0, $l = count($orders); $i < $l; $i++) { 
                    if ($i > 0) { // not first order clause 
                        $query .= ', '; 
                    } 

                    $query .= $this->quoteIdentifier('inner_tbl') . '.' . $aliases[$i] . ' '; 
                    $query .= (stripos($sorts[$i], 'asc') !== false) ? 'DESC' : 'ASC';
                }
            }
        }

        return $query;
    }

[DC-276] HAVING does not parse column aliases

Fri, 8 Jan 2010 16:52:19 +0000

How to reproduce

$query->having('<aggregate_alias> <operator> <column_alias>').

Expected SQL:

 
... HAVING <internal_aggregate_alias> <operator> a<N>__<column_name>...

Produced SQL

... HAVING <internal_aggregate_alias> <operator> <column_alias> ... /* yes, the same <column_alias> as in ->having() call! */

I've prepared this "real-world example".

Please fix it someone since I'm forced to invent weird workarounds to emulate this behaviour.

Thanks in advance.

[DC-274] Doctrine_Parser_Yml: hardcoded require calls for sfYaml

Mon, 23 Nov 2009 22:58:39 +0000

In Doctrine_Parser_Yml the requires calls to sfYaml are hardcoded:
if ( ! class_exists('sfYaml', false)) {
require_once dirname(_FILE_) . '/../../vendor/sfYaml/sfYaml.php';
require_once dirname(_FILE_) . '/../../vendor/sfYaml/sfYamlDumper.php';
require_once dirname(_FILE_) . '/../../vendor/sfYaml/sfYamlInline.php';
require_once dirname(_FILE_) . '/../../vendor/sfYaml/sfYamlParser.php';
}

This causes issues when one has already included sfYaml elsewhere. Either one needs to include the code for sfYaml twice or one needs to always load sfYaml before Doctrine parses any yaml files.

A better approach to handling the fact that sfYaml does not adhere to the standard naming convention is to do these require calls inside the autoloader, but only using "include" in order to allow a fallback autoloader to also be able to load the code.

[DC-267] Strange behaviour with HYDRATE_ARRAY

Thu, 19 Nov 2009 19:30:32 +0000

Some values of a join can be found in root component some in the relations - even for the same value

$q = Doctrine_Query::create()
-> select ('a.id, b.code, b.code as also_b_code')
-> from('Foo a')
-> innerJoin('a.Bar b')
->setHydrationMode(Doctrine::HYDRATE_ARRAY);

$res = $q->execute();

id is primary key of table Foo and code is primary key of table Bar

The result will contain id AND also_b_code as $res[0]['id'] and $res[0]['also_b_code'] but code is under $res[0]['Bar']['code']

Is there any way to get Doctrine::HYDRATE_ARRAY to work as under 1.1?

[DC-266] I18n - PostgreSQL - translation table name

Thu, 19 Nov 2009 19:02:46 +0000

Tables, which are translated, do not use table name from classes of tables with translations, but it seems that Doctrine just transforms
class name from camel case to underscore and appends translation. We use different class names, because of the problem with schemas described in
http://www.doctrine-project.org/jira/browse/DC-264.

Is it possible to fix this behavior - get table name from class?
Or at least write in documentation, that we can use atribute tableName in section I18n?

[DC-263] Incorrect type in PGSQL import.

Thu, 19 Nov 2009 19:39:08 +0000

in http://trac.doctrine-project.org/browser/branches/1.1/lib/Doctrine/Import/Pgsql.php

line 172

used if (strtolower($val['type']) === 'varchar') {

but in postgres varchar does not exist so you must compare with character varying like that :

if (strtolower($val['type']) === 'character varying') {

[DC-261] Query overrwtires explicitly passed connection with bound one

Thu, 19 Nov 2009 19:51:20 +0000

Doctrine_Query::loadRoot() and Doctrine_Query::buildInheritanceJoinSql() have the following snippet of code:

        // get the connection for the component
        $manager = Doctrine_Manager::getInstance();
        if ($manager->hasConnectionForComponent($name)) {
            $this->_conn = $manager->getConnectionForComponent($name);
        }

Seems clear that here we would like to use the connection bound to the given component instead of default one. But if we explicitly pass a connection in (like Doctrine_Query::create($conn)), then this code will also overwrite it with bound connection, although passed connection should have higher priority than bound one.

I would like to prepare a patch, but I'm not absolutely sure, why we check for bound connection only inside loadRoot() and buildInheritanceJoinSql()?

[DC-260] Improper translation of data types

Mon, 23 Nov 2009 22:46:16 +0000

Doctrine translates the datatype integer to oracle datatype number.
Doctrine generates number(8) for id's, numer(4) for left/right values
in nested sets and number(2) for the level column in nested sets.

This will result in corrupted trees or unexpected behavior if a tree
does contain more than 9.999 records or has more than 99 levels.

[DC-257] Missing identifier quoting in Doctrine_Connection_Oracle

Wed, 18 Nov 2009 21:38:54 +0000

If identifier quoting is enabled doctrine does not perform quoting in some subqueries.

$this->quoteIdentifier has to be added as below in Doctrine_Connection_Oracle oracle.php at line 106

$query = 'SELECT ' . $this->quoteIdentifier('b') . '.' .$column. ' FROM ('.
'SELECT a.*, ROWNUM AS doctrine_rownum FROM ('
. $query . ') a '.
') ' . $this->quoteIdentifier('b') .' '.
'WHERE doctrine_rownum BETWEEN ' . $min . ' AND ' . $max;

[DC-256] Identifier Quoting not made in generated subquery

Thu, 19 Nov 2009 20:00:35 +0000

If identifier quoting is enabled doctrine does not perform quoting in some subqueries.

$this->_conn->quoteIdentifier has to be added twice as below in query.php line 1426:

$subquery = 'SELECT doctrine_subquery_alias.' . $this->_conn->quoteIdentifier($table->getColumnName($table->getIdentifier()))

    . ' FROM (' . $subquery . ') doctrine_subquery_alias'
    . ' GROUP BY doctrine_subquery_alias.' . $this->_conn->quoteIdentifier($table->getColumnName($table->getIdentifier()))
    . ' ORDER BY MIN(ROWNUM)';

[DC-255] XML Export not in UTF-8

Thu, 19 Nov 2009 19:58:11 +0000

I'm testing doctrine 1.2.0-B3 connected to mssql 2000 for XML based exports.

We use latin1 (ISO-8859-1) as collation for the mssql server. When i now create an XML export in Doctrine, Doctrine uses SimpleXML which requires the strings encoded in UTF-8, but our mssql collation is latin1. This results in a invalid, non UTF-8 XML file.
It's required to add the ability to convert the strings from the database encoding to the 'export' encoding.
(similar to http://trac.doctrine-project.org/ticket/1887)

I have added a simple patch to add this possibility, then we can do something like this:

Doctrine_Parser::dump($data->toArray(false, true), 'xml', 'testExport.xml', 'ISO-8859-1');

[DC-254] getLimitSubquery Not Including All Order By Parts

Sun, 27 Feb 2011 18:39:08 +0000

The getLimitSubquery function is only including the first orderby field and ignoring all remaining parts. The function also needs to be changed to include mssql and odbc.

The diff's below provide corrections.

Thanks,

Mike

diff -r Doctrine-1.1.2\lib\Doctrine\Query.php Doctrine-1.1.2-Fixes\lib\Doctrine\Query.php
1367c1367
<         if ($driverName == 'pgsql' || $driverName == 'oracle' || $driverName == 'oci') {
---
>         if ($driverName == 'pgsql' || $driverName == 'oracle' || $driverName == 'oci' || $driverName == 'mssql' || $driverName == 'odbc') {
1371,1387c1371,1392
<                 $part_original = trim($e[0]);
<                 $callback = create_function('$e', 'return trim($e, \'[]`"\');');
<                 $part = trim(implode('.', array_map($callback, explode('.', $part_original))));
< 
<                 if (strpos($part, '.') === false) {
<                     continue;
<                 }
< 
<                 // don't add functions
<                 if (strpos($part, '(') !== false) {
<                     continue;
<                 }
< 
<                 // don't add primarykey column (its already in the select clause)
<                 if ($part !== $primaryKey) {
<                     $subquery .= ', ' . $part_original;
<                 }
---
> 
>                 foreach ($e as $f) {
>                     if ($f == 0 || $f % 2 == 0) {
>                         $part_original = trim($f);
>                         $callback = create_function('$e', 'return trim($e, \'[]`"\');');
>                         $part = trim(implode('.', array_map($callback, explode('.', $part_original))));
> 
>                         if (strpos($part, '.') === false) {
>                             continue;
>                         }
> 
>                         // don't add functions
>                         if (strpos($part, '(') !== false) {
>                             continue;
>                         }
> 
>                         // don't add primarykey column (its already in the select clause)
>                         if ($part !== $primaryKey) {
>                             $subquery .= ', ' . $part_original;
>                         }
>                     }
>                 }

[DC-253] Missing Data Types In DataDict For MsSql

Wed, 18 Nov 2009 17:24:36 +0000

There are some missing data types in the getPortableDeclaration function for MsSql. The following diff adds the types I found missing so far.

Thanks,
Mike

diff -r doctrine\lib\Doctrine\DataDict\Mssql.php Doctrine-1.1.2\lib\Doctrine\DataDict\Mssql.php
193,202d192
< case 'uniqueidentifier':
< $type[] = 'string';
< $length = 36;
< break;
< case 'sql_variant':
< case 'sysname':
< case 'binary':
< $type[] = 'string';
< $length = null;
< break;

[DC-252] Doctrine ignores UNIQUE:true on int

Wed, 18 Nov 2009 17:25:05 +0000

Consider this schema.yml:

Orden:
  tableName: ordenes    
  options:
    type: InnoDB  
  columns:
    id:
      primary: true
      type: integer(4)
      autoincrement: true
    numero: 
      type: integer(4)
      unique: true

sql:

CREATE TABLE ordenes (id INT AUTO_INCREMENT, numero INT, PRIMARY KEY(id)) COLLATE utf8_unicode_ci ENGINE = InnoDB;

unique constraint don't appear.

[DC-249] Doctrine_Manager::closeConnection() always changes current connection

Wed, 18 Nov 2009 20:36:27 +0000

Doctrine_Manager::closeConnection() should try to change current connection only if the just closed one was the current connection.

[DC-245] Table name prefix for table migration_version

Fri, 28 May 2010 06:19:49 +0000

The doctrine migration class sets a default table name for storing its version information, which bypasses any formatting rules, e.g. a table name prefix. Calling the function setTableName() does apply the format defined, but one cannot call _createMigrationTable() afterwards. This results in a situation, where on one hand I can have the table automatically created with the "wrong" name and on the other hand have the class use the correct table name but not create it at first.

The attached patch is a simple solution to this dilemma, it added a call from the contructor to set the table name by the function call.

[DC-244] Missing savepoint methods on Doctrine_EventListener_Chain

Tue, 17 Nov 2009 20:12:58 +0000

A couple of methods are missing to use savepoint argument in transaction methods from Doctrine_Transaction object.

If you want to process the bug, start a transaction, start a savepoint, and you will see a fatal error.

Below, you will find a piece of code with missing methods to add to Doctrine_EventListener_Chain class:

// @edit: Missing methods – loops
// @start

/**
* postSavepointCommit
* an event invoked after a Doctrine_Connection transaction with savepoint
* is committed
*
* @param Doctrine_Event $event
* @return void
*/
public function postSavepointCommit(Doctrine_Event $event)
{
foreach ($this->_listeners as $listener)

{ $listener->postSavepointCommit($event); }

}

/**
* preSavepointCommit
* an event invoked before a Doctrine_Connection transaction with savepoint
* is committed
*
* @param Doctrine_Event $event
* @return void
*/
public function preSavepointCommit(Doctrine_Event $event)
{
foreach ($this->_listeners as $listener)

{ $listener->preSavepointCommit($event); }

}

/**
* postSavepointRollback
* an event invoked after a Doctrine_Connection transaction with savepoint
* is being rolled back
*
* @param Doctrine_Event $event
* @return void
*/
public function postSavepointRollback(Doctrine_Event $event)
{
foreach ($this->_listeners as $listener)

{ $listener->postSavepointRollback($event); }

}

/**
* preSavepointRollback
* an event invoked before a Doctrine_Connection transaction with savepoint
* is being rolled back
*
* @param Doctrine_Event $event
* @return void
*/
public function preSavepointRollback(Doctrine_Event $event)
{
foreach ($this->_listeners as $listener)

{ $listener->preSavepointRollback($event); }

}

/**
* postSavepointCreate
* an event invoked after a Doctrine_Connection transaction with savepoint
* has been started
*
* @param Doctrine_Event $event
* @return void
*/
public function postSavepointCreate(Doctrine_Event $event)
{
foreach ($this->_listeners as $listener)

{ $listener->postSavepointCreate($event); }

}

/**
* preSavepointCreate
* an event invoked before a Doctrine_Connection transaction with savepoint
* is being started
*
* @param Doctrine_Event $event
* @return void
*/
public function preSavepointCreate(Doctrine_Event $event)
{
foreach ($this->_listeners as $listener)

{ $listener->preSavepointCreate($event); }

}
// @end

Please update.

Loops

[DC-243] Passing $options to Doctrine_Core::generateModelsFromDb() From Doctrine_Cli()

Tue, 17 Nov 2009 20:19:02 +0000

Here the issue when I create models from existing database.
I use doctrine-cli to do that, and make some configuration like this

Array
(
[connection_string] => mysql://user:password@localhost/db
[data_fixtures_path] => /data/fixtures/path
[models_path] => /models/path
[migrations_path] => /migrations/path
[sql_path] => /sql/path
[yaml_schema_path] => /yaml/schema/path
[generate_models_options] => Array
(
[generateBaseClasses] => 1
[classPrefixFiles] => Model
[classPrefix] => Model
[baseClassPrefix] => Base_
)
)

But, when I run the command with like this
php doctrine-cli.php generate-models-db

All models is created, but not follow the configuration.
I have look the code and find the real problem is options argument didn't pass when call Doctrine_Core::generateModelsFromDb()

Regards

Dolly

[DC-242] Linking (re-linking) o/m2m relations with array of ids in Doctrine_Record fromArray stopped working

Tue, 17 Nov 2009 21:20:10 +0000

class User extends Doctrine_Record
{
	public function setTableDefinition()
	{
		$this->hasColumn('username', 'string', 64, array('notnull' => true));
		$this->hasColumn('password', 'string', 128, array('notnull' => true));
	}
	
	public function setUp()
	{
		$this->hasMany('Role as Roles', array('local' => 'id_user', 'foreign' => 'id_role', 'refClass' => 'UserRole'));
		$this->actAs('SoftDelete');
	}
}

class Role extends Doctrine_Record
{
	public function setTableDefinition()
	{
		$this->hasColumn('name', 'string', 64);
	}
	
	public function setUp()
	{
		$this->hasMany('User as Users', array('local' => 'id_role', 'foreign' => 'id_user', 'refClass' => 'UserRole'));
	}
}

class UserRole extends Doctrine_Record
{
	public function setTableDefinition()
	{
		$this->hasColumn('id_user', 'integer', null, array('primary' => true));
		$this->hasColumn('id_role', 'integer', null, array('primary' => true));
	}
	
	public function setUp()
	{
		$this->hasOne('User', array('local' => 'id_user', 'foreign' => 'id', 'onDelete' => 'CASCADE'));
		$this->hasOne('Role', array('local' => 'id_role', 'foreign' => 'id', 'onDelete' => 'CASCADE'));
	}
}


$role = new Role();
$role->name = 'publisher';
$role->save();

$role = new Role();
$role->name = 'reviewer';
$role->save();

$role = new Role();
$role->name = 'mod';
$role->save();

$user = new User();
$user->fromArray(array(
	'username' => 'test',
	'password' => 'test',
	'Roles' => array(1, 2, 3),
));
$user->save();

//--------------------- here goes the failure
$user->fromArray(array(
	'Roles' => array(1, 3),
));
$user->save();

After a User is saved once with some roles, it cannot be saved here with another combination of roles, consisting of some roles which was saved to the User before.
In short, ->unlink() in fromArray method does not schedule a deletion of old relations before adding new ones on save, like it was in previous versions of Doctrine.
Hence the primary key constraint is violated and the exception is thrown:

SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '2-1' for key 1
#0 D:\projekty\cms-trunk\cms\library\Doctrine\Doctrine\Connection\Statement.php(269): Doctrine_Connection->rethrowException(Object(PDOException), Object(Doctrine_Connection_Statement))
#1 (...)\library\Doctrine\Doctrine\Connection.php(1032): Doctrine_Connection_Statement->execute(Array)
#2 (...)\library\Doctrine\Doctrine\Connection.php(677): Doctrine_Connection->exec('INSERT INTO `us...', Array)
#3 (...)\library\Doctrine\Doctrine\Connection\UnitOfWork.php(631): Doctrine_Connection->insert(Object(Doctrine_Table), Array)
#4 (...)\library\Doctrine\Doctrine\Connection\UnitOfWork.php(562): Doctrine_Connection_UnitOfWork->processSingleInsert(Object(UserRole))
#5 (...)\library\Doctrine\Doctrine\Connection\UnitOfWork.php(81): Doctrine_Connection_UnitOfWork->insert(Object(UserRole))
#6 (...)\library\Doctrine\Doctrine\Connection\UnitOfWork.php(452): Doctrine_Connection_UnitOfWork->saveGraph(Object(UserRole))
#7 (...)\library\Doctrine\Doctrine\Connection\UnitOfWork.php(137): Doctrine_Connection_UnitOfWork->saveAssociations(Object(User))
#8 (...)\library\Doctrine\Doctrine\Record.php(1690): Doctrine_Connection_UnitOfWork->saveGraph(Object(User))
#9 (...)\test3.php(124): Doctrine_Record->save()
#10

{main}

[DC-241] Count query with join o2m + limit + where throws an exception

Tue, 17 Nov 2009 23:08:51 +0000

class Poll extends Doctrine_Record
{
	public function setTableDefinition()
	{
		$this->setTableName('module_polls');
		
		$this->hasColumn('id_category', 'integer', null, array('notnull' => true));
		$this->hasColumn('question', 'string', 256);
	}
	
	public function setUp()
	{
		$this->hasMany('PollAnswer as Answers', array('local' => 'id', 'foreign' => 'id_poll', 'orderBy' => 'position'));
	}
}

class PollAnswer extends Doctrine_Record
{
	public function setTableDefinition()
	{
		$this->setTableName('module_polls_answers');
		
		$this->hasColumn('id_poll', 'integer', null, array('notnull' => true));
		$this->hasColumn('answer', 'string', 256);
		$this->hasColumn('votes', 'integer', null, array('notnull' => true, 'default' => 0));
		$this->hasColumn('position', 'integer');
	}
	
	public function setUp()
	{
		$this->hasOne('Poll', array('local' => 'id_poll', 'foreign' => 'id', 'onDelete' => 'CASCADE'));
	}
}

$query = Doctrine_Query::create()
	->from('Poll p')
	->leftJoin('p.Answers pa')
	->where('id_category = ?', 1)
	->limit(10);

print_r($query->count());

This fails with the following exception being thrown:

SQLSTATE[42000]: Syntax error or access violation: 1064 Something is wrong in your syntax obok '? LIMIT 10' w linii 1. Failing Query: "SELECT DISTINCT `m3`.`id` FROM `module_polls` `m3` LEFT JOIN `module_polls_answers` `m4` ON `m3`.`id` = `m4`.`id_poll` WHERE `m3`.`id_category` = ? LIMIT 10"
#0 (...)\library\Doctrine\Doctrine\Connection.php(1015): Doctrine_Connection->rethrowException(Object(PDOException), Object(Doctrine_Connection_Mysql), 'SELECT DISTINCT...')
#1 (...)\library\Doctrine\Doctrine\Query.php(1246): Doctrine_Connection->execute('SELECT DISTINCT...', Array)
#2 (...)\library\Doctrine\Doctrine\Query.php(1106): Doctrine_Query->buildSqlQuery()
#3 (...)\library\Doctrine\Doctrine\Query\Abstract.php(1126): Doctrine_Query->getSqlQuery(Array)
#4 (...)\library\Doctrine\Doctrine\Query\Abstract.php(1088): Doctrine_Query_Abstract->_getDqlCallbackComponents(Array)
#5 (...)\library\Doctrine\Doctrine\Query.php(1126): Doctrine_Query_Abstract->_preQuery()
#6 (...)\library\Doctrine\Doctrine\Query.php(1106): Doctrine_Query->buildSqlQuery()
#7 (...)\library\Doctrine\Doctrine\Query.php(1995): Doctrine_Query->getSqlQuery()
#8 (...)\library\Doctrine\Doctrine\Query.php(2091): Doctrine_Query->getCountSqlQuery()
#9 (...)\test2.php(123): Doctrine_Query->count()
#10

{main}

You can see the generated query which is wrong, moreover it is passed with an empty params array().
Without either where condition or limit removed, it works. With a combo of both of them it fails.

E.g. without where condition the following query is generated:
SELECT COUNT AS `num_results` FROM (SELECT `m`.`id` FROM `module_polls` `m` LEFT JOIN `module_polls_answers` `m2` ON `m`.`id` = `m2`.`id_poll` GROUP BY `m`.`id`) `dctrn_count_query`

[DC-240] Automatic ordering through orderBy in many-to-many relation definition issue

Tue, 17 Nov 2009 20:08:07 +0000

I would like to utilize new nice feature in Doctrine 1.2, namely automatic ordering of relations using orderBy property in relation definition. However I see small flaw with m2m relations:

class User extends Doctrine_Record
{
	public function setTableDefinition()
	{
		$this->setTableName('core_users');
		
		$this->hasColumn('username', 'string', 64, array('notnull' => true));
		$this->hasColumn('password', 'string', 128, array('notnull' => true));
	}
	
	public function setUp()
	{
		$this->hasMany('Role as Roles', array('local' => 'id_user', 'foreign' => 'id_role', 'refClass' => 'UserRole', 'orderBy' => 'position'));
		$this->actAs('SoftDelete');
	}
}

class Role extends Doctrine_Record
{
	public function setTableDefinition()
	{
		$this->setTableName('core_roles');
		
		$this->hasColumn('name', 'string', 64);
	}
	
	public function setUp()
	{
		$this->hasMany('User as Users', array('local' => 'id_role', 'foreign' => 'id_user', 'refClass' => 'UserRole'));
		$this->hasMany('Role as Parents', array('local' => 'id_role_child', 'foreign' => 'id_role_parent', 'refClass' => 'RoleReference', 'orderBy' => 'position'));
		$this->hasMany('Role as Children', array('local' => 'id_role_parent', 'foreign' => 'id_role_child', 'refClass' => 'RoleReference'));
	}
}

class UserRole extends Doctrine_Record
{
	public function setTableDefinition()
	{
		$this->setTableName('core_users_roles');
		
		$this->hasColumn('id_user', 'integer', null, array('primary' => true));
		$this->hasColumn('id_role', 'integer', null, array('primary' => true));
		$this->hasColumn('position', 'integer', null, array('notnull' => true));
	}
	
	public function setUp()
	{
		$this->hasOne('User', array('local' => 'id_user', 'foreign' => 'id', 'onDelete' => 'CASCADE'));
		$this->hasOne('Role', array('local' => 'id_role', 'foreign' => 'id', 'onDelete' => 'CASCADE'));
	}
}

class RoleReference extends Doctrine_Record
{
	public function setTableDefinition()
	{
		$this->setTableName('core_roles_reference');
		
		$this->hasColumn('id_role_parent', 'integer', null, array('primary' => true));
		$this->hasColumn('id_role_child', 'integer', null, array('primary' => true));
		$this->hasColumn('position', 'integer', null, array('notnull' => true));
	}
	
	public function setUp()
	{
		$this->hasOne('Role as Parent', array('local' => 'id_role_parent', 'foreign' => 'id', 'onDelete' => 'CASCADE'));
		$this->hasOne('Role as Child', array('local' => 'id_role_child', 'foreign' => 'id', 'onDelete' => 'CASCADE'));
	}
}

$query = Doctrine_Query::create()
	->from('User u')
	->leftJoin('u.Roles r')
	->orderBy('username ASC')
	->execute();

Results in following query being generated:

SELECT `c`.`id` AS `c__id`, `c`.`username` AS `c__username`, `c`.`password` AS `c__password`, `c`.`deleted_at` AS `c__deleted_at`, `c2`.`id` AS `c2__id`, `c2`.`name` AS `c2__name` FROM `core_users` `c` LEFT JOIN `core_users_roles` `c3` ON (`c`.`id` = `c3`.`id_user`) LEFT JOIN `core_roles` `c2` ON `c2`.`id` = `c3`.`id_role` WHERE (`c`.`deleted_at` IS NULL) ORDER BY `c`.`username` ASC, position

As you can see, 'position' column at the end is not prefixed by an alias (should be `c3` here), which may arise problems when there are more than one column of that name used within tables used in the query.

Same with lazy-loading:

$query = Doctrine_Query::create()
	->from('User u')
	->orderBy('username ASC');
	
$result = $query->execute();
foreach($result as $item)
{
	echo count($item->Roles);
}

SELECT `c`.`id` AS `c__id`, `c`.`name` AS `c__name`, `c2`.`id_user` AS `c2__id_user`, `c2`.`id_role` AS `c2__id_role`, `c2`.`position` AS `c2__position` FROM `core_roles` `c` LEFT JOIN `core_users_roles` `c2` ON `c`.`id` = `c2`.`id_role` WHERE (`c2`.`id_user` IN (?)) ORDER BY position

[DC-239] Doctrine_Query::getCountQuery() produces subquery if the query contains joins

Wed, 18 Nov 2009 01:46:20 +0000

Is there a reason why Doctrine_Query::getCountQuery() produces subquery only if there is joins or having clause.

Its not good for high performance environment like ours: it makes the temp file disk full and could kill the server performance.

Let's the developer the freedom to decide what the count method should return: I prefer it as option like that:

public function count($withSubquery = false)

{ ... }

But I want to know the reason why subquery.

[DC-238] Doctrine_Query::count() doesn't follow useResultCache

Tue, 17 Nov 2009 21:38:16 +0000

The count method of Doctrine_Query doesn't follow useResultCache option, so I have to make loop away with fetchOne or something like that use "execute()".

So I fixed this issue, see the attachment for details!

[DC-228] Doctrine_Record::fromArray() may fail when input contains related component IDs.

Tue, 17 Nov 2009 21:20:16 +0000

I try to explain through an example.

Data model:
===========

User — UserGroup — Group // many-to-many relation

Example code:
=============

$user = new User;
$user->name = 'user1';
$user->Group[0]->name = 'group1'; // id => 1
$user->Group[1]->name = 'group2'; // id => 2
$user->save();

$group3 = new Group;
$group3->name = 'group3';
$group3->save(); // id => 3

// edit user in an HTML form, and select all groups with checkboxes
// $_POST = array('name' => 'user1', 'Group' => array(1, 2, 3))
$user->fromArray($_POST);
$user->save(); // fail, unique constraint violation

Reason:
=======

Doctrine_Record::fromArray() calls unlink() in line 1948:

$this->unlink($key, array(), false);

unlink() fills _pendingUnlinks variable with User's Group relation IDs in line 2418:

if ( ! $ids) {
    $ids = $allIds;
}
foreach ($ids as $id) {
    $this->_pendingUnlinks[$alias][$id] = true;
}

So _pendingUnlinks[$alias] will be an array.

Then fromArray() calls link() in line 1949:

foreach ($value as $id) {
    $this->link($key, $id, false);
}

link() remove new IDs (1, 2 and 3) from _pendingUnlinks[$alias] in line 2496:

foreach ($ids as $id) {
    if (isset($this->_pendingUnlinks[$alias][$id])) {
        unset($this->_pendingUnlinks[$alias][$id]);
    }
}

So _pendingUnlinks[$alias] will be an empty array.

When we save user, this code is executed in Doctrine_Connection_UnitOfWork::saveGraph() (in line 103):

foreach ($record->getPendingUnlinks() as $alias => $ids) {
    if ($ids === false) {
        $record->unlinkInDb($alias, array());
    } else if ($ids) {
        $record->unlinkInDb($alias, array_keys($ids));
    }
}

BUT

Because _pendingUnlinks[$alias] is an empty array, neither IF branch will be executed, so relations won't be deleted.

And then Doctrine try to insert new relations, but it will fail because it will violate existed primary keys in UserGroup.

In an earlier version of Doctrine-1.2, the unlink() method set _pendingUnlinks[$alias] to false if $ids attribute was an empty array, so in saveGraph() all relations were deleted before new relations inserted.

/Sorry for my poor English/

[DC-198] refreshRelated inserts records when One-to-One relationships are used and a related record is deleted

Thu, 19 Nov 2009 20:11:00 +0000

When one-to-one relationship is used and a refreshRelated is performed after a delete a new record is inserted with NULL values.

schema.yml

Email:
columns:
user_id: integer
address: string(150)
relations:
User:
local: user_id
foreign: id
type: one
foreignType: one
foreignAlias: email

User:
columns:
name: string(150)

Sample code

Create.php - create the user/email

$u = new User();
$u->name = 'Test';
$u->email->address = 'foo@bar.com';
$u->save();

Delete.php - remove the email

$u = Doctrine_Query::create()~~>from('User')~~>fetchOne();
$u->name = 'Test2' ;
$u->email->delete();
$u->refreshRelated('email');
$u->save();

The profile (of Delete.php)

connect 0.005211

query 0.000493
SELECT u.id AS u_id, u.name AS u_name FROM user u
prepare 0.000028
SELECT e.id AS e_id, e.user_id AS euser_id, e.address AS e_address FROM email e WHERE e.user_id = ?
execute 0.001086
SELECT e.id AS e_id, e.user_id AS euser_id, e.address AS e_address FROM email e WHERE e.user_id = ?
Array
(
[0] => 1
)
Total time: 0.00681829452515
fetch 0.000024
SELECT e.id AS e_id, e.user_id AS euser_id, e.address AS e_address FROM email e WHERE e.user_id = ?
fetch 0.000016
SELECT e.id AS e_id, e.user_id AS euser_id, e.address AS e_address FROM email e WHERE e.user_id = ?
begin 0.000147

prepare 0.000019
DELETE FROM email WHERE id = ?
execute 0.000253
DELETE FROM email WHERE id = ?
Array
(
[0] => 1
)
Total time: 0.00727772712708
commit 0.001120

prepare 0.000021
SELECT e.id AS e_id, e.user_id AS euser_id, e.address AS e_address FROM email e WHERE e.user_id = ?
execute 0.000333
SELECT e.id AS e_id, e.user_id AS euser_id, e.address AS e_address FROM email e WHERE e.user_id = ?
Array
(
[0] => 1
)
Total time: 0.0087513923645
fetch 0.000017
SELECT e.id AS e_id, e.user_id AS euser_id, e.address AS e_address FROM email e WHERE e.user_id = ?
begin 0.000104

prepare 0.000020
UPDATE user SET name = ? WHERE id = ?
execute 0.000291
UPDATE user SET name = ? WHERE id = ?
Array
(
[0] => Test2
[1] => 1
)
Total time: 0.00918316841125
prepare 0.000021
INSERT INTO email (id, user_id, address) VALUES (?, ?, ?)
execute 0.000193
INSERT INTO email (id, user_id, address) VALUES (?, ?, ?)
Array
(
[0] =>
[1] =>
[2] =>
)
Total time: 0.00939726829529
commit 0.000823

a var_dump of $u->toArray() after refreshRelated shows

array(3) {
["id"]=>
string(1) "1"
["name"]=>
string(7) "Test2"
["email"]=>
array(3)

{ ["id"]=> string(1) "1" ["user_id"]=> string(1) "1" ["address"]=> string(8) "foo@bar.com" }

}

And in the database I have null values for the email record.

This does not happen if I change the relationship to hasMany

[DC-189] When applying changes to n:n relations that contain partially same data DELET is not fired

Tue, 17 Nov 2009 21:20:26 +0000

When changing n:n relations in a way that the new dataset contains relations that were present in the previous state (as in adding a relation to existing ones) DELETE is not fired before insert is done. In the example below I simply reapply an existing relation. This is done by Record::synchronizeWithArray().

Result:

Doctrine_Connection_Mysql_Exception: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '4-14' for key 'PRIMARY'

SQL Query pre r6611:
6 Query START TRANSACTION
6 Query UPDATE `structure` SET `user_create_id` = NULL, `user_modify_id` = NULL, `updated_at` = '2009-11-04 23:38:02' WHERE `id` = '4'
6 Query DELETE FROM `structure_category` WHERE (`structure_id` = '4')
6 Query INSERT INTO `structure_category` (`category_id`, `structure_id`) VALUES ('5', '4')
6 Query COMMIT

SQL QUERY with r6611:
5 Query START TRANSACTION
5 Query UPDATE `structure` SET `user_create_id` = NULL, `user_modify_id` = NULL, `updated_at` = '2009-11-04 23:36:33' WHERE `id` = '4'
5 Query INSERT INTO `structure_category` (`category_id`, `structure_id`) VALUES ('5', '4')
5 Query ROLLBACK

[DC-140] _createLimitSubquery generates a query with error ORA-00904

Wed, 18 Nov 2009 01:44:06 +0000

If I create a limited query with Oracle sometimes it generates an invalid query, giving ORA-00904 errror.

A simple DQL query like this:
$q = Doctrine_Query::create()
->select("$i.nome, $s.denominazione")
->from("Istituto $i")
->leftJoin("$i.Sedi $s");

creates this intermediate sql:
SELECT
"i"."id" AS "i__id",
"i"."nome" AS "i__nome",
"s"."id" AS "s__id",
"s"."denominazione" AS "s__denominazione"
FROM "istituto" "i"
LEFT JOIN "sede" "s" ON "i"."id" = "s"."id_istituto"

that passed to Doctrine_Pager generates this query:
SELECT "i"."id" AS "i__id",
"i"."nome" AS "i__nome",
"s"."id" AS "s__id",
"s"."denominazione" AS "s__denominazione"
FROM "istituto" "i"
LEFT JOIN "sede"
"s" ON "i"."id" = "s"."id_istituto"

WHERE "i"."id" IN (SELECT a."id" FROM (SELECT DISTINCT "i2"."id" FROM "istituto" "i") a WHERE ROWNUM <= 10)

The problem is in "i2" referenced in the inner SELECT statement. This inner query hasn't been correctly parsed, while the column has been aliased, the table name has an incorrect alias.

The problem is that the injected sql code necessary to pagination has some space missing inside the parenthesis, thus the tokenizer misses to alias the table name.
Here it follows a simple patch:

--- Oracle.php  2009-10-26 17:04:17.000000000 +0100
+++ Oracle-orig.php     2009-09-25 19:12:10.000000000 +0200
@@ -103,13 +103,13 @@
                 $column = $column === null ? '*' : $this->quoteIdentifier($column);
                 if ($offset > 0) {
                     $min = $offset + 1;
+                    $query = 'SELECT b.'.$column.' FROM ( '.
+                                 'SELECT a.*, ROWNUM AS doctrine_rownum FROM ( '
+                                   . $query . ' ) a '.
+                              ' ) b '.
-                    $query = 'SELECT b.'.$column.' FROM ('.
-                                 'SELECT a.*, ROWNUM AS doctrine_rownum FROM ('
-                                   . $query . ') a '.
-                              ') b '.
                               'WHERE doctrine_rownum BETWEEN ' . $min .  ' AND ' . $max;
                 } else {
+                    $query = 'SELECT a.'.$column.' FROM ( ' . $query .' ) a WHERE ROWNUM <= ' . $max;
-                    $query = 'SELECT a.'.$column.' FROM (' . $query .') a WHERE ROWNUM <= ' . $max;
                 }
             }
         }

[DC-138] WHERE LIKE doesn't work on JOIN SELECTs.

Wed, 18 Nov 2009 22:07:55 +0000

WHERE LIKE doesn't work on JOIN SELECTs.
Params are not passed to the query.

Reproduce code:

	public function addSearchTerm(Doctrine_Query $query, array $fields)
	{
		$phrase = $this->getRequest()->getPost('query');
		if(!empty($phrase) && !empty($fields))
		{
			$phrase = '%'. $phrase .'%';
			$searchString = array();
			$searchParams = array();
			foreach($fields As $field)
			{
				$searchString[] = "$field LIKE ?";
				$searchParams[] = $phrase;
			} 
			$searchString = implode(' OR ', $searchString);
			
			$query->andWhere($searchString, $searchParams);
		}	
	}

$this->addSearchTerm($query, array(
			'username',
			'email',
			'last_login_ip',
			'Data.title',
			'Data.first_name',
			'Data.last_name',
			'Data.company',
			'Roles.name'
		));

Result attached as screenshot.

Query produced is correct, however '?' are not replaced with params from the array.

Error message is:
Doctrine_Connection_Mysql_Exception: SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '? OR `c8`.`email` LIKE ? OR `c8`.`last_login_ip` LIKE ? OR `c9`.`title` LIKE ? O' at line 1

Query passed to MySQL by Doctrine:
SELECT DISTINCT `c8`.`id` FROM `core_users` `c8` LEFT JOIN `core_users_data` `c9` ON `c8`.`id` = `c9`.`id_user` LEFT JOIN `core_users_roles` `c11` ON (`c8`.`id` = `c11`.`id_user`) LEFT JOIN `core_roles` `c10` ON `c10`.`id` = `c11`.`id_role` LEFT JOIN `core_users_roles` `c13` ON (`c8`.`id` = `c13`.`id_user`) LEFT JOIN `core_roles` `c12` ON `c12`.`id` = `c13`.`id_role` LEFT JOIN `core_users_roles` `c14` ON `c12`.`id` = `c14`.`id_role` WHERE (`c8`.`username` LIKE ? OR `c8`.`email` LIKE ? OR `c8`.`last_login_ip` LIKE ? OR `c9`.`title` LIKE ? OR `c9`.`first_name` LIKE ? OR `c9`.`last_name` LIKE ? OR `c9`.`company` LIKE ? OR `c10`.`name` LIKE ?) ORDER BY `c8`.`username` ASC, `c14`.`position` LIMIT 20

[DC-62] decimal fields are rounded to 6 decimal places on save()

Wed, 14 Apr 2010 17:03:42 +0000

Decimal fields with high precision are rounded down to 6 decimal places prior to inserting into database on Doctrine_Record->save()

For example,
if you define a column to represent decimal numbers with 12 digits following the decimal:

MySet:
columns:
p_value:

{ type: decimal(18), scale: 12 }

Then create a new doctrine record object and save:

$record = new MySet();
$record['p_value'] = 5.012345678912;
$record->save();

The value in the database at this point has been rounded down to 6 decimal places:
value = 5.012346000000

print_r($record->toArray()) shows the correct value to 12 decimal places

[DC-37] Performing JOIN and ORDERBY and using the PAGER in MSSQL results in bad query

Mon, 23 Nov 2009 23:25:49 +0000

When adding an ORDER BY and a JOIN to a query when using MSSQL as the back end, an incomplete query is produced with duplicated ORDER BY sections.

$query = Doctrine_Query::create()
            ->from('User u')
            ->leftJoin('u.Roles')
            ->addOrderBy('u.lastname, u.firstname');

$pager = new Doctrine_Pager($query, $page, $per_page);

return array(
    'pager' => $pager,
    'results' => $pager->execute(),
);

Produces

SELECT 
	[u].[id] AS [u__id], 
	[u].[username] AS [u__username], 
	[u].[password] AS [u__password], 
	[u].[firstname] AS [u__firstname], 
	[u].[lastname] AS [u__lastname], 
	[u].[email] AS [u__email], 
	[u].[phone] AS [u__phone], 
	[u].[data] AS [u__data], 
	[u].[created_at] AS [u__created_at], 
	[u].[updated_at] AS [u__updated_at], 
	[u].[deleted_at] AS [u__deleted_at], 
	[r].[id] AS [r__id], 
	[r].[name] AS [r__name] 
FROM 
	[users] [u] 
	LEFT JOIN [user_has_role] [u2] ON ([u].[id] = [u2].[user_id]) 
	LEFT JOIN [role] [r] ON [r].[id] = [u2].[role_id] 
WHERE 
	[u].[id] IN (
		SELECT * FROM (
			SELECT TOP 1 * FROM (
				SELECT 
					DISTINCT TOP 1 [u].[id] 
				FROM 
					[users] [u] 
					LEFT JOIN [user_has_role] [u2] ON ([u].[id] = [u2].[user_id]
			) 
			LEFT JOIN [role] [r] ON [r].[id] = [u2].[role_id] 
			ORDER BY 
				[u].[lastname], [u].[firstname]
		) AS [inner_tbl] 
		ORDER BY 
			[inner_tbl].[ DESC, [inner_tbl].[ DESC
	) AS [outer_tbl] 
ORDER BY 
	[outer_tbl].[ asc, [outer_tbl].[ asc) 
ORDER BY 
	[u].[lastname], [u].[firstname]

Particularly, notice in the first ORDER BY blocks the incomplete clauses (e.g. [outer_tbl].[ )

*SCHEMA*

Role:
  columns:
    id:
      primary: true
      autoincrement: true
      type: integer
      notnull: true
    name:
      type: string(100)
      notnull: true
  relations:
    Users:
      foreignAlias: Roles
      class: User
      refClass: UserHasRole

UserHasRole:
  columns:
    user_id:
      type: integer
      primary: true
      notnull: true
    role_id:
      type: integer
      primary: true
      notnull: true
  relations:
    User:
      local: user_id
      foreign: id
    Role:
      local: role_id
      foreign: id

User:
  tableName: users
  actAs: [Timestampable, Softdelete]
  columns:
    id:
      primary: true
      autoincrement: true
      type: integer
      notnull: true
    username:
      type: string(255)
      notnull: true
    password:
      type: string(255)
      notnull: true
    firstname:
      type: string(255)
    lastname:
      type: string(255)
    email:
      type: string(255)
      email: true
    phone:
      type: string(20)
    data:
      type: string
  indexes:
    user_index:
      fields: [username]
      type: unique

I will be digging around to find out what the problem is later, and start playing with the 1.2 branch to see if the problem exists there. If someone could help me and point me in a good direction to start looking that would be great