[DBAL-1080] [GH-748] minor phpdoc fixes in the remaining files Created: 16/Dec/14  Updated: 19/Dec/14  Resolved: 19/Dec/14

Status: Resolved
Project: Doctrine DBAL
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Documentation Priority: Minor
Reporter: Doctrine Bot Assignee: Marco Pivetta
Resolution: Incomplete Votes: 0
Labels: None


 Description   

This issue is created automatically through a Github pull request on behalf of acrobat:

Url: https://github.com/doctrine/dbal/pull/748

Message:

This pr fixes the remaining phpcs issues left after #746 & #747



 Comments   
Comment by Doctrine Bot [ 19/Dec/14 ]

A related Github Pull-Request [GH-748] was closed:
https://github.com/doctrine/dbal/pull/748





[DBAL-1089] [GH-753] Add a unit test for broken backslashes on MySQL in LIKE Created: 19/Dec/14  Updated: 19/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 1
Labels: None


 Description   

This issue is created automatically through a Github pull request on behalf of nickvergessen:

Url: https://github.com/doctrine/dbal/pull/753

Message:






[DBAL-1084] MySQL DateTime fractional seconds exception Created: 18/Dec/14  Updated: 19/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: None
Affects Version/s: 2.4.2
Fix Version/s: None
Security Level: All

Type: New Feature Priority: Major
Reporter: Jachim Coudenys Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None
Environment:

MySQL 5.6.4+



 Description   

Since MySQL 5.6.4 it is possible to include fractional seconds in datetime/timestamp fields.

Doctrine cannot handle this at the moment, and it is not clear how this can be resolved.

Doctrine\DBAL\Types\ConversionException(#67): Could not convert database value "2014-12-16 17:06:38.385" to Doctrine Type datetime. Expected format: Y-m-d H:i:s
  #0 /var/www/vendor/doctrine/dbal/lib/Doctrine/DBAL/Types/DateTimeType.php(67): Doctrine\DBAL\Types\ConversionException::conversionFailedFormat('2014-12-16 17:0...', 'datetime', 'Y-m-d H:i:s')
  #1 /var/www/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/AbstractHydrator.php(330): Doctrine\DBAL\Types\DateTimeType->convertToPHPValue('2014-12-16 17:0...', Object(Doctrine\DBAL\Platforms\MySqlPlatform))
  #2 /var/www/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php(359): Doctrine\ORM\Internal\Hydration\AbstractHydrator->gatherRowData(Array, Array, Array, Array)
  #3 /var/www/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php(179): Doctrine\ORM\Internal\Hydration\ObjectHydrator->hydrateRowData(Array, Array, Array)
  #4 /var/www/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/AbstractHydrator.php(140): Doctrine\ORM\Internal\Hydration\ObjectHydrator->hydrateAllData()
  #5 /var/www/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php(804): Doctrine\ORM\Internal\Hydration\AbstractHydrator->hydrateAll(Object(Doctrine\DBAL\Driver\PDOStatement), Object(Doctrine\ORM\Query\ResultSetMapping), Array)
  #6 /var/www/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php(574): Doctrine\ORM\AbstractQuery->execute(NULL, 1)
  #7 /var/www/index.php(142): Doctrine\ORM\AbstractQuery->getResult()

The solution for MsSQL (see DBAL-860) cannot be used, because the fractional seconds definition can be different across fields:

  `insert` timestamp NULL DEFAULT NULL,
  `update` timestamp(3) NULL DEFAULT NULL,
  `delete` timestamp(5) NULL DEFAULT NULL,

My current approach is to create a custom type, but I would like to create a solution in Doctrine.

Any thoughts on this?



 Comments   
Comment by Marco Pivetta [ 18/Dec/14 ]

We don't currently support this nor can support a change in the DateTime type in 2.x. If you want support for this, then I suggest using a custom type (as you are doing) or proposing something new for 3.x

Comment by Jachim Coudenys [ 19/Dec/14 ]

For anyone that is having the same issue, I've added my version of the custom mapping in a Github Gist.





[DBAL-1088] [GH-752] Fix error handling restore Created: 19/Dec/14  Updated: 19/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

This issue is created automatically through a Github pull request on behalf of nicolas-grekas:

Url: https://github.com/doctrine/dbal/pull/752

Message:

PHP already handles an internal stack of error handlers






[DBAL-1087] [GH-751] Length of fixed string type (char) is ignored on Postgre schema update Created: 19/Dec/14  Updated: 19/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

This issue is created automatically through a Github pull request on behalf of SenseException:

Url: https://github.com/doctrine/dbal/pull/751

Message:

This PR should fix an issue for the fixed string type columns using PostgreSQL.

I've created a basic Symfony 2.6. Application and using a PostgreSQL 9.3. database. In this dummy project, I've created the following Entity:

``` php
<?php

namespace Acme\DemoBundle\Entity;

use Doctrine\ORM\Mapping AS ORM;

/**

  • @ORM\Entity
  • @ORM\Table(name="test")
    */
    class Test
    {
    /**
  • @ORM\Id
  • @ORM\Column(type="integer", name="id")
  • @ORM\GeneratedValue
    */
    public $id;

/**

  • @ORM\Column(type="string", name="name", length=2, options= {"fixed" = true}

    )
    */
    public $name;
    }
    ```

The table is successfully created after a schema update, but when another schema update is executed without any changes to the entity, an ALTER query is executed:

``` sql
ALTER TABLE test ALTER name TYPE CHAR(2);
```

After I searched in the code of doctrine dbal, I discovered that the length of the char column is not fetched correctly in `PostgreSqlSchemaManager::_getPortableTableColumnDefinition()`. In my example, length results as `null` and will be later set to 255 in the `Comparator`, which isn't the correct length from the database column. The comparation of the schemas results in a change of char-length.

For simplicity, I extended the if-condition in `PostgreSqlSchemaManager` to:

``` php
if (strtolower($tableColumn['type']) === 'varchar' || strtolower($tableColumn['type']) === 'bpchar') {
```

The fix I've contributed does help to parse the length in my case, but when I look into `PostgreSqlPlatform::initializeDoctrineTypeMappings()` there could be other types with the same problem, that are mapped to the doctrine string type.

``` php
// ... part of PostgreSqlPlatform::initializeDoctrineTypeMappings()
'varchar' => 'string',
'interval' => 'string',
'_varchar' => 'string',
'char' => 'string',
'bpchar' => 'string',
'inet' => 'string',
```

I guess we need to cover `char` and the others too.

`PostgreSqlSchemaManagerTest` also doesn't cover existing code. How is this going to be tested? Does the SchemaManagers need some tests in general?






[DBAL-1086] [GH-750] Store the parameters of the chosen connection when using Master/Slave. Created: 19/Dec/14  Updated: 19/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

This issue is created automatically through a Github pull request on behalf of xrash:

Url: https://github.com/doctrine/dbal/pull/750

Message:

When using the MasterSlaveConnection, the internal parameters array has a different structure than the usual Connection. This can cause the following methods to fail: getHost(), getPort(), getUsername() and getPassword().

I changed the MasterSlaveConnection so that it stores the parameters of the chosen connection in a private property. With this change, the methods that used to fail can now retrieve the configuration for the actual chosen connection, which is, in my opinion, clearly the expected behavior.

I chose to create a new private property in MasterSlaveConnection instead of inherit the $_params from Connection in order to not cause any confusion by having two different array structures in the same variable at different situations.

I also expanded the tests for this case.






[DBAL-1085] Custom Type Compare Fails To Generate Correct Migrations Created: 19/Dec/14  Updated: 19/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: 2.0, 2.1, 2.2, 2.3, 2.4
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Nicolas Vanheuverzwijn Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: migrations
Environment:

Everywhere



 Description   

// From doctrine 2.1
public function diffColumn(Column $column1, Column $column2)
{
$changedProperties = array();
if ( $column1->getType() != $column2->getType() )

{ $changedProperties[] = 'type'; }

...
}
The $column1->getType() will return the underlying platform object but the $column2->getType() will return the custom object type.

Because of the way the php compare function works, a custom type will always generate a changed property over the type of a column.

http://stackoverflow.com/questions/26964367/symfony2-doctrine-custom-types-generating-unneeded-migrations/27557785#27557785






[DBAL-1081] Paginator - Query Limit for SQL Server - SqlServerPlatform.php Created: 16/Dec/14  Updated: 18/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: None
Affects Version/s: 2.5
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Maciej Grajcarek Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: dbal, dql, orderBy, sqlserver
Environment:

Windows



 Description   

Hi!
I have a problem with Query results limit when ordering by SUM of a field.

My query looks like this:

            SELECT ypk.name keyword, SUM(ypk.value) total
            FROM YpBundle:YellowPageKeyword ypk
              JOIN ypk.yellowpage yp
              JOIN yp.listing l
            WHERE l.customer = :customerId
              AND ypk.originDate >= :dateFrom
              AND ypk.originDate <= :dateTo
            GROUP BY ypk.nameCrc, ypk.name
            ORDER BY SUM(ypk.value) DESC

First I was catching following error:
message: "[Syntax Error] line 0, col 395: Error: Expected known function, got 'SUM'"
class: Doctrine\ORM\Query\QueryException

It only accourse if SUM is used in ORDER BY clause.

I have registered new class Sum which extends FunctionNode.

Now, query is build and executed but it has an error:

'SELECT * FROM 
     (SELECT y0_.name AS name_0, SUM(y0_.value) AS sclr_1, ROW_NUMBER() OVER (ORDER BY value) DESC) AS doctrine_rownum FROM yellowpage_keywords y0_ INNER JOIN yellowpages y1_ ON y0_.yellowpage_id = y1_.id INNER JOIN listings l2_ ON y1_.listing_id = l2_.id WHERE l2_.customer_id = ? AND y0_.origin_date >= ? AND y0_.origin_date <= ? GROUP BY y0_.name_crc, y0_.name) AS doctrine_tbl WHERE doctrine_rownum BETWEEN 1 AND 10 ORDER BY doctrine_rownum' with params ["111", "2013-01-01 00:00:00.000000", "2014-12-31 23:59:59.000000"]

The line :

ROW_NUMBER() OVER (ORDER BY value) DESC)

should look like

ROW_NUMBER() OVER (ORDER BY SUM(y0_.value) DESC)

In doModifyLimitQuery method I have modified:

                $pattern = sprintf('/%s\.%s\s+(?:AS\s+)?([^,\s)]+)/i', $column['table'], $column['column']);

to:

                $pattern = sprintf('/%s\.%s\s+(?:AS\s+)?([^,\s)]+)/i', str_replace('(', '\(', $column['table']), str_replace(')', '\)', $column['column']));

Now preg_match founds matching strings and OVER part of query is build correctly.

I checked other issues about this problem (which are marked as already fixed) and I have no idea why it's not working for me.

Thanks in advance!



 Comments   
Comment by Marco Pivetta [ 17/Dec/14 ]

Seems like a bit of information is missing: Is this issue related to the paginator API or not?

Comment by Maciej Grajcarek [ 17/Dec/14 ]

First of all - thank you for formatting my issue.
Secondly yes - issue is directly connected with paginator API.

Here is a code which should help you to replicate the problem:

            SELECT ypk.name keyword, SUM(ypk.value) total
            FROM YpBundle:YellowPageKeyword ypk
              JOIN ypk.yellowpage yp
              JOIN yp.listing l
            WHERE l.customer = :customerId
              AND ypk.originDate >= :dateFrom
              AND ypk.originDate <= :dateTo
            GROUP BY ypk.nameCrc, ypk.name
            ORDER BY SUM(ypk.value) DESC

        $dql = $this->getEntityManager()->createQuery($query);
        $dql
            ->setParameter('customerId', $customerId)
            ->setParameter('dateFrom', $dateFrom)
            ->setParameter('dateTo', $dateTo);
        $dql->setMaxResults(10);    

        $keywords = $dql->getArrayResult(); 
Comment by Marco Pivetta [ 17/Dec/14 ]

That doesn't involve the paginator, just DQL.

SUM() and computed values are not supported in the ORDER BY clause: you have to select them first. Try:

            SELECT ypk.name keyword, SUM(ypk.value) total
            FROM YpBundle:YellowPageKeyword ypk
              JOIN ypk.yellowpage yp
              JOIN yp.listing l
            WHERE l.customer = :customerId
              AND ypk.originDate >= :dateFrom
              AND ypk.originDate <= :dateTo
            GROUP BY ypk.nameCrc, ypk.name
            ORDER BY total DESC
Comment by Maciej Grajcarek [ 18/Dec/14 ]

If I will do it, it will result in a following query:

SELECT *
FROM (
	SELECT y0_.name AS name_0, SUM(y0_.value) AS sclr_1, ROW_NUMBER() OVER (ORDER BY sclr_1 DESC) AS doctrine_rownum 
	FROM yellowpage_keywords y0_
	INNER JOIN yellowpages y1_ ON y0_.yellowpage_id = y1_.id
	INNER JOIN listings l2_ ON y1_.listing_id = l2_.id
	WHERE l2_.customer_id = 111
		AND y0_.origin_date >= '2014-01-01'
		AND y0_.origin_date <= '2014-12-01'
	GROUP BY y0_.name_crc, y0_.name
) AS doctrine_tbl
WHERE doctrine_rownum BETWEEN 1 AND 10
ORDER BY doctrine_rownum

It's an incorrect query for an SQL Server. Take a look on this part:

ROW_NUMBER() OVER (ORDER BY sclr_1 DESC) AS doctrine_rownum

SQL Server do not support aliasing in OVER clause.

It should look like this, to work:

ROW_NUMBER() OVER (ORDER BY SUM(y0_.value) DESC) AS doctrine_rownum

It looks like this is the copy of this issue: http://www.doctrine-project.org/jira/browse/DBAL-788
I'm on doctrine version 2.5 which has patch from that issue included, but I have no idea why it's not working for me.





[DBAL-1083] [GH-749] [DBAL-1082] Fix SchemaTool does not generate SQL for MySQL unsigned float Created: 18/Dec/14  Updated: 18/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

This issue is created automatically through a Github pull request on behalf of dchesterton:

Url: https://github.com/doctrine/dbal/pull/749

Message:






[DBAL-1082] SchemaTool does not generate SQL for MySQL unsigned float Created: 18/Dec/14  Updated: 18/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Platforms
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Minor
Reporter: Daniel Chesterton Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: dbal, mysql, schematool
Environment:

MySQL



 Description   

The schema update tool does not consider the possibility that MySQL double/float fields can be unsigned.

When running the CLI tool, it recognises that the schemas differ but it doesn't add the appropriate 'UNSIGNED' SQL statement. For example when the database is SIGNED but the entity is marked as UNSIGNED, running the tool with --dump-sql will generate SQL similar to below:

ALTER TABLE tablename CHANGE field field DOUBLE PRECISION NOT NULL;

Running this has no effect on the database and subsequent calls will try to run the same SQL.

I have created a pull request in GitHub which fixes the issue.






[DBAL-1079] [GH-747] minor phpdoc fixes in the schema files Created: 16/Dec/14  Updated: 16/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

This issue is created automatically through a Github pull request on behalf of acrobat:

Url: https://github.com/doctrine/dbal/pull/747

Message:

Some minor phpcs fixes in the dbal schema files






[DBAL-1078] [GH-746] minor phpdoc fixes in the platform files Created: 16/Dec/14  Updated: 16/Dec/14  Resolved: 16/Dec/14

Status: Resolved
Project: Doctrine DBAL
Component/s: Platforms
Affects Version/s: None
Fix Version/s: 2.6
Security Level: All

Type: Documentation Priority: Minor
Reporter: Doctrine Bot Assignee: Marco Pivetta
Resolution: Fixed Votes: 0
Labels: cs, docblock


 Description   

This issue is created automatically through a Github pull request on behalf of acrobat:

Url: https://github.com/doctrine/dbal/pull/746

Message:

Some minor phpcs fixes in the dbal platform files



 Comments   
Comment by Doctrine Bot [ 16/Dec/14 ]

A related Github Pull-Request [GH-746] was closed:
https://github.com/doctrine/dbal/pull/746

Comment by Doctrine Bot [ 16/Dec/14 ]

A related Github Pull-Request [GH-746] was assigned:
https://github.com/doctrine/dbal/pull/746





[DBAL-1077] Query limit for sql server Created: 16/Dec/14  Updated: 16/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Platforms
Affects Version/s: 2.4, 2.5
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: yannick LE LAN Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: dbal, regex, sqlserver
Environment:

sql server



 Description   

On branch master: tests/Doctrine/Tests/DBAL/platforms/AbstractSQLServerPlatformTestCase.php
On branch 2.4: tests/Doctrine/Tests/DBAL/platforms/SqlServerPlatformTest.php

public function testModifyLimitQueryFolcoerr()
{

    $sql = $this->_platform->modifyLimitQuery('SELECT son.label AS Name FROM SqlObjectName son WHERE ( SELECT COUNT(eso.identifier) FROM ExtractedSqlObject eso INNER JOIN ProductionDbName pdn ON eso.ref_ProductionDbName_ID = pdn.identifier AND (pdn.label IN (?, ?)) WHERE eso.ref_SqlObjectName_ID = son.identifier) > 0 ORDER BY son.identifier DESC', 1, 0);

    $this->assertEquals('SELECT * FROM (SELECT son.label AS Name, ROW_NUMBER() OVER (ORDER BY son.identifier DESC) AS doctrine_rownum FROM SqlObjectName son WHERE ( SELECT COUNT(eso.identifier) FROM ExtractedSqlObject eso INNER JOIN ProductionDbName pdn ON eso.ref_ProductionDbName_ID = pdn.identifier AND (pdn.label IN (?, ?)) WHERE eso.ref_SqlObjectName_ID = son.identifier) > 0) AS doctrine_tbl WHERE doctrine_rownum BETWEEN 1 AND 1 ORDER BY doctrine_rownum', $sql);

}

Correction proposed:
on both branches: lib/Doctrine/DBAL/Platforms/SQLServerPlatform.php

protected function doModifyLimitQuery(...)
{
    ...

    #ACTUAL: 
    $selectFromPattern = '/^(\s*SELECT\s+(?:(.*)(?![^(]*\))))\sFROM\s/i';

#CORRECT:

    $selectFromPattern = '/^(\s*SELECT\s+(?:(.*)(?![^(]*\))))\sFROM\s/iU';

explanation: "/U" pattern modifier => http://php.net/manual/en/reference.pcre.pattern.modifiers.php
Ungreedy behavior not susceptible to fall on ?! negative lookahead on parenthesis hunger

  1. has impacts on ORM with setMaxResults (which was failing in my case and made me investigate)


 Comments   
Comment by Marco Pivetta [ 16/Dec/14 ]

Can you send a PR for this change instead?

Comment by yannick LE LAN [ 16/Dec/14 ]

I will do so by the end of the week,





[DBAL-1075] [GH-744] Removed non-phpdoc @internal tags Created: 15/Dec/14  Updated: 16/Dec/14  Resolved: 16/Dec/14

Status: Resolved
Project: Doctrine DBAL
Component/s: None
Affects Version/s: 2.5, 2.4.3
Fix Version/s: 2.6
Security Level: All

Type: Documentation Priority: Minor
Reporter: Doctrine Bot Assignee: Marco Pivetta
Resolution: Fixed Votes: 0
Labels: annotation, docblock


 Description   

This issue is created automatically through a Github pull request on behalf of acrobat:

Url: https://github.com/doctrine/dbal/pull/744

Message:

Removed 2 @internal phpdoc tags as they were about old internal api usage and these methods are public api as @beberlei & @Ocramius confirmed on irc #doctrine

> ocramius: the @internal are old comments that were used to tell information about internal API usage
> ocramius: they are not the @internal of phpdoc, so you are welcome to send a PR to fix that



 Comments   
Comment by Doctrine Bot [ 16/Dec/14 ]

A related Github Pull-Request [GH-744] was assigned:
https://github.com/doctrine/dbal/pull/744

Comment by Doctrine Bot [ 16/Dec/14 ]

A related Github Pull-Request [GH-744] was closed:
https://github.com/doctrine/dbal/pull/744





[DBAL-1076] [GH-745] Added doModifyLimitQuery for SQLServer2012Platform that uses OFFSET... FETCH Created: 16/Dec/14  Updated: 16/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

This issue is created automatically through a Github pull request on behalf of zeroedin-bill:

Url: https://github.com/doctrine/dbal/pull/745

Message:

SQL Server 2012 introduced new syntax for paging records using the OFFSET... FETCH clause. See http://technet.microsoft.com/en-us/library/gg699618%28v=sql.110%29.aspx for details on the specification.

This pull request:

  • Adds doModifyLimitQuery specific to SQLServer2012Platform, using OFFSET ... FETCH instead of ROW_NUMBER() OVER(blah blah)
  • Duplicates tests from SQLServerPlatformTest, using simpler OFFSET ... FETCH syntax

This version of doModifyLimitQuery will be much more robust than the one for SQLServer 2008 and below.






[DBAL-1063] Exceptions from SchemaTool when running with DBAL 2.5.0 Created: 06/Dec/14  Updated: 15/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: flack Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

I'm not entirely sure it it is related to DBAL-1062, but I'm seeing some similar problems since updating to 2.5.0. The problem I'm having occurs only once per table, and only for some tables, so it is kind of hard to debug. What I've found out so far is that it happens when I manually define an index on a column which I name f.x. 'colname_idx', and then later Doctrine wants to add an index to the column automatically (e.g. when it is an association field), in which case it generates an index name like 'IDX_CF2713FD16F4F95B'.

An index with this name already exists (from the last run of the SchemaTool, presumably). The isFullfilledBy function in Doctrine\DBAL\Platforms\AbstractPlatform\Index detects that the both the manually-named index and the automatically-named one are identical and thus ignores them in Doctrine\DBAL\Schema\Table::_addIndex.

When the schema read from the database is compared to the one generated from metadata in Doctrine\DBAL\Schema\Comparator::diffTable, $table1 will list 'colname_idx' and the one from $table2 will say 'IDX_CF2713FD16F4F95B'. So it will be counted as a rename, and the rename SQL (for mysql at least) is

DROP INDEX colname_idx ON tablename
CREATE INDEX IDX_CF2713FD16F4F95B ON tablename (colname)

But as mentioned before, IDX_CF2713FD16F4F95B already exists, so I get

  [Doctrine\DBAL\Exception\DriverException]                                                          
  An exception occurred while executing 'CREATE INDEX IDX_CF2713FD16F4F95B ON tablename (colname)':          
  SQLSTATE[42000]: Syntax error or access violation: 1061 Duplicate key name 'IDX_CF2713FD16F4F95B'

Since the DROP statement before was executed successfully, on the next SchemaTool run, the existing index will be detected correctly for $table1, and thus no rename is issued.



 Comments   
Comment by Steve Müller [ 08/Dec/14 ]

From a first glance this issue is caused by ORM's schema tool which is creating indexes implicitly for unique columns and associations. Seems it doesn't play well with the new index renaming feature from DBAL 2.5 if you also define indexes explicitly that also fullfill those auto generated ones.
Maybe we have to asjust the schema tool to make a certain preference of explicitly defined indexes over auto generated ones if both fullfill the same criteria.

Comment by flack [ 08/Dec/14 ]

Well, making the schema tool smarter is certainly a good idea, since in 2.4 it created duplicate indexes (different name, same function) if I understood your theory correctly.

But there are some things on the DBAL side that I wonder about, e.g. shouldn't dbal do a sanity check before renaming? Or is there one that doesn't trigger because it cannot detect that an index with the target name already exists?

As far as I understood it, the schema read from the database was missing the one index (auto-generated one by SchemaTool), because Doctrine\DBAL\Schema\Table::_addIndex refused to add it on the grounds that it isFullfilledBy the manually-created index. This sounds like a very valid optimization for the case where I plan to write a schema to the database, but it is not at all helpful when creating a representation of an existing database. So I wonder if this case shouldn't be handled differently.

The other thing I'm still unclear about is why the second schema (the one coming from the driver) has the autogenerated index instead of the manual one. I suppose it could be a timing issue, but if the driver and the schematool both were to call _addIndex in the same order, the issue wouldn't even occur. Which might make it difficult to write a unit test for it

Comment by Steve Müller [ 09/Dec/14 ]

flack I did not test anything about this issue yet. It's all just speculation at the moment. What we'll have to do is try reproducing it in a test case. I would propose adding a test case to ORM first as this is your main entry point. Would be awesome if you could PR such a test case that reveals the issue then we can make further inspections. Still not sure whether it's a DBAL or ORM issue or maybe even both...

Comment by flack [ 15/Dec/14 ]

I've added two tests now:

https://github.com/doctrine/dbal/pull/743

The first one shows a discrepancy between Table::getIndexes and SchemaManager::listTableIndexes (which might be the root cause of the problem), and the second one reproduces the exception mentioned above.

BTW: Is there a way to make Jira recognize that a PR belongs to an already existing ticket? Right now, it always seems to create a new issue for each pull request





[DBAL-1074] [GH-743] Add failing unit test related to DBAL-1063 Created: 14/Dec/14  Updated: 14/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

This issue is created automatically through a Github pull request on behalf of flack:

Url: https://github.com/doctrine/dbal/pull/743

Message:

I am not 100% sure, but I think the behavior exposed by this test is one of the factors in DBAL-1063. But even if it wasn't, it is still very confusing that `Table::getIndexes` doesn't return the same thing as `SchemaManager::listTableIndexes` when they are both called on the same table.

I've written the test against the MySQL SM, but I suspect the same might happen with other platforms as well






[DBAL-1068] Microsoft SQL Server issues with ANSI_NULLS=OFF Created: 09/Dec/14  Updated: 13/Dec/14  Resolved: 13/Dec/14

Status: Resolved
Project: Doctrine DBAL
Component/s: Platforms
Affects Version/s: 2.5
Fix Version/s: None
Security Level: All

Type: Bug Priority: Critical
Reporter: man4red Assignee: Steve Müller
Resolution: Invalid Votes: 0
Labels: None
Environment:

Microsoft SQL Server 2012 (11.0.5058)

zf2, doctrine2, skeletonApplication, + modules
zf-commons/zfc-base v0.1.2
zf-commons/zfc-user 1.2.1
zf-commons/zfc-user-doctrine-orm dev-master
bjyoungblood/bjy-authorize dev-master


Attachments: PNG File bug1.png     PNG File bug2.png    

 Description   

So here what I have (see my Environment for more details)

$modules = array(
    'Application', //skeleton
    'DoctrineModule',
    'DoctrineORMModule',
    'DoctrineDataFixtureModule',
    'ZfcBase',
    'ZfcUser',
    'ZfcUserDoctrineORM',
    'BjyAuthorize',
    'User', // Entity copied from BjyAuthorize vendor folder
);
//module/User/Entity/User.php (copied from vendor\bjyoungblood\bjy-authorize\data\User.php.dist)
...
    /**
     * @var string
     * @ORM\Column(type="string", length=255, unique=true, nullable=true)
     */
    protected $username;
...
./vendor/doctrine/doctrine-module/bin/doctrine-module orm:schema-tool:create --dump-sql
CREATE TABLE [users] (id INT IDENTITY NOT NULL, username NVARCHAR(255), email NVARCHAR(255) NOT NULL, displayName NVARCHAR(50), password NVARCHAR(128) NOT NULL, state INT NOT NULL, PRIMARY KEY (id));
CREATE UNIQUE INDEX UNIQ_1483A5E9F85E0677 ON [users] (username) WHERE username IS NOT NULL;
...
and so on...

on create

[Doctrine\ORM\Tools\ToolsException]
Schema-Tool failed with Error 'An exception occurred while executing 'CREATE UNIQUE INDEX UNIQ_1483A5E9F85E0677 ON [users] (username) WHERE username IS NOT NULL':
SQLSTATE[HY000]: General error: 20018 Cannot create index. Object 'users' was created with the following SET options off: 'ANSI_NULLS'. [20018] (severity 16) [CREATE UNIQUE INDEX UNIQ_1483A5E9F85E0677 ON [users] (username) WHERE username IS NOT NULL]' while executing DDL: CREATE UNIQUE INDEX UNIQ_1483A5E9F85E06
77 ON [users] (username) WHERE username IS NOT NULL
....

Maybe this helps:

My database created in MSSQL and has default option ANSI_NULLS = OFF

see my options (bug1.png)

ok, changed it to ANSI_NULLS = ON according to documentation (but for now this sets OFF by default on create database)

got a new exception

[Doctrine\ORM\Tools\ToolsException]
Schema-Tool failed with Error 'An exception occurred while executing 'CREATE UNIQUE INDEX UNIQ_1483A5E9F85E0677 ON [users] (username) WHERE username IS NOT NULL':
SQLSTATE[HY000]: General error: 20018 CREATE INDEX failed because the following SET options have incorrect settings: 'CONCAT_NULL_YIELDS_NULL, ANSI_WARNINGS, ANSI_PADDING'. Verify that SET options are correct for use with indexed views and/or indexes on computed columns and/or filtered indexes and/or query noti
fications and/or XML data type methods and/or spatial index operations. [20018] (severity 16) [CREATE UNIQUE INDEX UNIQ_1483A5E9F85E0677 ON [users] (username) WHERE username IS NOT NULL]' while executing DDL: CREATE UNIQUE INDEX UNIQ_1483A5E9F85E0677 ON [users] (username) WHERE username IS NOT NULL

Then I copied to Microsoft SQL Management Studio code above.
No error!

Then I chcked off this checkbox in SQL Management Studio
(see bug2.png)

and SQL query now failed with almost the same error, but now in Management Studio!

Msg 1934, Level 16, State 1, Line 2
CREATE INDEX failed because the following SET options have incorrect settings: 'CONCAT_NULL_YIELDS_NULL'. Verify that SET options are correct for use with indexed views and/or indexes on computed columns and/or filtered indexes and/or query notifications and/or XML data type methods and/or spatial index operations.

So it seems that problem starts when we set for username unique=true and using MSSQL that trying to create UNIQUE INDEX ... WHERE fieled IS NOT NULL

Some related topics:
ANSI_NULLS
CONCAT_NULL_YIELDS_NULL
ANSI_WARNINGS
ANSI_PADDING
Creating a unique constraint that ignores nulls in SQL Server

Going to move my project to MySQL if this bug are not my fault



 Comments   
Comment by Steve Müller [ 12/Dec/14 ]

This is nothing we can control in Doctrine. According to the documentation http://msdn.microsoft.com/en-us/library/ms189292.aspx there are some requirements server-side that have to be met in order for such unique constraints to work. You will have to configure your server properly.

Comment by man4red [ 12/Dec/14 ]

Yep, thx for answer, but according to the same documentation:

The ANSI_NULLS connection-level option must be set to ON when the CREATE TABLE or ALTER TABLE statement that defines the computed column is executed. The OBJECTPROPERTY function reports whether the option is on through the IsAnsiNullsOn property.

The connection on which the index is created, and all connections trying INSERT, UPDATE, or DELETE statements that will change values in the index, must have six SET options set to ON and one option set to OFF. The optimizer ignores an index on a computed column for any SELECT statement executed by a connection that does not have these same option settings.
The NUMERIC_ROUNDABORT option must be set to OFF, and the following options must be set to ON:
ANSI_NULLS
ANSI_PADDING
ANSI_WARNINGS
ARITHABORT
CONCAT_NULL_YIELDS_NULL
QUOTED_IDENTIFIER
Setting ANSI_WARNINGS to ON implicitly sets ARITHABORT to ON when the database compatibility level is set to 90 or higher.

So my fixtures are created without correct connection-level option

Am I wrong?

Comment by man4red [ 12/Dec/14 ]

I think that problem is still persist.
According to documentation connection must have some options (see http://msdn.microsoft.com/en-us/library/ms189292.aspx and my comment)

Propose to pay more attention to the problem.

Comment by Steve Müller [ 12/Dec/14 ]

Hmmm I suppose that there is something wrong with your setup/configuration then. I just tested this using DBAL + sqlsrv driver + SQL Server 2012 and it adds the unique index on a nullable column without problems. Not sure what's wrong with your client/server setup then. Also I don't see how one would set those settings in the connection handle...
Which database driver are you using? sqlsrv or pdo_sqlsrv?

Comment by man4red [ 12/Dec/14 ]

Sorry for bothering you/
Now I began to suspect that the case in the client
I'll test today booth setup on a diffirent clients

Comment by Steve Müller [ 12/Dec/14 ]

Nevermind
If there truly is a bug we'll fix that somehow but as this is really common core functionality and nobody complained so far I suppose it's really related to your environment. Please let me know if I can close the issue as soon as you have checked your env.
Thanks!

Comment by man4red [ 12/Dec/14 ]

Steve, it seems, that problem persist only with FreeTDS/dblib driver (my env). Probably sqlsrv driver does set this parameters automatically.
Unfortunately there is no other mssql drivers for linux, so situation is very strange. Maybe there is nobody using linux + mssql env? Crzy Also forced to move to pdo_mysql.

Untill dblib not supported for doctrine - we can mark this as invalid.
Thx for your help

Comment by man4red [ 12/Dec/14 ]

Am I right at conclusion that:
1. Linux users can use dblib+mssql and had this issues. To fix this they can set connection-level paramters somehow before ALTER TABLE by querying "SET ANSI_NULLS ON; SET ANSI_PADDING ON; ..." etc...
2. There is only windows users are able to use doctrine DBAL with mssql without any problems thru sqlsrv driver provided by microsoft?
it bothers me in some way

Comment by Steve Müller [ 13/Dec/14 ]

Hehe we could have saved a lot of time if you had mentioned earlier that your are using dblib driver
But nevermind. dblib is not supported by Doctrine as it is an experimental driver and quite buggy. Not even sure how you managed to get DBAL connecting through dblib without a custom driver implementation. Whatsoever, unfortunately you are right, there currently is no officially supported way of connecting with DBAL to a SQL Server using a linux client. And yes only Windows users can do that by using either pdo_sqlsrv or native sqlsrv drivers provided by M$
If you are not bound to a Microsoft SQL Server backend and can use an alternative such as MySQL or any other database vendor I would then advise you to do that. I highly discourage you to start messing with dblib and its bugs/incompatibilities. If you are really bound to SQL Server you'll have to let your PHP application run on a windows machine and utilize one of Microsoft's PHP drivers. Hope that helps for now...
Closing then...





[DBAL-1073] [GH-742] Take care about mariadb platform Created: 12/Dec/14  Updated: 13/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers, Platforms
Affects Version/s: 2.5
Fix Version/s: 2.5.1
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Steve Müller
Resolution: Unresolved Votes: 0
Labels: detection, mariadb, mysql, platform, version


 Description   

This issue is created automatically through a Github pull request on behalf of nlegoff:

Url: https://github.com/doctrine/dbal/pull/742

Message:

Hi,
After upgrading to DBAL 2.5, I got an issue where I could not rename index while migrating because of MariaDB [versioning](http://en.wikipedia.org/wiki/MariaDB#Versioning) which outputs ```10.0.15-MariaDB-1~wheezy ``` as server version.

Because 10.x > 5.7 it loads new features from mysql 5.7 which are not available in mariadb ..



 Comments   
Comment by Doctrine Bot [ 13/Dec/14 ]

A related Github Pull-Request [GH-742] was assigned:
https://github.com/doctrine/dbal/pull/742





Generated at Fri Dec 19 22:54:10 UTC 2014 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.