[DBAL-1020] Postgres and using Schema tool throws cardinality errors Created: 22/Oct/14  Updated: 23/Oct/14

Status: Awaiting Feedback
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: 2.5
Fix Version/s: None
Security Level: All

Type: Bug Priority: Critical
Reporter: Dominic Watson Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: ddl, postgresql

Attachments: PNG File 6ZeW_jJFUbOCx5uGJ2feANtAsZOr72zhDL4szJ6p5VE.png     File pg_catalog_pg_class.csv    

 Description   

Postgres: 9.3.5.0 (Postgres App for OSX) w/ PostGIS extensions
doctrine/common: 2.4.x-dev ae92d076442e27b6910dd86a1292a8867cf5cfe4
doctrine/dbal: dev-master 1c9c24a7e2295b71249ae2a719ce38861fccd551
creof/doctrine2-spatial: https://github.com/intellix/doctrine2-spatial 4023ca8fbe703043012c31d6df26b9bc7b0a972d

It seems every now and again when I come to use the schema-tool I'm getting exceptions which can only be fixed by dropping the database and recreating from scratch.

The following SQL looks to be generated here: \Doctrine\DBAL\Platforms\AbstractPlatform::getListTableForeignKeysSQL

SELECT quote_ident(r.conname) as conname, pg_catalog.pg_get_constraintdef(r.oid, true) as condef
                  FROM pg_catalog.pg_constraint r
                  WHERE r.conrelid =
                  (
                      SELECT c.oid
                      FROM pg_catalog.pg_class c, pg_catalog.pg_namespace n
                      WHERE n.nspname NOT IN ('pg_catalog', 'information_schema', 'pg_toast') AND c.relname = 'state' AND n.nspname = ANY(string_to_array((select replace(replace(setting,'"$user"',user),' ','') from pg_catalog.pg_settings where name = 'search_path'),',')) AND n.oid = c.relnamespace
                  )
                  AND r.contype = 'f'

The full stack trace is as follows:

 ---~---~---~---~---~---~---~---~---~---~---~---~---~---~---~---~---~--
  Dropping database schema...
      ./bin/doctrine-module orm:schema-tool:drop --force --verbose
---~---~---~---~---~---~---~---~---~---~---~---~---~---~---~---~---~--
Dropping database schema...


                                                                                                                                                                                                       
  [Doctrine\DBAL\Exception\DriverException]                                                                                                                                                            
  An exception occurred while executing 'SELECT quote_ident(r.conname) as conname, pg_catalog.pg_get_constraintdef(r.oid, true) as condef                                                              
                    FROM pg_catalog.pg_constraint r                                                                                                                                                    
                    WHERE r.conrelid =                                                                                                                                                                 
                    (                                                                                                                                                                                  
                        SELECT c.oid                                                                                                                                                                   
                        FROM pg_catalog.pg_class c, pg_catalog.pg_namespace n                                                                                                                          
                        WHERE n.nspname NOT IN ('pg_catalog', 'information_schema', 'pg_toast') AND c.relname = 'state' AND n.nspname = ANY(string_to_array((select replace(replace(setting,'"$user"'  
  ,user),' ','') from pg_catalog.pg_settings where name = 'search_path'),',')) AND n.oid = c.relnamespace                                                                                              
                    )                                                                                                                                                                                  
                    AND r.contype = 'f'':                                                                                                                                                              
  SQLSTATE[21000]: Cardinality violation: 7 ERROR:  more than one row returned by a subquery used as an expression                                                                                     
                                                                                                                                                                                                       


Exception trace:
 () at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractPostgreSQLDriver.php:82
 Doctrine\DBAL\Driver\AbstractPostgreSQLDriver->convertException() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php:116
 Doctrine\DBAL\DBALException::driverExceptionDuringQuery() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:833
 Doctrine\DBAL\Connection->executeQuery() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:761
 Doctrine\DBAL\Connection->fetchAll() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:319
 Doctrine\DBAL\Schema\AbstractSchemaManager->listTableForeignKeys() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:284
 Doctrine\DBAL\Schema\AbstractSchemaManager->listTableDetails() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:268
 Doctrine\DBAL\Schema\AbstractSchemaManager->listTables() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:1039
 Doctrine\DBAL\Schema\AbstractSchemaManager->createSchema() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/SchemaTool.php:783
 Doctrine\ORM\Tools\SchemaTool->getDropSchemaSQL() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/SchemaTool.php:727
 Doctrine\ORM\Tools\SchemaTool->dropSchema() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/Console/Command/SchemaTool/DropCommand.php:100
 Doctrine\ORM\Tools\Console\Command\SchemaTool\DropCommand->executeSchemaCommand() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/Console/Command/SchemaTool/AbstractCommand.php:65
 Doctrine\ORM\Tools\Console\Command\SchemaTool\AbstractCommand->execute() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:252
 Symfony\Component\Console\Command\Command->run() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:891
 Symfony\Component\Console\Application->doRunCommand() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:195
 Symfony\Component\Console\Application->doRun() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:126
 Symfony\Component\Console\Application->run() at /Users/dominicwatson/Sites/flatscanner/api/bin/doctrine-module.php:58
 include() at /Users/dominicwatson/Sites/flatscanner/api/bin/doctrine-module:4




                                                                                                                    
  [Doctrine\DBAL\Driver\PDOException]                                                                               
  SQLSTATE[21000]: Cardinality violation: 7 ERROR:  more than one row returned by a subquery used as an expression  
                                                                                                                    


Exception trace:
 () at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:94
 Doctrine\DBAL\Driver\PDOConnection->query() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:830
 Doctrine\DBAL\Connection->executeQuery() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:761
 Doctrine\DBAL\Connection->fetchAll() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:319
 Doctrine\DBAL\Schema\AbstractSchemaManager->listTableForeignKeys() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:284
 Doctrine\DBAL\Schema\AbstractSchemaManager->listTableDetails() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:268
 Doctrine\DBAL\Schema\AbstractSchemaManager->listTables() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:1039
 Doctrine\DBAL\Schema\AbstractSchemaManager->createSchema() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/SchemaTool.php:783
 Doctrine\ORM\Tools\SchemaTool->getDropSchemaSQL() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/SchemaTool.php:727
 Doctrine\ORM\Tools\SchemaTool->dropSchema() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/Console/Command/SchemaTool/DropCommand.php:100
 Doctrine\ORM\Tools\Console\Command\SchemaTool\DropCommand->executeSchemaCommand() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/Console/Command/SchemaTool/AbstractCommand.php:65
 Doctrine\ORM\Tools\Console\Command\SchemaTool\AbstractCommand->execute() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:252
 Symfony\Component\Console\Command\Command->run() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:891
 Symfony\Component\Console\Application->doRunCommand() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:195
 Symfony\Component\Console\Application->doRun() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:126
 Symfony\Component\Console\Application->run() at /Users/dominicwatson/Sites/flatscanner/api/bin/doctrine-module.php:58
 include() at /Users/dominicwatson/Sites/flatscanner/api/bin/doctrine-module:4




                                                                                                                    
  [PDOException]                                                                                                    
  SQLSTATE[21000]: Cardinality violation: 7 ERROR:  more than one row returned by a subquery used as an expression  
                                                                                                                    


Exception trace:
 () at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:92
 PDO->query() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:92
 Doctrine\DBAL\Driver\PDOConnection->query() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:830
 Doctrine\DBAL\Connection->executeQuery() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:761
 Doctrine\DBAL\Connection->fetchAll() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:319
 Doctrine\DBAL\Schema\AbstractSchemaManager->listTableForeignKeys() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:284
 Doctrine\DBAL\Schema\AbstractSchemaManager->listTableDetails() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:268
 Doctrine\DBAL\Schema\AbstractSchemaManager->listTables() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:1039
 Doctrine\DBAL\Schema\AbstractSchemaManager->createSchema() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/SchemaTool.php:783
 Doctrine\ORM\Tools\SchemaTool->getDropSchemaSQL() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/SchemaTool.php:727
 Doctrine\ORM\Tools\SchemaTool->dropSchema() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/Console/Command/SchemaTool/DropCommand.php:100
 Doctrine\ORM\Tools\Console\Command\SchemaTool\DropCommand->executeSchemaCommand() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/Console/Command/SchemaTool/AbstractCommand.php:65
 Doctrine\ORM\Tools\Console\Command\SchemaTool\AbstractCommand->execute() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:252
 Symfony\Component\Console\Command\Command->run() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:891
 Symfony\Component\Console\Application->doRunCommand() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:195
 Symfony\Component\Console\Application->doRun() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:126
 Symfony\Component\Console\Application->run() at /Users/dominicwatson/Sites/flatscanner/api/bin/doctrine-module.php:58
 include() at /Users/dominicwatson/Sites/flatscanner/api/bin/doctrine-module:4


orm:schema-tool:drop [--dump-sql] [-f|--force] [--full-database]


 Comments   
Comment by Marco Pivetta [ 22/Oct/14 ]

What are the contents of pg_catalog.pg_class ?

Comment by Dominic Watson [ 22/Oct/14 ]

Uploaded CSV of that table

Comment by Dominic Watson [ 22/Oct/14 ]

After running the subquery as suggested in IRC:

  SELECT c.oid                                                                                                                                                                   
                        FROM pg_catalog.pg_class c, pg_catalog.pg_namespace n                                                                                                                          
                        WHERE n.nspname NOT IN ('pg_catalog', 'information_schema', 'pg_toast') AND c.relname = 'state' AND n.nspname = ANY(string_to_array((select replace(replace(setting,'"$user"'  
  ,user),' ','') from pg_catalog.pg_settings where name = 'search_path'),',')) AND n.oid = c.relnamespace  

oid
-------
40152
39687

Comment by Marco Pivetta [ 22/Oct/14 ]

Can you run the query:

SELECT
    c.*
FROM
   pg_catalog.pg_class c, pg_catalog.pg_namespace n
WHERE
    n.nspname NOT IN ('pg_catalog', 'information_schema', 'pg_toast')
    AND c.relname = 'state'
    AND n.nspname = ANY(string_to_array((
        select replace(replace(setting,'"$user"', user), ' ', '')
        from pg_catalog.pg_settings
        where name = 'search_path'
    ),','))
    AND n.oid = c.relnamespace
Comment by Dominic Watson [ 22/Oct/14 ]
  relname varchar,
  relnamespace oid,
  reltype oid,
  reloftype oid,
  relowner oid,
  relam oid,
  relfilenode oid,
  reltablespace oid,
  relpages int,
  reltuples real,
  relallvisible int,
  reltoastrelid oid,
  reltoastidxid oid,
  relhasindex bool,
  relisshared bool,
  relpersistence char(1),
  relkind char(1),
  relnatts smallint,
  relchecks smallint,
  relhasoids bool,
  relhaspkey bool,
  relhasrules bool,
  relhastriggers bool,
  relhassubclass bool,
  relispopulated bool,
  relfrozenxid xid,
  relminmxid xid,
  relacl _aclitem,
  reloptions _text

state,2200,40154,0,10,0,40152,0,0,0,0,0,0,true,false,p,r,2,0,false,true,false,true,false,true,6694,1,NULL,NULL
state,39587,39689,0,10,0,39687,0,0,0,0,39694,0,true,false,p,r,15,3,false,true,false,false,false,true,6629,1,NULL,NULL
Comment by Dominic Watson [ 22/Oct/14 ]

My ZF2 onBootstrap as well, in case it changes anything:

Unable to find source-code formatter for language: php. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
<?php
namespace Flatscanner;

use Doctrine\ORM\Mapping\UnderscoreNamingStrategy;
use ZF\Apigility\Provider\ApigilityProviderInterface;
use Zend\Uri\UriFactory;
use Doctrine\DBAL\Types\Type;

class Module implements ApigilityProviderInterface
{
    public function getConfig()
    {
        return include __DIR__ . '/../../config/module.config.php';
    }

    public function onBootstrap($e)
    {
        Type::addType('geometry', 'CrEOF\Spatial\DBAL\Types\GeometryType');
        Type::addType('point', 'CrEOF\Spatial\DBAL\Types\Geometry\PointType');
        UriFactory::registerScheme('chrome-extension', 'Zend\Uri\Uri');

        // Set naming strategy
        $em = $e->getTarget()->getServiceManager()->get('doctrine.entitymanager.orm_default');
        $em->getConnection()->getDatabasePlatform()->registerDoctrineTypeMapping("_text", "text"); // assuming it is a text LOB
        $em->getConfiguration()->setNamingStrategy(new UnderscoreNamingStrategy(CASE_LOWER));
    }

    public function getAutoloaderConfig()
    {
        return array(
            'ZF\Apigility\Autoloader' => array(
                'namespaces' => array(
                    __NAMESPACE__ => __DIR__,
                ),
            ),
        );
    }
}
Comment by Steve Müller [ 23/Oct/14 ]

Most probably also affects 2.4 as the codebase has not changed at the critical places. Possibly 2.3 is also affected by this. Could need a check.





[DBAL-819] Schema-tools does not work on multiple Oracle's schemas Created: 21/Feb/14  Updated: 22/Feb/14

Status: Open
Project: Doctrine DBAL
Component/s: Platforms, Schema Managers
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Antoine Descamps Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: Cli, oracle, orm, schematool
Environment:

DB: Oracle 11g



 Description   

The schema-tools, used via the CLI, is not able to detect schema's changes when working on multiple schemas.

For instance, the ORM is configured with a "global" Oracle's user, having permissions on every schemas. To specify which entities belong to which schema, I've prefixed the table name with the corresponding schema.

When trying to do the following command:

orm:schema-tool:update --dump-sql

Doctrine returns me the following message:

Nothing to update - your database is already in sync with the current entity metadata.

If, instead of using the "global" user in the Doctrine's configuration, I set the user of the specific schema I'm trying to generate a table on based from an entity, it works.



 Comments   
Comment by Steve Müller [ 21/Feb/14 ]

Moved this to DBAL for now. It seems to be related to schema prefixed table names not being evaluated in the platforms when generating the SQL for reverse engineering the database schema.

Comment by Steve Müller [ 22/Feb/14 ]

Antoine Descamps Okay after having investigated on this in detail and thinking about the possibilities we have to find a solution for this, I came to the following conclusion:
ORM's schema tool and DBAL's schema introspection are designed to be bound to what Doctrine defines as "database" for each platform. This is the scope of the whole operation. In case of Oracle it is the "user". You cannot break out of this scope in any way as the current DBAL implementation is not designed for a "complete" schema introspection and therefore does not allow it at some points. Fully understanding your concern I am afraid we cannot find a reasonable solution for your use case at this point in development (2.x). Furthermore there is a good reason for limiting the schema introspection to a certain layer. If the schema tool would introspect the "complete" schema without regard to the database it is connected to, it would suggest a lot of schema changes from other databases that do not belong to the context of the entity manager / database. This behaviour would even cause a lot more annoyance to users as it is the default use case that users are only interested in one database per entity manager.
Also this problem is completely independant from your mapping definitions. So it doesn't matter whether you prefix your table names or not. When running the "update" operation on the schema tool, it is the online schema introspection part that is preventing the schema tool from behaving as you would wish.
Changing the schema introspection behaviour in DBAL would completely break BC and is at some places in the code not even possible without changing the API.
I am sorry that I have to disappoint you with this conclusion but we I am afraid we cannot do anything about your issue until we start developing 3.x. We might reevaluate your use case their and see what we can do.





[DBAL-477] Just doublequote all schema names and field names in PostgreSQL sql command generation, and the same for MySQL Created: 28/Mar/13  Updated: 24/Dec/13

Status: Open
Project: Doctrine DBAL
Component/s: Platforms, Schema Managers
Affects Version/s: 2.3.2
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: jos de witte Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: mysql, postgresql
Environment:

Any PostgreSQL environment


Issue Links:
Reference
is referenced by DBAL-96 Make approach towards identifier quot... Open

 Description   

Generation of any SQL command to the database (From entities or migration versions) does not quote all the reserved keywords (For example a fieldname `right`.

Simple fix that always works: double-quote dbname, schemaname and fieldname

e.g "dbsecurity"."userschema"."users" or "tblusers"

MySQL : use the ` sign.

e.g `security`.`users` or `tblusers` (No support for schemas since I last checked some time ago)



 Comments   
Comment by Steve Müller [ 24/Jun/13 ]

If those are reserved keywords, they should be added to the "PostgreSQLKeywords" class and they will be quoted by Doctrine. As far as I can see those keywords mentioned are not present in this class. Maybe there is something missing?





[DBAL-235] Column order is not synchronized by Comparator Created: 08/Mar/12  Updated: 19/Dec/13

Status: Open
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: 2.2.1
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: Artem Goutsoul Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 1
Labels: None
Environment:

MySQL



 Description   

Comparator does not synchronize column order. It would be great if it did.



 Comments   
Comment by Chris Woodford [ 02/Apr/12 ]

I assume that this means adding AFTER to any ALTER ADD statements? if so, this is something that i would really appreciate as well

Comment by Artem Goutsoul [ 03/Apr/12 ]

Yup, that's exactly what I meant!

Comment by Steve Müller [ 18/Dec/13 ]

AFAIK only MySQL supports changing column order after table creation. All other vendors need a table recreation for this. Why exactly do you need this? I cannot think of a use case where the order in which columns are stored in the table is of any importance. And as this is a MySQL only specific feature which needs some heavy calculations in the comparator, I doubt this will be implemented...

Comment by Artem Goutsoul [ 19/Dec/13 ]

It's very convenient to use MySQL column order so that the columns show up in some sane order in DB management tools.
Also an application can show some tables in a sane way without explicit column order specification.

We are currently using it widely in our app. Our ORM is actually picking up the object specifications directly from the database and it's not specified in the application code.

Comment by Steve Müller [ 19/Dec/13 ]

Artem Goutsoul I get the point. But as I already stated only MySQL supports column reordering without requiring a table recreation AFAIK. How would you expect other platforms to behave in column reordering scenarios? I think it's weird if only MySQL platform can react to column reorderings and we have to adjust the comparator for a just a single platform. Benjamin Eberlei what would you say?





[DBAL-57] Handling of Quoted Elements Created: 30/Oct/10  Updated: 24/Jun/13

Status: Open
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: 2.0.0-BETA4
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Benjamin Eberlei Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Reference
is referenced by DDC-832 Not quoted class name when updating e... Resolved

 Description   

Handling of Quoted Table, Column (etc) names is not as good as it could be. Any input is currently accepted, however it is not processed further and used in conjunction with the platform.



 Comments   
Comment by Steve Müller [ 24/Jun/13 ]

May this be an issue fixed by https://github.com/doctrine/dbal/pull/302 ?





[DBAL-404] Support of index length for text fields Created: 29/Dec/12  Updated: 29/Dec/12

Status: Open
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: Maksim Lunochkin Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 2
Labels: None


 Description   

For table with text field description with index on it generated query is:

CREATE TABLE t (id INT UNSIGNED AUTO_INCREMENT NOT NULL, description LONGTEXT DEFAULT NULL, number INT DEFAULT NULL, INDEX index2 (description), INDEX index3 (description, number), PRIMARY KEY(id)) DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci ENGINE = InnoDB

This query lead to error, because for index on text field absented key length.






[DBAL-474] SchemaManager / Connection on PostgreSQL platform does not respect filterExpression for sequences Created: 27/Mar/13  Updated: 16/May/14

Status: In Progress
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: 2.2.2
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: jos de witte Assignee: Steve Müller
Resolution: Unresolved Votes: 1
Labels: postgresql, schematool
Environment:

Windows & Linux



 Description   

Dear Symfony team,

the filterExpression on AbstractSchemaManager seems not to work for sequences.

This only happens under postgres.

It seems the way the sequences are handled are the culprit: It tries to get min_value etc of sequences without matching sequence names to the filter expression in advance.

If for example access to the sequences is denied, (Different schema without permissions for the current entity manager), any higher-level ORM operations like generating migration versions fail.

--------------------- UPDATE

the context is when using migrations. Positive regexp expressions do not limit the migration to a single schema. eg ^schemaname.$
Instead, all sequences on the current database are returned.
When trying to limit a migration to a single schema consecutively this doesn't work.
We are using a per-schema connection, so this results in a lot of hassle for us.



 Comments   
Comment by Benjamin Eberlei [ 14/Apr/13 ]

Can you paste an exception trace? I see that filtering is applied to sequences, but your description seems to indicate this happens due to an SQL query much earlier?

Comment by jos de witte [ 24/Apr/13 ]

Dear Benjamin,

the context is when using migrations. Positive regexp expressions do not limit the migration to a single schema. eg ^schemaname.$
Instead, all sequences on the current database are returned.
When trying to limit a migration to a single schema consecutively this doesn't work.
We are using a per-schema connection, so this results in a lot of hassle for us.

Comment by Steve Müller [ 06/Jan/14 ]

jos de witte I think your issue got fixed in commit: https://github.com/doctrine/dbal/commit/8beb732fe9d5cd40a0d677f250d2be325482744f
This patch was first introduced in 2.3. Can you please confirm that this is fixed? Otherwise can you please provide a concrete example so that we can reproduce you issue?

Comment by Arnout Standaert [ 29/Jan/14 ]

I believe we are bumping into the same issue. Our webapp uses Migrations, but for our webapp we are limited to a certain schema within a bigger PostgreSQL database. We only have permissions on our own schema and public.
Now, listSequences in AbstractSchemaManager does filter the asset names correctly with the mentioned fix.

But the problem is with the step before, _getPortableSequencesList (see line 135 of AbstractSchemaManager):

        return $this->filterAssetNames($this->_getPortableSequencesList($sequences));

This function goes out and does a _getPortableSequenceDefinition on every sequence in the unfiltered list. For every sequence, the _getPortableSequenceDefinition in PostgreSqlSchemaManager performs a SELECT:

        $data = $this->_conn->fetchAll('SELECT min_value, increment_by FROM ' . $this->_platform->quoteIdentifier($sequenceName));

Now, our user role in the database doesn't have SELECT permissions on these sequences in other schemas, so the migration fails with a privilege error.

There should be some kind of filtering on the sequence list BEFORE the SELECT statement in the _getPortableSequenceDefinition function are performed, no?

Comment by Arnout Standaert [ 30/Jan/14 ]

I currently have a workaround running locally, which filters the sequences list before creating the PortableSequence's. This is probably hackish and a poor workaround, just posting here as a temporary solution and further illustration of the problem.

Altered line 135 in AbstractSchemaManager:

        return $this->_getPortableSequencesList($this->filterSequenceNames($sequences));

I added function filterSequenceNames() in AbstractSchemaManager for appropriate sequence filtering:

    /**
     * Filters sequence names if they are configured to return only a subset of all
     * the found elements.
     *5
     * @param array $sequenceNames
     *
     * @return array
     */
    protected function filterSequenceNames($sequenceNames)
    {
        $filterExpr = $this->getFilterSchemaAssetsExpression();
        if ( ! $filterExpr) {
            return $sequenceNames;
        }
        
        return array_values ( array_filter($sequences, function ($sequenceName) use ($filterExpr) {
                $sequenceName = $sequenceName["schemaname"].".".$sequenceName["relname"];
                return preg_match($filterExpr, $sequenceName);
            })
        );

    }

After these modifications, doctrine:migrations:migrate operations complete succesfully, even with our limited-permission database account.

Comment by Steve Müller [ 21/Feb/14 ]

Arnout Standaert Your fix looks promising and reasonable. Can you create a PR on the DBAL repo for that?

Comment by Viktor Sidochenko [ 15/Mar/14 ]

Why this fix is not in master?

Comment by Steve Müller [ 15/Mar/14 ]

Viktor Sidochenko because nobody has fixed it yet Feel free to provide a patch on GitHub.

Comment by Arnout Standaert [ 17/Mar/14 ]

I haven't gotten around to doing the PR on GitHub yet, I'm not yet too familiar with that.
I'll try to find some time for this the coming days.

Comment by Viktor Sidochenko [ 17/Mar/14 ]

Will be good. I`m not professional developer to make patches to upstream. So just voted for this issue.

Comment by Steve Müller [ 19/Mar/14 ]

Patch supplied in PR: https://github.com/doctrine/dbal/pull/546
jos de witte, Arnout Standaert, Viktor Sidochenko can you please test if the supplied PR fixes the problem?

Comment by Doctrine Bot [ 16/May/14 ]

A related Github Pull-Request [GH-546] was closed:
https://github.com/doctrine/dbal/pull/546





[DBAL-916] Some alter table statements do not respect @Table name value Created: 30/May/14  Updated: 04/Jun/14

Status: Open
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: 2.4
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Julia Smith Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None
Environment:

MacOS



 Description   

I have specified the table name for each class generating a table. I find that some table names are respected during doctrine update upon the database, and some are not.

Specifically it seems that simpler entities, one's annotated with entity/table become lower case, while more complicated ones (inheritance map) respect the given table name.



 Comments   
Comment by Julia Smith [ 03/Jun/14 ]

On further inspection, this does not appear to be a bug with Doctrine. A simple dump of the update statements generated shows no table name change, but:

ALTER TABLE Comment DROP FOREIGN KEY FK_5BC96BF03DBEAF48;
DROP INDEX IDX_5BC96BF03DBEAF48 ON Comment;
ALTER TABLE Comment CHANGE replyto_id replytox_id INT DEFAULT NULL;
ALTER TABLE Comment ADD CONSTRAINT FK_5BC96BF0484A4D29 FOREIGN KEY (replytox_id) REFERENCES Comment (id);
CREATE INDEX IDX_5BC96BF0484A4D29 ON Comment (replytox_id);

Somehow causes mysql to change the case of the table on MacOS.

weird.

Comment by Steve Müller [ 04/Jun/14 ]

Julia Smith can you please provide more details concerning your actual problem? I'm not sure I understand what the problem is here. I can't see how the SQL you pointed out changes a table's name in any way so I don't understand how that could change the table name's case.
Concerning identifier casing in general I would suggest you to read the following from the MySQL documentation:
http://dev.mysql.com/doc/refman/5.5/en/identifier-case-sensitivity.html

And a pretty good explanation of the problem with identifiers and case accross operating systems and database vendors:
http://www.alberton.info/dbms_identifiers_and_case_sensitivity.html#.U47DV3IZWlM

If the casing of identifiers (table names, column names etc.) in your schema is important for you, you will either have to quote those in your mapping explicitly or use a custom quoting strategy. Please refer to the documentation for further details:
http://docs.doctrine-project.org/en/latest/reference/basic-mapping.html#quoting-reserved-words

Hope that helps.





[DBAL-936] Doctrine type not found exception thrown before checking the comment Created: 10/Jul/14  Updated: 11/Jul/14

Status: Open
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: 2.4.2
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Maxime Veber Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: mapping, mysql


 Description   

Here is the piece of code:

$type = $this->_platform->getDoctrineTypeMapping($dbType);

// In cases where not connected to a database DESCRIBE $table does not return 'Comment'
if (isset($tableColumn['comment'])) {
    $type = $this->extractDoctrineTypeFromComment($tableColumn['comment'], $type);
    $tableColumn['comment'] = $this->removeDoctrineTypeFromComment($tableColumn['comment'], $type);
}

The method `getDoctrineTypeMapping` throw an exception if the type is not found. But for example if you have an enum type (see the doctrine cookbook on the subject), the type is setted as comment.

Doctrine throw the exception before having the occasion to get the type via comment.

Here are two solutions:

  • Check for comment before throw the exception
  • Adding the enum type to the doctrine platform and mapping it to string





[DBAL-970] Implement partial indexes for missing platforms Created: 12/Aug/14  Updated: 12/Aug/14

Status: Open
Project: Doctrine DBAL
Component/s: Platforms, Schema Managers
Affects Version/s: 2.5
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Major
Reporter: Steve Müller Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

As a follow-up to DBAL-900 there are some platforms left that support partial indexes too and need the implementation for that feature.
Currently partial indexes are only implemented for PostgreSQL.
Additionally there should be a functional test covering creation and introspection of partial indexes if possible.






[DBAL-1017] Altering a foreign key column is not done properly for MySQL Created: 21/Oct/14  Updated: 21/Oct/14

Status: Open
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: 2.5, 2.4.3
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Christophe Coevoet Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: mysql, schematool


 Description   

Altering a foreign key column column (for instance to make it not-nullable) or the index of a foreign key does not work on MySQL (to be exact, it does not work on some MySQL setups, but I haven't found the config setting impacting it yet). Making it work requires dropping the foreign key before altering the column/index and readding it after



 Comments   
Comment by Steve Müller [ 21/Oct/14 ]

Christophe Coevoet DBAL-732 related?





[DBAL-96] Make approach towards identifier quoting consistent Created: 26/Feb/11  Updated: 04/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Platforms, Schema Managers
Affects Version/s: None
Fix Version/s: 2.6

Type: Improvement Priority: Major
Reporter: Benjamin Eberlei Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 1
Labels: None

Issue Links:
Duplicate
is duplicated by DBAL-120 MySql platform getAlterTableSQL does ... Resolved
Reference
relates to DBAL-45 Add CLI tool that checks for Reserved... Resolved
relates to DBAL-477 Just doublequote all schema names and... Open
is referenced by DBAL-40 Transparent table&column names escaping Open

 Description   
  • Make the use of `` a general approach for explicit quoting of identifiers
  • introduce AbstractPlatform::getRegularSQLIdentifierCase($identifier)
  • Introduce AbstractPlatform::isRegularIdentifier($identifier)
  • Fix Schema Assets not to lower-case, but to check for explicit quoting before.
  • Filter values of identifiers passed to all platform functions when they are used in information schema queries according to `` explicit quoting rules.

Problem: Schema is independent of a vendor, this means we have to pick a behavior, i propose SQL-92

This means:

  • strtoupper() ALL tables, column, index, foreign key names that are not quoted by ``
  • For any Quoted identifiers by `` the case is kept.
  • We can introduce a validator to detect a schema that cannot be implemented with a given vendor platform.

In conjunction with the SQL reserved keywords tickets we can then improve the DatabaseDriver considerably to detect identifier casings



 Comments   
Comment by Steve Müller [ 24/Dec/13 ]

Benjamin Eberlei this is an interesting approach and I like it. But I have some complaints about it.
1. I doubt users will be happy about forced default casing rules (ALL upper or ALL lower). Therefore we should think about adding a simple configuration option in DBAL allowing to override the default casing behaviour to the user's preference.
2. Using a consistent default casing means we ALWAYS have to quote identifiers as otherwise the underlying database could silently change the case again (don't know if this is an issue).
3. Introducing this approach in 2.x branch is a BC break as it breaks users' mixed-case identifier mappings.

For 2.x we should maybe at least make use of Identifier class throughout the platforms where necessary.





[DBAL-1060] [GH-736] [DBAL-1058] Fix database and namespace introspection for SQL Server Created: 05/Dec/14  Updated: 05/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Platforms, Schema Managers
Affects Version/s: 2.4, 2.5
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Reference
relates to DBAL-1058 It seems that MSSQL syntax was changed Open
is referenced by DBAL-1061 [GH-737] [DBAL-1058] [2.4] Fix databa... Open

 Description   

This issue is created automatically through a Github pull request on behalf of deeky666:

Url: https://github.com/doctrine/dbal/pull/736

Message:

It looks like some SQL Server versions / setups need system tables to be queried lowercased otherwise causing errors.



 Comments   
Comment by Doctrine Bot [ 05/Dec/14 ]

A related Github Pull-Request [GH-736] was assigned:
https://github.com/doctrine/dbal/pull/736

Comment by Doctrine Bot [ 05/Dec/14 ]

A related Github Pull-Request [GH-737] was assigned:
https://github.com/doctrine/dbal/pull/737





[DBAL-1061] [GH-737] [DBAL-1058] [2.4] Fix database names introspection for SQL Server Created: 05/Dec/14  Updated: 05/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Platforms, Schema Managers
Affects Version/s: 2.4
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Reference
relates to DBAL-1058 It seems that MSSQL syntax was changed Open
relates to DBAL-1060 [GH-736] [DBAL-1058] Fix database and... Open

 Description   

This issue is created automatically through a Github pull request on behalf of deeky666:

Url: https://github.com/doctrine/dbal/pull/737

Message:

Backport of #736 to 2.4 branch.



 Comments   
Comment by Doctrine Bot [ 05/Dec/14 ]

A related Github Pull-Request [GH-737] was assigned:
https://github.com/doctrine/dbal/pull/737





[DBAL-1069] datetimetz type needs to be a commented one by default Created: 10/Dec/14  Updated: 10/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Platforms, Schema Managers
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Christophe Coevoet Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

By default, the AbstractPlatform creates datetimetz fields as datetime field for platforms not defining a specific creation for fields with timezones.
This can be an acceptable fallback if you know that your application always send the value as UTC even when the database can accept any timezone as input thanks to the datetimetz type (database not supporting timezone would still have consistent data if you always send the same timezone).
However it creates an issue for the SchemaTool: the field will obviously be reported as datetime when inspecting the DB, not as datetimetz (since it does not have a datetimetz). So for platforms not supporting datetimetz natively, the special comment should be used to avoid useless updates.
The special Doctrine command should however not be used for platforms supporting the type natively (PostgreSQL, SQLServer2008, Oracle and SqlAnywhere12).

I have an idea about a fix, so I may send a PR for this in the following days.






[DBAL-1085] Custom Type Compare Fails To Generate Correct Migrations Created: 19/Dec/14  Updated: 19/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: 2.0, 2.1, 2.2, 2.3, 2.4
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Nicolas Vanheuverzwijn Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: migrations
Environment:

Everywhere



 Description   

// From doctrine 2.1
public function diffColumn(Column $column1, Column $column2)
{
$changedProperties = array();
if ( $column1->getType() != $column2->getType() )

{ $changedProperties[] = 'type'; }

...
}
The $column1->getType() will return the underlying platform object but the $column2->getType() will return the custom object type.

Because of the way the php compare function works, a custom type will always generate a changed property over the type of a column.

http://stackoverflow.com/questions/26964367/symfony2-doctrine-custom-types-generating-unneeded-migrations/27557785#27557785






[DBAL-812] SchemaTool ignores multi-columns (composite) indexes when creating a foreign key Created: 13/Feb/14  Updated: 15/Feb/14

Status: Open
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: 2.4.2
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Peter Huynh Assignee: Steve Müller
Resolution: Unresolved Votes: 0
Labels: schematool
Environment:

MySQL



 Description   

When using the schema tool to generate the differences between the db and the entities, I notice the following:

ALTER TABLE sales ADD CONSTRAINT FK_36D222EF603EE73 FOREIGN KEY (vid) REFERENCES vendor (id);
CREATE INDEX IDX_36D222EF603EE73 ON sales (vid);

Normally, it doesn't bother me, however I have had an existing composite index consists of (vid, submit). This leads to the redundant index IDX_36D222EF603EE73 and therefore forcing unnecessary overheads.

It would be nice to have a mean to disable this from happening.

The code in question can be found at https://github.com/doctrine/dbal/blob/594e326bd58d1d7af578f0dc3143655b9d119d45/lib/Doctrine/DBAL/Schema/Table.php#L543.

A helpful member from #doctrine IRC also pointed out that it does not check the unique constraints also.

Regards,



 Comments   
Comment by Marco Pivetta [ 13/Feb/14 ]

Just a hint on this: looks like the current logic just checks hashed indexes, ignoring the "uniq" prefixed indexes to avoid duplicates.

Comment by Steve Müller [ 15/Feb/14 ]

Peter Huynh I get the point here. However after having a quick look into this it seems this behaviour exists for a good reason. See here: https://github.com/doctrine/dbal/blob/master/lib/Doctrine/DBAL/Schema/Index.php#L191-L195
The question is how could this reasonable extended to fulfil the needs you explained. I am not entirely sure about this and this peace of code is rather sensitive and critical. I don't want to rush any assumptions here.

Comment by Peter Huynh [ 15/Feb/14 ]

Hi Steve,

I fully understand your reservation.

Thanks for looking into things.

Peter





[DBAL-598] Schema Comparator is case insensitive for table names Created: 05/Sep/13  Updated: 07/Oct/13

Status: Open
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Minor
Reporter: Justin Martin Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

https://github.com/doctrine/dbal/blob/master/lib/Doctrine/DBAL/Schema/Comparator.php#L102

The schema comparator currently uses strtolower for all table name comparison, so differences in case will not be detected.

When updating a schema using the schema tool, the tool will not perform any modification if the case of a table name has changed.

An example of this case would be when one is writing entities for the ORM, and the naming includes uppercase letters. You generate your schema and it uses the equivalent case for the initial generation. Then, you write Table annotations to change the name of the tables to lowercase. The schema comparator would not detect this, and depending upon the case sensitivity of the SQL engine, this could result in SQL errors.



 Comments   
Comment by Jeremiah Small [ 07/Oct/13 ]

This is the closest existing issue I could find to a problem we are having with the schema tool update and case insensitivity. If this is a separate bug, I'd be happy to write it up, but I don't have create privs in Jira yet.

Here's the bug:

If we add a new association, and do php vendor/bin/doctrine-module orm:schema-tool:create, the new association will be created in MySQL information schema REFERENCED_TABLE_NAME will be created case sensitively.

Example (note second to last column):

CONSTRAINT_CATALOG CONSTRAINT_SCHEMA CONSTRAINT_NAME TABLE_CATALOG TABLE_SCHEMA TABLE_NAME COLUMN_NAME ORDINAL_POSITION POSITION_IN_UNIQUE_CONSTRAINT REFERENCED_TABLE_SCHEMA REFERENCED_TABLE_NAME REFERENCED_COLUMN_NAME
def mydb FK_E2C31786CF786E13 def mydb ObservationDiagnosticParameterValue diagnostic_parameter_value_id 1 1 mydb DiagnosticParameterValue id

However, if we add a new association, and do php vendor/bin/doctrine-module orm:schema-tool:update, the new association will be created in MySQL information schema REFERENCED_TABLE_NAME will be created case insensitively.

Example (note second to last column):

CONSTRAINT_CATALOG CONSTRAINT_SCHEMA CONSTRAINT_NAME TABLE_CATALOG TABLE_SCHEMA TABLE_NAME COLUMN_NAME ORDINAL_POSITION POSITION_IN_UNIQUE_CONSTRAINT REFERENCED_TABLE_SCHEMA REFERENCED_TABLE_NAME REFERENCED_COLUMN_NAME
def mydb FK_E2C31786CF786E13 def mydb ObservationDiagnosticParameterValue diagnostic_parameter_value_id 1 1 mydb diagnosticparametervalue id

In the second case, we then get errors like this:

An exception occurred while executing 'INSERT INTO ObservationDiagnosticParameterValue (observation_id, diagnostic_parameter_value_id) VALUES (?, ?)' with params [1211, 6916]:

SQLSTATE[23000]: Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails (`mydb`.`ObservationDiagnosticParameterValue`, CONSTRAINT `FK_E2C31786CF786E13` FOREIGN KEY (`diagnostic_parameter_value_id`) REFERENCES `diagnosticparametervalue` (`id`))

#0 /var/opt/theapp/vendor/doctrine/dbal/lib/Doctrine/DBAL/Statement.php(140): Doctrine\DBAL\DBALException::driverExceptionDuringQuery(Object(PDOException), 'INSERT INTO Obs...', Array)

To fix this, we can alter the affected table like this:

alter table `ObservationDiagnosticParameterValue` drop foreign key `FK_E2C31786CF786E13`;
alter table `ObservationDiagnosticParameterValue` add constraint `FK_E2C31786CF786E13` foreign key (`diagnostic_parameter_value_id`) REFERENCES `DiagnosticParameterValue` (`id`);

The behavior of schema-tool:create and schema-tool:update should not be different.





[DBAL-406] PostgreSqlSchemaManager::tablesExist() misses schema-qualified table names if they exist in the first schema on the search path Created: 07/Jan/13  Updated: 07/Jan/13

Status: Open
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: 2.3.1
Fix Version/s: None
Security Level: All

Type: Bug Priority: Minor
Reporter: Roger Hunwicks Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: postgresql
Environment:

Postgresql 9.1



 Description   

Please see https://github.com/doctrine/migrations/issues/99 for additional background.

To reproduce:

CREATE SCHEMA test_schema;
CREATE TABLE test_schema.test_table (test_column TEXT);

Then in Doctrine:

Unable to find source-code formatter for language: php. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
$connection->executeUpdate('SET search_path=test_schema;');
$result = $connection->getSchemaManager()->tablesExist(array('test_schema.test_table'));

$result is false when it should be true.

The error occurs because PostgreSqlSchemaManager returns the bare table name from getPortableTablesList() if the schema is the first one in the search path.

The full explanation is...

AbstractSchemaManager::tablesExist() calls $this->getPortableTablesList() before checking if the tables exist.

PostgreSqlSchemaManager overrides this in _getPortableTableDefinition() by comparing the schema for the table with the search path for the connection. If the table schema is the first one in the search path, then it returns the bare table name, if it isn't then it returns the schema-qualified table name (i.e. schema.table).

tablesExist() does an array_intersect to check that all the tables in the search array exist in the database.

If one of the tables in the search array was schema-qualified but also in the first schema on the search path, then you end up checking:

array_intersect(array('test_schema.test_table'), array('test_table'))

which fails.

One way to fix it would be to override tablesExist() in PostgreSqlSchemaManager so that it passes the search array through getPortableTableDefinition() before doing the array_intersect:

Unable to find source-code formatter for language: php. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
    /**
     * Return true if all the given tables exist.
     *
     * @param array $tableNames
     * @return bool
     */
    public function tablesExist($tableNames)
    {
        foreach ($tableNames as $key => $tableName) {
            if (strpos($tableName, '.') !== false) {
                $tableName = explode('.', $tableName, 2);
                $tableNames[$key] = $this->_getPortableTableDefinition(array('schema_name'=>$tableName[0], 'table_name'=>$tableName[1]));
            }
        }
        return parent::tablesExist($tableNames);
    }

I'm happy to provide a PR on GitHub if you want.






[DBAL-352] Running doctrine:schema:update throws an exception the first time, succeeds the second time. Created: 25/Sep/12  Updated: 25/Sep/12

Status: Open
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: None
Fix Version/s: None
Security Level: All

Type: Bug Priority: Minor
Reporter: Mark A. Hershberger Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Shell output from my symfony project:
$ app/console doctrine:schema:update --force
Updating database schema...

[Doctrine\DBAL\DBALException]
An exception occurred while executing 'ALTER TABLE Tn ADD CONSTRAINT FK_3504C6F07D3656A4 FOREIGN KEY (account) REFERENCES Account (id)':

SQLSTATE[42000]: Syntax error or access violation: 1072 Key column 'account' doesn't exist in table

[PDOException]
SQLSTATE[42000]: Syntax error or access violation: 1072 Key column 'account' doesn't exist in table

doctrine:schema:update [--complete] [--dump-sql] [--force] [--em[="..."]]

$ app/console doctrine:schema:update --force
Updating database schema...
Database schema updated successfully! "7" queries were executed
$



 Comments   
Comment by Christophe Coevoet [ 25/Sep/12 ]

Please run the command with the --dump-sql option instead of --force each time before launching it for real, to be able to see which SQL queries are executed in both cases

Comment by Mark A. Hershberger [ 25/Sep/12 ]

I'm not sure how to roll back the changes so that I can reproduce this.

Comment by Mark A. Hershberger [ 25/Sep/12 ]

Got it after a little poking around with git. Rolling back to a commit in my code before the one given here didn't didn't show this reproduce this.

$ php app/console doctrine:schema:create
ATTENTION: This operation should not be executed in a production environment.

Creating database schema...
Database schema created successfully!

$ git checkout d686a39fb664dca540167e1b3e96ea0ffd67bc00
Previous HEAD position was 0510a59... * First round of Customer-to-Accounts (1:n) mapping done * Start of Customer-to-Tn and Tn-to-Account mapping
HEAD is now at d686a39... * A little debugging for Entity::Customer::hasEntitlement * Add FK for Customer to Tn

$ php app/console doctrine:schema:update --dump-sql
ALTER TABLE Account CHANGE guid guid VARCHAR(255) NOT NULL;
ALTER TABLE Tn DROP FOREIGN KEY FK_3504C6F09B6B5FBA;
DROP INDEX IDX_3504C6F09B6B5FBA ON Tn;
ALTER TABLE Tn ADD tnType VARCHAR(10) NOT NULL, CHANGE account_id customer INT DEFAULT NULL;
ALTER TABLE Tn ADD CONSTRAINT FK_3504C6F07D3656A4 FOREIGN KEY (account) REFERENCES Account (id);
ALTER TABLE Tn ADD CONSTRAINT FK_3504C6F081398E09 FOREIGN KEY (customer) REFERENCES Customer (id);
CREATE INDEX IDX_3504C6F07D3656A4 ON Tn (account);
CREATE INDEX IDX_3504C6F081398E09 ON Tn (customer);
ALTER TABLE Customer CHANGE guid guid VARCHAR(255) NOT NULL, CHANGE authGuid authGuid VARCHAR(255) NOT NULL
mah@debian:~/comcast$ php app/console doctrine:schema:update --force
Updating database schema...

[Doctrine\DBAL\DBALException]
An exception occurred while executing 'ALTER TABLE Tn ADD CONSTRAINT FK_3504C6F07D3656A4 FOREIGN KEY (account)
REFERENCES Account (id)':

SQLSTATE[42000]: Syntax error or access violation: 1072 Key column 'account' doesn't exist in table

[PDOException]
SQLSTATE[42000]: Syntax error or access violation: 1072 Key column 'account' doesn't exist in table

doctrine:schema:update [--complete] [--dump-sql] [--force] [--em[="..."]]

$ php app/console doctrine:schema:update --dump-sql
ALTER TABLE Account CHANGE guid guid VARCHAR(255) NOT NULL;
ALTER TABLE Tn ADD account INT DEFAULT NULL;
ALTER TABLE Tn ADD CONSTRAINT FK_3504C6F07D3656A4 FOREIGN KEY (account) REFERENCES Account (id);
ALTER TABLE Tn ADD CONSTRAINT FK_3504C6F081398E09 FOREIGN KEY (customer) REFERENCES Customer (id);
CREATE INDEX IDX_3504C6F07D3656A4 ON Tn (account);
CREATE INDEX IDX_3504C6F081398E09 ON Tn (customer);
ALTER TABLE Customer CHANGE guid guid VARCHAR(255) NOT NULL, CHANGE authGuid authGuid VARCHAR(255) NOT NULL

$ php app/console doctrine:schema:update --force
Updating database schema...
Database schema updated successfully! "7" queries were executed

$

Comment by Christophe Coevoet [ 25/Sep/12 ]

hmm, the first update seems to miss the addition of the account field, which is done the second time through ALTER TABLE Tn ADD account INT DEFAULT NULL;





[DBAL-167] Schema comparator doesn't work properly with columnDefinition's Created: 17/Sep/11  Updated: 16/Jul/12

Status: Open
Project: Doctrine DBAL
Component/s: Drivers, Platforms, Schema Managers
Affects Version/s: 2.0.8, 2.1, 2.1.1, 2.1.2
Fix Version/s: None
Security Level: All

Type: Bug Priority: Minor
Reporter: Dmitry Strygin Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 3
Labels: None


 Description   

Schema comparator will mostly always return changed properties on columns for entities defined with columnDefinition even they are identical in the DB. This is due to weak low-lever compatibility of SchemaTool#getCreateSchemaSql() and SchemaTool#getSchemaFromMetadata() – the first one doesn't reconstruct columnDefinition, and the other one never supports 'fixed', 'default', cannot determine, whether it is boolean or integer (ex. TINYINT in the DB), etc...

All this results in extremely annoying unnecessary alter-table-change-columns surrounded by dropping and after that re-enabling constrains dependent on those columns.

I mean stuff like this:

symfony2#app/console doctrine:schema:update --dump-sql
...
ALTER TABLE es_hotels DROP FOREIGN KEY FK_527F88EE584598A3F92F3E70;
ALTER TABLE es_hotels DROP FOREIGN KEY FK_527F88EE584598A37A3ABE5D;
ALTER TABLE es_hotels DROP FOREIGN KEY FK_527F88EE584598A3EE551564;
ALTER TABLE es_hotels CHANGE is_active is_active TINYINT(1) NOT NULL DEFAULT '1', CHANGE checksum checksum CHAR(32) DEFAULT NULL;
ALTER TABLE es_hotels ADD CONSTRAINT FK_527F88EE584598A3F92F3E70 FOREIGN KEY (operator_id, country_id) REFERENCES es_countries(operator_id, id) ON DELETE CASCADE;
ALTER TABLE es_hotels ADD CONSTRAINT FK_527F88EE584598A37A3ABE5D FOREIGN KEY (operator_id, resort_id) REFERENCES es_resorts(operator_id, id) ON DELETE CASCADE;
ALTER TABLE es_hotels ADD CONSTRAINT FK_527F88EE584598A3EE551564 FOREIGN KEY (operator_id, subresort_id) REFERENCES es_subresorts(operator_id, id) ON DELETE CASCADE;
...

The simple solution would be to fix schema comparator not to signal any changes on columns with columnDefinition properties.
But would be much and much better to add some code to all *SchemaManager#getPortableTableColumnDefinition so they would reconstuct columnDefinition and they would be matched in the schema comparator.

I can do this



 Comments   
Comment by Roderick Schaefer | We handle IT [ 16/Oct/11 ]

I'm having the same issue on my production webserver, but not on the development webserver. I find that odd. It tries to drop all foreign keys and create them again, although without the CHANGE statement you are referring to, Dmitry.

Comment by Benjamin Eberlei [ 09/Jan/12 ]

This maybe fixable by making a hash out of the column definition and saving it into a database comment.

The Foreign Key problem maybe because of an old MySQL version 5.0.x

Comment by Joe Cai [ 16/Jul/12 ]

@beberlei, sounds good to me. any plan of implementing this?





[DBAL-175] Table comments in Doctrine\DBAL\Schema\Table Object Created: 06/Oct/11  Updated: 17/Oct/11

Status: Open
Project: Doctrine DBAL
Component/s: Platforms, Schema Managers
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Minor
Reporter: Asmir Mustafic Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 1
Labels: None


 Description   

Should be useful discover the table comments from database schema.

This feature is already available for column comments, but not for table comments






[DBAL-879] Sequence default value [PGSQL] Created: 29/Apr/14  Updated: 26/Oct/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers, Schema Managers
Affects Version/s: 2.4.2
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Mohammad Niknam Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: autoincrement, postgresql, sequence
Environment:

ArchLinux
PostgreSQL 9.3.4



 Description   

Hi
I'm using dbal to generate schmea from database via Schema-Manager. The problem is that my primary field 'id' have default value of 'nextval('test1_id_seq'::regclass)' but when I retrive columns using Doctrine\DBAL\Schema\AbstractSchemaManager::listTableDetails() or Doctrine\DBAL\Schema\Table::getColumns() , default value of the column 'id' is null.
In Doctrine\DBAL\Schema\PostgreSqlSchemaManager::_getPortableTableColumnDefinition() method at line 292 default value replaced with null, I don't know why but I guess It's because Driver compatibility.
Also Doctrine\DBAL\Schema\Sequence has no method to retrieve that table.
So I don't have the default value (pointing at sequence) and I can't find out what Sequence is linked to this table either.



 Comments   
Comment by Steve Müller [ 26/Oct/14 ]

Can you please provide a code example of how you create table + sequence and retrieve it?





[DBAL-1048] Function based index Created: 20/Nov/14  Updated: 20/Nov/14

Status: Open
Project: Doctrine DBAL
Component/s: Platforms, Schema Managers
Affects Version/s: 2.4.3, 2.3.5
Fix Version/s: None
Security Level: All

Type: New Feature Priority: Minor
Reporter: Grégoire Paris Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: postgresql


 Description   

I would like to create a function-based index, but it does not seem to be possible ATM. The final goal is to be able to have case insensitive string filters with Postgresql without losing performance.

I tried this mapping :

  indexes:
        my_entity_name_index:
            columns: [ LOWER(name) ]

But here is what doctrine answers :

[Doctrine\DBAL\Schema\SchemaException]

There is no column with name 'LOWER(name)' on table 'my_entity'.



 Comments   
Comment by Marco Pivetta [ 20/Nov/14 ]

I don't think that we can provide platform-specific index column name support here.





Generated at Fri Dec 19 04:05:08 UTC 2014 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.