[DMIG-37] PostgreSQL generated SQL statements in migrations that use doublequotes create syntax errors in PHP Created: 28/Mar/13  Updated: 28/Mar/13

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: jos de witte Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: PostgreSQL, schematool
Environment:

PostgreSQL backend, using migration versions with field or schemanames that use reserved names.



 Description   

When using some reserved field names in PostgreSQL DBAL, the generated SQL statements will quote those field names. (Make a table with fieldname "left" for example)

When the migrationversion is generated, the doublequote is not escaped:

addSQL("ALTER table x add column "left" integer");

This does not compile of course, should be

addSQL("ALTER table x add column \"left\" integer");

Regards!






[DMIG-36] Iterating over versions is very slow Created: 05/Mar/13  Updated: 05/Mar/13

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Eric Durand-Tremblay Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

When iterating migration versions, a call to SchemaManager::createSchema() is done for every versions. Four our schema, this call take about 1 second to execute. With already 30 migrations, this begin to be a problem.

$versions = $configuration->registerMigrationsFromDirectory($this->_migration_folder);
foreach($versions as $version){
    if(!$version->isMigrated()){
        $version->markMigrated();
    }

}

The problem is in Migrations\Configuration\Configuration::createMigrationTable() which does not set the migrationTableCreated flag when the table already exists.

public function createMigrationTable()
{
        $this->validate();

        if ($this->migrationTableCreated) {
            return false;
        }

        $schema = $this->connection->getSchemaManager()->createSchema();
        if ( ! $schema->hasTable($this->migrationsTableName)) {
            $columns = array(
                'version' => new Column('version', Type::getType('string'), array('length' => 14)),
            );
            $table = new Table($this->migrationsTableName, $columns);
            $table->setPrimaryKey(array('version'));
            $this->connection->getSchemaManager()->createTable($table);

            $this->migrationTableCreated = true;

            return true;
        }
        return false;
}





[DMIG-35] Ignoring unique key Created: 07/Sep/12  Updated: 07/Sep/12

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: venu Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: migrations, unique
Environment:

PHP 5.4.6-2~lucid
5.1.63-0ubuntu0.10.04.1-log



 Description   

I have a table design style, which has api_id field with unique key
CREATE TABLE `design_style` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`api_id` varchar(5) COLLATE utf8_unicode_ci NOT NULL,
.....
PRIMARY KEY (`id`),
UNIQUE KEY `api_id` (`api_id`),
UNIQUE KEY `name` (`name`),
KEY `category_id_idx` (`category_id`),
CONSTRAINT `design_style_category_id_design_style_category_id` FOREIGN KEY (`category_id`) REFERENCES `design_style_category` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=7 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci

I reverse engineered the database tables and created entities. I am using it in symfony2 app.

Then I executed
php app/console doctrine:schema:update --dump-sql
and also checked with migrations diff
php app/console doctrine:migrations:diff

but has given me
DROP INDEX api_id ON design_page






[DMIG-30] Make :migrations:diff smart and generate Schema object changes, not SQL directly Created: 16/Nov/11  Updated: 17/Feb/12

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major
Reporter: Benjamin Eberlei Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 2
Labels: None

Attachments: Text File DMIG-30.patch    

 Description   

Currently when using :migrations:diff the generated output will use addSql() exclusively. However there has to be a way to generate changes on the schema objects themselves by using the SchemaDiff objects and generating code from them.



 Comments   
Comment by Tyler Sommer [ 03/Feb/12 ]

I think this would be something that would greatly increase the usefulness of migrations.

After looking things over, I think doing something like this could work:

  • Instantiate an instance of Doctrine\ORM\Tools\Comparator, and retrieve a Doctrine\DBAL\Schema\SchemaDiff using $comparator->compare($fromSchema, $toSchema);
  • Iterate over each of the SchemaDiff's properties to retrieve new tables, dropped tables, columns, indexes, etc to generate both up and down code

It seems fairly simple, but I get kind of lost thinking about how it can be tested.

Another option, which may in the end be easier but more dirty, is to create a 'PhpPlatform' that only actually implements the getCreateTableSQL, getDropTableSQL, etc methods of Doctrine\DBAL\Platforms\AbstractPlatform.

Comment by Tyler Sommer [ 17/Feb/12 ]

I've gone and followed what I outlined in my last comment, but there is a small snag.

Comparator and SchemaDiff check for schema changes that are not possible to make using public methods. An example is a dropped index. Both Comparator and SchemaDiff support dropping indexes, but there is no Table#dropIndex to actually make it happen.

However, if you use Reflection to modify the underlying Table#$_indexes array, then the proper SQL will be generated by SchemaDiff. Terrible and dirty, I know. We should do something about it Should I create a ticket?

Anyway, I've attached a patch (based on the current master, 9e81984) of what I've come up with. I think I've covered everything. This will even generate code to drop indexes and foreign keys (using Reflection). I've tried it on a schema of around 90 entities and it works beautifully. Of course, it's definitely not done or ready, but I wanted to see what was thought about what I've done.

Finally, what would be considered an acceptable test for this kind of thing? I suppose if I refactor all of the code generating bits into individual methods, I could pretty easily test for expected output. Am I on the right track?

edit: by the way, use migrations:diff --no-platform to generate the migration





[DMIG-32] add annotation tag like author which defaults to current system username Created: 06/Feb/12  Updated: 06/Feb/12

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Minor
Reporter: Justinas Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

would be nice if generated migrations could include @author tag with systems username
it would help when migrations are created by several users to find which migration belongs to who.






[DMIG-26] Remove implicit schema diff to sql in Migration tasks for explicit solution Created: 16/Nov/11  Updated: 21/Nov/11

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Benjamin Eberlei Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Currently the following leads to an implicit drop table statement:

public function up(Schema $schema)
 {
    $schema->dropTable("foo");
}

However this should be an explicit operation that internally puts everything onto the "addSQL" stack, then resets the schema diffs to zero by cloning the current schema into the "diff schema" again.

public function up(Schema $schema)
 {
    $schema->dropTable("foo");
    $this->syncSchema($schema);
    $schema->dropTable("bar");
    $this->syncSchema($schema);
}


 Comments   
Comment by Beau Simensen [ 20/Nov/11 ]

So each call to syncSchema() would result in processing all changes on the schema instance (dropTable(), etc.) to actual SQL commands and placed on the "operations" stack? And at that point the schema instance should show no diffs so that any future calls to syncSchema() would only include schema changes since the previous call?

Comment by Benjamin Eberlei [ 20/Nov/11 ]

Yes exactly

Comment by Beau Simensen [ 21/Nov/11 ]

Alright, I'll see what I can do. I have a feeling I will get a pretty good idea how everything works if I start digging into this one.





[DMIG-29] Add callback operation Created: 16/Nov/11  Updated: 16/Nov/11

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major
Reporter: Benjamin Eberlei Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

When DMIG-27 is implemented, add a new operation "callback" that is executed at the position in the stack and can do arbitrary things (for example use the EntityManager to migrate data!)






[DMIG-27] Add stack for operations Created: 16/Nov/11  Updated: 16/Nov/11

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major
Reporter: Benjamin Eberlei Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Up and down operations should only put commands on a stack that is then executed after up() or down() have been processed.

Currently there are two implicit stacks:

1. addSql()
2. The changes done to the $schema instance.

With DMIG-26 $schema is an explicit change that adds to a stack.

This should be unified to be one single stack called "operations" and they should be linearly processed.






[DMIG-28] Interactive :migrate execution: Show all commands that would be executed when pressing [y] Created: 16/Nov/11  Updated: 16/Nov/11

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major
Reporter: Benjamin Eberlei Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Currently schema console only asks if you really want to do the migration, however you dont know what will actually happen.

all the operations should be posted before asking the question to do the migration.






[DMIG-24] False positives in Comparator on default values Created: 14/Jul/11  Updated: 16/Nov/11

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Yaroslav Zenin Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

At the moment I found only one way how to work with default values

/**
 * @var string $isActive
 *
 * @Column(name="isActive", type="boolean", nullable=false)
 */
private $isActive;

and then in the construct I use something like $this->setIsActive(true);
on db level it looks like "isActive BIT default 1"
But the problem happened when I try to compare schemas for migrations. It always false positives on defaults.

How to set defaults for prevent false positives in Comparator?






[DMIG-18] diffing causes lots of false positives (on postgresql) Created: 18/Feb/11  Updated: 16/Nov/11

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Lukas Kahwe Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 2
Labels: None
Environment:

PostgreSQL 9.0



 Description   

I am playing around with the diff tool and noticing that a lot of false positives are generated, like using SERIAL creates Sequences that are not explicitly listed, causing the diff tool to believe that they should be dropped. Or using "array" as a datatype makes the diff tool believe that there was a type change since they are setup as TEXT. now the alter statement then actually specifies to alter it form TEXT to TEXT again.

I guess the fact is that just more work needs to be put in here and I guess if I want things to improve I should step up, especially if I care about such an "obsure" database as Postresql



 Comments   
Comment by Ilan [ 14/May/11 ]

I have noticed the same problems when using @GeneratedValue(strategy="IDENTITY").
Postgres will create a sequence which doctrine does not manage. When running a schema validation doctrine fails because the diff tool
believes the sequences should not be in the database.
Same problem when trying to update the schema with --complete, doctrine will run queries to drop the sequences.

Regards,

Ilan

Comment by Lukas Kahwe [ 05/Jul/11 ]

likely solved with http://www.doctrine-project.org/jira/browse/DBAL-42

Comment by Finjon Kiang [ 10/Sep/11 ]

Currently, sequences won't be dropped when using diff tool (latest version from github downloaded today). But it still drops the default value like 'nextval('xxx_seq')'. As the database wasn't used by doctrine2 only, dropping default value like that caused some problems in my environment.

Comment by Finjon Kiang [ 10/Sep/11 ]

well... the diff is not reliable currently as all the indexes, constraints were affected and serial, bigserial fields would be replaced using int and bigint...





[DMIG-17] bug in reverting migrations Created: 15/Jan/11  Updated: 16/Nov/11

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Stepan Tanasiychuk Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 4
Labels: None
Environment:

Linux stfalcon-laptop 2.6.35-22-generic #35-Ubuntu SMP Sat Oct 16 20:45:36 UTC 2010 x86_64 GNU/Linux
Installed packages, channel pear.doctrine-project.org:
======================================================
Package Version State
DoctrineCommon 2.0.0 stable
DoctrineDBAL 2.0.0 stable
DoctrineORM 2.0.0 stable

DoctrineMigration from git://github.com/doctrine/migrations.git (2.0 alpha?)


Attachments: Zip Archive Entities.zip     File migrations.yml     File Version20110115224434.php    

 Description   

I create entities from doctrine getting started tutorial http://www.doctrine-project.org/docs/orm/2.0/en/tutorials/getting-started-xml-edition.html
It's Bug, User, Project (in attach)
and create config migrations.yml (in attach)

after that i run command "./doctrine migrations:diff" which generated migration file Version20110115224434.php (in attach)
"./doctrine migrations:migrate" work normal
but "./doctrine migrations:migrate 0" dosn't work:

Migrating down to 0 from 20110115224434

– reverting 20110115224434

-> ALTER TABLE bug_product DROP FOREIGN KEY
Migration 20110115224434 failed during Execution. Error SQLSTATE[HY000]: General error: 1005 Can't create table 'doctrine2.#sql-531_108' (errno: 150)

[PDOException]
SQLSTATE[HY000]: General error: 1005 Can't create table 'doctrine2.#sql-531_108' (errno: 150)



 Comments   
Comment by Karsten Dambekalns [ 13/Jun/11 ]

The problem is the missing constraint name. I looked up the name the constraint had and added it to the statement and it worked.

Since the names are generated by MySQL automatically, looking them up when generating the migration will probably not work, because the generated name might differ on another setup. What would work is creating names for the constraints and use them in create and drop operations.





[DMIG-6] Put Migrations under own namespace to avoid problems with Autoloader Created: 30/Jun/10  Updated: 16/Nov/11

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Marcus Stöhr Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

I think it would be a good idea to put the Migrations classes under their own namespace. If you use \Doctrine\Common\ClassLoader for loading both, DBAL and Migrations, you need to load Migrations before DBAL to avoid raising an exception.

Throws exception:

$classLoader = new \Doctrine\Common\ClassLoader('Doctrine\DBAL', realpath(APPLICATION_PATH . '/../library/Doctrine/DBAL/lib'));
$classLoader->register();
$classLoader = new \Doctrine\Common\ClassLoader('Doctrine\DBAL\Migrations', realpath(APPLICATION_PATH . '/../library/Doctrine/Migrations/lib'));
$classLoader->register();

Works:

$classLoader = new \Doctrine\Common\ClassLoader('Doctrine\DBAL\Migrations', realpath(APPLICATION_PATH . '/../library/Doctrine/Migrations/lib'));
$classLoader->register();
$classLoader = new \Doctrine\Common\ClassLoader('Doctrine\DBAL', realpath(APPLICATION_PATH . '/../library/Doctrine/DBAL/lib'));
$classLoader->register();


 Comments   
Comment by Jonathan H. Wage [ 30/Jun/10 ]

I am not sure we should do this. The migrations are an extension of the DBAL so it makes sense to be under that namespace. I thin it is normal to have to register the autoloaders in the right order, no?

Comment by Marcus Stöhr [ 01/Jul/10 ]

Yes, with the migrations being an extension of the DBAL I agree with you. As of today I wasn't aware of the fact that the autoloader using a stack (I should have checked that prior). However, it would be nice to have either some documentation about it or extend the ClassLoader to handle this implicit.

Comment by Jonathan H. Wage [ 01/Jul/10 ]

I will add some documentation about using multiple class loaders and order.





[DMIG-8] MySQL + Doctrine 2 + Column of type decimal with no precision or scale manually set results in infinite migration diffs generated Created: 07/Oct/10  Updated: 16/Nov/11

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Daniel Cousineau Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 1
Labels: None
Environment:

Debian 5.0, PHP 5.3.3, MySQL 5.1.50, Doctrine 2.0.0BETA4, Annotations



 Description   

To reproduce:

Create a new entity with a column type decimal @Column(type="decimal")

Run the CLI tool "migrations:diff"

Run the CLI tool "migrations:migrate"

If you run "migrations:diff" again a new migration will be created with an ALTER TABLE that only affects the columns of type Decimal. You can apply these new migrations and generate more using Diff ad nauseum.

My assumption is there is a mismatch between the Migrations Diff and annotation's defaults, causing it to continually identify decimal columns as requiring a migration.

To avoid this issue manually set a precision and scale, everything will then work as expected.






[DMIG-21] Call to undefined method Doctrine\DBAL\Migrations\Tools\Console\Command\StatusCommand::getApplication() Created: 29/Mar/11  Updated: 16/Nov/11

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Lee Feistel Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 2
Labels: None
Environment:

Doctrine 2.0.3, PHP 5.3.5



 Description   

I get the following error when trying to run doctrine migrations:status in a newly configured setup:

PHP Fatal error:  Call to undefined method Doctrine\DBAL\Migrations\Tools\Console\Command\StatusCommand::getApplication() in /library/vendor/migrations/lib/Doctrine/DBAL/Migrations/Tools/Console/Command/AbstractCommand.php on line 82

Changing line 82 of AbstractCommand.php to the following gets it working, but I am unsure of the implications:

if ($this->getHelper('db')) {


 Comments   
Comment by Romain D. [ 30/Mar/11 ]

I have this issue too with the "packaged" Doctrine 2.0.3 availible at http://www.doctrine-project.org/downloads/DoctrineORM-2.0.3-full.tar.gz.

Weirdly enough, the method is availible in the latest master, cf. https://github.com/symfony/Console/blob/master/Command/Command.php#L88

Comment by Wil Moore III [ 04/Apr/11 ]

I also confirm this behavior. I've also verified that this issue can be resolved by grabbing the latest (HEAD) from the Symfony repo.





[DMIG-19] migrations diff with columnDefinition Created: 06/Mar/11  Updated: 16/Nov/11

Status: Open
Project: Doctrine Migrations
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: arnaud-lb Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 3
Labels: None


 Description   

When an attribute has a @Column(columnDefinition=...) parameter, migrations:diff always causes false positives.






Generated at Sat May 18 15:38:43 UTC 2013 using JIRA 5.2.7#850-sha1:b2af0c8dc8537b36121c6a579fabbdf79fc919e5.