[DBAL-106] Doctrine\DBAL\Schema\Comparator false positives Created: 29/Mar/11  Updated: 06/Apr/11  Resolved: 06/Apr/11

Status: Resolved
Project: Doctrine DBAL
Component/s: Schema Managers
Affects Version/s: 2.0.2
Fix Version/s: 2.0.4

Type: Bug Priority: Major
Reporter: Lee Feistel Assignee: Benjamin Eberlei
Resolution: Fixed Votes: 1
Labels: None

Doctrine 2.0.3, PHP 5.3.5, MySQL 5.5.9


I am using the model class annotations for my Doctrine schema definition and a MySQL database for my target. When testing code that uses Doctrine\DBAL\Schema\Comparator, either by using $schemaTool->updateSchema($classes) which is defined in the core functionality or by running doctrine migrations:diff from the Doctrine Migrations project, a set of bugs are revealed. The symptoms of this are that immediately after calling $schemaTool->createSchema($classes) any call that uses the Comparator class to calculate the diffs between the real database and the schema definition produce a large set of false positives. Most of these false positives can be traced back to the logic in Doctrine\DBAL\Schema\Comparator::diffColumn(Column $column1, Column $column2). There are two issues here I can see right away:

One is that if the model annotations rely on defaults rather than explicitly defining certain attributes, this function sees them as different. For example, if I have a column defined like this: @Column(type="decimal", scale=2, nullable=true) then the Comparator believes it is different from the real database because precision is 10 in the real data and 0 in my schema. If I change the line to this: if (($column1->getPrecision() ?: 10) != ($column2->getPrecision() ?: 10)) then it works, but I am sure a real solution needs to determine what the defaults actually are.

The second issue is when I use the 'columnDefinition' attribute with a custom type to declare an enum or set, for instance. Since this function does not actually check what the columnDefinition field is set to at all, it has no way to know if I have actually made a change to the list of values, or anything else about that columnDefinition. I have been reading that using the comments field might be another way to do this, but I am not yet clear on what the best practice really is here. I know that http://www.doctrine-project.org/jira/browse/DBAL-42 touches on this with regard to storing Array type metadata in the comments field. I need more direction on how this should work in custom field types.

In addition, I am getting this strange set of diffs on the index for a ManyToMany association's join table:

        $this->addSql("CREATE INDEX IDX_44B8C6D89AF7860 ON firm_firm (firm_id)");
        $this->addSql("DROP INDEX primary ON firm_firm");
        $this->addSql("CREATE UNIQUE INDEX primary ON firm_firm (firm_id, firm_id)");

I am not sure why this only happens on this one table and not others, but it seems to be another case of the Comparator getting a false positive for some reason this time in Comparator::diffIndex(Index $index1, Index $index2). The model annotations for this association are:

   * @var Firm
   * @ManyToMany(targetEntity="Firm")
  protected $Children = array();

I will add more to this as I learn more details about the issues, but hopefully this is enough to start grokking the issues in Comparator. Getting the Comparator class in the core Doctrine\DBAL library working properly for calculating diffs will go a long way toward making well defined migrations a reality for Doctrine2 projects running in production. Hopefully, someone will also be able to address this minor issue in the Doctrine Migrations (http://www.doctrine-project.org/jira/browse/DMIG-21) so that it can be deployed without needing a patch.

Thanks to everyone for Doctrine2, and I am looking forward to being among the first to put this new library into production on a real world project in a few months.

Comment by Benjamin Eberlei [ 06/Apr/11 ]

Can you tell me how the indexes on the join table look like?

Comment by Lee Feistel [ 06/Apr/11 ]

Now that I look at this schema, I think I need to apologize for my oversight because I didn't explain the problem properly. The real issue is that this self-referential ManyToMany association needs to have the id fields explicitly defined or it will not work. Doctrine2 is generating this table by apparently assuming both id fields have the same name, and so the second index that is created gets a randomly generated name assigned to it. There is no way this association is actually working properly at all, so I'll talk to the programmer that wrote this. I think we can fix this part of the problem ourselves, but perhaps some error checking here would be helpful. The first half of the problem (in diffColumn) remains. Thanks again for looking into this.

`firm_id` int(11) NOT NULL,
PRIMARY KEY (`firm_id`),
KEY `IDX_44B8C6D89AF7860` (`firm_id`)

ALTER TABLE `firm_firm`
ADD CONSTRAINT `firm_firm_ibfk_1` FOREIGN KEY (`firm_id`) REFERENCES `firm` (`id`) ON DELETE CASCADE;

Comment by Benjamin Eberlei [ 06/Apr/11 ]

Fixed the precision issue, please open a new ticket for the many to many.

Comment by Lee Feistel [ 06/Apr/11 ]

Thanks. At least the precision issue was easy, so glad to have that one out of the way. Is there any hope for a way to handle the use of columnDefiniton? I guess the alter statements being generated are just changing it to the same value, so it might not actually be broken in and of itself. Those statements could take a small amount of time to execute what is essentially a noop. Maybe we just have to manually go through and clean out the unwanted alter statements for the time being. It might be workable.

Comment by Benjamin Eberlei [ 06/Apr/11 ]

Updated version

Comment by Benjamin Eberlei [ 06/Apr/11 ]

columnDefinition always comes up in the diff. There is just no way around it. The problem is that there is no generic way to create a "columnDefinition" from the existing databse, so no good value to compare it too. If you can think of a solution for this problem i would be very happy, but we havent come up with one yet.

Generated at Tue Dec 01 19:06:20 EST 2015 using JIRA 6.4.10#64025-sha1:5b8b74079161cd76a20ab66dda52747ee6701bd6.