Sunsetting Doctrine DBAL 2

Posted on January 22, 2022 by Sergei Morozov


Since the release of Doctrine DBAL 3.0.0 in November 2020, the 2.x release series effectively went into the maintenance mode. In the past year, we've been accepting mostly the following types of patches for DBAL 2:

  1. Development dependency updates
  2. Security fixes
  3. Improvements to compatibility with PHP 8.1
  4. Improvements in the upgrade path to DBAL 3

Except for dependency updates, at the moment, there are no known issues in DBAL 2 that would fall into any of the above categories.

Many projects that depend on Doctrine DBAL depend on it indirectly via Doctrine ORM which until release 2.10.0 didn't support DBAL 3. It was one of the blockers of the DBAL 3 adoption which is no longer the case.

With all that said, the DBAL team announces the plan for sunsetting DBAL 2 in 6 months as of the ORM 2.10.0 release which is April 3, 2022. After that date, we plan to release DBAL 2 only to address security related and other critical issues for at most a year.

All Doctrine DBAL users are encourarged to upgrade to the latest stable version which is 3.3.0 as of the time of this writing.

New Release: Doctrine ORM 2.11 with Enums, Virtual Columns, Read-Only Properties, Nested Attributes and more

Posted on January 11, 2022 by Benjamin Eberlei


We have released a new minor version 2.11 of Doctrine ORM with several improvements and new features.

See all changes and contributors in the Changelog on Github.

This blog post gives an overview over all the new features and improvements that are user facing. Please see the changelog and UPGRADE notes for new deprecations.

PHP 8.1 Enum Support

With PHP 8.1 the language has first class support for enumerations and Doctrine ORM 2.11 supports the mapping of database values to Backed Enums.

The support is not integrated on DBAL Type level, but using a new mapping option called enumType on column/field declaration level:

enum Suit: string {
    case Hearts = 'H';
    case Diamonds = 'D';
    case Clubs = 'C';
    case Spades = 'S';
}

#[Entity]
class Card
{
    /** ... */

    #[Column(type: 'string', enumType: Suit::class)]
    public $suit;
}

Virtual and Generated Columns

There has been constant demand for this feature for a long time, to add support for columns that are not insertable/updatable and might have their value updated on the database side.

We have worked along the lines of Java Persistence API support of insertable, updatable and generated options for field mappings.

There are two major use cases for this:

  1. Map a column several times, for example with join columns:
#[Entity]
class User
{
     #[ManyToOne(targetEntity: Country:class), JoinColumn(name: "country_code", referencedColumnName: "country_code")]
     public $country;

     #[Column(type: "string", name: "country_code", insertable: false, updatable: false)]
     public $countryCode;
}
  1. Columns updated by the database
#[Entity]
class Article
{
    #[Column(type: "datetime",
        columnDefinition: "TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP",
        insertable: false,
        updatable: false,
        generated: "ALWAYS")]
    public $created;
}

Support for Readonly Properties

Another PHP 8.1 feature is the new readonly keyword that prevents the value of a property to be written again after it has been initialized in the constructor of an object.

With ORM 2.11 the support now works as you would expect with no additional mapping options necessary:

#[Entity, Table(name: 'author')]
class Author
{
    #[Column, Id, GeneratedValue]
    private readonly int $id;

    #[Column]
    private readonly string $name;
}

AssociationOverrides and AttributeOverrides in Attribute Driver

The new AttributeDriver for PHP 8 did not support the equivalent mapping options for association and attribute overrides that are available for XML and Annotation mapping, because in PHP 8.0 it was not possible to nest complex attributes.

With the support now available in PHP 8.1 we have added these attributes.

<?php
use Doctrine\ORM\Mapping\AssociationOverride;
use Doctrine\ORM\Mapping\AssociationOverrides;
use Doctrine\ORM\Mapping\AttributeOverride;
use Doctrine\ORM\Mapping\AttributeOverrides;
use Doctrine\ORM\Mapping\Column;
use Doctrine\ORM\Mapping\Entity;

#[AssociationOverrides([
    new AssociationOverride(
        name: "groups",
        joinTable: new JoinTable(
            name: "ddc964_users_admingroups",
        ),
        joinColumns: [new JoinColumn(name: "adminuser_id")],
        inverseJoinColumns: [new JoinColumn(name: "admingroup_id")]
    )
])]
#[AttributeOverrides([
    new AttributeOverride(
        name: "id",
        column: new Column(name: "guest_id", type: "integer", length: 140)
    )])]
class DDC964Admin extends DDC964User
{
}

For PHP 8.0 we have already moved Index and JoinColumn mappings to the top level to avoid nesting and decided not to allow nesting for these to mimic annotation support.

Allow arithmetic expressions within IN operator

It is now possible to use arithmetic expressions or functions inside the IN operator:

SELECT u FROM User u WHERE u.id IN (1 + 1, FOO(u.id))

Ignore entity classes in schema tool

You can now specify Entity FQCNs to ignore during schema tool creation and comparison. SchemaTool will then skip these (e.g. when comparing schemas).

<?php
$config->setSchemaIgnoreClasses([$fqcn]);
$config->getSchemaIgnoreClasses();

New Release: Doctrine MongoDB ODM 2.3 with Attributes, JSON Schema Validation, and more

Posted on December 4, 2021 by Ion Bazan


We have released a new minor version 2.3 of Doctrine MongoDB ODM, the first version with support for using PHP 8 Attributes as a new driver for mapping documents and several other changes. See all changes and contributors in the Changelog on GitHub.

Attributes Mapping Driver

The following code example shows many of the mappings that are re-using the annotation classes for familiarity:

use Doctrine\ODM\MongoDB\Mapping\Annotations as MongoDB;
use Doctrine\ODM\MongoDB\Types\Type;

#[MongoDB\Document(repositoryClass: PostRepository::class)]
class Post
{
    #[MongoDB\Id]
    private string $id;

    #[MongoDB\Field(type: Type::BOOLEAN)]
    private bool $published = false;

    #[MongoDB\Field(type: Types::COLLECTION)]
    private array $text = [];

    #[MongoDB\ReferenceOne(targetDocument: User::class)]
    public $author;

    #[MongoDB\ReferenceMany(targetDocument: Tag::class)]
    public Collection $tags;
}

You may want to use Rector with DoctrineSetList::DOCTRINE_ODM_23 set to convert all your annotation mappings to attributes in seconds!

JSON Schema Validation

MongoDB ≥ 3.6 offers the capability to validate documents during insertions and updates through a JSON schema associated with the collection. See MongoDB documentation.

Doctrine MongoDB ODM now provides a way to take advantage of this functionality thanks to the new #[Validation] mapping.

use Doctrine\ODM\MongoDB\Mapping\Annotations as MongoDB;
use Doctrine\ODM\MongoDB\Mapping\ClassMetadata;

#[MongoDB\Document]
#[MongoDB\Validation(
    validator: SchemaValidated::VALIDATOR,
    action: ClassMetadata::SCHEMA_VALIDATION_ACTION_WARN
)]
class SchemaValidated
{
    public const VALIDATOR = <<<'EOT'
{
    "$jsonSchema": {
        "required": ["name"],
        "properties": {
            "name": {
                "bsonType": "string",
                "description": "must be a string and is required"
            }
        }
    },
    "$or": [
        { "phone": { "$type": "string" } },
        { "email": { "$regex": { "$regularExpression" : { "pattern": "@mongodb\\.com$", "options": "" } } } },
        { "status": { "$in": [ "Unknown", "Incomplete" ] } }
    ]
}
EOT;
}

Once defined, those options will be added to the collection after running the odm:schema:create or odm:schema:update command.

Psalmified APIs

In-code documentation has been immensely improved to make sure static analysis tools and IDEs know about the right document classes returned from DocumentManager, ClassMetadata, and other public APIs. This includes generics support for your own repositories extending DocumentRepository.

use Doctrine\ODM\MongoDB\Repository\DocumentRepository;
use App\Document\User;

/**
 * @template-extends DocumentRepository<User>
 */
class UserRepository extends DocumentRepository
{
}

Deprecations

Doctrine MongoDB ODM 2.3 introduces several minor deprecations:

  • The Doctrine\ODM\MongoDB\Proxy\Resolver\ClassNameResolver interface has been deprecated in favor of the Doctrine\Persistence\Mapping\ProxyClassNameResolver interface
  • Annotation classes no longer extend Doctrine\Common\Annotations\Annotation class
  • Annotation arguments switched to @NamedArgumentConstructor for Attribute compatibility
  • @Inheritance annotation has been removed as it was never used
  • Document Namespace Aliases ('MyBundle:User) - use fully qualified class names instead (User::class)

Read more in our upgrading document.

Coding Standard Support

Doctrine MongoDB ODM 2.3 now supports and fully validates against Doctrine Coding Standard version 9.0+. This greatly improves automatic pull request checks as all new violations in a PR get caught and inlined into the PR as comments.

New Release: Doctrine DBAL 3.2.0

Posted on November 26, 2021 by Sergei Morozov


We are happy to announce the immediate availability of Doctrine DBAL 3.2.0. As most of the minor releases, this one focuses on new features, improvements and deprecations of the old APIs. Here are some details on the most significant features and improvements:

Platform-aware schema comparison (#4746)

Up until this release, the logic of comparing database schemas had a major design flaw: it took into account only the abstract schema definitions without taking the target platform into account.

This flaw would lead to multiple issues which shared the same root cause: the two definitions could be considered different by the DBAL, but they would produce the same DDL.

For instance, consider the two column definitions:

// old schema
$column1 = new Column('contents', Type::getType('text'));

// new schema
$column2 = new Column('contents', Type::getType('text'), ['default' => 'Hello, world!']);

If we compared them with the comparator, we'd get a diff:

$comparator = new Comparator();
$comparator->diffColumn($column1, $column2);
// array(1) {
//   [0] =>
//   string(7) "default"
// }

This might be valid for the platforms that support the DEFAULT constraint on TEXT columns but isn't valid for those that don't support it (e.g. MySQL). Regardless of the diff, both definitions would produce the same DDL on MySQL:

contents LONGTEXT NOT NULL

An attempt to migrate the old schema to the new one would produce a false-positive diff but applying it wouldn't result in any schema changes.

A false-negative diff was also possible. Consider these following example:

// old schema
$column1 = new Column('settings', Type::getType('json'));

// new schema
$column2 = new Column('settings', Type::getType('json'), ['length' => 16777215]);

Comparison of the above column definitions should have triggered a diff on MySQL and migrate the underlying column from TEXT to MEDIUMTEXT but it didn't, because the DBAL would ignore the length of the TEXT columns.

Apart from that, the DBAL would compare only a subset of the definitions, so some column options as the character set and collation weren't taken into account during comparison at all.

The new approach

Instead of comparing abstract definitions on a per-property basis, the new implementation compares the DDL that is generated from both definitions for the target database platform. If the definitions produce the same DDL, they are considered equal. According to the tests and the number of resolved issues, this approach should be more accurate and less error-prone.

Implementing this approach was impossible without introducing a new API which rendered the existing API obsolete.

Prior to DBAL 3.2.0, the schema comparator could be only instantiated directly via the new keyword:

$comparator = new Comparator();

Instantiated like this, the comparator doesn't have a notion of the target database platform and cannot perform the comparison properly. That is why, this way of instantiation is deprecated in favor of instantiating the comparator by the schema manager:

$schemaManager = $connection->createSchemaManager();
$comparator = $schemaManager->createComparator();

This way, the schema manager can instantiate a platform-specific comparator and provide it with the necessary context (e.g. the default collation used by the database).

While the old API is still available, it is recommended to use the new API for more accurate comparison.

Support for psr/cache (#4620)

Since the Doctrine Cache library is being sunset, the new DBAL release introduced the ability to use a PSR-6 compatible implementation for result caching.

While both the doctrine/cache and psr/cache APIs will be supported until the next major DBAL release, we recommend users to switch to a PSR-6 compatible implementation in their projects.

Support for psr/log (#4967)

The SQLLogger interface was designed long ago and has certain limitations: there is no way to log messages at different logger levels and it is really challenging to extend the logger functionality without introducing breaking API changes.

The new DBAL release introduces a new middleware that can delegate logging to a PSR-3 compatible implementation.

Note that the new logger won't produce the messages identical to the ones produced by the old one. If you have any processes built around analysing log messages, you may need to make some changes before adopting the new API.

Always cache the full result (#5003)

The implementation of the result cache prior to DBAL 3.2.0 would store the result set in the cache only once it was fetched completely. It led to the following issues:

  1. If the result isn't yet cached and its consumer didn't fetch it completely, the query would be executed again.
  2. In case of a cache miss, the DBAL would get() the cache entry twice: once to fetch the data and once to merge the just fetched result with other results that may be stored in the cache.

The new implementation stores the results in the cache right after they were fetched. It simplifies the caching layer significantly and makes its behavior more straightforward.

Add events for Transaction begin/commit/rollback (#4622)

The new DBAL version introduces three more transaction-related events:

  • onTransactionBegin,
  • onTransactionCommit,
  • onTransactionRollBack.

Subscribing to those might be helpful if the application logic integrates the database transaction flow with the business logic implemented outside the database. For instance, in the filesystem.

Basic exception handling in IBM DB2 and SQL Server drivers (#4929, #4928)

The DBAL provides a mechanism that converts driver-specific error codes to portable error-specific exceptions. For instance an attempt to insert NULL into a column that has a NOT NULL constraint applied will result in error with the code 1566 on MySQL and in ORA-01400 on Oracle. The DBAL will convert these two errors to a portable NotNullConstraintViolationException.

Historically, the DBAL drivers based on the ibm_db2, sqlsrv and pdo_sqlsrv extensions did not support this feature and would thow a generic DriverException.

As of DBAL 3.2.0, this feature is supported by all bundled drivers.

Improved AbstractPlatform::getLengthExpression() (#4855)

Although the LENGTH expression was implemented for all supported database platforms, the different implementations didn't have consistent semantics:

  1. Most implementations would return the length in characters (Unicode code points), which is the most expected behavior.
  2. The implementations for MySQL and IBM DB2 would return the number of bytes. It worked fine for the strings that consisted only of the ANSI characters, but an attempt to use it with a wider range of characters would produce an unexpected result. For instance, the length of the string 'Привет, мир!' might be reported as 19 instead of 12.

As of DBAL 3.2.0, all platforms return the length in Unicode points according to the character set used by the database connection. Note, SQL Server supports UTF-8 only as of SQL Server 2019.

You can find more details in the release notes.

DBAL 3 SQL Injection Security Vulnerability fixed (CVE-2021-43608)

Posted on November 11, 2021 by Benjamin Eberlei


We have released a new version Doctrine DBAL 3.1.4 that fixes a critical SQL injection vulnerability in the LIMIT clause generation API provided by the Platform abstraction.

We advise everyone using Doctrine DBAL 3.0.0 up to 3.1.3 to upgrade to 3.1.4 immediately.

The vulnerability can happen when unsanitized input is passed to many APIs in Doctrine DBAL and ORM that ultimately end up calling AbstractPlatform::modifyLimitQuery.

As a workaround you can cast all limit and offset parameters to integers before passing them to Doctrine APIs.

This vulnerability has been assigned CVE-2021-43608.

New Release: Doctrine ORM 2.9 with Attributes, Typed Properties, more

Posted on May 24, 2021 by Benjamin Eberlei


We have released a new minor version 2.9 of Doctrine ORM, the first version with support for using PHP 8 Attributes as a new driver for mapping entities and several other changes. See all changes and contributors in the Changelog on Github.

Attributes Mapping Driver

The following code example shows many of the mappings that are re-using the annotation classes for familiarity:

use Doctrine\DBAL\Types\Types;
use Doctrine\ORM\Mapping AS ORM;

#[ORM\Entity(repositoryClass: PostRepository::class)]
class Post
{
    #[ORM\Column(type: Types::INTEGER)]
    #[ORM\Id, ORM\GeneratedValue(strategy: 'AUTO')]
    private ?int $id;

    #[ORM\Column(type: Types::BOOLEAN)]
    private bool $published = false;

    #[ORM\Column(type: Types::SIMPLE_ARRAY)]
    private array $text = [];

    #[ORM\ManyToOne(targetEntity: User::class)]
    public $author;

    #[ORM\ManyToMany(targetEntity: Tag::class)]
    #[ORM\JoinTable(name: "post_tags")]
    #[ORM\JoinColumn(name: "post_id", referencedColumnName: "id")]
    #[ORM\InverseJoinColumn(name: "tag_id", referencedColumnName: "id")]
    public Collection $tags;
}

Typed Property Defaults

Since PHP 7.4 types can be declared on class properties and Doctrine now uses these type declarations to reduce amount of mapping boilerplate:

  • Columns don't need the type definitions
  • ManyToOne and OneToOne don't need target entity definitions

Example:

use Doctrine\ORM\Mapping AS ORM;

#[ORM\Entity(repositoryClass: UserRepository::class)]
class User
{
    #[ORM\Id, ORM\Column, ORM\GeneratedValue]
    public ?int $id = null;

    #[ORM\Column]
    public \DateTime $created;

    #[ORM\ManyToOne]
    public Email $email;
}

Psalmified APIs

Improved the documentation to make sure static analysis tools and IDEs know about the right entity classes returned from EntityManager, EntityRepository and other public ORM APIs. This includes generics support when you extend EntityRepository.

use Doctrine\ORM\EntityRepository;
use App\Entity\User;

/**
 * @template-extends EntityRepository<User>
 */
class UserRepository extends EntityRepository
{
}

Query::HINT_READ_ONLY

A new query hint is added that allows hydrating entities through DQL that are marked as read only for the unit of work session, as long as they are not yet loaded as writeable:

$dql = 'SELECT u FROM ' . ReadOnlyEntity::class . ' u WHERE u.id = ?1';

$query = $entityManager->createQuery($dql);
$query->setParameter(1, $user->id);
$query->setHint(Query::HINT_READ_ONLY, true);

$user = $query->getSingleResult();

Index/UniqueConstraints using Field Names

Instead of specifying column names for an index or unique-constraint declaration you can now alternatively use field names.


use Doctrine\ORM\Mapping AS ORM;

#[ORM\Entity]
#[ORM\Index(fields: ["isPublished"])]
class Post
{
    #[ORM\Column]
    public bool $isPublished = false;
}

This simplifies mapping as the column names passed through the naming strategy do not need to be known.

INDEX BY Associations

Previously DQL INDEX BY was not possible for assocations, now you can:

$dql = 'SELECT p, u FROM Post INDEX BY p.author JOIN p.author u WHERE p.id = 3';

Deprecations

Doctrine ORM 2.9 rethinks deprecations and integrates with our new doctrine/deprecations library.

  • Undeprecate merge() and detach() as no replacements are available yet
  • Notify Persist Change Tracking: Use Explicit Change Tracking instead
  • DQL SELECT PARTIAL syntax, use Value Objects with SELECT NEW instead
  • EntityManager::flush() with arguments
  • EntityManager::clear() with arguments (use detach)
  • Named Queries in Mapping (use Repository)
  • cli-config.php for console command configuration, inject EntityManagerProvider instead.
  • Deprecate doctrine/cache for metadata caching, use PSR-6 cache instead

Cache Deprecations and PSR-6

Over the next versions we will deprecate use of doctrine/cache and replace it with PSR-6. If you are still using doctrine/cache code in your own application make sure to force the versions to "^1.10" in composer.json. Details

PHP 7.1 Support

ORM 2.9 reintroduces PHP 7.1 support, because it wasn't technically unsupported anyways. No changes were necessary to the code to allow it again except in the testsuite.

The PHP 7.1 support was re-added to allow a very broad approach to prepare for some of the deprecations that are introduced in ORM 2 and will be removed in version 3.0.

Coding Standard Support

Doctrine ORM 2.9 now supports and fully validates against Doctrine Coding Standard version 9.0+. This greatly improves automatic pull request checks as all new violations in a PR get caught and inlined into the PR as comments.

New Release: Doctrine DBAL 2.13.1 and 3.1.0 with important Forward Compatibility fix

Posted on April 19, 2021 by Benjamin Eberlei


Last month we released DBAL 2.13.0 as an important push for the ecosystem towards DBAL 3 with an extensive deprecation and forward compatibility layer.

We made a few mistakes, given that the forward compatibility layer is quite complex. As such we have now released Doctrine DBAL 2.13.1 and 3.1.0 with two new APIs that improve the forward compatiblity.

The problem lies in Statement::execute(): 2.13.0 would return a bool and 3.0.0 would return a Result from this method. This kind of API change cannot be handled with a forward compatibility.

As such we introduced two new APIs on Statement that replace execute(). When the old code was:

$statement = $connection->prepare('SELECT * FROM tbl WHERE col = ?');
$statement->execute();

$rows = $statement->fetchAll();

Then the new code is now:

$statement = $connection->prepare('SELECT * FROM tbl WHERE col = ?');
$result = $statement->executeQuery();

$rows = $result->fetchAllAssociative();

The DBAL 2.13 forward compatiblity layer supports both versions of all code and returns a Statement/Result hybrid that has all the APIs that the DBAL Statement had up until version 2.12. This way you can move at your own pace from the old to the new API in your code.

Thank you again to mdumoulin for the work on improving the forward compatiblity and to Sergei Morozov for the thorough reviews and comments.

Again I want to highlight the Runtime Deprecations library that we introduced to support this migration. You can integrate this with your log stack during development and testing:

use Doctrine\Deprecations\Deprecation;
use Monolog\Logger;
use Monolog\Handler\StreamHandler;

$log = new Logger('doctrine');
$log->pushHandler(new StreamHandler('deprecations.log', Logger::INFO));

Deprecation::enableWithPsrLogger($log);

Or alternatively using PHP's global error handler:

Deprecation::enableWithTriggerError();

See the 2.13 blog post for more information about the migration to DBAL 3 and strategy recommendations.

New Release: Doctrine DBAL 2.13 with Deprecations and Forward Compatibility

Posted on March 29, 2021 by Benjamin Eberlei


We have released DBAL 2.13, what we plan to be the last minor version in the 2.x family of Doctrine DBAL. This release includes additional forward compatibility to DBAL 3 around Statement and Result API and with an integration in our new deprecations logging library.

In addition this DBAL release re-enables PHP 7.1 and 7.2 compatibility to give as much flexibility as possible to everyone with forward compatibility.

Statement and Result Forward Compatibility

DBAL 3.0 extracts all fetch-methods from the Statement API and moved them to a new Result API that is returned from Statement::execute. We have backported this API to 2.13 - so that you can support writing code for both DBAL 2 and 3 at the same time.

Old code:

$statement = $connection->prepare('SELECT * FROM tbl WHERE col = ?');
$statement->bindParam(1, $value);
$statement->execute();

while (($row = $statement->fetch()) !== false) {
}

$connection->executeQuery('SELECT * FROM tbl')->fetchAll();

New Code:

$statement = $connection->prepare('SELECT * FROM tbl WHERE col = ?');
$statement->bindParam(1, $value);
$result = $statement->executeQuery();

while (($row = $result->fetchAssociative()) !== false) {
}

$connection->executeStatement('SELECT * FROM tbl')->fetchAllAssociative();

The Result Fetching API was improved to use more human-readable names:

// Old
$stmt->fetch();
$stmt->fetch(PDO::FETCH_ARRAY);
$stmt->fetchColumn();
$stmt->fetchAll();

// New
$result = $stmt->execute();
$result->fetchAssociative();
$result->fetchNumeric();
$result->fetchOne();
$result->fetchAllAssociative();

Many more changes have been made on the public API and also for the internals, but these are the most common ones.

Thank you to mdumoulin for the work on improving the forward compatiblity.

Deprecations Logging

We have wrestled internally for a long time with the strategy on runtime deprecation going forward and settled on introducing a small, new API for reporting the usage of deprecated APIs.

The reason for this abstraction is the potential for side effects caused by an error handler and the potential overhead. We expect our deprecations to be triggered a few hundred times in some requests as such the production overhead must be minimal.

This means deprecation logging is disabled by default and you must enable it to either use @trigger_error or a PSR-3 compatible logger.

See the deprecation library README.md for details on how to configure and use it.

PHP 7.1 and 7.2 Support

A few large Doctrine DBAL deployments still support older versions of PHP that are not officially supported anymore, but are covered by support of a few Linux distributions.

To provide the largest possible flexibility to the ecosystem to run code with both DBAL 2 or 3 this version of Doctrine DBAL will work again with PHP 7.1 and 7.2

Migrate to DBAL 3

We recommend a three step strategy to move your code-base and that of your dependents to DBAL 3. It depends if you are working on a library or platform that is dependent upon, or if you are working on a standalone application.

For a standalone application:

  1. Upgrade to DBAL 2.13 and enforce "^2.13" as a version constraint in composer.json
  2. Enable deprecation tracking and eliminate all deprecations triggered in your codebase.
  3. After fixing all deprecations, update composer constraint to "^3.0". Doctrine will upgrade to version 3 if all other dependencies you are using are ready as well to upgrade to version 3. If it fails, you need to identify and update the dependencies as well.

For a library, framework or platform:

  1. Upgrade to DBAL 2.13 and enforce "^2.13" as a version constraint in composer.json
  2. Enable deprecation tracking and eliminate all deprecations triggered in your codebase. Release a version so that all plugins and downstream users can be notified of using deprecated Doctrine DBAL directly themselves.
  3. After fixing all deprecations, update composer constraint to "^2.13 | ^3.0". Doctrine will only upgrade to version 3 if all other dependencies you are using are ready as well to upgrade to version 3.

We recommend the following strategies to detect the use of deprecated code:

  • Use Psalm, other static analyzers or IDEs to detect the use of deprecated code.
  • If you have an extensive test-suite, register a PSR-3 logger with Doctrine Deprecations to catch all deprecations while running the tests.
  • Otherwise register a PSR-3 logger with Doctrine Deprecations in development or staging only and collect and fix them as well you can. This could be done for a longer amount of time of days, weeks or months.
  • If that is not possible, register a PSR-3 logger in production. Make sure to eliminate high frequency deprecations quickly or call ignoreDeprecation to snooze them to avoid overhead.

New Major Release: Doctrine DBAL 3.0

Posted on November 17, 2020 by Benjamin Eberlei


We have released a new major version of Doctrine DBAL, version 3.0.0. This new major version comes almost 10 years after DBAL 2.0 was released on December 2010, then coupled into the ORM 2.0.

Today Doctrine DBAL is released independent of the ORM, thanks to Composer and sees its new major version before the ORM.

This release was made possible foremost by Sergei Morozov, our primary DBAL maintainer, who has spent countless hours working on the package since 2016. Thank you!

See the Release Notes for a detailed list of changes.

This blog post covers a few of the major changes in a bit more detail to give you an idea of what DBAL 3.0 looks like.

Decouple DBAL from PDO

The major theme of DBAL 3.0 is the decoupling from PDO. Instead of copying the API verbatim like DBAL 2.0 did, DBAL 3.0 grows it into a better, more usable direction.

We extract all fetch-methods from the Statement class and moved them to a new Result class that is returned from Statement::execute.

Old code:

$statement = $connection->prepare('SELECT * FROM tbl WHERE col = ?');
$statement->bindParam(1, $value);
$statement->execute();

while ($row = $statement->fetch()) {
}

New Code:

$statement = $connection->prepare('SELECT * FROM tbl WHERE col = ?');
$statement->bindParam(1, $value);
$result = $statement->execute();

while ($row = $result->fetchAssociative()) {
}

The Result Fetching API was improved to use more human-readable names:

// Old
$stmt->fetch();
$stmt->fetch(PDO::FETCH_ARRAY);
$stmt->fetchColumn();
$stmt->fetchAll();

// New
$stmt->fetchAssociative();
$stmt->fetchNumeric();
$stmt->fetchOne();
$stmt->fetchAllAssociative();

Many more changes have been made on the public API and also for the internals, but these are the most visible ones.

Upgrading to DBAL 3 from 2

DBAL 3 is a real new major release with significant changes to the public API. Depending on your codebase a migration could require non-trivial work. However we do not intend to leave you hanging with DBAL 2 and a future migration:

  1. We intend to support DBAL 2.12 a while longer, including support for the upcoming PHP 8.0, so that there is no rush for you to upgrade to DBAL 3.

  2. DBAL 2.12 already includes forwards compatible API changes for all the new APIs, so that you can migrate your code step by step to the new APIs already.

  3. Deprecated methods in DBAL 2.12 are tagged with the @deprecated doc-block and static analysis tools such as Psalm, PHPStan and Phan can already help you detect using or calling this deprecated code in your application.

  4. We intend to release another version of DBAL 2 which includes optional triggering of deprecation messages at runtime similar to how Symfony deals with deprecations.

As you can see, with these approaches it will be possible for you to smoothly migrate your application from DBAL 2 to 3.

As a note to libraries and frameworks that need to support multiple versions of Doctrine DBAL: We recommend you start out with DBAL 2.12 and migrate all usages of deprecated APIs to their newer counterparts. Then once you have managed this, you can allow 3.0 and try to get your code working with both versions. Unfortunately this may not be possible for all cases, because we could not provide replacement APIs for everything and some features have been dropped between DBAL 2 and 3.

Outlook

DBAL 3 is a huge first step towards a modern database abstraction layer, independent from the legacy of PHP and PDO API design. In the future we plan to improve DBAL in other ways that we haven't gotten around yet, such as more API modernization, increased safety with use of strict scalar types in the code base, better error handling and more.

Released doctrine/migrations 3.0-alpha

Posted on April 10, 2020 by Asmir Mustafic


doctrine/migrations 3.0-alpha has been published on the 29th March 2020.

The upcoming 3.0 new major release is the result of almost 6 months of work and brings a completely refactored/rewritten internal structure and some interesting new features.

Why a new major release?

The doctrine/migrations v1.x codebase is 10 years old, and in the past years a lot of features have been added on top of its initial architecture.
doctrine/migrations 2.0 was released a bit more than a year ago. This major release did a bit of cleanup, but the general structure remained the same. In this schema you can see the dependencies between classes in the latest 2.3.x branch:

The red lines are circular dependencies (and we already know that in software development circular dependencies are not a good thing).

In doctrine/migrations 3.x, most of the internal classes have been re-written and dependency injection has been widely adopted.
In this schema you can see the dependencies between classes in the latest master branch (release v3.0):

As you can see the circular dependencies are gone. This has been possible thanks to extensive use of dependency injection and applying SOLID principles. To reduce future backward incompatibilities, many classes have been marked as final or as @internal while keeping the functionalities intact. Extensibility is still possible by using dependency injection and providing classes implementing dedicated interfaces.

These schemas have been generated using PhpDependencyAnalysis with this configuration.

New features and improvements

Beside the code quality improvements, there is a a long list of improvements (see below), but the main user-facing feature is the ability to collect migrations from multiple folders/namespaces and to specify dependencies between migrations.

Here a (probably not complete) list of improvements implemented in the upcoming 3.0 release:

  • ability to collect migrations from multiple folders/namespaces and to specify dependencies between migrations
  • doctrine/migrations will write to your database only when running migrations (previously the metadata table was created on the very first command run even if it was a read-only command)
  • Output verbosity can be controlled using the -v, -vv or -vvv command parameters
  • Use of dependency injection allows you to decorate/replace most services
  • Removed usage of console helpers to provide the connection or entity manager in favor of dependency injection
  • Introduced migrations:list command to list the available/executed migrations
  • Introduced migrations:sync-metadata-storage command to explicitly update the metadata schema in case a newer version introduces changes to the metadata table
  • Multiple migrations can be passed to the migrations:execute command
  • More organized output of the migrations:status command
  • Configurations and Dependency Factory are read-only during the migration process
  • The down() migration is optional now
  • Multi-namespace migrations
  • Custom migrations metadata storage
  • Added warning when using the migrations:diff if there are not executed migrations

Backward compatibility

In doctrine/migrations 3.0 a lot of things changed, but for end-users most of the things will look the same. Your migration files do not need any update.

You will have to change your configuration files, as the configuration format has changed. The official documentation contains more information about these changes. This documentation should be particularly helpful if you did also some custom integration with third party frameworks or libraries.

If you wrote custom event listeners, please take a look at them as the method signatures for event listeners have been updated.

Symfony Integration

If you are using DoctrineMigrationsBundle then things are even easier: the 2.3.0 release introduced some deprecation notices and if you have already solved them your configuration is already compatible. If you want you can have a look to the latest configuration format available on the official documentation. You can look more in detail to which changes are needed in the upgrading document.

What is next

In the upcoming weeks, we will be preparing the first beta release and starting the process to reach a stable release. To be able to deliver a good stable release it is important that you test the pre-release and share your feedback!

To try the alpha version, you can run:

composer require 'doctrine/migrations:^[email protected]'

If you are using Symfony:

composer require 'doctrine/doctrine-migrations-bundle:^[email protected]' 'doctrine/migrations:^[email protected]'

You can be also more brave trying the development versions by specifying @dev instead of @alpha when requiring the composer dependencies above.

You can also have a look at the release notes and the upgrading document.

Similarly you can also have a look at the release notes and the upgrading document for the Symfony bundle.

In the alpha release, breaking changes are still possible. In the beta, release breaking changes are possible but will happen only if we will find very unexpected behaviors. When the alpha and beta phase will be completed, a stable version will be made available.

Note

This post was initially published on https://www.goetas.com/blog/released-doctrinemigrations-30-alpha/.

View Blog Archive