Doctrine ORM 3 and DBAL 4 Released

Posted on February 3, 2024 by Jonathan H. Wage


We are thrilled to announce the release of Doctrine ORM 3.0 and DBAL 4.0. These releases are the culmination of over a decade of hard work across dozens of contributors and the Doctrine maintainers.

What's New

A Slimmer, More Efficient ORM: The new Doctrine ORM 3.0 comes in at 326KB, down from 400KB in ORM 2.18.0. This reduction not only makes the ORM lighter but also signals our efforts to streamline and optimize every aspect of our library and focus our maintenance efforts on the core functionality of an ORM and less on tooling and helpers that are only useful by a small number of our users.

Enhanced Code Quality and Coverage: With ORM 3.0, we've pushed our code coverage from 84% to 89%. For DBAL 4.0, we've pushed our code coverage from 86% to 94%. This improvement underscores our commitment to reliability and the stability of the Doctrine ecosystem, ensuring that your applications run smoothly.

Leaner Dependencies: In Doctrine ORM 3.0, we have finally eliminated dependencies on doctrine/cache and doctrine/common. This change reduces complexity and improves maintainability of Doctrine as we now depend on PSR-6: Caching Interface for our caching responsibilities. Implementing a PSR means we are more interoperable with other frameworks and easier to use by a broader amount of users.

A Growing Community: The Doctrine project now boasts 1029 contributors across all its projects. This vibrant community is the backbone of Doctrine, providing valuable insights, feedback, and contributions that drive the project forward.

Upgrading

We understand that upgrading to a new major version can be difficult. The best way to upgrade is to first upgrade to the latest Doctrine ORM 2.x and DBAL 3.x version and address any deprecation warnings that are reported. You can read more about how Doctrine handles deprecations here. Once you have addressed all of the deprecations, you should have a clear path to upgrade.

In addition to that, we've maintained comprehensive documentation about every change, deprecation and BC break to facilitate a smooth transition to ORM 3.0 and DBAL 4.0.

The Future of Doctrine ORM 2

We plan to maintain Doctrine ORM 2 for at least the next 2 years by providing bug and security fixes. We may also add or deprecate things in 2.x to improve the existing forward-compatbility layer to make the transition to ORM 3 smoother.

Looking Forward

ORM 3 and DBAL 4 are a big step forward towards modernizing the API of our libraries, increasing safety with the use of scalar types in the code base, better error handling and generally cleaning up the code to make it easier to maintain. We look forward to continuing work on Doctrine and focusing on being the most stable and reliable PHP database persistence related libraries available.

Archiving Unmaintained Packages

Posted on January 30, 2024 by Andreas Braun


After long consideration, we have decided to archive a number of repositories that have not seen any activity in a while. This affects the CouchDB and OrientDB ODMs and their respective libraries, as well as the KeyValueStore project. The following repositories and composer packages are affected:

The composer packages will remain available and installable, but we will not be making any bug fixes or security fixes in the affected libraries. If you or your business depends on one of these libraries, please fork them and maintain them yourself in the future.

ORM 3.0 Beta 1, DBAL 4 RC 1 and future plans

Posted on October 11, 2023 by Benjamin Eberlei


We have released the first beta of the long awaited Doctrine ORM 3 and a release candidate of DBAL 4.

The target audience for these releases are framework integration and extension library authors. ORM 3 is not yet production ready and the APIs may change.

Our goal is to release ORM 3.0 as soon as possible and to gather feedback from greenfield project authors first.

This beta release is the result of a lot of work by many contributors, especially Grégoire, Alexander, Claudio and Matthias on ORM, Sergei and Alexander on DBAL. To iron out the final details, we met in Düsseldorf for a Doctrine Core Team meeting, generously funded by our sponsors through OpenCollective and GitHub. We also welcomed Matthias as the latest member of the Doctrine Core Team.

Continued ORM 2 support and forward compatibility

We will maintain the latest branch of the 2 line in ORM for at least another 2 years, possibly longer, to give you enough time to upgrade and us more time to learn from upgrader feedback and improve forward compatibility.

This means that we will be making ORM 2.x work with newer versions of PHP, fixing security bugs, and introducing layers and features that help with forward compatibility in the upgrade path to ORM 3.

Current users of ORM 2 should note that there is no urgency right now to update to ORM 3, as we are still working on replacement APIs and forward compatibility, and do not intend to ship them all with ORM 3.0, but with later versions.

ORM 2 users can already prepare for 3 by addressing deprecations

But there is already work to be done as an ORM 2 user: to help you find all the places where things may be deprecated or changing behaviour, we have created the doctrine/deprecations library and integrated it heavily into DBAL, ORM and other components.

It allows the use of deprecated behaviour to be logged at runtime with low overhead, automatic suppression of the same deprecation occurring multiple times, and a way to ignore selected deprecations for the time being. Each deprecation message always links to a GitHub issue with more details.

Many deprecated features have no replacement, such as Mapping Exporters, Generate Mapping from Database, Named Queries.

For some of the deprecations in ORM, we are still planning replacement APIs, especially:

  • There is currently no way to limit the number of entities that the flush operation considers changed. Flush will currently always calculate change sets on all entities that are not read-only.
  • As a replacement for removing PARTIAL object hydration, we are looking at making embeddable objects lazy, perhaps improving nesting of the new DTO expression in DQL. We are also looking to introduce subselect or batch loading for collections for more efficient multi-level hydration.

These will be released in 2.x as forward compatible APIs so that you can switch to using them before upgrading to ORM 3.

Doctrine ORM Team Meetup in Bonn, Germany

Posted on August 21, 2023 by Benjamin Eberlei


We are organizing a Doctrine ORM Core Team Meetup in Düsseldorf, Germany from Monday, 9.10.2023 to Wednesday, 11.10.2023 at the offices of one of our primary sponsors Tideways GmbH.

The goal is to get the current team together, discuss and work on the missing pieces for the long-awaited Doctrine ORM 3.0 release that is planned for later this year.

From annotations to attributes

Posted on November 4, 2022 by Grégoire Paris


Last month, we migrated the tests of the ORM from annotations to attributes. Let us look back on what lead to this moment.

Annotations

Let's go 22 years back in time. In October 2000, Ulf Wendel introduces phpdoc comments at the PHP-Kongress. These comments follow a structure that allows to produce API documentation from them. They are inspired by javadoc.

In 2002, Alex Buckley, a Specification lead for the Java language publishes JSR-175, thus proposing to add user-defined annotations to the language, allowing to tag language elements with extra information. 2 years later, it gets approved and Java 1.5, also known as Java 5 is released, with support for annotations.

4 more years elapse and in 2006, Jano Suchal publishes Addendum, a PHP library that adds support for using "Docblock/JavaDoc" as annotations, meaning that contrary to what is done in Java, Addendum annotations are contained inside phpdoc comments, like this:

/** @test */
function test_it_throws_on_invalid_argument(): void
{}

That is because they are implemented in userland, without requiring a change in PHP itself.

Doctrine ORM 2.0 is not released yet at that point, but the library is used to build an annotation driver in Doctrine 2 in early 2009. At that time, Doctrine was a project in a single repository, with Common, DBAL and ORM as top-level namespaces. Addendum is replaced 6 months later, with a new namespace under Common called Annotations.

In the summer of 2010, Guilherme Blanco and Pierrick Charron submit an RFC to add annotations support to PHP, but it gets declined. The RFC already mentions the need for annotations in PHPUnit, Symfony, Zend Framework, FLOW3 and of course, Doctrine.

Late 2010, Doctrine 2 is tagged, and the single repository is split into 3 repositories.

Finally, in 2013, the namespace above is isolated in its own repository, and doctrine/annotations 1.0.0 is tagged.

Today, the package is widely used in the PHP ecosystem and has a little short of 300 million downloads on Packagist, and is depended on by over 2 thousand packages, including major frameworks and tools. It is fair to say annotations have proven valuable to many users.

Attributes

The RFC mentioned above is only one among many. As mentioned before, annotations were implemented as phpdoc comments, which has several drawbacks:

  • The comments are necessary to run the code, and need to be kept in the opcode cache.
  • They are obtained at runtime, by using the reflection API, and because of that, can only be detected as invalid at runtime.
  • They are not well supported by IDEs if at all.
  • They clutter comments, which were originally intended for humans.
  • They can be confused with phpdoc, which are something else entirely.

In March 2020, Benjamin Eberlei resurrects Dmitry Stogov's attributes RFC and submits the seventh RFC on this topic, introducing attributes to PHP.

A few rounds of RFCs about syntax later, PHP 8.0 is released, with a notable feature missing: nested attributes. PHP 8.0 attributes use a syntax that is forward-compatible with them though, and finally, with PHP 8.1, nested attributes are supported.

Migrating from one to the other

Since attributes are much better than annotations, with doctrine/orm 3.0, annotations will no longer be supported, which means applications using them as a way to map entities to tables need to migrate towards attributes (or another driver). As maintainers of that library, even we needed to migrate: most of the test suite of doctrine/orm used annotations.

Enter Rector. Rector is a standalone tool that is invaluable when it comes to performing such migrations: it is able to understand PHP code and apply so-called Rectors to it. It is extensible, so it is possible to define such Rectors in order to address upgrades for anything, including Doctrine.

What's more, it comes with batteries included: when you install rector/rector, what you get is code from rector/rector-src and its official extensions, among which you will find rector/rector-doctrine. That's right, there is already an entire extension dedicated to Doctrine.

Rules are grouped together in sets, and the set that interests us here is Rector\Doctrine\Set\DoctrineSetList::ANNOTATIONS_TO_ATTRIBUTES.

To migrate doctrine/orm's test suite to annotations, here is how we proceeded:

  1. Install Rector: composer require --dev rector/rector.
  2. Create a file called rector.php at the root of the library with the following contents:

        <?php
    
        declare(strict_types=1);
    
        use Rector\Config\RectorConfig;
        use Rector\Doctrine\Set\DoctrineSetList;
    
        return function (RectorConfig $rectorConfig): void {
            $rectorConfig->paths([
                __DIR__ . '/tests',
            ]);
            $rectorConfig->sets([
                DoctrineSetList::ANNOTATIONS_TO_ATTRIBUTES,
            ]);
        };
  3. Run vendor/bin/rector, which obeys the above configuration.
  4. Uninstall Rector: composer remove rector/rector && rm rector.php
  5. Run vendor/bin/phpcbf to make the migrated codebase compliant with our coding standard.

Or at least, it was the plan, because some annotations were not perfectly migrated. All in all, I found only 2 bugs, which looks great given how so many edge cases should appear in our test suite.

I went on and reported those 2 bugs, and this is where the experience went from great to stellar: the issue template leads to a playground, much like the one you can find for other tools such as Psalm or PHPStan.

This one comes with 2 buttons: "Create an issue", which pre-fills the Github issue with useful information, and "Create a test", that lets you create a test in the right directory (and also, the right repository, which is rectorphp/rector-src, and not rectorphp/rector).

If you want to add a new test for the bug you reported, you should let the official tutorial walk you through that, it is very well written.

Anyway, now that these 2 bugs are fixed and you know how to migrate, plan that migration, and let us know how it goes! Happy Rectoring!

New Release: Doctrine DBAL 3.4.0

Posted on August 6, 2022 by Sergei Morozov


Doctrine is proud to announce the release of Doctrine DBAL 3.4.0. Below is a summary of the most noteworthy changes in the new release:

Database schema introspection optimization (#5268)

Older DBAL versions, in order to introspect database schema, performed a set of queries for each table individually. This caused noticeable performance issues on some platforms like Oracle which seemingly rebuild their internal views for each such query.

As of this release, the entire schema is introspected in a fixed number of queries. The more tables the schema contains, the more noticeable this optimization should be.

It was impossible to make these optimizations while using the schema introspection platform methods (e.g. getListTableColumnsSQL()). As a result, although these methods are kept in the codebase for backward compatibility, the DBAL itself no longer uses them. The SQL queries used for schema introspection are no longer considered part of the public DBAL API.

Support for foreign key constraints on SQLite (#5427)

Although SQLite has supported foreign key constraints since its earliest versions, their support in the DBAL was quite limited. One of the reasons for that was that managing foreign key constraints in SQLite is quite different from the rest of the supported platforms.

For example, when a foreign key constraint is declared, platforms like MySQL require that the referenced table must already exist. To support creating tables with mutually referencing constraints, the DBAL would create the tables first and create the constraints via ALTER TABLE … ADD FOREIGN KEY ….

This approach doesn't work with SQLite since it doesn't allow adding constraints to an existing table. Fortunately, it doesn't require the referenced table to exist at the time of creating the foreign key either.

The new DBAL release introduces a new API for building CREATE TABLE and DROP TABLE statements for multiple tables which could be tailored to the requirements of a given platform. The AbstractPlatform::supportsForeignKeys() method is now deprecated since the DBAL supports foreign key constraints on all supported platforms.

Support for TEXT/BLOB default values on MariaDB (#5332)

The platform layer in the DBAL is organized in the way that the code implementing the support for MySQL is also used to support MariaDB. As a result, even though MariaDB may support certain features the DBAL doesn't support them because they are not supported by MySQL. One of such features is the default values for TEXT and BLOB columns.

As of the new release, the default TEXT and BLOB values are supported on MariaDB but are still unsupported on MySQL, even though MySQL supports them as of release 8.0.13.

Support for result caching in QueryBuilder (#5539)

The recently added enableResultCache() method of the QueryBuilder class allows specifying the query cache profile to be used for performing the queries built by the builder.

PHP 7.4 or newer is required (#5459)

The DBAL no longer supports PHP 7.3 since its support by the community ended last year. The codebase now actively uses such features of PHP 7.4 as covariant return types and typed properties.

Deprecations

In light of the DBAL 4 release planned for later this year, the 3.4.0 release introduces over 30 deprecations which, as usual, focus on cleaning up obsolete features and making the API more robust and clearer from the static analysis standpoint.

To learn more about upgrading your application, see the upgrade notes. You can find the full list of changes in the release milestone.

On Doctrine Sponsoring with the help of OpenCollective

Posted on March 24, 2022 by Benjamin Eberlei


To simplify our own administrative burden, we have decided to move over the funds to OpenCollective, primarily motivated by the success and positive experience of the new PHP Foundation.

We have started raising money for the Doctrine Project in 2019 through Patreon and Github Sponsors to support the project. It was planned to organize core team get-togethers/hackathons, but due to the pandemic we haven't been able to do this. In addition the legal and tax implications of raising money also caused us some headaches.

The move to OpenCollective will allow us to delegate much of the administrative work to them for a small percentage of the raised capital. The fee is a much smaller amount than the taxes that we had to pay on the raised money previously, so it is a win-win for us.

We want to assure our sponsors that we still plan to make use of the funds to further the Doctrine project and we are actively looking to increase our funding for these goals:

  • Regularly organize hackathons for Doctrine Core team contributors, including 3-4 days of accommodation, food and travel for roughly 10-15 people.

  • Infrastructure, servers and hosting.

  • Marketing material such as stickers, t-shirts and other small things.

  • If the budget increases significantly we might be able to pay someone part- or full-time to do maintenance work such as responding to and processing issues, prepare for new PHP releases and general cleanups of the codebase.

As such we hope to convince you to sponsor Doctrine through either our Github Sponsors or OpenCollective directly.

Sunsetting Doctrine DBAL 2

Posted on January 22, 2022 by Sergei Morozov


Since the release of Doctrine DBAL 3.0.0 in November 2020, the 2.x release series effectively went into the maintenance mode. In the past year, we've been accepting mostly the following types of patches for DBAL 2:

  1. Development dependency updates
  2. Security fixes
  3. Improvements to compatibility with PHP 8.1
  4. Improvements in the upgrade path to DBAL 3

Except for dependency updates, at the moment, there are no known issues in DBAL 2 that would fall into any of the above categories.

Many projects that depend on Doctrine DBAL depend on it indirectly via Doctrine ORM which until release 2.10.0 didn't support DBAL 3. It was one of the blockers of the DBAL 3 adoption which is no longer the case.

With all that said, the DBAL team announces the plan for sunsetting DBAL 2 in 6 months as of the ORM 2.10.0 release which is April 3, 2022. After that date, we plan to release DBAL 2 only to address security related and other critical issues for at most a year.

All Doctrine DBAL users are encouraged to upgrade to the latest stable version which is 3.3.0 as of the time of this writing.

For migrating from DBAL 2 to 3, see our two blog posts on DBAL 2.13 Forward Compatibility Layer:

New Release: Doctrine ORM 2.11 with Enums, Virtual Columns, Read-Only Properties, Nested Attributes and more

Posted on January 11, 2022 by Benjamin Eberlei


We have released a new minor version 2.11 of Doctrine ORM with several improvements and new features.

See all changes and contributors in the Changelog on Github.

This blog post gives an overview over all the new features and improvements that are user facing. Please see the changelog and UPGRADE notes for new deprecations.

PHP 8.1 Enum Support

With PHP 8.1 the language has first class support for enumerations and Doctrine ORM 2.11 supports the mapping of database values to Backed Enums.

The support is not integrated on DBAL Type level, but using a new mapping option called enumType on column/field declaration level:

enum Suit: string {
    case Hearts = 'H';
    case Diamonds = 'D';
    case Clubs = 'C';
    case Spades = 'S';
}

#[Entity]
class Card
{
    /** ... */

    #[Column(type: 'string', enumType: Suit::class)]
    public $suit;
}

Virtual and Generated Columns

There has been constant demand for this feature for a long time, to add support for columns that are not insertable/updatable and might have their value updated on the database side.

We have worked along the lines of Java Persistence API support of insertable, updatable and generated options for field mappings.

There are two major use cases for this:

  1. Map a column several times, for example with join columns:
#[Entity]
class User
{
     #[ManyToOne(targetEntity: Country:class), JoinColumn(name: "country_code", referencedColumnName: "country_code")]
     public $country;

     #[Column(type: "string", name: "country_code", insertable: false, updatable: false)]
     public $countryCode;
}
  1. Columns updated by the database
#[Entity]
class Article
{
    #[Column(type: "datetime",
        columnDefinition: "TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP",
        insertable: false,
        updatable: false,
        generated: "ALWAYS")]
    public $created;
}

Support for Readonly Properties

Another PHP 8.1 feature is the new readonly keyword that prevents the value of a property to be written again after it has been initialized in the constructor of an object.

With ORM 2.11 the support now works as you would expect with no additional mapping options necessary:

#[Entity, Table(name: 'author')]
class Author
{
    #[Column, Id, GeneratedValue]
    private readonly int $id;

    #[Column]
    private readonly string $name;
}

AssociationOverrides and AttributeOverrides in Attribute Driver

The new AttributeDriver for PHP 8 did not support the equivalent mapping options for association and attribute overrides that are available for XML and Annotation mapping, because in PHP 8.0 it was not possible to nest complex attributes.

With the support now available in PHP 8.1 we have added these attributes.

<?php
use Doctrine\ORM\Mapping\AssociationOverride;
use Doctrine\ORM\Mapping\AssociationOverrides;
use Doctrine\ORM\Mapping\AttributeOverride;
use Doctrine\ORM\Mapping\AttributeOverrides;
use Doctrine\ORM\Mapping\Column;
use Doctrine\ORM\Mapping\Entity;

#[AssociationOverrides([
    new AssociationOverride(
        name: "groups",
        joinTable: new JoinTable(
            name: "ddc964_users_admingroups",
        ),
        joinColumns: [new JoinColumn(name: "adminuser_id")],
        inverseJoinColumns: [new JoinColumn(name: "admingroup_id")]
    )
])]
#[AttributeOverrides([
    new AttributeOverride(
        name: "id",
        column: new Column(name: "guest_id", type: "integer", length: 140)
    )])]
class DDC964Admin extends DDC964User
{
}

For PHP 8.0 we have already moved Index and JoinColumn mappings to the top level to avoid nesting and decided not to allow nesting for these to mimic annotation support.

Allow arithmetic expressions within IN operator

It is now possible to use arithmetic expressions or functions inside the IN operator:

SELECT u FROM User u WHERE u.id IN (1 + 1, FOO(u.id))

Ignore entity classes in schema tool

You can now specify Entity FQCNs to ignore during schema tool creation and comparison. SchemaTool will then skip these (e.g. when comparing schemas).

<?php
$config->setSchemaIgnoreClasses([$fqcn]);
$config->getSchemaIgnoreClasses();

New Release: Doctrine MongoDB ODM 2.3 with Attributes, JSON Schema Validation, and more

Posted on December 4, 2021 by Ion Bazan


We have released a new minor version 2.3 of Doctrine MongoDB ODM, the first version with support for using PHP 8 Attributes as a new driver for mapping documents and several other changes. See all changes and contributors in the Changelog on GitHub.

Attributes Mapping Driver

The following code example shows many of the mappings that are re-using the annotation classes for familiarity:

use Doctrine\ODM\MongoDB\Mapping\Annotations as MongoDB;
use Doctrine\ODM\MongoDB\Types\Type;

#[MongoDB\Document(repositoryClass: PostRepository::class)]
class Post
{
    #[MongoDB\Id]
    private string $id;

    #[MongoDB\Field(type: Type::BOOLEAN)]
    private bool $published = false;

    #[MongoDB\Field(type: Types::COLLECTION)]
    private array $text = [];

    #[MongoDB\ReferenceOne(targetDocument: User::class)]
    public $author;

    #[MongoDB\ReferenceMany(targetDocument: Tag::class)]
    public Collection $tags;
}

You may want to use Rector with DoctrineSetList::DOCTRINE_ODM_23 set to convert all your annotation mappings to attributes in seconds!

JSON Schema Validation

MongoDB ≥ 3.6 offers the capability to validate documents during insertions and updates through a JSON schema associated with the collection. See MongoDB documentation.

Doctrine MongoDB ODM now provides a way to take advantage of this functionality thanks to the new #[Validation] mapping.

use Doctrine\ODM\MongoDB\Mapping\Annotations as MongoDB;
use Doctrine\ODM\MongoDB\Mapping\ClassMetadata;

#[MongoDB\Document]
#[MongoDB\Validation(
    validator: SchemaValidated::VALIDATOR,
    action: ClassMetadata::SCHEMA_VALIDATION_ACTION_WARN
)]
class SchemaValidated
{
    public const VALIDATOR = <<<'EOT'
{
    "$jsonSchema": {
        "required": ["name"],
        "properties": {
            "name": {
                "bsonType": "string",
                "description": "must be a string and is required"
            }
        }
    },
    "$or": [
        { "phone": { "$type": "string" } },
        { "email": { "$regex": { "$regularExpression" : { "pattern": "@mongodb\\.com$", "options": "" } } } },
        { "status": { "$in": [ "Unknown", "Incomplete" ] } }
    ]
}
EOT;
}

Once defined, those options will be added to the collection after running the odm:schema:create or odm:schema:update command.

Psalmified APIs

In-code documentation has been immensely improved to make sure static analysis tools and IDEs know about the right document classes returned from DocumentManager, ClassMetadata, and other public APIs. This includes generics support for your own repositories extending DocumentRepository.

use Doctrine\ODM\MongoDB\Repository\DocumentRepository;
use App\Document\User;

/**
 * @template-extends DocumentRepository<User>
 */
class UserRepository extends DocumentRepository
{
}

Deprecations

Doctrine MongoDB ODM 2.3 introduces several minor deprecations:

  • The Doctrine\ODM\MongoDB\Proxy\Resolver\ClassNameResolver interface has been deprecated in favor of the Doctrine\Persistence\Mapping\ProxyClassNameResolver interface
  • Annotation classes no longer extend Doctrine\Common\Annotations\Annotation class
  • Annotation arguments switched to @NamedArgumentConstructor for Attribute compatibility
  • @Inheritance annotation has been removed as it was never used
  • Document Namespace Aliases ('MyBundle:User) - use fully qualified class names instead (User::class)

Read more in our upgrading document.

Coding Standard Support

Doctrine MongoDB ODM 2.3 now supports and fully validates against Doctrine Coding Standard version 9.0+. This greatly improves automatic pull request checks as all new violations in a PR get caught and inlined into the PR as comments.

View Blog Archive