README

This lightweight library allows you to perform safer Doctrine migrations during deployment in cluster-based environments like kubernetes where rolling-update takes place. Each migration has two up phases, no down phase.

before
- to be called before any traffic hits the new application version
- typically contains ADD COLUMN etc.
after
- to be called after the deployment is done and no traffic is hitting the old application version
- typically contains DROP COLUMN etc.

You can see Czech talk about this library on YouTube.

Installation:

composer require shipmonk/doctrine-two-phase-migrations

Configuration in Symfony application:

return [
    // ...
    ShipMonk\Doctrine\Migration\Bridge\Symfony\TwoPhaseMigrationsBundle::class => ['all' => true],
];

Then configure it in config/packages/two_phase_migrations.yaml:

two_phase_migrations:
    migrations_dir: '%kernel.project_dir%/migrations'

    # optional parameters:
    # migration_table_name: 'doctrine_migration'
    # migration_class_namespace: 'YourCompany\Migrations'
    # migration_class_prefix: 'Migration'
    # excluded_tables: ['my_tmp_table']
    # template_file_path: '%kernel.project_dir%/migrations/my-template.txt'
    # template_indent: "\t\t"

The bundle requires symfony/http-kernel ^6.4+. All commands and services are registered automatically, and Doctrine\ORM\EntityManagerInterface is autowired. If you register a custom MigrationExecutor, MigrationAnalyzer, or MigrationVersionProvider service, it will be picked up automatically.

All commands accept an optional Psr\Log\LoggerInterface. When a logger is available (e.g. via MonologBundle), structured log messages with context are emitted. When no logger is injected, commands fall back to Symfony\Component\Console\Logger\ConsoleLogger and write directly to the console output.

Manual service registration (without the bundle)

If your Doctrine\ORM\EntityManagerInterface is autowired, just register few services in your DIC and tag the commands:

_instanceof:
    Symfony\Component\Console\Command\Command:
        tags:
            - console.command

services:
    ShipMonk\Doctrine\Migration\Command\MigrationInitCommand:
    ShipMonk\Doctrine\Migration\Command\MigrationRunCommand:
    ShipMonk\Doctrine\Migration\Command\MigrationSkipCommand:
    ShipMonk\Doctrine\Migration\Command\MigrationCheckCommand:
    ShipMonk\Doctrine\Migration\Command\MigrationGenerateCommand:
    ShipMonk\Doctrine\Migration\MigrationService:
    ShipMonk\Doctrine\Migration\MigrationConfig:
        $migrationsDir: "%kernel.project_dir%/migrations"

    # more optional parameters:
        $migrationClassNamespace: 'YourCompany\Migrations'
        $migrationTableName: 'doctrine_migration'
        $migrationClassPrefix: 'Migration' # will be appended with date('YmDHis') by default
        $excludedTables: ['my_tmp_table'] # migration table ($migrationTableName) is always added to excluded tables automatically
        $templateFilePath: "%kernel.project_dir%/migrations/my-template.txt" # customizable according to your coding style
        $templateIndent: "\t\t" # defaults to spaces

Commands:

Initialization:

After installation, you need to create migration table in your database. It is safe to run it even when the table was already initialized.

$ bin/console migration:init

# example output:
[info] Initializing migration table doctrine_migration
[info] Migration table doctrine_migration created successfully

Generating new migration:

You can generate migration from database <=> entity diff automatically. This puts all the queries generated by Doctrine to before stage, which will NOT be correct for any destructive actions. Be sure to verify the migration and move the queries to proper stage or adjust them. When no diff is detected, empty migration class is generated.

$ bin/console migration:generate

# example output:
[info] Starting migration generation
[info] 1 schema changes detected
[info] Migration version 20230217063818 generated successfully

The generated file then looks like this:

<?php declare(strict_types = 1);

namespace App\Migrations;

use ShipMonk\Doctrine\Migration\Migration;
use ShipMonk\Doctrine\Migration\MigrationExecutor;

class Migration20230217063818 implements Migration
{

    public function before(MigrationExecutor $executor): void
    {
        $executor->executeQuery('CREATE INDEX IDX_542819F35080ECDE ON my_table (my_column)');
    }

    public function after(MigrationExecutor $executor): void
    {
    }

}

You can adjust it by providing custom $templateFilePath to MigrationConfig, but it needs to implement Migration interface.

Status verification:

You can check awaiting migrations and entity sync status:

$ bin/console migration:check

# example success output:
[info] Starting migration check
[info] Phase before fully executed, no awaiting migrations
[info] Phase after fully executed, no awaiting migrations
[info] Database is synced with entities, no migration needed
[info] Migration check completed

$ bin/console migration:check

# example failure output:
[info] Starting migration check
[info] Phase before fully executed, no awaiting migrations
[error] Phase after has executed migrations not present in /app/migrations: 20220208123456
[warning] Database is not synced with entities, 1 missing updates
[info] Migration check completed

Skipping all migrations:

You can also mark all migrations as already executed, e.g. when you just created fresh schema from entities. This will mark all not executed migrations in all stages as migrated.

$ bin/console migration:skip

# example output:
[info] Starting migration skip
[info] Found 1 migrations to skip in phase after
[info] Migration 20230214154154 phase after skipped
[info] Migration skip completed, 1 skipped

Executing migration:

Execution is performed without any interaction and does not fail when no migration is present for execution.

$ bin/console migration:run before

# example output:
[info] Starting migration execution (phase before)
[info] 1 pending migrations found
[info] Executing migration 20220224045126 phase before
[info] Migration 20220224045126 phase before executed successfully, 0.032 s elapsed
[info] Migration execution completed (phase before)

$ bin/console migration:run after

# example output:
[info] Starting migration execution (phase after)
[info] 1 pending migrations found
[info] Executing migration 20220224045126 phase after
[info] Migration 20220224045126 phase after executed successfully, 0.033 s elapsed
[info] Migration execution completed (phase after)

When executing all the migrations (e.g. in test environment) you probably want to achieve one-by-one execution. You can do that by:

$ bin/console migration:run both

# example output:
[info] Starting migration execution (phase both)
[info] 2 pending migrations found
[info] Executing migration 20220224045126 phase before
[info] Migration 20220224045126 phase before executed successfully, 0.032 s elapsed
[info] Executing migration 20220224045126 phase after
[info] Migration 20220224045126 phase after executed successfully, 0.033 s elapsed
[info] Migration execution completed (phase both)

Advanced usage

Run custom code for each executed query:

You can hook into migration execution by implementing MigrationExecutor interface and registering your implementations as a service. Implement executeQuery() to run checks or other code before/after each query. Interface of this method mimics interface of Doctrine\DBAL\Connection::executeQuery().

Alter generated migration SQLs:

You can implement custom MigrationAnalyzer and register it as a service. This allows you to alter generated SQLs (e.g. add ALGORITHM=INSTANT) and assign them to proper phase.

Hook to migration execution:

If you pass Psr\EventDispatcher\EventDispatcherInterface to MigrationService, you can hook into migration execution. Dispatched events are:

MigrationExecutionStartedEvent
MigrationExecutionSucceededEvent
MigrationExecutionFailedEvent

All events have MigrationPhase enum and Migration instance available.

Run all queries within transaction:

You can change your template (or a single migration) to extend TransactionalMigration. That causes each phases to be executed within migration. Be aware that many databases (like MySQL) does not support transaction over DDL operations (ALTER and such).

Checking execution duration:

Migration table has started_at and finished_at columns with datetime data with microseconds. But those columns are declared as VARCHARs by default, because there is no microsecond support in doctrine/dbal yet. That may complicate datetime manipulations (like duration calculation). You can adjust the structure to your needs (e.g. use DATETIME(6) for MySQL) manually in some migration.

+----------------+--------+-----------------------------+---------------------------+
| version        | phase  | started_at                  | finished_at               |
+----------------+--------+-----------------------------+---------------------------+
| 20220224045126 | before | 2023-02-17 05:19:50.225048 | 2023-02-17 05:19:50.672871 |
| 20220224045126 | after  | 2023-02-17 05:23:11.265727 | 2023-02-17 05:23:11.982982 |
+------------------------------------------------------+----------------------------+

Differences from doctrine/migrations

The underlying diff checking and generation is equal to what happens in doctrine/migrations as it uses doctrine/dbal features. Main difference is that we do not provide any downgrade phases.

This library is aiming to provide only core functionality needed for safe migrations within rolling-update deployments. Basically all the logic is inside MigrationService, which has only ~300 lines. We try to keep it as lightweight as possible, we do not plan to copy features from doctrine/migrations.

shipmonk / doctrine-two-phase-migrations

Maintainers

Package info

Statistics

Security