Search by

cresset / magebuild

pingiun

A fast, parallel, DAG-ordered Magento 2 build orchestrator: composer install, di-compile, static-content deploy, autoload dump, and packaging in one process, for CI / Deployer build steps. Fetches a prebuilt Rust binary on demand.

Package info

github.com/cresset-tools/magebuild

Language:Rust

pkg:composer/cresset/magebuild

Statistics

Installs: 6

Dependents: 0

Suggesters: 0

Stars: 1

Open Issues: 1

v0.13.0 2020-01-01 00:00 UTC

This package is auto-updated.

Last update: 2026-11-16 21:59:17 UTC


README

Build a Magento 2 release as fast as possible: a parallel, DAG-ordered orchestrator that runs composer install, DI compile, static-content deploy, autoload dump, and packaging in one process, replacing the build half of a Deployer recipe.

magebuild links magequery's write-side engines in-process (magecommand-engine for the DI compile, magecommand's static_deploy for static content) together with a native Composer client, so a full build needs no PHP and no framework bootstrap. The steps run on a bounded parallel ready-queue over a shared rayon pool, with DI compile and static-content deploy overlapped:

composer-install → { di-compile → autoload-dump } ∥ { static-deploy } → package

Install

curl (Linux / macOS)

curl --proto '=https' --tlsv1.2 -LsSf \
  https://github.com/cresset-tools/magebuild/releases/latest/download/magebuild-installer.sh | sh

Prebuilt binaries ship for Linux (gnu and musl, x86_64 and arm64), macOS (Apple Silicon). Windows is not supported. Every release archive is Sigstore-signed.

Composer (for Deployer / CI projects)

composer require --dev cresset/magebuild

This puts magebuild in vendor/bin/; the package fetches the prebuilt binary for your platform on first run (checksum-verified, cached).

Without installing (bgx)

bougie's bgx (like npx) runs magebuild in an isolated, globally-cached environment without adding it to your project. Everything after the package is forwarded straight to magebuild:

$ bgx cresset/magebuild plan
$ bgx cresset/magebuild --artifact release.tar.zst

Usage

magebuild                              # build the Magento root in the current dir
magebuild --root path/to/store         # ... or an explicit root
magebuild --artifact release.tar.zst   # also package (gzip/zstd chosen by extension)
magebuild plan                         # print the DAG + parallel schedule, run nothing
magebuild node di-compile              # run one node and its dependencies
magebuild --json                       # machine-readable output
magebuild db-status                    # does the database need setup:upgrade?
magebuild config-status                # does the config need app:config:import?
magebuild theme-status                 # does setup:upgrade have a theme to register?
magebuild queue-status                 # ... or a database queue to add?
magebuild deploy --from release.tar.gz # ship a built release to the [deploy] host

Common flags: --jobs N, --only <ids>, --skip <ids>, --dry-run, --profile, --preset <name>, --deployed-version <v>, --exclude-from <file>.

Presets

--preset hyva shapes the graph for a Hyvä production build, following the Hyvä deploy docs:

  1. a Tailwind npm ci --ignore-scripts && npm run build per discovered Hyvä theme (app/design/frontend/*/*/web/tailwind if you have a project theme, else vendor/hyva-themes/*/web/tailwind), wired to run before static-deploy;
  2. a static-content deploy with --no-parent --no-less --no-js-bundle --no-html-minify --symlink=file — Hyvä ships pre-built Tailwind CSS and no RequireJS bundles, and --symlink=file makes every byte-identical asset a relative symlink to its vendor/app/lib source instead of copying it (smaller, faster; the artifact already ships vendor beside pub/static).

The preset is applied over the built-in defaults but under magebuild.toml, so any field it sets can still be overridden per node.

Packaging picks the compressor from the artifact extension: .tar (none), .tar.gz (parallel gzip), .tar.zst (multi-threaded zstd). Both compressors fan out across --jobs cores.

Configuration (magebuild.toml)

Placed at the Magento root, it overrides node fields or adds project steps:

[build]
jobs = 8

[nodes.di-compile]
fused = true                       # fused interceptors

[nodes.static-deploy]
locales = ["en_US", "nl_NL"]
no_parent = true                   # don't deploy a theme's parent(s); see below
symlink = true                     # symlink pure copies to source; see below

[nodes.composer-install]
hardlink = true                    # see below — off by default

# add a custom step to the graph
[nodes.my-assets]
after = ["composer-install"]
run = "npm run build"

hardlink (or the MAGEBUILD_HARDLINK env var) makes composer-install hard-link packages out of a decompress-once store instead of extracting on every run. It's off by default because it only pays off with a persistent, uncompressed store — self-hosted CI with a cache disk, a Docker layer, or repeated local builds. On ephemeral CI that restores the store from a compressed actions/cache, the restore re-decompresses everything, so a plain extract is just as fast.

Composer patches. composer-install applies a project's patches/ directory automatically (zero-config — the same convention as bigbridge/patcher), so the installed tree matches a real composer install even though the in-process installer skips composer plugins. Each *.patch file's target package and strip depth are inferred from its diff; package patches apply inside vendor/<pkg>, project-root patches at the root. Application is idempotent — the applied set is fingerprinted in patches.lock.json, so a re-run only re-patches packages whose patch set changed, and a failing patch aborts the build (never a silently unpatched tree). Projects with no patches/ dir are unaffected. This is what makes, e.g., a Hyvä store's checkout modules Tailwind-v4-clean before the --preset hyva Tailwind build runs.

no_parent (static-deploy, off by default) maps to magecommand's --no-parent: a child theme normally pulls its ancestor themes into the deploy (Magento's quick strategy). With it on, only the themes you deploy are emitted. Static deploy resolves the parent→child fallback from source at deploy time, so the child tree is self-contained and the parent theme's own pub/static output is redundant when nothing serves it — e.g. a Hyvä storefront that only touches Magento/luma via the fallback checkout doesn't need Magento/blank shipped. Leave it off unless you know the parent is never requested at runtime.

symlink (static-deploy, off by default; also the MAGEBUILD_SYMLINK env var, and implied by --preset hyva) maps to magecommand's --symlink=file: pure-copy files — images, fonts, verbatim JS — become relative symlinks back to their vendor//app//lib/web/ source instead of duplicated bytes; only derived files (LESS-compiled CSS, generated RequireJS, bundles) stay real. It's well suited to a magebuild artifact — the tarball ships vendor beside pub/static, so the relative links resolve after extraction to any path and survive an atomic current → releases/N swap. On a large multi-store the win is significant: the deploy stops writing, and package stops re-reading, the gigabytes of duplicated asset bytes, so the artifact shrinks several-fold (≈3.5× on one production multi-store tree: 2.6 GiB → 0.7 GiB). The one requirement is that the serve-time web server follows symlinks (nginx's default, disable_symlinks off); leave it off for a bare pub/static shipped without its source tree.

Deployer integration

Install magebuild on the CI build host (curl one-liner above, or the Composer package), then call it from deploy.php. Either run the whole build in one parallel call:

task('magento:compile', function () {
    run('cd {{release_path}} && magebuild --deployed-version {{content_version}}');
});

or swap it in for individual bin/magento steps (di-compile ∥ static-deploy still overlap when run as one call; here they are separate recipe tasks):

task('magento:compile', function () {
    run('cd {{release_path}} && magebuild --only di-compile --skip composer-install');
    run('cd {{release_path}} && composer dump-autoload -o --no-dev');
});
task('magento:deploy:assets', function () {
    run('cd {{release_path}} && magebuild --only static-deploy --skip composer-install'
        . ' --deployed-version {{content_version}}');
});

--skip composer-install when Deployer already installed the vendor tree; drop it to let magebuild install from composer.lock too.

Gating setup:upgrade with db-status

magebuild db-status is a native bin/magento setup:db:status: it compares the declarative schema, the setup patches and the module versions against the database named in app/etc/env.php, in tens of milliseconds rather than a framework bootstrap. It uses the same exit codes as Magento: 0 when the database is up to date, 2 when an upgrade is required (including when the database is unreachable), 1 on an error such as a store that isn't installed. --json prints what differs.

It runs on the deploy target, so put magebuild on that host's PATH (the curl installer above), then override the recipe's check in deploy.php:

use Deployer\Exception\RunException;

set('database_upgrade_needed', function () {
    try {
        run('magebuild db-status --root {{release_or_current_path}}');
    } catch (RunException $e) {
        if ($e->getExitCode() == 2) { // upgrade required
            return true;
        }
        throw $e;
    }
    return false;
});

Gating app:config:import with config-status

magebuild config-status is a native bin/magento app:config:status. Like Magento, it hashes each imported section of the deployment config (scopes, themes, system, plus any section a module registers with ImporterPool) and compares it with the hash the last app:config:import stored in the flag table. Exit codes match Magento's: 0 when up to date, 2 when an import is needed. It exits 1 on an error, and also when the config holds something it doesn't reproduce exactly (a PHP constant, an exponent-form float, MAGENTO_DC__OVERRIDE); run bin/magento app:config:status then. In deploy.php, the recipe's config_import_needed takes it the same way as db-status above.

Themes and queues: theme-status and queue-status

setup:db:status doesn't look at themes or message queues, so a release that only adds one of them looks up to date to it. Mage-OS added theme:registration:status and queue:config:status for this, and these are native versions with the same exit codes (0 when nothing is missing, 2 when setup:upgrade has something to register):

  • theme-status lists the themes that a registration.php declares but the theme table doesn't hold;
  • queue-status lists the topology queues (queue_topology.xml, plus the older queue.xml when Magento_Amqp is enabled) missing from the queue table.

RabbitMQ keeps its own state, which these commands don't read. Instead, magebuild deploy fingerprints what Magento declares there from (env.php's queue section and the modules' queue_topology.xml, queue.xml and communication.xml), keeps the live release's fingerprint in .dep/magebuild-amqp-topology, and upgrades when it changes. On a store with RabbitMQ, the first deploy with this upgrades once. A queue deleted on the broker by hand goes unnoticed. queue-status prints the fingerprint.

Deploying (magebuild deploy)

magebuild deploy replaces Deployer's deploy stage. It opens one ssh session to the host and runs a small agent there: the same magebuild binary, uploaded to ~/.cache/magebuild/agent-<sha256> only when that build is not there yet. The agent does the release work that a Deployer recipe does one remote command at a time, so a deploy costs one round trip instead of about a hundred:

  1. takes a lock (flock on .dep/magebuild.lock, released however the agent exits) and creates releases/<N>;
  2. reuses every file the live release already has and unpacks only the rest as it streams in (see "Sending only what changed" below);
  3. links shared/ into the release, derives app/etc/env.php from a template with a fresh cache id_prefix per release, and sets writable dirs;
  4. checks natively whether the database needs an upgrade (db-status), the config an import (config-status), and whether there is a theme to register or a queue to set up (theme-status, queue-status, and the RabbitMQ topology fingerprint below), asking bin/magento app:config:status only when the config holds something the native check can't hash exactly;
  5. pauses cron, runs the import or upgrade only when needed (maintenance mode on the live release meanwhile), then swaps current atomically. Maintenance stays on through the swap, so the old code never serves the upgraded database: the flag is set in both the live and the new release, and after the swap and the opcache reset only the new release's is cleared. The old release keeps its flag, so rolling back to it after an upgrade comes up in maintenance until you run maintenance:disable there. If the live release was already in maintenance before the deploy, the new one inherits it and the deploy leaves it on. If an import or upgrade fails partway, the database may match neither release, so the agent fails closed: maintenance stays on, cron stays stopped, and .dep/upgrade-failed tells the next deploy to clear that flag once it gets through. Once an upgrade has started, or the swap is next, the agent finishes even if the ssh connection drops (CI cancelled or timed out): it ignores SIGHUP, keeps PHP's output off the dead connection, and writes how the deploy ended to .dep/last-deploy.log. Before that point a dropped connection still stops the deploy with the store untouched. The agent sets and clears var/.maintenance.flag itself, exactly as maintenance:enable/disable do, without booting PHP. Like Deployer's recipe, share var/.maintenance.ip but not the flag: a missing shared file is created empty, which is maintenance on, and Magento's disable deletes the link rather than the shared file;
  6. resets opcache over FastCGI, deletes the previous release's cache keys, resumes cron, and sets old releases aside for the next deploy to delete while it uploads.

The layout is Deployer's (releases/<N>, shared/, a relative current, .dep/latest_release, .dep/releases_log), so a Deployer recipe still works on the same host as a fallback.

[deploy]
host = "magento@shop.example.com"
path = "/var/www/shop"
php = "/usr/bin/php8.4"
shared_files = ["var/.maintenance.ip"]          # env.php is derived, below
links = { "auth.json" = "shared/auth.json" }    # linked, never touched
keep_releases = 5

[deploy.env_php]
source = "shared/app/etc/env.php"               # rolls cache/frontend/*/id_prefix to r<N>_

[deploy.cron]                                   # default: backend = "none"
backend = "systemd"
stop = ["magento-cron.timer", "magento-cron.service"]
start = ["magento-cron.timer"]

[deploy.opcache]
fastcgi = "127.0.0.1:9000"                      # or a socket path
tmp_dir = "/var/www/tmp"                        # a dir php-fpm sees at the same path

Sending only what changed

Before connecting, the runner reads a manifest out of the artifact: every path with its mode and the sha256 of its contents. The agent compares it with the manifest it stored for the live release (.dep/manifests/<N>.json.zst) and hard-links each file whose content and mode are unchanged from the live release, so the runner sends a zstd tar of only the new and changed files. A rebuild with no input changes sends a few KB instead of the whole artifact. The first deploy, or one whose live release has no stored manifest, sends the whole artifact as before.

Hard-linked files are the same file in every release that has them, so nothing may change a release's files in place after its deploy. app/etc/ is always copied rather than linked, because bin/magento may rewrite config.php. A file edited on the host since its deploy no longer matches the size and mtime in its stored manifest, so it is sent again rather than linked. On hosts where something does write into releases, reuse = "copy" gives every release its own copy of each file; that costs disk, but nothing extra on the network.

shared_dirs and writable_dirs default to Deployer's magento2 recipe. upgrade = "check" refuses to swap when an upgrade or import is needed instead of running it. --agent <path> picks the binary to run on the host (a static Linux build; by default the running magebuild), --host overrides the host, and --ssh the ssh command.

How it relates to magequery

magequery is the read-side tool (inspect a Magento codebase) and magecommand its write-side companion (DI compile, static deploy). magebuild is the orchestrator: it links those engines as libraries and runs them, plus Composer, as a parallel build graph.

License

EUPL-1.2.