cresset / magebuild
A fast, parallel, DAG-ordered Magento 2 build orchestrator: composer install, di-compile, static-content deploy, autoload dump, and packaging in one process, for CI / Deployer build steps. Fetches a prebuilt Rust binary on demand.
Requires
- php: >=8.1
Requires (Dev)
None
Suggests
- ext-curl: Faster, more reliable binary downloads
Provides
None
Conflicts
None
Replaces
None
README
Build a Magento 2 release as fast as possible: a parallel, DAG-ordered
orchestrator that runs composer install, DI compile, static-content deploy,
autoload dump, and packaging in one process, replacing the build half of a
Deployer recipe.
magebuild links magequery's
write-side engines in-process (magecommand-engine for the DI compile,
magecommand's static_deploy for static content) together with a native
Composer client, so a full build needs no PHP and no framework bootstrap. The
steps run on a bounded parallel ready-queue over a shared rayon pool, with DI
compile and static-content deploy overlapped:
composer-install → { di-compile → autoload-dump } ∥ { static-deploy } → package
Install
curl (Linux / macOS)
curl --proto '=https' --tlsv1.2 -LsSf \ https://github.com/cresset-tools/magebuild/releases/latest/download/magebuild-installer.sh | sh
Prebuilt binaries ship for Linux (gnu and musl, x86_64 and arm64), macOS (Apple Silicon). Windows is not supported. Every release archive is Sigstore-signed.
Composer (for Deployer / CI projects)
composer require --dev cresset/magebuild
This puts magebuild in vendor/bin/; the package fetches the prebuilt binary
for your platform on first run (checksum-verified, cached).
Without installing (bgx)
bougie's bgx (like npx) runs magebuild in an isolated,
globally-cached environment without adding it to your project. Everything after
the package is forwarded straight to magebuild:
$ bgx cresset/magebuild plan $ bgx cresset/magebuild --artifact release.tar.zst
Usage
magebuild # build the Magento root in the current dir magebuild --root path/to/store # ... or an explicit root magebuild --artifact release.tar.zst # also package (gzip/zstd chosen by extension) magebuild plan # print the DAG + parallel schedule, run nothing magebuild node di-compile # run one node and its dependencies magebuild --json # machine-readable output magebuild db-status # does the database need setup:upgrade? magebuild config-status # does the config need app:config:import? magebuild theme-status # does setup:upgrade have a theme to register? magebuild queue-status # ... or a database queue to add? magebuild deploy --from release.tar.gz # ship a built release to the [deploy] host
Common flags: --jobs N, --only <ids>, --skip <ids>, --dry-run,
--profile, --preset <name>, --deployed-version <v>, --exclude-from <file>.
Presets
--preset hyva shapes the graph for a Hyvä production build,
following the Hyvä deploy docs:
- a Tailwind
npm ci --ignore-scripts && npm run buildper discovered Hyvä theme (app/design/frontend/*/*/web/tailwindif you have a project theme, elsevendor/hyva-themes/*/web/tailwind), wired to run before static-deploy; - a static-content deploy with
--no-parent --no-less --no-js-bundle --no-html-minify --symlink=file— Hyvä ships pre-built Tailwind CSS and no RequireJS bundles, and--symlink=filemakes every byte-identical asset a relative symlink to itsvendor/app/libsource instead of copying it (smaller, faster; the artifact already shipsvendorbesidepub/static).
The preset is applied over the built-in defaults but under magebuild.toml,
so any field it sets can still be overridden per node.
Packaging picks the compressor from the artifact extension: .tar (none),
.tar.gz (parallel gzip), .tar.zst (multi-threaded zstd). Both compressors
fan out across --jobs cores.
Configuration (magebuild.toml)
Placed at the Magento root, it overrides node fields or adds project steps:
[build] jobs = 8 [nodes.di-compile] fused = true # fused interceptors [nodes.static-deploy] locales = ["en_US", "nl_NL"] no_parent = true # don't deploy a theme's parent(s); see below symlink = true # symlink pure copies to source; see below [nodes.composer-install] hardlink = true # see below — off by default # add a custom step to the graph [nodes.my-assets] after = ["composer-install"] run = "npm run build"
hardlink (or the MAGEBUILD_HARDLINK env var) makes composer-install
hard-link packages out of a decompress-once store instead of extracting on every
run. It's off by default because it only pays off with a persistent,
uncompressed store — self-hosted CI with a cache disk, a Docker layer, or
repeated local builds. On ephemeral CI that restores the store from a compressed
actions/cache, the restore re-decompresses everything, so a plain extract is
just as fast.
Composer patches. composer-install applies a project's patches/ directory
automatically (zero-config — the same convention as bigbridge/patcher), so the
installed tree matches a real composer install even though the in-process
installer skips composer plugins. Each *.patch file's target package and strip
depth are inferred from its diff; package patches apply inside vendor/<pkg>,
project-root patches at the root. Application is idempotent — the applied set is
fingerprinted in patches.lock.json, so a re-run only re-patches packages whose
patch set changed, and a failing patch aborts the build (never a silently
unpatched tree). Projects with no patches/ dir are unaffected. This is what
makes, e.g., a Hyvä store's checkout modules Tailwind-v4-clean before the
--preset hyva Tailwind build runs.
no_parent (static-deploy, off by default) maps to magecommand's --no-parent:
a child theme normally pulls its ancestor themes into the deploy (Magento's quick
strategy). With it on, only the themes you deploy are emitted. Static deploy
resolves the parent→child fallback from source at deploy time, so the child
tree is self-contained and the parent theme's own pub/static output is
redundant when nothing serves it — e.g. a Hyvä storefront that only touches
Magento/luma via the fallback checkout doesn't need Magento/blank shipped.
Leave it off unless you know the parent is never requested at runtime.
symlink (static-deploy, off by default; also the MAGEBUILD_SYMLINK env var,
and implied by --preset hyva) maps to magecommand's --symlink=file: pure-copy
files — images, fonts, verbatim JS — become relative symlinks back to their
vendor//app//lib/web/ source instead of duplicated bytes; only derived
files (LESS-compiled CSS, generated RequireJS, bundles) stay real. It's well
suited to a magebuild artifact — the tarball ships vendor beside pub/static,
so the relative links resolve after extraction to any path and survive an atomic
current → releases/N swap. On a large multi-store the win is significant: the
deploy stops writing, and package stops re-reading, the gigabytes of duplicated
asset bytes, so the artifact shrinks several-fold (≈3.5× on one production
multi-store tree: 2.6 GiB → 0.7 GiB). The one requirement is that the serve-time
web server follows symlinks (nginx's default, disable_symlinks off); leave it
off for a bare pub/static shipped without its source tree.
Deployer integration
Install magebuild on the CI build host (curl one-liner above, or the Composer
package), then call it from deploy.php. Either run the whole build in one
parallel call:
task('magento:compile', function () { run('cd {{release_path}} && magebuild --deployed-version {{content_version}}'); });
or swap it in for individual bin/magento steps (di-compile ∥ static-deploy
still overlap when run as one call; here they are separate recipe tasks):
task('magento:compile', function () { run('cd {{release_path}} && magebuild --only di-compile --skip composer-install'); run('cd {{release_path}} && composer dump-autoload -o --no-dev'); }); task('magento:deploy:assets', function () { run('cd {{release_path}} && magebuild --only static-deploy --skip composer-install' . ' --deployed-version {{content_version}}'); });
--skip composer-install when Deployer already installed the vendor tree; drop
it to let magebuild install from composer.lock too.
Gating setup:upgrade with db-status
magebuild db-status is a native bin/magento setup:db:status: it compares the
declarative schema, the setup patches and the module versions against the
database named in app/etc/env.php, in tens of milliseconds rather than a
framework bootstrap. It uses the same exit codes as Magento: 0 when the
database is up to date, 2 when an upgrade is required (including when the
database is unreachable), 1 on an error such as a store that isn't installed.
--json prints what differs.
It runs on the deploy target, so put magebuild on that host's PATH (the
curl installer above), then override the recipe's check in deploy.php:
use Deployer\Exception\RunException; set('database_upgrade_needed', function () { try { run('magebuild db-status --root {{release_or_current_path}}'); } catch (RunException $e) { if ($e->getExitCode() == 2) { // upgrade required return true; } throw $e; } return false; });
Gating app:config:import with config-status
magebuild config-status is a native bin/magento app:config:status. Like
Magento, it hashes each imported section of the deployment config (scopes,
themes, system, plus any section a module registers with ImporterPool)
and compares it with the hash the last app:config:import stored in the flag
table. Exit codes match Magento's: 0 when up to date, 2 when an import is
needed. It exits 1 on an error, and also when the config holds something it
doesn't reproduce exactly (a PHP constant, an exponent-form float,
MAGENTO_DC__OVERRIDE); run bin/magento app:config:status then. In
deploy.php, the recipe's config_import_needed takes it the same way as
db-status above.
Themes and queues: theme-status and queue-status
setup:db:status doesn't look at themes or message queues, so a release that
only adds one of them looks up to date to it. Mage-OS added
theme:registration:status and queue:config:status for this, and these are
native versions with the same exit codes (0 when nothing is missing, 2 when
setup:upgrade has something to register):
theme-statuslists the themes that aregistration.phpdeclares but thethemetable doesn't hold;queue-statuslists the topology queues (queue_topology.xml, plus the olderqueue.xmlwhenMagento_Amqpis enabled) missing from thequeuetable.
RabbitMQ keeps its own state, which these commands don't read. Instead,
magebuild deploy fingerprints what Magento declares there from (env.php's
queue section and the modules' queue_topology.xml, queue.xml and
communication.xml), keeps the live release's fingerprint in
.dep/magebuild-amqp-topology, and upgrades when it changes. On a store with
RabbitMQ, the first deploy with this upgrades once. A queue deleted on the
broker by hand goes unnoticed. queue-status prints the fingerprint.
Deploying (magebuild deploy)
magebuild deploy replaces Deployer's deploy stage. It opens one ssh session
to the host and runs a small agent there: the same magebuild binary, uploaded
to ~/.cache/magebuild/agent-<sha256> only when that build is not there yet.
The agent does the release work that a Deployer recipe does one remote command
at a time, so a deploy costs one round trip instead of about a hundred:
- takes a lock (
flockon.dep/magebuild.lock, released however the agent exits) and createsreleases/<N>; - reuses every file the live release already has and unpacks only the rest as it streams in (see "Sending only what changed" below);
- links
shared/into the release, derivesapp/etc/env.phpfrom a template with a fresh cacheid_prefixper release, and sets writable dirs; - checks natively whether the database needs an upgrade (
db-status), the config an import (config-status), and whether there is a theme to register or a queue to set up (theme-status,queue-status, and the RabbitMQ topology fingerprint below), askingbin/magento app:config:statusonly when the config holds something the native check can't hash exactly; - pauses cron, runs the import or upgrade only when needed (maintenance mode on
the live release meanwhile), then swaps
currentatomically. Maintenance stays on through the swap, so the old code never serves the upgraded database: the flag is set in both the live and the new release, and after the swap and the opcache reset only the new release's is cleared. The old release keeps its flag, so rolling back to it after an upgrade comes up in maintenance until you runmaintenance:disablethere. If the live release was already in maintenance before the deploy, the new one inherits it and the deploy leaves it on. If an import or upgrade fails partway, the database may match neither release, so the agent fails closed: maintenance stays on, cron stays stopped, and.dep/upgrade-failedtells the next deploy to clear that flag once it gets through. Once an upgrade has started, or the swap is next, the agent finishes even if the ssh connection drops (CI cancelled or timed out): it ignores SIGHUP, keeps PHP's output off the dead connection, and writes how the deploy ended to.dep/last-deploy.log. Before that point a dropped connection still stops the deploy with the store untouched. The agent sets and clearsvar/.maintenance.flagitself, exactly asmaintenance:enable/disabledo, without booting PHP. Like Deployer's recipe, sharevar/.maintenance.ipbut not the flag: a missing shared file is created empty, which is maintenance on, and Magento's disable deletes the link rather than the shared file; - resets opcache over FastCGI, deletes the previous release's cache keys, resumes cron, and sets old releases aside for the next deploy to delete while it uploads.
The layout is Deployer's (releases/<N>, shared/, a relative current,
.dep/latest_release, .dep/releases_log), so a Deployer recipe still works
on the same host as a fallback.
[deploy] host = "magento@shop.example.com" path = "/var/www/shop" php = "/usr/bin/php8.4" shared_files = ["var/.maintenance.ip"] # env.php is derived, below links = { "auth.json" = "shared/auth.json" } # linked, never touched keep_releases = 5 [deploy.env_php] source = "shared/app/etc/env.php" # rolls cache/frontend/*/id_prefix to r<N>_ [deploy.cron] # default: backend = "none" backend = "systemd" stop = ["magento-cron.timer", "magento-cron.service"] start = ["magento-cron.timer"] [deploy.opcache] fastcgi = "127.0.0.1:9000" # or a socket path tmp_dir = "/var/www/tmp" # a dir php-fpm sees at the same path
Sending only what changed
Before connecting, the runner reads a manifest out of the artifact: every
path with its mode and the sha256 of its contents. The agent compares it with
the manifest it stored for the live release (.dep/manifests/<N>.json.zst)
and hard-links each file whose content and mode are unchanged from the live
release, so the runner sends a zstd tar of only the new and changed files.
A rebuild with no input changes sends a few KB instead of the whole artifact.
The first deploy, or one whose live release has no stored manifest, sends the
whole artifact as before.
Hard-linked files are the same file in every release that has them, so nothing
may change a release's files in place after its deploy. app/etc/ is always
copied rather than linked, because bin/magento may rewrite config.php. A
file edited on the host since its deploy no longer matches the size and mtime
in its stored manifest, so it is sent again rather than linked. On hosts where
something does write into releases, reuse = "copy" gives every release its
own copy of each file; that costs disk, but nothing extra on the network.
shared_dirs and writable_dirs default to Deployer's magento2 recipe.
upgrade = "check" refuses to swap when an upgrade or import is needed instead
of running it. --agent <path> picks the binary to run on the host (a static
Linux build; by default the running magebuild), --host overrides the host,
and --ssh the ssh command.
How it relates to magequery
magequery is the read-side tool
(inspect a Magento codebase) and magecommand its write-side companion (DI
compile, static deploy). magebuild is the orchestrator: it links those engines
as libraries and runs them, plus Composer, as a parallel build graph.