README

Elasticsearch index lifecycle for Symfony. The query and faceted-UI half lives in survos/search-bundle; this bundle owns everything that writes.

Moving an app off Meilisearch? See Migrating from Meilisearch: the decision, the app checklist, and the fsn1 production setup.

Why it's separate

survos/search-bundle adapters are read-only, and it must not require an engine client — an app using only the SQLite FTS5 or Postgres BM25 adapter shouldn't pay for elasticsearch/elasticsearch, elastic/transport and OpenTelemetry. Index ownership therefore lives in the engine's own bundle, exactly as survos/meili-bundle already does for Meilisearch.

This is deliberately not FOSElasticaBundle: that brings its own query layer (Finder, its own Query objects), which would compete with search-bundle's Query/ResultSet/facets, and its mappings live in ES-specific YAML instead of deriving from #[Field] metadata.

Commands

bin/console elastic:index:status   [code]
bin/console elastic:index:create   [code] [--drop] [--strict|--no-strict] [--triggers|--no-triggers]
bin/console elastic:index:populate [code] [--batch-size=250] [--limit=N]
bin/console elastic:index:rebuild  [code] [--keep-old] [--batch-size=250]
bin/console elastic:index:delete   [code] [--force]
bin/console elastic:outbox:consume [--once] [--dispatch] [--batch-size=500] [--idle-timeout=5] [--time-limit=N]

Mappings are not computed here — they come from the search's resolved adapter parameters in search-bundle, so querying and indexing can't drift apart.

Demo: a faceted, stemmed search in five minutes

Reproduced end to end against survos-sites/bench (Elasticsearch 9.5.0, 300 films). Every number below is from that run.

# config/packages/survos_search.yaml
survos_search:
    default_adapter: es
    adapters:
        es: { dsn: '%env(ELASTICSEARCH_DSN)%' }

# config/packages/survos_elastic.yaml
survos_elastic:
    analysis:
        language: english
        ascii_folding: true

bin/console elastic:index:create app_movie     # bench_movie -> bench_movie_20260816113526
bin/console elastic:index:populate app_movie   # indexed 300 documents

Then open /entity/app_movie/search. The page is served by search-bundle's ux-search components against the Elasticsearch adapter — 300 results, a Year range slider whose 1986–2023 bounds come from a live stats aggregation, and term facets (mcu, superhero, marvel, …) from terms aggregations.

Stemming is the part worth demonstrating, because it is invisible until you look for it:

query	results
`?query=village`	2
`?query=villages`	2
`?query=war`	13

Singular and plural returning the same hits is the english stemmer. Without survos_elastic.analysis those numbers are 0 and 2 — the default standard analyzer does no stemming at all. On the same index you can see both behaviours side by side:

curl "$ES/bench_movie/_analyze" -H 'Content-Type: application/json' \
  -d '{"analyzer":"standard","text":"Running through Kovács'"'"' villages"}'
#   -> running, through, kovács, villages

curl "$ES/bench_movie/_analyze" -H 'Content-Type: application/json' \
  -d '{"analyzer":"survos_text","text":"Running through Kovács'"'"' villages"}'
#   -> run, through, kovac, villag

Changing the analyzer

index.analysis is a static setting — editing the config does nothing to an index that already exists. The admin page says so rather than letting you wonder:

Configured for "hungarian" but this index was built with "english". index.analysis is a static setting, so the configuration has had no effect on it — run elastic:index:rebuild.

bin/console elastic:index:rebuild app_movie

builds a new generation, populates it, swaps the alias atomically, and drops the old one. The old index serves every query until the moment of the swap, and a failure mid-populate leaves the live alias untouched.

Admin pages

/admin/elastic/ lists every registered search with its index, document source and schema status; /admin/elastic/{code} is the diagnostic page — alias, analyzers, dynamic mapping, declared-vs-actual drift, field count, deep-paging headroom, and a Field intent table tracing each #[Field] through the adapter into the live mapping. That last one answers "why isn't my facet showing up" without guessing which of the three layers dropped it.

Doctrine sync

ElasticSpoolDoctrineListener collects changed ids in postPersist/postUpdate/postRemove and hands them off in postFlush. It never calls Elasticsearch inline: a database write must not depend on the search engine being up, and an HTTP request must not wait on it.

Messages carry ids, never documents. The worker loads current state when it runs, so duplicate messages are cheap and correct -- an import that flushes the same entity four times produces one reindex from the final state, not four racing writes.

# config/packages/survos_elastic.yaml
survos_elastic:
    spool_dir: '%kernel.project_dir%/var/elastic-spool'
    spool_enabled: true
    async: true        # dispatch through Messenger
    batch_size: 500    # ids per message

Two modes

async: true (default, needs symfony/messenger) dispatches ReindexDocuments / RemoveDocuments, chunked by batch_size, so one enormous flush becomes several bounded jobs.

async: false, or no bus installed, writes a JSONL spool drained by elastic:spool:flush. This is the right mode for a bulk import: a line per id costs nothing, and reconciling 400k ids once at the end beats dispatching 400k messages.

bin/console elastic:spool:flush [FQCN] [--batch-size=500] [--async]

Write path under load

A workflow flushes once per transition, so a busy pipeline produces a stream of small ReindexDocuments messages. The write path is built for that:

The handler batches. ReindexDocumentsHandler is a Messenger BatchHandlerInterface: it collects handler_batch_size messages (default 50), merges their ids per class, and reconciles them with one query and one bulk request. A partial batch goes out after handler_idle_timeout seconds of worker idleness (default 1).
Incremental writes are upserts. Reconciliation sends update + doc_as_upsert, so Elasticsearch's detect_noop skips documents whose indexed fields didn't change, such as a transition that only touches unindexed columns. Mapped fields missing from a document are sent as null, so a merge can't keep a stale value.
No forced refresh on incremental writes or deletes. Each forced refresh creates a segment. refresh_interval (1s) makes changes searchable. elastic:index:populate refreshes once at the end.
Rebuilds load with refresh off. elastic:index:rebuild sets refresh_interval: -1 on the new generation, restores the default, refreshes once, then swaps the alias.
Bulk requests are sized by bytes as well as count (ElasticIndexService::MAX_BULK_BYTES, 10 MB), because OCR and AI output make document size vary by orders of magnitude.

survos_elastic:
    handler_batch_size: 50
    handler_idle_timeout: 1

Routing

Unrouted messages are handled synchronously -- which still works, but the flush then waits on Elasticsearch, defeating the point. Route them to get real async:

# config/packages/messenger.yaml
framework:
    messenger:
        routing:
            'Survos\ElasticBundle\Message\ReindexDocuments': async
            'Survos\ElasticBundle\Message\RemoveDocuments': async

Postgres trigger outbox (trial)

The listener only sees writes that go through the ORM. On Postgres, triggers can capture every write instead -- raw SQL, DQL bulk updates, ON DELETE CASCADE, psql -- in the same transaction, so a crash after COMMIT cannot lose a change. See survos/mono#63.

bin/console elastic:index:create app_package --triggers     # create elastic_outbox + install triggers (idempotent)
bin/console elastic:outbox:consume                          # long-running: LISTEN, drain, reindex
bin/console elastic:index:create app_package --no-triggers  # remove them again

Statement-level triggers record (class, id) in elastic_outbox, one row per id however often it changed, and NOTIFY on commit, so the consumer wakes in well under a second.
The consumer claims a batch, reconciles it with indexIds() (or --dispatch sends ReindexDocuments to the bus), then deletes it. A failed batch stays queued; a write landing on a claimed id is queued again rather than lost.
elastic:index:status shows which searches have triggers and how deep the outbox is.
The table is created by the command, not a migration; a doctrine.dbal.schema_filter hides it from doctrine:migrations:diff.
Not Postgres (SQLite, MySQL)? --triggers says so, and the listener keeps handling changes.
The listener is unaffected. Running both is safe -- reindexing is idempotent and detect_noop skips the duplicate write -- which is how the trial compares them.

Not built yet

The embedding cache. Vectors stay off until it exists — see search-bundle's docs/elasticsearch.md.
Streaming/resumable populate, conditional indexing, relation traversal in auto-mapping, a raw request-body hook, search_after deep paging and suggesters — see survos/mono#42 items 3–8.

Settings/analyzer management and schema validation are done — see the demo above and the admin pages.

SearchBench lifecycle validation (September 2026)

SearchBench's es branch now evaluates the full lexical browser UI using InstantSearch and client-side Twig, with 9,751 movies, 5,076 cars, 1,807 Marvel characters and 17,092 WCMA records. See the browser contract and the app's setup and live tests.

elastic:index:populate now creates a missing index with the declared mapping before loading records. rebuild fills a new generation before swapping the alias; pause writes or replay changes during that operation. Incremental indexing refuses a missing index rather than letting Elasticsearch auto-create an incompatible mapping. Bulk item failures are surfaced.

The Doctrine listener selects Elasticsearch-backed entities, captures generated identifiers before removal, and dispatches reconciliation IDs after flush. Reconciliation loads current DB state: present rows are indexed, absent rows deleted. Both old remove messages and new index messages follow this rule so a delayed delete does not blindly remove a replacement record.

Use a dedicated Messenger queue/consumer. Sharing the ORM database connection with the Doctrine transport keeps queue inserts inside an explicit outer transaction until commit. A process crash between a standalone flush commit and dispatch is not covered by a transactional outbox. DQL/SQL bulk writes bypass ORM events and need explicit reindexing. Concurrent consumers do not guarantee version ordering; one indexing consumer per dataset is the evaluated configuration.

With async: false (or no bus), IDs are spooled for elastic:spool:flush. Queue dispatch failures also fall back to the spool. Schedule drains/monitor queue failures in deployed applications. A drain claims the current file under a lock, retains failed claims for retry, and leaves new appends for a subsequent drain. This requires a persistent shared spool volume if multiple application processes need to drain the same files. JSONL, filesystem, finder and lock are now explicit runtime dependencies.

Unit tests cover generated-ID capture, cleared units of work, failed queue dispatch, failed spool claims, concurrent appends, and bulk errors. SearchBench additionally tests population, Doctrine CRUD, replacement identifiers and rollback reconciliation against a local ES node.

Admin navbar and local/hosted Elasticsearch

With Tabler and an Elasticsearch-backed search, the Elastic dropdown in ADMIN_NAVBAR_MENU shows the configured ES endpoints (without credentials), app indexes, Kibana, Kibana index management, and Dev Tools. In Dev Tools, GET _tasks?detailed=true shows running Elasticsearch tasks; these are distinct from application Messenger jobs. Menu rendering never makes cluster requests.

In debug mode, loopback-only ES connections default to http://localhost:5601. For a hosted cluster, container hostname, SSH tunnel, or Kibana space, configure the browser-facing URL explicitly; it is independent of the ES DSN:

survos_elastic:
    kibana_url: '%env(KIBANA_URL)%'

Set KIBANA_URL in the app environment (for example http://localhost:5602 for a tunnel). Set kibana_url: '' to hide Kibana links. Changing this URL does not switch the search backend: configure the SearchBundle adapter DSN separately. Use separate per-app index prefixes when sharing a server. A remote authenticated node also needs correct TLS trust and credentials in the app's connection setup. SearchBundle alone keeps its engine-neutral Search menu; ES administration belongs in ElasticBundle, like the Meilisearch tools belong in MeiliBundle.

Symfony profiler

In debug mode, the Elasticsearch toolbar and profiler panel show calls, query bodies, round-trip and Elasticsearch timings, hit counts, aggregations and errors. The toolbar uses Symfony's built-in search icon and turns amber for empty or repeated searches.

Repeated searches are counted beyond the first identical query on the same client and index; each matching row shows the total occurrence count. Bulk and other write calls are excluded because their recorded payloads are summaries, not complete requests.

Copy as cURL replays a search against its configured HTTP endpoint. Commands use POSIX shell quoting and omit connection credentials; supply authentication and any custom CA configuration yourself. The button uses Symfony's profiler clipboard support. It is available with the SearchBundle factory's endpoint-aware client; older versions or custom clients without endpoint metadata still show query bodies and timings.

survos / elastic-bundle

Package info

Security

Fund package maintenance!

Statistics