README

Two cooperating subsystems:

Telemetry destination seam (Sink/, Collector/, Buffer/, Heartbeat/, Driver/) — the §12.4 "emitting ≠ monitoring" plane. The app emits OTLP to a local OpenTelemetry Collector sidecar; the collector batches, disk-buffers and retries to an off-host backend. The only swap point is a driver (MetricsSinkInterface / ErrorSinkInterface); switching backends never touches app code. See "Telemetry seam" below.
Template publisher (Service/, Command/, Resources/) — optional starter dashboards/alert rules for common stacks. It does not export telemetry, call external services, or run in the request path.

Telemetry seam (collector + off-host sinks)

Drivers (the only place a backend name appears): grafana (OTLP metrics/traces/ logs, off-host), glitchtip (error sink, disk-buffered), null (explicit no-op). Vendor SDK drivers (datadog, sentry, …) are deferred split packages.
Collector config is generated for the selected sink — loopback-only OTLP receiver, memory_limiter + batch, a cardinality deny-list, and a file_storage persistent queue (retry_on_failure + sending_queue) so a backend blip buffers to disk and drains on recovery. Backend credentials are referenced via ${env:...} — never inlined into the committed config.
Log aggregation rides the same sidecar and is on by default when the sink carries the Logs signal (grafana does). A filelog receiver tails the host's Docker json-file container logs, unwraps the Docker envelope, and parses the app's structured JSON into attributes; a transform processor drops secret-named keys and masks secret-shaped values (JWTs, bearer tokens, cards, emails) before logs leave the host; a dedicated disk-backed queue isolates log delivery from metrics/traces. To read the root-owned container logs the collector sidecar runs as root and bind-mounts the container-log directory read-only — the standard log-agent posture (Promtail, Fluent Bit, Grafana Alloy). Disable with --no-logs. Note: the redaction is a transform processor, not the dedicated redaction processor, because the pinned collector (0.103.0) does not support redaction on the logs signal. The regex escaping for OTTL is validated end-to-end against that image.
Error sink spools captured errors to a bounded, crash-safe on-disk queue (BoundedSpool: byte-capped, drop-oldest + counter, CRC-checked, atomic rewrite); capture() never blocks the request path and never throws. Errors are PII-scrubbed by construction (MessageScrubber / CapturedError).
Dead-man heartbeat (HttpHeartbeatEmitter): the app pushes a periodic check-in to an external monitor; absence pages (detected off-host) — the only detector that catches "host dead AND its monitoring dead."

Telemetry-seam commands

php bin/console vortos:observability:collector --sink=grafana   # generate sidecar config (logs on by default)
php bin/console vortos:observability:collector --no-logs         # metrics/traces only
php bin/console vortos:observability:collector --log-sample=0.5  # forward 50% of log records
php bin/console vortos:observability:collector --dry-run         # preview
php bin/console vortos:observability:heartbeat --status=success  # dead-man check-in (cron, ~60s)

Configuration (env)

Var	Default	Purpose
`OBSERVABILITY_METRICS_SINK`	`grafana`	Selected metrics sink driver key
`OBSERVABILITY_ERROR_SINK`	`glitchtip`	Selected error sink driver key
`OBSERVABILITY_GRAFANA_OTLP_HOST`	—	Off-host OTLP gateway host
`OBSERVABILITY_GRAFANA_OTLP_HEADERS`	—	Auth header (resolved by the collector at runtime)
`OBSERVABILITY_GRAFANA_OTLP_PORT`	443 (TLS) / 4318	Gateway port; omit to let the sink derive it (a TLS gateway is 443, not the bare-OTLP 4318)
`OBSERVABILITY_GRAFANA_OTLP_PATH`	`/otlp` (http)	Gateway base path the exporter appends `/v1/{signal}` onto
`OBSERVABILITY_LOGS_DISABLED`	`false`	Set truthy to omit the log-aggregation pipeline
`OBSERVABILITY_LOG_INCLUDE`	Docker json-file glob	Comma/space-separated container-log globs the filelog receiver tails
`OBSERVABILITY_LOG_SAMPLE_RATIO`	`1.0`	Fraction of log records forwarded (1.0 = all)
`OBSERVABILITY_LOG_START_AT`	`end`	`end` (only new lines) or `beginning` on a new file
`OBSERVABILITY_GLITCHTIP_DSN`	—	Error backend ingest URL (read at use time, never logged)
`OBSERVABILITY_SPOOL_DIR`	system temp	Error spool directory
`OBSERVABILITY_SPOOL_MAX_BYTES`	256 MiB	Error spool byte cap (drop-oldest past this)
`OBSERVABILITY_HEARTBEAT_URL`	—	External dead-man monitor base URL

Observability Templates

This module publishes optional starter assets for common observability stacks. It does not export telemetry, call external services, or run in the request path.

Vortos runtime modules emit standard signals:

metrics through Prometheus or StatsD
traces through OpenTelemetry OTLP
logs as structured JSON to stdout/stderr
health endpoints through the foundation module

The templates help teams bootstrap dashboards and alert rules for those signals.

Messaging applications also export operational gauges for the transactional outbox and dead-letter queue:

vortos_outbox_backlog_size{transport,status}
vortos_outbox_oldest_pending_age_seconds{transport}
vortos_dlq_backlog_size{transport,event}
vortos_dlq_oldest_failed_age_seconds{transport}

With Prometheus these are refreshed during /metrics scrapes. With push-style backends such as StatsD, schedule php bin/console vortos:metrics:collect from one worker per environment so gauges are emitted without adding work to normal request handling.

Commands

php bin/console vortos:observability:list
php bin/console vortos:observability:publish --stack=grafana-oss
php bin/console vortos:observability:publish --stack=datadog
php bin/console vortos:observability:publish --stack=newrelic

Published files are written under observability/.

Use --dry-run to preview and --force to overwrite existing files.

Stacks

prometheus: recording and alert rules
grafana: Grafana dashboard JSON
alertmanager: Alertmanager routing example
grafana-oss: Prometheus + Grafana + Alertmanager
datadog: Datadog dashboard and monitor examples
newrelic: New Relic dashboard and alert examples

All thresholds and notification routes are examples. Review and tune them per environment before production use.

vortos / vortos-observability

Maintainers

Package info

Statistics

Security