README

Local LLM inference for PHP, in-process.
Chat, embeddings, and reasoning models — no Python sidecar, no remote API.

Status

Feature-complete for its scope. Maintained: dependency updates and new-model support ship as patch/minor releases — see the CHANGELOG. Last verified: PHP 8.3/8.4/8.5 (NTS on macOS arm64 + Linux x86_64; ZTS 8.4/8.5 on Linux), llama-cpp-2 0.1.160 (llama.cpp, October 2026), Qwen3 0.6B reference models, 2026-10-29.

Remaining ideas are tracked as future issues. Co-maintainers welcome — if you run this in production and want a say in where it goes, open an issue or a PR.

What is ext-infer?

ext-infer is a PHP 8.3+ extension that loads a GGUF model and runs inference in the PHP process via llama.cpp. PHP-native semantic search, RAG pipelines, and CLI/worker inference work without shelling out to Python or hitting a remote API.

Written in Rust on top of ext-php-rs and the llama-cpp-2 bindings. The public PHP surface is fluent and role-aware — building a chat prompt looks like Prompt::system(...)->withUser(...), not a string of <|im_start|> tokens.

💬 Chat completions via an immutable Prompt builder that renders through the model's embedded template — no manual <|im_start|> plumbing.
🧱 Structured output — pass a JSON Schema (or raw GBNF grammar) and sampling is constrained so malformed output is impossible, not retried. A 0.6B model becomes a dependable extractor.
🧠 Reasoning-model aware — Response::answer() and Response::reasoning() split Qwen3 / R1-style <think>…</think> output automatically.
📊 Embeddings — Model::embed() returns an Embedding with dimensions(), normalize(), cosineSimilarity(), and packed() (zero-copy handoff to vector indexes) built in.
🎯 Reranking — RerankModel scores (query, document) pairs through Qwen3-Reranker's calibrated yes/no judgment; completes the embed → rerank two-stage retrieval pipeline.
⚡ In-process — no subprocess fork, no IPC, no daemon. Latency is whatever the model takes to decode.
🛠️ Apple Metal acceleration is opt-in (make release FEATURES=metal); CPU is the portable default.
🧵 Thread-safe — LlamaBackend is a Sync-guarded singleton and each call builds its own context, so ZTS PHP + parallel works by design.

Quick start

mkdir -p models
curl -L -o models/Qwen3-0.6B-Q8_0.gguf \
    https://huggingface.co/Qwen/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B-Q8_0.gguf

<?php
use Displace\Infer\Model;
use Displace\Infer\Prompt;

$model    = Model::load('models/Qwen3-0.6B-Q8_0.gguf');
$response = $model->chat(
    Prompt::system('You are a helpful assistant.')
        ->withUser('What is 2+2?'),
    maxTokens: 256,
    temperature: 0.0,
);

echo $response->answer(), PHP_EOL;   // "2 + 2 equals 4."
echo $response->reasoning() ?? '';    // captured <think>…</think>, if any

$model->close();

make build       # produces target/debug/libinfer.{so,dylib}
php -d extension=$PWD/target/debug/libinfer.dylib hello.php

Full walkthrough — including the interactive Symfony Console chat and pairwise-similarity embedding example — under examples/.

Documentation

infer.displace.tech hosts the full guide:

Getting started — install via PIE or from source, verify, troubleshoot.
Guide — prompts, chat, raw, embeddings, choosing a model.
Recipes — multi-turn chat, semantic search, RAG over markdown, worker pools.
Reference — full API surface, exceptions, environment variables, compatibility matrix.
Advanced — threading, Metal, performance tuning.

The site is built from docs/ with mdbook and deploys automatically on every push to main.

Compatibility

	macOS arm64	Linux x86_64	Linux arm64	Windows
PHP 8.3	✅	✅	✅	—
PHP 8.4	✅	✅	✅	—
PHP 8.5	✅	✅	✅	—

ZTS is supported by design (the code is thread-safe) and enabled in composer.json; CI runs the PHPT suite on ZTS PHP 8.4 and 8.5 (Linux) every week. Prebuilt release binaries are NTS; ZTS installs build from source through PIE. Windows is intentionally out of scope.

Roadmap

Shipped chat completions · raw completions · grammar/JSON-Schema constrained generation · embeddings (+ packed float32 output) · RerankModel · reasoning split · typed exceptions · PHPT suite · CI matrix · PIE-compatible composer.json · tag-triggered binary release workflow · THIRD-PARTY-NOTICES + cargo about license manifest · a current llama.cpp with newer architectures (0.2.1).

Remaining ideas live as future issues, each with its reasoning and any locked constraint (streaming, for one, is deliberately v0.3+). PLAN.md holds the design notes and decision history; RELEASE.md is the cut-a-release flow.

Deliberately out of scope

Training and fine-tuning · GPU-first serving (CPU is the portable default; Metal is opt-in) · frontier-model-class inference — the target is small local models on a 4GB+ box · an orchestration framework — chains and agents belong to the frameworks integrating ai-contracts · Windows. (Platform-wide non-goals; see the umbrella ARCHITECTURE.md.)

Part of a stack

Transcribe → chunk → embed → search → answer, entirely on your hardware:

Stage	Package	What it does
Transcribe	ext-whisper	Speech-to-text over whisper.cpp
Chunk	ai-toolkit	Text chunkers and packed-vector math, pure PHP
Embed · rerank · answer	ext-infer (this repo)	Local LLM chat, embeddings and reranking over llama.cpp
Search	ext-turbovec	Compressed in-memory vector search (TurboQuant)
Integrate	ai-contracts	The interfaces frameworks code against
Reference app	rag-over-markdown	The whole pipeline over a folder of markdown

Every package links the others; the extensions share one scaffold, ext-template.

Writing about this

The stack's launch series ran Sept 22 – Oct 1, 2026; everything else is at eric.mann.blog/writing.

displace / ext-infer

Package info

Security

Statistics