Search by

php-loves-ai / multimodal-ai-runner

Oleksii Fedorchak

Run AI models locally from PHP: text-to-image, image-to-image, text-to-text, image-to-text, speech-to-text, text-to-speech, text-to-video, image-to-video, face recognition and multiple-choice decisions with JEV classifiers.

Package info

github.com/AlexeyFedorchak/php-loves-ai

pkg:composer/php-loves-ai/multimodal-ai-runner

Statistics

Installs: 75

Dependents: 0

Suggesters: 0

Stars: 0

Open Issues: 0

v8.0.0 2026-11-01 05:40 UTC

This package is auto-updated.

Last update: 2026-11-02 16:22:13 UTC


README

██████╗ ██╗  ██╗██████╗      ████ ████      █████╗ ██╗
██╔══██╗██║  ██║██╔══██╗    ███████████    ██╔══██╗██║
██████╔╝███████║██████╔╝    ███████████    ███████║██║
██╔═══╝ ██╔══██║██╔═══╝      █████████     ██╔══██║██║
██║     ██║  ██║██║            █████       ██║  ██║██║
╚═╝     ╚═╝  ╚═╝╚═╝              █         ╚═╝  ╚═╝╚═╝

Run AI models locally from PHP. Generate images, write and answer text, describe pictures, transcribe speech, read text aloud, enlarge photos, make short videos and recognize faces — on your own machine, from PHP code or the command line, with any matching model from Hugging Face.

Task What it does Example model
🎨 text-to-image Generate an image from a text prompt stabilityai/sd-turbo
🔎 image-to-image Enlarge a photo, or redraw it following a prompt caidas/swin2SR-classical-sr-x2-64
✍️ text-to-text Answer a prompt, or continue it Qwen/Qwen2.5-0.5B-Instruct
👀 image-to-text Describe an image, or answer questions about it HuggingFaceTB/SmolVLM-256M-Instruct
🎧 speech-to-text Transcribe speech in audio or video openai/whisper-tiny
🗣️ text-to-speech Read text aloud into an audio file facebook/mms-tts-eng
🎬 text-to-video Generate a video from a text prompt Wan-AI/Wan2.1-T2V-1.3B-Diffusers
🎞️ image-to-video Animate an image into a video stabilityai/stable-video-diffusion-img2vid-xt
🤔 jev Answer a multiple-choice question about text, an image, audio or video akhilaaa3/Jev-Omni
🙂 face Find, count and fingerprint the faces in a photo immich-app/buffalo_l
🤝 face-match Check whether two images show the same person immich-app/buffalo_l

🚀 Easy to start

composer require php-loves-ai/multimodal-ai-runner

vendor/bin/loves-ai setup                  # 🧰 one-time setup
vendor/bin/loves-ai setup text-to-image    # 🎨 add the task you need

vendor/bin/loves-ai pull stabilityai/sd-turbo   # ☕ grab the model you like
vendor/bin/loves-ai text-to-image stabilityai/sd-turbo "a cozy cat by the fireplace" --steps=1 --guidance=0
🎨 Painting your image with stabilityai/sd-turbo… Masterpieces take a moment — perfect time for a hot chocolate and a cookie 🍪
🎉 Image saved to /Users/you/tmp/hugging-face/images/20260916-142501-a3f09c.png

🐘 …and the same from PHP:

use PhpLovesAi\Runner\TextToImage;

$image = (new TextToImage())->generate(
    model: 'stabilityai/sd-turbo',
    prompt: 'a cozy cat by the fireplace',
    outputPath: storage_path('app/cat.png'),
    steps: 1,
    guidanceScale: 0.0,
);

💡 Run vendor/bin/loves-ai on its own to see every task and which ones are ready to use.

🔒 No Python, no API keys, no cloud. Models run on your machine; nothing is sent anywhere. A Hugging Face key is only needed for private or gated models.

Requirements

  • PHP 8.2+
  • proc_open enabled (used to invoke the binaries)
  • allow_url_fopen and the openssl extension (used by setup to download the binaries)
  • tar on the PATH (built into macOS, Linux and Windows 10+)

Supported platforms: macOS arm64, Linux x86_64, Linux arm64, Windows x86_64.

Installation

composer require php-loves-ai/multimodal-ai-runner
vendor/bin/loves-ai setup                  # downloads the puller (~17 MB) and asks for your Hugging Face API key
vendor/bin/loves-ai setup text-to-image    # optional: the image generation runner (a few hundred MB)
vendor/bin/loves-ai setup text-to-text     # optional: the text generation runner (a few hundred MB)
vendor/bin/loves-ai setup image-to-text    # optional: the image description runner (a few hundred MB)
vendor/bin/loves-ai setup speech-to-text   # optional: the speech transcription runner (a few hundred MB)
vendor/bin/loves-ai setup text-to-speech   # optional: the speech synthesis runner (a few hundred MB)
vendor/bin/loves-ai setup image-to-image   # optional: the image enlarging and redrawing runner (a few hundred MB)
vendor/bin/loves-ai setup text-to-video    # optional: the video generation runner (a few hundred MB)
vendor/bin/loves-ai setup image-to-video   # optional: the image animation runner (a few hundred MB)
vendor/bin/loves-ai setup jev              # optional: the multiple-choice decision runner (a few hundred MB)
vendor/bin/loves-ai setup face             # optional: the face runner (~40 MB, no PyTorch; face-match uses it too)
🧰 Setting up php-loves-ai (v6.0.0) for darwin-arm64
🔑 Hugging Face API key (optional)
   Public models, like stabilityai/sd-turbo, are pulled without a key. Private and gated models need one:
   create it at https://huggingface.co/settings/tokens
   Paste your key (hidden), or press Enter to use public models only:
✅ Saved your Hugging Face API key to /var/www/my-app/.local/huggingface/credentials.json
✅ The puller is installed at /var/www/my-app/.local/runners/puller-darwin-arm64
🎉 All set! Happy hacking 🍪
👉 Pull a model: vendor/bin/loves-ai pull <model>

Commands

Everything runs through one command, vendor/bin/loves-ai, so nothing in vendor/bin clashes with other packages. Running it without arguments lists the tasks and marks the runners that are installed:

🧰 php-loves-ai v6.0.0 — run small AI models locally, without installing Python

Usage: vendor/bin/loves-ai <command> [arguments] [options]

     setup           Download the runner binaries for this computer
     pull            Pull a model from the Hugging Face Hub

Tasks (✅ = runner installed, run `setup <task>` for the others):
  ✅ text-to-image   Generate an image from a text prompt
     image-to-image  Enlarge an image, or redraw it following a prompt
     text-to-video   Generate a video from a text prompt
     image-to-video  Animate an image into a video
  ✅ text-to-text    Generate text: answer a prompt or continue it
     image-to-text   Describe an image, or answer a question about it
     speech-to-text  Transcribe speech in an audio or video file
     text-to-speech  Read text aloud into an audio file
     jev             Answer a question by choosing one of its options
  ✅ face            Find, count and fingerprint the faces in an image
  ✅ face-match      Check whether two images show the same person

Run 'vendor/bin/loves-ai <command> --help' for a command's arguments and options.

vendor/bin/loves-ai --version prints the installed package version.

Hugging Face API key

A key is optional. Public models are pulled without one; private and gated models need it (create one at https://huggingface.co/settings/tokens).

setup asks for it once and saves the answer in .local/huggingface/credentials.json, readable only by its owner. Pressing Enter is remembered too, so setup does not ask again. To save or replace a key later, run vendor/bin/loves-ai setup --token=hf_... or pass --token=hf_... to pull. When there is no terminal to ask in (Docker builds, CI, deploy scripts), setup skips the question; pass --token there if you need a key.

The key is never read from environment variables, so every process running the project uses the same one.

Where models and runners live

Everything is stored inside your project, next to vendor/, in one fixed place:

<project root>/.local/
├── models/        models pulled by `vendor/bin/loves-ai pull`, as models/<model id>
├── runners/       binaries installed by `vendor/bin/loves-ai setup`
└── huggingface/   credentials.json with your Hugging Face API key

These paths cannot be changed. They depend only on the project directory, never on the user, HOME, the working directory or environment variables. So the CLI, the web server (php-fpm running as www-data), queue workers and other containers sharing the project directory all find the same models and runners, and no code or config ever needs a path. On a server or in Docker, run setup and pull once in the project and forget about it. The web server's user needs read and execute access to .local.

setup and pull report the path they installed to, and add a .gitignore to .local, so its hundreds of MB and your API key are never committed. Add .local/ to .dockerignore if you build images from the project directory.

setup --force re-downloads, and setup --debug shows the URLs and paths used. After upgrading the package, run vendor/bin/loves-ai setup --force (plus setup text-to-image --force if you use it) to get the matching binaries.

Running a command before its binary is installed fails with The puller is not installed yet. and a hint to run setup.

Pulling models

Public models need no API key. Private and gated models use the key saved in the project (see Hugging Face API key).

From the command line

vendor/bin/loves-ai pull openai-community/gpt2 [--revision=main] [--token=hf_...] [--all] [--log-file=PATH] [--debug]
☕ Pulling openai-community/gpt2… Big downloads take a moment — perfect time for a cup of tea and some cookies 🍪
If you wish to see all logs, re-run the command with the "--debug" option.
🎉 Pulled openai-community/gpt2 into /var/www/my-app/.local/models/openai-community/gpt2
   Skipped 3 files (450.0 MB) the runners cannot read: other frameworks or training leftovers.

Only the files the runners can read

Hugging Face repositories usually publish the same weights several times over, for PyTorch, TensorFlow, Flax and ONNX. The runners read PyTorch, plus ONNX where that is the only format a repository ships, so the rest is downloaded for nothing. pull leaves those files out:

Model Repository Pulled
openai/whisper-tiny 0.61 GB 0.16 GB
SfinOe/stable-diffusion-v1.5 10.96 GB 5.48 GB
facebook/mms-tts-eng 0.29 GB 0.15 GB
immich-app/buffalo_l 0.36 GB 0.18 GB
nlpconnect/vit-gpt2-image-captioning 0.98 GB 0.98 GB

Skipped are weights for other frameworks (.h5, .msgpack, .gguf, .ckpt, and .armnn, .rknn, .mnn, which are one vendor's NPU build of the same weights and no runner loads), single-file copies of a diffusers pipeline, alternative versions such as fp16 and non_ema, leftovers from training (optimizer.pt, trainer_state.json, checkpoint-*) and sample media. A .bin file is skipped only when the same repository also publishes it as .safetensors, so models that ship .bin alone — the vit-gpt2 row above — are pulled untouched.

.onnx works the same way round: it is skipped next to PyTorch weights, where it is the same model exported a second time, and kept when it is all a repository has — which is what makes the InsightFace packs behind face pullable. On immich-app/buffalo_l that keeps the 182 MB of .onnx and drops 175 MB of .armnn and .rknn.

Pass --all to download the repository as it is, if a model ever needs a file these rules leave out.

The opening message is picked at random from a few cozy variants (see PullCommand::INTROS).

When Hugging Face refuses a model, pull explains why and what to do:

Error: meta-llama/Llama-3.2-1B is not available: it is a gated model, which needs a Hugging Face API key.
Re-run with your key: vendor/bin/loves-ai pull meta-llama/Llama-3.2-1B --token=<your Hugging Face API key> (create one at https://huggingface.co/settings/tokens)

--token saves the key for next time. A gated model also needs its terms accepted on its Hugging Face page, with the account the key belongs to.

Pulls one model at a time into .local/models/<model id>. The puller's own output is hidden unless --debug is given. To keep that output, set a log file — each run is appended to it with a timestamp, the puller's output and the outcome. Output is colored on terminals; set NO_COLOR=1 to disable colors. Exit codes: 0 success, 1 pull failed, 2 invalid usage. Run vendor/bin/loves-ai pull --help for details.

Defaults come from config/pull.php:

Key Default Overridden by
revision main --revision
log_file null (output discarded) --log-file

From PHP

use PhpLovesAi\Process\ModelPuller;

$paths = (new ModelPuller())->pull(
    ['openai-community/gpt2', 'distilbert/distilbert-base-uncased'],
    onProgress: fn (string $chunk) => fwrite(STDERR, $chunk),
    // allFiles: true,  // download every file, as `pull --all` does
    onSkipped: fn (string $model, int $files, int $bytes) => fwrite(STDERR, "{$model}: skipped {$files} files\n"),
);
// ['openai-community/gpt2' => '/var/www/my-app/.local/models/openai-community/gpt2', ...]

Failures throw exceptions implementing PhpLovesAi\Exception\PhpLovesAiException. A model Hugging Face refuses throws ModelAccessDeniedException ($model, $reason: gated or not_found, $apiKeyUsed); other failures throw PullFailedException. Both list the models pulled before the failure in $pulled. To save a key from PHP, use (new PhpLovesAi\HuggingFace\Credentials())->saveApiKey('hf_...').

The binary can also be used directly:

[HUGGING_FACE_API_KEY=hf_...] .local/runners/puller-darwin-arm64 --dir .local/models [--revision main] [--all] -- openai-community/gpt2

It writes one JSON line per model to stdout, {"model": "...", "path": "...", "skipped_files": 3, "skipped_bytes": 471859200} when pulled or {"model": "...", "error": "gated|not_found"} when refused, and progress to stderr. Exit codes: 0 success, 1 at least one model failed, 2 invalid arguments.

Generating images

Pull a complete Diffusers text-to-image model first, such as stabilityai/sd-turbo. Its repository has a model_index.json. Add-ons like embeddings or LoRAs, and single-file checkpoints, cannot be used on their own: the runner rejects them with an explanation before starting.

Then:

From the command line

vendor/bin/loves-ai pull stabilityai/sd-turbo
vendor/bin/loves-ai text-to-image stabilityai/sd-turbo "a cozy cat by the fireplace" --steps=1 --guidance=0
🎨 Painting your image with stabilityai/sd-turbo… Masterpieces take a moment — perfect time for a hot chocolate and a cookie 🍪
If you wish to see all logs, re-run the command with the "--debug" option.
🎉 Image saved to /Users/you/tmp/hugging-face/images/20260915-142501-a3f09c.png
Option Meaning
--output=PATH Image file to write (default: timestamped .png in output_dir)
--negative-prompt=TEXT What the image should not contain
--steps=N Inference steps (default: the model's own)
--guidance=SCALE Guidance scale; turbo models use 0 (default: the model's own)
--width=PX, --height=PX Image size (default: the model's native size)
--seed=N Random seed, for reproducible images
--device=DEVICE cpu, cuda, mps… (default: the best available)
--log-file=PATH Append the runner's output to this file
--debug Show the runner's output while generating

Values are passed to the model as-is — one it cannot handle (a prompt that is too long, an unsupported size…) makes the run fail. Defaults come from config/text-to-image.php (output_dir, log_file).

From PHP

use PhpLovesAi\Runner\TextToImage;

// Finds the runner and the pulled model in the project's .local directory by itself.
$image = (new TextToImage())->generate(
    model: 'stabilityai/sd-turbo',
    prompt: 'a cozy cat by the fireplace',
    outputPath: __DIR__ . '/cat.png',
    steps: 1,
    guidanceScale: 0.0,
);
// '/.../cat.png'

Throws BinaryNotInstalledException when setup text-to-image has not been run, ModelNotFoundException when the model was not pulled yet, UnsupportedModelException when the pulled model is not a complete Diffusers text-to-image model, and RunFailedException (with the runner's error output) when generation fails.

Enlarging and redrawing images

Two kinds of image-to-image models work, and the runner tells them apart by the model's own files:

  • Upscaling models, e.g. caidas/swin2SR-classical-sr-x2-64 (2× larger) or caidas/swin2SR-realworld-sr-x4-64-bsrgan-psnr (4×). They enlarge a photo and clean it up, and take no prompt. This is the one to use for making images bigger than a plain resize can.
  • Diffusers image-to-image pipelines, e.g. stabilityai/sd-turbo or timbrooks/instruct-pix2pix. They redraw the image following a prompt, keeping more or less of the original depending on --strength.

(For plain resizing to a smaller size, PHP's own GD or Imagick extension is faster and needs no model.)

From the command line

vendor/bin/loves-ai pull caidas/swin2SR-classical-sr-x2-64
vendor/bin/loves-ai image-to-image caidas/swin2SR-classical-sr-x2-64 photo.jpg --output=photo-2x.png

vendor/bin/loves-ai pull stabilityai/sd-turbo
vendor/bin/loves-ai image-to-image stabilityai/sd-turbo photo.jpg --prompt="a watercolor painting" --strength=0.6 --steps=2 --guidance=0
🔎 Making it bigger and better with caidas/swin2SR-classical-sr-x2-64… How about a hot chocolate while you wait? ☕
If you wish to see all logs, re-run the command with the "--debug" option.
🎉 Image saved to /var/www/my-app/photo-2x.png
Option Meaning
--output=PATH Image file to write (default: a timestamped .png in output_dir)
--prompt=TEXT What the result should look like; needed by diffusers models, refused by upscaling models
--negative-prompt=TEXT What the result should not contain (diffusers models)
--strength=N How much of the original to keep, 0 to 1; higher changes more (diffusers models)
--steps=N Inference steps (default: the pipeline's own)
--guidance=SCALE Guidance scale; turbo models use 0 (default: the pipeline's own)
--seed=N Random seed, for reproducible images
--device=DEVICE cpu, cuda, mps… (default: the best available)
--log-file=PATH Append the runner's output to this file
--debug Show the runner's output while working

Defaults come from config/image-to-image.php (output_dir, log_file).

From PHP

use PhpLovesAi\Runner\ImageToImage;

// Finds the runner and the pulled model in the project's .local directory by itself.
$imageToImage = new ImageToImage();

$bigger = $imageToImage->transform(
    model: 'caidas/swin2SR-classical-sr-x2-64',
    imagePath: storage_path('app/photo.jpg'),
    outputPath: storage_path('app/photo-2x.png'),
);

$painting = $imageToImage->transform(
    model: 'stabilityai/sd-turbo',
    imagePath: storage_path('app/photo.jpg'),
    outputPath: storage_path('app/painting.png'),
    prompt: 'a watercolor painting',
    strength: 0.6,
    steps: 2,
    guidanceScale: 0.0,
);

Throws ImageNotFoundException when the image does not exist, BinaryNotInstalledException when setup image-to-image has not been run, ModelNotFoundException when the model was not pulled yet, UnsupportedModelException when the model does not produce images, and RunFailedException (with the runner's error output) when the run fails, e.g. when a prompt is missing or given to a model that takes none.

Upscaling works on the whole image at once, so memory use grows with the picture: a large photo can need several GB. Enlarging in a queue job, and shrinking very large photos first, keeps web requests safe.

Generating text

Pull a transformers text generation model first. Both kinds of text models work:

  • Chat and completion models (task text-generation), e.g. Qwen/Qwen2.5-0.5B-Instruct or HuggingFaceTB/SmolLM2-360M-Instruct. Chat models get the prompt wrapped in their chat template, so they answer it; other models continue it.
  • Encoder-decoder models (task text2text-generation), e.g. google/flan-t5-base.

The runner rejects models it cannot load before starting, with an explanation: GGUF files (made for llama.cpp and Ollama), ONNX-only repositories, LoRA adapters, image models, and models that need their own Python code (trust_remote_code), which it never runs.

From the command line

vendor/bin/loves-ai pull Qwen/Qwen2.5-0.5B-Instruct
vendor/bin/loves-ai text-to-text Qwen/Qwen2.5-0.5B-Instruct "Write a haiku about PHP." --system="You are a poet."
✍️ Writing with Qwen/Qwen2.5-0.5B-Instruct… Good words take a moment — perfect time for a cup of tea and a cookie 🍪
If you wish to see all logs, re-run the command with the "--debug" option.
🎉 Qwen/Qwen2.5-0.5B-Instruct wrote:
<the generated haiku>
Option Meaning
--system=TEXT Instructions for chat models, e.g. "You are a helpful assistant."
--max-new-tokens=N Maximum length of the answer in tokens (default: 256)
--temperature=T Randomness: 0 always picks the likeliest words (default: the model's own)
--top-p=P Nucleus sampling probability, e.g. 0.9 (default: the model's own)
--seed=N Random seed, for reproducible text
--device=DEVICE cpu, cuda, mps… (default: the best available)
--log-file=PATH Append the runner's output to this file
--debug Show the runner's output, and the text as it is written

Defaults come from config/text-to-text.php (log_file).

From PHP

use PhpLovesAi\Runner\TextToText;

// Finds the runner and the pulled model in the project's .local directory by itself.
$answer = (new TextToText())->generate(
    model: 'Qwen/Qwen2.5-0.5B-Instruct',
    prompt: 'Summarize in one sentence: PHP is a popular general-purpose scripting language...',
    systemPrompt: 'You are a concise assistant.',
    maxNewTokens: 100,
    temperature: 0.0,
);

Throws BinaryNotInstalledException when setup text-to-text has not been run, ModelNotFoundException when the model was not pulled yet, UnsupportedModelException when the model is not a transformers text model, and RunFailedException (with the runner's error output) when generation fails.

Small models run on CPU, but larger ones get slow quickly: a 0.5B model writes a few words per second on a laptop CPU, and each run loads the model from disk again. Run generation in a queue job rather than in a web request.

Describing images

Pull a transformers image-to-text model first. Both kinds work:

  • Vision-language models (task image-text-to-text), e.g. HuggingFaceTB/SmolVLM-256M-Instruct or HuggingFaceTB/SmolVLM-500M-Instruct. They answer a question about the image; without one, they describe it.
  • Captioning models (task image-to-text), e.g. Salesforce/blip-image-captioning-base or nlpconnect/vit-gpt2-image-captioning. They write a short caption; a prompt is the start of the caption, e.g. "a photography of".

As with text models, the runner rejects models it cannot load before starting, with an explanation: text-only models, GGUF files, ONNX-only repositories and models that need their own Python code (such as Florence-2 or Moondream).

From the command line

vendor/bin/loves-ai pull HuggingFaceTB/SmolVLM-256M-Instruct
vendor/bin/loves-ai image-to-text HuggingFaceTB/SmolVLM-256M-Instruct photo.jpg "What are the animals doing?"
👀 Studying your picture with HuggingFaceTB/SmolVLM-256M-Instruct… Grab a warm drink while it finds the right words ☕
If you wish to see all logs, re-run the command with the "--debug" option.
🎉 HuggingFaceTB/SmolVLM-256M-Instruct says:
The animals are sleeping.
Option Meaning
--max-new-tokens=N Maximum length of the text in tokens (default: 256)
--temperature=T Randomness: 0 always picks the likeliest words (default: the model's own)
--seed=N Random seed, for reproducible text
--device=DEVICE cpu, cuda, mps… (default: the best available)
--log-file=PATH Append the runner's output to this file
--debug Show the runner's output, and the text as it is written

Defaults come from config/image-to-text.php (log_file).

From PHP

use PhpLovesAi\Runner\ImageToText;

// Finds the runner and the pulled model in the project's .local directory by itself.
$imageToText = new ImageToText();

$description = $imageToText->generate('HuggingFaceTB/SmolVLM-256M-Instruct', storage_path('app/photo.jpg'));

$answer = $imageToText->generate(
    model: 'HuggingFaceTB/SmolVLM-256M-Instruct',
    imagePath: storage_path('app/photo.jpg'),
    prompt: 'Is there any text in this image? Write it out.',
    maxNewTokens: 100,
    temperature: 0.0,
);

Throws ImageNotFoundException when the image file does not exist, BinaryNotInstalledException when setup image-to-text has not been run, ModelNotFoundException when the model was not pulled yet, UnsupportedModelException when the model cannot read images, and RunFailedException (with the runner's error output) when generation fails, e.g. because the file is not an image.

Transcribing speech

Pull a transformers speech recognition model first. Both kinds work:

  • Whisper-style models, e.g. openai/whisper-tiny, openai/whisper-base or openai/whisper-small (larger is more accurate and slower). They are multilingual: they detect the spoken language, can be told it, and can translate the speech into English.
  • CTC models, e.g. facebook/wav2vec2-base-960h. They transcribe the one language they were trained on.

The runner decodes audio itself, with no FFmpeg installation needed: WAV, MP3, M4A/AAC, FLAC, OGG/Opus and the audio track of video files work, at any length. It rejects models it cannot load before starting, with an explanation, including whisper.cpp (GGML) and faster-whisper (CTranslate2) conversions, which are common on Hugging Face.

From the command line

vendor/bin/loves-ai pull openai/whisper-tiny
vendor/bin/loves-ai speech-to-text openai/whisper-tiny interview.m4a
vendor/bin/loves-ai speech-to-text openai/whisper-tiny interview.m4a --timestamps
🎧 Listening carefully with openai/whisper-tiny… Perfect time for a cup of tea and a cookie 🍪
If you wish to see all logs, re-run the command with the "--debug" option.
🎉 Transcript by openai/whisper-tiny:
[00:00.00 → 00:05.56] He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat
[00:05.56 → 00:11.04] mutton pieces to be ladled out in thick peppered flower fatten sauce.
Option Meaning
--language=LANGUAGE Spoken language for Whisper-style models, e.g. en or french (default: detected)
--translate Translate the speech into English (Whisper-style models)
--timestamps Print when each segment is spoken: phrases for Whisper-style models, words for CTC models
--device=DEVICE cpu, cuda, mps… (default: the best available)
--log-file=PATH Append the runner's output to this file
--debug Show the runner's output while transcribing

Defaults come from config/speech-to-text.php (log_file).

From PHP

use PhpLovesAi\Runner\SpeechToText;

// Finds the runner and the pulled model in the project's .local directory by itself.
$speechToText = new SpeechToText();

$transcript = $speechToText->transcribe('openai/whisper-tiny', storage_path('app/interview.m4a'));

$english = $speechToText->transcribe(
    model: 'openai/whisper-small',
    audioPath: storage_path('app/interview-uk.mp3'),
    language: 'uk',
    translate: true,
);

// For subtitles: seconds from the start of the audio; the last segment may have no end.
$segments = $speechToText->transcribeWithTimestamps('openai/whisper-tiny', storage_path('app/interview.m4a'));
// [['start' => 0.0, 'end' => 5.56, 'text' => 'He hoped there would be stew for dinner, …'], ...]

Throws AudioNotFoundException when the file does not exist, BinaryNotInstalledException when setup speech-to-text has not been run, ModelNotFoundException when the model was not pulled yet, UnsupportedModelException when the model cannot transcribe speech (or cannot be told a language or translate), and RunFailedException (with the runner's error output) when transcription fails, e.g. because the file has no audio.

Small models (whisper-tiny, whisper-base) are quick; larger models and long recordings take a while, especially on CPU, and each run loads the model from disk again, so run transcription in a queue job.

Reading text aloud

Pull a transformers text-to-speech model first, one that needs nothing but text:

  • VITS and MMS models, e.g. facebook/mms-tts-eng (one repository per language, such as mms-tts-deu or mms-tts-ukr) or kakao-enterprise/vits-ljs. They are small and fast.
  • Bark, e.g. suno/bark-small, which has named voices such as v2/en_speaker_6, chosen with --voice.

Models that need extra files or their own Python code are rejected before starting, with an explanation: SpeechT5 (which needs a speaker embedding file), Kokoro and Parler-TTS (which ship their own code), and speech recognition models given to the wrong runner.

From the command line

vendor/bin/loves-ai pull facebook/mms-tts-eng
vendor/bin/loves-ai text-to-speech facebook/mms-tts-eng "PHP loves AI, and now it can speak." --output=hello.wav
🎵 Turning your words into sound with facebook/mms-tts-eng… How about a hot chocolate while you wait? ☕
If you wish to see all logs, re-run the command with the "--debug" option.
🎉 Audio saved to /var/www/my-app/hello.wav
Option Meaning
--output=PATH Audio file to write; its extension picks the format: .wav, .mp3, .m4a, .flac, .ogg (default: a timestamped .wav in output_dir)
--voice=VOICE Voice of models that have several, e.g. a Bark preset like v2/en_speaker_6, or a speaker number
--speed=RATE Speaking rate of VITS-style models, e.g. 0.8 slower, 1.2 faster (default: the model's own)
--seed=N Random seed, for reproducible audio
--device=DEVICE cpu, cuda, mps… (default: the best available)
--log-file=PATH Append the runner's output to this file
--debug Show the runner's output while speaking

Defaults come from config/text-to-speech.php (output_dir, log_file).

From PHP

use PhpLovesAi\Runner\TextToSpeech;

// Finds the runner and the pulled model in the project's .local directory by itself.
$file = (new TextToSpeech())->speak(
    model: 'facebook/mms-tts-eng',
    text: 'PHP loves AI, and now it can speak.',
    outputPath: storage_path('app/hello.mp3'),
    speed: 0.9,
);
// '/var/www/my-app/storage/app/hello.mp3'

Throws BinaryNotInstalledException when setup text-to-speech has not been run, ModelNotFoundException when the model was not pulled yet, UnsupportedModelException when the model cannot speak, and RunFailedException (with the runner's error output) when generation fails, e.g. for an output format it cannot write.

Small voice models speak a sentence in a second or two on a laptop CPU, but each run loads the model again, so run longer texts in a queue job.

Generating videos

Pull a diffusers video model first, e.g. Wan-AI/Wan2.1-T2V-1.3B-Diffusers, zai-org/CogVideoX-2b or an AnimateDiff pipeline.

Video models are the heaviest thing here. They are several gigabytes to pull, want a lot of memory, and a few seconds of video takes minutes on a GPU and up to hours on a CPU. Start with the smallest model, few frames and a small frame size, and always generate in a queue job.

From the command line

vendor/bin/loves-ai pull Wan-AI/Wan2.1-T2V-1.3B-Diffusers
vendor/bin/loves-ai text-to-video Wan-AI/Wan2.1-T2V-1.3B-Diffusers "a cat walking through tall grass" --frames=33 --output=cat.mp4
🎬 Rolling the camera with Wan-AI/Wan2.1-T2V-1.3B-Diffusers… Films take their time — perfect for a pot of tea and some cookies 🍪
If you wish to see all logs, re-run the command with the "--debug" option.
🎉 Video saved to /var/www/my-app/cat.mp4
Option Meaning
--output=PATH Video file to write; its extension picks the format: .mp4, .webm, .mkv, .gif (default: a timestamped .mp4 in output_dir)
--negative-prompt=TEXT What the video should not contain
--frames=N Number of frames to generate (default: the pipeline's own)
--fps=N Frames per second of the written file (default: 8)
--steps=N Inference steps (default: the pipeline's own)
--guidance=SCALE Guidance scale (default: the pipeline's own)
--width=PX, --height=PX Frame size (default: the pipeline's own)
--seed=N Random seed, for reproducible videos
--device=DEVICE cpu, cuda, mps… (default: the best available)
--log-file=PATH Append the runner's output to this file
--debug Show the runner's output while filming

Defaults come from config/text-to-video.php (output_dir, log_file).

From PHP

use PhpLovesAi\Runner\TextToVideo;

// Finds the runner and the pulled model in the project's .local directory by itself.
$clip = (new TextToVideo())->generate(
    model: 'Wan-AI/Wan2.1-T2V-1.3B-Diffusers',
    prompt: 'a cat walking through tall grass',
    outputPath: storage_path('app/cat.mp4'),
    frames: 33,
    fps: 16,
);

Throws BinaryNotInstalledException when setup text-to-video has not been run, ModelNotFoundException when the model was not pulled yet, UnsupportedModelException when the model is not a diffusers video pipeline (a still-image pipeline says so and points to text-to-image), and RunFailedException (with the runner's error output) when generation fails, e.g. for an output format it cannot write or when memory runs out.

Animating images

Pull a diffusers image-to-video model first. They differ in what they take:

  • Image only, e.g. stabilityai/stable-video-diffusion-img2vid-xt. It animates the picture on its own and refuses a prompt.
  • Image and prompt, e.g. Wan-AI/Wan2.1-I2V-14B-480P-Diffusers, zai-org/CogVideoX-5b-I2V or LTX-Video. The prompt says what should happen in the clip.

A model whose repository only ships its text-to-video pipeline is converted to the image-to-video pipeline of the same family automatically, when diffusers has one.

The same warning as for text-to-video applies: these models are large and slow, so generate in a queue job.

From the command line

vendor/bin/loves-ai pull stabilityai/stable-video-diffusion-img2vid-xt
vendor/bin/loves-ai image-to-video stabilityai/stable-video-diffusion-img2vid-xt photo.jpg --output=clip.mp4
🎬 Bringing your picture to life with stabilityai/stable-video-diffusion-img2vid-xt… Films take their time — perfect for a pot of tea and some cookies 🍪
If you wish to see all logs, re-run the command with the "--debug" option.
🎉 Video saved to /var/www/my-app/clip.mp4

The options are the same as text-to-video, plus --prompt=TEXT for models that accept one. Defaults come from config/image-to-video.php (output_dir, log_file).

From PHP

use PhpLovesAi\Runner\ImageToVideo;

// Finds the runner and the pulled model in the project's .local directory by itself.
$clip = (new ImageToVideo())->generate(
    model: 'Wan-AI/Wan2.1-I2V-14B-480P-Diffusers',
    imagePath: storage_path('app/photo.jpg'),
    outputPath: storage_path('app/clip.mp4'),
    prompt: 'the camera slowly zooms out',
    frames: 33,
    fps: 16,
);

Throws ImageNotFoundException when the image does not exist, BinaryNotInstalledException when setup image-to-video has not been run, ModelNotFoundException when the model was not pulled yet, UnsupportedModelException when the model is not a diffusers video pipeline, and RunFailedException when the run fails, e.g. when the model takes no prompt but one was given.

Answering multiple-choice questions

JEV models are decision classifiers: give them a question and its options, and they return a probability for each option instead of writing an answer, so the result is always one of your options. The question can be about text (the state), an image, an audio file or a video. Useful for routing, moderation, yes/no checks and triage, where the answer must be one of a fixed set.

Pull a JEV model first, e.g. akhilaaa3/Jev-Omni. It is a Gemma 4 12B model with a decision head (head.pt and decision_config.json); chat models without that head are rejected before starting, with an explanation. The model's own Python loader, jev_omni.py, is never run: the runner rebuilds the head from its weights alone.

Jev-Omni is large: a 24 GB pull, and about as much free GPU or unified memory to run. Its authors test it on CUDA GPUs; on Apple Silicon it needs a Mac with 32 GB or more, and on a CPU a single question can take minutes. Decide in a queue job, not in a web request.

From the command line

vendor/bin/loves-ai pull akhilaaa3/Jev-Omni
vendor/bin/loves-ai jev akhilaaa3/Jev-Omni "Has the meeting started?" Yes No --state="The meeting starts at 10 AM. It is now 9 AM."
vendor/bin/loves-ai jev akhilaaa3/Jev-Omni "What is on the table?" "A cat" "A dog" "Nothing" --media=photo.jpg
🤔 Weighing the options with akhilaaa3/Jev-Omni… Perfect time for a cup of tea and a cookie 🍪
If you wish to see all logs, re-run the command with the "--debug" option.
🎉 akhilaaa3/Jev-Omni chose: No (98.6%)
      Yes    1.4%
   👉 No    98.6%
Option Meaning
--state=TEXT What the question is about, as text
--media=PATH Image, audio or video file the question is about
--modality=KIND image, audio or video (default: told by the file's extension)
--frames=N Frames sampled evenly from a video (default: 16)
--device=DEVICE cpu, cuda, mps… (default: the best available)
--log-file=PATH Append the runner's output to this file
--debug Show the runner's output while deciding

Give 2 to 256 different options; the model answers best with up to 20. Audio is read up to its first 30 seconds, and decoded by the runner itself, as are videos, with no FFmpeg installation needed. Defaults come from config/jev.php (log_file).

From PHP

use PhpLovesAi\Runner\Jev;

// Finds the runner and the pulled model in the project's .local directory by itself.
$jev = new Jev();

$decision = $jev->decide(
    model: 'akhilaaa3/Jev-Omni',
    question: 'Has the meeting started?',
    options: ['Yes', 'No'],
    state: 'The meeting starts at 10 AM. It is now 9 AM.',
);
$decision->prediction;    // 'No'
$decision->index;         // 1
$decision->confidence;    // 0.986
$decision->probabilities; // ['Yes' => 0.014, 'No' => 0.986]

$mood = $jev->decide(
    model: 'akhilaaa3/Jev-Omni',
    question: 'How does the caller sound?',
    options: ['Happy', 'Neutral', 'Upset'],
    mediaPath: storage_path('app/call.m4a'),
);

Throws \InvalidArgumentException for fewer than two options, duplicate options or an unknown modality, MediaNotFoundException when the media file does not exist, BinaryNotInstalledException when setup jev has not been run, ModelNotFoundException when the model was not pulled yet, UnsupportedModelException when the model has no decision head, and RunFailedException (with the runner's error output) when the run fails, e.g. when the media cannot be decoded.

Recognizing faces

Four things, from one model pack: tell whether a picture has a face in it, count the faces, check whether two pictures show the same person, and turn each face into a short face id you can store in a database and match against later.

Pull an InsightFace pack first. They hold two ONNX graphs — a detector that finds faces and their landmarks, and a recognizer that turns each aligned face into 512 numbers:

Model Pull size Notes
immich-app/buffalo_l ~182 MB The default recommendation; what the numbers below were measured with
immich-app/buffalo_s ~113 MB Smaller and faster, a little less accurate
immich-app/antelopev2 ~360 MB Larger; some report it finding fewer faces than buffalo_l

These are the InsightFace models, republished for Immich in a layout the runner reads. Check the model card for the licence before shipping one in a product: InsightFace releases its pretrained models for non-commercial research use.

This runner needs no PyTorch. It is onnxruntime, numpy and Pillow, so setup face downloads ~40 MB (115 MB unpacked) against 218 MB for text-to-image, and a photo with three faces takes about half a second on a CPU.

Faces are personal data almost everywhere. Storing face ids or embeddings for identifiable people brings GDPR, BIPA and similar obligations with it, whatever the model licence says.

From the command line

vendor/bin/loves-ai setup face
vendor/bin/loves-ai pull immich-app/buffalo_l

vendor/bin/loves-ai face immich-app/buffalo_l crew.jpg
vendor/bin/loves-ai face immich-app/buffalo_l crew.jpg --count
vendor/bin/loves-ai face-match immich-app/buffalo_l passport.jpg selfie.jpg
🙂 Looking for faces with immich-app/buffalo_l… Should be quick — no time for a whole cookie 🍪
🎉 Found 3 faces in crew.jpg
   👤 1  91.3% confident, 80×114 px at 201,254
      v1:256:34ae1b70385ab1df212d044a4c36cd46421b2bc1cb5b80ae01fbf87709288745
   👤 2  91.0% confident, 75×108 px at 426,117
      v1:256:22bc1f64552ec40f8a1d5c9e3b7740d1e8a2f05b6c3d9417ab28e6f5103c7d82
   👤 3  89.6% confident, 81×115 px at 657,266
      v1:256:4a6e0df6c0617b2f94e8d13a5026cb8f7d41920e3b6a5c8df0e71248a93b6054

--count only runs the detector, which skips loading the 166 MB recognizer — use it when all you need is "is there a face here?".

For a backend, --json prints one object and nothing else, so it pipes straight into your code:

vendor/bin/loves-ai face immich-app/buffalo_l crew.jpg --json --embeddings
{"model":"immich-app/buffalo_l","image":"crew.jpg","count":3,"faces":[
  {"id":"v1:256:34ae1b70…","score":0.9125,"box":[201,254,281,368],
   "landmarks":[[232,302],[268,303],[255,323],[229,335],[266,336]],
   "embedding":[0.035172,0.021544,-0.029648, …]}]}

From PHP

use PhpLovesAi\Runner\Face;

$face = new Face();

$face->hasFace('immich-app/buffalo_l', storage_path('app/upload.jpg'));     // true
$face->countFaces('immich-app/buffalo_l', storage_path('app/crew.jpg'));    // 3

foreach ($face->analyze('immich-app/buffalo_l', storage_path('app/crew.jpg')) as $found) {
    $found->id;         // "v1:256:34ae1b70…" — store this per face
    $found->embedding;  // the 512 numbers it is derived from
    $found->box;        // [left, top, right, bottom] in pixels
    $found->score;      // 0.9125
    $found->width();    // 80
}

hasFace() and countFaces() only run the detector; analyze() also runs the recognizer. Pass withEmbeddings: false to get the ids without the 512 numbers.

Matching two pictures:

$match = $face->compare(
    model: 'immich-app/buffalo_l',
    imagePath: storage_path('app/passport.jpg'),
    otherImagePath: storage_path('app/selfie.jpg'),
    threshold: 0.45,
);

$match->samePerson;      // true
$match->similarity;      // 0.6819
$match->threshold;       // 0.45 — stored results still say what they meant
$match->wasAmbiguous();  // true when either picture held more than one face

When a picture holds several faces, compare() uses the most confident one in each and facesInA / facesInB say how many there were.

Face ids, and how to compare them

A face id is a fingerprint, not a primary key. It is a locality-sensitive hash of the embedding: similar faces get similar bit strings, so two photos of one person give two different ids that differ in only some of their bits. Never compare them with === — compare the distance:

use PhpLovesAi\Runner\FaceId;

FaceId::distance($rowA->face_id, $rowB->face_id);   // 68  -> same person
FaceId::distance($rowA->face_id, $rowC->face_id);   // 109 -> someone else

Measured with buffalo_l at the default 256 bits:

Pair Cosine similarity Distance
Same person, two different photos 0.68 68 / 256
Same photo at ⅓ scale, JPEG quality 60 0.72 63 / 256
Same photo rotated 5°, or 35% brighter 0.97 16–20 / 256
Two different people 0.01–0.02 109–122 / 256

So a cut-off around 90 separates them, and unrelated faces centre on half the bits (the expected distance is bits × θ / π, where θ is the angle between the embeddings). Measure it on your own images before relying on it — only you know what a wrong match costs. The same goes for threshold: config/face.php ships 0.40 as a starting point, not a safe default.

Two ids only mean anything side by side when they came from the same model, the same hash_bits and the same hash version — FaceId::distance() refuses to compare ids that disagree, rather than returning a meaningless number. hash_bits lives in config/face.php rather than in each call for that reason: changing it invalidates every id you have stored.

On the CPU provider ids are reproducible across machines: one photo hashed byte-identically on macOS arm64, Linux arm64 and Linux x86_64, so a stored id still matches after a move from a developer's Mac to a container on either chip, and a queue worker agrees with a web server. They are not identical across execution providers — CPU and CoreML came out a bit or two apart on the same image — which is one more reason to compare by distance, never equality.

Fewer bits is a shorter string but a blunter one. At 64 bits the same-person and different-person distances overlap, which is why the default is 256 (a 71-character id, prefix included):

$table->string('face_id', 72)->index();   // "v1:256:" + 64 hex characters

For an exact answer, keep the embedding too and compare that — FaceId::similarity() does it in PHP, and a vector column (pgvector, or any vector store) does it at scale. The usual shape is to shortlist candidates by id distance, then re-rank the shortlist by cosine similarity on the embeddings.

analyze() throws ImageNotFoundException when the image does not exist, BinaryNotInstalledException when setup face has not been run, ModelNotFoundException when the model was not pulled, UnsupportedModelException when the model is not an InsightFace pack, \InvalidArgumentException for an unknown bit count or an out-of-range score, and RunFailedException (with the runner's error output) when the run itself fails. compare() additionally throws NoFaceDetectedException when either picture has no face in it, and FaceId::distance() throws InvalidFaceIdException for a malformed id or two that cannot be compared.

How it works

The Composer package is tiny and holds only PHP code; the heavy parts live outside of it:

  1. Puller binary — a Python program that downloads models from Hugging Face and saves them in the project.
  2. Runner binaries — one per task, each holding its own copy of the libraries that task needs: PyTorch for most, and for face only onnxruntime, which is why it is a tenth of their size. Within a task, one runner serves many models: diffusers, transformers and onnxruntime pick the right architecture from the model's own files, so new models work without a new release.
  3. Both are compiled with PyInstaller into standalone programs, built per platform by GitHub Actions and attached to each GitHub release. setup downloads the ones matching the current OS, and the other commands find them automatically. Nothing needs Python installed.
  4. You choose which models to pull; weights never travel through Composer.
  5. The PHP classes run those programs through Symfony Process and give you a plain, typed API.

Releasing binaries

Publishing a GitHub release runs .github/workflows/release-binaries.yml, which builds every binary on each supported platform and attaches <tool>-<os>-<arch>.tar.gz plus a .sha256 checksum to the release. setup downloads from the release matching the installed package version (development installs use the latest release). The workflow can also be started by hand for an existing tag.

To build and pack locally (PyInstaller does not cross-compile, so this covers the current platform only):

python/puller/build.sh                    # → python/puller/dist/puller-<os>-<arch>
python/runners/text-to-image/build.sh     # → python/runners/text-to-image/dist/text-to-image-<os>-<arch>/
python/runners/text-to-text/build.sh      # → python/runners/text-to-text/dist/text-to-text-<os>-<arch>/
python/runners/image-to-text/build.sh     # → python/runners/image-to-text/dist/image-to-text-<os>-<arch>/
python/runners/speech-to-text/build.sh    # → python/runners/speech-to-text/dist/speech-to-text-<os>-<arch>/
python/runners/text-to-speech/build.sh    # → python/runners/text-to-speech/dist/text-to-speech-<os>-<arch>/
python/runners/image-to-image/build.sh    # → python/runners/image-to-image/dist/image-to-image-<os>-<arch>/
python/runners/text-to-video/build.sh     # → python/runners/text-to-video/dist/text-to-video-<os>-<arch>/
python/runners/image-to-video/build.sh    # → python/runners/image-to-video/dist/image-to-video-<os>-<arch>/
python/runners/jev/build.sh               # → python/runners/jev/dist/jev-<os>-<arch>/
python/runners/face/build.sh              # → python/runners/face/dist/face-<os>-<arch>/
python/package.sh                         # → python/release/*.tar.gz + *.sha256

Linux binaries can be built from any Docker host (including a Mac): python/build-in-docker.sh linux/amd64 or python/build-in-docker.sh linux/arm64 builds and packs them on an old glibc base, so they run on Debian 11+, Ubuntu 20.04+ and RHEL 9+. The release workflow uses the same script.

Name the binaries you want after the output directory to build only those: python/build-in-docker.sh linux/arm64 release face takes minutes, where the torch-based runners take hours. .github/workflows/build-runner.yml does the same on all four platforms and publishes nothing — push a ci/** branch to find out whether a runner survives PyInstaller on a platform you do not have, including a smoke test that loads a real model.

The runners bundle torch (plus diffusers or transformers, torchvision for image-to-text and PyAV with FFmpeg's libraries for the audio runners), so they are built as directories (~600–700 MB, ~210–220 MB packed) rather than single files; Linux builds use CPU-only torch to stay within GitHub's release asset size limit. face is the exception: onnxruntime, numpy and Pillow only, so it is ~115 MB built and ~40–50 MB packed. To test setup against local assets, serve python/release over HTTP and set PHP_LOVES_AI_DOWNLOAD_URL to its URL.

Structure

bin/                 loves-ai, the only script exposed via vendor/bin
config/              Package config (pull.php and one file per runner)
python/              Python sources compiled into standalone binaries (not shipped via Composer)
  puller/            Pulls models from Hugging Face and saves them locally
  runners/           One runner per task, running locally saved models
    text-to-image/   Generates images with diffusers models
    image-to-image/  Enlarges images, or redraws them with diffusers models
    text-to-video/   Generates videos with diffusers pipelines
    image-to-video/  Animates images with diffusers pipelines
    text-to-text/    Generates text with transformers models
    image-to-text/   Describes images with transformers models
    speech-to-text/  Transcribes speech with transformers models
    text-to-speech/  Reads text aloud with transformers models
    jev/             Answers multiple-choice questions with JEV decision classifiers
    face/            Finds, counts, matches and fingerprints faces (onnxruntime, no torch)
src/
  Enum/              Model registry
  Binary/            Platform detection and installing (Installer) the prebuilt binaries
  Config/            Config loading and validation
  Console/           Application (the loves-ai command) and one Command per subcommand
  Filesystem/        LocalStorage (the fixed paths inside .local) and path helpers
  HuggingFace/       Credentials: the optional API key saved in .local/huggingface/credentials.json
  Process/           PHP wrapper that invokes the puller binary (ModelPuller)
  Runner/            One class per task running pulled models (TextToImage, ImageToImage, TextToVideo, ImageToVideo,
                     TextToText, ImageToText, SpeechToText, TextToSpeech, Jev, Face), sharing the Runner
                     interface, plus FaceId for comparing stored face ids
  Exception/         Package exceptions
tests/
  Unit/
  Integration/

License

MIT