README

Xberg

liter-llm

One API for every LLM — in your language, without the rewrites.

What and Why?

liter-llm gives you a single, consistent way to call any large language model. Reach 174 providers — OpenAI, Anthropic, Google, Bedrock, and more — through one client, and switch models by changing a name instead of your code. The same client ships to 14 languages, all built on one Rust core, so you get identical behavior everywhere. Need a gateway? Point your existing OpenAI SDK at the built-in, drop-in proxy — no rewrites, no separate service to run.

Work with any provider without rewrites — 174 providers behind one API; change the model name to switch, your code stays the same.
Use it from your language — Python, TypeScript, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, and WebAssembly, all built on the same Rust core, plus a C/FFI surface for everything else.
Secure by default — API keys are wrapped, redacted from logs, and never serialized; cloud auth for Azure, AWS, and Vertex refreshes and rotates for you.
One small binary — the client, a drop-in OpenAI-compatible proxy, and an MCP server for AI agents ship in a single 35 MB binary with no interpreter and no dependency tree to compromise.
Stay up when a provider fails — route by cost, latency, or weight and fall back automatically, with built-in rate limiting, caching, and cost tracking.

Deeper details — observability, middleware, and the full endpoint list — live in the documentation.

We credit litellm for proving the category; our provider registry was bootstrapped from theirs. See ATTRIBUTIONS.md.

Features

Feature	Description
174 providers	OpenAI, Anthropic, Google, AWS Bedrock, Groq, Mistral, Together, Fireworks, DeepSeek, Cohere, and 130+ more — compiled at build time
14 native bindings	Rust, Python, Node.js, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, WebAssembly — plus a shared C/FFI surface
First-class streaming	SSE and AWS EventStream binary protocol with zero-copy buffers
Proxy & MCP server	Drop-in OpenAI-compatible proxy (22 endpoints) and MCP tool server in a 35 MB Docker image
Tower middleware	Rate limiting, caching (40+ OpenDAL backends), cost tracking, budget enforcement, health checks, and fallback — all composable
Observability	OpenTelemetry with GenAI semantic conventions, cost-tracking spans, and HTTP-level tracing
Tool calling	Parallel tools, structured outputs, and JSON-schema validation
Search & OCR	Web search across 12 providers, document OCR across 4
TOML configuration	`liter-llm.toml` auto-discovery, custom providers, cache backends, and middleware config
Local LLM support	Ollama, LM Studio, vLLM, llama.cpp, LocalAI, and llamafile via OpenAI-compatible APIs

⭐ Star this repo to show your support — it helps others discover liter-llm.

Quick Start

Language Packages

Python

pip install liter-llm

See Python README for full documentation.

Node.js

pnpm add @xberg-io/liter-llm

See Node.js README for full documentation.

Rust

cargo add liter-llm

See Rust crate for full documentation.

go get github.com/xberg-io/liter-llm/packages/go/v2

See Go README for full documentation.

Java

Available on Maven Central as io.xberg.literllm:liter-llm. See Java README for the dependency snippet and current version.

Ruby

gem install liter_llm

See Ruby README for full documentation.

PHP

composer require xberg-io/liter-llm

See PHP README for full documentation.

dotnet add package LiterLlm

See .NET README for full documentation.

Elixir

Add {:liter_llm, "~> 1.6"} to your mix.exs dependencies. See Elixir README for full documentation.

Dart / Flutter

dart pub add liter_llm

See Dart README for full documentation.

Swift

Add via Swift Package Manager. See Swift README for full documentation.

Kotlin (Android)

Available on Maven Central as io.xberg.literllm:liter-llm-android. See Kotlin README for the dependency snippet and current version.

Zig

See Zig README for installation and usage.

WebAssembly

pnpm add @xberg-io/liter-llm-wasm

See WebAssembly README for full documentation.

C/C++ (FFI)

Build from source as part of this workspace. See FFI crate for full documentation.

CLI, Proxy & MCP Server

The liter-llm CLI ships both the OpenAI-compatible proxy and the MCP tool server. Install it any of these ways:

brew install xberg-io/tap/liter-llm
cargo install liter-llm-cli                 # from crates.io
cargo binstall liter-llm-cli               # prebuilt binary from GitHub Releases
pnpm dlx @xberg-io/liter-llm-cli --help         # npm (self-installs the binary)
docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-your-key ghcr.io/xberg-io/liter-llm

Windows users can install the same binary through Scoop:

scoop bucket add xberg https://github.com/xberg-io/scoop-bucket
scoop install liter-llm

Then run the proxy or the MCP server:

liter-llm api --config liter-llm-proxy.toml   # OpenAI-compatible proxy (22 endpoints)
liter-llm mcp --transport stdio               # MCP tool server (stdio)
liter-llm mcp --transport http --port 3001    # MCP tool server (Streamable HTTP)

See the MCP server guide and the proxy guide for transports, routing, virtual keys, and budgets. To use the MCP server inside a coding agent, install the liter-llm plugin (below) — it registers the server for your MCP client. The CLI release compatibility requirements below also apply to auto-installed binaries.

AI Coding Assistants

Install the liter-llm plugin from xberg-io/liter-llm. It ships the liter-llm agent skills (chat, streaming, tools, embeddings across 174 providers) — select your runtime below.

The bundle targets Claude Code, Codex, Cursor, Gemini CLI, Kimi, OpenCode, Factory Droid, Hermes, Pi, Agent Plugins and GitHub Copilot CLI. Read the plugin guide for runtime resources, CLI prerequisites and configuration. MCP-capable adapters expose the product's tools; Hermes uses a local stdio bridge and explicitly selected skills. Pi loads skills and prompt templates and requires the product CLI for executable operations.

Install only the public Agent Skills, without a plugin or MCP registration, using the skills CLI:

npx skills add xberg-io/liter-llm --list
npx skills add xberg-io/liter-llm

Choose the skills and target agents interactively. Installing instructions does not install the product CLI or configure provider credentials.

Published CLI 2.2.3 predates the local-stdio trust and deferred-provider-credential fixes. Until a CLI release includes them, use a source build for startup without an extra auth config:

cargo install --git https://github.com/xberg-io/liter-llm \
  --rev f14fb8862f9cc73e2c4ab38a2876efe77fab8bf4 --locked liter-llm-cli

Put the installed binary on the assistant's PATH. If a plugin has already cached an older binary, remove that plugin's bin/liter-llm (or bin/liter-llm.exe) before retrying so it selects the source build. The released 2.2.3 binary requires mcp.stdio_trust_local = true or a configured mcp.stdio_key_id; configured providers may also require credentials at startup. The source build defers missing provider credentials until a request uses that provider. HTTP authentication remains configured separately.

The source-built stdio MCP server trusts its local host by default and starts without a provider key. Set provider credentials such as OPENAI_API_KEY or ANTHROPIC_API_KEY before sending requests to that provider; missing credentials fail the request. HTTP MCP access retains its configured authentication. Use liter-llm api for the proxy and liter-llm mcp for MCP; chat and streaming are SDK, proxy or MCP operations rather than CLI subcommands.

Claude Code

/plugin marketplace add xberg-io/liter-llm
/plugin install liter-llm@liter-llm

Codex CLI

/plugins add https://github.com/xberg-io/liter-llm

Then search for liter-llm and select Install Plugin.

Cursor

Settings → Plugins → Add from URL → https://github.com/xberg-io/liter-llm, then select liter-llm.

Gemini CLI

From a checkout of this repository:

gemini extensions install ./plugin

The manifest is inside plugin/. See the Gemini extension reference for local installation and updates.

Factory Droid

droid plugin marketplace add https://github.com/xberg-io/liter-llm
droid plugin install liter-llm@liter-llm

GitHub Copilot CLI

copilot plugin marketplace add https://github.com/xberg-io/liter-llm
copilot plugin install liter-llm@liter-llm

Install from a local checkout now:

pi install ./plugin

After the next plugin release, install the same package used by OpenCode:

pi install npm:@xberg-io/opencode-liter-llm

Pi loads the bundled skills and prompt templates. Use /liter-llm-chat and /liter-llm-stream to start a task. Install the CLI separately as shown in the plugin README. This package does not register MCP servers, hooks, or executable extensions in Pi.

opencode

Add the package to opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "plugin": ["@xberg-io/opencode-liter-llm"]
}

Documentation

Full guides, the unified chat() API for every binding, multimodal I/O, the proxy/gateway, and the complete provider list live at docs.liter-llm.xberg.io.

Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines.

Join our Discord community for questions and discussion.

Part of Xberg.io

Xberg — document intelligence: text, tables, metadata from 101 formats with optional OCR.
Xberg Enterprise — managed extraction API with SDKs, dashboards, and observability.
crawlberg — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
html-to-markdown — fast, lossless HTML→Markdown engine.
liter-llm — universal LLM API client with native bindings for 14 languages and 174 providers.
tree-sitter-language-pack — tree-sitter grammars and code-intelligence primitives.
alef — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.

License

MIT — see LICENSE for details.

xberg-io / liter-llm

Package info

Security

Statistics