An open-source LLM gateway developed by BerriAI in the United States. It lets you call 140+ providers — including GPT, Claude, and Gemini — through a single OpenAI-compatible API, so developers no longer need to learn a different SDK or request format for each vendor. Switching models or falling back when one fails becomes a matter of configuration rather than code. Since its release in July 2023, it has grown beyond simply normalizing API calls: team-level budgets, rate limiting, cost tracking, audit logs, and PII masking have turned it into a gateway that consolidates an organization’s entire LLM usage into one place. It is MIT-licensed (the enterprise directory is under a separate license) and runs on your own servers or Kubernetes cluster.
Key Features
- 140+ providers behind one API: OpenAI, Anthropic, Google Vertex AI, Amazon Bedrock and other major providers are normalized into OpenAI’s request/response format. Exceptions and errors are mapped to the OpenAI standard as well, so you can swap models without rewriting application code
- Two ways to use it — SDK or proxy: A Python SDK you embed directly in your application, and a standalone proxy (AI gateway) server. With the proxy, any OpenAI-compatible client can connect as-is, requiring no changes to existing code
- Routing and fallbacks: Distribute load across multiple deployments and automatically switch to another model or region when rate limits or errors occur. Auto-routing can also send simple prompts to cheaper models and complex requests to stronger ones
- Virtual keys and budgets: Issue virtual API keys per user, team, or organization, with hard spending caps and RPM/TPM limits. Requests are stopped automatically once a cap is reached, preventing surprise bills
- Cost tracking and observability: Track spend by key, team, or user, and export Prometheus metrics. Integrates with logging platforms such as Langfuse, Arize Phoenix, LangSmith, and OpenTelemetry
- Guardrails and MCP gateway: Includes guardrails such as PII masking, prompt injection detection, and secret detection. It can also act as a central hub that aggregates MCP (Model Context Protocol) servers and distributes them with access control
Pricing
| Plan | Price | Key features |
|---|---|---|
| Open Source (self-hosted) | $0 | 140+ providers, virtual keys, user/team management, budgets and rate limits, fallbacks, request/response logging, Prometheus metrics |
| Enterprise | Contact sales (annual contract) | Everything above plus SSO + SCIM, OIDC/JWT auth, audit logs, secret manager integration and key rotation, org and team admin roles, multi-region control plane, 24/7 support with SLAs, and air-gapped deployment |
Pricing is as of August 2026. Enterprise is quoted individually based on annual request capacity, deployment architecture, and support needs — the company states explicitly that it is not priced per token. Check the official site for the latest pricing.
Note that while the software itself is free, you still pay each underlying LLM provider for usage. And because you host it yourself, server costs and operational effort are on you.
Pros & Cons
✅ Pros
- No need to learn provider-specific SDKs, which dramatically lowers the cost of switching models
- Open source (MIT) and self-hostable, so prompts and responses never have to pass through a third party
- Virtual keys and spending caps make LLM costs visible and controllable per team or project
- Fallbacks and load balancing are built in, providing resilience against provider outages and rate limits
- Supports Docker, Kubernetes, Helm, and Terraform, making it easy to fit into existing infrastructure
⚠️ Cons
- It is infrastructure software for developers and platform engineers, not a chat app for end users
- Self-hosting means you own the burden of setup, upgrades, and monitoring
- Enterprise governance features such as SSO, audit logs, and SCIM sit behind the Enterprise tier, whose price is not published
- The breadth of supported providers comes with many configuration options; tuning routing and guardrails takes some learning
- Because everything is normalized to the OpenAI format, the newest provider-specific features may not be usable directly
Comparison with Similar Services
| Criteria | LiteLLM | OpenRouter | Portkey | Cloudflare AI Gateway |
|---|---|---|---|---|
| Delivery model | Self-hosted (OSS) | Hosted service | Hosted (OSS edition available) | Managed on Cloudflare |
| Open source | Yes (MIT) | No | Partly | No |
| Primary role | Internal LLM gateway | Unified access and billing across many models | Gateway + observability | Caching, rate limiting, and logging layer |
| Budgets and virtual keys | Built in | Managed via credits | Built in | Mainly limits |
| Data path | Inside your own servers | Through the service | Through the service (inside your own network for the OSS edition) | Through Cloudflare |
Although all of these sit “in front of the LLM,” their roles differ. Choose LiteLLM if you want to keep control of the data path in-house; choose OpenRouter if you would rather consolidate contracts and payments.
Who Is It For
- Development teams using several LLM providers whose calling code is scattered across vendor-specific SDKs
- AI program leads who need visibility into LLM spend by department or project, with enforceable caps
- Companies handling confidential data that want to avoid routing prompts and responses through external services
- Product operators who need fallback paths so a single provider’s outage or rate limit does not stop the service
- Teams that want to consolidate internal MCP servers in one place and distribute them under access control
Summary
LiteLLM is an open-source foundation that funnels a crowded field of LLM providers into a single entry point and layers cost management, access control, and guardrails on top. It is not a casual chat tool for individuals, but for organizations that have reached the stage of running multiple models in production, it is a practical way to keep both model flexibility and operational control. A sensible path is to start with the free self-hosted edition to make your usage patterns visible, then consider Enterprise once you need SSO and audit logs.