LLM Gateway
Open-source OpenAI-compatible gateway routing one API to 200+ models across 40+ providers
0Visit LLM Gateway
llmgateway.ioAbout LLM Gateway
LLM Gateway is an open-source, OpenAI-compatible routing layer that puts one API in front of more than 200 models across 40-plus providers, including OpenAI, Anthropic and Google. The problem it solves is operational rather than intellectual: teams that use several providers end up juggling separate keys, separate dashboards, separate billing and separate SDK quirks, and switching provider for a given model means a code change. Pointing an existing OpenAI-compatible client at the gateway collapses that into one endpoint with real-time cost tracking across all of it. Around the core routing sit the features that make a gateway worth running in production — automatic provider selection and custom routing rules, budgets and spend controls, prompt caching, guardrails, reliability failover, and activity logs with exportable data. The commercial model is unusual and worth reading carefully: the free tier is free forever with all 200+ models available and bring-your-own-keys included, and the platform fee is 5% on credit usage rather than a monthly subscription, so cost scales with spend rather than seats. Metadata retention is free while full payload storage is billed at $0.01 per million tokens. Supporting surfaces include an MCP server, an AI SDK provider, a token cost calculator, model comparison and rankings pages, and the vendor states it is SOC 2 Type II certified.
Does ChatGPT recommend your AI tool?
If you're building in AI Agent Infrastructure, run a free AI-visibility scan on your own product — we ask ChatGPT across 5 prompt angles and score how often you get named. ~30 seconds, no signup, no card.
Key Features
LLM Gateway Pros & Cons
✅ Pros
- +Usage-based 5% fee instead of a seat subscription
- +BYOK on the free tier means you can route without moving billing
- +SOC 2 Type II certified, which is rare for an open-source gateway
⚠️ Cons
- −The 5% platform fee compounds on top of provider costs at scale
- −Full payload logging is billed separately from metadata
- −Free-tier log retention is capped at 30 days
Tags
Is LLM Gateway your tool?
This is the page buyers and AI assistants read when they look up LLM Gateway. Claim your listing for $19 one-time — no subscription, nothing to cancel — and get a Featured badge, top placement in your category, and a permanent dofollow backlink. Prefer it ongoing? Monthly is one click away on the next page.
Complete Your AI Agent Stack
Other ai agent tools in our catalog:
1Password
Try FreeSecrets and credential manager
Keep API keys and .env secrets out of your repo
Consensus
Try FreeAI search across 200M research papers
Source real evidence behind your analysis
Gamma
Try FreeAI presentation builder
Turn ideas into polished decks instantly
💰 Affiliate disclosure: We may earn a commission if you sign up through these links at no extra cost to you.
Stay updated on AI Agent Infrastructure tools — join our weekly newsletter
One concise email with fresh launches, trending picks, and featured standouts.
Alternatives to LLM Gateway
View all LLM Gateway alternatives →More AI Agent Infrastructure tools
CMDOP
Outbound-only messenger and MCP server for running commands across a fleet of machines with no open ports, no VPN and no infrastructure to provision
Fetch Hive
Build, deploy and trace AI agents and prompt workflows as APIs, with per-step cost, token and duration tracing on every run
FlurryPORT
Webhook capture, signature verification and replay that doubles as an action broker giving agents typed actions with server-side credentials
LobsterMail
Instant email inboxes for autonomous agents — no API keys, no human signup, with custom domains and an SDK
Lunary
Observability and prompt management for LLM apps — event logs, traces, cost and latency tracking, versioned prompts and human review.
Lutra AI
Productivity agent that connects email, spreadsheets, documents and CRMs, building custom MCP/REST/OpenAPI integrations to automate repetitive work.
Agent connectivity: not yet verified