As an indie developer building AI products, I kept hitting the same wall: how do I control LLM costs without vendor lock-in?
Direct API keys meant zero visibility into per-project spending. Month-end surprise bills. No way to cap costs per feature. Rate limit headaches.
So I built CostLLM — an OpenAI-compatible API gateway that sits between your code and the LLM provider.
What it does
Virtual API Keys — Create separate keys per project, environment, or team member
Usage Tracking — Real-time token consumption dashboard
Budget Controls — Hard caps prevent overspend
Rate Limiting — Configurable per key
Drop-in Compatibility — Works with any OpenAI SDK
One line change
# Before
client = OpenAI(api_key="sk-proj-...")
# After
client = OpenAI(
base_url="https://api.costllm.ai/v1",
api_key="YOUR_COSTLLM_API_KEY"
)
Built with
- Python (FastAPI) gateway + backend
- Redis for rate limiting
- PostgreSQL for user & usage data
- Nginx reverse proxy
- Creem for payments
Supported models
| Model | Type |
|---|---|
| deepseek-v4-flash | Fast chat |
| deepseek-v4-pro | Complex reasoning |
| deepseek-chat | General purpose |
| deepseek-reasoner | Deep analysis |
What I learned
Building a production API gateway taught me a lot about concurrency, streaming, error handling, and payment webhooks. The hardest part wasn't the code — it was designing a pricing model that works for both free users and growing teams.
Try it
- Website:
- Free tier: $10 signup credit, no credit card required
I'd love feedback from other developers working with LLM APIs — especially around cost visibility, key management, and what you'd want from a gateway.
SOCIAL SHARE CARD GENERATOR