The provider dashboards show you one number — your total bill. That's like getting an electricity bill with no breakdown. You just see the total and hope nobody left the AC on.
Tbh, if you look closely at your API logs, you are probably wasting around 43% of your budget. I spent the last few weeks analyzing LLM usage across different teams, and the same leaks happen everywhere.
Here is where your money is actually going:
1. Retry Storms (34% of waste)
Your prompt fails to return valid JSON. The agent retries. It fails again. Next thing you know, your while-loop has fired 40 times. At 10k tokens a pop on Claude 3.5 Sonnet, that single user interaction just cost you a lot.
2. Duplicate Calls
Users ask the same questions. Without semantic caching, you are paying OpenAI to generate the exact same answer 100 times a day.
3. Context Bloat
Sending the entire chat history in every single request without truncation. You only need the last few turns, but your wrapper is sending 50k tokens "just in case."
4. Wrong Model Selection
Using GPT-4o for basic routing or classification tasks when a much smaller, cheaper model could do it 10x faster.
How to stop the bleeding
You can't fix what you can't see. If you don't have per-tenant cost attribution, you are flying blind. You need to know exactly which user, model, and feature is burning tokens.
I built LLMeter (open-source AGPL-3.0) to solve this. It tracks costs per model, per user, per day. It connects directly to OpenAI, Anthropic, DeepSeek, and OpenRouter to give you the exact breakdown without needing to route your traffic through a proxy.
Stop guessing. Track your per-user costs: https://llmeter.org?utm_source=devto&utm_medium=article&utm_campaign=devto-hidden-43-percent