"The price table most AI cost tools multiply by has one automated test: jq empty

Most tools that tell you what your AI usage costs work the same way: count tokens locally, multiply by a shared price table. For most of the ecosystem that table is a single JSON file in the LiteLLM repo, and its only automated check is jq empty, which verifies the JSON parses. That is the entire quality gate: no value validation, no cross-check against provider pages, no guard against swapped input and output prices, no dates, no provenance.

This is not an obscure file. LiteLLM (BerriAI/litellm, ~51k stars, MIT) prices roughly 2,783 models. ccusage (~16.4k stars) re-pins it hourly, tokencost vendors a copy, ccost embeds it. An error in it is not one tool's bug; it is the whole ecosystem agreeing on a wrong number, with nothing left over to disagree.

Why I was in there

I maintain a spend monitor for Claude Code called BudgetClaw: it tails the local JSONL logs Claude Code already writes, computes cost in real time, and on a budget breach sends SIGTERM to the process. Zero keys, zero prompts, zero added latency. What sent me upstream was a Saturday spent grouping a month of usage by branch: about $2,770 total, one refactor branch at $543 by itself, mostly retry loops re-sending an 80K+ token context. But a tool that kills processes over dollars has to get the dollars right, and cache pricing makes that delicate: a cache read bills at 0.1x the base input rate, a 5-minute cache write at 1.25x, a 1-hour write at 2x. One wrong multiplier and the kill switch fires at the wrong moment or not at all. Quietly wrong is worse than loudly wrong. So before trusting the table, I read it.

Findings

1. Records that cite their own refutation. Four xai/grok-4.20-* rows are priced at $2 input / $6 output per million tokens. The source field on those same rows points at docs.x.ai, which says $1.25 / $2.50.

"input_cost_per_token":  2e-06,    // $2.00 per million

"output_cost_per_token": 6e-06,    // $6.00 per million

"source": "https://docs.x.ai/..."  // the cited page says $1.25 / $2.50

The wrong number and its own correction, in one record. jq empty passes; it parses flawlessly.

2. No guard against transposition. Input and output prices have been swapped before. The command-r7b fix had to bring its own test, because there was none to fail.

3. Provenance is optional. A source appears on about 31% of records. There is no last_validated, no confidence, no date field of any kind. For two thirds of the table, the reason to believe a number is that it is in the file.

4. The past is deleted. When a price changes, the old value is overwritten and gone. ccusage issue #764, closed unresolved, is users pointing out that today's prices get applied to all historical usage, so reported costs do not match what was charged. They asked for point-in-time pricing; the upstream model deletes the past, so it could not be given.

5. Merging a fix may not fix anything. The default branch is litellm_internal_staging. Public main can run about nine days behind it, serving a bug already fixed on staging, and a bulk staging sync can overwrite a community fix after it merges. A correct price can land, merge, and silently regress.

A control experiment

A static audit only goes so far, so I filed the grok fix and watched the pipeline handle it: BerriAI/litellm#30849, opened 2026-06-19. Twenty-eight lines, correcting both the root map and the bundled backup so they agree, plus the cost-calculator test that did not exist.

Weeks later it is open and unmerged, reviewed by exactly three entities: a CLA signature bot, greptile, and codecov. No human. I report this as data, not a grievance, and per finding 5, even a merge might not stick.

None of this is a scandal about people; it is what jq empty looks like at scale. That repo has fielded on the order of 1,158 pricing PRs, and no volunteer team hand-verifies that volume against provider pages. The maintainers run the most successful price table in the ecosystem exactly as well as its architecture allows. Every finding above is structural.

Diagnosis

The instinct is to call this a coverage problem and fix it with more rows. It is not. LiteLLM carries ~2,783 models, models.dev ~5,278; the set I actually need is 94. One bad price kills a cost tool, and 2,700 correct ones do not save it. A price with no source and no date is fine for an estimate and wrong for a bill.

Provenance-first is not exotic: every record carries a first-party source URL, an effective date, a last_validated date, and a confidence label. Corrections supersede old rows instead of deleting them, so priceOn(model, date) returns the rate actually in effect that day, which is what ccusage #764 wanted. A bot diffs each provider's own pricing page; a human verifies every change before it lands. I keep such an index for my 94 models, with history back to GPT-4's 2023 launch (github.com/RoninForge/ai-price-index, CC-BY).

The point

Parsing usage logs is the easy 90%; the tokens sit on your disk with model names attached. The dollar at the bottom is only as honest as the price it was multiplied by, and for most of the AI-cost stack that price lives in a JSON file whose sole certification is that it parses. jq empty answers one question: is this valid JSON. Every tool downstream is answering another: what did this cost. Right now nothing stands between the two.

The index is open. If I have a price wrong, the fix is a PR.