An append-only audit log caught two accounting bugs in a 216-star usage tracker

Last month I wrote about , a 216-star Rust usage tracker for agentic CLIs — and within twelve days of the first report, both were fixed in an official release: one through my PR, one through a maintainer-authored patch that adopted the report's analysis. Here's the play-by-play, with the numbers.

Bug 1: your history quietly rewrites itself

Claude Code rewrites session JSONL files in place when you resume or compact a conversation. Between two scans of the same file, five of my assistant messages vanished — usage that was already incurred, gone from the record. Any tracker that recomputes totals from live files inherits this drift: yesterday's numbers change while you sleep.

I filed it as , since merged): it fails red on 3.5.9 and passes green on 3.6.0, which is exactly what you want a regression test to do.

The part I care most about: on the transcripts splitrail scans, 3.6.0 agreed with my append-only log token-exact — 18,548,947 output tokens on both sides across 13.5k messages. Two independent implementations, different languages, different dedup strategies, same number to the digit. (The tiny message-count delta was zero-usage api-error records splitrail intentionally skips.) When two systems reconcile exactly, every remaining discrepancy is a finding, not noise.

Bug 2: the discrepancy that was a finding

Because the log records where every message came from, I could classify all of them against the live tree:

Row two is the second bug. Claude Code writes subagent transcripts (Task tool: Explore, general-purpose, custom agents) under projects/<slug>/<sessionId>/subagents/, at directory depth ≥ 4. Splitrail's discovery hard-caps at depth 2 in three places (WalkDir…min_depth(2).max_depth(2), a components() == 2 path check, and the glob pattern). Those files are structurally invisible — 54% of my live messages, and roughly a third of the dollars, never entered any total.

The model mix makes the blind spot vivid: splitrail saw 22 of my 5,059 Sonnet messages and 0 of 1,566 Haiku messages. Those models run almost exclusively inside subagents. If you delegate heavily to cheap models — which is exactly what cost-conscious agent users do — your tracker undercounts the most, on the workflows you optimized hardest.

Filed as "Include Claude Code subagent transcripts" (+428/−100 across seven files), whose description restates the issue's analysis and adopts its central caution — simply lifting the depth cap isn't enough; subagent records have to flow through the same local_hash dedup semantics as main transcripts, or you trade an undercount for a double-count. Both fixes shipped together in splitrail 3.6.1, which hit Homebrew within a day. On my corpus, the newly visible class is row two of the table above: 16,160 messages and 13.7M output tokens finally entering the totals. A fresh 3.6.1-vs-log reconciliation of the live tree is the obvious next check — part 3 starts there.

What I'd generalize

Live mutable files are not an audit trail. If your source can rewrite history, recomputation is not accounting. Append-only ingest with a stable per-record identity (message.id, last-write-wins on partials) is cheap insurance — mine is a few hundred lines and a SQLite file.

Reconcile token-exact or you know nothing. "Close enough" totals hide entire bug classes. It was exact agreement on the scanned subset that turned the remaining gap into two nameable, fixable defects instead of a shrug.

Decompose the gap before you blame anyone. "Your numbers are lower than mine" is an accusation; "the gap is exactly deleted-files + an unscanned directory class, here's the table" is a bug report a maintainer can act on in hours. Both turnarounds here — #200→#204 in eight days, #207→#209 in two — happened because the mechanism arrived with the report.

Frozen snapshot + isolated $HOME is the whole trick for tracker A/B. Clone the data, pin HOME (and XDG_STATE_HOME/XDG_DATA_HOME — on Linux, state dirs honor XDG and will escape your sandbox; CodeRabbit's review caught that one in my own fixture, fair is fair), and two binaries scanning identical bytes become a controlled experiment.

Timeline

Twelve days, two bugs, zero arguments about whose numbers were right — the log settled every question before it became a debate.

The layer underneath

All of this ran on ; the subagent fix shipped in splitrail 3.6.1.