Ten days ago I had a benchmark: reorder the expert weights inside a MoE model file by measured co-activation, and disk reads per token drop 2.23×. It was real, reproducible, and measured entirely on my own machine — which is to say, it was worth almost nothing yet.
Today the strongest number attached to this work is one I didn't produce: +32.3% decode throughput and −26.3% time-to-first-token on a 235B-parameter model running on a 48 GB MacBook, measured by someone I've never met, on an inference engine I'd never run, using my script unmodified, with the arms swapped as a page-cache control. Along the way, two of my three original pitches were refuted with data, my one pre-registered scaling prediction failed its own threshold, and both of my attempts to forecast a key parameter missed in opposite directions.
The refutations were the productive part. This post is about the process that made them cheap: five engines, a half-dozen strangers, one issue thread, and the discipline that emerged in it — because I think the process generalizes and the write-ups I usually see don't mention it.
The setup
The idea (project: , where @mabaeyens was shipping an expert-offload streamer for MLX, and measured his 744B engine's container — which already stores each expert as one contiguous ~19 MB read — at a residual +1.3%, below his own promotion gate. He posted it as a no-go, I agreed, and the ~+20% GLM-class number I'd floated earlier was retired. The coalescing ladder is now fully priced: nine-scattered → +14–32% depending on share; per-projection → +31% at deep offload; per-expert-contiguous container → approximately nothing, at device bandwidth ceiling. An optimization whose domain of inapplicability is measured and published is worth more than one with only success stories.
Nulls are product
The thread's graveyard, as of today: predictive prefetch (three independent nulls — the structural one, the properly-controlled A/B, the cross-architecture shuffle test), heat-pinning versus LRU, smarter-than-LRU eviction, and my original cross-layer framing (retracted). Each null is scoped — batch-1, decode, these architectures — and each one is a roadmap item somebody now doesn't have to build.
Then the ending wrote itself. Late in the thread, PhilipJohnBasile open-sourced his engine — and its README turned out to contain an independent replication of nearly the entire design theory, measured on a fifth engine before we'd ever spoken: router-lookahead prefetch reverted at −6.96%, smarter eviction dead at −26% with zero hit-rate change, static pinning dead, hotset training dead. His summary line: "the cache-policy frontier is closed: capacity vs reuse distance is the wall." We had converged on the same wall from four different codebases.
(That open-sourcing also resolved a citation mystery in one of the great small moments of the thread: a throughput figure everyone kept attributing to our measurements turned out to be an Italian code comment at line 6141 of the parent project his engine forks — measured on Linux in early July, a platform qualifier his fork's copy had dropped. Two of us had "verified" the wrong provenance because GitHub's code search silently doesn't index files that large. That's in the traps section now too.)
What I took away
Take claims to the people with the machines to kill them. Four engine authors falsified, scoped, or extended every major claim within days. No general audience does that. The niche thread was not the humble alternative to a launch; it was the higher-bandwidth channel.
Pre-registration is what makes being wrong cheap. Every number I committed to in advance either survived (and now carries weight) or died cleanly (and produced a law). The two forecasts I hedged on are the ones that cost follow-up rounds to untangle.
Public concession is the currency. The thread's author said it at the close, better than I can: "I learned more from the parts where I was wrong than the parts where I was right. Good thread."
Keep one versioned, red-line-able summary. It outperforms both the scrollback and any paper: it's current, it's editable, and every participant has standing to correct it.
Ship your nulls with your wins. The parity result on stock llama.cpp, posted up front, is why the right people engaged at all.
The thread is still open. Co-activation-aware ordering is now the named follow-up on the mlx-lm PR; the colibri community is building an expert atlas with the co-activation format from this work; there's a co-signed, pre-declared cross-engine experiment waiting on two routing traces. The consolidated summary is , and the repo is (the 744B endpoint, iliria), @pierre427 (production pager, the registered-band point), and the colibri crew — @JustVugg, @ZacharyZcR, @bokiko, @mohamedmastouri2000-boop — who are taking the co-activation work somewhere none of us planned.
For more information and other projects go to https://getnadir.com
SOCIAL SHARE CARD GENERATOR