Retrieval-Augmented Self-Recall — What the Comments Taught Me (RE-call v0.3)

A follow-up to . Code: .

I published a thesis post about agent memory and got five comments that were better than the post.

Two of them didn't just critique the design — they described, precisely, why it would fail and what would fix it. So I did the only reasonable thing: I turned both into experiments, ran them on the same eval harness the series is built on, and shipped what survived. That's RE-call v0.3, and this post is the receipt.

I want to be explicit about why I'm writing it this way. The point of publishing this series was never broadcast — it was error-correction. A design you keep in a drawer accumulates conviction; a design you publish accumulates objections, and objections are the cheapest high-quality signal you will ever get. The comment section of Part 1 did more for this codebase than any week of solo iteration. This post exists to pay that back with the thing commenters almost never receive: evidence that someone listened, measured, and changed the code.

Comment 1: "A similarity score is not a confidence score"

had independently named the target shape:

Retrieval should return confidence + provenance + validity, not just relevance.

That sentence is now, almost verbatim, how RE-call's trust layer describes itself. Every hit returns a verdict (ok / superseded / expired / …), a calibrated confidence, and provenance; a memo declares supersedes: old-memo.md in its frontmatter at write time, and retrieval returns the current head of the chain instead of a resolved-but-still-embedded old decision.

For v0.3 I added the experiment that closes the "why not just timestamps?" question — against the steelman, not a strawman: "among the confidently-relevant hits, trust the newest", with the stale docs re-touched after their successors, the way any living corpus re-syncs constantly. Superseded-trust rate (how often the stale memo is handed back as the answer — lower is better):

Look at the bge-small row: the timestamp heuristic is worse than plain relevance ranking — the tie-break actively promotes the freshly-re-synced stale memo in the one case where ranking had preferred the successor. A per-document timestamp cannot see a two-document relation, and making the timestamp "smarter" makes it more confidently wrong. The declared relation holds at 0.00 in the same runs.

Vinicius also called the residual failure mode in advance: write-time binding is only as good as the author's discipline — a forgotten link is an orphan memo that looks valid forever. But, as I replied then: impossible to infer becomes possible to enforce. So v0.3 ships recall lint — dangling supersedes: references, cycles, ambiguous successors, versioned siblings with no declared edge, closures declared only in prose. No DB, exit 1 on errors, drops into CI in one line. (It paid for itself before it shipped: writing its tests uncovered a real parser bug where a scalar [[wikilink]] was read as a YAML list, producing an edge that silently never resolved.)

The experiment I still owe

's point — a typed coverage_check slot so the "no real match" signal can't be silently dropped — sharpened a design rule that now runs through the whole stack: the retriever computes the signal, the schema carries it. Computed, it's a measurement; self-reported by the model, it's a declaration. Those fail very differently.

What I'm actually arguing for

Five people I've never met read a post about a niche RAG problem and, between them, produced: a falsifiable critique of my abstention mechanism, the correct architecture for supersession, a proposed alternative worth benchmarking, and a schema-design principle. Total cost to me: publishing something concrete enough to be wrong about, and taking the replies seriously enough to run them.

That's the whole model. Not "content", not reach — working in public as a form of peer review. The asymmetry is absurdly favorable: you contribute one design and get back the failure modes it would have taken you months to hit alone. The only price is that you have to be willing to write "I was wrong, here's the measurement" — which, in a series whose thesis is calibrated honesty, is not a price at all. It's the product.

So: thank you Vinicius, Mateo, Nazar, Tae — and (MIT). The full v0.3 study with every table: .