Retrieval-Augmented Self-Recall — Part 3: Teaching RAG to Say \"I Don't Know\

Part 3 of Retrieval-Augmented Self-Recall. Code: .

Ask your agent "have we tried this filter on this market before?" when the honest answer is never. A ranking retriever hands back the three closest memos anyway — something about a different filter, on a different market — and the agent, looking at three confident results, concludes: yes, we've looked at this.

It just made a decision on a hallucination. Nothing in the stack noticed. No error was raised, because from the retriever's point of view nothing went wrong: you asked for the nearest neighbours and it gave you the nearest neighbours.

Everything in Part 2 made retrieval good. Good ranking makes this failure worse, not better — it returns confident noise faster. This post is about making retrieval honest, which for agent memory is the part that actually decides whether you can trust it.

So RE-call wraps retrieval in honesty guards — this post covers the original three, each answering a question ranking metrics never ask. (The current repo has grown that table to six, and the growth story is .

Guard 3: anti-re-litigation — "did we already settle this?"

The most agent-specific guard of the three. Before the agent proposes an idea, it queries memory for closed decisions on that topic — the "we tried X, it failed, here's why" memos — and the guard surfaces them.

The failure it prevents is subtle and expensive: an agent re-proposing a dead idea because the memo that killed it three months ago didn't happen to rank in the top results for today's phrasing. Ranking-optimized retrieval is bad at this specifically, because a settled-decision memo is often lexically distant from the fresh proposal even though it's the most decision-relevant document in the store.

The implementation leans on structure: decision-type memos (closed hypotheses, postmortems) are typed, and a targeted retrieval path prioritizes them when the agent is in "propose" mode. Memory that can't defend its own past decisions is condemned to relive them.

The unifying idea

Retrieval answers one question: what's closest? The guards answer the three that actually govern whether the agent should act:

• 
  Should you trust it? (gap_warning)


• 
  Is it still current? (freshness)


• 
  Did we already decide this? (anti-re-litigation)

That's the whole difference between a search index and a memory. A search index ranks. A memory knows its own limits.

(Since this was drafted, the guard table grew: trust verdicts with declared supersession, an opt-in entailment judge for the high-similarity-but-wrong case a threshold can never catch, and a write-time lint for the supersession graph. All three exist because readers argued with this post's ancestors — that story, with measurements, is . This is the layer that makes Claude Code, Beyond the Prompt's memory trustworthy, not just searchable.