The judge gate: why a passing validator isn't a finished feature

There's a failure mode shared by every autonomous coding agent I've watched: the agent declares victory the moment its checks pass. Tests are green. Linter is happy. Build succeeds. Done.

Except the test was checking the wrong thing. The linter doesn't know about placeholders. The build doesn't care that the function returns null on the path that mattered. The agent rationalized those away because they didn't trigger any red.

A passing validator is necessary. It is not sufficient.

This post is about the gap, and one pattern for closing it.

Two existing patterns

Two prior tools name the durable-autonomous-goal shape. They're worth knowing.

OpenAI Codex /goal. You give Codex an objective and a stopping condition; it works toward both across many turns. The contract is "complete X without stopping until Y." When Y matches, Codex stops. (, by Geoffrey Huntley. The simplest possible agentic loop:

while :; do cat PROMPT.md | claude-code; done

Ralph leans on validators — compile, test, static analysis — as the back-pressure that keeps each iteration honest. The famous Ralph rule:

DO NOT IMPLEMENT PLACEHOLDER OR SIMPLE IMPLEMENTATIONS. WE WANT FULL IMPLEMENTATIONS. DO IT OR I WILL YELL AT YOU.

That rule is in the prompt because Ralph noticed the same thing every long-running agent notices: validators alone don't catch placeholders. The model chases compilation reward, ships stubs that compile, declares done. So Ralph yells at it.

Both tools work. Both have the same hole: a passing validator can ship a stubbed implementation.

The judge as a separate gate

The pattern I want to talk about is putting a second agent — let's call it the judge — between "validator passes" and "goal is done." The judge is a fresh-context subagent, spawned new each time, that:

• Receives the contract's explicit Definition of Done as a checklist


• Reads the full diff and each modified file end-to-end via the Read tool


• Returns a structured verdict: approve or reject + an actionable fix-list


• Never sees the executing agent's reasoning, so it can't be argued into accepting weak work

The key part is the fresh context. The executing agent has spent the last hour persuading itself that the implementation is fine. The judge has spent zero minutes on it. When the judge reads function getThreshold() { return null; // TODO: pick value }, it doesn't have the executing agent's "yeah but the tests pass" history. It just sees a TODO. Reject.

A worked example

I shipped this pattern as a Claude Code plugin called / with every verdict line preserved.

Star goalkeeper on GitHub and try the judge gate on your next agent loop →

A question for you

I'm curious: have you shipped a feature where the tests passed, the lint was clean, the build succeeded — and the feature was still fundamentally broken because the agent stubbed something out and you didn't catch it in review?

How are you currently gating autonomous workflows in your stack? Manual review only? Validator-only with a strong anti-placeholder prompt? Something with a judge or critic step? I'd genuinely like to hear what's working and what's failing — drop a comment with the shape of your loop and where the gaps are.

goalkeeper is MIT. Shipped 2026-05-10. The CHANGELOG is the dogfood log.