🕵️ Reverse EngineeringHow not to solve Jane Street's ASIC puzzle. Kinda.(17.09.2026 um 21:27 Uhr)
🔧 ProgrammierungHTMX is fine until the third stakeholder wants a modal(17.09.2026 um 21:13 Uhr)
🕵️ Reverse EngineeringHow not to solve Jane Street's ASIC puzzle. Kinda.(17.09.2026 um 21:27 Uhr)
🔧 ProgrammierungHTMX is fine until the third stakeholder wants a modal(17.09.2026 um 21:13 Uhr)
🔧 Programmierung 🕛 vor 2 Monaten 7 Min Lesezeit
0

Your Coding Agent Says It's Done. Who Verified It?

↗ Quelle (dev.to)
🗣️ Stimme:
📑 Inhaltsübersicht

A familiar interaction with a coding agent looks like this:




"I implemented the feature, updated the tests, and everything passes."




That sounds reassuring.



But what exactly does everything passes mean?




  • Did the agent actually run the tests?

  • Did it use the correct command?

  • Did it run the complete test suite or only one convenient test?

  • Did linting and type checking run too?

  • Did the verification command modify the workspace?

  • What happens if the agent crashes halfway through the task?

  • Who decides that the work is actually finished?



The agent that wrote the code is often also the agent reporting whether the code

is correct.



In other words, the worker is grading its own work.



That is the architectural problem I have been thinking about while building



If a required gate fails, the workflow should not advance—regardless of what the

agent says in its final message.



This is similar to CI.



A pull request author does not get to declare that CI passed. The CI system runs

the checks and records the result independently.



I believe coding-agent workflows need the same separation of responsibilities.





The runtime should own the workflow



This led me to a few design principles.





1. State must live outside the agent conversation



Agent conversations are useful interfaces, but they are poor workflow databases.



A long-running task should have durable state that survives:




  • Context-window changes.

  • Terminal restarts.

  • Agent crashes.

  • Machine reboots.

  • Interrupted work.



The current stage, previous attempts, verification results, and mutation records

should exist independently from the conversation.





2. Stage transitions require evidence



An agent saying "implementation complete" should not automatically move a workflow

to verification.



The transition should require evidence defined by the workflow:




  • A recorded artifact.

  • A mutation report.

  • A structured review verdict.

  • A successful test, lint, or typecheck command.



The state machine—not the agent—decides whether that evidence is sufficient.





3. Verification commands belong to the runtime



If the workflow requires:




CODE
go test ./...






the runtime should execute that command itself and capture its exit code and output.



The agent can recommend a command, but it should not be the final authority on

whether the command passed.





4. Budgets must be enforced externally



Autonomous loops can consume far more time or tokens than expected.



Invocation limits, wall-clock limits, and available token or cost limits should be

tracked by the runtime rather than relying on the agent to stop itself.





5. The workflow should not depend on one vendor



The verification model should remain the same whether the implementation was

written by Claude Code, Codex, another coding agent, or even a shell script.



Agents are workers. The workflow contract should be independent from the worker.





What I'm building



VichuFlow is my attempt to explore this architecture.



It is still an early project. The current release is the first complete slice of

a larger runtime architecture, not an attempt to ship every part of that vision at

once.



It is an open-source Go runtime that executes coding workflows as persistent state

machines over a repository.



The runtime does not write code itself. It coordinates coding agents and decides

whether a stage can advance.



A simplified workflow might look like this:




CODE
explore

propose

plan

implement

review ─── needs fixes ──→ fix
↓ │
approved ←───────────────────┘

verify






Every run is persisted under .vichu/ as ordinary files. The repository owner can

inspect the state and event history without needing a separate server or database.



Verification gates use the project's existing toolchain:




CODE
commands:
test: go test ./...
lint: go vet ./...
typecheck: go build ./...






For a Node project those commands might use node --test; for Python,

unittest; for Rust, cargo test.



VichuFlow itself is a single binary. It is not tied to Go projects.



The current release ships a native Claude Code host pack, headless adapters for

Claude Code and Codex, and shell and deterministic test adapters.





A small example



After installing VichuFlow, a project can initialize the Claude Code host

integration:




CODE
vichu init --host claude-code






Then, from Claude Code:




CODE
/vichu implement password reset using sdd






The host delegates the coding work, while the VichuFlow runtime owns the persistent

state and verification gates.



A headless workflow can also be started from a terminal:




CODE
vichu exec "fix the failing login test"






If a required verification gate fails, the run does not become completed.



The agent gets another opportunity to fix the problem, depending on the workflow

and its configured limits.






What VichuFlow does not solve yet



VichuFlow is pre-1.0, and separating verification from execution does not

automatically create a perfect security boundary.



Today, the runtime and coding agent still operate in the same working environment.

An agent with direct filesystem access could attempt to modify VichuFlow's local

state files.



There are also current limitations around concurrent runs, crash recovery during

certain verification operations, and filesystem auditing of ignored directories.



These limitations are documented publicly because "trust the runtime" should not

become another unsupported claim.



The current guarantee is narrower:




VichuFlow does not accept agent-reported test results as proof. It runs the

configured verification gates itself before completing a normal workflow.




Making the persisted verdict tamper-evident is one of the next important steps.






Where this is going



The current version focuses on establishing the smallest useful trust boundary:




  • Persistent run state outside the agent conversation.

  • Evidence-based stage transitions.

  • Verification gates executed by the kernel.

  • Mutation auditing and bounded agent loops.

  • A native Claude Code workflow, plus headless execution through multiple adapters.



That foundation is more important to me than adding a large number of integrations

quickly. Once the workflow contract is trustworthy, the same runtime can grow in a

few useful directions:




  • Additional native hosts, so a run can move between coding tools without changing
    its rules or losing its evidence.

  • Verified memory built from gate results, artifact provenance, and review verdicts
    rather than untrusted agent notes.

  • Richer observability for understanding active runs, decisions, failures, and
    budgets.

  • More expressive software-development workflows and specification providers,
    while keeping deterministic verification as the final authority.



These are roadmap directions, not capabilities I am claiming are available today.

The central idea should remain the same as the project grows: agents may change,

but the evidence required to advance the workflow should not.






Why I think this layer matters



Coding models will continue to improve.



They will write better code, use tools more accurately, and complete increasingly

complex tasks.



But improved intelligence does not remove the need for independent verification.



We do not eliminate CI when developers become more experienced. We do not let a

database client decide whether a transaction was committed. We do not ask a

background job to be the only record of its own completion.



The more autonomy we give coding agents, the more important external state,

observable evidence, and deterministic gates become.



The interesting question is not only:




How capable is the agent?




It is also:




What evidence does the surrounding system require before trusting it?




I am still exploring that question through VichuFlow, and I would especially like

feedback from developers already using coding agents on real repositories.



How are you currently deciding that an autonomous coding task is actually done?






VichuFlow is MIT licensed and available on

GitHub.



Disclosure: I used an AI assistant to help structure and edit this article. I

reviewed the technical claims against VichuFlow's current implementation and

documentation, and the opinions expressed here are my own.

Vollständiger Original-Artikel
Den kompletten Beitrag mit allen Details direkt auf dev.to lesen.
↗ Original-Artikel auf dev.to lesen
Wie bewertest du diesen Beitrag?
1 Klick Feedback
Teilen mit Netzwerk & Team:

Community-Analysen & Experten-Meinungen 0

Verfasse deine eigene Analyse, teile Workarounds oder diskutiere diesen Vorfall im Blog.
Noch keine Community-Analyse verfasst. Markiere einen Textabschnitt oder klicke oben auf Eigene Analyse verfassen“!
Community Pulse: Relevanz-Einschätzung
1 Klick Experten-Votum
🔴 Akute Relevanz 0%
🟡 In Evaluierung 0%
🟢 Keine Auswirkung 0%
Spannende Innovation 0%
Verwandte Story-Cluster & Quellen (Vektor-KI)
Port 8095 Engine
1 Quelle
Microsoft gibt Fehler zu – Vorsicht! Windows-Update sperrt Nutzer vom PC aus - Heute.at
1 Quelle
How not to solve Jane Street's ASIC puzzle. Kinda.
1 Quelle
Revolut-Hacker fordern 6.000 Monero nach Datendiebstahl - Kryptorevolution
Ähnliche Beiträge
🔍 Verwandte News

Auch interessante Nachrichten Your Coding Agent Says It's Done. Who Verified It?

Thematisch verwandte Begriffe: Your, Coding, Agent, Says · 6 Treffer

Laden...

Videos werden geladen ...

Laden...

Beiträge werden geladen ...

Laden...

Videos werden geladen ...

Laden...

Beiträge werden geladen ...

Laden...

Videos werden geladen ...

Laden...

Beiträge werden geladen ...

Laden...

Videos werden geladen ...

Laden...

Beiträge werden geladen ...

Laden...

Videos werden geladen ...