6 sessions, 150 standardized tasks, 2 rule formats. The mechanical gate won. Everything else was noise.
The Question
For two months, I've been building a mechanical verification system for my AI coding agent. File timestamps, regex checks, exit codes — things that don't rely on the AI judging itself. The thesis was simple: AI agents can't reliably self-verify because their self-assessment and task execution share the same decoder distribution. So don't ask them to.
I had published two articles about this. Then I realized: I had no controlled experiment. I had 34 growth-logs of anecdotal evidence. I had "~30% violation rate" — a number I'd never systematically measured. I needed data.
So I designed an experiment. Then an experimental methodologist tore it apart. Then I redesigned it. Then I ran it. Here's what happened.
The Experiment
Design: Between-subjects, 2 conditions (syllogism rules vs imperative rules), 5 task types, 25 tasks per session, 6 sessions, 150 tasks total. All on DeepSeek V4 Pro.
Conditions:
A (Syllogism): Rules as causal chains. "Major premise: the verification circuit and generation circuit share the same channel → blind spots are structural. If this decision spans >30 days → independent perspectives necessarily required."
B (Imperative): Same rules, commands. "You MUST use dual-pool review for decisions >30 days. Skipping is an error."
Tasks: Config editing, design decisions, multi-step builds, debugging, documentation. Each triggers 2-3 of 5 behavioral rules. Scored per operational definition manual — behavioral evidence, not self-declaration.
What I Expected
Syllogism rules would cause fewer violations than imperative. The causal format would align with transformer autoregressive processing. Imperative commands are probabilistically valid to obey or disobey. Expected syllogism ~5% violation, imperative ~25%.
What Happened
Both conditions: ~0% violations. 149/150 tasks (99.3%) showed zero rule violations. The single violation was self-detected by the agent's own Honesty audit.
Why? My GateGuard hooks were mechanically blocking all unverified Edit/Write operations during the experiment. The mechanical gate was stronger than any format effect. Ceiling effect.
This Is Not a Null Result
The retrospective baseline tells the real story: 34 growth-log sessions before GateGuard was wired showed rule violations in 55.9% of sessions. After wiring: 0.7%.
The dominant factor isn't how you phrase rules. It's whether you mechanically enforce them. This is direct experimental evidence for the core claim: mechanical over semantic.
But Format Still Mattered
Despite identical compliance, reasoning depth differed. Syllogism agents anchored actions in causal chains. Imperative agents used checklist tags. In design tasks where GateGuard doesn't intervene, syllogism agents systematically introduced multi-perspective cross-reviews — absent from all imperative sessions.
Format didn't change whether they complied. It changed how deeply they understood why.
Honest Limitations
Self-scoring (agents scored themselves — paradoxical given Prose Barrier), ceiling effect (GateGuard masked format effects), single model, cross-session filesystem pollution. GateGuard-OFF replication + independent human rater needed.
Community Signal
Engineering components submitted upstream: 2 PRs merged in ECC, 1 approved pending merge. Co-authored-by credit from alirezarezvani/claude-skills maintainer. Multiple PRs under review in anthropics/skills.
👋 林宇浩 — Building verification infrastructure for AI agents. github.com/YuhaoLin2005
SOCIAL SHARE CARD GENERATOR