Why I Rejected an Event Bus for My Solo Agent Fleet: State Is Truth, Events Are Rumors

Originally published on hexisteme notes.

I run a personal fleet on one machine: a handful of small agent CLIs, a pile of cron jobs and LaunchAgents, and several MCP servers, some of them third-party tools I didn't write and can't modify. At some point I wanted a single inbox that could answer one question — what needs my attention right now? New outputs I haven't read yet. Decisions only a human can close. Dependencies that broke overnight without telling anyone.

The obvious architecture for that is an event bus. I sketched it, then rejected it on purpose and built a pull-based design instead. The reasoning generalizes past my particular setup, so here it is.

For a small, high-churn fleet you don't fully control, pull (scan on-disk state) beats push (an event bus). An event bus needs every producer — including third-party tools you can't modify — to emit; anything that doesn't is silently invisible. A poller that reads on-disk state instead — file mtimes, health checks, process liveness — is self-healing: it reflects reality even for components that never report anything, and a new tool shows up the moment it writes a file, with zero instrumentation.

The architecture I almost built

The event-bus version looks clean on a whiteboard. Every component emits events — job.finished, output.created, decision.pending — onto an append-only log. The inbox is just a reader of that log. Closing an item writes a close-event that advances whatever comes next. It's the textbook "single source of truth" pattern, and it's the first thing most people reach for.

I killed it for four reasons.

Reason 1: the instrumentation tax is a project killer

Push means every producer has to emit. In a fleet that grows most weeks, that's a standing tax on every new agent, every new cron line, and — the one that actually killed the design — every third-party MCP server I didn't write. You cannot add an emit() call to a server someone else built. So the moment I add a component and forget to instrument it, or simply can't, it goes invisible in the inbox.

That's the trap: the components you least control — third-party tools, things that crash early — are exactly the ones an event bus can't see. Push optimizes for the easy case (code you own) and silently fails the hard case (code you don't). An observability layer whose blind spots grow with the system is worse than no observability layer at all, because it still looks authoritative while quietly lying.

Reason 2: state is truth, events are rumors

An event is a claim that depends on the claimant surviving long enough to make it. If an agent crashes before it emits done, an event-based monitor shows nothing — the failure is invisible. State is the evidence left behind regardless of whether anyone remembered to report it: a stale output file, a log that stopped growing, a health check that fails, a process that just isn't there anymore.

# pull: derive status from evidence the component already leaves behind

status = {

    "alive":   process_is_running(job),               # ps / launchctl print

    "fresh":   newest_output_mtime(job) > expected,    # <project>/report/ glob

    "healthy": health_check(dependency),               # poll endpoint

}

# no emit() anywhere — a component that never reports is still seen

State is truth, events are rumors. An event only exists if something remembered to send it; state is evidence left behind regardless. A pull monitor reads that evidence and reflects a crash without needing the agent's cooperation — which is what makes it self-healing, converging on reality every cycle, while push is only ever as honest as its least-reliable emitter.

Reason 3: orchestration schizophrenia

A closed-loop bus, where "close this item" emits an event that advances the next step, quietly turns the inbox into a workflow engine. I already run an orchestration hub that decomposes goals into steps. Building a second one inside the monitor duplicates that responsibility and doubles the debugging surface — now a stuck task could be the hub's fault or the inbox's, and I have to check both. A monitor should report state, not drive it. Keeping those two jobs in two separate systems is what keeps either one debuggable.

Reason 4: the bus itself becomes an unreviewed dump

An append-only event log isn't free infrastructure. It accrues schema drift — the shape of output.created changes and old readers break — plus duplicate events, events nobody ever closes, and unbounded growth that eventually demands compaction. I'd be adding a database with none of a database's guarantees, and it would need its own monitoring just to know if it was healthy. Pull doesn't produce that artifact: there's nothing to compact, because there's nothing stored beyond the state that already exists on disk.

What pull looks like instead

The inbox ends up as a computed view over state that already exists — no new store, no emit() calls anywhere:

Priority is deterministic, not a model's guess: BLOCKED → STALE → NEW, where each is a hard fact — a failed health check, a file newer than its read-timestamp, a schedule past due. An LLM-generated priority number would be unfalsifiable and would drift over time; a deterministic trigger is reproducible and debuggable. I keep the model out of the ranking entirely.

Freshness is "unread," not "old"

Pull also fixed a metric I had wrong. My first instinct for "freshness" was elapsed time — this ran three days ago, so it's stale. But age isn't actually the problem; unread output is. A report that finished an hour ago and that I haven't opened yet demands more attention than one from last week that I already read.

So freshness is computed as a join: does an output exist whose mtime is newer than the last time I opened it? Clicking a card records a read-timestamp; unread items rise to the top, read ones sink. This join is cheap only because the design already scans state for everything else — freshness falls out of the same glob. In a push system it would have been yet another event to emit and reconcile.

The boundary that keeps it honest

Choosing pull forces a discipline I've come to think is the actual point: the monitor must never mutate fleet state. "Closing" an inbox item means acknowledge, and deep-link to the real place the work gets closed — it does not reach in and change a job, a ledger, or an agent's state directly. The moment a monitor starts writing back, it's an orchestrator again, and reasons 1 through 3 all come back. Read the world, link to the controls, never become the controls.

When push is actually right

None of this means event buses are a bad idea in general — it means they fit a different shape of problem. If you own every producer, can instrument all of them, and need high-throughput, low-latency fan-out, push is the right tool for that job. The pull argument wins specifically for a small, heterogeneous, high-churn fleet with components you don't fully control, where the dominant cost is instrumentation and the failure that hurts most is the silent one. The question isn't "push or pull" in the abstract — it's which cost is fatal for your system: throughput, or blind spots.

More notes at hexisteme.github.io/notes.