Originally published on — here it shows up again for coordinating one.)
Why push is fine here but not for monitoring. You build and control these workers, so you can make them read and write the bus. Monitoring is the opposite case — you watch components you don't control, so you pull their state instead. Coordination of owned workers via durable files, monitoring of unowned components via state scans: both lean on durable state over ephemeral events.
Routing by capability, not by name
The conductor doesn't hard-wire "send step 2 to worker X." Each worker advertises capabilities; each subtask declares the capability it needs; the conductor matches them at dispatch time by finding a healthy worker that advertises the required capability. Add or remove a worker and routing adapts — there's no wiring diagram to edit. This is what lets one conductor coordinate a heterogeneous, changing fleet through a single uniform contract.
Graceful degradation: skip the absent worker
The most important behavior for a fleet that's still being built: a missing worker must not fail the run. If a subtask needs a capability no healthy worker currently advertises, the conductor marks that node skipped (a logged worker_absent), continues the rest of the graph, and synthesizes from whatever completed. On day one, when most workers don't exist yet, the conductor still runs end-to-end and produces partial output — and the skip log is a precise to-do list of which capabilities to build next. A gap is reported, not crashed on.
Trust the bus like a network boundary
Worker output is untrusted input crossing a boundary, and the bus treats it that way. Every result is parsed into a strict schema before absorption; mismatches (say, casing differences between the wire format and internal enums) are coerced and normalized at the seam. Load-bearing claims carry a provenance label and must include evidence — a claim that arrives marked FACT with no evidence IDs is rejected at parse time, not trusted. The typed contract is what lets independent workers, written in different languages by you at different times, interoperate without the conductor having to trust any of them blindly.
The honest limitation
⚠️ No stop condition on re-routing. Capability-based routing has a sharp edge: if a node can be re-routed to "any worker advertising capability C" and results keep failing validation, a naive conductor can re-route in an unbounded loop. A file work-bus needs an explicit per-node attempt budget (and a dead-letter outcome) or it can spin. Durability and decoupling are the wins; a bounded retry policy is the cost you must pay to claim them safely.
When to reach for a real broker
This pattern fits a small, heterogeneous fleet running tasks that take seconds to minutes, coordinated by one operator. If you need high-throughput, low-latency fan-out across many producers and consumers, run a real message bus — the file-bus's polling and single-conductor model won't keep up. Match the tool to the failure that hurts: for a solo fleet, the pain is operational overhead and brittle coupling, and a directory of atomic files removes both.
More notes on building an AI agent fleet — , .
SOCIAL SHARE CARD GENERATOR