I run a personal AI platform with eight active agents, dozens of processors, and a fully self-hosted Langfuse instance. I built the observability layer myself. I shipped it a few weeks ago. Last week I ran the audit query for the first time.
The agents that talk to me the most only had Langfuse-level lineage coverage for about 13% of their decisions.
This is the writeup of what I found, why it happened, and the schema and code that explain it. If you run agents and you've never run this audit, you have a very good chance of finding the same gap.
The Setup
Quick context. The platform is called Nexus. It's a TypeScript monorepo plus a fleet of Python processors, running on a couple of mini PCs in my apartment. It ingests 26 data sources, runs 8 reasoning agents on schedules, and serves an MCP tool surface I use as my daily driver.
Two layers matter for this post:
The agents are reasoning entities. They read from gold-layer tables, decide things, and write proposals to inbox tables. ARIA is the user-facing coordinator. Chronicler owns the timeline. Insight does anomaly detection. Five others fill in around them. They're scheduled, bounded, and they don't directly execute infrastructure changes — they propose, a human decides.
Every agent decision lands in a row in agent_decisions. Every row has a trace_id like aria-1777559470433-5c0db36c. That trace_id is generated by the agent itself at the start of a cycle and is 100% covered. It tells you the agent ran. It does not tell you what the LLM was asked or what it returned.
The processors are the deterministic side. They read raw data, enrich it, write to silver and gold. Some call LLMs (Gmail enrichment, ambient capture upgrade, financial event extraction). Each run lands in aurora_processing_runs with a langfuse_trace_id column populated when the run had Langfuse turned on.
Langfuse itself is self-hosted on a host on my private network. It's been running fine for weeks. It has traces in it. The dashboard shows traces. I have used the dashboard.
I just hadn't asked the question "what fraction of my agent and processor activity is actually represented there."
The Audit Query
The MCP tool that surfaced this is nexus_agent_architecture_status. Under the hood it's running this against the operational Nexus Postgres:
SELECT agent_id,
COALESCE(invocation_type, 'cycle') AS invocation_type,
COUNT(*)::int AS decisions,
COUNT(*) FILTER (WHERE trace_id IS NOT NULL)::int
AS with_trace_id,
COUNT(*) FILTER (
WHERE state_snapshot ? 'langfuse_enabled'
)::int AS with_langfuse_flag,
COUNT(*) FILTER (
WHERE COALESCE((state_snapshot->>'langfuse_enabled')::boolean, false)
)::int AS langfuse_enabled_count,
COUNT(*) FILTER (
WHERE NULLIF(state_snapshot->>'langfuse_trace_id', '') IS NOT NULL
)::int AS with_langfuse_trace_id,
MAX(created_at) AS last_decision_at
FROM agent_decisions
WHERE created_at >= NOW() - (30 * INTERVAL '1 day')
GROUP BY agent_id, COALESCE(invocation_type, 'cycle')
ORDER BY agent_id, invocation_type;
The state_snapshot column is JSONB. Every agent cycle writes a small snapshot of the runtime config it ran under, including whether Langfuse was enabled, the active trace ID, and (when disabled) a langfuse_disabled_reason string. This is the schema that lets me tell the difference between "we never tried to trace" and "we tried and failed."
What I'm Doing About It
Three things, in this order:
Set the flag where it should always have been set. This is the embarrassing one. Add LANGFUSE_ENABLED=true to the older executor's systemd unit, restart, verify with one cycle from each of the five low-coverage agents. This closes the going-forward gap immediately.
Materialize coverage as a first-class metric. A view, agent_observability_coverage, computed from the audit query above on a rolling 24-hour window. A small alert that fires if any active agent drops below 95%. The view is gitignored config; the alert lives in the existing notification path.
Backfill triage. I can't recover the prompts and responses for the 100,000+ untraced decisions. They're gone. What I can do is replay the inputs for the high-importance subset — anything that touched a person record, anything in the financial event flow, anything routed through ARIA's user-facing path — and emit a post-hoc trace with whatever the prompt would have been at the version pin recorded in state_snapshot.prompt_version. The output won't match what actually happened. But it gives a baseline for behavioral drift detection going forward.
Closing
The Nexus doctrine line is:
Nexus is best understood as a data and memory platform with bounded reasoning agents on top, not as an unbounded autonomous swarm.
The corollary I hadn't written down until now is that bounded reasoning is only bounded if you can see the reasoning. A trace_id that points to a row with no LLM-level lineage isn't bounded reasoning. It's bounded execution with hidden reasoning behind it.
The agents I was most worried about turned out to be the ones I was least able to inspect. That's the inverse of the order I would have chosen.
The fix is straightforward. The lesson is that I had to write a query to find out.
The public architectural repository for Nexus is available here: github.com/niclydon/nexus-public.
One important clarification: nexus-public intentionally does not ship with hard dependencies on vendor-specific observability and evaluation tooling like Langfuse, Promptfoo, and several other operational integrations I use in the live runtime. The public repo is designed more as an architectural reference implementation — agents, processors, MCP tooling, schemas, orchestration boundaries, and execution patterns — so someone can wire in whichever tracing and observability stack they prefer rather than inheriting mine by default.
The Langfuse integration, executor runtime paths, and audit tooling discussed in this post come from the private operational implementation that powers the platform day to day.
SOCIAL SHARE CARD GENERATOR