Agent observability gets weirdly polite at the exact moment it should get nosy. It records the model call, stores the prompt, counts tokens, and then loses interest right when the agent starts touching software.
That makes the trace look clean and the incident feel impossible.
The failure rarely lives inside the LLM span. matters even more with agents. Logs answer the question a developer guessed in advance. Traces let the team follow the shape of the system after the weird thing has already happened. Agent systems add a worse version of the same problem because the execution path is partly selected at runtime.
A token count will not explain why the refund tool wrote to the wrong account. A prompt transcript will not explain why a sub-agent retried the same API call through two paths. A model latency chart will not explain why a background reconciliation job created the user-visible failure.
The agent caused work. The trace has to follow the work.
.
The phrase reads like metadata, but it decides whether the team has a trace or a receipt.
The trace ID describes one distributed execution. The conversation ID describes the user-facing unit of work across traces, services, and turns. A support agent classifies a request, fetches account state, hands off to a billing agent, and waits for a queue worker before it sends the final response. Those steps can land in different traces. The customer experienced one conversation. The system created a pile of spans.
Without gen_ai.conversation.id, the pile stays a pile.
There is one easy mistake here: do not fake the identifier at the leaf. The OpenTelemetry GenAI agent-span conventions say gen_ai.conversation.id should be populated only when a real identifier is readily available, and should not fall back to a new UUID, trace ID, or hash of request content . Conversation IDs add the user-facing thread across those technical boundaries. Trace context keeps parent-child relationships intact. Conversation context lets the team ask, "show me everything this agent conversation caused." Both have to survive the tool call.
Span ancestry is not paperwork
LangSmith's OpenTelemetry docs include a nasty little failure mode: a child span whose parent never reaches LangSmith can be accepted with a 200, buffered, and then dropped later if the parent never arrives is the right operating model. The app team cannot sprinkle tracing on the agent wrapper and call it done. The platform has to make propagation easy, collectors safe, sampling legible, and missing-span failures visible.
Agent names decide accountability
Multi-agent systems make the naming problem uglier.
Honeycomb's docs warn that each agent needs a unique gen_ai.agent.name, and that sub-agents should not inherit the parent agent's name because duplicate or missing names make investigation impossible . Handoffs are not vibes. They are ownership transfers. The receiving agent needs state. The sending agent needs a receipt. The trace needs to show who acted, who delegated, who observed the result, and which name owns the failure.
Agent names are also release artifacts. A renamed agent can break dashboards. A reused name can hide a new implementation behind an old label. A temporary experiment name can leak into live traffic and turn a week of traces into archeology. Name the agent like a service. Version the behavior somewhere else.
Prompts are data with a governance boundary
A useful agent trace wants the thing security teams hate storing: prompts, responses, tool arguments, tool outputs, retrieved context, and evaluation notes.
That tension will not go away. It has to be designed.
Honeycomb recommends storing full prompts, chat history, and completions as span events because they may be large or contain sensitive data, while the docs call out that an OpenTelemetry Collector can filter or redact them before ingestion .
That collector draws the line between operational evidence and data sprawl.
. Good. Trace context is not harmless just because it is metadata.
A public request can carry lies. Internal propagation can carry evidence. The gateway has to know the difference.
The trace should feed control
The point of agent observability is not a prettier incident screenshot.
Candidly's LangSmith writeup is useful because it shows traces turning into a control surface. Their Cait financial-planning agent moved from post-hoc conversation evaluation toward turn-level state inference. The labeling pipeline reached 92.3% agreement with a human-labeled LangSmith dataset, and trace-derived features predicted resolved versus abandoned conversations with 0.90 AUC . The trace cannot stop at telemetry collection. If a live conversation exposes a broken handoff, the fix should not be a screenshot in Slack. It should become a test case and a control.
The best traces become software.
Instrument the boring path first
The first pass does not need a grand observability platform. It needs a clean propagation contract.
Start with the product conversation or session boundary. Create or retrieve the real conversation ID there. Pass it through the agent runtime. Attach it to the root invoke_agent span. Carry it into LLM calls, tool execution, service calls, queues, and database work. Add gen_ai.agent.name at every agent boundary and use stable names for sub-agents. Set gen_ai.operation.name with the closest standard value instead of inventing labels for basic operations.
Then break the system on purpose.
Force a tool timeout. Drop a parent span in a staging environment. Send a request through a queue. Trigger a sub-agent handoff. Verify the timeline still tells the story. Check that the collector redacts the fields it claims to redact. Confirm public trace headers get stripped at the edge. Find the first span where the conversation ID disappears and fix that before buying another dashboard.
This work feels unglamorous because it is. Also because it is the part that decides whether the next incident review has evidence.
AI agent observability follows what the agent caused. The conversation ID is the thread. Pull it through the stack, or accept that live systems will keep handing back neat little model traces while the actual failure happens somewhere else.
SOCIAL SHARE CARD GENERATOR