I built TraceGate because my AI agent demo passed, but the traces told a different story

Most AI agent demos stop at the final answer.

Mine did too at first. I had a support agent that could answer a refund question, call a policy tool, and avoid a prompt-injection path. From the outside, it looked fine.

Then I asked a less comfortable question: if this agent broke in production, would I actually know what happened?

That question became TraceGate.

TraceGate is a release gate for AI agents built with OpenTelemetry and SigNoz. It runs agent scenarios, sends telemetry to SigNoz, then checks whether the run produced enough evidence to safely ship.

The architecture

The stack is:

Vite

React

TypeScript

Node.js

OpenTelemetry

SigNoz

OpenAI

YAML contracts

The flow looks like this:

Scenario -> Agent runner -> OpenTelemetry -> SigNoz -> TraceGate contract evaluator -> Pass or block

The Node runner executes the scenario. OpenTelemetry records spans, metrics, and logs. SigNoz receives the telemetry through OTLP. TraceGate reads the run result and evaluates it against the contract.

For the OpenTelemetry setup, I used the OTLP HTTP exporters:

const endpoint =

  process.env.OTEL_EXPORTER_OTLP_ENDPOINT ?? "http://localhost:4318";



const sdk = new NodeSDK({

  traceExporter: new OTLPTraceExporter({

    url: `${endpoint}/v1/traces`

  }),

  logRecordProcessor: new BatchLogRecordProcessor(

    new OTLPLogExporter({

      url: `${endpoint}/v1/logs`

    })

  ),

  metricReader: new PeriodicExportingMetricReader({

    exporter: new OTLPMetricExporter({

      url: `${endpoint}/v1/metrics`

    }),

    exportIntervalMillis: 1000

  })

});

In my local setup, SigNoz received spans for:

agent.run

llm.call

tool.ticket.lookup

tool.policy.search

tool.trace.lookup

This makes the product easier to test without asking someone to set up my whole local environment first.

What I learned

The main thing I learned is that observability is more useful when it is tied to a decision.

Before this project, I mostly treated observability as something I would open after a bug. With AI agents, that feels too late. If an agent takes a risky path, skips metadata, hides cost, or retries a flaky tool several times, I want to know during release validation.

The second thing I learned is that “the agent worked” is too broad. I now split it into two questions:

Did the agent produce the expected outcome?

Did the agent produce enough evidence to debug that outcome?

TraceGate focuses on the second question.

What I would build next

The next version should have a visual contract editor. Writing YAML is fine for a hackathon, but a team should be able to create a release gate by choosing required spans, attributes, budgets, and scenarios in the UI.

I would also add GitHub Actions support. The natural place for TraceGate is in CI, where it can block an agent release before merge.

The final improvement would be deeper SigNoz artifacts. Each contract should generate a matching dashboard, alert, and investigation prompt so the release gate and the debugging workflow stay connected.

Final thought

TraceGate started from one uncomfortable question: if my AI agent fails later, will I have the evidence to understand it?

SigNoz already gives teams a strong place to inspect telemetry. TraceGate adds a release workflow on top of that telemetry.

The project is small, but the idea feels useful: agents should not ship only because the final answer looked good. They should ship when their behavior is traceable enough to trust.