How Do You Measure AI Agent Reliability?

Your agent passed the eval, so you shipped. The next day a user sends almost the same input and it fails. Nothing changed. You just learned that "it passed" was one sample of a distribution, and you shipped on a coin flip that landed heads.

Part 1 defined the bar. Part 2 built the eval set that measures it. Part 3 is the uncomfortable part: an agent will not give the same answer twice, so a single green run tells you almost nothing about AI agent reliability.

One run measures luck, not reliability

A deterministic function passes a test or it does not, forever. An agent is a random variable. Run the same case ten times and you might see nine passes and one confident wrong answer. A single run showing the pass is not evidence the agent is good, it is evidence you got lucky once. Reliability is not a checkmark. It is a distribution, and you have to measure its shape.

Measure the distribution, not one draw

• Run every case many times. Report a pass rate, not pass or fail. If a case passes 8 of 10 runs, that is an 80 percent case, and you should know before a user does. (This is the idea behind ), harmless variance stops counting as failure and the failures that matter become visible.

Reduce variance on purpose

Once you can see the distribution, narrow it where it counts. Lower the temperature on calls that must be consistent. Constrain the output format so there are fewer ways to go wrong. Add a guardrail that catches the disqualifying failure before it reaches the user. None of this makes an agent deterministic, and chasing determinism is the wrong goal. The goal is a distribution tight enough, with a floor high enough, that you can trust it.

So the release gate is not "the eval passed." It is "the pass rate over many runs clears the bar, and the worst case is bounded." A green checkmark from a single run is not a decision, just an anecdote.

Part 4 turns to the other constraint that decides whether an agent ships: what all of this costs, in latency and dollars, at real volume.

Run it once and you measured luck. Reliability is the shape of the distribution, not a single pass.