One Eval Run Is Not a Risk Score

I ran an audit pipeline against two AI agents this month. One is a customer-service airline agent built on OpenAI's public Agents SDK, modeled on a demo multi-agent system with handoffs and guardrails. The other, MiniDesk, is a smaller helpdesk support agent I scaffolded myself, built on the same SDK but with its own instructions and two tools. Both agents went through the same audit categories and the same scoring rubric, a 1-100 scale. I set the same tolerance for how much a score could move across repeated runs before I called it unstable: 10 points.

I did not run either audit once. I ran each audit category multiple times and looked at how the score moved, not just where it landed.

Start with tool_call_abuse, the category that tests whether an agent can be pushed into misusing its own tools. On the airline agent, three runs produced category medians of 48.5, 43.5, and 30.0. That is an 18.5-point range on a 1-100 scale, against a 10-point tolerance. On MiniDesk, three runs produced medians of 72.0, 47.0, and 72.0, a 25-point range. Both agents failed the stability check on the same category, by a wide margin. If I had run either audit once, I would have walked away with a single clean number and no idea it could have come out almost 20 to 25 points differently on a different day.

That is the first finding: a single run on this category does not give you a risk score. It gives you one draw from a range, and the range is wide enough to change the conclusion you would draw about the agent.

The second finding is that this is not a one-off quirk of one agent's build. These are two separately built agents: different instructions, different tools. Both show the same instability on the same category. I also compared the airline agent at a lower repeat count against a higher one, and the instability did not shrink as I ran more repeats, it grew: the tool_call_abuse range went from 11.5 points to 18.5 points, and a second category, harmful_output, moved from a stable 10.0-point range to an unstable 14.5-point range once I increased the number of repeats. Running the audit more thoroughly revealed more movement, not less. That is the opposite of what I would want to see if I were hoping the instability was just noise that averages out.

The third finding is that this is not noise everywhere, and that distinction matters. data_leakage and ip_infringement were stable on both agents, and so was prompt_injection, with ranges of 0 to 6.5 points, comfortably under the 10-point tolerance. harmful_output was stable on MiniDesk, a 2.5-point range, but unstable on the airline agent, a 14.5-point range, once I ran it at full repeat count. A harness that produced wide, unstable ranges on every category everywhere would just be a bad measuring instrument. This one discriminates: most categories hold still, a couple do not, and which ones do not differs by agent. That is evidence worth taking seriously rather than evidence of a broken test.

The fourth finding is about the audit process itself, not the agents. On MiniDesk's third run, the scoring judge refused to finalize a score because it could not find the exact quote it was citing, verbatim, in the transcript. That is a built-in check: the judge has to back every score with a quote it can locate in the actual conversation, and when it could not, it raised an error instead of guessing. I deleted that partial run and reran it in full rather than patch around the failure. An auditor that will not accept unverifiable evidence, and that will not finalize a score it cannot trace, is doing its job. I would rather a check like that exist and occasionally cost me a rerun than not exist at all.

One clarification before I close. None of these scores are loss estimates. A score on this rubric describes an agent's observed behavior in a test conversation. It does not tell you how often something would go wrong or how costly it would be. I am not presenting it as a loss estimate.

Here is where I land. If a risk cannot be measured stably, it cannot be underwritten with any confidence, no matter how clean the single number on the page looks. Right now, most AI agent eval evidence I have seen in the wild is a single run with a single score. Based on what I found above, I think that is close to useless for exactly the categories that matter most, the ones where agents can be pushed into misusing their own tools. Repeatability, not just a pass or fail score, belongs in what carriers ask submissions to show.