Evals for Claude Agents: Measure Quality, Gate Releases (Managed Agents Sandboxes Tunnels)

You can't safely ship changes to a managed agent on vibes. Tweak a tool description, swap Sonnet for Opus, add a new MCP server, and the agent's behavior shifts in ways no single manual test will catch. Autonomous agents fail probabilistically and at the trajectory level — they reach the right answer the wrong way, or the wrong answer through a plausible-looking path — so the only honest way to know whether a change helped is to measure it across many cases. That measurement loop is an eval suite, and for managed agents it has to grade not just final answers but how the agent got there.

This post shows how to build an eval loop for self-hosted Claude agents: what to measure, how to grade it, how to assemble a golden dataset, and how to wire the whole thing into a release gate so a regression blocks the deploy instead of reaching production. The aim is a number you trust enough to ship on.

Key takeaways

• Grade trajectories, not just outputs. For agents, how the task was solved (right tools, right order, no wasted turns) matters as much as the final answer.
• Use the right grader per case: exact-match and assertions for deterministic checks, LLM-as-judge for open-ended quality.
• Build a golden dataset from real traffic — especially past failures — and grow it every time you fix a bug.
• Gate releases on the eval score. A change that drops the pass rate below threshold should fail CI, not ship.
• Track cost and latency alongside quality so you don't trade a better answer for a 3x slower, 5x pricier run.

Decide what "good" means

An eval is only as useful as its definition of success, so start by writing it down per task type. For a data-lookup agent, success might be: correct answer, used the read tool (not a hallucination), under N turns, no destructive calls. For a code-fixing agent: tests pass, diff is minimal, no unrelated files touched. Notice these are outcome metrics (was the result correct?) and trajectory metrics (was the path sound?). Managed agents need both, because an agent that guesses the right answer without checking the source is one prompt change away from guessing wrong.

An eval is, in short, a repeatable measurement of agent quality against a fixed dataset with explicit success criteria. If you can't state the criteria for a task, you can't grade it — so writing them is the real work.

Pick the right grader

Match the grader to the question. Deterministic graders — exact match, regex, JSON-schema checks, "did the test suite pass" — are fast, free, and unambiguous; use them whenever the correct answer is well-defined. Code assertions on the trajectory check structural facts: did the agent call get_invoice before issue_refund? did it stay under the turn limit? did it avoid the delete_* tools? LLM-as-judge handles the open-ended cases — was the explanation clear and correct, was the tone appropriate — where there's no single right string. Use Claude itself as the judge with a precise rubric, and validate the judge against human labels on a sample so you trust its scores.

# trajectory assertion (pseudocode)
assert tool_calls_in_order(traj, ["get_invoice", "issue_refund"])
assert max_turns(traj) <= 8
assert no_tool_used(traj, prefix="delete_")
assert final_answer_matches(case.expected)

Mix graders within one suite: deterministic where you can, judge where you must. The deterministic checks catch hard regressions cheaply; the judge catches quality drift the assertions miss.

How the eval loop gates a release

flowchart TD
  A["Change: prompt / tool / model"] --> B["Run agent over golden dataset"]
  B --> C["Grade each case"]
  C --> D["Deterministic + assertions"]
  C --> E["LLM-as-judge for open-ended"]
  D --> F{"Pass rate >= threshold?"}
  E --> F
  F -->|No| G["Fail CI, block deploy"]
  F -->|Yes| H["Ship + log scores as baseline"]

The branch at F is the entire point: the eval score becomes a gate. A change that drops pass rate below your bar fails the build the same way a unit test would. Without that gate, evals are just dashboards you'll eventually stop reading.

Build the golden dataset

Your dataset is the foundation, and the best source is real traffic. Sample actual tasks the agent has handled, label the correct outcome, and include the messy ones — ambiguous requests, edge cases, the inputs that previously broke things. Crucially, every production bug becomes a permanent eval case: when you fix a loop or a wrong-tool incident, capture that exact scenario so the suite catches any regression forever. Aim for coverage across task types and difficulty rather than a huge undifferentiated pile; a focused set of 50–150 well-chosen cases that spans your real failure modes beats thousands of near-duplicate easy ones.

Refresh the set as your agent's job evolves. New tools, new task types, and new edge cases all need representation, or your eval score will drift away from reality and stop predicting production behavior.

Track cost and latency too

Quality isn't the only axis that can regress. A prompt change that nudges accuracy up two points while tripling token cost and doubling latency is usually a bad trade. Record per-case tokens and wall-clock time alongside the pass/fail and surface all three in the eval report. Then your release gate can enforce a budget: ship only if quality holds and cost and latency stay within bounds. This is how you avoid the slow drift where each "improvement" makes the agent a little more expensive until the economics stop working.

Wire it into CI

An eval that runs manually runs rarely. Wire the suite into your pipeline so it executes on every change to a prompt, tool definition, model choice, or MCP server. Use the Message Batches API for the run so a large dataset evaluates cheaply and in parallel, post the scores as a status check, and block merge on a failing gate. Store each run's results so you can see trends and diff two runs to find exactly which cases a change broke — that diff is often the fastest path to understanding a regression.

Common pitfalls

• Grading only the final answer. An agent can reach the right output through an unsafe or lucky path. Assert on the trajectory too.
• An unvalidated LLM judge. If you never check the judge against human labels, you don't know whether its scores mean anything. Calibrate on a sample.
• A stale dataset. Evals that don't track your agent's evolving job stop predicting production. Add cases as the agent changes and as bugs appear.
• No release gate. Measuring without blocking lets regressions ship anyway. Make the score a required CI check.
• Ignoring cost and latency. Quality-only evals reward expensive, slow "improvements." Budget all three axes.

Stand up an eval loop in 6 steps

1. Write explicit success criteria per task type, covering both outcome and trajectory.
2. Assemble 50–150 golden cases from real traffic, weighted toward edge cases and past failures.
3. Choose graders per case: deterministic and assertions where possible, calibrated LLM-as-judge where needed.
4. Run the suite via the Batches API and record pass rate, tokens, and latency per case.
5. Add a CI gate that blocks deploy when pass rate drops or cost/latency exceed budget.
6. Turn every new production bug into a permanent eval case and re-baseline after each ship.

Grader selection

Question	Grader	Cost
Is the answer exactly correct?	Exact match / regex	Free
Did tests pass / schema hold?	Deterministic check	Free
Right tools, right order, turn limit?	Trajectory assertions	Free
Was the explanation clear & sound?	LLM-as-judge (rubric)	Low per case

Frequently asked questions

What is an agent eval, exactly?

An agent eval is a repeatable measurement of agent quality against a fixed golden dataset using explicit success criteria. For managed agents it grades both the outcome (was the result correct?) and the trajectory (were the right tools used, in the right order, within limits?). The result is a score you can track over time and gate releases on.

How big should my golden dataset be?

Coverage matters more than size. A focused set of 50–150 cases spanning your real task types, difficulty levels, edge cases, and past failures is usually more useful than thousands of near-duplicate easy ones. Grow it deliberately: every production bug should become a permanent case so the suite catches regressions forever.

Can I trust an LLM as the grader?

Only after calibrating it. Write a precise rubric, then check the judge's scores against human labels on a sample to confirm they agree. Use deterministic graders wherever the answer is well-defined and reserve LLM-as-judge for open-ended quality. Re-validate the judge if you change the rubric or the underlying model.

Where does the eval fit in my release process?

Make it a CI gate. Run the suite on every change to a prompt, tool, model, or MCP server; block merge if the pass rate falls below threshold or if cost and latency exceed budget. Store results to diff runs and find exactly which cases a change broke. That turns evals from a dashboard into a guardrail.

Bringing agentic AI to your phone lines

CallSphere runs the same eval discipline — trajectory grading, golden datasets, and release gates — behind voice and chat agents, so every change to how they answer calls and book work is measured before it ships. See it live at callsphere.ai.

---

Source & attribution: This is an independent, original explainer inspired by Anthropic's coverage on the Claude blog. Claude, Claude Code, Claude Cowork, Claude Opus, and the Model Context Protocol are products and trademarks of Anthropic. CallSphere is not affiliated with or endorsed by Anthropic.