Evals for Claude Agents: Measure Quality, Gate Releases (Claude Api Skill Ecosystem)

You can't ship an agent you can't measure, and "it seemed to work when I tried it" is not measurement. The teams that ship Claude agents confidently all have the same thing: an eval loop that turns the vague question "is this agent good?" into a number that goes up or down when they change a prompt, swap a model, or add a tool. Without it, every change is a gamble and every regression is discovered by a user.

Evals for agents are harder than evals for single LLM calls, because an agent produces a trajectory — a sequence of tool calls and intermediate decisions — not just a final string. A good final answer reached through a chaotic, expensive, dangerous path is still a problem. This post covers how to build an eval loop that measures both the destination and the journey, and how to wire it into a release gate.

Start with gradeable criteria, not vibes

The foundation of any eval is a rubric of explicit, independently checkable criteria. "The report is good" is ungradeable. "The output is valid JSON, contains a numeric price field per SKU, cites at least one source per claim, and never calls issue_refund" is gradeable — each clause is a check that passes or fails. A defining sentence: an agent eval is a repeatable measurement that scores an agent's output and trajectory against a fixed rubric of independently gradeable criteria, producing a score you can track across versions.

Build your eval set from real cases. Mine production transcripts (and incident reports) for the inputs that mattered, including the ones that went wrong — every bug you fix should become a permanent eval case so it can never regress silently. Aim for coverage of the distribution, not volume: a focused set of 50–200 cases that span your real input types beats thousands of near-duplicates.

Three things to measure

An agent eval scores three layers. Outcome: did the final output meet the rubric? Trajectory: were the right tools called, in a sensible order, without forbidden actions or wasteful loops? Cost: how many tokens and how much latency did it take? A change that improves outcome quality but triples token spend is a tradeoff you want to see, not discover on the invoice.

flowchart TD
  A["Eval case\n(input + rubric)"] --> B["Run agent\ncapture output + trajectory"]
  B --> C["Deterministic checks\nschema, forbidden tools, cost"]
  B --> D["LLM-as-judge\nscore vs rubric"]
  C --> E{"Aggregate score\n>= threshold?"}
  D --> E
  E -->|Yes| F["Gate passes — release"]
  E -->|No| G["Gate fails\ninspect regressions"]
  G --> H["Fix & add failing case\nto eval set"]
  H --> A

Capture the trajectory by logging every tool_use block, its arguments, and response.usage on each turn. Many checks are then pure code: assert the output validates against a schema, assert no tool from your forbidden list was called, assert the run stayed under a token budget, assert no tool was called more than N times (your loop-detector). These deterministic checks are cheap, fast, and never flaky — run as many of your criteria through them as possible.

LLM-as-judge for the subjective criteria

Some criteria resist code: "is the tone appropriate," "did the answer actually address the question," "is the explanation correct." For these, use a separate Claude call as a judge. Give the judge the rubric, the input, and the agent's output, and ask it to score each criterion with a brief justification. Use structured outputs (output_config.format with a JSON schema) so the judge returns a parseable object — per-criterion pass/fail plus a one-line reason — rather than prose you have to scrape.

A few rules keep LLM-as-judge honest. Judge one criterion at a time or in a small explicit set; a judge asked for a single holistic "score out of 10" is noisy and unstable. Make the judge cite evidence from the output for each verdict — this both improves accuracy and gives you something to read when a score looks wrong. And periodically calibrate the judge against human labels on a sample; if the judge and a human disagree often, your rubric is ambiguous, not the agent. A capable model like Opus 4.8 makes a strong judge precisely because it follows a precise rubric literally.

The loop and the gate

The eval becomes valuable the moment it gates releases. Wire it into CI: on every prompt change, model bump, or tool edit, run the full eval set and compute an aggregate score per criterion. Set thresholds — overall pass rate, plus hard gates on the criteria you can't regress (no forbidden-tool calls, ever; schema validity at 100%). A change that drops below threshold doesn't merge.

Run the suite efficiently with the Batches API — eval cases are latency-tolerant and independent, so submit them as a batch at 50% cost, with a shared cached system prompt across all cases. When a gate fails, the per-criterion breakdown points straight at the regression: if "forbidden tool" suddenly fails on three cases after a prompt edit, you know exactly what your change broke. Fix it, add the failing input as a new permanent eval case, and the loop tightens with every iteration. Over time the eval set becomes the institutional memory of every way your agent has ever been wrong.

A note on managed outcome grading

If you build on Anthropic's managed agents, the outcome-grading loop formalizes this pattern at runtime: you send a user.define_outcome event with a rubric, and a separate grader (in its own context window) scores each iteration and feeds per-criterion gaps back to the agent until the work satisfies the rubric or hits a max-iterations cap. It's the same iterate → grade → revise shape as an offline eval, run live — and the discipline of writing a precise, gradeable rubric pays off in both places.

Frequently asked questions

What should an agent eval measure beyond the final answer?

The trajectory and the cost. Capture every tool call and its arguments to check that the right tools ran in a sensible order without forbidden actions or loops, and track token and latency budgets so quality gains that blow up cost are visible.

When should I use LLM-as-judge versus deterministic checks?

Use code for anything checkable — schema validity, forbidden-tool calls, cost budgets, call counts. Reserve LLM-as-judge for genuinely subjective criteria like tone, relevance, or correctness of an explanation, and have it score one criterion at a time with cited evidence.

How big should my eval set be?

Prioritize coverage over volume — 50 to 200 cases spanning your real input distribution, including every past failure, beats thousands of near-duplicates. Add each new bug as a permanent case so it can't silently regress.

How do I gate a release on evals?

Run the full suite in CI on every prompt, model, or tool change; compute per-criterion scores; and set thresholds, including hard gates on criteria you can't regress. A change below threshold doesn't merge.

Bringing agentic AI to your phone lines

CallSphere gates every change to its voice and chat agents on exactly this kind of eval loop — scoring real call transcripts against gradeable rubrics before anything reaches a live caller. See the measured-quality approach at callsphere.ai.

---

Source & attribution: This is an independent, original explainer inspired by Anthropic's coverage on the Claude blog. Claude, Claude Code, Claude Cowork, Claude Opus, and the Model Context Protocol are products and trademarks of Anthropic. CallSphere is not affiliated with or endorsed by Anthropic.