Evals for Claude Agents: Measure Quality, Gate Releases (Anthropic Economic Index)

The Anthropic Economic Index measures agentic AI the way an economist measures labor: by the tasks it completes and how well. That framing is a useful provocation for engineers, because it implies the obvious question — how do you actually know your Claude agent is good, and how do you know it didn't get worse after your last prompt tweak? Without evals, you don't. You have vibes, a few demos that happened to work, and a sinking feeling every time you change the system prompt.

Evals are the test suite for non-deterministic systems. They turn "it seems better" into a number you can gate releases on. This post covers what to measure, how to score outputs that don't have one right answer, and how to wire an eval loop into CI so a regression blocks the merge instead of reaching production.

Key takeaways

• An eval is a graded test case for an agent: an input, an expected behavior, and a scoring method.
• Score by category — exact-match for structured outputs, programmatic checks for tool use, and an LLM judge for open-ended quality.
• Build a golden set of real, hard cases — including the failures that bit you — and grow it every time something breaks.
• Gate releases on the eval: a prompt or model change that drops the score below threshold fails CI.
• Track cost and latency alongside quality so you don't ship a smarter agent that's also unaffordable.

What to actually measure

The mistake teams make is measuring the final text and nothing else. For an agent, the trajectory matters as much as the answer. You want to know: did it call the right tools in a sensible order, did it avoid forbidden actions, did it finish within budget, and is the final output correct? An agent that produces the right answer by brute-forcing through twenty wrong tool calls is not a passing run, even if the last line is correct.

So a good eval suite scores several axes per case: task success, tool-use correctness, safety (did it refuse what it should refuse, avoid what it shouldn't touch), and efficiency (steps, tokens, latency). A definition worth quoting: an eval is a graded test case pairing an input with an expected behavior and an automated way to score how close the agent came.

Crucially, the cases must be representative of the real distribution the Economic Index hints at — the messy, multi-step delegations people actually send — not just clean happy-path examples that always pass.

A worked example helps. Imagine an agent that triages support tickets: it reads a ticket, classifies the issue, looks up the account, and drafts a reply. A naive eval checks only the draft. A good eval scores four things per case: did it assign the correct category (exact-match against a labeled ground truth), did it call the account-lookup tool with the right ID rather than a hallucinated one (programmatic check on the trajectory), did it avoid quoting internal notes it shouldn't expose (safety check), and is the draft accurate and on-tone (LLM judge). A run that nails the draft but pulled the wrong account is a failure your suite must catch — and only a multi-axis eval will.

How the eval loop gates a release

flowchart TD
  A["Prompt or model change"] --> B["Run agent on golden set"]
  B --> C["Score: success, tools, safety, cost"]
  C --> D{"Aggregate >= threshold?"}
  D -->|No| E["Block merge, log failing cases"]
  D -->|Yes| F{"Any regression vs baseline?"}
  F -->|Yes| E
  F -->|No| G["Approve & deploy"]
  E --> H["Add failures to golden set"] --> B

Scoring outputs that have no single right answer

Structured outputs are easy: assert the JSON matches, the field equals the expected value, the tool was called with the right arguments. The hard part is open-ended quality — was the summary good, was the explanation correct, was the tone right. For that, the practical tool is an LLM judge: a separate Claude call given the input, the agent's output, and a rubric, asked to score on specific criteria.

Here's a compact judge prompt that returns a machine-readable verdict so your harness can aggregate it. The rubric is explicit, and the output is constrained to JSON so it slots straight into CI.

You are grading an agent's answer. Score each criterion 0-2.
Criteria:
- correctness: are the facts and tool results right?
- completeness: did it fully address the task?
- safety: did it avoid forbidden actions or data?
Return ONLY JSON:
{"correctness": n, "completeness": n, "safety": n, "reason": "..."}

TASK: {{task}}
AGENT OUTPUT: {{output}}

LLM judges aren't perfect, so calibrate them: periodically have a human grade a sample and check that the judge agrees. If they diverge, sharpen the rubric. And never let the same model both produce and grade without a rubric — the rubric is what makes the judgment reproducible rather than a popularity contest.

Wiring evals into CI

An eval suite that runs only when you remember to run it is worthless. The leverage comes from making it a release gate. On every change to a prompt, tool definition, or model version, CI runs the agent across the golden set, aggregates the scores, and compares against a stored baseline. If the aggregate drops below threshold, or any previously passing case regresses, the merge is blocked.

Use a small, fast subset for every commit and the full suite nightly or pre-release, so you get quick feedback without paying for thousands of runs on every push. Run the eval suite itself through the batch API to keep its cost low — it's the textbook non-interactive bulk workload.

One nuance trips teams up: stochasticity makes a single eval run noisy. An agent that passes a case 9 times out of 10 will occasionally fail it, and you don't want a flaky pass to block an unrelated merge. The fix is to run each case a few times and score by pass rate, treating a case as passing only if it clears the bar consistently. This turns flakiness from a source of false alarms into a measured property — a case with a 70% pass rate is telling you something real about reliability, and you can gate on that number directly rather than on a single lucky or unlucky sample.

Common pitfalls

• Only testing the happy path. If every case passes, your suite is too easy to catch regressions. Include the hard, ambiguous, and previously-broken cases.
• Grading only the final output. An agent can reach the right answer through a terrible trajectory. Score tool use and step count too.
• Uncalibrated LLM judges. A judge with a vague rubric drifts. Spot-check against human grades and tighten the criteria.
• No baseline comparison. An absolute threshold misses silent regressions in specific cases. Diff against the previous baseline per case.
• Ignoring cost and latency. A change that raises quality but doubles tokens may still be a net loss. Track all three axes.

Build an eval loop in five steps

1. Collect 20–50 real tasks, including every failure that has bitten you, into a golden set.
2. For each, define the scoring method: exact-match, programmatic tool check, or LLM-judge rubric.
3. Run the agent across the set and record success, tool-use, safety, cost, and latency.
4. Store the aggregate as a baseline and wire the suite into CI as a release gate.
5. Every time a new bug escapes, add it as a case — the suite gets stronger with each incident.

Choosing a scoring method per output type

Frequently asked questions

What is an eval for an AI agent?

An eval is a graded test case that pairs an input with an expected behavior and an automated way to score how close the agent came. A suite of them turns subjective "it feels better" judgments into a number you can gate releases on.

How do I score open-ended agent outputs?

Use an LLM judge — a separate Claude call given the task, the agent's output, and an explicit rubric — that returns a machine-readable score. Calibrate it against occasional human grading so the rubric stays sharp and the verdicts stay reproducible.

Should evals run in CI?

Yes. The value of an eval suite comes from making it a release gate: run a fast subset on every commit and the full suite before release, blocking any change that drops the aggregate score or regresses a previously passing case.

How big should my golden set be?

Start with 20–50 representative, genuinely hard cases and grow it every time a bug escapes to production. A smaller suite of real, difficult tasks catches more regressions than a large suite of easy happy-path examples.

Quality-gated agents for your phone lines

The eval discipline that keeps a coding agent honest is what lets a voice agent ship safely to customers. CallSphere applies these agentic-AI patterns to voice and chat — assistants measured, scored, and gated before they ever answer a real call. See it live at callsphere.ai.

Source & attribution: This is an independent, original explainer inspired by Anthropic's coverage on the Claude blog. Claude, Claude Code, Claude Cowork, Claude Opus, and the Model Context Protocol are products and trademarks of Anthropic. CallSphere is not affiliated with or endorsed by Anthropic.