Evals for Claude Code workflows: gating releases right

The most dangerous moment in an agentic project is the one where the workflow starts working. It handles the demo case, the team is delighted, and someone ships it. Then a slightly different input arrives in production and the agent does something subtly wrong that nobody catches for a week. The problem was never that the workflow couldn't work — it was that nobody had a way to know whether a change made it better or worse. Without evals, you are shipping on vibes, and vibes do not survive contact with real traffic.

An eval is a repeatable test that measures the quality of an agent's output against a known dataset, producing a score you can compare across versions. For dynamic workflows — where the same input can take different paths — evals are not a nice-to-have. They are the only thing standing between a deliberate, measured release and a coin flip. This post covers how to build an eval loop that actually gates releases: what to measure, how to grade it, and how to wire it into your shipping process.

Why agentic systems need their own eval discipline

Traditional software testing assumes determinism: same input, same output, assert equality. Dynamic workflows violate that assumption. The agent samples tokens, chooses its own path, and can reach a correct answer two different ways or a wrong answer via a path that looked fine. You cannot assert exact output equality, because there often is no single correct output — there is a space of acceptable ones and a space of unacceptable ones.

That shifts evaluation from "did it match" to "was it good enough," which means you need graders that judge quality, not just diffs. It also means a single run tells you almost nothing. Because of sampling variance, the same workflow can pass on one run and fail on the next, so you evaluate over a dataset of cases and look at aggregate pass rates, not individual outcomes. One green run is not a signal; a stable pass rate across many cases is.

Build the eval dataset first

An eval is only as good as its dataset. Start by collecting real cases — the actual inputs your workflow will see — and label each with what a good outcome looks like. Include the easy happy-path cases, but spend most of your effort on the edges: the ambiguous request, the malformed input, the case that previously caused a loop or a wrong tool call. Your dataset should encode every failure you have ever seen, so that no past bug can silently return.

This is the most underrated practice in agentic engineering: every time a workflow fails in production, turn that failure into an eval case before you fix it. Over time your dataset becomes a precise map of your system's weak points, and your pass rate becomes a meaningful number. A dataset that only contains happy paths produces a reassuring score that means nothing; a dataset full of hard cases produces a score you can actually trust.

flowchart TD
  A["Code or prompt change"] --> B["Run workflow over
eval dataset"]
  B --> C["Grade each output"]
  C --> D["Aggregate score
& per-case results"]
  D --> E{"Score >= gate
& no regressions?"}
  E -->|Yes| F["Allow release"]
  E -->|No| G["Block; show
failing cases"]
  G --> H["Fix & add
new eval case"]
  H --> A

Choosing graders: code, model, and human

How you grade depends on what you are measuring. The cheapest and most reliable grader is code: a deterministic check that the output has the right structure, contains a required value, passes the tests, or satisfies a rule. Whenever a quality criterion can be expressed as code, do it that way — it is fast, free, and never flaky. Reach for fancier graders only for the criteria code cannot capture.

For subjective quality — was the explanation clear, was the tone right, did the agent reason soundly — use an LLM-as-judge grader: a separate model call that scores the output against a rubric you write. This is powerful but must be calibrated. Write a precise rubric, test the judge against human-labeled examples until its scores correlate with yours, and be aware that a judge can be lenient or inconsistent if its instructions are vague. A miscalibrated judge gives you confident, wrong numbers.

Human review remains the gold standard for a small, rotating sample. You will never hand-grade thousands of cases, but spot-checking a handful each cycle catches the failures your automated graders miss and keeps your judge honest. The practical setup is layered: code graders for everything mechanical, a calibrated LLM judge for quality, and human review on a sample — each catching what the cheaper layer cannot.

Gate releases on the eval, not on a demo

The eval loop only matters if it has teeth. Wire it so that no change to the workflow's prompt, tools, or model ships without running the full dataset and clearing a defined bar. Set a gate: the aggregate score must exceed a threshold and no previously-passing case may regress. A change that raises the average but breaks three cases that used to work is not an improvement — it is a trade you must make consciously, not by accident.

Track scores over time so you can see trajectory, not just a snapshot. When you upgrade the underlying model, change a tool description, or rewrite a prompt, run the eval before and after and compare. This is what turns model and prompt changes from anxiety-inducing guesses into measured decisions. The number tells you whether the change helped, and the per-case breakdown tells you exactly what it helped and what it hurt.

Finally, keep the loop fast enough to run often. If your eval takes an hour, people will skip it under deadline pressure. Sample a representative subset for quick iteration and run the full dataset before a real release. The discipline that survives is the one that is cheap to follow, so invest in making the eval loop quick and automatic rather than a heavyweight ceremony.

Frequently asked questions

What is an eval for an agentic workflow?

An eval is a repeatable test that runs the workflow over a labeled dataset of real cases and grades each output, producing an aggregate score you can compare across versions. Because dynamic workflows are non-deterministic, you measure aggregate pass rates over many cases rather than asserting exact output equality on a single run.

How do I grade outputs that aren't deterministic?

Layer your graders. Use code checks for anything mechanical — structure, required values, passing tests. Use a calibrated LLM-as-judge with a precise rubric for subjective quality like clarity or reasoning. Spot-check a human sample each cycle to catch what the automated graders miss and keep the judge honest.

How do I stop old bugs from coming back?

Turn every production failure into an eval case before you fix it, and gate releases so no previously-passing case may regress. Over time your dataset becomes a map of every weak point your system has ever shown, and the regression check ensures fixed bugs stay fixed.

Can I trust an LLM judge to grade quality?

Only if you calibrate it. Write a precise rubric and test the judge against human-labeled examples until its scores correlate with yours. An uncalibrated judge with vague instructions produces confident but unreliable scores, so validate it the way you would validate any measurement instrument.

Quality-gated agentic AI for your phone lines

CallSphere runs the same eval discipline behind its voice and chat agents — measuring quality on real cases and gating releases — so the agents that answer every call and message stay reliable as they evolve. See it live at callsphere.ai.

Source & attribution: This is an independent, original explainer inspired by Anthropic's coverage on the Claude blog. Claude, Claude Code, Claude Cowork, Claude Opus, and the Model Context Protocol are products and trademarks of Anthropic. CallSphere is not affiliated with or endorsed by Anthropic.