Evals for Claude Legal Agents: Gating Releases on Quality

The most dangerous moment in a legal-agent deployment is the quiet one: a prompt tweak that looks harmless, ships on a Friday, and starts missing a class of indemnification clauses that nobody notices until a deal closes on a bad summary. There is no stack trace for "the agent got subtly worse." The only thing standing between you and that silence is an eval loop — a systematic way to measure quality and refuse to ship regressions. When you deploy Claude across the legal industry, your evals are your safety rail.

An eval is a repeatable test that scores an agent's output against a known-good expectation. Unlike a unit test, it tolerates fuzziness — there are many acceptable ways to summarize a clause — but it still must produce a number you can compare across versions. Building that number, honestly, is the hardest and most valuable engineering work in the whole project.

Start with a golden set drawn from real matters

Every eval loop begins with a dataset. For legal agents, build a golden set of representative tasks with vetted correct answers: contracts paired with the clauses a senior associate actually flagged, intake messages paired with the correct matter type, research questions paired with the controlling authority. Scrub privilege, but keep the difficulty — your golden set must include the gnarly cross-referenced clauses and the ambiguous edge cases, because those are exactly where regressions hide.

Cover the distribution, not just the easy center. Stratify the set by document type, clause category, and difficulty so a passing score actually means the agent handles the variety it will meet in production. A golden set of fifty clean NDAs will tell you nothing about how the agent handles a 200-page credit agreement, and the credit agreement is where the money is.

Scoring: exact match, rubric, and LLM-as-judge

How you score depends on the task. For classification — matter type, clause presence, risk tier — use exact or set-based matching against the labeled answer; these give you precision, recall, and accuracy directly. For open-ended outputs like clause summaries or risk explanations, exact match is useless, so you score against a rubric: does the summary identify the obligation, the trigger, the party, and the remedy?

flowchart TD
  A["New prompt / model version"] --> B["Run against golden set"]
  B --> C{"Task type?"}
  C -->|Classification| D["Exact / set match score"]
  C -->|Open-ended| E["LLM-as-judge vs rubric"]
  D --> F["Aggregate metrics"]
  E --> F
  F --> G{"Score >= gate & no critical regression?"}
  G -->|Yes| H["Promote release"]
  G -->|No| I["Block, inspect failures, iterate"]

LLM-as-judge — using a capable Claude model to grade outputs against a rubric — scales rubric scoring to thousands of cases. It works well when the rubric is concrete and the judge sees the reference answer. But it has known biases: judges can favor longer answers, reward confident tone over correctness, and drift if the rubric is vague. Calibrate the judge against human grades on a sample, pin the judge model version, and periodically re-check that machine and human scores still agree. A judge you never audit is a number you cannot trust.

Metrics that map to legal risk

Aggregate scores hide the failures that matter. In legal review, a false negative — missing a present risk — is far worse than a false positive, because a flagged-but-benign clause costs a minute of a lawyer's time while a missed indemnity costs a lawsuit. So track precision and recall separately, weight recall on high-severity clause categories, and set a hard floor on recall for the categories where a miss is unacceptable.

Report per-category breakdowns, not just a global mean. An agent that scores 92% overall might be at 99% on confidentiality clauses and 70% on limitation-of-liability — and the 70% is the one that will hurt you. Your dashboard should make that visible at a glance, and your release gate should key on the worst critical category, not the average.

Wiring evals into a release gate

An eval that runs only when someone remembers is theater. Wire it into the release path: every prompt change, tool change, or model upgrade triggers the full eval suite automatically, and promotion is blocked unless the suite clears its thresholds and shows no regression on any critical category versus the current production version. This is the gate that catches the harmless-looking Friday tweak.

Define the gate as a comparison, not just an absolute. "Score above 85%" lets a change that drops you from 96% to 86% sail through. "No critical category regresses by more than one point against production" catches it. Pin the eval dataset and the judge version so the comparison is apples-to-apples, and version your eval set alongside your code so you can reproduce any past result.

Closing the loop with production feedback

Your golden set will never anticipate everything. The richest source of new eval cases is production itself: every time a lawyer corrects the agent, every time a reviewer overrides a risk score, capture that as a candidate eval case. Triage these weekly, scrub them, and fold the instructive ones back into the golden set. Over time your evals stop reflecting what you imagined the agent would face and start reflecting what it actually faces — which is the only standard that matters.

This feedback loop is also how you earn trust with the lawyers who use the system. When a partner sees that their correction last month is now a permanent test the agent must pass, the agent stops being an opaque black box and becomes a system that demonstrably learns from its mistakes. That trust, more than any benchmark, is what gets a legal-AI deployment renewed.

Frequently asked questions

What is an eval for an AI agent?

An eval is a repeatable test that scores an agent's output against a known-good expectation, producing a number you can compare across versions. It tolerates the fuzziness of open-ended answers while still gating releases on measurable quality, unlike a brittle exact-match unit test.

Can I trust LLM-as-judge for legal evals?

Yes, with calibration. Judges scale rubric scoring but carry biases toward length and confident tone, so calibrate the judge against human grades on a sample, pin its model version, and re-audit agreement periodically. An uncalibrated judge produces numbers you cannot defend.

What metric matters most for legal review agents?

Recall on high-severity clause categories. Missing a present risk is far costlier than over-flagging, so track precision and recall separately, break results down per category, and set a hard recall floor on the categories where a miss is unacceptable.

How do I keep my eval set relevant over time?

Feed production back into it. Capture every lawyer correction and reviewer override as a candidate case, scrub it, and fold the instructive ones into the golden set weekly. The set then reflects what the agent actually faces rather than what you guessed it would.

Bringing measured quality to your phone lines

CallSphere runs the same eval discipline — golden sets, calibrated judges, and hard release gates — on its voice and chat agents, so every conversational improvement is measured before it reaches a live caller. See it live at callsphere.ai.

Source & attribution: This is an independent, original explainer inspired by Anthropic's coverage on the Claude blog. Claude, Claude Code, Claude Cowork, Claude Opus, and the Model Context Protocol are products and trademarks of Anthropic. CallSphere is not affiliated with or endorsed by Anthropic.