Evals for Claude Finance Plugins That Gate Releases

You cannot ship an agent into a finance close on vibes. A plugin that summarizes ledgers and proposes journal entries needs the same release discipline as any system that touches money: a way to measure whether it is actually correct, and a gate that blocks a regression from reaching production. Yet evals are the step most teams skip, because writing them feels slower than just trying the plugin once and watching it work. That instinct is exactly how a silent regression — a tool description you tweaked, a model version you bumped — ends up restating a close two days before the deadline.

This post is about building an eval loop for Claude Cowork finance plugins: what to measure, how to score agentic behavior that is non-deterministic, and how to wire the eval into a release gate so a drop in quality blocks the deploy automatically. The emphasis throughout is on finance-specific correctness — not just "did the agent answer" but "is the number right and did it use the right tool to get there."

Key takeaways

• An eval is a fixed set of inputs with known-correct expectations that you run on every plugin change to measure quality objectively.
• Finance evals must score three things: final-answer accuracy, tool-call correctness, and adherence to required guardrails like approval gates.
• Use programmatic checks for anything verifiable (a reconciliation should sum to zero) and an LLM judge only for the genuinely subjective parts.
• Gate releases on a threshold: if accuracy or tool-correctness drops below the bar, the deploy fails — no manual override on a close-critical plugin.
• Grow the eval set from production failures so every real-world bug becomes a permanent regression test.

What to measure in a finance plugin

Generic agent evals ask "was the response helpful." Finance evals have to be sharper, because a confident-but-wrong number is worse than no answer. Score three distinct dimensions. First, final-answer accuracy: for a reconciliation, does the net difference match the known-correct value; for a variance report, do the figures tie to a golden dataset. Second, tool-call correctness: did the agent call the right tools with the right arguments, regardless of whether it stumbled into a plausible final answer. Third, guardrail adherence: did it actually pause for human approval before proposing a journal entry, or did it skip the gate.

The reason to separate tool-call correctness from answer accuracy is subtle but important. An agent can produce the right final number through the wrong path — calling a tool with a hallucinated filter that happens to return the same total this period — and that path will break next period. Scoring the path catches latent bugs that the answer alone hides.

flowchart TD
  A["Plugin change / model bump"] --> B["Run eval set: golden cases"]
  B --> C["Programmatic checks: sums tie?"]
  B --> D["Tool-call trace scored"]
  B --> E["LLM judge: subjective parts"]
  C --> F{"All thresholds met?"}
  D --> F
  E --> F
  F -->|Yes| G["Release to close"]
  F -->|No| H["Block deploy, file failing case"]
  H --> B

Programmatic checks beat LLM judges where you can use them

Finance is unusually friendly to deterministic scoring, and you should exploit that. Whenever a property is verifiable by code, check it by code: a reconciliation's components should sum to the stated total, a balance sheet should balance, a re-forecast should reconcile to the prior actuals. These checks are fast, free, and unambiguous. Reserve the LLM judge — using a model to grade output quality — for the genuinely subjective parts, like whether a narrative variance explanation is clear and correctly attributes the driver.

A minimal eval case for a reconciliation plugin captures the input, the expected number, and the required tool path. Running it asserts all three dimensions at once:

{
  "name": "q3_ap_recon_entity_12",
  "input": { "task": "reconcile AP", "entity": "12", "period": "2026-09" },
  "expect": {
    "net_difference": 0.00,
    "must_call_tools": ["get_ap_ledger", "get_bank_statement"],
    "must_not_call": ["post_journal_entry"],
    "requires_human_approval_before": "post_journal_entry"
  }
}

The must_not_call and requires_human_approval_before fields encode guardrails directly into the eval, so a regression that lets the agent post an entry without approval fails the test loudly rather than slipping into production.

Scoring non-deterministic behavior

Agents are stochastic, so a single pass is not a reliable measurement. Run each eval case several times and look at the distribution: a plugin that gets the right answer eight times out of ten is meaningfully different from one that gets it ten out of ten, and for a close-critical workflow you may demand the latter. Track pass rate per case, not just an aggregate, because an aggregate can hide one critical case that fails half the time while easy cases inflate the average.

Pin everything that should be fixed: the model version, the input snapshot, the tool definitions. When the eval result moves, you want to know whether it was your change or an upstream model update. Treat the model version as part of the release artifact — bumping from one Claude version to another is a change that must pass the full eval before it reaches the close.

Gating the release

An eval that nobody enforces is documentation, not a gate. Wire the eval run into the deploy path so that shipping a plugin change requires the eval to pass at or above your thresholds. For a finance plugin those thresholds should be strict: perhaps 100% on the guardrail checks (an approval gate is never optional), a high bar on tool-call correctness, and a final-answer accuracy floor on the golden set. If any dimension drops below its bar, the deploy fails automatically. No human override for a system that posts to the ledger.

Common pitfalls

• Scoring only the final answer. A right number via the wrong tool path is a latent bug. Score the tool trace, not just the output.
• Using an LLM judge for verifiable facts. If sums should tie, check them in code. Judges are for subjective quality, and they cost tokens and add noise.
• Running each case once. Agents are stochastic; a single pass over-reports reliability. Run several times and track per-case pass rate.
• Not pinning the model version. An upstream model bump can shift results silently. Treat the version as part of what the eval gates.
• A gate with manual overrides. The first time someone overrides "just this once" before a deadline, the gate is gone. Make guardrail checks non-overridable.

Build your eval loop in five steps

1. Assemble a golden set of representative finance tasks with known-correct numbers and the required tool paths for each.
2. Write programmatic checks for everything verifiable — sums tie, balances balance — and reserve an LLM judge for subjective narrative quality.
3. Encode guardrails (required approvals, forbidden tools) directly as assertions in each eval case.
4. Run each case multiple times, track per-case pass rate, and pin the model version and input snapshot.
5. Wire the eval into the deploy path so a drop below threshold blocks the release automatically, and add every production failure as a new case.

Frequently asked questions

What is an eval for an agentic finance plugin?

An eval is a fixed set of representative finance tasks paired with known-correct expectations — the right reconciliation number, the required tool path, the mandatory approval gate — that you run on every plugin change to measure quality objectively. It turns "it worked when I tried it" into a repeatable, enforceable measurement.

When should I use an LLM judge versus a programmatic check?

Use a programmatic check for anything verifiable by code, which in finance is most of the important properties: sums tie, balances balance, totals match a golden dataset. Reserve an LLM judge for the genuinely subjective parts, such as whether a narrative variance explanation reads clearly and attributes the right driver.

How do I handle the fact that the agent is non-deterministic?

Run each eval case several times and track the per-case pass rate rather than a single result, so a plugin that passes eight of ten times is distinguished from one that passes every time. Pin the model version and input snapshot so that when results move you can tell whether your change or an upstream model update caused it.

Should the eval ever be overridable to hit a deadline?

The guardrail checks — required approvals and forbidden tools — should never be overridable, because the first "just this once" exception under deadline pressure permanently weakens the gate. Soft, advisory dimensions like narrative quality can be warnings, but anything that touches ledger integrity must block the deploy.

Bringing agentic AI to your phone lines

The same eval-and-gate rigor keeps CallSphere's voice and chat agents trustworthy — every release is measured against golden conversations before it ever answers a customer call or books work. See it live at callsphere.ai.

Source & attribution: This is an independent, original explainer inspired by Anthropic's coverage on the Claude blog. Claude, Claude Code, Claude Cowork, Claude Opus, and the Model Context Protocol are products and trademarks of Anthropic. CallSphere is not affiliated with or endorsed by Anthropic.