Debugging Claude Clinical Abstraction Agents That Fail

A clinical abstractor is a person who reads a messy chart — discharge summaries, lab panels, free-text nursing notes — and pulls out the specific structured facts a registry or quality measure needs: principal diagnosis, admit date, ejection fraction, the exact ICD-10 code. When you ask Claude to do that same job as an agent, with tools to look up codes and write rows into a database, you inherit a new class of bugs that don't exist in a single prompt-and-response call. The agent doesn't just answer; it acts in a loop, and loops fail in ways that are harder to see.

This post is about debugging those failures specifically. We'll walk through the three failure modes that dominate abstraction agents — runaway loops, wrong tool calls, and hallucinated arguments — and the instrumentation and prompt changes that actually move the needle. The examples assume a Claude Code or Agent SDK harness with a few MCP tools: a terminology lookup, a chart-fetch tool, and a write-record tool.

Why agent bugs hide better than prompt bugs

In a single completion, if Claude gets the answer wrong you see it immediately in the text. In an agent, the wrong decision happens three tool calls deep, gets partially corrected, and the final output looks plausible. A clinical abstractor agent might extract the correct diagnosis, call the terminology tool with a slightly wrong query, accept a near-match code, and write a confidently wrong row. Nothing crashed. The bug is in the trajectory, not the answer.

The first rule of debugging agents is therefore: log the full trajectory, not just the final message. Capture every tool call with its arguments, every tool result, and the model's text between calls. In the Claude Agent SDK you get this through the streamed message events; in Claude Code you can write a hook that appends each tool invocation to a JSONL trace file. Without a trajectory you are guessing. With one, most bugs are obvious within a minute of reading.

Failure mode one: the loop that never ends

The most common abstraction-agent failure is the loop. Claude fetches the chart, doesn't find the ejection fraction, searches again, reformulates, searches again, and never decides to stop. Each turn looks individually reasonable, which is exactly why the loop survives. The agent has no stopping criterion strong enough to overrule its own helpfulness.

flowchart TD
  A["Start abstraction"] --> B["Fetch chart sections"]
  B --> C{"Field found?"}
  C -->|Yes| D["Validate & record"]
  C -->|No| E{"Retries < 2?"}
  E -->|Yes| F["Try alternate section"] --> C
  E -->|No| G["Emit 'not documented'"]
  D --> H["Next field"]
  G --> H

The fix is rarely a smarter model — it's an explicit budget and an explicit "not found" exit. Give the agent a hard retry cap per field, and make "this field is not documented in the chart" a first-class, rewarded outcome rather than a failure. Abstractors do this constantly: a missing field is a valid finding. If your prompt implies every field must be filled, the agent will loop or hallucinate to satisfy that pressure. State plainly that returning not_documented is correct and expected.

At the harness level, enforce the budget independently of the prompt. Count tool calls per task and inject a system message when the agent crosses a threshold: "You have used your search budget for this field; record what you have or mark it not documented." Models follow a budget far more reliably when the harness reminds them mid-run than when they have to track it themselves.

Failure mode two: the wrong tool call

The second mode is calling the right action with the wrong tool, or the wrong tool entirely. The agent needs to look up an ICD-10 code and instead calls the chart-fetch tool with the diagnosis name. This usually traces back to tool descriptions that overlap or under-specify when each tool applies. Claude routes on the description text, so vague descriptions produce vague routing.

Write tool descriptions as decision rules, not labels. Instead of "Searches the terminology database," write "Use this tool ONLY to convert a finalized diagnosis or procedure phrase into a billing code. Do not use it to read chart text." Add a one-line negative example of when not to use it. In practice, the single highest-leverage debugging move for wrong-tool-call bugs is tightening descriptions until each tool's boundary is unambiguous.

When you still see misrouting, add a lightweight pre-flight in the tool itself: if the terminology tool receives an argument that looks like a raw note rather than a clean diagnosis phrase, return a structured error telling Claude what kind of input it expects. Claude reads tool errors and self-corrects well when the error is specific. A tool that fails silently or returns garbage teaches the agent nothing.

Failure mode three: hallucinated arguments

The most dangerous mode in clinical work is the hallucinated argument: Claude calls the write-record tool with an ejection fraction of 55% that appears nowhere in the chart, because 55% is a common value and the model pattern-matched. The tool call succeeds, the data is wrong, and nothing flags it. This is where abstraction agents earn their reputation for being unsafe — not because they reason badly, but because confident fabrication slips through tool calls.

The defense is provenance. Require every recorded value to come with the source span it was extracted from, and validate that span server-side. Change the write-record tool's schema so it demands a source_quote field, then have the tool verify that quote literally exists in the fetched chart text before accepting the write. If the quote isn't found, reject the call. This single constraint eliminates most hallucinated arguments because the model can no longer record a value it can't ground.

Provenance also makes your traces auditable. When a human reviewer disputes a recorded field, the source quote shows exactly where Claude got it, turning a black-box disagreement into a two-second check. Grounding is both a correctness control and a debugging accelerant.

Reproduce deterministically before you fix

Agent bugs are slippery because the loop is partly nondeterministic — re-run the same chart and the trajectory can diverge, so a fix that seems to work might just be luck. Pin down a reproduction first. Capture the exact chart, the exact tool definitions, and the model version that produced the failure, and re-run that frozen setup several times. If the bug appears most of the time, you have a real reproduction to fix against. If it appears once in twenty runs, you're chasing variance, and the right response is usually a structural guard — a budget, a validation, a provenance check — rather than a prompt tweak aimed at a ghost.

Keep these reproductions as named cases. The chart that triggered a hallucinated ejection fraction, the chart that sent the agent into a loop, the chart that misrouted a tool call — each becomes a permanent fixture in your regression suite. Over a few months this collection becomes the most valuable debugging asset you own, because it encodes every failure mode your real data has thrown at the agent, and it runs automatically on every change.

A repeatable debugging loop

Put these together into a routine. When an abstraction run produces a wrong field, pull the trajectory, find the tool call that introduced the error, and classify it: loop, wrong tool, or hallucinated argument. Each class has a default fix — budget and a 'not found' exit; sharper tool descriptions and tool-side errors; or provenance enforcement. Re-run the same chart and confirm the fix holds. Then add that chart to your regression set so the bug can't silently return.

Resist the urge to fix everything in the system prompt. Prompt edits are global and brittle; a single added sentence to handle one chart can shift behavior on a hundred others. Prefer fixes that live in the harness and the tools — budgets, schemas, validations — because they are testable, local, and don't degrade with prompt drift.

Frequently asked questions

How do I tell a loop from legitimate thorough searching?

Look at whether successive tool calls add new information. Legitimate searching narrows or changes the query and pulls new chart sections. A loop repeats near-identical queries against the same data. If three consecutive calls return overlapping results, it's a loop, and a retry cap will end it without hurting genuine thoroughness.

Should I use a more capable model to reduce these bugs?

A stronger model like Claude Opus 4.8 reduces wrong-tool and hallucination rates, but it will not invent a stopping criterion or a provenance check you didn't build. Model choice helps; harness design is what makes the failures rare and catchable. Use the capable model and the controls together.

What's the fastest way to start debugging an existing agent?

Turn on full trajectory logging before anything else. Most teams debug blind because they only keep the final output. One JSONL trace per run, with every tool call and result, turns hours of speculation into minutes of reading.

Can I auto-detect hallucinated arguments without human review?

Yes, partially. Server-side validation that every recorded value has a verbatim source span in the chart catches the majority. For numeric fields you can add range checks. Humans still review edge cases, but provenance plus validation removes the bulk of fabricated arguments automatically.

From charts to phone calls

CallSphere runs the same agentic discipline on voice and chat — assistants that pull facts from your systems mid-conversation, ground every action in real data, and never loop a caller into oblivion. See how it works at callsphere.ai.

Source & attribution: This is an independent, original explainer inspired by Anthropic's coverage on the Claude blog. Claude, Claude Code, Claude Cowork, Claude Opus, and the Model Context Protocol are products and trademarks of Anthropic. CallSphere is not affiliated with or endorsed by Anthropic.