Risk management for Claude agents: contain the blast radius

The first time an agent does something genuinely wrong in production — refunds an order it shouldn't, deletes the wrong record, leaks a snippet of one customer's data into another's reply — the lesson lands hard: an agent is not a chatbot with extra steps. It is a system that takes actions, and actions have blast radius. The teams that ship durable agents on Claude are not the ones that never fail; they are the ones who designed for failure before it happened. This post is about doing that deliberately.

Key takeaways

• Catalog failure modes by category — wrong action, runaway loop, data exposure, tool error, hallucinated state — not one-off bugs.
• Blast radius is determined by tool design: read-only tools, scoped credentials, and confirmation gates shrink it dramatically.
• Put a human-in-the-loop gate on any irreversible or high-value action; reversible actions can run autonomously.
• Hard budgets and timeouts on steps and tokens stop runaway loops before they cost real money.
• Treat the agent like an untrusted client of your own APIs — authorize every call server-side, never trust the model.

The failure modes that actually bite

Agent failures cluster into a small number of categories, and naming them is half the battle. Wrong action: the agent calls a valid tool with valid-looking but incorrect arguments — refunding the wrong order. Runaway loop: the agent retries, second-guesses, and burns tokens or hammers an API without converging. Data exposure: context from one user, document, or tenant bleeds into another's output. Tool error cascade: a tool returns an error the agent mishandles, then compounds. Hallucinated state: the agent acts as if a step succeeded when it didn't.

Each category has a different containment strategy, which is exactly why categorizing matters. You do not fix "the agent did a bad thing" — you fix the class.

How do you contain blast radius?

Containment is an architecture decision made at the tool boundary, long before a single prompt is written. The cleanest mental model is a gate: every action the agent proposes flows through checks before it touches the real world.

flowchart TD
  A["Claude proposes action"] --> B{"Reversible?"}
  B -->|Yes| C{"Within budget & scope?"}
  B -->|No| D["Require human approval"]
  C -->|Yes| E["Execute via scoped credential"]
  C -->|No| F["Block & alert"]
  D --> G{"Human approves?"}
  G -->|Yes| E
  G -->|No| F
  E --> H["Log trace for audit"]

The key insight is that the model never directly holds the power to do damage. It proposes; your code disposes. A confirmation gate on irreversible actions, scoped credentials so the agent literally cannot reach data it shouldn't, and a budget check that kills runaway loops — these are not model features, they are engineering you own.

A concrete example: a guarded tool wrapper

The single highest-leverage pattern is wrapping every consequential tool in a server-side guard that re-checks authorization and reversibility, independent of what Claude asked for. Here is the shape of it.

HIGH_RISK = {"issue_refund", "delete_record", "send_payment"}

def execute_tool(name, args, ctx):
    # 1) Server-side authorization — never trust the model
    if not authorized(ctx.user, name, args):
        return {"error": "unauthorized"}
    # 2) Budget & loop guard
    if ctx.steps > MAX_STEPS or ctx.spend > MAX_SPEND:
        return {"error": "budget_exceeded"}
    # 3) Human gate for irreversible / high-value actions
    if name in HIGH_RISK and not ctx.approved(name, args):
        return {"status": "pending_approval"}
    result = TOOLS[name](**args)
    audit_log(ctx.trace_id, name, args, result)  # 4) always audit
    return result

Notice the authorization check uses ctx.user, not anything the model supplied. If an agent is tricked or simply wrong and tries to refund another customer's order, the guard rejects it because the server, not the model, decides who is allowed to do what. This one wrapper neutralizes the majority of wrong-action and data-exposure failures.

Common pitfalls in agent risk management

• Trusting the model to enforce policy. Putting "never refund without approval" in the system prompt is a hint, not a guarantee. Fix: enforce policy in code at the tool boundary.
• One giant credential. Giving the agent an admin key means any failure is maximal. Fix: scope credentials to the narrowest set of actions and data the task needs.
• No step or token ceiling. Without budgets, a looping agent can run for thousands of steps overnight. Fix: hard caps on steps, wall-clock time, and token spend per task.
• Treating all actions equally. Gating reads as strictly as deletes adds friction and trains people to rubber-stamp approvals. Fix: gate by reversibility and value, let safe actions flow.
• No audit trace. When something goes wrong you can't reconstruct it. Fix: log every tool call with arguments and a trace ID, immutably.

Build your containment plan in 7 steps

1. List every tool the agent can call and label each reversible or irreversible.
2. For each tool, assign the narrowest credential and data scope that still works.
3. Add a server-side authorization check keyed to the real end user, not the model.
4. Put a human-approval gate on every irreversible or high-value action.
5. Set hard ceilings on steps, time, and token spend per task; fail closed.
6. Log every tool call with arguments, result, and trace ID for audit and replay.
7. Run a quarterly "red team" pass where you actively try to make the agent misbehave.

Reversible vs. irreversible action handling

Property	Reversible action	Irreversible / high-value
Example	Draft email, tag record	Send payment, delete data
Default mode	Autonomous	Human approval required
Credential scope	Read/write to safe namespace	Tightly scoped, time-boxed
Rollback plan	Undo or re-run	Pre-action confirmation only
Blast radius	Low	High — contain aggressively

Blast radius is the total scope of harm a single agent action can cause if it is wrong — measured in data exposed, money moved, or records changed — and the core job of agent risk management is to make that radius as small as the task allows. An agent that can only ever touch one tenant's reversible data is a fundamentally safer system than one prompt away from disaster, regardless of how good the model is.

Frequently asked questions

Can I rely on Claude's safety training to prevent harmful actions?

Model safety helps with content, but it cannot know your business rules — that this user can't refund that order. Authorization and reversibility must be enforced in your code, not the model.

How do I stop an agent from looping forever?

Set hard ceilings: maximum steps per task, a wall-clock timeout, and a token-spend cap. When any limit is hit, fail closed and surface the partial state to a human rather than retrying.

When should a human be in the loop?

On any action that is irreversible or above a value threshold — payments, deletions, external sends. Reversible, low-value actions can run autonomously to keep the agent useful.

What is the most common single point of failure?

Over-broad credentials. An agent holding an admin key turns any small mistake into a large incident. Scope every credential to the minimum the task requires.

Bringing safe agents to your phone lines

CallSphere brings this same containment discipline to voice and chat — agents with scoped tools, approval gates, and full audit trails that answer every call and message safely, day and night. See it in action at callsphere.ai.

---

Source & attribution: This is an independent, original explainer inspired by Anthropic's coverage on the Claude blog. Claude, Claude Code, Claude Cowork, Claude Opus, and the Model Context Protocol are products and trademarks of Anthropic. CallSphere is not affiliated with or endorsed by Anthropic.