By Sagar Shankaran, Founder of CallSphere
A staged, reversible rollout plan for moving a threat-detection workflow onto a Claude Code agent: baseline, shadow mode, canary, and instant rollback.
Key takeaways
You already have a threat-detection workflow. Maybe it's a pile of SIEM correlation rules, a runbook your analysts follow by hand, and a few brittle scripts. It works, mostly, and people depend on it. Now you want to move triage onto a Claude Code agent. The temptation is to flip the switch and let the agent take over. Don't. The fastest way to lose trust in an agentic system — and to actually miss a breach — is a big-bang cutover. Migrating safely is a discipline of running old and new side by side until the new one has earned its place. This post is the playbook.
Before you build anything, document the existing process as a precise specification: what triggers a triage, what data gets pulled, what decisions get made, and what the acceptable error rates are today. This sounds obvious and almost nobody does it. The current false-positive and false-negative rates are your baseline; without them you can't tell whether the agent is an improvement or a quiet regression. Capture a few weeks of historical alerts with their eventual ground-truth outcomes — these become both your eval set and your shadow-mode comparison data.
This step also surfaces the implicit knowledge living in analysts' heads. The runbook says "investigate the login," but the senior analyst actually checks three specific things in a specific order. That tacit expertise is what you'll encode into the agent's instructions and tools, and writing it down is half the migration.
The first deployment of the agent takes no real actions. It runs in shadow mode: it receives the same alerts the live workflow does, performs its full investigation, and writes its verdict to a log — but the existing workflow remains in control and the agent's output changes nothing. Now you can compare, alert by alert, what the agent would have decided against what the current system did and what the truth turned out to be. Shadow mode is where you build confidence cheaply, because mistakes cost nothing.
Hear it before you finish reading
Talk to a live CallSphere AI voice agent in your browser — 60 seconds, no signup.
Run shadow mode long enough to cover the variety of real traffic — including the rare, weird alerts, since those are where agents and rules disagree most. Track agreement rate, and for every disagreement, have an analyst adjudicate: was the agent right, or the old system? Each disagreement is a finding that either fixes the agent or, sometimes, reveals a gap in the legacy rules.
flowchart TD
A["Document legacy workflow & baseline rates"] --> B["Deploy agent in shadow mode"]
B --> C["Agent verdict logged, takes no action"]
C --> D{"Agrees with legacy & ground truth?"}
D -->|Disagreements| E["Analyst adjudicates, fix agent"]
E --> B
D -->|Agreement stable| F["Canary: agent owns low-risk slice"]
F --> G{"Metrics hold on live traffic?"}
G -->|No| H["Roll back to legacy instantly"]
G -->|Yes| I["Widen scope, keep human approval for containment"]
When shadow agreement is high and disagreements consistently favor the agent (or are explainable), promote it to a real but tightly bounded role. Pick the lowest-risk slice of traffic — perhaps auto-closing the highest-volume, most obviously benign alert category — and let the agent actually act there while everything else stays on the legacy path. This is where the agent first earns trust by carrying real load, but with a small blast radius. If it misbehaves, the damage is contained to one well-understood category and you learn fast.
Keep destructive actions human-gated throughout. Even as the agent takes over triage, containment steps — isolating a host, disabling an account — should require analyst approval well past the point where auto-close is fully delegated. The order of delegation is: first let the agent recommend, then let it auto-close benign, and only much later, if ever, let it act on high-stakes outcomes unattended.
Every stage must be reversible in seconds, not in a deploy cycle. Put the agent's authority behind a feature flag so that if the canary metrics slip — override rate climbs, a false-negative slips through — you flip back to the legacy workflow immediately while you diagnose. A migration where rollback is hard is a migration that pressures people to push through problems rather than retreat from them. The whole point of staging is to make stepping back cheap, so that stepping forward is safe. Run the legacy workflow in parallel, ready to resume, until the agent has held its metrics across enough real traffic and enough edge cases that the team genuinely trusts it.
Widen the agent's scope one slice at a time, each expansion justified by the metrics from the previous one, never by enthusiasm. As coverage grows, your eval set grows with it — every production override feeds back as a new test case, exactly as in your release gate. Only when the agent has owned a category through its full range of seasonal and adversarial variation should you consider retiring the corresponding legacy rules, and even then, archive them rather than delete them. The legacy system is your ultimate fallback; keep it runnable long after you've stopped relying on it. A migration is finished not when the agent works on a good day, but when the team is comfortable that a bad day is still handled safely.
Shadow mode runs the new agent on real production alerts in parallel with the existing workflow, logging its verdicts but taking no action and changing no outcomes. You start there because it lets you measure the agent against both the legacy system and ground truth at zero risk, building confidence before the agent ever affects anything.
Still reading? Stop comparing — try CallSphere live.
CallSphere ships complete AI voice agents per industry — 14 tools for healthcare, 10 agents for real estate, 4 specialists for salons. See how it actually handles a call before you book a demo.
Stage the rollout: document the legacy workflow and its baseline error rates, run the agent in shadow mode, then canary it on the lowest-risk slice of traffic with a small blast radius, and expand only as metrics hold. Keep every stage behind a feature flag so rollback to the legacy path is instant.
Not early, and ideally not unattended even late. Delegate in order — first recommendations, then auto-closing obviously benign alerts, and keep containment actions like host isolation and account disabling behind human approval well past the point where low-risk triage is fully handed over.
Only after the agent has owned each category through its full range of normal, seasonal, and adversarial variation while holding its quality metrics, and even then archive the legacy rules rather than delete them. Keep the old workflow runnable as a fallback long after you've stopped depending on it day to day.
CallSphere rolls out its voice and chat agents the same careful way — shadow, canary, then widen — so assistants that answer every call and message and book work 24/7 earn trust before they carry load. See it live at callsphere.ai.
Source & attribution: This is an independent, original explainer inspired by Anthropic's coverage on the Claude blog. Claude, Claude Code, Claude Cowork, Claude Opus, and the Model Context Protocol are products and trademarks of Anthropic. CallSphere is not affiliated with or endorsed by Anthropic.

Written by
Sagar Shankaran· Founder, CallSphere
LinkedInSagar Shankaran is the founder of CallSphere, where he builds production AI voice and chat agents deployed across healthcare, hospitality, real estate, and home services. He writes about agentic AI, LLM engineering, and shipping voice agents that handle real calls in production.
See how AI voice agents work for your industry. Live demo available -- no signup required.
Anthropic's Claude Fable 5 and Mythos 5 explained: pricing, availability, frontier benchmarks, the dual-model safeguard architecture, and what they mean for AI agents.
Where Claude Code, MCP, and multi-agent systems are taking GTM engineering next, and how to prepare your team now for standing and multi-agent workflows.
Where Claude Cowork and the Claude agent ecosystem are heading next — standing agents, MCP, skills as a moat — and the concrete moves to prepare your team now.
The metrics, leading signals, and anti-metrics that prove Claude Cowork is working — acceptance rate, time-to-outcome, and why usage counts mislead.
Shipping an agentic GTM workflow is easy; proving it works is hard. The metrics, signals, and eval loops that show a Claude Code rebuild is paying off.
A realistic end-to-end Claude Cowork use case: a quarterly vendor-spend review from vague ask to shipped deliverable, with every agentic step shown.
© 2026 CallSphere Inc. All rights reserved.
Made within San Francisco
Watch how CallSphere handles real customer calls, schedules appointments, and processes payments — live.
Try Live DemoBook a DemoCalculate Your ROI