Prompt and Context Design for Claude Security Agents (Security Program AI Accelerated Offense)

Two security agents can use the same model, the same tools, and the same architecture, and one will be trustworthy while the other quietly makes bad calls. The difference is almost always context design — what you put in front of the model, what you deliberately withhold, and how you frame the rest. In security work this is not a stylistic choice; the wrong context produces confident, wrong, and occasionally dangerous decisions. This post is about getting it right.

The instinct most engineers have is to give the model everything: all the logs, all the policy, the whole asset inventory, the entire threat-intel feed. That instinct is wrong. More context is not more intelligence — past a point it is more noise, more cost, more attack surface, and more opportunity for the model to latch onto the wrong detail. Good context design is mostly about disciplined subtraction.

The job of the system prompt

The system prompt is the agent's standing operating procedure, and it should read like one. It states the single responsibility, names every tool and the precise conditions for using it, defines the evidence standard required before any serious conclusion, and fixes the output format. It also encodes values the model must hold under pressure: calibrated uncertainty, a bias toward escalation when ambiguous, and an absolute prohibition on recommending state changes without corroboration. These are not suggestions buried in prose; they are stated as hard rules with worked examples.

Keep the system prompt stable. It is the part you want cached across every event, so it should not contain anything that changes per alert. Volatile data belongs in a separate, clearly delimited section. This split is both an economic decision — caching the stable prefix slashes per-event cost — and a clarity decision, because the model reasons better when standing instructions and live evidence are visibly separate.

flowchart TD
  A["Candidate context items"] --> B{"Decision-relevant for THIS alert?"}
  B -->|No| C["Leave out: noise, cost, attack surface"]
  B -->|Yes| D{"Trusted instruction or untrusted data?"}
  D -->|Instruction| E["Stable cached prefix: SOP, policy, tools"]
  D -->|Data| F["Delimited evidence block, tagged untrusted + freshness"]
  E --> G["Assembled context window"]
  F --> G
  G --> H["Claude reasons & emits verdict"]

The diagram captures the two questions every candidate piece of context must pass: is it relevant to this decision, and is it instruction or data? Relevance decides whether it goes in at all; the instruction-versus-data distinction decides where it goes and how it is framed. Most context bugs come from skipping one of these questions.

What to put in: decision-relevant evidence, tagged

Include only what changes the verdict for this specific alert. For an impossible-travel sign-in that means the user's normal locations and devices, the risk tier of what they can access, the reputation of the source network, and any recent related events — not the entire authentication history of the org. Tag each item with its source and freshness, because a security decision built on stale intel presented as current is how agents reach confidently wrong conclusions. The model can discount a fact it knows is three days old; it cannot discount one you hid the age of.

Provide just enough policy to make the judgment, not the entire policy library. If this is an identity alert, the identity-response policy is relevant; the data-retention policy is not. Pruning to the decision keeps the window sharp and cheap, and it reduces the chance the model anchors on an irrelevant rule.

What to leave out, and why it matters more than what you include

Leave out raw firehose logs the model cannot meaningfully use — summarize or pre-filter first. Leave out secrets and credentials entirely; the model never needs them to reason, and any token in context is a token that can leak. Leave out other tenants' or users' data unless it is directly relevant, both for privacy and to keep the reasoning focused. And leave out anything you have not labeled as trusted-or-untrusted, because unlabeled external content is exactly how prompt injection slips in.

That last point deserves its own discipline. Any content the agent ingests from the outside world — an email body it is triaging, a webpage pulled during enrichment, an attacker-controlled field in a log — is data, never instructions. Wrap it in clear delimiters, mark it untrusted, and instruct the model explicitly that nothing inside that block can direct its actions. The whole class of "the email told the agent to disable the admin account" attacks dies when external content is structurally framed as inert evidence.

Framing for defensibility and calibration

Ask the agent to produce its reasoning as an evidence-cited chain, not a verdict from the ether. When every conclusion must point to a tagged piece of context, two good things happen: the output is auditable by a human, and the model is far less prone to invent facts. Pair this with an explicit instruction to express uncertainty numerically and to escalate rather than guess when the evidence is thin. A calibrated "I am not sure, here is what I would need" is worth more than a confident verdict that happens to be wrong.

Finally, give the model one worked benign example and one worked malicious example inside the context. Examples calibrate thresholds better than adjectives ever will — "suspicious" means little, but a concrete case of what crosses the line into malicious anchors the model's judgment precisely. Keep the examples canonical and version them alongside the prompt so you can see, in a diff, exactly when and why the agent's calibration changed.

Frequently asked questions

Why not just give the agent all available context?

Because beyond the decision-relevant set, more context adds noise, cost, latency, and attack surface without improving judgment. It also increases the chance the model anchors on an irrelevant detail. Disciplined subtraction produces sharper, cheaper, safer reasoning.

How do I keep external content from acting as instructions?

Structurally frame all ingested external content as untrusted data inside clear delimiters, and instruct the model that nothing within that block can direct actions. Enforce the real boundary server-side too — context framing reduces injection risk, scoped tools and schemas eliminate the impact.

What belongs in the cached prefix versus the per-alert block?

Stable instructions — the SOP, tool rules, policy, and examples — go in the cached prefix. Anything that changes per alert — entity context, intel, the alert payload — goes in a separate delimited block. This split powers prompt caching and keeps reasoning clear.

How does context design reduce hallucination?

By requiring evidence-cited reasoning over tagged context items, the model must ground every claim in something you provided rather than inventing plausible facts. Tagging freshness and source further lets it discount weak evidence instead of treating everything as equally certain.

Bringing agentic AI to your phone lines

The same context discipline — relevant evidence in, untrusted content fenced off, calibrated and cited reasoning out — is what makes CallSphere's voice and chat agents dependable on a live call. Experience it at callsphere.ai.

Source & attribution: This is an independent, original explainer inspired by Anthropic's coverage on the Claude blog. Claude, Claude Code, Claude Cowork, Claude Opus, and the Model Context Protocol are products and trademarks of Anthropic. CallSphere is not affiliated with or endorsed by Anthropic.