Claude Prompt & Context Design: What to Keep, What to Cut

Most advice about prompting Claude focuses on what to add: more instructions, more examples, more guardrails. That advice quietly produces bloated, expensive, confused agents. The harder and more valuable skill is subtraction — knowing what to leave out of context and why. A context window is not a backpack you stuff full; it's a working memory you curate. This post is about that curation: what earns a place in front of the model, what should be loaded only on demand, and what should never be there at all.

Context is a budget, not a bucket

Every token in context does two things: it costs money and it competes for the model's attention. A 1M-token window tempts you to dump everything in, but a model swamped in irrelevant history reasons worse, not better — the signal you care about gets diluted by noise you didn't curate. The discipline is to treat context as a budget you spend deliberately, where every block justifies its presence by changing the answer.

The useful question for any candidate piece of context is: would the model's output change if this weren't here? If a 4,000-token document never influences the response, it's pure cost. If a one-line constraint silently fixes a recurring error, it's worth more than its length suggests. Designing context is mostly answering that question, block by block, and being willing to cut the things that don't pass.

What belongs in context

Three categories earn a permanent seat. First, stable instructions — the agent's role, hard constraints, and output format — go in a frozen system prompt that never changes byte-for-byte, so it caches cleanly. Second, the immediate task and the few prior turns the model needs to stay coherent. Third, tool definitions for the actions actually relevant to this agent, with prescriptive descriptions. Everything in this tier shares a property: it changes the model's behavior on most requests, so paying for it on every request is justified.

flowchart TD
  A["Candidate context item"] --> B{"Changes the answer\nmost of the time?"}
  B -->|Yes| C["Keep in context (cache it)"]
  B -->|Sometimes| D{"Loadable on demand?"}
  D -->|Yes| E["Move to a Skill or tool result"]
  D -->|No| C
  B -->|No / secret| F["Leave out entirely"]

What to load on demand instead

A large middle category should not live in context but be reachable from it. Task-specific procedures belong in Agent Skills: a skill's short description sits in context, and the model reads the full instructions only when the task makes it relevant. Reference data belongs behind a tool — let the model search or fetch it when needed rather than pre-loading every document. Rarely-used tools belong behind tool search, so their schemas are appended on demand instead of weighing down the prefix.

This is the progressive-disclosure principle, and it's the single highest-leverage move in context design. It keeps the fixed cost small while preserving capability, because the detail is available the instant it's needed and invisible the rest of the time. An agent with five always-loaded skills' worth of instructions reasons worse than one with five discoverable skills it pulls in only when relevant — same capability, a fraction of the standing context.

What to leave out entirely

Some things should never be in context. Secrets — API keys, tokens, credentials — are the clearest case: they leak into history, listings, and compaction summaries, and they belong at the transport layer, injected after the call leaves the model's context. Volatile noise — timestamps, request IDs, per-turn UUIDs placed early in the prefix — silently shreds your prompt cache; if you need them, push them to the very end. And stale tool output — the thousand-row dump from twenty turns ago that's no longer relevant — should be pruned by context editing rather than carried forever.

The compaction nuance deserves a callout because it's a subtraction that requires care. When the API compacts history to fit the window, it returns a compaction block, and you must append response.content in full — block included — on the next turn. Strip it to just the text and you silently lose the summarized state. Subtraction done by the API still demands discipline from you.

Designing for the cache, not against it

Context layout and caching are the same problem viewed twice. Caching is a prefix match: the stable content must physically precede the volatile content, or no marker will save you. So the design rule falls out naturally — frozen instructions and deterministic tool lists first, then per-session context, then the per-turn question last. Order your context by how often each part changes, most stable to least, and the cache mostly takes care of itself.

Verify rather than assume. If cache_read_input_tokens stays zero across requests that should share a prefix, something volatile slipped forward — a date in the system prompt, an unsorted JSON serialization, a per-user ID interpolated too early. The fix is always to move the volatile piece later or make it deterministic. Good context design is observable: you can watch the cache hits accrue and know your layout is sound.

Tuning the prompt for the model in front of you

Finally, what you write matters as much as what you include. Recent Opus models follow instructions literally and reach for tools conservatively, so prompts tuned for older, more reluctant models often misfire — aggressive "you MUST always" language overtriggers, and missing trigger conditions cause under-calling. Write trigger conditions explicitly ("call X when Y"), state desired verbosity rather than assuming a default, and grant autonomy on small decisions while keeping caution on destructive ones. The cut-versus-keep question applies to your phrasing too: every sentence in the prompt should change behavior, or it's just more noise competing for attention.

Frequently asked questions

If I have a million-token window, why not just include everything?

Because relevance, not capacity, drives quality. Irrelevant context dilutes the model's attention and inflates cost on every request. Include what changes the answer, make the rest reachable on demand, and leave noise out entirely.

Where should I put the current date and user ID?

Not in the system prompt. Both are volatile, and placing them early in the prefix invalidates your prompt cache on every request. Put them in a user-turn message near the end, after your last cache breakpoint.

What's the difference between a skill and just putting instructions in context?

A skill is loaded on demand — its description stays in context, but the full instructions are read only when relevant. Inline instructions are paid for on every request whether the task needs them or not. Skills give you the capability without the standing cost.

How do I keep secrets out of context safely?

Never place credentials in the system prompt or messages — they persist in history and compaction. Inject them at the transport layer (a vault and proxy, or your client's request layer) after the tool call leaves the model's context, so the model orchestrates the call without ever seeing the secret.

Bringing agentic AI to your phone lines

CallSphere applies this same context discipline — keep what changes the answer, load the rest on demand, keep secrets out — to voice and chat agents that stay sharp and fast across long live conversations while pulling in tools and knowledge exactly when a caller needs them. Hear curated context at work at callsphere.ai.

Source & attribution: This is an independent, original explainer inspired by Anthropic's coverage on the Claude blog. Claude, Claude Code, Claude Cowork, Claude Opus, and the Model Context Protocol are products and trademarks of Anthropic. CallSphere is not affiliated with or endorsed by Anthropic.