Prompt and context design for Claude agents that work (Founders Playbook AI Native Startup)

Two teams build the same Claude agent with the same tools and the same model. One agent is sharp and reliable; the other is vague and erratic. The difference is almost never the model — it is what each team puts in the context window, and just as importantly, what they leave out. Prompt and context design is the highest-leverage skill in building AI-native products, and it is mostly about subtraction. This post is about deciding, deliberately, what belongs in context on every turn and why the things you exclude matter as much as the things you include.

Context is a budget, and attention is the scarce resource

The instinct of every engineer new to agents is to add more: more instructions, more examples, more retrieved documents, more history. It feels safer. It is the opposite of safe. A model's attention is finite, and past a certain point additional context degrades performance — the signal you care about gets diluted by material that merely seems relevant. The right mental model is a budget: every token you spend competes with every other token for the model's focus, so you spend deliberately and you cut aggressively.

Context design, defined plainly, is the practice of deciding exactly what information a model sees on each turn so it has what it needs to act well and nothing that distracts it. That definition contains the whole discipline. "What it needs to act well" forces you to be specific about the task; "nothing that distracts it" forces you to delete. The teams that internalize both halves build agents that stay sharp as they scale; the teams that only do the first half build agents that get slower and dumber with every feature.

The four things that almost always belong

Some content earns its place on nearly every turn. First, the system prompt: identity, hard rules, and operating constraints, written concretely. Second, the tool definitions: names, descriptions, and schemas, since the agent reasons about its options from these. Third, the task-relevant memory: the durable facts about this specific user or job that change how the agent should behave. Fourth, the immediate working context: the current request and the recent turns needed to understand it.

flowchart TD
  A["New turn"] --> B["Stable prefix: system prompt + tools (cached)"]
  B --> C["Add durable memory for this user"]
  C --> D{"Need external knowledge?"}
  D -->|Yes| E["Retrieve top 1-3 relevant docs"]
  D -->|No| F["Skip retrieval"]
  E --> G["Add recent conversation only"]
  F --> G
  G --> H["Send minimal sufficient context to Claude"]

Notice that the stable prefix — system prompt and tool definitions — sits at the front. That ordering is not cosmetic. Putting the unchanging material first lets you use prompt caching so the model does not re-process it on every call, which cuts both latency and cost meaningfully at volume. Good context design and good unit economics turn out to be the same discipline viewed from two angles.

The things to leave out — and why

Now the harder half. Leave out full conversation history once it grows long; replace it with a running summary plus the last few turns. The model rarely needs turn forty verbatim, and replaying it crowds out what matters. Leave out documents that merely match a keyword but do not bear on the current task; retrieval that returns thirty marginally-relevant chunks is worse than retrieval that returns the three that actually answer the question. Leave out instructions for situations that are not happening — a giant prompt covering every edge case the agent might someday face dilutes the rules for the case in front of it right now.

The principle underneath all of these is sufficiency, not completeness. You are not trying to give the model everything that could conceivably be useful; you are trying to give it the minimal set that lets it act correctly on this turn. When you are unsure whether something belongs, the default should be to leave it out and add it back only if you can show the agent fails without it. This is the opposite of how most engineers reason about code, where more information is usually harmless, and it takes deliberate practice to invert.

Retrieval is context design, not a database query

When your knowledge base outgrows the context window, you retrieve — but retrieval is a context-design decision, not just a search. The quality lever is not how much you fetch but how relevant and how well-scoped what you fetch is. Retrieve the few passages that genuinely bear on the task, and pass them in a clean, labeled form so the model knows what it is reading. A common failure is dumping raw search results into context and hoping the model sorts them out; a better pattern is to retrieve, then compress to the parts that matter, then include only those.

For larger jobs, subagents become a context-design tool in their own right. A subagent can read a large source in its own fresh window and return a tight summary to the orchestrator, so the main agent's context holds the distilled conclusion rather than the raw material. This is one of the most effective ways to handle volume without blowing the budget — you push the bulky reading into isolated windows and keep only the conclusions where they are needed.

Write the briefing, then cut it in half

The practical workflow I recommend is to draft the context the way you would brief a sharp colleague who reads it once and acts: state the goal, give them the few facts that matter, hand them the tools, and stop. Then go back and cut it in half. Almost every first draft of an agent's context is too long, and almost every edit that improves an agent is a deletion. Measure the result with your eval set — change one thing, run the cases, keep what helps. Context design is empirical, and the teams that treat it that way, tightening relentlessly while watching real outcomes, end up with agents that feel uncannily reliable while everyone else wonders why theirs drifts.

Frequently asked questions

Is a bigger context window a substitute for context design?

No. A large window lets you include more, but including more is usually the wrong move — attention is still finite, and irrelevant context degrades reasoning. The window raises the ceiling on what fits; it does not change the discipline of including only what helps.

How much conversation history should I keep in context?

Keep the recent turns needed to understand the current request, and replace older history with a running summary. Replaying long verbatim history crowds out the system prompt, tools, and task facts that matter more.

What is the most common context-design mistake?

Over-inclusion — stuffing in every instruction, document, and past message "just in case." It dilutes the model's attention, raises cost and latency, and makes behavior less reliable, not more. When in doubt, leave it out and add back only if the agent demonstrably needs it.

How does retrieval relate to context design?

Retrieval is a context-design decision. The goal is relevance and scoping, not volume: fetch the few passages that bear on the task, compress them, and include only those. Dumping raw search results into context is a frequent cause of flaky behavior.

Sharp context, on every conversation

CallSphere applies this same context discipline to Claude-powered voice and chat agents — minimal, sufficient context per turn so they stay sharp across thousands of live calls. Hear the difference disciplined context makes at callsphere.ai.

Source & attribution: This is an independent, original explainer inspired by Anthropic's coverage on the Claude blog. Claude, Claude Code, Claude Cowork, Claude Opus, and the Model Context Protocol are products and trademarks of Anthropic. CallSphere is not affiliated with or endorsed by Anthropic.