When to use Claude prompt caching and when not to

Most writing about prompt caching is cheerleading: turn it on, save 90%, done. The honest version is more interesting, because caching is a trade with real losing cases. There are workloads where enabling it raises your bill, workloads where it adds complexity for negligible gain, and workloads where a completely different lever — batching, a smaller model, or just a shorter prompt — would serve you better. Knowing when not to cache is as valuable as knowing how, because the wrong call quietly taxes every request. This post gives you the decision criteria and the alternatives, with the trade-offs stated plainly.

Key takeaways

• Cache when you have a large stable prefix reused at least twice within the TTL; skip it otherwise.
• Caching loses money on one-shot calls, per-request-varying prefixes, and prefixes below the cacheable token minimum.
• For non-latency-sensitive bulk work, the Batches API's flat 50% discount often beats the complexity of caching.
• Sometimes the right move is a shorter prompt or a cheaper model, not caching a bloated one.
• Caching and batching compose — you can cache a shared prefix inside a batch for both discounts at once.

The cases where caching clearly wins

Caching pays off when one condition holds above all others: a substantial prefix stays byte-identical across many requests within the cache's time-to-live. The textbook example is a conversational agent or assistant with a large frozen system prompt and tool set, where each turn appends a small question. The prefix is written once and read on every subsequent turn at roughly one-tenth the input price, and the latency drop from skipping prefill is felt directly by an interactive user. Retrieval-augmented setups with a fixed instruction block and shared few-shot examples followed by a varying query are the same shape.

The defining definition to anchor on: prompt caching is worth enabling when the reused prefix is both large enough to clear the model's cacheable minimum and read often enough that the cumulative read savings exceed the one-time write premium. If both halves of that sentence are true, cache. If either is false, the rest of this post is about what to do instead.

The cases where caching loses

The clearest losing case is the one-shot request: a prefix written once and never read pays the ~1.25x write premium for zero benefit, making it strictly more expensive than not caching. Equally bad is the prefix that varies every request — if the first kilobyte of your prompt changes per call because it contains a timestamp, a per-request ID, or freshly assembled content, there is no reusable prefix and every request is a fresh write. A third trap is the sub-minimum prefix: below roughly 4096 tokens on Opus and Haiku 4.5 (2048 on Sonnet 4.6), the prefix silently won't cache at all, so you get no error and no benefit while believing caching is on.

flowchart TD
  A["Workload"] --> B{"Latency-sensitive?"}
  B -->|No, bulk| C["Batches API > flat 50% off"]
  B -->|Yes| D{"Large stable prefix reused 2+ times in TTL?"}
  D -->|Yes| E["Prompt caching"]
  D -->|No| F{"Prefix below cacheable minimum?"}
  F -->|Yes| G["Shorten prompt or skip caching"]
  F -->|No, varies per request| H["No caching > restructure or drop"]
  C --> I{"Shared prefix across batch items?"}
  I -->|Yes| J["Cache inside the batch > both discounts"]

There is also a complexity trade-off that does not show up on the bill. Caching introduces a fragile invariant — the byte-stable prefix — that every future change to your prompt code must respect. For a workload where the savings are marginal, that ongoing maintenance cost can exceed the dollars saved. Honesty requires admitting that sometimes the right answer is to skip caching not because it would lose money but because it would not save enough to justify the discipline it demands.

The main alternative: the Batches API

When latency does not matter — overnight document processing, bulk classification, large-scale extraction — the Batches API is frequently the better lever. It processes requests asynchronously at a flat 50% discount on all token usage, with no prefix-stability requirement and no per-request invalidation risk. You submit up to many thousands of requests, most complete within an hour (maximum 24), and you pay half price across the board. There is no cache to break, no byte-identical prefix to maintain, and no minimum-prefix gotcha.

The choice between batching and caching comes down to two axes: latency tolerance and prefix reuse. If you can tolerate asynchronous turnaround, batching gives you a guaranteed 50% with near-zero engineering risk. If you need real-time responses, batching is off the table and caching becomes the primary cost lever. They are not mutually exclusive — a batch whose requests all share a large preamble can carry a cache_control breakpoint on that preamble and earn the cache discount on top of the 50% batch discount, which is the best of both for high-volume shared-context jobs.

Other alternatives worth weighing

Before caching a giant prompt, ask whether the prompt should be giant at all. A 50K-token system prompt stuffed with rarely-relevant instructions is a candidate for trimming, or for Agent Skills and tool search that load detail on demand rather than holding it all in the fixed prefix. A smaller prompt is cheaper on every single request, cached or not, and it sidesteps the cacheable-minimum question entirely for the parts you remove. Caching a bloated prompt optimizes the wrong thing; shrinking it optimizes the root cause.

Model choice is the other lever. If a task runs fine on a cheaper, faster model, moving it there can save more than caching would on the expensive model — and the two stack, since you can cache on the cheaper model too. The honest framing is that caching is one tool among several, and the discipline is to reach for the one that matches the workload rather than defaulting to caching because it is the most talked-about.

Common pitfalls

• Caching reflexively. Enabling caching on every workload regardless of reuse pattern means paying write premiums on one-shot and per-request-varying calls. Check reuse first.
• Missing the batch alternative. Teams cache aggressively on latency-insensitive bulk jobs when a flat 50% batch discount would be simpler and often cheaper.
• Caching below the minimum. A 3K-token prefix on Opus silently won't cache. Either grow the prefix above the threshold deliberately or accept it won't cache and don't rely on it.
• Not stacking discounts. Running a batch with a shared preamble but no cache breakpoint leaves the cache discount on the table. Add the breakpoint and earn both.
• Optimizing a prompt that should be shorter. Caching a bloated 50K prompt instead of trimming it to 10K optimizes the wrong variable. Shrink first, then cache the remainder if it still pays.

Pick the right lever in five steps

1. Classify the workload by latency tolerance — real-time or batch-friendly.
2. If batch-friendly, default to the Batches API for the flat 50% and skip caching unless there's a shared preamble.
3. If real-time, measure your reusable prefix size and reuse count within the TTL.
4. Cache only if the prefix clears the minimum and is read at least twice; otherwise shorten the prompt or change models.
5. Where a batch shares a large preamble, add a cache breakpoint to stack both discounts.

Stacking caching inside a batch

This shows the both-discounts pattern: a shared system block carries the cache breakpoint, and the whole thing is submitted as a batch so every request also earns the 50% batch reduction.

shared_system = [
    {"type": "text", "text": "You are a document analyst."},
    {"type": "text", "text": LARGE_SHARED_PREAMBLE,
     "cache_control": {"type": "ephemeral"}},  # cached across batch items
]
# each request reuses shared_system; batch adds a flat 50% on top
batch = client.messages.batches.create(requests=[
    Request(custom_id=f"doc-{i}",
            params=MessageCreateParamsNonStreaming(
                model="claude-opus-4-8", max_tokens=1024,
                system=shared_system,
                messages=[{"role": "user", "content": q}]))
    for i, q in enumerate(questions)
])

Caching vs the alternatives

Situation	Best lever	Why
Real-time, large reused prefix	Prompt caching	~0.1x reads + lower latency
Bulk, latency-insensitive	Batches API	Flat 50%, no prefix to maintain
Bulk + shared preamble	Both	Cache discount stacks on batch
One-shot call	Neither	Caching is a pure write loss
Bloated prompt	Shorten first	Cheaper on every request

Frequently asked questions

When should I not use prompt caching?

Skip it for one-shot calls (the write premium buys nothing), for prompts whose prefix changes every request (no reusable prefix), and for prefixes below the model's cacheable minimum (they silently won't cache). Also skip it when the savings are too marginal to justify maintaining a byte-stable prefix across all future prompt changes.

Is the Batches API better than caching?

For latency-insensitive bulk work, often yes — it gives a flat 50% discount with no prefix-stability requirement and no invalidation risk. For real-time workloads it isn't an option, so caching becomes the lever. They also compose: a batch with a shared preamble can cache that preamble and earn both discounts.

Can I use caching and batching together?

Yes. Put a cache_control breakpoint on the shared portion of the requests and submit them as a batch. The cache discount applies to the reused prefix and the 50% batch discount applies to all token usage, so high-volume shared-context jobs get the best of both.

Should I cache a very large system prompt?

First ask whether it should be that large. Trimming rarely-relevant instructions, or moving them into Skills and tool search that load on demand, makes every request cheaper regardless of caching. Cache the remainder only if it still clears the minimum and is reused enough to beat the write premium.

Bringing agentic AI to your phone lines

CallSphere brings these same deliberate agentic trade-offs to voice and chat — multi-agent assistants that answer every call and message, use tools mid-conversation, and book work 24/7. See the live system at callsphere.ai.

---

Source & attribution: This is an independent, original explainer inspired by Anthropic's coverage on the Claude blog. Claude, Claude Code, Claude Cowork, Claude Opus, and the Model Context Protocol are products and trademarks of Anthropic. CallSphere is not affiliated with or endorsed by Anthropic.