Prompt caching

Prompt caching can reduce the cost of repeated input. It reuses a matching prompt prefix, not a previously generated answer. Availability, minimum prompt size, and cache duration depend on the model and provider.

Organize your prompt

Stable prefix: ReuseKeep shared instructions, tool definitions, and reference material at the beginning in a consistent order.
Changing suffix: AppendPut the current question and changing context after the reusable prefix.
Measure: Cache readsInspect reported usage and Activity. A successful response does not necessarily mean there was a cache hit.

Changing models, routes, or prompt prefixes can affect reuse. Do not assume cache state is shared across every provider or that a short test prompt meets the cache threshold.

Anthropic cache breakpoint

For a model supporting Anthropic prompt caching, a cache_control marker identifies the end of a reusable content block. This Messages API example is a request-shape example; replace the short reference text with your actual reusable material. It does not guarantee a cache hit.

import os
from anthropic import Anthropic

client = Anthropic(
    base_url="https://api.straitly.ai",
    api_key=os.environ["STRAITLY_API_KEY"],
)

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=256,
    system=[{
        "type": "text",
        "text": "Answer using this reference: <your reusable reference>",
        "cache_control": {"type": "ephemeral"},
    }],
    messages=[{"role": "user", "content": "Summarize the reference."}],
)
print(response.usage)
for block in response.content:
    if block.type == "text":
        print(block.text)

Use the provider's current documentation for minimum token thresholds, breakpoint limits, and supported TTLs. OpenAI-format requests do not use this exact Anthropic content-block contract.

Read cache usage

The first eligible request may write cache entries; a later matching request may read them. Cache reads and cache writes are different billing buckets.

Inspect cache-read, cache-write, and fresh-input tokens in Activity. When using the Anthropic SDK, inspect the usage fields returned by your selected model; do not infer hits from latency alone.

Request activity Token rates

Cache costs and misses

Cache writes can cost more than fresh input, while cache reads can cost less. Check all token rates, not only input and output.
5-minute and 1-hour cache writes can have different prices. Use the applicable rate when estimating a workload.
On a miss, check model support, prompt size, expiry, and exact prefix changes. Caching is not a general persistence guarantee.
Do not use caching as a data-retention promise. Ask support about the handling requirements of sensitive workloads.

Models & pricing Contact support