Prompt cache diagnostics

Prompt cache diagnostics help explain why a request reused fewer tokens than expected. Compare a request with an earlier response to identify changes to the model, tools, settings, or input that prevented reuse.

Diagnostics are available in the Responses API for GPT-5.6 and later supported models. Use them to investigate individual requests, and use the Prompt Caching Dashboard to monitor cache performance across your application.

Compare with an earlier response

Set comparison_response_id in prompt_cache_options to the ID of a recent, completed response from the same organization. Choose a response whose prompt prefix you expected the current request to reuse, such as the preceding conversation turn.

The two response IDs serve different purposes:

• previous_response_id carries earlier conversation context into the request. It does not request diagnostics.
• comparison_response_id in prompt_cache_options selects a response as the diagnostic baseline. It does not load that response’s conversation content or restrict which matching cache the request can use. The request can still reuse matching cached content from other requests.

The IDs can refer to the same response or different responses. If you manage history yourself, send it in input and set only the comparison ID.

Diagnostic records are retained for a short period. If the record has expired, the comparison returns comparison_response_not_found, even if the response itself is still available through the API.

The following example continues a conversation using previous_response_id and requests a cache comparison against the same response. It uses a different prompt_cache_key for each request to demonstrate how changing the key can prevent reuse.

Use your own policy document in support-policy.txt. The reusable prefix must meet the model’s minimum cacheable length, which is 1,024 tokens for GPT-5.6 and later.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30from pathlib import Path

from openai import OpenAI

client = OpenAI()
policy = Path("support-policy.txt").read_text()

first = client.responses.create(
    model="gpt-5.6",
    instructions=policy,
    input="Summarize the refund policy.",
    prompt_cache_key="support-policy:request-1",
)

second = client.responses.create(
    model="gpt-5.6",
    instructions=policy,
    previous_response_id=first.id,
    input="Does the policy also apply to sale items?",
    prompt_cache_key="support-policy:request-2",
    prompt_cache_options={
        "comparison_response_id": first.id,
    },
)

diagnostics = second.prompt_cache_diagnostics
if diagnostics is not None and diagnostics.type == "cache_miss":
    print(diagnostics.reason)
    print(diagnostics.comparison_reusable_tokens)
    print(diagnostics.cache_missed_tokens)

If the key change causes a miss, the result may look like this. Token counts vary with the input.

12345678{
  "prompt_cache_diagnostics": {
    "type": "cache_miss",
    "reason": "prompt_cache_key_changed",
    "comparison_reusable_tokens": 1280,
    "cache_missed_tokens": 1280
  }
}

To preserve reuse, use the same key for both requests, such as support-policy:session-42. See Tune prompt cache keys.

See Conversation state for continuation examples. Top-level instructions do not carry over through previous_response_id, so the example supplies the same instructions on both requests.

Multi-turn conversations

To compare consecutive turns, save each completed response’s id and pass it as comparison_response_id in prompt_cache_options on the next request. Omit the comparison ID on the first turn.

When testing a fix, keep the comparison ID set to the baseline response.

Streaming

When stream=True, read prompt_cache_diagnostics from event.response in the response.completed event.

Understand the result

Read prompt_cache_diagnostics.type to determine the comparison outcome.

A request with 2,500 input tokens can report cache_hit if it reuses the comparison response’s 2,000-token prefix. The remaining 500 tokens are new and still need processing.

For a cache miss, comparison_reusable_tokens, when present, is the comparison response’s cached_tokens + cache_write_tokens. cache_missed_tokens estimates how many of the comparison response’s reusable tokens were not reused. Use the current response’s usage fields for actual cache usage and billing.

Diagnostics report the first classified reason, not every difference between requests. After addressing that reason, repeat the comparison to check for other causes.

Fix a cache miss

Some changes, such as switching models or compacting a conversation, are intentional. You may choose to keep them even if they reduce cache reuse.

Confirm the improvement

After making a change:

1. Send another representative request and compare it with the intended baseline.
2. Check the diagnostic result for any remaining difference.
3. Compare cached_tokens, cache_write_tokens, and total cost across several requests.

See Monitor cache performance for usage metrics and cost calculations.

Pricing and rate limits

Prompt cache diagnostics have no additional cost and do not count separately toward rate limits. Any extra baseline or retry requests to the Responses API are billed normally and count toward rate limits.

Zero Data Retention

Prompt cache diagnostics are compatible with Zero Data Retention. OpenAI does not store raw prompts or model outputs for this feature. Diagnostic records contain configuration metadata, token-count estimates, and hashes used to compare cache-sensitive content. These records are scoped to the organization, expire after a short period, and are used only to explain prompt-cache hits or misses.

Setting comparison_response_id does not retrieve or persist the earlier response’s content. See Your data for OpenAI’s data controls.

Limitations

Diagnostics are available in the Responses API for GPT-5.6 and later supported models.

Diagnostics are best effort and may not classify every cache miss. An unavailable result does not indicate whether a cache hit or miss occurred.

Diagnostics never block or fail your request. They do not change how the model generates output. If the comparison is not ready, the response returns unavailable.