Prompt Caching

Prompt caching can reduce repeated input processing for supported models when requests share a reusable prompt prefix.

Caching availability varies by model. Cache hits are not guaranteed, and any latency or cost improvements depend on the selected model's current pricing. Applications should never depend on cache state for functional correctness.

How it works

Some models automatically cache eligible prompt prefixes. When you send a request with a repeated prefix, cached input is reused where available.

Implementation

You do not need to send special parameters to enable caching on supported models. Structure your prompts so that identical, reusable content appears at the very beginning of the request.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

# Long system prompt that stays the same across requests
SYSTEM_PROMPT = """You are a helpful AI assistant with deep expertise in Python.\n[... thousands of tokens of instructions or context ...]\n"""

# First request - full processing
response1 = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Pro",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "How do I use list comprehensions?"},
    ],
)

# Second request - matching prefix may be reused
response2 = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Pro",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "What are Python generators?"},
    ],
)

Best practices

Common use cases

When constructing your prompt architecture, these elements are typically the best candidates for the static prefix:

Prompt cache retention

On supported models, you can keep a prompt prefix cached for 5 minutes or 1 hour. The first request writes the cache at the model's retention write price. Every request within the retention window that sends the same prefix reads it at the cached input price.

Write the cache

{
  "model": "moonshotai/Kimi-K3",
  "messages": [
    { "role": "system", "content": "<long, stable instructions or documents>" },
    { "role": "user", "content": "First question" }
  ],
  "prompt_cache_key": "my-app-session-42",
  "prompt_cache_options": { "mode": "explicit", "ttl": "1h" }
}

Read the cache

Send the same prompt_cache_key and the same prefix, without prompt_cache_options:

{
  "model": "moonshotai/Kimi-K3",
  "messages": [
    { "role": "system", "content": "<long, stable instructions or documents>" },
    { "role": "user", "content": "Next question" }
  ],
  "prompt_cache_key": "my-app-session-42"
}

Rules

• Send prompt_cache_options only on the request that writes the cache. Every request that includes it writes the cache again and is billed at the write price.
• Keep the cached prefix identical and at the start of the prompt. Caching works in whole blocks whose size depends on the model and is shown in its Prompt cache retention table; a prefix shorter than one block is not cached.
• Retention is available on Chat Completions, for models that show a Prompt cache retention table on their model page. It cannot be combined with service_tier.
• Unsupported requests return HTTP 400 with cache_retention_unsupported.

Pricing

A 5-minute write is billed at the model's 5-minute write price, and a 1-hour write at its 1-hour price. Reads are billed at the cached input price and reported in usage.prompt_tokens_details.cached_tokens. A 5-minute write pays for itself from the first read; a 1-hour write from the second.

Checking cache usage

Cache usage metadata may be available when the model returns it. When supported, the response usage object indicates how many tokens were served from cache:

{
  "usage": {
    "prompt_tokens": 5000,
    "completion_tokens": 50,
    "total_tokens": 5050,
    "prompt_tokens_details": {
      "cached_tokens": 4800
    }
  }
}