Prompt Caching Strategies to Cut LLM API Costs

In a typical AI workflow you pass the same input tokens to the model over and over: the same system prompt, the same tool definitions, the same long document corpus. Every repetition is money you did not need to spend. Prompt (context) caching exists to turn that repeated prefix into a ~90%-discounted lookup.

The Gemini API offers two caching mechanisms, and choosing between them is the first strategy decision you’ll make.

Prerequisites

• Python 3.10+
• A Gemini API key (GEMINI_API_KEY)
• google-genai installed

Implicit vs explicit caching

• Implicit caching is automatically enabled on Gemini 2.5 and newer models. If a request hits an existing cache, the system passes the cost saving on — but there is no guarantee you’ll hit, because caching is driven by common prompt prefixes and system load. Zero developer work, zero guarantee.
• Explicit caching is a cache object you create with a Time-To-Live (TTL). You declare the content once, then reference it in later prompts. You guarantee the discount (90% on Gemini 2.5+), in exchange for a bit of developer work and storage costs.

There is a minimum token count to cache (2,048 for Gemini 2.x models; 4,096 for the Gemini 3 family). Below that, the cache isn’t worth billing.

How explicit caching is billed

Three factors drive the cost:

1. Cache creation — the input tokens that seed the cache are billed at the standard input token rate (a one-time write).
2. Cached reads — tokens you reference from the cache are billed at the discounted rate, about 10% of standard input. In real pricing terms a $1.50/1M input drops to ~$0.15/1M.
3. Storage (TTL) — you pay an hourly rate per million tokens stored, prorated down to the minute. The default TTL is 1 hour; you can set anything from 1 minute to unbounded.

The discount roughly breaks even at modest volumes: a 100k-token cache written once and read hundreds of times is dramatically cheaper than re-sending those tokens on every request. The read count is what makes caching pay.

Creating an explicit cache

from google import genai
from google.genai import types

client = genai.Client()

# 1. Cache the stable content ONCE with a TTL
system_docs = "A 50,000-token product manual, repeated for every support query..."

cache = client.caches.create(
    model="gemini-2.5-pro",
    config=types.CreateCachedContentConfig(
        display_name="product-manual-v1",
        ttl="3600s",  # 1 hour
        contents=[system_docs],
    ),
)
print("cached tokens:", cache.usage_metadata.total_token_count)

Referencing the cache in requests

Now subsequent requests point at the cache instead of resending the corpus. The cached content is a prefix to your prompt — nothing between it and the model changes.

resp = client.models.generate_content(
    model="gemini-2.5-pro",
    contents="What is the return policy?",
    config=types.GenerateContentConfig(cached_content=cache.name),
)
print(resp.usage_metadata)
# cachedContentTokenCount appears here so you can measure hit rate

The number of cached tokens shows up in usage_metadata.cached_content_token_count on the response — that field is your cache-hit SLI.

Strategy: structure prompts for cache topology

Caching is not free to trigger. Any change to an earlier part of the prompt invalidates the cache for everything after it. So the layout of your prompt is a cost decision:

• Stable content first: system prompt, tool definitions, static docs. These hit the cache on every call.
• Dynamic content after: the user’s new message, freshly retrieved chunks, current date.

If your assembly engine inserts a freshly-retrieved chunk before the system prompt, every single call misses the cache for the entire prompt. This single ordering rule is the difference between ~60–80% cost savings and nothing.

Region	TTL	Contents	Cache behavior
System prompt + tools	Long (1h)	Instructions, function schemas	Hits every call
Corpus context	Medium	Retrieved docs, manuals	Hits for follow-ups
Conversation tail	Short (5m)	Last assistant message	Grows each turn
User input	None	The new message	Never cached

For chat and agent loops, cache the growing conversation prefix too — each turn’s input is “everything before + new message,” and caching that prefix with a short TTL makes the whole session progressively cheaper.

Putting It All Together

A client wrapper that hides caching so business logic never thinks about it:

class CachedClient:
    def __init__(self, model: str, ttl: str = "3600s"):
        self.model = model
        self.cache = client.caches.create(
            model=model,
            config=types.CreateCachedContentConfig(ttl=ttl, contents=[self._system()]),
        )

    def ask(self, question: str):
        return client.models.generate_content(
            model=self.model,
            contents=question,
            config=types.GenerateContentConfig(cached_content=self.cache.name),
        )

Conclusion & Next Steps

Prompt caching is the single largest cost lever most applications are leaving on the table: a guaranteed 90% discount on the tokens you send most often, if you structure prompts for cache topology and pick the right TTL. Next: watch cached_content_token_count like an error rate, refresh caches when your corpus changes (bump a version in the display name), and model TTL storage costs against recreation cost so you aren’t paying hourly storage for content you update daily.

References / Sources

• Gemini API context caching guide. https://ai.google.dev/gemini-api/docs/generate-content/caching
• Gemini API pricing. https://ai.google.dev/gemini-api/docs/pricing
• Vertex AI context caching. https://cloud.google.com/blog/products/ai-machine-learning/vertex-ai-context-caching
• Context caching API reference. https://ai.google.dev/api/caching