Most developers treat OpenAI Prompt Caching as a toggle you flip once and forget about. It is not that. The discount is real and so is the latency drop, but both depend entirely on how you order the strings you send. Get the order wrong and you keep paying full price while wondering why the announcement promised 90% off.
I have spent the last 14 years chasing bottlenecks in WordPress and WooCommerce, and they were nearly always Redis or object caching. In the LLM world the bottleneck moved to token processing, which is what this tutorial is about. If you want the background first, I covered why prompt caching matters separately.
What OpenAI prompt caching actually does
Prompt caching stores the computation from the pre-fill stage of a request. Send the same system prompt or the same RAG extract twice and OpenAI reuses the processed tokens instead of running them through the model again. Those reused tokens cost 90% less, and the response can come back up to 80% faster.
The 1,024 token threshold
The first hurdle: caching only kicks in for prompt prefixes longer than 1,024 tokens. A lean three-sentence system prompt saves you nothing at all. That is why RAG pipeline caching is the usual place people meet this feature, since those context windows get heavy fast.
Trying it out in Python
Here is what it looks like in a real script. The setup is a deliberately bloated system prompt sent twice against gpt-4o-mini, which supports caching with no extra flags.
from openai import OpenAI
import time
client = OpenAI(api_key="YOUR_API_KEY")
# We need at least 1,024 tokens to trigger the cache.
# This is a common "Senior Dev" hack to test cache hits.
long_prefix = """
You are a technical architect specializing in high-scale WordPress environments.
You provide advice on database sharding, object caching, and API optimization.
""" * 150
def make_request(query):
start = time.time()
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": long_prefix},
{"role": "user", "content": query}
]
)
latency = time.time() - start
return response, latency
# First Request (Cache Miss)
resp1, time1 = make_request("How do I fix a race condition in a plugin?")
print(f"First Request Time: {round(time1, 2)}s")
# Second Request (Potential Cache Hit)
resp2, time2 = make_request("What about transient deadlocks?")
print(f"Second Request Time: {round(time2, 2)}s")Run it and the second request comes back noticeably faster. OpenAI hashes the prefix, and when the hash matches something it processed recently, it serves from the cache. Nothing in your code has to change for that to happen. The order of your prompt is what decides it.
How you break the cache without noticing
The mistake I have watched people make a dozen times is putting dynamic data at the beginning of the prompt. Prepend a user ID or a timestamp in front of a 2,000 word system prompt and the cache is gone. OpenAI Prompt Caching matches on the prefix, so anything that shifts the start of the string counts as a brand new prompt.
- Breaks the cache:
"User: 123 | System: [Long Prompt]" - Keeps the cache:
"System: [Long Prompt] | User: 123"
Per the official OpenAI documentation, the system hashes the first 256 tokens and uses that to route your request to a machine that probably already holds your cache. Change the hash and there is nothing left to hit. The OpenAI Cookbook goes deeper into the mechanics.
If this prompt caching work is eating your dev hours, hand it over to me. I have been wrestling with WordPress and API integrations since the 4.x days.
Build prompts for reuse
The habit worth breaking is treating a prompt as a one-off message. Treat it as layers: static, heavy instructions at the top, dynamic user variables at the very bottom. That single refactor is the difference between a $1,000 monthly bill and a $100 one. Portkey’s deep dive is worth a read if you want more on the optimization side.
