On this page
01
What you'll set up
Many apps send the same request over and over: a standard prompt, a classification of a repeated input, an embedding of a known text. Each repeat costs money and time for an answer you already have. In about eleven minutes you will turn on the exact response cache, watch what it would save, and then start serving repeats from cache.
- The exact response cache on a policy, in Monitor only
- A savings estimate from your own traffic
- How long answers are kept, to suit your content
- Serve mode, so repeated requests are answered from cache
- A way to see a hit in the response
You need an owner or admin role and a policy bound to your traffic.
02
How the exact cache works
Before a request goes to a provider, Cloptima checks whether it has answered an identical request before under the same rules. If it has, the stored answer comes back and the provider is never called.
1Request arrives
Policy and guardrails applied
2Cache looked up
Same request, same rules?
3Hit
Stored answer returned, no provider cost
4Miss
Provider answers, the answer is stored
Exact means exact. A request must match on the prompt, the system prompt, the tools, and every setting that changes the answer, such as temperature. Cloptima does not guess that two different requests are close enough.
03
Start in Monitor only
Monitor only records every request that would have been a hit and serves nothing. You learn the value of the cache before it changes a single response.
- 1
Open the policy
Go to AI → Policies, open the policy that governs the traffic, and choose Edit.
- 2
Open Caching
Switch on Exact response cache.
- 3
Choose Mode: Monitor only
Nothing is served from cache yet.
- 4
Choose Keep answers for
One hour is a good start. The next section helps you pick.
- 5
Save
Cloptima starts recording would-hits at once.
- 2Exact response cache
- Answers a repeat of an identical request from the cache, without calling the provider.On
- 3Mode
- Monitor only
- 4Keep answers for
- 1 hour
04
Read the savings estimate
For a policy that covers one app, the card estimates what the cache would save on that app's traffic over the last 30 days.
| You see | Meaning |
|---|---|
| Requests that could use the cache | How many of the app's requests in the window matched an earlier one |
| Estimated savings | What those repeats would save in a month, once Monitor only has data |
05
Choose how long answers live
Keep answers for is how long a stored answer may be reused. Pick it from how fast your answers go stale.
| Content | A good choice |
|---|---|
| Static reference answers, embeddings of fixed text | 7 days |
| Classification of repeated inputs | 1 hour to 1 day |
| Answers that depend on changing data | 1 hour, or do not cache |
The choices are 1 hour, 1 day, and 7 days. Your plan sets the longest one offered.
06
Switch to Serve
When the estimate and a few days of Monitor only look good, serve from cache.
- 1
Choose Mode: Serve
Repeated requests are now answered from cache.
- 2
Start with one app
Do this on a policy bound to a single app first.
- 3
Watch the savings
The Dashboard's Realized Caching Savings card shows Cloptima exact cache savings.
To clear stored answers on demand, for example after you correct a wrong answer at the source, choose Invalidate cache on the policy's row in AI → Policies. Confirm, and every stored answer for that policy is dropped. The action is recorded in the audit log.
07
What makes two requests identical
A hit needs everything that shapes the answer to match.
| Part of the request | Must match? |
|---|---|
| The prompt and messages | Yes |
| The system prompt or instructions | Yes |
| Tools offered to the model | Yes |
| Settings such as temperature and top-p | Yes |
| The model and provider | Yes |
| Your organization, credential, and policy version | Always |
| App, team, and environment | Always |
| Caller labels, request ids, and user ids | No. They do not change the answer |
Answers are kept apart per customer and per credential. Another organization can never see them.
08
Recognize a hit
A response served from cache carries a header that says so.
HTTP/1.1 200 OK
content-type: application/json
x-cloptima-cache: hitUse it in tests and logs to confirm the cache is working. A request with no header went to the provider.
09
What is never cached
Some requests should always reach a provider. Cloptima leaves these out for you.
- Streams that did not finish. A completed stream is stored once it ends
- Responses too large to store
- Requests that use tools the provider runs for you
- Models the policy does not allow
- Answers that were blocked or cut short by a guardrail
10
Keep it as code
Cache settings live in Terraform with the rest of the policy.
resource "cloptima_llm_gateway_policy" "classifier" {
name = "classifier"
mode = "enforce"
exact_cache_enabled = true
exact_cache_mode = "enforce"
exact_cache_ttl_seconds = 86400
}11
If something goes wrong
Most surprises are about what counts as identical.
| What you see | Likely cause | Fix |
|---|---|---|
| No hits in Serve | Requests differ in a setting such as temperature or a timestamp in the prompt | Make the prompt and settings stable, or use the semantic cache |
| Hits are low | Answers are kept for a short time, or traffic is mostly unique | Keep answers longer, and check the estimate |
| A response is not cached | Its stream did not finish, or it was too large to store | Expected. Repeats of complete answers still hit |
| Old answers appear | Answers are kept long for changing content | Choose a shorter Keep answers for |