On this page
01
What you'll set up
A cache that is too cautious saves little. One that is too loose shares what it should not. Cloptima takes care of safety, so tuning is about hit rate. In about seven minutes you will see what the cache keeps apart, shape requests so repeats match, and pick how long answers live.
- An understanding of who shares an answer
- Requests shaped so their repeats match
- How long answers are kept, per policy
- A routine to raise the hit rate safely
You need an exact cache already running, at least in Monitor only. If not, start with the guide on caching identical requests.
03
Shape requests so repeats match
A hit needs an identical request. Small differences turn a repeat into a miss.
- Keep fixed instructions in the system prompt, word for word
- Leave timestamps, request ids, and user names out of the prompt
- Use the same temperature, tools, and settings for the same task
- Send the same model name every time
04
Sensitive data
Guardrails redact a request before the cache sees it. There is nothing to set.
- The cache stores the masked answer, never a redacted value
- A hit gets the new request's own values back in place of the placeholders
05
Streamed responses
Streamed replies reach the user as they are written. Cloptima stores the finished answer once the stream completes, so a repeat of the request can hit.
06
What the cache covers
The cache follows the policy. Every model the policy allows is cacheable.
- Every model the policy allows, from every provider that serves it
- Requests that use tools the provider runs for you are never cached
- Responses too large to store are returned normally
- A slow or unavailable cache never blocks a request. It simply behaves as a miss
07
Choose how long answers live
Keep answers for is the one setting that trades hit rate against freshness.
| Keep answers for | Use it for |
|---|---|
| 1 hour | Answers that depend on data that changes during the day |
| 1 day | Classification of repeated inputs, daily reports |
| 7 days | Static reference answers, embeddings of fixed text |
Your plan sets the longest length offered.
08
When the cache starts fresh
Answers must never outlive the rules that produced them.
- Changing the policy starts the cache fresh for that policy
- Changing a guardrail profile does the same, because the profile is part of the key
- Keep answers for always applies, so old answers expire on their own
- Invalidate cache, on the policy's row in AI → Policies, clears every stored answer for that policy at once
09
A tuning routine
Change one thing at a time, in Monitor only.
- 1
Note the estimate
Read the savings estimate on the policy's cache card.
- 2
Change one thing
Make a prompt stable, or keep answers longer.
- 3
Wait a few days
Compare the estimate.
- 4
Keep what helps
Undo what did not.
10
If something goes wrong
Most surprises come from requests that differ.
| What you see | Likely cause | Fix |
|---|---|---|
| Two apps do not share answers | Each app keeps its own answers | Expected. Each app still saves on its own repeats |
| Repeats never hit | A timestamp, id, or setting differs between them | Make the prompt and settings stable |
| A streamed request did not hit | The earlier stream did not finish | Expected. The next complete answer is stored |