On this page
01
What you'll set up
People rarely ask the same question the same way. 'How do I reset my password' and 'I forgot my password, what now' need the same answer, and an exact cache treats them as different. The semantic cache matches by meaning. In about eleven minutes you will turn it on, choose who pays for its embeddings, and move from Monitor only to serving.
- The semantic cache on a policy, in Monitor only
- How long answers are kept
- Funding for the embeddings that make matching possible
- A rollout from Monitor only to Serve
You need an owner or admin role and a policy bound to your traffic.
02
Exact or semantic
Use both. They solve different problems.
Exact cache
- Matches identical requests only
- No extra cost to look up
- Best for fixed prompts and repeated inputs
Semantic cache
- Matches requests that mean the same thing
- A lookup uses a small embedding call
- Best for questions people word differently
03
How a match is judged
A stored answer is reused only if it passes every check. Each one protects you from a bad match.
| Check | Why it matters |
|---|---|
| Same organization | Answers never cross organizations |
| Same app | An answer written for one app is never reused by another |
| Same model and provider | The answer was written by the model you are now calling |
| Same tools and settings | A different tool list or temperature can change the answer |
| Redacted values it can restore | A stored answer that holds redacted values is reused only when each one belongs to the new request |
| Similar enough | The match must be close. Cloptima sets the bar, so a loose match is never served |
| Fresh enough | The stored answer must be newer than Keep answers for |
| Checked by your guardrails | An answer stored under another version of your guardrails is checked again before it is served |
04
Turn it on in Monitor only
Monitor only records the matches Cloptima would have made. Nothing is served.
- 1
Open the policy
Go to AI → Policies, open the policy, and choose Edit.
- 2
Open Caching
Switch on Semantic response cache.
- 3
Choose Mode: Monitor only
Nothing changes for your users.
- 4
Choose Keep answers for
Start at 1 day.
- 5
Choose Paid by
Cloptima credits, or one of your embeddings keys.
- 6
Save
Matches start being recorded.
- 2Semantic response cache
- Answers a request that means the same as an earlier one from the cache.On
- 3Mode
- Monitor only
- 4Keep answers for
- 1 day
- 5Paid by
- Cloptima credits
05
Fund the embeddings
Each lookup embeds the request. Your embeddings key or Cloptima credits pay for it. Without either, semantic caching is skipped and requests go straight to the provider.
| Paid by | How it works |
|---|---|
| Cloptima credits | Embeddings are paid from your credits at list price |
| Your embeddings key | Embeddings use your Vertex AI or self-hosted embeddings key, in the cloud or on your edge gateway |
06
Monitor only, then serve
Two modes take you from learning to serving.
| Mode | What happens |
|---|---|
| Monitor only | Matches are recorded. Nothing is served |
| Serve | Matches are served from cache |
You choose when to move. A response served from the semantic cache carries x-cloptima-cache: semantic-hit.
07
Roll it out
Raise your confidence in steps.
- 1
Monitor for a week
Read real matches and check that they would have been right.
- 2
Serve for one app
Start with traffic like FAQs, on a policy bound to that app.
- 3
Widen
Turn it on for more apps as the results hold.
08
If something goes wrong
Most questions are about funding and scope.
| What you see | Likely cause | Fix |
|---|---|---|
| No matches in Monitor only | Traffic is unique, or nothing pays for the embeddings | Check Paid by, and add credits or an embeddings key |
| Another app's answers are not reused | Each app keeps its own answers | Expected. Turn the cache on for each app that repeats |
| Matches vanish after a model change | Answers are tied to the model that wrote them | Let the cache refill on the new model |
| Answers expire sooner than you want | Keep answers for is short | Choose a longer length |