On this page
01
What you'll do
Caching only matters if it pays. In about eight minutes you will read the savings, tell real hits from would-hits, and decide where to cache next.
- Read the Realized Caching Savings card
- Tell the three layers of savings apart
- Compare Monitor only estimates with Serve results
- Find the apps worth caching next
02
Three layers of savings
Three different mechanisms can lower a cost. Keep them apart when you measure.
| Layer | How it saves | Where it shows |
|---|---|---|
| Provider prompt cache | The provider charges less for a repeated prompt prefix | Provider prompt cache on the savings card |
| Cloptima exact cache | An identical request is answered without calling the provider | Cloptima exact cache on the savings card |
| Semantic cache | A similar request is answered without calling the provider | Semantic cache on the savings card |
03
Read the savings card
Open AI → Dashboard and find Realized Caching Savings.
| Line | Meaning |
|---|---|
| Provider prompt cache | Savings from your provider's own prompt caching this month |
| Cloptima exact cache | Savings from requests answered from the exact cache this month |
| Semantic cache (observed) | What semantic matching has found, and its savings once you serve from it |
| Total saved this month | The layers added together |
| Cache read tokens and Cache write tokens | How much prompt text was read from or written to the provider's cache |
| Provider prompt cache | $128.40 |
| Cloptima exact cache | $220.00 |
| Semantic cache (Monitor only) | Monitor only — savings not yet quantified |
| Total saved this month | $348.40 |
| Cache read tokens | 41.2M |
| Cache write tokens | 3.8M |
04
A worked example
Take a classifier that makes 50,000 calls a month at $0.002 each. The numbers below are an illustration.
| Measure | Monitor only estimate | Serve, first month |
|---|---|---|
| Calls that repeat an earlier request | 34,000 (68%) | 31,500 (63%) |
| Cost of those calls at list price | $68.00 | $63.00 |
| Spend without caching | $100.00 | $100.00 |
| Spend with caching | $37.00 | |
| Saved this month | $63.00 |
Real savings landed a little under the estimate, which is normal: the first hours of a cache are empty, and some repeats come after an entry has expired. A gap of this size is healthy. A gap of half or more means answers are kept for less time than the repeats need.
05
Why a hit rate alone can mislead
A high hit rate on cheap calls saves less than a modest hit rate on costly ones.
| App | Hit rate | Cost per call | Monthly calls | Saved |
|---|---|---|---|---|
| Classifier | 63% | $0.002 | 50,000 | $63 |
| Research assistant | 12% | $0.08 | 20,000 | $192 |
Read savings in dollars, then ask where a higher hit rate would be worth the most. The research assistant has the lower hit rate and the bigger saving.
06
Monitor only against Serve
Monitor only estimates what a cache would save. Serve is the real number. Compare them.
- 1
Note the estimate in Monitor only
Use the savings estimate on the exact cache card of a policy bound to one app.
- 2
Switch to Serve on that app
Wait a week of ordinary traffic.
- 3
Compare
If real savings are well below the estimate, check Keep answers for and whether the prompts are stable.
07
Confirm hits in your own tests
The response header lets you check a hit without a dashboard.
| Header value | Meaning |
|---|---|
| x-cloptima-cache: hit | Served from the exact cache |
| x-cloptima-cache: semantic-hit | Served from the semantic cache |
| No header | The request went to the provider |
Send the same request twice. The second should carry the header.
08
Find the next app to cache
Use the Explorer to find where repeated spend lives.
- 1
Group by App
In AI → Explorer, choose a 30d window and sort by Spend.
- 2
Pick apps with repetitive work
Classifiers, FAQ bots, and fixed-prompt pipelines repeat the most.
- 3
Run Monitor only on each
Read its savings estimate before you serve.
09
A monthly review
A short review keeps caching honest.
- Read Total saved this month and compare with last month
- Check each cache's time to live against how fast your content changes
- Look at apps that cost a lot and cache nothing
- Retire a cache that saves little and adds risk
10
If something goes wrong
Most surprises come from mode and scope.
| What you see | Likely cause | Fix |
|---|---|---|
| Cloptima exact cache shows zero | The cache is in Monitor only, or no requests repeated | Check the mode, and the estimate |
| Savings are lower than the estimate | Answers are kept for less time than the repeats need | Keep answers longer in Monitor only and compare |
| Provider prompt cache shows nothing | Prompts do not repeat a stable prefix | Put fixed instructions first and variable text last |
| Semantic cache shows no savings | It is in Monitor only | Switch to Serve when you are ready |