We’ve gotten a little too focused on token reduction as a measure of whether an AI agent is actually becoming more efficient.
It sounds logical. Cut a tool response from 10,000 tokens to 2,000 and you’d assume the agent gets cheaper. But they don’t work that way.
An agent is constantly searching, reading, reasoning, acting, checking and sometimes going back and doing the whole thing again. So if you remove an important piece of information just to make the response shorter, the agent may have to search for it again, reread files, resolve ambiguity or rerun a patch or test. The saving shows up in one place on your dashboard. The cost shows up three steps later, where nobody’s looking.
The evidence is worse than the intuition
There’s some pretty interesting evidence here. A paper called Token Reduction Is Not Cost Reduction looked at 2,908 invoiced Claude Code executions and found that tool output was reduced by 38.4%, while the actual cost increased by 6.8%.
Worth being clear about what that study is. It ran entirely inside Claude Code with unmodified Claude Code as the baseline, and the measurement plan was locked in before a single session ran, which is more discipline than most cost claims in this space get. It was also authored at PointFive and evaluates their own experimental build alongside third-party tools, so it isn’t independent research. Read it with that in mind. The direction of the finding still holds up, and the mechanism behind it is the part that matters.
That mechanism is the prompt cache. Cache traffic dominated the cost composition, accounting for roughly 87% of reconstructed cost and about 80% of the actual bill. Which means most of what you’re paying for isn’t fresh tokens at all. It’s the repeated prefix of a conversation that the provider is caching for you at a discount.
Compression breaks that. Every time a compressor rewrites a tool output mid-trajectory, the prefix after that point no longer matches what was cached. You’ve invalidated the cheap part of your bill to shave the expensive part, and the expensive part was never the bulk of it. The study found exactly the pattern you’d expect from this: the harder a tool compressed, the more it cost. The one intervention that didn’t make agents more expensive was the one that barely compressed at all.
And that’s the part that’s easy to miss. Saving tokens in one step doesn’t necessarily mean saving money across the whole task. If the agent needs those tokens to make the right decision, removing them just creates more work somewhere else.
Compression isn’t the problem, blind compression is
I’m definitely not saying compression is bad. There are cases where it works really well, especially when you’re removing irrelevant information rather than useful context.
The distinction is doing real work there. Truncating a 40,000-line dependency install log down to the errors is removing noise. That log was never going to inform a decision. Stripping the surrounding lines out of a stack trace so only the exception message survives is removing signal, because the frames were the thing that told the agent which file to open. Both look identical as a percentage. One is free. One costs you a re-read, a re-search, and a fresh cache miss.
The trouble is that compressors run at the tool boundary, where they can see the size of the output but not what the agent was about to do with it. They’re optimising a number they can measure against a need they can’t.
Measure the thing you actually care about
So instead of measuring the percentage of tokens saved, measure the actual cost of each successful task. Success rate, retries, repeated searches, tool calls, memory traffic, latency and the cost of the compressor itself all matter.
That last one gets forgotten constantly. If your compressor is an LLM call, you’re paying for inference to decide what to throw away, on every tool response, forever. It has to earn that back before it breaks even, and the paper suggests it often doesn’t.
The rest of the list is there because it captures the work that gets displaced. A retry is a whole extra trajectory. A repeated search is a tool call plus the tokens it returns plus the reasoning to interpret them. If you’re only counting what left the context, none of that is visible to you.
Forty percent fewer tokens doesn’t mean much if the agent takes twenty percent longer and costs more to get the same result.