Caveman Mode: Cutting AI Output Tokens 75% Without Losing Accuracy

Look at any frontier pricing table and one thing jumps out: output costs about 6× input. $5 in, $30 out. $10 in, $50 out. Every provider, same shape.

Now consider what your agent spends those output tokens on. Preambles. Restating your question. Explaining what it's about to do before doing it. Apologising. Summarising what you just watched it do. On a normal working day, most of my agent's output was prose I didn't read.

So I compressed it. Roughly 75% fewer output tokens, no loss of technical accuracy. Here's the whole thing.

What it actually looks like

The mode is simple: abbreviations, arrows for causality, fragments over sentences, one word where one word does. Exact paths, exact error strings and exact numbers are preserved absolutely — the compression targets prose, never precision.

// normal — 34 words
"I've finished refactoring the authentication module and all tests
 are now passing after fixing the token expiry bug. Let me know if
 you'd like me to look at anything else."

// caveman — 8 words
auth refactor done · token-expiry bug fixed · tests green

Another, from a real debugging session:

// normal
"I looked into the issue and it appears that the Supabase client is
 hanging when multiple concurrent calls are made to getSession(). This
 is likely related to the navigator locks API. I would suggest
 disabling the auth lock in createClient as a potential fix."

// caveman
root cause: supabase getSession() hangs on concurrent calls
  -> navigator locks deadlock
fix: auth.lock=false in createClient

The second version is more useful, not less. It's scannable, the causality is explicit, and the fix is a line I can act on. The verbose version buries the same three facts in hedging.

Why this matters more than it looks: the re-billing problem

Here's the bit that makes compression compound rather than just save.

Output tokens don't get billed once. Everything the model says becomes part of the conversation, so on the next turn it's re-sent as input. On the turn after that, again. A verbose paragraph on turn three is still being paid for on turn forty.

verbose reply, 400 output tokens
  turn 3:  400 out    @ $30/M
  turns 4-40: 400 in  x 37 turns = 14,800 in @ $5/M

compressed reply, 90 output tokens
  turn 3:   90 out
  turns 4-40:  90 in  x 37 turns =  3,330 in

Same information, roughly a quarter of the lifetime cost. And the second-order effect is bigger than the money: less context consumed means fewer compactions, and compaction is where agents lose the thread on long tasks. Every compaction you avoid is a class of bug you don't get.

The highest-leverage trigger: right before you fan out

If you adopt one rule from this post, make it this one. Compress before dispatching parallel sub-agents.

Sub-agent output flows back into the main context. Five agents each returning a chatty 800-token report is 4,000 tokens of prose in your main window, which then rides along as input on every remaining turn of the session. Five agents returning 150-token structured findings is 750.

The other high-value triggers, in rough order:

Sweeping a large codebase — inventory and audit passes generate enormous amounts of "here's what I found" narration.
Long iterative tool sequences — mass refactors, batch migrations, template application. Per-file commentary × 200 files is the whole budget.
Build/check/build loops — you only need pass/fail and the error string, never a paragraph about what a passing build means.

The four cases where you must switch it off

This is the part that keeps compression from being a footgun. Terse is wrong in exactly four situations, and I switch back deliberately:

1. Security warnings. If something might expose credentials or data, I want full sentences and explicit consequences. Compressed risk warnings get skimmed, and skimming a security warning is how you get an incident.

2. Irreversible-action confirmations. Before a force-push, a migration, a delete: spell out exactly what will be affected. "rm 3 files · force-push main" is not enough information to consent to something you can't undo.

3. Customer-facing copy. Obvious but worth stating — marketing copy, blog posts, support replies. Compression is for the working channel, never the artifact.

4. Explaining to someone you're mentoring. I teach a colleague this stack. Terse output assumes shared context that a learner doesn't have yet, and it strips exactly the reasoning they need to see. Full prose, every time.

Note the shape of that list: compress the working channel, never the deliverable, and never a decision that needs informed consent.

The catch: it's a comms mode, not a thinking mode

I got this wrong initially and it's worth flagging clearly.

Compressing the output is free. Compressing the reasoning is not. If you push terseness so hard that the model stops working through problems properly, you've traded a small token saving for worse answers — a spectacularly bad deal at any price.

The rule I settled on: reasoning stays as long as it needs to be, the report gets compressed. And during genuine debugging — where the value is in watching the chain of inference and spotting the bad assumption — I turn compression off entirely. Clarity of reasoning is the product there.

Also: code, commit messages, PR descriptions and tickets stay written normally. Compressed code comments are just bad code comments, and a terse commit message fails the one job a commit message has.

Make it the default, not a toggle

Last practical note: this only pays if it's the baseline. A compression mode you have to remember to enable gets enabled on maybe one session in five, and always the sessions where you were already thinking about cost — never the runaway ones where it would have mattered.

So put it in your standing rules as the default state, with the four exceptions named explicitly. Then it applies to every session, every sub-agent, every project, without a decision.

The verdict

Output tokens cost 6× input and then get re-billed as input for the rest of the session. Compressing how your agent talks back is the highest-ratio optimisation available: about 75% savings, no accuracy cost, and fewer compactions as a bonus.

Make it the default. Fan-out is the trigger that matters most. Switch to plain English for security, irreversible actions, customer copy and teaching — and never compress the reasoning, only the report.

auth refactor done · tests green · do not @ me.

Want your AI spend audited and the obvious waste cut out? That's most of what I get hired for — start a conversation, or see recent projects.