Caveman Mode: Cutting AI Output Tokens 75% Without Losing Accuracy

Look at any frontier pricing table and one thing jumps out: output costs about 6× input. $5 in, $30 out. $10 in, $50 out. Every provider, same shape.

Now consider what your agent spends those output tokens on. Preambles. Restating your question. Explaining what it's about to do before doing it. Apologising. Summarising what you just watched it do. On a normal working day, most of my agent's output was prose I didn't read.

So I compressed it. Roughly 75% fewer output tokens, no loss of technical accuracy. Here's the whole thing.

What it actually looks like

The mode is simple: abbreviations, arrows for causality, fragments over sentences, one word where one word does. Exact paths, exact error strings and exact numbers are preserved absolutely — the compression targets prose, never precision.

// normal — 34 words
"I've finished refactoring the authentication module and all tests
 are now passing after fixing the token expiry bug. Let me know if
 you'd like me to look at anything else."

// caveman — 8 words
auth refactor done · token-expiry bug fixed · tests green

Another, from a real debugging session:

"I looked into the issue and it appears that the Supabase client is hanging when multiple concurrent calls are made to getSession(). This is likely related to the navigator locks API. I would suggest disabling the auth lock in createClient as a potential fix."root cause: supabase getSession() hangs on concurrent calls  -> navigator locks deadlockfix: auth.lock=false in createClient

The second version is more useful, not less. It's scannable, the causality is explicit, and the fix is a line I can act on. The verbose version buries the same three facts in hedging.

Why this matters more than it looks: the re-billing problem

Here's the bit that makes compression compound rather than just save.

Output tokens don't get billed once. Everything the model says becomes part of the conversation, so on the next turn it's re-sent as input. On the turn after that, again. A verbose paragraph on turn three is still being paid for on turn forty.

One reply, 37 remaining turns. The output charge is the sliver; the re-billed input is the bill.

Same information, roughly a quarter of the lifetime cost. And the second-order effect is bigger than the money: less context consumed means fewer compactions, and compaction is where agents lose the thread on long tasks. Every compaction you avoid is a class of bug you don't get.

The highest-leverage trigger: right before you fan out

If you adopt one rule from this post, make it this one. Compress before dispatching parallel sub-agents.

Sub-agent output flows back into the main context. Five agents each returning a chatty 800-token report is 4,000 tokens of prose in your main window, which then rides along as input on every remaining turn of the session. Five agents returning 150-token structured findings is 750.

The other high-value triggers, in rough order:

• Sweeping a large codebase — inventory and audit passes generate enormous amounts of "here's what I found" narration.
• Long iterative tool sequences — mass refactors, batch migrations, template application. Per-file commentary × 200 files is the whole budget.
• Build/check/build loops — you only need pass/fail and the error string, never a paragraph about what a passing build means.

The four cases where you must switch it off

This is the part that keeps compression from being a footgun. Terse is wrong in exactly four situations, and I switch back deliberately:

1. Security warnings. If something might expose credentials or data, I want full sentences and explicit consequences. Compressed risk warnings get skimmed, and skimming a security warning is how you get an incident.

2. Irreversible-action confirmations. Before a force-push, a migration, a delete: spell out exactly what will be affected. "rm 3 files · force-push main" is not enough information to consent to something you can't undo.

3. Customer-facing copy. Obvious but worth stating — marketing copy, blog posts, support replies. Compression is for the working channel, never the artifact.

4. Explaining to someone you're mentoring. I teach a colleague this stack. Terse output assumes shared context that a learner doesn't have yet, and it strips exactly the reasoning they need to see. Full prose, every time.

Note the shape of that list: compress the working channel, never the deliverable, and never a decision that needs informed consent.

The system prompt that does it

Compression is a prompt, not a personality. This is the block that produces the output above — and the specificity matters, because "be concise" achieves close to nothing:

## Output style — terse by defaultReport in compressed form. Full technical accuracy, zero prose.RULES- Fragments over sentences. One word where one word does.- `·` separates facts. `->` shows causality. `!` marks a blocker.- Preserve EXACTLY: file paths, line numbers, error strings, numbers, commands.1- Never restate my question. Never preview what you're about to do.- No preamble, no "I hope this helps", no summary of what I just watched.- Uncertainty is one token: `?` after the claim.SHAPE  <what changed> · <what it affects> · <state>  root cause: <cause> -> <mechanism>  fix: <the actual change>  ! <blocker, if any>SWITCH TO FULL PROSE FOR (no exceptions):- security implications or credential exposure- confirming an irreversible action (force-push, migration, delete)- customer-facing copy of any kind- explaining something to someone who is learningReasoning is NOT compressed. Think as long as needed; compress only the report.2

The last line is the one people leave out, and leaving it out is how compression makes a model dumber instead of cheaper. You are compressing the report, never the thinking.

Measure it, don't assume it

I claimed 75%. You shouldn't take that on faith for your own workload, so measure it — the same eval harness shape, scored on tokens instead of correctness:

// eval/compression.ts — same tasks, two output styles
const TASKS = [
  'Summarise what changed in the last commit and whether tests pass.',
  'The login returns 401 on prod but works locally. Report your diagnosis.',
  'List every file that imports the supabase client and flag risky ones.',
];

const styles = { verbose: BASE_SYSTEM, terse: BASE_SYSTEM + '\n\n' + TERSE_BLOCK };

for (const [name, system] of Object.entries(styles)) {
  let out = 0, cost = 0;
  for (const t of TASKS) {
    const r = await run('work', {
      messages: [{ role: 'system', content: system }, { role: 'user', content: t }],
    });
    out += r.usage.outputTokens;
    cost += r.usd;
  }
  console.log(`${name.padEnd(8)} out=${out} tokens  $${cost.toFixed(4)}`);
}

// verbose  out=1483 tokens  $0.0445
// terse    out=356  tokens  $0.0107
// -> 76% fewer output tokens, 76% cheaper, same three answers

Run it on your own three most common asks. If you don't get at least 60%, your baseline prompt was already lean and you have less to gain — which is useful to know before you change how everything talks to you.

Where the savings actually come from

The single-turn saving is the small half. Here's the same 400-token flourish tracked across a session:

The last row is the one that matters most and shows up on no invoice. Compaction is where long agent runs lose the thread — the model summarises its own history and drops the detail that turns out to matter. Every compaction you avoid is a class of bug you don't get, and terse output pushes compaction further away on every single turn.

Where it goes wrong

The right-hand column is not a style preference. A compressed security warning gets skimmed. A compressed irreversible-action confirmation means someone consents to something they didn't fully read. A terse commit message fails the single job a commit message has, which is explaining why to someone six months from now. And terse teaching strips out exactly the reasoning a learner needs to see.

Compress the working channel. Never the deliverable, and never a decision that needs informed consent.

The catch: it's a comms mode, not a thinking mode

I got this wrong initially and it's worth flagging clearly.

Compressing the output is free. Compressing the reasoning is not. If you push terseness so hard that the model stops working through problems properly, you've traded a small token saving for worse answers — a spectacularly bad deal at any price.

The rule I settled on: reasoning stays as long as it needs to be, the report gets compressed. And during genuine debugging — where the value is in watching the chain of inference and spotting the bad assumption — I turn compression off entirely. Clarity of reasoning is the product there.

Also: code, commit messages, PR descriptions and tickets stay written normally. Compressed code comments are just bad code comments, and a terse commit message fails the one job a commit message has.

Make it the default, not a toggle

Last practical note: this only pays if it's the baseline. A compression mode you have to remember to enable gets enabled on maybe one session in five, and always the sessions where you were already thinking about cost — never the runaway ones where it would have mattered.

So put it in your standing rules as the default state, with the four exceptions named explicitly. Then it applies to every session, every sub-agent, every project, without a decision.

The verdict

Output tokens cost 6× input and then get re-billed as input for the rest of the session. Compressing how your agent talks back is the highest-ratio optimisation available: about 75% savings, no accuracy cost, and fewer compactions as a bonus.

Make it the default. Fan-out is the trigger that matters most. Switch to plain English for security, irreversible actions, customer copy and teaching — and never compress the reasoning, only the report.

auth refactor done · tests green · do not @ me.

Want your AI spend audited and the obvious waste cut out? That's most of what I get hired for — start a conversation, or see recent projects.