Velocity X's Brain runs on Anthropic's Claude API. When you're sending thousands of requests per day across a customer base, the cost model shifts. A $0.003 per-request toy becomes a $90/day line item. At scale, every optimisation compounds. This post covers the patterns we use to run Claude cheaply without tanking quality.
The Naive Cost Model That Breaks at Scale
If you've never scaled an LLM product, you probably think: "Sonnet costs ~$3 per million input tokens, ~$15 per million output tokens. I'll just run Sonnet for everything and move on." This works until you don't. Let's do the math.
A 10-person field team uses Velocity X's Brain for follow-up drafts, lead scoring, and next-best-action routing. 30 visits/day per person = 300 visits/day. Each Brain invocation sends ~2,000 input tokens (visit notes, CRM context, examples) and gets ~300 output tokens (draft email). 300 visits × 2,300 tokens per visit = 690,000 tokens/day. At Sonnet's pricing: ~$2.07/day input + $4.50/day output ≈ $6.57/day ≈ $197/month. Per-customer margin starts disappearing. Add more customers and the math breaks fast. Plus, not every Brain job is equal. Some drafts matter more than others.
Model Tiering — Run Cheap When You Can
The first optimisation is ruthless model selection. Don't run Opus for classification. Don't run Sonnet for grunt work. Use the smallest model that solves the problem.
Haiku (small task routing). Velocity X's Brain starts with Haiku for classification: "Is this a cold prospect or a returning customer? Should the follow-up be brief or detailed?" Haiku costs ~1/10th what Sonnet costs and is fast enough that latency doesn't matter. 300 visits/day × 500 classification tokens = 150,000 tokens/day. At Haiku's ~$0.08 per million input tokens: $0.012/day. Rounding error.
Sonnet (default work). Draft emails, deal summaries, routing logic — these use Sonnet. We've tested Haiku here; the draft quality drops ~30%. Sonnet is the floor for customer-facing work that reps will see and potentially edit.
Opus (high-stakes only). When a deal is above $50k or the rep has marked "I need your best draft", Brain routes to Opus. Slower, ~10× the cost per token, but materially better at handling nuance. A $500k deal justifies $0.05 in API spend. A $2k deal doesn't.
Tiering rule of thumb: if you're running the same model for everything, you're leaving money on the table. Profile your workload, identify the cognitive ceiling for each job type, assign accordingly.
Prompt Caching — Reuse Expensive Context
This is the move that surprised us most in testing. Anthropic's prompt caching feature lets you cache the static parts of your prompt (your brand.json, your few-shot examples, your system instructions) across multiple API calls. Cached tokens cost 90% less than input tokens. If your cached block is 2,000 tokens and you hit it 100 times in a day, that's 200,000 token-days becoming 20,000 token-days. Instant 90% savings on that slice.
Velocity X's Brain caches:
- brand.json — voice rules, tone, phrases to use/avoid. 800 tokens, cached per customer account.
- Few-shot examples — 5 real follow-up email pairs showing "visit notes → good draft". 1,200 tokens, cached per rep.
- System instructions — role definition, output format (JSON), confidence scoring logic. 400 tokens, globally cached.
Total cacheable per request: ~2,400 tokens. If a rep does 30 visits/day, they hit the cache 30 times. Cost: first request pays full price (~$0.007 input), next 29 are ~$0.0007 each (cached). Monthly savings for one rep: ~$5. Scale that across 100 reps and you're saving $500/month from caching alone, with zero quality loss.
Setup is straightforward: wrap your static prompt content in the `cache_control` parameter (Anthropic SDK v0.25+) and Anthropic handles the rest. You don't have to manage cache invalidation; it's automatic on content change.
Batch API — Trade Latency for Savings
Not every Brain job needs instant turnaround. A "send this follow-up tomorrow at 9am" job can wait 24 hours. Anthropic's Batch API runs jobs asynchronously and charges ~50% less. Velocity X uses batching for:
- Scheduled follow-ups — queued at 5pm, processed overnight, results back by 8am next day.
- Bulk re-drafting — a rep says "re-draft my top 20 open deals with the new tone rules" — batches 20 requests, pays half price, gets results in a few hours.
- Analytics-only jobs — scoring historical visit notes for pipeline health reports. No time pressure, batched, 50% savings.
The trade-off is latency. Batch jobs typically return within 24 hours but can take longer during high-load periods. For reps sitting in a car needing a draft now, we use synchronous API. For anything that can wait, batch. Rough split: 60% of our Brain traffic goes through batch (saving 50%), 40% is real-time.
Combining All Three — Real Numbers
Let's model a realistic customer: 10-person field team, 300 visits/day, mixed workload.
Naive (Sonnet for everything): 300 visits × 2,300 tokens = 690,000 tokens/day × $18.15 per million = $12.54/day = $376/month.
With tiering (Haiku + Sonnet + Opus): 150k Haiku tokens (classification) + 450k Sonnet tokens (drafts) + 90k Opus tokens (5% high-stakes) = $1.20 + $7.50 + $1.35 = $10.05/day = $301/month. Savings: ~20%.
With caching (50% of requests hit cache): Same workload, but half the input tokens are cached. Input cost drops from ~$6 to ~$3. New daily cost: ~$6.70/day = $201/month. Savings: ~47%.
With batch (60% of traffic batched): 60% of the remaining costs are halved. $201 → $120/month. Total savings vs naive: 68%. Your 10-person team now costs $120/month instead of $376/month, with the same quality on time-sensitive jobs and better quality on high-stakes deals.
Frequently Asked Questions
Doesn't caching add complexity?
Not much. Anthropic's SDK handles it automatically if you use the `cache_control` parameter. You specify which prompt chunks should be cached, set it once, and forget it. If your context changes, the cache invalidates automatically.
What if I'm already using Sonnet and can't drop to Haiku?
Start with caching. It's a 30-minute integration and saves 50% on static context. Then profile your actual workload — you might find that Haiku works fine for 70% of your classification jobs. Tiering is optional. Caching is not.
How much latency does batching add?
Usually 1-4 hours, but contractually can be up to 24 hours. For applications where "ready by morning" is fine, batch is a no-brainer. For anything user-facing that needs sub-second response, use synchronous API.
Do these patterns work for other providers?
Model tiering works everywhere. Batch API exists on OpenAI and Anthropic but with different performance profiles. Prompt caching is Anthropic-only (and Google Gemini, similar concept). If you switch providers, some techniques port over, some don't.
At what scale does this matter?
A solo maker running 10 requests/day won't notice the difference. A startup with 5 customers at 100 requests/day each might save $20/month. A team of 50+ customers, 5,000+ daily requests? You're talking $3k-$5k/month in savings. Worth the engineering time.
Can I do all three at once?
Yes. This is what Velocity X does. The patterns don't conflict — model tiering is decision logic, caching is static-context optimization, batching is transport-layer optimization. Stack them.
The Bottom Line
Running Claude API at scale looks expensive on the surface. A few pattern changes cut your bill by 50-70% without cutting quality — in fact, tiering often improves quality because you can afford Opus for high-stakes jobs. Start with caching (smallest lift, biggest reward), add tiering next (profile first), and batch whatever can tolerate latency. The numbers compound fast. See the pricing page for Brain's detailed cost breakdown, or read how Brain drafts follow-ups to understand where these patterns apply in practice.