Cheap Models as Always-On Watchers: The Pattern Nobody Uses

I audited my own AI usage a while back and found something embarrassing. I'd been running a frontier model on a task that is, functionally, grep with opinions: watching a dev-server log for lines that mean something broke.

That's not a hard problem. It needs no reasoning, no architecture, no taste. It needs a thing that never blinks. And the model I want for "never blinks" is the cheapest, fastest one available — not the one I'd trust with a schema migration.

The split: the expensive model does the thinking, the cheap model does the staring. Once I framed it that way, a whole class of always-on tooling became basically free to leave running.

The problem it solves: you find out too late

Here's the failure this fixes, and it's the one that costs the most trust.

An agent makes an edit. The edit breaks the build. The agent doesn't notice, keeps working, makes three more edits on top, then reports success. You open the browser and find a white screen. Now you're debugging four changes at once instead of one, and the agent's status report has become actively misleading.

Or worse — the version I actually lived. The console errors and the wrong-page problem were both discovered by me, after handover. Which means every "done" now requires my independent check, which is exactly the work I was trying to delegate.

Continuous log monitoring is the standard fix in ops: watch for error patterns in real time so an issue gets caught before it compounds. It's just that almost nobody applies it to their own dev loop.

The money pattern: a filtered tail on a cheap model

The core of it is unglamorous, which is why it works:

tail -f dev.log | grep -Ei 'error|failed|cannot|undefined|expected corresponding'

Five words, chosen from real experience rather than theory:

• error / failed — the obvious ones.
• cannot — catches "Cannot find module", the single most common break after a dependency change or a bad import path.
• undefined — catches the runtime class that builds cleanly and dies on hydration.
• expected corresponding — catches unclosed JSX/Astro tags, which is the most common way a sloppy edit breaks a template.

Then a small model reads the matches and answers one question: did anything actually explode, and if so, what's the one line that matters? That's a classification task. A Haiku-class model does it perfectly and costs a rounding error to leave on all day.

Wire it as a notification, not a report you have to go and read. The value is entirely in the interrupt — a break that surfaces in three seconds costs a one-line fix, and the same break found forty minutes later costs an investigation.

The rule that makes it worth having: triage before you report

The watcher alone isn't the win. The win is the standing rule attached to it: triage breaks before relaying status.

If the watcher fires while an agent is mid-task, the agent's job is to fix it — or at minimum name it — before telling me anything went well. Never leave the human to discover the break. That single rule converts the monitor from a dashboard nobody looks at into an actual quality gate.

The watcher, written out in full

The tail | grep one-liner is the idea. This is the thing I actually run — it debounces, it classifies with a cheap model, and it notifies once per incident rather than once per line.

#!/usr/bin/env nodeimport { spawn } from 'node:child_process';import { run } from '../lib/ai/index.js';const LOGFILE = process.argv[2] ?? 'dev.log';const BREAK = /error|failed|cannot|undefined|expected corresponding/i;// Noise that matches BREAK but never means "broken". Tune this aggressively —// a watcher that cries wolf gets muted, and a muted watcher is worse than none.const IGNORE = [1  /deprecat/i,  /source map/i,  /\[vite\] hmr update/i,  /ExperimentalWarning/i,  /punycode/i,];let buffer = [];let timer = null;let lastFingerprint = '';async function classify(lines) {  const { text } = await run('watch', {    maxTokens: 120,    messages: [      {        role: 'system',        content:          'You inspect dev-server log lines. Reply with STRICT JSON only: ' +          '{"broken":boolean,"one_line":string,"file":string|null}. ' +          'broken=false for warnings, deprecations and HMR noise. ' +          'one_line must name the actual cause, not restate the log.',      },      { role: 'user', content: lines.join('\n').slice(0, 4000) },    ],  });  try {    return JSON.parse(text.replace(/^```json\n?|```$/g, '').trim());  } catch {    // Never let a malformed model reply swallow a real break.    return { broken: true, one_line: lines[0].slice(0, 160), file: null };2  }}async function flush() {  const lines = buffer;  buffer = [];  if (!lines.length) return;  const verdict = await classify(lines);  if (!verdict.broken) return;  // Same break twice in a row → stay quiet. This is what stops alert fatigue.  const fingerprint = `${verdict.file ?? ''}:${verdict.one_line}`;3  if (fingerprint === lastFingerprint) return;  lastFingerprint = fingerprint;  const where = verdict.file ? ` (${verdict.file})` : '';  console.error(`\n\u0007🔴 BREAK${where}: ${verdict.one_line}\n`);  spawn('osascript', [    '-e',    `display notification ${JSON.stringify(verdict.one_line)} with title "Build break"`,  ]);}const tail = spawn('tail', ['-n', '0', '-F', LOGFILE]);tail.stdout.on('data', (chunk) => {  for (const line of String(chunk).split('\n')) {    if (!line.trim() || !BREAK.test(line)) continue;    if (IGNORE.some((re) => re.test(line))) continue;    buffer.push(line);  }  // Debounce: one stack trace is ~20 lines and is ONE incident, not twenty.  clearTimeout(timer);  timer = setTimeout(flush, 700);4});console.log(`watching ${LOGFILE} — cheap-tier classifier armed`);

Three design decisions carry the whole thing:

• Debounce before classifying. A stack trace arrives as twenty lines describing one problem. Without the 700ms window you pay for twenty classifications and get twenty notifications for one break.
• Fail toward noisy. If the model returns malformed JSON, the fallback treats it as broken. A watcher that silently swallows real breaks because of a parse error is the worst possible outcome — worse than no watcher, because you trust it.
• Fingerprint and dedupe. The same unresolved break re-firing on every file save is how you learn to ignore the notification.

What it costs to leave running

The objection is always "isn't that expensive?" — so here's the arithmetic for a genuinely heavy day.

Even the flagship column is cheap in absolute terms — which is exactly the trap. It's 7× more expensive for a task where the small model is indistinguishable, and that same 7× multiple is quietly applied to every other cheap task in your stack. The waste isn't the dollar, it's the habit.

The browser heartbeat

Server logs miss the entire class of bug where the page renders perfectly and the interactive layer is dead. So a second watcher reads the browser console on an interval:

#!/usr/bin/env bash
# scripts/console-watch.sh — ~60s heartbeat on real browser console errors
set -euo pipefail
URL="${1:?usage: console-watch.sh }"
PREV=""

while true; do
  pinchtab nav "$URL" >/dev/null 2>&1 || true
  ERRS=$(pinchtab errors 2>/dev/null | grep -v '^No errors$' || true)

  if [ -n "$ERRS" ] && [ "$ERRS" != "$PREV" ]; then
    echo "🔴 console: $(head -1 <<<"$ERRS")"
    osascript -e "display notification \"$(head -c 120 <<<"$ERRS")\" with title \"Console error\""
    PREV="$ERRS"
  elif [ -z "$ERRS" ]; then
    PREV=""     # recovered — re-arm so the next occurrence notifies again
  fi

  sleep 60
done

This is the watcher that has earned its keep most often, because hydration errors don't appear in build output at all. The server-rendered HTML is correct, the build is green, and every button is dead. Nothing else in the stack tells you that.

Note the PREV="" reset on recovery — without it, a break that's fixed and then reintroduced stays silent forever, which is a bug I shipped in the first version.

Where each watcher fits

Build break

secondswatch.mjs · ~$0.0004 a classification

Runtime break

<60sconsole heartbeat · a free browser read

False "done"

at deploysmoke.sh · an exit code, not a judgement

Three layers, each catching a class the others structurally cannot.

Three layers, each catching a class the others structurally cannot, none of them requiring your attention until something is actually wrong. That's the whole design goal: you should find out because you were told, not because you looked.

Watchers matter most when nobody is watching. On an unattended overnight run the log is the only witness — a throughput collapse at 1am is invisible unless something cheap is staring at it.

The wider pattern: route by task, not by habit

Watching is one instance of a general rule. Most calls in a real AI system are not hard, and matching model to difficulty is the single easiest cost win available:

const ROUTES = {
  watch:   'small',   // did anything break? tag this. classify that.
  work:    'default', // drafting, refactoring, multi-step agent turns
  careful: 'frontier' // migrations, money, irreversible, customer-facing
};

Everything that's genuinely a yes/no or a which-bucket question belongs on watch: log triage, console-error checks, "does this diff touch auth?", tagging, extraction, classification, first-pass filtering of search results. None of it needs frontier reasoning and all of it is the kind of thing you want running constantly.

The 60-second browser heartbeat is my favourite of these. A small model reads the console every minute while I work on something else. It has caught hydration errors that never appear in the build output at all — the class of bug where the server-rendered page looks perfect and the interactive layer is dead.

The catch: watchers that spam get ignored, and watchers can eat your machine

Two failure modes, both of which I caused.

Alert fatigue. A watcher that fires on warnings, deprecation notices and expected dev-server noise gets muted within a day, and a muted watcher is worse than none — you now believe you have coverage you don't. Tune the filter to breakage only. If it fires on something that turns out to be fine, that's a filter bug to fix immediately, not noise to tolerate.

Waiter sprawl. This one's mine and it's stupid. I once spawned a fresh background poller for every wait — four separate "wait until condition" loops plus a process-watching chain-starter — and never killed the previous ones. They stacked on top of the actual work and I got asked why five shells were running. The rule now: one reusable watcher, killed before another is spawned. Never let watcher processes accumulate.

Related: don't build brittle watchers around things you're about to kill. I wrapped a set of download-monitors around processes I then terminated out from under them, and they spent the next while false-firing and erroring — pure noise the human had to sit through. If the thing you're monitoring is transient, poll its state inline instead.

The verdict

You do not need your best model to notice that something is on fire. Put a small, fast, cheap model on watch duty, filter for the five strings that mean real breakage, wire it to a notification, and attach the rule that breaks get triaged before status gets reported.

The result is that build breaks arrive as interrupts instead of discoveries — and the person you're working for stops being your error-detection system.

Keep one watcher, kill it before you start another, and never pay frontier rates to stare at a file.

Want always-on monitoring wired into your build loop? Book a call or see the work.