smithfam.ai · postmortem · 26 Jul 2026

The model nobody set

I had a written rule about which model does the thinking on this project. It was correct. It also turned out to reach almost none of the actual bill.

I run two sites, pointsandprompts.com and wifiodds.com, almost entirely with Claude, across two Max 20x plans. Month to date: 10,076M tokens, $8,920. I pulled the usage dashboard on 26 July 2026 expecting a bad number. I did not expect to find out why so fast.

Haiku's share of the month, on its own, came to 0.03%. Almost nothing here got delegated downward, and that's odd, because a document on this project says exactly what should happen. MODEL-DOCTRINE.md reads: "Opus decides and checks. Sonnet does." A second line was written specifically to close the obvious loophole: "'Is there work?' is not an Opus question." That second line is the whole point of the doctrine, and it turned out to cover almost nothing.

The rule itself was correct. It was enforced in exactly one place: watch.sh, a shell script that called claude -p --model sonnet inside a relay loop. That relay had been paused for eight days. So the rule had nowhere left to apply, while every recurring job on the account ran through 27 scheduled tasks the rule never named.

None of those 27 tasks set a model. They didn't override anything on purpose. They inherited the app default, which is Opus. A task that checks whether a Chrome Web Store review has cleared is a yes-or-no question with no reasoning in it, and it ran on opus-4-8 and fable-5, four times a day, for as long as the review sat in the queue.

One detail wasted my first attempt at a fix, so it gets a paragraph of its own. The model isn't a field in a task's SKILL.md front matter — that file only carries name and description. It lives on the task's entry in scheduled-tasks.json, the registry the scheduler actually reads. Writing model: claude-sonnet-5 into the SKILL.md changes nothing, and it looks enough like a fix that nobody would think to check further. I'd already been caught by this exact shape once, on a different project: an .assetsignore file everyone believed was keeping a folder off a public site, deleted by the build command before Cloudflare's uploader ever ran. A rule that lives somewhere the runtime doesn't read protects nothing.

Then the fix itself turned out to be a third instance of the same shape. I set 17 tasks by editing that registry file — backed up, written atomically, re-read and confirmed. Twenty minutes later, 25 of 29 were unset again. A second session running on the same machine set one task twice, verified both times, and found it reverted; a task it set to Opus came back as claude-haiku-4-5-20251001, a value neither of us wrote. The app keeps the registry in memory and rewrites the file, so edits to the active session's registry get clobbered. The durable route is the scheduler's own UI. The file is for reading.

What makes that worth writing down is not the bug, it's the verification. My check passed. I re-read the file immediately, saw the value I'd written, and reported success — in the one window where the check could not fail. Write, then wait, then re-read. A test that runs inside the interval where the fault hasn't surfaced yet isn't weak evidence, it's the wrong test.

The intended fix, for whatever route survives: model: claude-sonnet-5, on the registry entry for 22 of the 27 tasks. The other five run once and don't recur, as far as I can tell, so there was nothing to set. I picked Sonnet, not Haiku, even though Haiku costs less and is closer to what the doctrine argues for. Sonnet is the one setting I've watched hold up end to end in this exact setup, and a poller that fails without warning costs more in cleanup than the tokens it would have saved. I checked one task's output against the Sonnet run before touching the other twenty-one.

What I haven't done yet is measure a result. I made this change today, and the number that matters is next month's bill. I'll report it here, whichever way it moves.

What I'm taking from this

1. An unset model is a bug, not a default. Silent inheritance is how a yes-or-no check ends up on the most expensive model on the account.

2. A rule enforced in one code path is not enforced. Ask where else it needs to hold. Mine died the moment I paused one loop.

3. Polling is not reasoning. Where a check is really curl piped to grep, it shouldn't call a model at all. Save the model for the run where something actually changed.

4. Frequency multiplies cost. Four checks a day against a process that moves in days buys latency nobody was using.

5. Audit the meters, not the intent. My doctrine said the default was holding. The dashboard said 49% Opus. I believe the dashboard now.

One more note, since it's worth more than a line in a spreadsheet. Tokens are the unit of the bill and the unit of the footprint at the same time, so a lower count moves both together. The biggest lever this week was noticing how much of that 10 billion tokens never needed a model in the first place, not routing it more cleverly.