I audited my own tool: what limitcast got wrong and what I fixed

Sam Stull · 7 November 2026

I read limitcast the way I would read a stranger's tool: corrupt its state, feed it bad input, time it on a large account, and read the code that writes to disk. It passed most of that. Here is what it did not pass, what I changed, and what the measurements say.

limitcast held up where it matters most, and had four real defects. No hook can block a tool call on bad state, but a mistyped option was silently ignored, settings were written in place, and results walked my transcripts twice, and a cold report --why ran 26 seconds in silence. All four are fixed in 0.18.2. The speedup turned out smaller than I first measured.

What held up

I started from 222 passing tests, a clean Biome run, npm's latest version equal to the repository head (0.18.1), and no open issues or pull requests. Then I overwrote all 21 state files on my account with {garbage and ran every hook: guard, prompt-check, tool-limit, tool-check, subagent-receipt, drift and statusline. All seven exited 0, so none can block a tool call. Out-of-range input (budget subagents 150, plan set bogus, --days -1) prints the usage and exits 1.

A typo that looked like success

limitcast trim --bogus-flag printed the normal report and exited 0. The CLI only looked for the options it knew, so install --gaurd would have run a plain install and looked fine. Mistyped options now fail with Unknown option --gaurd and exit 1. The hooks are exempt, so they still fail open. A test checks the list of options against the ones the code reads, so a new option cannot be forgotten.

Settings written in place

Seven places wrote config.json with writeFileSync, which empties the file before it writes. A hook that reads at that moment sees an empty file and treats budgets and the keep-list as unset for that call. In a stress test (a 3.5 MB file rewritten in a loop while another process parsed it), 25 of 1,710 reads failed with the in-place write and 0 of 1,639 with a temp file and a rename. Real config files are a few hundred bytes, so the window is far narrower than that. It is still a window, and the fix is one helper, now used for the settings, the usage cache, last.json and the pruned samples.jsonl.

Reading the same transcripts twice

On my account there are 6,492 transcript files and a 17.3 MB usage cache. results made 15,030 directory reads in the transcript folder, including two full walks of it, and parsed and rewrote that cache twice. It now reads usage once and makes 9,666 reads, one walk fewer. The usage and report --why caches are rewritten only when a transcript changed. The first report --why on a cold cache took 25.8 seconds with no output; it now says "reading N transcripts for the first time" first.

I timed the committed version against the new one, alternating runs, median of five:

Only results got faster, by about a fifth. The rest is noise: my own session was writing transcripts the whole time, so the caches had changed anyway. My first timings, taken against a warm cache, suggested a much bigger win, and I had written that into the changelog before I re-measured. I corrected it.

Smaller things

The changelog had an "Unreleased" block that had already shipped in 0.15.0, sitting between 0.15.0 and 0.14.0. Nine comments still labelled features "Pro" after everything became free. The structured data on the home page said 0.18.0 while npm had 0.18.1. All three are fixed.

What I did not fix

About 0.6 seconds of every report --why is the transcript walk itself, roughly 5,400 directory reads per pass here. Skipping old directories would be faster, but it could miss a subagent file that was appended to after its folder last changed, so I left it. Two commands changing the same setting at the same instant can still lose one update, because that needs a lock, not a rename. The fixes shipped in 0.18.2 (commit 38b3302); the 228 tests, six of them new, pass.

Measure your own numbers

npm install -g limitcast
limitcast doctor
limitcast report --why --days 7
limitcast results

If one of those is slow on your machine, open an issue with the transcript count and the time. The guide lists every command, and the 0.18 post covers the last release.