Flow
The order my gate says no
1Provider said back off
Wait until the time it gave.
2No reading
Ask me to type one in.
3No reading yet today
A new local day needs a fresh one.
4No price for this model
Wait for a starting reading for the model in use.
5Session window full
Pause until it resets.
6Weekly limit
Stop at the allowance minus the reserve.
7Daily limit
Pause at the day's start plus an even share of the week.
8Price-finding batch used up
Ask me for another reading.
The hosted model tells me how many tokens each call used, but not what that cost. The only cost number is a percentage on the provider's dashboard: how much of this week's allowance is gone. I couldn't find a way to get it from the API, so I type it in by hand.
The loop that spends it runs on its own all day. It writes test chats for my desktop assistant, checks them, plays them out, and has a model grade them. Left alone, it would use up the week, and I share that allowance for my own chats. So I put a gate in front of it.
Typing in the gauge
When I type in a reading, the app's ledger saves its own totals next to it: calls so far and tokens so far. Between readings the gate guesses where the gauge is. It starts from the last reading, adds the tokens spent since, adds tokens set aside for work in progress, and turns that into percent with a learned price. The next reading replaces the guess.
A price from two readings
Two readings give one price sample: how far the gauge rose, divided by how many tokens were spent. The math is small.
The price is the rise in the gauge divided by the millions of tokens spent between the two readings.
Throw away samples where the gauge didn't move. Use the median of the last few. Tie the price to one model and wipe it when the model changes.
Before the price is known
With no price, the gate allows a small batch of single calls, then stops and asks me for a new reading. It won't start a full evaluation until it has a price.
A floor for the week, a share for the day
Two limits sit on the guess:
- a weekly limit: the whole allowance minus a small reserve the loop can't touch
- a daily limit: where the gauge stood when my day began, plus an even day's share of the week
A new day needs a new reading. A reading lower than the last one means the week reset. If the provider says to back off, the gate waits. My own chats go into the same ledger and are never blocked, so they count toward the price instead of skewing it.
Whole units, not single calls
A test that stops halfway wastes what it spent. So the gate estimates and sets aside everything one full test will need before it starts. Each call takes from that pot. A call that fails early gives its share back, and leftovers are freed at the end. Set-asides expire, so a crash can't hold them. In strict, a new reading mid-test means the test has to ask again.
All of this lives in one ledger file. Changes take a lock, writes are atomic, and if the file won't read, the gate pauses instead of starting fresh.
The estimator I replaced
My first estimate for a test used fixed numbers per turn and per case. When I replaced it later the same day, the tests showed the same work reserving between a third and a half as many calls. It also charged a resumed test for turns that had already run. I'd guess it turned away work it could afford, though I have no numbers showing that.
Now it learns from recent finished tests of the same kind, adds some slack, and charges a resumed test only for what's left.
The warn-only setting
The same change added a setting that warns instead of refusing. A provider back-off still stops work, and per-test and per-call caps still apply. The evaluation loop runs in this warn-only setting unless I opt into strict, so the limits above only refuse when strict is on. Since I share this allowance with my own work, that default is the part I'd look at again.