Back to Feed
article 3m read

Tool cost report billed one model call three times

Every tool in a burst claimed the full cost of the one model call that followed it; I moved ownership to the call, then split its cost by tool output size, and the tests still don't check the math.

Article DepthPractice

bar chart

What a tiny write pays for one call, by rule (illustrative)

The call costs 12 units, and the write ran last in the burst.

Nothing in the output looked off while the tool cost report charged one model call three times.

Tool runs and model calls, matched by time

The report shows which tools my coding agents spend money on. I log tool runs, and I log model calls with their cost. But no call says which tool result it read. So the report matches them by time. If a call comes soon after a tool in the same session, that call gets linked to the tool. That's a guess, and the data can't prove it right.

Why three tools paid for one call

At first, each tool run looked for the first call after it and took that call's full cost. Agents often run three tools in a row and then make one call that reads all three results. All three tools found the same call. All three took its full price. So one bill was counted three times, and bigger bursts made it worse.

Each row looked fine. Only the total was wrong. You'd only catch it by comparing the tool total with the real spend, and the report didn't do that.

The cost belongs to the call. A tool should only get part of it.

Handing the bill to whoever went last

My first repair went the other way. Each call picked just the latest tool before it. Now each call counted once, and the totals matched. But the last tool got the whole bill. A small write after a big file read would pay for the read. I changed it again the same day.

Splitting by output size

Now each call's cost is split across every tool that ran just before it. The split follows how much text each tool sent back. A big read gets a big slice, and a tiny write gets a small one. Each tool gets at least a small minimum, so an empty result still counts. The slices for one call always add up to that call's cost, so the total can't run past what I paid.

Where length and timing mislead

Text length is only a rough stand-in for how much a result mattered. A tool that falls near two calls gets counted in both, so its run count reads high. Calls with no tool nearby are left out. My tests check that the query has the right parts, like the split and the filter that keeps one organization's data apart from another's. They don't run it on a known bill.

Until the agent names its inputs

The real answer is for the agent to say which results each call read. Until it does, I'm guessing from timestamps.

Keep reading

Pick up a connected idea or branch into a nearby one.

5 paths forward