Back to Feed
article 3m read

Token logs from the agents had four ways to lie, and the report admits doubt

Adding up my coding agents' token logs would have double counted copies, nested counters, running totals and the wrong day, so the report that landed in that commit dedupes, checks the math, rolls helpers into their parent job and says incomplete instead of zero.

Article DepthPractice

Flow

Hazard, safeguard, proof

  1. 1Duplicate responses

    Safeguard: map keyed by response id. Proof: same record in an active and archived file counts once.

  2. 2Cumulative events

    Safeguard: skip and tally them. Proof: a huge running total never reaches the sum.

  3. 3Wrong day

    Safeguard: named-zone date from the event timestamp. Proof: seconds around local midnight and a winter date.

  4. 4Hidden helpers

    Safeguard: parent walk with visited set. Proof: child usage appears under the parent with a task count.

  5. 5Bad or missing data

    Safeguard: validate invariants, mark incomplete. Proof: negative counters and absent cache fields change status.

  6. 6Leaked content

    Safeguard: counts only. Proof: a planted secret string never appears in output.

Knowing how many tokens my coding agents used yesterday was the goal. The logs were on disk, so the quick answer was to add up every usage line. That would have given me a number that was too big in four ways, and it would have looked just as sure of itself every time.

Which files and lines get read

The report reads both the live and the archived session folders. It goes line by line and skips anything that isn't a usage record or task info, so the chat text never gets loaded. It also reads older files, since a session from weeks ago can still be working today.

What to dedupe, skip and re-date

Duplicates. The same response can be saved in two places. Keep the first record for each response ID. If a copy matches, count it as a repeat. If a copy disagrees, mark the result incomplete.

Counts inside counts. Cached input is part of input. Reasoning is part of output. Don't add them on top. Check each record before you use it.

Each record has to show that its total equals input plus output.

Running totals. Some old events hold a total so far. Adding those counts the same tokens again and again. Skip them, but keep a count of how many you skipped.

The wrong day. Date each response by its own time, in a time zone you name. Not by when the session started, and not by the folder. Test the seconds around midnight and a date on each side of daylight saving.

The order of the steps

Filter by date before checking counters, so a record outside your window with bad numbers doesn't matter. A line you can't parse or date always counts as broken, because you can't tell which day it belongs to. Check a record before you store it. Work out helpers last, after every file is read, because the link to a parent can be in a later file.

Rolling up delegated tasks

Follow each task's parent link up to the top task, and stop if you loop. Show each top task with how many tasks fed into it. If a parent link is missing on this machine, the job will look cheaper than it was, so say so.

The ok and incomplete statuses

The result is ok only if there was data and nothing was broken, conflicting or missing a cache count. Anything else is incomplete, and the command exits with an error code. An empty day is incomplete too. Zero usage and missing logs look the same, so never print them the same way. Show a few problem spots by file and line, never the line itself. Plant a fake secret in a test and check that it never shows up. Use a closed day for a baseline, since live logs can grow while you read them.

The worktree check

The same change added a read-only look at my checkout and its worktrees. It doesn't fetch or delete. It only checks the current checkout for unsaved changes and marks the rest as not checked.

The output cap

The first draft of the notes in that change said to cap tool output. A later commit that day took it out. Only cut output once you know the full evidence is still there.

One thing these logs can't tell me is which tokens bought work I kept. That question is still open.

Keep reading

Pick up a connected idea or branch into a nearby one.

5 paths forward