Every tool call left two rows, and part of my dashboard counted both

Flow

Finding and stopping a two-events-per-call count

1. 1Check the bottom of each ratio⌄

   Dividing by all rows is exposed. Successes over successes plus failures is safe.

2. 2Filter to results⌄

   Every tool query counts result rows only, even ones that were right by luck.

3. 3Label the kind⌄

   A view that holds both kinds needs a column that says which, kept in the key.

4. 4Count each kind⌄

   Now and then, count each kind and check the two counts are close.

Every tool call a coding assistant makes shows up in my data twice, and part of my dashboard was counting both.

The dashboard shows what coding assistants do: which tools they use, how often those tools work, and how long they take. Each tool call writes two events: first a decision, recording whether the user approved it, and then a result, recording whether the tool succeeded and how long it took. Once my ingestion normalizes them, both carry the same event name, and one extra field holding the original name is the only way to tell them apart.

Where the two kinds end up

I have two views over this data. One keeps every event as its own row, including both kinds and that extra field. The other adds events up per hour, per team and per tool. It also takes both kinds, but it has no field that says which kind a row came from.

Why the math goes wrong

Only the result row knows if the tool worked. So if you divide "rows that worked" by "all rows", the bottom is twice as big as it should be:

const rate = rows.filter(r => r.ok).length / rows.length; // half

A tool that never fails shows up at one half, average duration is cut in half too, and counts double. All of it looks like a genuinely unreliable tool.

One form is safe: successes divided by successes plus failures. Decision rows add nothing to either side. Anything that divides by the row count needs a filter.

A guard that was doing a different job

The performance query already skipped rows with no success value. Decision rows have no success value, so that one was right by luck. I can't say the same for the other tool queries. I added an explicit "results only" filter to all of them, because luck stops working the day someone edits the other filter.

The rule I wrote for my agent

I lean on an agent for most of this code, so I put the rule in the file it reads before this kind of work:

• if a view takes more than one kind of event, give it a column that says which kind
• put that column in the grouping key, so rows of different kinds never merge
• filter on it in every query
• now and then, count each kind and check the two counts are close

The example in that file is supposed to show the wrong number next to the right one. It filters both, so it would show the same number twice. That's the file I wrote for the agent to copy.

Team charts that ask for a missing column

The real goal of the change was team charts. I added the user to the hourly views, so I can pull rows per person, look up teams, and sum them per team.

The catch: the hourly tool view doesn't have the kind column, and the team query filters on it. I'd bet that query fails the first time the team page opens. I also doubt the new user column shows up on a database that already has the old views, unless they get rebuilt. If either guess is right, that page has nothing true to show yet.