Back to Feed
article 3m read

A busy second fools an agent timeline that pairs events by clock

The agent console's transcript store and its telemetry store got merged by timestamp. The duplicate check asks whether anything is nearby, not whether the matching thing is, so real tool results can vanish in bursts, and the join loosened the telemetry access filter along the way.

Article DepthPractice

Flow

How I'd match events between two stores

  1. 1Shared event ID

    If both sides can carry one, join on it and drop the clock.

  2. 2Same kind, one to one

    Without an ID, pair by kind and time, and use each entry once.

  3. 3Keep leftovers

    An unpaired event stays on the timeline.

  4. 4Say when a store failed

    An outage should not look like an empty session.

  5. 5Check the viewer's org

    Not just that a row has one.

The first cut of my merged session timeline showed every prompt and every tool result twice. That bug was easy to spot. The one that replaced it is much harder to see.

Same session, no shared event ID

My agent console reads two stores. One has the full text of each session. The other has small telemetry events: requests, tool calls, results, prompts and errors, with timings. Both use the same session ID. But no single event has an ID that both sides know. And in the samples the notes compared, the text side only kept whole seconds. So lining things up by time needs some slack.

Matching by clock

The merge, mostly agent work by the look of it, does three things. It reads both stores at once and treats a failed read as empty. It attaches every telemetry event within a couple of seconds to each text entry. Then it gives some telemetry events their own rows. Requests, decisions and errors always get one. Prompts and tool results only get one if no text entry is close in time.

That last rule is the problem. It asks "is anything close?" and not "is the matching thing close?" Agents do a lot in a few seconds. So one reply can hide several real tool results. Short of an event ID both sides carry, a safer rule pairs events one to one, by kind.

An event keeps its place on the timeline whenever it has no unused partner of the same kind nearby.

I haven't built that yet, and I haven't seen the current code drop a real event. I'm going by the code. But a lost result doesn't look broken. It looks like a tool that never answered.

The attached timings have the same flaw. They can belong to a nearby event, not the one they sit on. And new event types fall under the "is anything close" rule, so they'd mostly vanish in busy stretches.

Who signed off

A planning note suggested matching on time and type. The code only checks time. The review checked one session by hand and called it fine. I didn't push back either.

Outages and the wrong viewer

A failed read turns into "no data", so an outage looks like a quiet session. The session summarizer reads that same timeline.

The two stores also write organization IDs in different formats. So the telemetry access check only asks whether a row has an organization at all, not whether it's yours. It's marked as temporary. A wrong row on a timeline is a bug. A row shown to the wrong person would be a leak.

Keep reading

Pick up a connected idea or branch into a nearby one.

5 paths forward