Screenshots moved out of transcripts, and one big one puts them all back

Flow

Each choice and what it costs

1. 1Names from account, session, message, counter⌄

   Resending the same log writes to the same names.

2. 2Two picture formats⌄

   Old rows stay readable, but every reader must handle both.

3. 3Too-big picture throws⌄

   One huge screenshot fails the whole session.

4. 4One error handler for everything⌄

   Failures store the heavy original with no mark.

5. 5Uploads one at a time⌄

   Screenshot-heavy sessions are slow to accept.

Flow

Checks this change still needs

1. 1One oversize picture⌄

   The others should still move out, and the big one should be marked.

2. 2Storage down⌄

   Count what gets stored inline and whether anyone can tell.

3. 3Send the same log twice⌄

   The number of stored files should not grow.

4. 4Picture in a tool reply⌄

   See whether it stays in the database.

5. 5Open a stored link⌄

   Find out who can fetch it.

Every screenshot pasted into an agent session ends up in the transcript as a long string of encoded text, and my analytics service used to store it as is. Each transcript line went into a database built for fast scans over many small values, and a pasted picture went in as one giant value. I'd bet one screenshot is often bigger than the whole chat around it. Nobody searches those bytes. They're only there to be looked at.

Swapping pictures for links before storage

I added a step right before the data gets stored. It reads the transcript, finds image blocks in the user's messages, uploads each picture to file storage, and swaps the picture for a short link. The database gets the light version. Each stored file is named from the account, the session, the message and a counter, so sending the same transcript twice writes to the same names. The old picture format stays readable next to the new link format, because old rows don't get rewritten. That means every reader has to understand both, including any parser that runs later.

That parser part nearly bit me. The data gets parsed again after my step, and the parser used to turn every image back into the old format. A link would have come out as an empty picture. The same change taught the parser about links.

One big screenshot undoes the rest

The whole step is wrapped in one error handler, and on any error it stores the original transcript. A picture that's too big doesn't get skipped. It throws.

If any single upload fails, the handler gives back the original transcript with every picture still inside it.

So one huge screenshot sends every picture in that session back into the database, including the ones that uploaded fine. Those uploads stay in storage with nothing pointing at them. The request still looks successful, so nothing retries. And the stored row doesn't say anything went wrong, so a failed session looks just like one with no pictures. Handle each picture on its own instead: if one fails, leave it in place, mark why, and keep going.

Gaps I can see

Pictures returned inside a tool's reply sit one level deeper and never get moved. Uploads happen one at a time inside the request, so a session full of screenshots is slow to accept. The size limit for incoming requests went up several times over, which lets big sessions in, and big abusive ones too.

I also removed a log line that printed every incoming request in full, screenshots and all. I'm glad that one's gone.

Before turning this on, I'd check who's allowed to open those links. The pictures are stored with a cache setting that keeps them for a year, and nothing in this change sets access on them. I'd want that answered before the first screenshot of someone's private work goes up.