Back to Feed
article 3m read

Telemetry installer turned on prompt logging and never asked

The day I taught my assistant-telemetry installer a second assistant, it switched on prompt logging without asking, merged settings in two opposite directions, wrote a token in plain text, and the same commit removed the server cap on stored conversation content. An editor extension from the same day sent full file paths on every edit.

Article DepthPractice

Flow

Checks I would run on a setup like this

Plant fake secrets, then look at what leaves and what gets stored.

  1. 1Fake secret

    Paste one into a prompt and look for it in storage.

  2. 2Old setting

    Turn prompt logging off first, then rerun the installer.

  3. 3Padded token

    Use a token ending in equals signs and see if it arrives whole.

  4. 4Folder name

    Put a fake client name in a path and look for it in traces.

  5. 5Long message

    Send a very long message and see how much gets saved.

Without telling you, the telemetry installer turns on prompt logging. The same commit also let the server keep whole conversations.

The installer is a small tool. It points my AI coding assistants at my own collector by editing their settings files. When I added a second assistant, I had it turn on prompt logging for that one. It doesn't ask. It just writes the file and prints where the data goes now.

The same commit changed the server too. And a second commit that day added an editor extension with loose defaults of its own.

Two settings files, two merge rules

The installer merges its values into what's already in each file. That part is good. But for one assistant, your old values win. The precedence is not the same for both assistants.

On the new assistant, my defaults go in first and your values win. On the old one, your values go in first and mine win.

So on the new assistant, prompt logging is on unless you had turned it off yourself. On the old one, my address and an access token replace yours. The token is written as plain text into a settings file.

Pick one rule and use it everywhere. I'd want a clear no from you to always beat my default.

The server stopped trimming

The server used to cut long messages down to a fixed size before saving them. I removed that limit. Now the full text, plus a full copy of each raw event, goes into the database, in a column that's easy to search.

The commit message is three words and says nothing about why. Put both changes together and nothing in the pipeline asks whether prompts should be kept at all. A password pasted into a chat would likely be saved in full.

The editor extension

The extension runs every time the editor opens. On every edit, it sends the file's full path, along with the language and how many lines changed. Paths often hold a username or a client name.

It also turns line counts into metric labels, which can pile up into a huge number of series. Its header parsing splits on equals signs and keeps only two pieces, so a token that ends in equals signs loses that ending. It sends over plain HTTP by default. It has no tests.

Before you copy this setup

If you're building something like it, run a few checks first. Put a fake secret in a prompt and a fake client name in a folder path. Then look at what actually leaves the machine and what lands in storage. Turn prompt logging off yourself, rerun the installer, and see if your choice survives. Use a token that ends in equals signs and see if it arrives whole. Send a very long message and see how much of it gets saved. Nothing in this day's commits runs these checks.

Keep reading

Pick up a connected idea or branch into a nearby one.

5 paths forward