A guardrail generator that writes tests which only pass for the guardrail that catches nothing

Flow

Scanner versus generated guardrail

1. 1Type names⌄

   Scanner: long labels with spaces. Generator: short keys. They never match, so the stand-in is used.

2. 2Card numbers⌄

   Scanner: digit groups plus a checksum. Generator: digit groups only, so order numbers get flagged.

3. 3Government IDs⌄

   Scanner: separators optional. Generator: hyphens required, so it can miss its own finding.

4. 4Coverage⌄

   Some scanner types have no generator rule at all.

5. 5Stand-in rule⌄

   Matches one plain word, which also sits in every test sentence.

A guardrail that catches nothing would still pass the tests my generator writes. As written, they don't even run.

My trace scanner reads saved AI chat logs in the browser and flags things that shouldn't be there, like card numbers and API keys. Each finding has a "create a fix" button. That day I swapped its fake demo for a real pipeline that writes a small guardrail, puts it on a new branch, and opens a pull request. I spent my care on the pipeline. The generator got a template.

I'm going off the diff here. I haven't seen this open a real pull request.

From button to pull request

It checks that the finding is yours, limits how many fixes you can ask for per hour, saves a run record, and replies at once. Then it works in the background, step by step. If a step fails, it closes the pull request and deletes the branch.

The lookup never hits

The generator picks a pattern from a table using short names. The scanner hands it long labels with spaces. They never match. On a miss, the generator uses a stand-in pattern that matches one plain word, and the bundle still looks done.

The tests reward the wrong guardrail

Every test uses the same sentence, and that sentence has the stand-in word in it.

One of them only asserts that a count is at least zero, which is true for every list ever made.

Several tests are about that strong. The one test that can fail passes for the stand-in and fails for a real detector. And the file calls a name it never imports, so every test breaks before it checks anything.

Even with matching names, the scanner and the guardrail use different rules. The scanner checks card numbers with a checksum. The guardrail doesn't, so it would flag order numbers.

Holes around the generator

The finding is looked up in a database table that nothing in that day's commits fills. The branch name has spaces, which git won't take. The hourly limit counts failed runs too. And a late bookkeeping error can close a pull request that worked.

Proof I want in the bundle

One real sample, masked, that the guardrail must catch, and one lookalike it must skip. One set of detectors shared by the scanner and the guardrail. A hard stop on an unknown label. Tests that run before the pull request opens.

As generated, the pull request just asks the reviewer to tick boxes for testing steps.