Path guard given to my tools, and some of them never call it

Flow

Inputs to try against each tool

Run every input against every tool, not only the ones you think touch files.

1. 1Typed values⌄

   Send a list, a true-or-false value and a value with spaces.

2. 2Escaping paths⌄

   Try parent-folder steps, an outside absolute path and a shortcut that points out.

3. 3Patterns⌄

   Give search and multi-file tools a pattern that climbs out of the project.

4. 4Big files⌄

   Read a large text file and a large binary file and see what comes back.

5. 5Risky tools⌄

   Ask the writer and the shell runner to act and see whether anything asks first.

Flow

A build order with the rules in the registry

My own runner stops after the transport step today.

1. 1Prompt⌄

   Put the schema in the prompt and ask for one JSON line.

2. 2Parse⌄

   Check the tool name and show the raw reply on failure.

3. 3Transport⌄

   Pass inputs as one JSON document.

4. 4Declare⌄

   Mark path inputs, writing tools and output budgets in the registry.

5. 5Enforce⌄

   Check paths before and after, ask before writes, clip output.

6. 6Return⌄

   Hand results back as data, not as instructions.

Chat request after chat request, the local model server turned me away. The last rejection said why: the model I picked can't use tools. I had sent it a tool description anyway, and it refused the whole request.

A fallback for models without tools

So the program now puts the tool's description into the prompt instead. It asks the model for one line of JSON with the tool name and its inputs. The program reads that line and checks the name is the tool I chose. Then it runs the tool and shows the result. A second model call turns the result into a normal answer. If the line can't be read, I see what the model actually said.

It's fragile. Any chatter before the JSON breaks it. The program also never checks the inputs against the description, so every tool has to check its own.

One shape for every call

The first version passed inputs as command-line flags. Lists and true-or-false values got turned into plain words. Now each tool gets all its inputs as one JSON document. That one change fixed the mangling.

A shared guard that only some tools call

Later that day I added a whole set of tools: file readers, a writer, searches and a shell runner. A shared helper does the safety work. It checks that any file path stays inside one project folder, even after following shortcuts, and it cuts long text short.

Some tools never call the helper. The multi-file reader builds its own paths and never checks them. So does a resource reader. Reading the code, a pattern with a couple of parent-folder steps would reach outside the project. Nothing shows a model doing it, but nothing would stop one.

There's more. One reader returns big binary files whole. The writer overwrites without asking. The shell runner keeps its starting folder inside the project but runs any command. And the tool output goes back to the model with the same authority as my own instructions.

What I'd want in a registry

If you build something like this, don't leave the rules to each tool. Have the registry say which inputs are paths, which tools write, and how much output is allowed. Then let the runner enforce that:

• check paths before the tool runs, and check matches after
• ask before anything that writes or runs commands
• clip output, and return it as plain data

The chart below lists inputs worth trying against each tool to see which of these hold.

I haven't built that yet. Until I do, I won't point the shell runner at a folder I'd miss.