Flow
Which checks each kind of change gets
1One API route plus tests or docs
Narrow lane before going live: whole-app type check, lint on changed files, tests that import the route.
2A second route or a shared helper
Full checks before going live.
3Schema, config or package changes
Full checks before going live, and the full deep suite after.
4A deleted or moved file
Full checks; a move counts as a delete plus an add.
5A file no rule recognizes
The deep planner runs everything until a rule is written.
Flow
Building a deep-test planner in order
1Save one list of changes
Next to the build, renames off, an unknown line when the old commit is missing.
2Allow only one narrow lane
One route plus tests or docs; full checks can be forced, narrow cannot.
3Tag every deep test
Always runs, belongs to areas, or belongs to one route file.
4Widen on doubt
Empty diff, deletions, shared files, unknown files and baseline-only plans all run everything.
5Generate route checks
One per route file, found by walking the folder.
6Report the choice
Mode, reason, areas and test counts in every run.
When a new build of the ordering system went live, the deploy checked exactly two things: is it answering, and did it leave junk on the host? The deep tests, the ones that run real purchasing and reporting workflows against the live app, had their own commands, and the deploy never called them.
Now it calls them. It reads the diff, runs the deep tests the change has earned, and writes down why it chose them. If it can't tell what a change touches, it runs every one.
One saved list of changes
The deploy saves the list of changed files next to the candidate build, with each line marked added, modified or deleted. Rename detection is off on purpose, so a moved file counts as a delete plus an add. If the last deployed commit is missing or isn't an ancestor of the new one, the list gets a single "unknown" line instead. Every later step reads that same file rather than working it out again.
Two gates read it
The first gate runs before the new build goes live and decides how hard to check it. I added a lane for one narrow kind of change: exactly one API route, plus tests or docs, with nothing deleted. That lane still type-checks the whole application, lints the changed files with no warnings allowed, and runs the tests that import the route, and a route with no related tests fails. An older lane covers changes that only touch the PDF renderer. Everything else gets the full checks. I can force the full checks by hand, but nobody can force the narrow lane.
The second gate runs after the build is live and picks the deep tests. It runs after a full check too, so a change can get the full checks first and a smaller deep run afterward.
Every deep test gets a tag
Each deep test needs a stable ID and one tag: it always runs, it belongs to one or more areas like purchasing or reporting, or it belongs to one route file. Keep the always-run set small. Mine is the health endpoints, the service being active, and a clean host. Then write path rules that map changed files to areas.
When the planner widens
The planner runs the full deep suite when the diff is empty or has an unknown line, when a file was deleted, when a shared file changed, or when no rule recognizes a production file. Shared means things like pipeline config, the database schema, auth, and the deploy and test code itself, so a change to the picker can't shrink its own check.
Watch for one trap. A plan can end up holding only the always-run tests, which looks green while checking nothing about the change. So widen in that case too.
If only the always-run tests survive, run everything.
As a last guard, the runner stops with an error if it is handed zero tests.
The route check that overpromises
The route checks are generated, one per route file, by walking the folder at startup, so a new route gets its check automatically. Each check confirms the route is present in the build. It does not confirm the route's methods, even though its name lists them, and that needs fixing.
The choice goes in the report
Every run writes its mode, reason, areas and test counts into the report, which makes a deliberate narrow run easy to tell apart from an accident.
I don't know yet how often it widens. A docs-only change still runs everything, which is a lot of work for a typo. And the area tags are typed by hand, so I have to trust that I filed each test in the right place.