Flow
How the AI arm got its own path
Order of the changes, oldest first.
1Plugged in
The model's numbers went back through the planner and its rules.
2Small fixes
Per-row checks, rounding slack, saved pages and a slot patch.
3Missing history
Older sales added so empty shelves are not judged on a window of zeros.
4Own path
Own menu of amounts, whole-case rounding, extras to the most urgent stores.
The planner I run skipped more than a thousand low-stock lines that were still selling. The AI arm I built to question it could order only a handful of them. That wasn't the model agreeing. My design left it almost no room to disagree.
How the test was set up
My rule-based planner is the only thing that writes real orders for the stores. After each draft, a model is asked about every store row: does it need more, what kind of demand is it, how much, should an empty shelf be stocked, and how urgent is it. A fixed rule turns that into one number per row. Both orders are saved side by side. The model's order never goes anywhere real.
Where it went wrong
The agent built the model's order by feeding its numbers back into the planner. The idea was that the model picks demand and the planner only checks what's legal.
But the planner's "legal" included all its own rules: a cap on how much stock an order can create, a gate on whether a product is worth ordering, a floor for empty stores, and a rule for empty slots. Those rules ran after the model answered. So the model's order mostly matched the planner's.
Totals hide this. To spot it, pick lines where you expect the two to differ and count how many the model's order really carries.
The repairs I stacked on the wrong design
A lot of work went into that version first. One bad answer used to throw out a whole batch, so checks now run per row and record why a row fell back. Probabilities that missed adding up to one by a rounding error got some slack. Big suppliers took too long, so runs now save in pages and a background job finishes them. Pooled slots blocked a couple of plans, and that got patched too. Each fix was real. Some of them tied the test tighter to the planner.
The missing history
The model also turned down a couple hundred empty-shelf lines. Almost all had no recent sales, and it guessed stockouts had held sales down or the products were too new. But nothing sells from an empty shelf, and it had never seen older sales. Now each row carries older sales and how long since the product last sold and was last in stock, all from before the decision.
Making it a separate pipeline
I told the agent to split the arm out. The planner went back to exactly what it was. The model's arm builds its own menu of amounts, labels each with how many days of stock it would give, and never sees the planner's limits. The allocation step is plain code.
Add up what the model picked, round the total up to a whole case, and hand the extra units to the most urgent of the stores it picked.
A row the model didn't answer keeps the planner's number and says so. Products that can't be ordered stay at zero.
What I'd check before trusting it
This shows where the two orders differ, not which is right. Fallbacks borrow the planner's answer, so lots of failures would hide disagreement again. The arm has no budget limit either. What I don't know yet is whether older history fixes the empty-shelf declines.