Two formulas checked against each other instead of against a real order

Flow

Auditing a planner against a real order

1. 1Load the accepted order⌄

   Map every line to a product. Stop if one won't map.

2. 2Replay in the same scope⌄

   Only the order's stores and products, as of when it was made.

3. 3Sort the rows⌄

   Same, different, or found on only one side, based on whether the row exists.

4. 4Flag blockers⌄

   Plans needing review or leaving units unplaced beat good numbers.

5. 5Print a read-only receipt⌄

   Say nothing was written. Optionally fail the run if anything differs.

The first version of my migration audit said two formulas disagreed. True, and not much use.

The system buys whole cases and splits them across stores. An old allocator does that for every supplier. A new planner runs next to it in shadow, and I want to know what switching a supplier would change.

The agent built a read-only audit. For each side it made a table of units and cost per store and product. Then it sorted every row: same on both sides, different, only on one side, or only on the other. It flagged plans that needed review, and it gave one ready-or-not answer. It reused an existing diff tool. Good work.

The wrong thing to compare against

Version one ran the old allocator on today's demand and compared that with the new planner. All that tells you is whether two formulas agree right now. Nobody had shown the old one was right, and today's demand isn't a real order. When they disagreed, I couldn't tell which side was off.

There was already an older audit that compares against accepted orders. The agent copied its tools and skipped its baseline. Easy to miss, and that annoys me.

Compare against a real order

The follow-up switched the reference to an order the stores had actually accepted. If you build something like this:

• Load the accepted order and map every line to a product. If a line won't map, stop.
• Ask the new planner about only those stores and products, as of the moment the order was made.
• Decide "missing" from whether a row exists, never from a difference of zero.
• Let review flags beat good numbers.
• Keep it read-only and say so in the output.

What it showed

On a couple of real orders, the new planner had nothing for any line on one order and missed most lines on the other. The problem is coverage, not math.

If you build one, expect accepted orders to include hand edits, and settle on a cost tolerance before you let it block anything. I haven't done that second part yet.