Back to Feed
article 3m read

Allocator tests guarded almost everything except the promise in its README

The live allocator in my ordering system never reads the stock ceiling its README promises, the random tests check five rules but not that one, and a ceiling-aware allocator sits unused in the next folder.

Article DepthPractice

Flow

The live allocator, step by step

No step compares a store's stock with its ceiling.

  1. 1Compare

    Total asks against supply picks the branch.

  2. 2Short supply

    Shrink every ask by the same fraction, round down, give spare units to the biggest fractions. Stop.

  3. 3Enough supply

    Fill every ask in full.

  4. 4Extra

    Split by recent sales speed, only among stores that asked.

  5. 5Anything left

    One unit at a time to the asking store given the least so far.

Flow

Random-test checks, present and missing

Mint checks exist today. Coral checks are the two missing ones.

  1. 1Never over supply

    Total handed out is at most the supply.

  2. 2No negatives

    No store gets a negative count.

  3. 3Askers only

    A store that asked for nothing gets nothing.

  4. 4Exact accounting

    Handed out plus leftover equals supply, using a new leftover field.

  5. 5Ceiling

    Each store's stock plus what it got doesn't go over its ceiling. Needs a rule for asks bigger than the room.

The README for the ordering system's allocator says no store ends up over its stock ceiling. The allocator never reads the ceiling. Every store record has one, and no step looks at it.

This came in with the first commit of that system, tests and all. I'm going on what I read there. The allocator's job: a shipment lands, several stores want it, who gets how many?

How it splits a shipment

It adds up the requests and compares them with the supply. If supply is short, every request shrinks by the same fraction. Shares are rounded down, and the spare units go to the stores that lost the most to rounding. The total comes out exact, and nobody gets more than they asked.

If supply is enough, every request is filled. The extra is split by recent sales speed, but only among stores that asked. Anything left goes one unit at a time to the asking store that has been given the least so far.

Five random-test rules, no ceiling

The rules are written as tests, which I like. Random tests feed in fake stores and fake supply and check five things:

  • never more than the supply
  • never a negative count
  • nothing for a store that asked for nothing
  • every request filled when there's enough
  • something handed out when there's demand and supply

No test checks the ceiling. The fake stores have ceilings, some start above them, and neither random test compares. A hand-run simulation script in the same commit does check it, but it needs a database and isn't in the test suites.

The supply rule is also loose. It says at most, not exactly. If nobody asks, the allocator hands out zero and the shipment just sits there. Every test passes, and one test calls that correct. The result has no leftover field, so nobody can see it.

A leftover count and a ceiling check

Give the result a leftover count, then assert exact accounting.

The amounts given plus the leftover have to add up to the supply.

Then check that each store's stock plus what it got doesn't go over its ceiling. That check fights with "fill every request" when a store asks for more than it has room for. You have to pick which rule wins. The live allocator never picked, because it skips the ceiling.

The allocator in the next folder

The same commit has another allocator in its own folder. It does read the ceiling. It caps each share at a store's room, sends extra to asking stores by sales speed, offers overflow to stores that didn't ask, and has a last step that breaks the ceiling on purpose to finish capped requests. So it picked: requests win, but only at the end.

Nothing calls it. Only its first step is tested. It also breaks the live rule that a store which didn't ask gets nothing.

Probably an agent wrote both, judging by notes in the code. I shipped the README and didn't catch that it didn't match the code that runs. Still open: do I teach the live allocator about ceilings, or test the other three steps of the modular one and switch over?

Keep reading

Pick up a connected idea or branch into a nearby one.

5 paths forward