Flow
Where to save the memory relative to the send
1What I shipped: save, then send
A failed send is already marked sent, so it gets skipped next run.
2Send, then save each alert
A crash in between means one repeat. A stable key lets the receiver drop it.
The supplier alerts I built can nag a supplier about the same low shelf again and again. They can also lose an alert after one failed send and never try again. Both problems came in the same change.
I found them by reading my own code and tests. I haven't seen a supplier get these alerts yet, so how often it happens is a guess.
One job, one row per alert
A scheduled job checks each supplier who has alerts turned on. It finds items that are low, items at zero, and items with way too much stock. It sends each list to the supplier as a webhook.
The first version had no memory, so it repeated itself every run. In the same change I gave it one: a row for each supplier, alert type and item. The row holds a hash of the alert and a flag for whether it's active. New or changed alerts get sent. Matching ones get skipped.
The hash watches the wrong thing
The hash covers every number on the item: stock on hand, sales rate, days of supply, and so on. Those numbers come from a rolling window of sales, so they shift a bit every day. The check is exact, so any tiny shift counts as new. My notes say it re-sends when things change in a way that matters. The code can't tell what matters. My bet is that most items that sell will re-fire most runs.
If you build one of these, ask what change should earn a second message. Usually it's a new alert type, or the item moving into a worse band. It's almost never the second decimal of a sales rate.
The memory is saved too early
All the rows for a supplier are saved in one database transaction. That save finishes before any alert goes out. If a send fails, the job stops with an error, but the rows already say the alert was sent. Next run it gets skipped. The rest of that supplier's alerts are lost the same way.
The two bugs cover for each other. A noisy item gets its lost alert back on its next wobble. A steady item stays lost.
My tests didn't catch any of this. They check the memory step alone, where the saved hash comes from the very same item, so it has to match. The sending step isn't in them.
Saving after the send, not before
Most of this comes down to a few changes:
- Hash the alert type plus a band, not the raw numbers.
- Save the row only after the supplier's endpoint says the send worked.
- Put a stable key in each alert so the receiver can throw away a repeat.
- Wait for a margin before clearing, so an item sitting on the line doesn't flip each run.
- Base clearing on the full list, not the capped one.
In code, the core idea is tiny:
if post(alert, dedupe_key=alert.key):
memory.remember(alert)
A crash between those two steps means one repeat next time, and the stable key lets the receiver drop it.
To test it, use a fake receiver that fails on purpose. Run the whole job twice. Then check what the receiver got, not what the table says.