Flow
Five symptoms traced to their causes
1The warehouse copy had far fewer receipts than live
Why: receipts are filed by delivery date but typed in days later. Change: re-read about a month back, and treat newer days as maybe.
2A vendor's transfers vanished from the list
Why: the fast mode skipped the pending checks along with the newest stretch. Change: always run the pending checks.
3A few zero-dollar invoices looked like a bug
Why: they were real free samples, already received. Change: none needed there.
4Old rows sat inside a range marked done
Why: the stamp ignored checks that failed. Change: move the stamp back to just before the oldest one.
5A wrong answer stuck in the cache
Why: only the newest day was re-read. Change: re-read the whole cached range after a few hours.
Weeks of warehouse deliveries were all there, my receipt archive told the ordering system. It had about a quarter of them, and it still said the range was complete.
The setup is simple. The store system's API is slow and limits how often I can call it. So my own service keeps a copy, and answers come from the copy. Each answer has a "good through here" stamp. My vendor-deal checker reads receipts to see which vendor deliveries came in and what they cost. It trusts that stamp: it caches the range and never asks again.
The warehouse was way off
When the copy was checked against the live API, the warehouse was way off. Live had about four times as many receipts, including one big delivery. The stores were close. So the problem was in how I copied, not in an outage.
Receipts typed in days late
The API sorts receipts by delivery date. But people type them in days later. My copy job read a short recent window, marked it done, and moved on. Late receipts landed in windows it never read again.
I switched reads back to live first. Then every few hours the copy re-reads about the last month. Rows that didn't change are skipped. And a read only counts as proof for deliveries that were already a month old when it ran. Newer stuff stays "maybe."
The fast mode skipped the pending checks
A transfer starts as pending and later becomes received. My copy saw it once, while pending. The service made up for that by checking each pending record live before answering.
Listings were slow, so a fast mode got added. Its one switch skipped two things: the newest stretch, which is fine to skip, and the pending checks, which are not. The checker drops pending rows. So a vendor's transfers just disappeared.
On the dev box, that vendor showed just a couple of invoices for zero dollars. The commit that switched the fast mode off read it as vendor names not matching and blank amounts. Wrong read. Those were real free samples, already received. Probably they were never pending. One switch covered both the safe skip and the unsafe one, and that stung.
The stamp stayed put
When the API is busy, a pending check can fail, and the old copy goes out. The answer said "partial," but the stamp didn't move. The stamp now backs up instead of overstating coverage.
When a check fails, the stamp is pulled back to just before the oldest record that couldn't be checked.
The tests now check the stamp value itself, not just the partial flag.
On the reading side
The checker stopped trusting the stamp alone:
- A partial answer counts as done only up to its stamp.
- The whole cached range gets re-read after a few hours.
- Identical requests share one read.
- Background reads get a long timeout, and clicks a short one.
- Vendor deliveries are read only where vendors deliver.
The fast mode came back after a few locations matched live exactly. A month is my guess at how late a receipt can be, and I haven't checked it against the worst one.