My discount tracker could add and fix rows but never let one go

Flow

Where the sync looks for one day

1. 1Exact saved copy⌄

   A saved copy of exactly that day goes first.

2. 2Exact earlier run⌄

   Next, an earlier run of exactly that day.

3. 3Broader saved copy⌄

   Then a copy that covers more than the day.

4. 4Upstream⌄

   If nothing saved fits, read the day fresh.

5. 5Cut-short copies are skipped⌄

   A copy that looks incomplete is refused, and the sync moves to the next one on the list.

My discount tracker could add a row and fix a row. It could not tell when a row should be gone.

The tracker keeps every discount and return the stores ring up, tied to the supplier behind it. It syncs from the upstream sales system. Earlier that day I'd made the sync faster. It could replay saved copies of old responses, checked against a hash, and it could re-read one day from a dated summary. Both give a better answer for a day than an older, looser read.

A better answer can have fewer rows. But my sync only did upserts: add the row, or update it if it's there. So if an old read of Tuesday had a row and the trusted read doesn't, the old row just stays. It still counts, and the supplier's total is too high. Nothing warns you.

Letting a good read speak for its whole day

Now a write can say which window it speaks for, and what kind of source backs it. The window includes its start and stops just before its end, so two days never overlap. Rows outside the window are dropped. After saving, any live row in that window that this run didn't write gets marked superseded, with a note saying when, which run, and why.

dropUntouched({ span: [day.start, day.end], except: thisRun })

Retirement only checks kind and time, so a read for one store would retire every other store's rows for that day. As far as I know, every trusted read today covers every store.

Nothing gets deleted. If a later trusted read has the row again, it goes back to live.

Teaching the reports and my checking script to skip retired rows

This only works if everything that counts rows skips superseded ones. I changed the reports, the single-event lookup, the backfill job and my reconciliation script. That script had its own bug: it compared one day against the whole upstream cache. Now it asks for just that day. The new columns default to live, so old rows don't change when it ships.

Choosing which evidence wins

When several saved copies could answer a day, the exact match wins, then the narrowest, then the newest. The sync works one day at a time and goes down a list: exact saved copy, exact earlier run, a broader copy, then upstream. A copy that looks cut short is refused and the sync moves to the next one on the list, since retiring rows from an incomplete read would hide real ones.