Back to Feed
article 3m read

Scraper reads loyalty points now, so it gets a second opinion

The lookup tool started reading customers' loyalty balances off a point-of-sale page, so every balance now gets checked against my archive, sorted into one of five outcomes so a missing number can't pass as a match, and written to an append-only log before anyone sees it.

Article DepthPractice

Flow

Each row, in order

A row is saved before it shows up, and a failed save stops the run.

  1. 1Find customers

    The browser searches for the requested customer.

  2. 2Read the screen balance

    Only for a single exact match.

  3. 3Ask the archive

    Look up the customer number; if the archive is down, the row says so.

  4. 4Pick one answer

    One of five, with missing numbers settled first.

  5. 5Save the row

    Written to the audit table; if this fails, the run stops.

  6. 6Show it

    Only now does the row appear on the operator's screen.

The lookup tool reads a customer's loyalty balance now, and I don't trust the number it reads.

The stores' point-of-sale system only shows a balance on a page meant for a person at a register. There's no way to ask for it directly. So a headless browser logs in, searches the customer list, and until today it only read the search table.

One profile, opened on purpose

Now, if exactly one row matches the request exactly, it opens that one profile, reads the points, and goes back. No match or two matches, and it opens nothing. It never clicks anything that could change points, and the thing it drives has no way to.

Before the click it checks the link is still the only one. After the click it waits until the page names the same customer. Only that customer gets a balance. Everyone else gets none, not a zero.

That's careful. It's still a number read off a page, and someone's points depend on it.

A second witness that never sees the page

So every customer the browser finds gets looked up again in my archive, which keeps copies of the point-of-sale data. The archive takes a customer number and nothing else. It hashes that number with a secret key and finds the current copy. If it finds two, it reports an error instead of guessing. It sends back the balance, lifetime earned and spent, and two times: when it saw the value and when it recorded it.

Where a missing number turns into a match

The easy way to compare is a trap.

It converts both values to numbers and compares them for equality.

In JavaScript, an empty value becomes zero. Two missing numbers look equal, and a check that never happened looks like a pass. So my comparison picks one of five answers, in order: the archive was down, the archive doesn't know the customer, a number is missing, they agree, or they disagree. Missing values never reach the equals sign.

If the archive is down, that row says so and the batch keeps going. "42" and "42.00" count as the same.

Written down before anyone sees it

Each row is saved to an audit table first. Only then does it show on the operator's screen. If the save fails, the run stops. Saved rows can't be edited or deleted, and passwords are never saved. A read-only history page lets you search old rows by name or customer number.

Fakes, and the number they can't give me

Tests use a fake browser, archive and audit store. They check that every row is saved, the run finishes, and "42" matches "42.00". They can't tell me how often the two really disagree.

Both witnesses also start at the same point-of-sale system, so if that system is wrong, they agree on the wrong number. When they do disagree, I'd bet on the archive running behind, and the saved times let a person check that.

Keep reading

Pick up a connected idea or branch into a nearby one.

5 paths forward