Flow
My publish step after a rollback, then and now
1Then: the hub resent its saved note
It saw the same release as its last note and sent that note again, old number and all.
2Then: the box said no
The number was lower than the last one it took, so it stayed on the broken release.
3Now: every publish signs a new note
Even an old release gets a new, higher number, because going back is a new decision.
4Now: the box takes it
The number is higher, and the box sees it already has those files, so it skips the download.
After rolling back a bad release, the one machine I actually needed to roll back said no.
A log line that paused my TV
The agent added a log line to the code on the little box behind my TV. One field in it had the same name as a parameter of the log function itself, so the call crashed. It crashed in the loop that keeps the audio output alive, right before the step that renews it. The audio lease ran out and a guard paused playback. That bug got its own fix in a separate release. But the quick way out of a bad release is a rollback, so I rolled back.
The hub went back, the box stayed put
The hub went back to the old release. The box stayed on the broken one.
The box has an updater. It asks the hub for a signed note that says which release to run, with a number on it. The box only takes a note with a higher number than the last one it took. That rule stops anyone from handing my box an old note and pushing it back onto a version with a known hole. Going back to an old version is exactly what I'd asked for.
Where the old number came from
The rollback put back the hub's whole app folder. That folder also holds the hub's record of the last note it sent. So the old note came back with its old, lower number. When the hub started, its publish step saw "same release as my last note" and sent that note again as-is:
If the last note was for this same release, it just sent that note again.
The box saw a lower number and said no, which is its job. The code that makes a fresh number sat right under that check and never ran. My own test even demanded the shortcut.
Why I left the box alone
I could have let the box accept lower numbers on a rollback. But a real rollback and a replayed old note look the same to the box. Let one in and you let both in. The box was right. My publisher was lazy.
Every publish is a new decision
Now every publish signs a new note with a new, higher number, even when the release is one the box has run before. Going back gets a new number because it's a new decision. The box can tell it already has those files, so it doesn't download them again. I didn't have to touch the box. The note also says when a release shipped before, and the deploy prints a loud line when it's a rollback.
What the hub-side test can't see
My test only plays the hub's side. It never runs the box, so I know about the refusal from the day it happened, not from a test. And the number comes from the clock. If my clock ever jumps far ahead, the box will refuse every honest number until real time catches up. That's the next thing I'd poke at.