Flow
The order I would build it in
Receipts go first because every later step retries.
1Receipts
Same receipt and request return the old answer; a changed request is refused.
2Versioned claim
Check the seen version, accepted dependencies and attempts left.
3All paths or none
One clash, parent or child, rolls back the whole claim.
4Token on every write
A reclaim issues a new token and locks the late worker out.
5Two read positions
Reading hands out a page; only the reader commits it.
Late workers were the problem. If you have two agents, or any two workers, sharing one store, a lease plus a version number won't stop a worker that stalls, gets replaced and then comes back with old work. It can just re-read the version and pass.
That's what pushed my agent bridge to a second version in one day. The first had leases on paths and receipts on sends, and it trusted agents to save their read position at the right time. The second moved those rules into the server. If I were building it again, I'd do it in this order:
- Put a receipt on every write. Store it with a fingerprint of the request and the result, in the same transaction as the change. Same receipt, same request: return the old answer. Same receipt, different request: refuse.
- Claim with the version the worker saw, and check that its dependencies were reviewed and accepted and that it hasn't used up its attempts.
- Reserve every path the job touches at once, or none. Treat a parent folder and its child as a clash.
- Hand back a random token with each claim, and require it on every later write. A reclaim gets a new token, so the old worker is locked out even if it knows the latest version.
- Track two read positions per reader: handed out and acknowledged. Reading never moves the acknowledged one. The reader commits it after it has kept what it read.
Expect one surprise from step one. Retrying a lost claim gives back the original token, and if that claim has since expired, the token fails on the next write. That's right. The worker needs to know it lost the job.
Test it with two real processes on one database file, not mocks. My demo does that, but with scripted clients. Live agents are next. And remember the leases only bind workers that go through the API, so give each agent its own worktree.