Back to Feed
article 2m read

The stalled agent that still knew the right version number

A lease and a version check couldn't stop a coding agent that stalled, got replaced and came back with old work, so my agent bridge got a second version the same day: a fresh token on every claim, a receipt on every write, and a read cursor only the reader can move.

Article DepthPractice

Flow

The order I would build it in

Receipts go first because every later step retries.

  1. 1Receipts

    Same receipt and request return the old answer; a changed request is refused.

  2. 2Versioned claim

    Check the seen version, accepted dependencies and attempts left.

  3. 3All paths or none

    One clash, parent or child, rolls back the whole claim.

  4. 4Token on every write

    A reclaim issues a new token and locks the late worker out.

  5. 5Two read positions

    Reading hands out a page; only the reader commits it.

Late workers were the problem. If you have two agents, or any two workers, sharing one store, a lease plus a version number won't stop a worker that stalls, gets replaced and then comes back with old work. It can just re-read the version and pass.

That's what pushed my agent bridge to a second version in one day. The first had leases on paths and receipts on sends, and it trusted agents to save their read position at the right time. The second moved those rules into the server. If I were building it again, I'd do it in this order:

  1. Put a receipt on every write. Store it with a fingerprint of the request and the result, in the same transaction as the change. Same receipt, same request: return the old answer. Same receipt, different request: refuse.
  2. Claim with the version the worker saw, and check that its dependencies were reviewed and accepted and that it hasn't used up its attempts.
  3. Reserve every path the job touches at once, or none. Treat a parent folder and its child as a clash.
  4. Hand back a random token with each claim, and require it on every later write. A reclaim gets a new token, so the old worker is locked out even if it knows the latest version.
  5. Track two read positions per reader: handed out and acknowledged. Reading never moves the acknowledged one. The reader commits it after it has kept what it read.

Expect one surprise from step one. Retrying a lost claim gives back the original token, and if that claim has since expired, the token fails on the next write. That's right. The worker needs to know it lost the job.

Test it with two real processes on one database file, not mocks. My demo does that, but with scripted clients. Live agents are next. And remember the leases only bind workers that go through the API, so give each agent its own worktree.

Keep reading

Pick up a connected idea or branch into a nearby one.

5 paths forward
Continue the thread

Reorder approvals that lived in memory while the job outlived the request

The slowest assistant job, building a reorder, ran inside the request on an in-memory approval token, so a timeout or restart left nobody knowing whether a draft order existed. Now apply writes a synced journal entry before it answers, workers hold fenced leases, an old approval can never restart a job, and an expired lease reads as interrupted rather than failed.

via #idempotency · #distributed-systems
Related idea

The draft button checked for duplicates, then left a gap wide enough for a second one

The button that saves a delivery as a draft in the stores' inventory system checked first and wrote its record only after a slow outside call, so a second click could slip through. Now it claims the delivery in the database first, lets a partial unique index refuse the second claim, fixes a payload hash that couldn't see nested fields, and adds a narrow admin cancel for claims that get stuck.

via #concurrency · #idempotency