Back to Feed
article 2m read

Half a project in the tracker, then the export button forgets it did

Reading back a new export that pushes a planning document into an issue tracker, I found it keeps every created ticket ID in memory until the end, so one failed call leaves orphan tickets and a retry builds them all again.

Article DepthPractice

Flow

An export that can pick up where it stopped

  1. 1Claim the export

    Set in progress only if it isn't already set, with an expiry.

  2. 2Read saved IDs

    Load the ticket IDs already stored on the document.

  3. 3Skip what exists

    Leave items with an ID alone, and reuse a saved epic ID for missing stories.

  4. 4Create, then save

    Store each new ID right after the tracker returns it.

  5. 5Mind the gap

    A crash between create and save can still strand one ticket.

Half a project in the issue tracker is what the new export button can make before forgetting it was ever there.

A planning document has a few epics, each with some stories. One button turns that into tracker tickets. I didn't look hard at the loop when it went in. When I read it back, I saw it only works if every call works.

Where the memory lives

The export makes tickets one by one, epics first, since each story needs its epic's tracker ID. Each ID the tracker sends back is kept in memory. Only at the very end does the code save those IDs onto the document.

If a call fails halfway, the IDs in memory are lost. The document gets a failed status and an error message, and nothing else. Some tickets now exist in the tracker, and my app has no record of them.

The second click

Click export again and it starts from the top. It doesn't check for saved IDs, so it makes every ticket a second time.

Two clicks at once also both get through, because the "in progress" flag is set without checking what it was before. By my reading, the run that finishes last overwrites the other's IDs, so a full set of tickets ends up orphaned.

I haven't seen this happen. There are no tests for it and no failure data. It's from reading the code.

Saving each ticket as it lands

If you're building something like this, the changes are small:

  • Save each ticket's ID to the document right after the tracker returns it.
  • Only set "in progress" if it isn't already set, and let that claim expire so a crash can't lock the document.
  • On every run, skip anything that already has an ID. If an epic made it but some stories didn't, reuse the epic's saved ID.

A retry now creates only the tickets that don't have an ID yet.

One gap stays open. If the app crashes after the tracker says yes but before the save, one ticket is stranded. A key the tracker uses to spot repeats would cover that. So would a tag on each ticket that I can search before creating.

Keep reading

Pick up a connected idea or branch into a nearby one.

5 paths forward
Continue the thread

Reorder approvals that lived in memory while the job outlived the request

The slowest assistant job, building a reorder, ran inside the request on an in-memory approval token, so a timeout or restart left nobody knowing whether a draft order existed. Now apply writes a synced journal entry before it answers, workers hold fenced leases, an old approval can never restart a job, and an expired lease reads as interrupted rather than failed.

via #reliability · #idempotency · #distributed-systems