The only place my idempotency key matched was a test that stopped time

Flow

Where each key should come from

1. 1Webhook seat report⌄

   The provider's event ID. It stays the same on every retry.

2. 2Checkout button⌄

   One attempt ID made when the page loads. A double click reuses it.

3. 3New subscription⌄

   Customer plus price, plus an attempt marker only where buying again is allowed.

4. 4Never in a key⌄

   The current time or a fresh random value.

Subscription keys only matched themselves in a test that froze the clock. The test sets time to one moment, makes a subscription, and checks the key: customer, price and that frozen time. In real use the clock keeps going, so two tries a second apart send two different keys. The payment provider can't tell it's the same request.

I was adding per-seat billing to an admin app that many companies share. When I reread that day's changes, I found the same mistake in three places.

What the provider does with a repeat

The provider runs the first request with a key and saves the answer. If the same key shows up again, it sends back the saved answer and does nothing new. That only works if every retry builds the same key. Put the time in it and every retry looks new.

My usage reporter built its key like this:

key = f"{customer_id}-{now_millis()}"

A comment above it said this stopped duplicates. It did the opposite.

Why the events keep coming back

The webhook handler checks the provider's signature first. Then three events report seats: a new subscription, a subscription turning active, and a finished checkout. Each one sends a usage record with that time-based key. If anything fails, the handler returns an error and the provider sends the event again later.

So if the usage call goes through but the reply gets lost, the retry sends a second record with a new key. If the meter keeps only the last number, that's fine. If it adds them up, someone pays for extra seats. The only note I found said it keeps the last number, and that was a doc that got deleted, not the live setting.

A good key got worse

The subscription key started the day as customer plus price, which was right. A later change, called a test fix, added the time to it. The test was changed to freeze the clock so it still passed. Checkout got a time-based key in the same change, and customer creation switched to a random key on every call.

Keys built from the request, not the time

Build the key from what you meant to do. The webhook can use the provider's event ID, which stays the same on every retry. The checkout button can get one attempt ID when the page loads. Subscriptions can use customer plus price. Then test that building the key twice gives the same answer, with the clock running.

I haven't made these changes yet.