Skip to content
Curiosity
#api #reliability #distributed-systems

Idempotency keys on write endpoints

Why every non-GET endpoint that touches money or external state needs an idempotency key, and how to implement one without footguns.

Updated May 9, 2026
On this page

The problem teams keep rediscovering

A client retries a POST /payments. The first call succeeded but the response was lost. The retry creates a second payment. Now the system owes the user a refund and the team owes itself an incident review.

This happens every time a non-idempotent write meets an unreliable network. Which is always.

Postmortem
A typical double-charge incident (illustrative)
Symptom

Support tickets: “I was charged twice for one order.” Several arrive within hours after a brief network blip in a mobile client.

Investigation

Logs show the mobile app retrying POST /payments after a socket timeout. The first call had succeeded; the response never made it back. The retry is treated as a new request.

Root cause

POST /payments has no idempotency contract. The server has no way to recognise the retry.

Fix

Add Idempotency-Key header support. Store attempts keyed by the header in a dedicated table. Replays return the original response.

Prevention

Every new write endpoint ships with idempotency wired through the framework. A lint rule can flag POST handlers that don’t accept the header.

The contract

The client sends a header — Idempotency-Key: <opaque uuid> — that uniquely identifies the attempt, not the resource. The server promises:

  1. First time we've seen this key

    Process the request. Store the full response keyed by the header. Return it.

  2. Replayed key, completed

    Return the stored response. Do not re-process anything.

  3. Replayed key, still in flight

    Block (or return 409) until the original finishes, then return its response.

  4. Expire after the window

    24h is standard. The key + response can be garbage-collected after that.

The key is the client’s responsibility. They mint it once per logical operation and reuse it across retries.

The flow

The decision: where to store the record

Decision
Chose: Dedicated idempotency_records table

Durability matters more than the extra ~2ms. The database is already operated; a separate cache would be one more system to run. Hashing was tempting and would have been the wrong call.

✓ Dedicated idempotency_records table
Pros
  • Simple
  • Audit-friendly
  • Works with existing RDBMS
Cons
  • One extra round-trip per write
Redis with TTL
Pros
  • Fast
  • Built-in expiry
Cons
  • Another moving part
  • Loses durability guarantees on eviction
Hash the request body
Pros
  • No client cooperation
Cons
  • Two semantically identical retries with different timestamps charge twice — the standard footgun

The footguns

Anti-pattern
Hashing the request body to derive the key
Smell

Server-side: idempotencyKey = sha256(body). No header required.

Why it happens

It avoids a client API change. Looks like a free win.

Fix instead

Make the client mint a UUID per attempt. Two retries with a differing requestedAt timestamp will hash differently and double-charge.

Other common mistakes:

  • Storing only the resource ID, not the full response. On replay you can’t reconstruct the original 201 Created body.
  • Ignoring request-body equality. If the client reuses a key with a different body, that’s a client bug — return 422, don’t silently honour the cached response.
  • Forgetting the in-flight state. Concurrent retries arrive before the first finishes. Use INSERT ... ON CONFLICT DO NOTHING and treat the lookup as the source of truth.

What to remember

Related on the read side: 0001 JPA N+1 query problem · Java — same shape of “cheap up front, very expensive once in production”.

Related