Skip to main content

Preventing Duplicate Transactional Sends With Idempotency Keys

Stop duplicate receipts and double password resets: derive stable idempotency keys from business events, understand provider key windows, and add a local guard.

Duplicate receipts are one of the few email defects a customer will report immediately, and they almost never come from a bug in your send code. They come from a retry after a response that was lost in transit. This guide covers deriving a key that makes the retry harmless, and the local guard that covers the cases the provider's key cannot.

Root Cause: The Response Is Lost, Not the Request

A send call has three possible outcomes, and only two of them are visible to the caller. The request may fail before the provider receives it, in which case nothing was sent and a retry is correct. It may succeed and return, in which case everything is clear. Or it may succeed at the provider and fail on the way back — a timeout, a dropped connection, a load balancer terminating an idle socket.

From your side, the third case is indistinguishable from the first. Both surface as an exception with no response body. A retry policy that cannot tell them apart must retry, because the alternative is silently dropping messages that genuinely failed — and each retry of the third case produces one more copy in the recipient's inbox.

An idempotency key resolves the ambiguity by moving it to the party who knows the answer. The provider recorded the first request under that key; when the same key arrives again it returns the original result rather than performing the work a second time.

Why a retry cannot tell the cases apart A request that never arrived and a request whose response was lost look identical to the caller, so only the provider can distinguish them. Two Failures Look the Same From Here never arrived connection refused Nothing was sent. A retry is correct. caller sees: an exception response lost timeout after acceptance The message was sent. A retry duplicates it. caller sees: an exception with a stable key provider recognises it The retry returns the original result. exactly one message sent
The caller cannot distinguish the first two cases and should not try — the key moves the decision to the only party with the answer.

The Exact Fix: Derive the Key From the Business Event

A key generated at send time defeats the mechanism, because each retry generates a different one. The key must be a deterministic function of what the message is, computable identically on every attempt.

# keys.py — a key that is identical on every attempt for the same logical message.
import hashlib


def send_key(event_type: str, entity_id: str, recipient: str, version: str = "v1") -> str:
    """Stable across retries, distinct across genuinely different sends.

    event_type  the reason for the message: "order.receipt", "password.reset"
    entity_id   the specific thing it is about: the order id, the reset token id
    recipient   the address, so a re-send to a corrected address is a new message
    version     bumped deliberately when a resend of the SAME event is intended
    """
    material = f"{version}:{event_type}:{entity_id}:{recipient}".lower()
    return hashlib.sha256(material.encode()).hexdigest()[:48]


# Correct: derived from the order, so every retry produces the same key.
key = send_key("order.receipt", order.id, customer.email)

# Wrong: a fresh value per attempt means the provider sees each retry as new work.
# key = str(uuid.uuid4())

Including the recipient in the material matters more than it first appears. If a receipt is initially sent to a mistyped address and then re-sent to a corrected one, those are genuinely different messages and must not collapse to one key. Including a version component gives you a deliberate escape hatch: a support agent re-sending a receipt on request bumps the version, producing a new key and an intentional second copy.

Provider Key Windows

Providers remember keys for a bounded period, and the window is shorter than most retry budgets assume.

Provider Mechanism Retention
Amazon SES x-ses-idempotency-key on v2 send Minutes, best-effort
SendGrid Custom args plus your own guard No native key
Postmark Message stream deduplication Short, per stream
Mailgun v:idempotency variable plus your own guard No native key

The gap this leaves is real: a message that dead-letters, is triaged and is replayed an hour later will be outside every provider's window. A local guard closes it.

-- One row per logical message. The unique constraint is the guarantee; the
-- application-level check is only an optimisation in front of it.
CREATE TABLE sent_message (
  idempotency_key  text PRIMARY KEY,
  provider_message_id text,
  recipient        text        NOT NULL,
  event_type       text        NOT NULL,
  sent_at          timestamptz NOT NULL DEFAULT now()
);
def send_once(db, esp, key, **message):
    # Claim the key first. If another worker already holds it, this raises and
    # we know the message is already sent or in flight.
    try:
        db.execute(
            "INSERT INTO sent_message (idempotency_key, recipient, event_type) "
            "VALUES (%s, %s, %s)",
            (key, message["to"], message["event_type"]),
        )
    except UniqueViolation:
        return {"deduplicated": True}

    result = esp.send(idempotency_key=key, **message)
    db.execute(
        "UPDATE sent_message SET provider_message_id = %s WHERE idempotency_key = %s",
        (result.message_id, key),
    )
    return result

Claiming the key before the send rather than after is what makes this correct under concurrency. Recording it afterwards leaves a window in which two workers both check, both find nothing, and both send.

Variant: When a Duplicate Is Actually Wanted

Some messages are legitimately sent more than once with the same content: a reminder, a re-sent verification code the user requested again, a monthly statement. These are not duplicates — they are distinct events that happen to look alike.

Key material for repeats and retries A retry of the same event reuses every component of the key material, while a legitimate repeat differs in the entity or version component. The Material Decides What Counts as a Repeat a retry — same key reset token 88f2, same address, v1 deduplicated — one message sent Every component is unchanged. a second request — new key reset token c41a, same address, v1 sent — a genuinely new message The token id differs, so the key does.
Include the component that makes each occurrence specific, and wanted repeats separate themselves without any special handling.

Model them as such. A re-requested verification code has a new token, so the token id in the key material differs and the key is naturally distinct. A monthly statement includes the period. The rule is that if the key material contains everything that makes the message a specific occurrence, wanted repeats produce distinct keys automatically and unwanted ones do not.

Pipeline Integration

Compute the key at enqueue time, not in the worker, and carry it in the job payload. A key computed inside the worker is recomputed on each attempt, and any drift in the inputs — a lazily-loaded field, a changed timestamp — produces a different key on the retry, which is the exact failure the mechanism exists to prevent.

Pass the same key to both the local guard and the provider. The two protect different windows: the local table covers replays hours or days later, the provider's key covers the tight retry loop where your own insert may not yet have committed.

Validation and Deployment Checklist

Debugging: Named Symptoms

Symptom: duplicates persist despite a key being passed. Cause: the key is being generated per attempt rather than carried in the job. Fix: log the key on every attempt and confirm it is identical across retries of the same message — if it differs, it is being computed rather than carried.

Symptom: a legitimate re-send is silently swallowed. Cause: the key material does not include anything that distinguishes the occurrences, so a genuinely new message collides with an old one. Fix: include the specific token, order revision or period in the material, or bump the version component for a deliberate re-send.

Symptom: duplicates only appear during deploys. Cause: a rolling restart interrupting in-flight sends after the provider accepted them, with the replacement worker retrying under a freshly generated key. Fix: carry the key in the payload so the replacement worker inherits it.

Computing the key at enqueue versus in the worker A key computed at enqueue time is carried in the payload and survives a restart, while a key computed inside the worker is regenerated by the replacement. Compute It Once, Carry It Everywhere computed in the worker worker restarts mid-send replacement generates a new key The provider sees unrelated work. computed at enqueue key travels in the payload replacement reuses the same key The provider recognises the repeat.
A key that lives in the job payload survives every restart, redeploy and replay; one computed in the handler does not.

Frequently Asked Questions

How long should the local guard table retain rows?

Longer than your longest replay window — thirty days is a reasonable default for transactional mail. The table grows by one row per message sent, so a retention job is required at volume, but pruning too aggressively re-opens the window that the provider's short key retention already leaves open.

Does this protect against a user clicking "send" twice?

Only if the two clicks produce the same key material. A user requesting two password resets legitimately generates two tokens, so the keys differ and both messages send — which is correct, since the first token may already have been invalidated. Debouncing user-initiated actions is a separate concern, handled at the application layer rather than in the send path.

What if the insert succeeds but the send then fails permanently?

The key is claimed for a message that was never delivered, so a later retry will be deduplicated away. Delete the row when a send dead-letters permanently, or record the outcome on the row and treat a failed status as un-claimed on replay. Claiming without releasing is the one way this pattern can lose a message.

Is a hash necessary, or can the key be readable?

Readable keys are fine and easier to debug — order.receipt:12345:a@example.com works. Hashing helps only where the key length is constrained by the provider or where the material contains something you would rather not send to a third party, such as an address in a system where that is sensitive.

← Back to Email Queueing and Retry Architecture