Idempotent Commands, Deduplication Windows, Request IDs, and Exactly-Once Illusions

1. Ambiguous failure is why idempotency matters

AtlasMart sends CapturePayment(req-77, 50). The payment service commits the charge, but the network connection closes before the client receives the result. Retrying is necessary for reliability—but unsafe if the service cannot recognize that the retry represents the same logical command.

An operation is idempotent when applying the same intended operation multiple times has the same intended effect as applying it once. HTTP defines several request methods as idempotent, but application-level POST-like commands often need an explicit idempotency key or request ID. The server scopes that identity to the tenant/operation and stores enough durable result state to replay the original outcome.

2. The dedup record is part of the correctness contract

A useful dedup record includes request ID, tenant/account scope, operation type, a fingerprint or normalized payload identity, status, durable business-result identifier, response summary, creation time, and expiry/retention policy. Reusing one request ID with a different payload must be rejected; otherwise a retry identity becomes an accidental alias for a different command.

3. Retention windows mean the guarantee is bounded

If AtlasMart retains dedup entries for 60 minutes, a retry at minute 10 can replay the original result. A retry after the entry is deleted may execute again unless another durable business identity—such as unique payment intent ID or order ID—still prevents duplication. Therefore “idempotent for 60 minutes” and “this business action can never happen twice” are different promises.

The retention horizon must exceed realistic retry/replay delays, including queue backlogs, mobile offline periods, and disaster recovery replays. Longer retention costs storage/index space and may raise privacy/governance questions.

4. Side-effect ordering creates two classic crash windows

If a service writes the dedup marker before the external side effect and crashes, a retry may see “already processed” and suppress work that never happened. If it performs the side effect before writing the marker and crashes, a retry may repeat the side effect. The cleanest solution is to place the business mutation and dedup record in one local atomic transaction when they share a datastore.

When the side effect crosses systems, use a protocol such as transactional outbox/inbox, downstream idempotency, durable workflow state, and reconciliation. “Exactly once” cannot be inferred from one component's dedup table because failures can occur between components.

5. AtlasMart lab: response replay, mismatch rejection, and expiry

The lab charges twice under a naive retry, then introduces a dedup store that atomically records the business effect and response inside the simulator. It replays the original response, rejects reuse with a different amount, and demonstrates that a retry after expiry can execute again.

print("AMBIGUOUS PAYMENT TIMEOUT")
ledger = []

def naive_capture(request_id, amount):
    ledger.append((request_id, amount))
    return {"status": "captured", "amount": amount}

naive_capture("req-77", 50)
print("first capture committed; response lost")
naive_capture("req-77", 50)
print("naive ledger:", ledger, "charged total:", sum(x[1] for x in ledger))

print("\nIDEMPOTENT HANDLER WITH RESPONSE REPLAY")
ledger = []
dedup = {}

def capture(request_id, amount, now, ttl):
    row = dedup.get(request_id)
    if row and now <= row["expires_at"]:
        if row["amount"] != amount:
            raise ValueError("same request ID reused with different payload")
        return {**row["response"], "replayed": True}
    response = {"status": "captured", "amount": amount, "ledger_id": len(ledger) + 1}
    # Simulator treats business mutation + dedup record as one atomic durable boundary.
    ledger.append((request_id, amount))
    dedup[request_id] = {"amount": amount, "response": response, "expires_at": now + ttl}
    return {**response, "replayed": False}

print("first:", capture("req-77", 50, now=100, ttl=60))
print("retry:", capture("req-77", 50, now=110, ttl=60))
print("ledger:", ledger, "charged total:", sum(x[1] for x in ledger))

print("\nPAYLOAD MISMATCH IS REJECTED")
try:
    capture("req-77", 70, now=120, ttl=60)
except ValueError as e:
    print("rejected:", e)

print("\nDEDUP WINDOW IS A BOUNDED GUARANTEE")
print("after expiry:", capture("req-77", 50, now=170, ttl=60))
print("ledger now:", ledger, "charged total:", sum(x[1] for x in ledger))
print("late retry can repeat after retention unless business identity supplies a stronger invariant")

print("\nSIDE-EFFECT ORDERING FAILURE MODES")
print("marker-first then crash before charge -> retry may suppress a missing charge")
print("charge-first then crash before marker -> retry may duplicate a charge")
print("safe design needs one atomic boundary where possible, or an outbox/reconciliation protocol across boundaries")

6. Exactly-once is a claim about a whole end-to-end path

Messaging products may offer exactly-once features within carefully defined boundaries, but an application side effect can still be repeated outside that boundary. A consumer can atomically commit a broker offset yet fail to atomically commit a payment at a third-party provider. A producer can deduplicate broker writes while a downstream database transaction retries. The useful architecture statement is precise: “delivery is at least once; handlers are idempotent by event ID for N days; business rows have a unique operation identity; reconciliation detects divergence.”

RFC 9110 also illustrates why idempotency matters for retry: idempotent request semantics permit repeating a request after communication failure without changing the intended effect. For non-idempotent application commands, an explicit key plus server-side handling provides the analogous application contract.

7. Deliberately wrong approach: random ID per retry

If a client generates a new request ID every time it retries, the server cannot know the calls represent one logical command. The key must identify the business attempt, not the transport attempt. Conversely, reusing one key forever for unrelated commands causes false deduplication. Generate once per logical operation, persist it with client/workflow state, and reuse it only for retries of the same intent.

8. Production judgment and bridge to TTL/ephemeral state

Observe dedup hit rate, key-collision/mismatch rejects, age distribution of retries, entries near expiry, duplicate side-effect incidents, outbox/inbox backlog, and reconciliation discrepancies. Protect request IDs from cross-tenant probing, avoid putting secrets in keys, and rate-limit pathological replay. Test crash points before/after durable mutations and side effects.

Dedup retention introduces an expiry problem: when may state safely disappear, how exact is expiry timing, and what happens under replication or memory pressure? Chapter 19 continues directly with TTL, lazy versus active expiry, eviction, and ephemeral-state reconstruction.

References

Foundational claims use primary research/specifications where practical. Product documentation is used only as a current implementation example and is not required for the mandatory labs.

• RFC 9110 — HTTP Semantics, Idempotent Methods — Standards-track definition and retry rationale for idempotent HTTP methods.
• Stripe — Designing robust APIs with idempotency — Practical public implementation discussion of idempotency keys and ambiguous network failures.
• IETF HTTPAPI Idempotency-Key draft -07 — Work-in-progress header design; the -07 draft expired April 18, 2026, so it is not cited as a final RFC.
• Saito & Shapiro — Optimistic Replication — Background for duplicate/retry/reconciliation concerns in optimistic distributed systems.