What Is Idempotency? Designing an Idempotency-Key to Prevent Double Charges on API Retries, Explained With a Real Experiment

1. What Is Idempotence?

Idempotence is the property that performing an operation once versus performing it multiple times leaves the system in the same resulting state. Mathematically, a function f is idempotent when it satisfies

f(f(x)) = f(x)

This sounds abstract, but everyday examples make it concrete.

• An idempotent operation: flipping a light switch to “on.” No matter how many times you flip an already-on switch to “on,” the room’s brightness doesn’t change. In programming, the assignment x = 5 is idempotent for the same reason — no matter how many times you run it, x stays 5.
• A non-idempotent operation: a toggle switch for on/off. One press flips the state; a second press flips it back — the result depends on how many times it runs. In programming, x += 1 is the classic example: every execution changes the value of x further.

Carrying this property over into the context of an API (particularly a Web API) gives us:

An API request is idempotent when sending it once to the server, or accidentally sending it multiple times due to network conditions, produces the same “intended effect” on the server either way.

This isn’t a definition of my own invention — it’s exactly the definition adopted by RFC 9110, the formal HTTP specification discussed below, and it forms the foundation for the rest of this article. Importantly, idempotence does not require that the response be identical every time. The first request might return 201 Created with a newly created resource’s ID, and the second (a retransmission of the same request) might return 200 OK with the existing resource’s information — as long as the intended effect on the server, e.g. “exactly one record exists for that order,” stays the same, the operation is still considered idempotent.

2. Why Idempotence Is Necessary: Retries and Duplicate Delivery

The biggest reason idempotence matters in practice is the reality that communication over a network has to be designed around the assumption that retries will happen. As touched on in the article on exponential backoff and jitter , timeouts and retries are basic building blocks for constructing reliable distributed systems, and that article closed by noting that “retries assume idempotence as a precondition. Retrying an API with side effects, such as creating a payment, as-is can cause duplicate processing.” This article is a deep dive into that single sentence.

Here’s how a retry turns into a duplicate, step by step:

1. A client sends a request to a payment API.
2. The server successfully completes the processing (the charge succeeds) and tries to return the result (the ACK) to the client.
3. But the response is lost somewhere on the return trip over the network, or the client times out.
4. The client, unable to tell whether the request got through, retransmits the same request.
5. The server receives this as a new request and executes the payment processing again.

The fundamental problem here is that the client has no reliable way to know whether the server actually finished processing. This is exactly the structure exposed by the classic communication-theory thought experiment known as the “Two Generals’ Problem,” which shows that “over an unreliable communication channel, no finite exchange of messages can guarantee that both parties have reached certain agreement”¹. Because the ACK itself can be lost, the client cannot externally distinguish between “the server processed it but the ACK was lost” and “the server never received it in the first place.”

This is often phrased as “exactly-once delivery is impossible.” In fact, Amazon SQS’s standard queues, a managed queueing service from AWS, explicitly state in their official documentation that they “ensure at-least-once message delivery, but due to the highly distributed architecture, more than one copy of a message might be delivered… and messages may occasionally arrive out of order,” and their recommended use cases assume “applications that can handle messages that might arrive more than once or out of order”².

In other words, the industry’s practical conclusion is not to try to make delivery itself exactly-once, but rather to settle on a design where “delivery remains at-least-once (duplicates are possible), and making the receiver’s processing idempotent yields the same effect as exactly-once (effectively-once).” The Idempotency-Key pattern discussed in this article is one of the most concrete implementations of that approach.

3. Idempotence of HTTP Methods: The RFC 9110 Definition

How idempotent HTTP methods ought to be is spelled out clearly in RFC 9110, the specification defining HTTP semantics. Section 9.2.2 of that RFC defines it as follows:

A request method is considered “idempotent” if the intended effect on the server of multiple identical requests with that method is the same as the effect for a single such request.³

Building on this, RFC 9110 states that, among the methods it defines, “PUT, DELETE, and safe request methods are idempotent”³. “Safe methods” are defined in RFC 9110 §9.2.1 as the four methods GET, HEAD, OPTIONS, and TRACE. This is summarized in the table below.

RFC 9110 also explains, in the same §9.2.2, why idempotent methods are singled out for special treatment: “Idempotent methods are distinguished because the request can be repeated automatically if a communication failure occurs before the client is able to read the server’s response,” while cautioning that “a client SHOULD NOT automatically retry a request with a non-idempotent method unless it has some means to know that the request semantics are actually idempotent”³. In other words, safely auto-retrying a non-idempotent method like POST requires establishing, separately from the method type, an application-layer guarantee that “this particular request is idempotent.” That guarantee is exactly what the Idempotency-Key pattern, discussed next, provides. As for PATCH in the table, it’s defined not in RFC 9110 but in RFC 5789, and is not declared idempotent — so while an implementation can be designed to be idempotent, that remains the implementer’s responsibility rather than a guaranteed property of the method itself.

4. The Idempotency-Key Pattern: How Stripe and the IETF Draft Design It

The widely used solution for safely making non-idempotent methods like POST retry-safe is the Idempotency-Key pattern. Stripe, the payments platform, was among the earliest to run this in production, and its implementation is often cited as a de facto industry standard. According to Stripe’s official documentation, the mechanism can be summarized as follows⁴.

Generating the key is the client’s responsibility. Stripe recommends “a V4 UUID, or a random string with sufficient entropy” (maximum length 255 characters, avoiding sensitive data such as email addresses), and the client attaches it via the Idempotency-Key request header. The server processes the first request normally, then stores a fingerprint of that request’s parameters together with the response (status code and body), keyed by the idempotency key. This storage happens regardless of success or failure — the documentation explicitly states that “subsequent requests with the same key return the same result, including 500 errors.” When a second or later request arrives with the same key and matching parameters, the server does not re-execute the charge and instead returns the stored response as-is; if the parameters differ, it returns an error to guard against misuse of the key. If a second request with the same key arrives while the first is still in-flight, the result is not saved, and the client is told it’s safe to retry. Keys are stored for at least 24 hours, and once a key expires, reusing it is treated as a brand-new request. Note also that GET/DELETE requests, being inherently idempotent HTTP methods, don’t need an Idempotency-Key at all (this point is elaborated on in the FAQ).

Beyond Stripe, there’s an ongoing effort to define Idempotency-Key as a standard HTTP header. A leading example is draft-ietf-httpapi-idempotency-key-header, under consideration by the IETF’s httpapi working group⁵. As of this writing (July 2026), the latest version available (version 07, dated October 2025) has expired as a draft, but it remains a valuable primary source pointing toward the direction of standardization. Its contents largely align with Stripe’s implementation, but one point is more specific: when a second request with the same in-flight key arrives, the server SHOULD return a 409 Conflict. On key expiration, the draft says only that the resource owner (the server side) SHOULD define its own policy and publish it — meaning that even at the IETF level, there’s no single “correct” retention period defined.

Taken together, these two primary sources — Stripe’s battle-tested production implementation and the IETF draft attempting to standardize it — suggest that the essence of the Idempotency-Key pattern is “adding a small caching layer on the server that remembers a request’s content together with its result.” The next section quantitatively verifies how much double charging actually occurs, with and without this mechanism.

5. Experiment: A Double-Charge Simulation Under At-Least-Once Delivery

5.1 Model Setup

We reproduce the “at-least-once delivery + ACK loss” scenario described in Section 2 with a simple discrete simulation. The model is as follows.

• When a client sends a payment request, the server always receives and processes it (we assume the outbound leg of the network never loses the request).
• The ACK (response) carrying the processing result is lost with probability p. A client that doesn’t receive an ACK, unsure whether the server processed the request, retransmits the identical request.
• The ACK for the retransmission can itself be lost with the same probability p, so retries can in principle continue indefinitely until an ACK gets through. The simulation caps this at 20 retries as a safety margin, but for p ≤ 10% the probability of 20 consecutive ACK losses is astronomically small (0.1^20), so this cap is essentially irrelevant to the results.
• We simulate 10,000 payment requests with a fixed random seed of 42, comparing ACK loss rates p of 1%, 5%, and 10%.

Two receiver-side implementations are compared:

• (a) No idempotence: the server unconditionally processes every request it receives as a new charge. Retries caused by ACK loss directly become double (or more) charges.
• (b) With an Idempotency-Key: the server detects duplicates using the key embedded in the request. Any receipt of the same key after the first simply returns the stored response, with no new charge.

The core of the simulation (Python with numpy installed in a venv, scratchpad/idempotency/sim_idempotency.py) is as follows.

def simulate_attempts(n_requests, p_ack_loss, max_retries, seed):
    """For each payment request, returns how many times it was actually delivered to the server."""
    rng = np.random.default_rng(seed)
    attempts = np.ones(n_requests, dtype=np.int64)
    still_retrying = np.ones(n_requests, dtype=bool)
    for _ in range(max_retries):
        draws = rng.random(n_requests)
        lost = still_retrying & (draws < p_ack_loss)
        attempts[lost] += 1
        still_retrying = lost  # requests whose ACK arrived (or hit the cap) stop retrying
        if not still_retrying.any():
            break
    return attempts

def summarize(attempts):
    # (a) No idempotence: any request with attempts>=2 gets double- (or multi-) charged
    double_charge_incidents = int(np.sum(attempts >= 2))
    total_extra_charges = int(np.sum(attempts - 1))
    # (b) Idempotency-Key: double charges are always 0. Duplicate receipts (= prevented count) match the above
    return double_charge_incidents, total_extra_charges

For each payment request, we count how many times in a row the ACK was lost, following a geometric-style distribution; whenever attempts >= 2, at least one retry — i.e., a duplicate delivery to the server — occurred. In the no-idempotence implementation, this directly becomes the double-charge count; in the Idempotency-Key implementation, the same count is tallied as “duplicates prevented.”

5.2 Measured Results

The measured results for N=10,000 requests with a fixed seed of 42 are as follows.

The results first show that without idempotence, double charges rise roughly in proportion to the ACK loss rate. Out of 10,000 requests, the double-charge count was 101 at p=1%, 508 at p=5%, and 984 at p=10% — the loss rate shows up almost directly as the double-charge rate (1.01% / 5.08% / 9.84%). This is by no means a rare edge case: on unstable networks such as mobile carrier networks or mobile apps, an ACK loss rate of 5–10% is entirely realistic, meaning double charges could occur on the order of one in a few dozen requests.

By contrast, with Idempotency-Key deduplication, double charges were completely zero at all three loss rates. What’s interesting is that “duplicates prevented” matches the “total extra charge events” from the no-idempotence implementation (532 at p=5% and 1,082 at p=10% are slightly larger than the corresponding double-charge counts, since they also include multi-retry cases with attempts >= 3), confirming that the Idempotency-Key mechanism doesn’t just catch a single duplicate — it faithfully blocks every duplicate no matter how many retries occur.

6. Implementation Notes

To realize the effects shown by the simulation in a real system, here are the points worth attending to.

6.1 Where to Store Keys, and TTL

The safest place to store the Idempotency-Key/response pair is in the same database table as the payment processing itself. A separate store like Redis for keys alone is faster, but it can introduce inconsistencies — e.g., “the charge succeeded but saving the key failed” — so it’s preferable to commit the payment transaction and the key storage within the same DB transaction. There’s no single correct TTL value (the IETF draft says the same), but the basic policy is to set it comfortably longer than the client’s retry grace period; Stripe’s “at least 24 hours” is a useful reference for a realistic lower bound.

6.2 Handling the Same Key Arriving While Still In-Flight

The simulation doesn’t model this for simplicity, but production systems must handle the case where a second request with the same key arrives before the first has finished processing. Naively treating it as a new request just because “there’s no key yet” causes two processes to run concurrently, defeating the whole point of preventing double charges. As the IETF draft recommends with 409 Conflict, you need a design that explicitly locks the key into an “in-progress” state, returning a “please retry later” error to any request with the same key that arrives during that window. On an RDB, this can be achieved by combining a row lock on the key column (SELECT ... FOR UPDATE) with the unique-constraint error handling described below.

6.3 Storing the Response

Stripe’s explicit statement that “subsequent requests with the same key return the same result, including 500 errors” reflects an important design decision. Storing and replaying failed results as well as successful ones avoids unpredictable behavior where a client expects different behavior after a prior failure. At minimum, the HTTP status code and response body should be stored.

6.4 The Simplest Implementation: A DB Unique Constraint

The Idempotency-Key implementation might sound complex, but in its simplest form, basic deduplication can be achieved just by placing a unique constraint on an idempotency_key column in the charges table.

CREATE TABLE charges (
    id BIGSERIAL PRIMARY KEY,
    idempotency_key VARCHAR(255) NOT NULL,
    amount INTEGER NOT NULL,
    status VARCHAR(20) NOT NULL,
    response_body JSONB,
    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
    UNIQUE (idempotency_key)
);

Attempting an INSERT into this table, and treating a unique-constraint violation (e.g. 23505) as “already processed with this key,” then looking up the existing row and returning its stored response_body as-is — this is the easiest first step, achievable without introducing a dedicated cache layer or lock mechanism. However, the “same key arrives while still in-flight” scenario from 6.2 isn’t solved by a unique constraint alone; it needs to be combined with row locking.

7. Operations That Cannot (or Are Hard to) Be Made Idempotent

Not every operation can be made idempotent. Here are the representative cases where applying idempotence is difficult.

• Sequential number issuance: processes that issue sequence numbers that must never repeat system-wide, such as invoice numbers or slip numbers, are inherently non-idempotent in their simplest form — “hand out the next number every time it’s called.” To make this operation safely retryable, the Idempotency-Key needs to be applied to the numbering logic itself, with a table design where “a repeated request with the same key returns the previously issued number instead of issuing a new one.” If only the payment processing is made idempotent while the internal numbering logic is left untouched, you can end up in a half-fixed state where the charge doesn’t duplicate but the number does.
• Side effects on external services: even if your own API is made idempotent via an Idempotency-Key, if a third-party API you call internally (credit checks, shipping instructions to a carrier, etc.) doesn’t support idempotency keys, room for duplication remains there. Where the downstream service doesn’t support it, you need to track on your own side whether the call has already been made before making it.
• Notifications such as email/SMS: notifications that reach the user directly, like a payment confirmation email, can be sent twice if the notification logic runs independently, even when the internal charge processing is fully idempotent. A typical countermeasure is the transactional outbox pattern, in which the commit of the payment processing and the registration of the email-sending job happen in the same transaction, combined with the sending worker itself tracking already-sent message IDs to prevent resends — a two-layer approach to idempotence that tends to be the realistic landing point.

What these cases have in common is that idempotence isn’t something you can guarantee at a single entry point (the API endpoint) alone — for every side effect that cascades from there, you need to decide, as a design judgment, how far idempotence should extend.

8. Conclusion

This article defined idempotence as the API-level version of the mathematical property f(f(x)) = f(x), then organized the idempotence of each HTTP method as defined by RFC 9110, and the concrete mechanics of the Idempotency-Key pattern as defined by Stripe and the IETF draft, from primary sources. Building on that, it simulated a realistic scenario of at-least-once delivery combined with ACK loss and quantitatively confirmed the following:

• Without idempotence, double charges occur roughly in proportion to the ACK loss rate (measured: 101 at p=1%, 508 at p=5%, and 984 at p=10%, out of 10,000).
• With Idempotency-Key deduplication, double charges under the same conditions drop to completely zero.
• The Idempotency-Key prevents not just single duplicates but every duplicate arising from multiple retries (1,082 at p=10%).

Given the basic constraint in distributed systems that “exactly-once delivery is impossible in principle,” idempotence design is an unavoidable foundation for retrying safely, and both the primary sources and the measured results here confirm that the Idempotency-Key pattern is the most practical way to implement it.

FAQ

Q1. Does a GET request also need an Idempotency-Key? A. No. By RFC 9110’s definition, GET is inherently both safe and idempotent, and Stripe’s documentation explicitly states that “GET/DELETE don’t need to send an Idempotency-Key.” An Idempotency-Key is only needed for methods like POST that don’t guarantee idempotence on their own.

Q2. Who generates the Idempotency-Key? A. The client generates it. Stripe recommends “a V4 UUID, or a similar random string with sufficient entropy” — the server does not issue the key on the client’s behalf. A client is expected to mint a unique key per semantic unit of the request (e.g., a single user action) and reuse that same key across retries.

Q3. How long should the key’s TTL be set? A. There is no single absolute value agreed upon as an industry standard. The IETF draft states only that “the server side should define and publish its own policy.” Stripe’s production practice is “at least 24 hours,” and setting it comfortably longer than the client’s retry grace period is a realistic starting point.

Q4. Isn’t a DB unique constraint alone enough? A. It’s an effective first step toward the minimal goal of preventing duplicate “creation,” but it falls short in two ways. First, when the unique-constraint violation (error) occurs, you need to additionally implement logic that looks up and returns the stored response rather than simply surfacing the error to the client as-is. Second, a unique constraint alone cannot detect the “in-flight conflict” scenario where a second request with the same key arrives before the first is committed — that requires combining it with row locking or other exclusive-control mechanisms.

Related Articles

• Exponential Backoff and Jitter Explained: The Retry Design Playbook, Verified by Simulation - This article is a deep dive into the line “retries assume idempotence as a precondition.” The design of retries themselves (backoff, jitter, circuit breakers) is covered in detail there.
• Modeling Database Connection Pool Size With Queueing Theory (M/M/c) - Covers DB-side concurrency control — the foundation for the unique constraints and row locking discussed in this article — from a queueing-theory perspective.

For staple books in other areas, see 10 Recommended Technical Books for Engineers .

References