What's the first step in a system design interview or a real design doc?

Clarify requirements: functional needs (what must it do), non-functional constraints (latency, availability, scale), and success metrics. Everything downstream — the API shape, the storage choice, the failure handling — follows from those, which is why skipping straight to components is the most common design mistake.

Why does a client retry a request at all?

Usually because it didn't get a clear answer — a timeout, a dropped connection, a 502 from a proxy — not because it wants to submit the same order twice. The server has no way to tell 'the client wants two orders' apart from 'the client is retrying one' unless the client sends something that identifies the attempt, which is exactly what an idempotency key is for.

Is there one correct system architecture for a given set of requirements?

No — there are tradeoffs. Two designs can both satisfy the same requirements while making different choices about cost, latency, consistency, and complexity; a good design states which tradeoffs it made and why, rather than presenting one option as objectively correct.

Should every endpoint support idempotency keys?

Only ones with a side effect worth deduplicating — creating an order, charging a card, sending a payment. A read-only GET is naturally idempotent already (asking twice doesn't create two of anything), so adding key-based deduplication to it would be bookkeeping with no benefit.

System Design Essentials: Requirements Into a Real API Contract

Requirements and API contracts stay theoretical right up until a client retries a request — the idempotency key is where system design theory turns into code that has to actually work.

Retrying a request shouldn’t create it twice

idempotency_store = {}
next_order_id = [1000]

def create_order(idempotency_key, user_id, items):
    if idempotency_key in idempotency_store:
        cached = idempotency_store[idempotency_key]
        print(f"  duplicate request, returning cached order {cached['orderId']}")
        return cached
    order_id = next_order_id[0]
    next_order_id[0] += 1
    response = {"orderId": order_id, "status": "CREATED"}
    idempotency_store[idempotency_key] = response
    print(f"  new order created: {response}")
    return response

print("Client sends the same request twice (e.g. after a timed-out response):")
create_order("key-abc-123", "user-1", ["widget"])
create_order("key-abc-123", "user-1", ["widget"])

print()
print("A different idempotency key creates a genuinely new order:")
create_order("key-xyz-789", "user-1", ["widget"])

$ python idempotent_create_order.py
Client sends the same request twice (e.g. after a timed-out response):
  new order created: {'orderId': 1000, 'status': 'CREATED'}
  duplicate request, returning cached order 1000

A different idempotency key creates a genuinely new order:
  new order created: {'orderId': 1001, 'status': 'CREATED'}

The second call with key-abc-123 never reaches the order-creation logic at all — it’s caught by the if idempotency_key in idempotency_store check and handed back the exact same response the first call produced. This is the concrete version of a requirement that’s easy to state in prose (“retries shouldn’t cause duplicate orders”) and easy to get wrong in code if the check is missing.

This is also where requirements become an API contract: the client has to know to generate and send an idempotency key, and the server has to document that the same key returns the same result rather than erroring. Neither side can improvise this at request time — it has to be part of the design before either is written.

Practice

1. Remove the idempotency check entirely and see what a retry actually does without it.

next_order_id = [1000]

def create_order_naive(user_id, items):
    order_id = next_order_id[0]
    next_order_id[0] += 1
    response = {"orderId": order_id, "status": "CREATED"}
    print(f"  new order created: {response}")
    return response

print("Client sends the same request twice with NO idempotency key:")
create_order_naive("user-1", ["widget"])
create_order_naive("user-1", ["widget"])
print("  (two separate orders now exist for the same intended purchase)")

$ python naive_no_idempotency.py
Client sends the same request twice with NO idempotency key:
  new order created: {'orderId': 1000, 'status': 'CREATED'}
  new order created: {'orderId': 1001, 'status': 'CREATED'}
  (two separate orders now exist for the same intended purchase)

Nothing here is “broken” in the sense of raising an error — that’s what makes this bug dangerous. It silently does the wrong thing (charges or ships twice) instead of failing loudly.

2. Detect a conflict when the same idempotency key is reused for a genuinely different request.

idempotency_store = {}
next_order_id = [1000]

def create_order(idempotency_key, user_id, items):
    if idempotency_key in idempotency_store:
        cached_request, cached_response = idempotency_store[idempotency_key]
        if cached_request != (user_id, tuple(items)):
            print(f"  CONFLICT: key {idempotency_key!r} was already used for a different request")
            return None
        print(f"  duplicate request, returning cached order {cached_response['orderId']}")
        return cached_response
    order_id = next_order_id[0]
    next_order_id[0] += 1
    response = {"orderId": order_id, "status": "CREATED"}
    idempotency_store[idempotency_key] = ((user_id, tuple(items)), response)
    print(f"  new order created: {response}")
    return response

create_order("key-abc-123", "user-1", ["widget"])
print("Same key, but a different item list this time:")
create_order("key-abc-123", "user-1", ["gadget"])

$ python conflicting_reuse.py
  new order created: {'orderId': 1000, 'status': 'CREATED'}
Same key, but a different item list this time:
  CONFLICT: key 'key-abc-123' was already used for a different request

Without this check, a buggy client that accidentally reuses a key for a different purchase would silently get back the wrong order — comparing the stored request, not just the key, turns that into a detectable error instead.

3. Measure how many requests get deduplicated when a flaky client retries some of them.

idempotency_store = {}
next_order_id = [1000]

def create_order(idempotency_key, user_id, items):
    if idempotency_key in idempotency_store:
        return idempotency_store[idempotency_key]
    order_id = next_order_id[0]
    next_order_id[0] += 1
    response = {"orderId": order_id, "status": "CREATED"}
    idempotency_store[idempotency_key] = response
    return response

# A client retries key-1 three times (e.g. flaky network) but key-2 and key-3 are only sent once
requests = ["key-1", "key-1", "key-1", "key-2", "key-3"]
for key in requests:
    create_order(key, "user-1", ["widget"])

print(f"Requests sent: {len(requests)}")
print(f"Orders actually created: {next_order_id[0] - 1000}")

$ python dedup_savings.py
Requests sent: 5
Orders actually created: 3

5 requests, only 3 distinct keys among them, so only 3 orders exist — the store’s job is exactly this collapsing of “how many times did the client ask” into “how many times did it actually need to happen.”

4. Add a TTL so an idempotency key stops being honored after 60 seconds.

idempotency_store = {}
next_order_id = [1000]
TTL_SECONDS = 60

def create_order(idempotency_key, user_id, items, now):
    if idempotency_key in idempotency_store:
        response, created_at = idempotency_store[idempotency_key]
        if now - created_at < TTL_SECONDS:
            print(f"  duplicate within TTL, returning cached order {response['orderId']}")
            return response
        print(f"  key expired ({now - created_at}s old), treating as a new request")
    order_id = next_order_id[0]
    next_order_id[0] += 1
    response = {"orderId": order_id, "status": "CREATED"}
    idempotency_store[idempotency_key] = (response, now)
    print(f"  new order created: {response}")
    return response

create_order("key-abc", "user-1", ["widget"], now=0)
create_order("key-abc", "user-1", ["widget"], now=30)
create_order("key-abc", "user-1", ["widget"], now=90)

$ python ttl_expiry.py
  new order created: {'orderId': 1000, 'status': 'CREATED'}
  duplicate within TTL, returning cached order 1000
  key expired (90s old), treating as a new request
  new order created: {'orderId': 1001, 'status': 'CREATED'}

At now=30, the key is 30 seconds old — still within the 60-second TTL, so it’s treated as a duplicate. By now=90, it’s aged out and a second order gets created — a real system needs this kind of expiry so the idempotency store doesn’t grow forever for keys nobody will ever retry again.

Retrying a request shouldn’t create it twice

Practice

Frequently Asked Questions