Is retrying always the right response to a failure?

No — only for failures that are plausibly transient (timeouts, 500s, 429s) and for operations safe to repeat (idempotent, or made idempotent per the previous lesson). Retrying a validation error just wastes time re-running a request that will fail identically every time, so a well-behaved retry loop checks the failure type first and gives up immediately on anything non-transient.

Why cap the backoff instead of letting it keep doubling?

Uncapped exponential growth means a client that's been failing for a while ends up waiting minutes or hours between attempts, which is rarely what anyone wants — a cap (2 seconds here) keeps the worst case bounded while still backing off aggressively early on, when backing off matters most.

How is jitter different from just picking a slightly different backoff formula?

Jitter adds randomness on top of the deterministic backoff specifically to decorrelate multiple clients that failed at the same moment (a dependency going down affects everyone calling it at once) — without jitter, every client retrying the same attempt number gets the identical wait and they all hammer the dependency again in lockstep.

Does this replace circuit breakers?

No — they solve different problems and stack together. Retry-with-backoff handles a single caller's individual request; a circuit breaker (covered in the microservices topic) stops a caller from even attempting a call once a dependency is known to be down, which retries alone can't do.

Scaling & Reliability: Retries With Backoff and Jitter

Retrying immediately at a fixed interval just re-hits a struggling dependency at full speed; backing off — and jittering that backoff — actually gives it room to recover.

A flaky call, backed off and retried until it succeeds

import random
import time

random.seed(42)

class TransientError(Exception):
    pass

def call_with_retry(fn, max_attempts=4):
    for attempt in range(1, max_attempts + 1):
        try:
            return fn()
        except TransientError as e:
            if attempt == max_attempts:
                raise
            backoff = min(2.0, 0.1 * (2 ** (attempt - 1)))
            jitter = random.uniform(0, 0.05)
            wait = backoff + jitter
            print(f"  attempt {attempt} failed ({e}), waiting {wait:.2f}s before retry")
            time.sleep(wait)
    raise RuntimeError("unreachable")

attempts = {"count": 0}

def flaky_call():
    attempts["count"] += 1
    if attempts["count"] < 3:
        raise TransientError("gateway busy")
    return "success"

start = time.perf_counter()
result = call_with_retry(flaky_call)
elapsed = time.perf_counter() - start
print(f"Result: {result} after {attempts['count']} attempts, {elapsed:.2f}s total")

$ python retry_with_backoff.py
  attempt 1 failed (gateway busy), waiting 0.13s before retry
  attempt 2 failed (gateway busy), waiting 0.20s before retry
Result: success after 3 attempts, 0.33s total

flaky_call fails on its first two calls and succeeds on the third — a real, played-out sequence, not a described one. The waits themselves grow (0.1 * 2^0, then 0.1 * 2^1, each plus a small random jitter), and the total elapsed time genuinely includes both real sleep calls, which is why it lands a little above their sum rather than exactly on it.

Practice

1. Make the call always fail and confirm retries exhaust and the error propagates.

import random
import time

random.seed(42)

class TransientError(Exception):
    pass

def call_with_retry(fn, max_attempts=3):
    for attempt in range(1, max_attempts + 1):
        try:
            return fn()
        except TransientError as e:
            if attempt == max_attempts:
                raise
            backoff = min(2.0, 0.1 * (2 ** (attempt - 1)))
            jitter = random.uniform(0, 0.05)
            wait = backoff + jitter
            print(f"  attempt {attempt} failed ({e}), waiting {wait:.2f}s before retry")
            time.sleep(wait)
    raise RuntimeError("unreachable")

def always_fails():
    raise TransientError("gateway permanently down")

try:
    call_with_retry(always_fails, max_attempts=3)
except TransientError as e:
    print(f"All retries exhausted, final error: {e}")

$ python exhaust_retries.py
  attempt 1 failed (gateway permanently down), waiting 0.13s before retry
  attempt 2 failed (gateway permanently down), waiting 0.20s before retry
All retries exhausted, final error: gateway permanently down

The third and final attempt’s failure isn’t followed by another wait — if attempt == max_attempts: raise re-raises immediately instead of sleeping before a retry that will never happen.

2. Compare 5 clients' first-retry wait times with and without jitter.

import random

random.seed(7)

def wait_without_jitter(attempt):
    return min(2.0, 0.1 * (2 ** (attempt - 1)))

def wait_with_jitter(attempt):
    backoff = min(2.0, 0.1 * (2 ** (attempt - 1)))
    return backoff + random.uniform(0, 0.05)

print("5 clients retrying attempt 1, no jitter:")
waits = [wait_without_jitter(1) for _ in range(5)]
print(f"  wait times: {[round(w, 3) for w in waits]}")
print(f"  all identical? {len(set(waits)) == 1}")

print()
print("5 clients retrying attempt 1, with jitter:")
waits = [wait_with_jitter(1) for _ in range(5)]
print(f"  wait times: {[round(w, 3) for w in waits]}")
print(f"  all identical? {len(set(waits)) == 1}")

$ python jitter_decorrelation.py
5 clients retrying attempt 1, no jitter:
  wait times: [0.1, 0.1, 0.1, 0.1, 0.1]
  all identical? True

5 clients retrying attempt 1, with jitter:
  wait times: [0.116, 0.108, 0.133, 0.104, 0.127]
  all identical? False

Without jitter, every client that failed at the same instant retries at the exact same instant too — a synchronized retry storm hitting the dependency right as it’s trying to recover. Jitter spreads that same set of clients across a small window instead.

3. Show the backoff plateau across 8 attempts with a 2-second cap.

def backoff_for(attempt, cap=2.0):
    return min(cap, 0.1 * (2 ** (attempt - 1)))

for attempt in range(1, 9):
    print(f"attempt {attempt}: backoff = {backoff_for(attempt):.2f}s")

$ python backoff_cap.py
attempt 1: backoff = 0.10s
attempt 2: backoff = 0.20s
attempt 3: backoff = 0.40s
attempt 4: backoff = 0.80s
attempt 5: backoff = 1.60s
attempt 6: backoff = 2.00s
attempt 7: backoff = 2.00s
attempt 8: backoff = 2.00s

The doubling stops mattering once 0.1 * 2^(attempt-1) exceeds the 2-second cap — from attempt 6 onward, min(cap, ...) always returns the cap itself, no matter how many more attempts follow.

4. Skip retries entirely for a non-retryable validation error.

class TransientError(Exception):
    pass

class ValidationError(Exception):
    pass

def call_with_retry(fn, max_attempts=3):
    for attempt in range(1, max_attempts + 1):
        try:
            return fn()
        except TransientError as e:
            if attempt == max_attempts:
                raise
            print(f"  attempt {attempt} failed (retryable: {e}), retrying")
        except ValidationError as e:
            print(f"  attempt {attempt} failed (NOT retryable: {e}), giving up immediately")
            raise
    raise RuntimeError("unreachable")

def bad_request():
    raise ValidationError("item_count must be positive")

try:
    call_with_retry(bad_request)
except ValidationError:
    print("Caller correctly sees a validation error, not silently retried 3 times")

$ python non_retryable_error.py
  attempt 1 failed (NOT retryable: item_count must be positive), giving up immediately
Caller correctly sees a validation error, not silently retried 3 times

Catching ValidationError separately and re-raising immediately means a request that’s wrong in a way retrying can never fix fails fast on the first attempt, instead of wasting 2 more attempts and their backoff delays on a certain failure.

A flaky call, backed off and retried until it succeeds

Practice

Frequently Asked Questions