How is a bulkhead different from a circuit breaker?

A circuit breaker stops calling a dependency that's already failing outright. A bulkhead limits the blast radius when a dependency is merely slow (not failing) by capping how many resources — threads, connections — it can occupy, so it can't starve calls to unrelated dependencies even while it's still technically 'up'. They solve different failure modes and are commonly used together: a bulkhead limits how much damage a slow dependency can do, and a circuit breaker stops trying once it's clearly dead.

Why does the shared pool in this lesson get stuck on 3 seconds instead of some other number?

With 2 workers and 3 slow calls queued ahead of the fast one, the fast call has to wait for 2 full rounds of the slow service's 1.5-second response time before a worker is free to run it — 2 x 1.5s = 3s. The exact number is a direct consequence of pool size, queue position, and the slow dependency's real latency, not a fixed constant.

Doesn't giving every dependency its own pool waste resources compared to one shared pool?

It does use more total threads/connections at rest, which is the real cost of the isolation. The tradeoff is deliberate: a shared pool is more resource-efficient until the moment one dependency misbehaves, at which point that efficiency is exactly what lets it take everything else down with it.

Does making the isolated pool bigger fix the starvation instead of separating dependencies?

It can, for a specific workload — a shared pool sized well above your peak concurrent load might never queue at all. But dependency isolation and pool sizing solve different problems: sizing protects against a burst of legitimate load; isolation protects against one dependency's failure mode (going slow) consuming resources meant for others, no matter how the pool is sized.

Bulkhead: Isolating Failure So One Slow Dependency Doesn't Sink Everything

Giving every downstream dependency its own bounded pool of resources means one dependency misbehaving can’t consume the resources another dependency’s calls need to run at all.

A shared pool lets a slow dependency starve an unrelated fast one

import http.server
import threading
import time
import urllib.request
from concurrent.futures import ThreadPoolExecutor

class SlowHandler(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        time.sleep(1.5)
        self.send_response(200)
        self.end_headers()
        self.wfile.write(b"slow ok")

    def log_message(self, *args):
        pass

class FastHandler(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.end_headers()
        self.wfile.write(b"fast ok")

    def log_message(self, *args):
        pass

slow_server = http.server.HTTPServer(("127.0.0.1", 8026), SlowHandler)
fast_server = http.server.HTTPServer(("127.0.0.1", 8027), FastHandler)
threading.Thread(target=slow_server.serve_forever, daemon=True).start()
threading.Thread(target=fast_server.serve_forever, daemon=True).start()
time.sleep(0.2)

def call_slow():
    return urllib.request.urlopen("http://127.0.0.1:8026/x", timeout=10).read().decode()

def call_fast():
    return urllib.request.urlopen("http://127.0.0.1:8027/x", timeout=10).read().decode()

shared_pool = ThreadPoolExecutor(max_workers=2)

print("Submitting 3 slow requests, then 1 fast request, to a SHARED pool of 2 workers...")
slow_futures = [shared_pool.submit(call_slow) for _ in range(3)]

start = time.perf_counter()
fast_future = shared_pool.submit(call_fast)
result = fast_future.result()
elapsed = time.perf_counter() - start
print(f"Fast request finished in {elapsed:.2f}s (result: {result!r})")

for f in slow_futures:
    f.result()

shared_pool.shutdown()
slow_server.shutdown()
fast_server.shutdown()

$ python bulkhead_shared.py
Submitting 3 slow requests, then 1 fast request, to a SHARED pool of 2 workers...
Fast request finished in 3.09s (result: 'fast ok')

call_fast targets a completely healthy, instant-responding service, but it still takes 3 full seconds — because the pool only has 2 worker threads, both immediately claimed by the first 2 slow requests, and the 3rd slow request sits ahead of the fast one in the same shared queue. The fast call has to wait for 2 full rounds of the slow service’s 1.5-second latency before a worker is even free to run it, even though the fast call itself takes almost no time at all.

Giving each dependency its own pool stops the bleeding

import http.server
import threading
import time
import urllib.request
from concurrent.futures import ThreadPoolExecutor

class SlowHandler(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        time.sleep(1.5)
        self.send_response(200)
        self.end_headers()
        self.wfile.write(b"slow ok")

    def log_message(self, *args):
        pass

class FastHandler(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.end_headers()
        self.wfile.write(b"fast ok")

    def log_message(self, *args):
        pass

slow_server = http.server.HTTPServer(("127.0.0.1", 8026), SlowHandler)
fast_server = http.server.HTTPServer(("127.0.0.1", 8027), FastHandler)
threading.Thread(target=slow_server.serve_forever, daemon=True).start()
threading.Thread(target=fast_server.serve_forever, daemon=True).start()
time.sleep(0.2)

def call_slow():
    return urllib.request.urlopen("http://127.0.0.1:8026/x", timeout=10).read().decode()

def call_fast():
    return urllib.request.urlopen("http://127.0.0.1:8027/x", timeout=10).read().decode()

slow_pool = ThreadPoolExecutor(max_workers=2)
fast_pool = ThreadPoolExecutor(max_workers=2)

print("Submitting 3 slow requests to their OWN pool, then 1 fast request to a SEPARATE pool...")
slow_futures = [slow_pool.submit(call_slow) for _ in range(3)]

start = time.perf_counter()
fast_future = fast_pool.submit(call_fast)
result = fast_future.result()
elapsed = time.perf_counter() - start
print(f"Fast request finished in {elapsed:.2f}s (result: {result!r})")

for f in slow_futures:
    f.result()

slow_pool.shutdown()
fast_pool.shutdown()
slow_server.shutdown()
fast_server.shutdown()

$ python bulkhead_isolated.py
Submitting 3 slow requests to their OWN pool, then 1 fast request to a SEPARATE pool...
Fast request finished in 0.10s (result: 'fast ok')

Same slow dependency, same 3 backed-up slow calls, same fast dependency — the only change is that call_fast now runs through its own fast_pool instead of sharing shared_pool with the slow calls. It finishes in 0.10s regardless of how backed up the slow dependency’s pool gets, because the two pools have nothing to contend over. This is exactly what a bulkhead on a ship does: a compartment flooding can’t sink the whole vessel if the bulkheads between compartments hold.

Practice

1. Instead of isolating the fast dependency, just make the shared pool bigger (4 workers instead of 2) and confirm starvation disappears without any isolation at all.

import http.server
import threading
import time
import urllib.request
from concurrent.futures import ThreadPoolExecutor

class SlowHandler(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        time.sleep(1.5)
        self.send_response(200)
        self.end_headers()
        self.wfile.write(b"slow ok")

    def log_message(self, *args):
        pass

class FastHandler(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.end_headers()
        self.wfile.write(b"fast ok")

    def log_message(self, *args):
        pass

slow_server = http.server.HTTPServer(("127.0.0.1", 8028), SlowHandler)
fast_server = http.server.HTTPServer(("127.0.0.1", 8029), FastHandler)
threading.Thread(target=slow_server.serve_forever, daemon=True).start()
threading.Thread(target=fast_server.serve_forever, daemon=True).start()
time.sleep(0.2)

def call_slow():
    return urllib.request.urlopen("http://127.0.0.1:8028/x", timeout=10).read().decode()

def call_fast():
    return urllib.request.urlopen("http://127.0.0.1:8029/x", timeout=10).read().decode()

shared_pool = ThreadPoolExecutor(max_workers=4)
slow_futures = [shared_pool.submit(call_slow) for _ in range(3)]

start = time.perf_counter()
fast_future = shared_pool.submit(call_fast)
result = fast_future.result()
elapsed = time.perf_counter() - start
print(f"Fast request finished in {elapsed:.2f}s (result: {result!r})")

for f in slow_futures:
    f.result()
shared_pool.shutdown()
slow_server.shutdown()
fast_server.shutdown()

$ python bigger_shared_pool.py
Fast request finished in 0.15s (result: 'fast ok')

With 4 workers and only 3 slow calls, every slow call gets its own worker and the fast call still gets one immediately — nothing ever queues. This is a real fix for this specific workload, but it only works because the pool happens to be sized above the current slow-call volume; if a 4th or 5th slow call arrived, the fast call would start queuing again. Isolation protects against that regardless of how many slow calls show up.

2. Shrink the isolated fast pool down to a single worker (max_workers=1) and confirm it's still fast — isolation is about separation, not size.

import http.server
import threading
import time
import urllib.request
from concurrent.futures import ThreadPoolExecutor

class SlowHandler(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        time.sleep(1.5)
        self.send_response(200)
        self.end_headers()
        self.wfile.write(b"slow ok")

    def log_message(self, *args):
        pass

class FastHandler(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.end_headers()
        self.wfile.write(b"fast ok")

    def log_message(self, *args):
        pass

slow_server = http.server.HTTPServer(("127.0.0.1", 8030), SlowHandler)
fast_server = http.server.HTTPServer(("127.0.0.1", 8031), FastHandler)
threading.Thread(target=slow_server.serve_forever, daemon=True).start()
threading.Thread(target=fast_server.serve_forever, daemon=True).start()
time.sleep(0.2)

def call_slow():
    return urllib.request.urlopen("http://127.0.0.1:8030/x", timeout=10).read().decode()

def call_fast():
    return urllib.request.urlopen("http://127.0.0.1:8031/x", timeout=10).read().decode()

slow_pool = ThreadPoolExecutor(max_workers=2)
fast_pool = ThreadPoolExecutor(max_workers=1)

slow_futures = [slow_pool.submit(call_slow) for _ in range(3)]
start = time.perf_counter()
fast_future = fast_pool.submit(call_fast)
result = fast_future.result()
elapsed = time.perf_counter() - start
print(f"Fast request finished in {elapsed:.2f}s (result: {result!r})")

for f in slow_futures:
    f.result()
slow_pool.shutdown()
fast_pool.shutdown()
slow_server.shutdown()
fast_server.shutdown()

$ python small_isolated_pool.py
Fast request finished in 0.10s (result: 'fast ok')

Even a pool of exactly 1 worker keeps the fast call fast, because nothing else is competing for that single worker — slow_pool and fast_pool are entirely separate resources. Isolation doesn’t require generous sizing; it requires that unrelated dependencies never share the same pool.

3. Send 3 requests to the same dependency through its own single-worker pool, and confirm a bulkhead doesn't remove the need for adequate sizing within one dependency.

import http.server
import threading
import time
import urllib.request
from concurrent.futures import ThreadPoolExecutor

class MediumHandler(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        time.sleep(0.5)
        self.send_response(200)
        self.end_headers()
        self.wfile.write(b"medium ok")

    def log_message(self, *args):
        pass

server = http.server.HTTPServer(("127.0.0.1", 8033), MediumHandler)
threading.Thread(target=server.serve_forever, daemon=True).start()
time.sleep(0.2)

def call_medium():
    return urllib.request.urlopen("http://127.0.0.1:8033/x", timeout=10).read().decode()

pool = ThreadPoolExecutor(max_workers=1)
start = time.perf_counter()
futures = [pool.submit(call_medium) for _ in range(3)]
for i, f in enumerate(futures):
    f.result()
    print(f"call {i} finished at t={time.perf_counter() - start:.2f}s")
pool.shutdown()
server.shutdown()

$ python within_dependency_queueing.py
call 0 finished at t=0.56s
call 1 finished at t=1.08s
call 2 finished at t=1.60s

All 3 calls target the exact same dependency through its own dedicated pool — there’s no cross-dependency starvation here to isolate against, and the calls still queue behind each other roughly 0.5 seconds apart, one per worker slot. A bulkhead solves contention between dependencies; it does nothing about a single dependency simply being called faster than its own pool can keep up with, which is a capacity/sizing problem, not an isolation problem.

A shared pool lets a slow dependency starve an unrelated fast one

Giving each dependency its own pool stops the bleeding

Practice

Frequently Asked Questions