What Is the Circuit Breaker Pattern? A Deep Dive into Closed/Open/Half-Open via Simulation

1. Why Cascading Failures Happen

In a microservices architecture, handling a single request often means calling through a chain of dependencies — service A calls service B, which calls service C. One of the nastiest failure modes in this kind of setup is cascading failure. When one service (say, C) becomes overloaded or starts timing out, the threads or connections in the service calling it (B) get blocked for a long time waiting for C’s response (or its timeout). Once B’s thread pool or connection pool is exhausted, even requests that have nothing to do with C can no longer be served, and A — which calls B — gets dragged down in turn. A single point of failure propagates through the entire call hierarchy; that’s the basic structure of a cascading failure.

This mechanism can be quantified using Little’s Law, \(L = \lambda W\) , covered in https://yuhi-sa.github.io/en/posts/20260716_db_pool_queueing/1/. Suppose a call to some dependency normally responds in an average of \(W=20\text{ms}\) , at an arrival rate of \(\lambda=50\text{ req/s}\) . The average number of threads (or connections) blocked at any moment just for this one call is about

But once the dependency fails and calls sit there until they hit the timeout of \(W=2000\text{ms}\) instead of returning promptly, the number of simultaneously blocked threads jumps 100-fold:

If the caller’s thread pool only has, say, 200 threads, this single dependency alone eats up half the pool, dragging down the throughput of every other endpoint that shares it. The essence of cascading failure is this paradox: the timeout — the very safety mechanism meant to bound how long a call can take — ends up tying up resources indefinitely for as long as the dependency stays fully down.

Michael Nygard, in his book Release It!, proposed the circuit breaker pattern as a defense against exactly this kind of failure propagation¹. The name is a direct analogy to an electrical circuit breaker: once a downstream failure is detected, stop sending any more current (requests) to that dependency, protecting both the caller’s own resources and the failing dependency itself. Martin Fowler later summarized the pattern in a bliki post, capturing the essential idea — wrap the protected call in something that keeps monitoring for failures, and once failures cross a threshold, immediately fail subsequent calls without even attempting the real one — along with the standard formalization into three states: Closed, Open, and Half-Open².

2. The Circuit Breaker’s Three States

A circuit breaker can be modeled as a state machine with the following three states.

• Closed: Normal operation. All requests pass through to the dependency. Meanwhile, the outcome of each recent call (success/failure) is recorded in a sliding window. Once the failure rate within that window exceeds a threshold, the breaker transitions to Open.
• Open: The tripped state. No requests are sent to the dependency at all; every call fails immediately (without waiting for a timeout). After a certain wait period elapses, the breaker transitions to Half-Open.
• Half-Open: A trial state. Only a small number of requests are allowed through to the dependency, as a test. If the failure rate of these trial calls is below the threshold, the dependency is assumed to have recovered and the breaker returns to Closed; if it’s still at or above the threshold, the dependency is assumed to still be down and the breaker returns to Open.

The transitions among these three states are illustrated below.

The key point is that the instant the breaker becomes Open, calls to the dependency drop to zero. Instead of “waiting until the timeout resolves things,” the caller can use the information it accumulated while Closed to conclude immediately that calling would be pointless, and give up right away. This resolves the thread/connection pinning described in Section 1, and also frees the dependency from a flood of pointless requests so it can focus on recovering. This “helps the dependency recover” effect is measured quantitatively in the experiment in Section 4.

3. What the Key Parameters Mean

Resilience4j, a widely used Java library implementing this pattern, documents concrete names and meanings for the parameters that drive this state machine³. The table below summarizes the key parameters from that documentation.

Both slidingWindowSize and minimumNumberOfCalls default to a fairly large 100, which is a deliberate balance so the breaker doesn’t overreact to ordinary noise (a handful of incidental failures during normal operation). Conversely, the smaller these values are, the faster the breaker reacts — but the more prone it becomes to overreacting to a small number of coincidental failures. The same trade-off applies to waitDurationInOpenState: too short, and the breaker keeps darting into Half-Open and hammering the still-broken dependency again before it has recovered; too long, and the breaker stays tripped well after the dependency has actually recovered.

The fact that automaticTransitionFromOpenToHalfOpenEnabled defaults to false is easy to overlook. It means the breaker does not switch to Half-Open the instant the Open wait duration elapses via some background timer; instead, it checks — lazily, on the next incoming request — whether the wait duration has elapsed, and if so, treats that very request as the first Half-Open trial call. This keeps the implementation simple (no background timer needed) and avoids ever making a pointless trial call when no request is actually coming in. The hand-rolled implementation in Section 4 reproduces this exact behavior.

4. Experiment: Comparing With and Without a Breaker Under Fault Injection

To back up this theory with numbers, I built a discrete-event simulation (DES) using nothing but Python’s standard heapq, and measured exactly what changes with and without a circuit breaker.

4.1 Model Setup

• Clients → dependency: Clients call the dependency via a Poisson process with arrival rate \(\lambda=50\text{ req/s}\) .
• Dependency behavior: Under normal conditions it responds successfully with an average latency of 20ms (exponentially distributed). But between \(T_1=15\text{s}\) and \(T_2=35\text{s}\) , it enters a “fault” state: every call actually sent to it during this window fails after a 2000ms timeout.
• Scenarios compared: (1) no breaker (every call goes directly to the dependency), and (2) with breaker (calls pass through a hand-rolled state machine using the parameter names from Section 3).
• Breaker configuration: failureRateThreshold=50%, slidingWindowType=COUNT_BASED, slidingWindowSize=20, minimumNumberOfCalls=20, waitDurationInOpenState=5 seconds, permittedNumberOfCallsInHalfOpenState=5. The defaults (a 100-call window and a 60-second wait) are conservative production values; they’re scaled down here so the full arc of state transitions is visible within a 60-second simulation.
• Random seed: fixed at 42.

The hand-rolled circuit breaker state machine looks like this.

class CircuitBreaker:
    """Minimal circuit breaker modeled on Resilience4j's CircuitBreaker (CLOSED/OPEN/HALF_OPEN)."""

    def __init__(self, failure_rate_threshold=0.5, sliding_window_size=20,
                 minimum_number_of_calls=20, wait_duration_in_open_state=5.0,
                 permitted_number_of_calls_in_half_open_state=5):
        self.failure_rate_threshold = failure_rate_threshold
        self.sliding_window_size = sliding_window_size
        self.minimum_number_of_calls = minimum_number_of_calls
        self.wait_duration_in_open_state = wait_duration_in_open_state
        self.permitted_number_of_calls_in_half_open_state = permitted_number_of_calls_in_half_open_state

        self.state = "CLOSED"
        self.closed_window = deque(maxlen=sliding_window_size)  # 1=success, 0=failure
        self.half_open_results = []
        self.half_open_admitted = 0
        self.opened_at = None

    def try_acquire_permission(self, now):
        """Decide whether to permit a call. Returns (True, tag) or (False, None)."""
        if self.state == "OPEN":
            # Equivalent to automaticTransitionFromOpenToHalfOpenEnabled=false:
            # lazily check on the next call attempt whether the Open wait duration has elapsed.
            if now - self.opened_at >= self.wait_duration_in_open_state:
                self._transition_to_half_open(now)
            else:
                return False, None

        if self.state == "HALF_OPEN":
            if self.half_open_admitted < self.permitted_number_of_calls_in_half_open_state:
                self.half_open_admitted += 1
                return True, "HALF_OPEN"
            return False, None

        return True, "CLOSED"  # CLOSED

    def on_result(self, now, success, tag):
        if tag == "CLOSED":
            if self.state != "CLOSED":
                return
            self.closed_window.append(1 if success else 0)
            if len(self.closed_window) >= self.minimum_number_of_calls:
                failure_rate = 1 - sum(self.closed_window) / len(self.closed_window)
                if failure_rate >= self.failure_rate_threshold:
                    self._transition_to_open(now)
        elif tag == "HALF_OPEN":
            if self.state != "HALF_OPEN":
                return
            self.half_open_results.append(1 if success else 0)
            if len(self.half_open_results) >= self.permitted_number_of_calls_in_half_open_state:
                failure_rate = 1 - sum(self.half_open_results) / len(self.half_open_results)
                if failure_rate >= self.failure_rate_threshold:
                    self._transition_to_open(now)
                else:
                    self._transition_to_closed(now)

    def _transition_to_open(self, now):
        self.state = "OPEN"
        self.opened_at = now
        self.half_open_results = []
        self.half_open_admitted = 0

    def _transition_to_half_open(self, now):
        self.state = "HALF_OPEN"
        self.half_open_results = []
        self.half_open_admitted = 0

    def _transition_to_closed(self, now):
        self.state = "CLOSED"
        self.closed_window.clear()

On each arrival event, the breaker’s try_acquire_permission decides whether to allow the call; only if allowed does the simulation schedule a completion event for the call to the dependency. When the call completes, its outcome (success/failure) is recorded via on_result, which trips a state transition if the threshold is exceeded. Each admitted call carries a tag recording the state it was admitted under (CLOSED or HALF_OPEN); if the state has already changed by the time a long (timed-out) call completes, its result is simply dropped instead of being applied to the now-stale window, keeping the state transitions consistent with timing.

4.2 Measured Results: Latency Time Series

Aggregated over the fault window (15–35 seconds):

The time series of mean and p99 latency, binned per second, is plotted below.

Without a breaker, both mean and p99 latency stay pinned at 2000ms (the timeout value itself) for the entire fault window — unsurprising, since every request without exception waits for the dependency’s response (or its timeout). This is the direct cause of the thread pinning described in Section 1.

With a breaker, about 2.17 seconds after the fault begins (t=17.17s) the breaker trips to Open, and from that point mean latency collapses to about 0.5ms. This is because calls that hit Open fail immediately, before ever reaching the dependency. What’s interesting, though, is that p99 latency occasionally spikes back up toward 2000ms even during the fault. This happens at the moments the breaker transitions to Half-Open (t≈22.17s, 29.22s): a small number of trial calls actually get sent to the dependency, and since the fault is still ongoing, those trial calls themselves take the full 2000ms timeout. In other words, the measurements reveal a non-obvious fact: the breaker dramatically improves mean latency, but the trial calls made during Half-Open unavoidably incur the full timeout latency themselves.

4.3 Measured Results: Requests Reaching the Dependency

Another critical metric is how many requests actually reach the dependency. If the breaker is doing its job, load on the dependency during the fault should drop sharply.

Aggregating the requests that reached the dependency during the fault window: 987 (100% reach rate) without a breaker, versus just 102 (10.33% reach rate) with one. That’s 885 requests — roughly 90% of the total — blocked before ever reaching the dependency. Of those 102, about 20 were early calls that were already in flight before the breaker tripped to Open, plus two rounds of Half-Open trials (5 calls each, with a few incidental overlaps from other requests). From the perspective of a dependency that’s still failing, this means going from “being hammered with 40–60 requests per second continuously” to “almost zero” — a concrete, quantitative confirmation of the claim that the breaker helps the dependency recover.

4.4 Measured Results: Automatic Recovery via Half-Open

Finally, here’s the breaker’s own state transitions visualized as a timeline.

The measured sequence of events is as follows.

t= 0.00s  CLOSED (simulation start)
t=17.17s  CLOSED -> OPEN       (2.17s after the fault began; the 20-call window filled with failures)
t=22.17s  OPEN   -> HALF_OPEN  (waitDurationInOpenState=5s elapsed)
t=24.22s  HALF_OPEN -> OPEN    (5 trial calls, all failed since the fault was still ongoing, back to Open)
t=29.22s  OPEN   -> HALF_OPEN  (another 5 seconds elapsed)
t=31.23s  HALF_OPEN -> OPEN    (2nd trial also failed entirely)
t=36.25s  OPEN   -> HALF_OPEN  (3rd attempt; by this point the dependency had already recovered at t=35s)
t=36.38s  HALF_OPEN -> CLOSED  (trial calls succeeded, returning to normal)

The dependency’s fault actually ended at \(T_2=35\text{s}\) , but the breaker didn’t actually return to Closed until \(t=36.38\text{s}\) — a lag of about 1.38 seconds. This wasn’t because the breaker deliberately timed its Half-Open attempt to coincide with the fault ending; it just happened that the next 5-second trial cycle (following the one at \(t=31.23\text{s}\) ) landed at \(t=36.25\text{s}\) . Even so, the breaker automatically detected and recovered the moment the first trial call after the dependency’s actual recovery went out. The practical value of this pattern lies precisely in this: recovery happens automatically, without any human intervention and without actively polling the dependency’s health — just by riding along on the normal flow of trial calls.

5. Relationship to Retries and Timeouts

A circuit breaker doesn’t work in isolation — it only delivers its full value alongside timeouts and retries. Comparing it with the exponential backoff and jitter covered in https://yuhi-sa.github.io/en/posts/20260720_exponential_backoff/1/, the two patterns can be understood as a division of labor across different failure timescales.

• Retry + backoff: targets short-lived, transient failures — a network blip or a momentary overload — that resolve naturally within hundreds of milliseconds to a few seconds. It fits situations where, from a single request’s point of view, “trying again might well succeed” is a reasonable assumption.
• Circuit breaker: targets sustained failures — a dependency that stays clearly down for seconds, tens of seconds, or longer. In this regime, retrying an individual request is pointless, and the retries themselves become extra load on the dependency; the breaker stops the entire call path, retries included. The AWS Builders’ Library, alongside its three pillars of timeouts, retries, and backoff, also describes a client-side token bucket that caps the volume of retries themselves — a mechanism it positions as a circuit breaker in the broad sense⁴.

As the exponential backoff article showed, backoff without jitter never breaks synchronization — the “size of the colliding wave” of clients stays the same even as large numbers of them fail simultaneously. A circuit breaker is a different layer of defense: it stops that colliding wave from ever reaching the dependency in the first place. In practice, the standard design layers all three together for a single request: set a timeout → if it fails, retry a few times with backoff and jitter → and if the breaker is Open, defer to it and give up immediately rather than retrying at all.

6. Implementation: Resilience4j and Python

6.1 Resilience4j (Java)

When used with Spring Boot, it’s common to configure each instance in application.yml.

resilience4j.circuitbreaker:
  instances:
    paymentService:
      failureRateThreshold: 50
      slidingWindowType: COUNT_BASED
      slidingWindowSize: 100
      minimumNumberOfCalls: 100
      waitDurationInOpenState: 60000
      permittedNumberOfCallsInHalfOpenState: 10
      automaticTransitionFromOpenToHalfOpenEnabled: false

Configuring it directly from code looks like this.

CircuitBreakerConfig config = CircuitBreakerConfig.custom()
    .failureRateThreshold(50)
    .slidingWindowType(SlidingWindowType.COUNT_BASED)
    .slidingWindowSize(100)
    .minimumNumberOfCalls(100)
    .waitDurationInOpenState(Duration.ofSeconds(60))
    .permittedNumberOfCallsInHalfOpenState(10)
    .build();

CircuitBreaker circuitBreaker = CircuitBreaker.of("paymentService", config);

The parameter names map directly onto the ones used in Sections 3 and 4 of this article. In production, it’s safest to start from the defaults (a 100-call window, a 60-second wait) and tune from there based on actual traffic volume and your SLOs.

6.2 Python (Hand-Rolled Class)

The Python ecosystem has existing libraries like pybreaker, but the CircuitBreaker class built in Section 4 can also be used directly as a wrapper around a function call.

breaker = CircuitBreaker(
    failure_rate_threshold=0.5,
    sliding_window_size=20,
    minimum_number_of_calls=20,
    wait_duration_in_open_state=5.0,
    permitted_number_of_calls_in_half_open_state=5,
)

def call_with_breaker(now, dependency_call):
    allowed, tag = breaker.try_acquire_permission(now)
    if not allowed:
        raise CircuitOpenError("breaker is open, failing fast")
    try:
        result = dependency_call()
        breaker.on_result(now, success=True, tag=tag)
        return result
    except Exception:
        breaker.on_result(now, success=False, tag=tag)
        raise

In production, pass time.monotonic() instead of a bare now, and pass your actual HTTP client or DB driver call as dependency_call, and this drops directly into application code as-is.

7. Operational Notes

Based on the measurements and implementation above, here are some operational considerations.

1. Set thresholds based on your production error-rate variance. Setting failureRateThreshold too low (say, 20%) makes the breaker prone to tripping open from ordinary error-rate noise that occurs even during normal operation. Setting it too high (say, 80%) means genuine failures take a long time to detect. Start near the 50% default and tune while watching your actual error-rate histogram.
2. Design fallbacks ahead of time. When the breaker is Open, the caller needs a plan for what to do besides “return an error immediately.” Returning stale cached data, a default value, or gracefully degrading a feature are all fallback paths that work without the dependency — without one prepared in advance, a circuit breaker ends up being nothing more than “failing faster.”
3. Monitor the breaker’s own state changes. Turn the Open/Half-Open/Closed transitions themselves into metrics and set up alerting on them. If a breaker is flapping frequently between Open and Half-Open, either the dependency’s outage is dragging on, or waitDurationInOpenState is set too short. Reproducing a state timeline like Figure 4 from Section 4 on a production dashboard makes it much easier to reconstruct what happened between an outage and its recovery after the fact.
4. Scope breakers appropriately. Rather than wrapping one giant dependency (say, “the entire database”) in a single breaker, splitting breakers along lines like endpoint or table — matched to the actual blast radius of a failure — avoids taking down unrelated functionality along with the thing that actually broke.

8. Conclusion

A circuit breaker is a mechanism for protecting a client’s own resources against a dependency’s sustained bad weather, while also helping that dependency recover. This simulation confirmed the following points quantitatively:

• The breaker tripped to Open about 2.17 seconds after the fault began, and from that point client-observed mean latency improved dramatically, from 2000ms down to about 0.5ms.
• Of the requests during the fault window, 987 (100%) reached the dependency with no breaker, versus only 102 (10.33%) with one — roughly 90% of the load was blocked.
• p99 latency briefly returned to around 2000ms at each Half-Open trial — a non-obvious fact showing that while the breaker dramatically improves mean latency, the trial calls themselves can’t avoid incurring the full timeout.
• The breaker automatically returned to Closed via Half-Open just 1.38 seconds after the dependency actually recovered (t=35s), with no human intervention.

The idea of “just trip and stop calling when things fail” sounds simple on its own, but its real effect only becomes concrete once you actually build and run something like this. Retries, backoff, timeouts, and circuit breakers each handle failures at a different timescale — no single one of them is a substitute for the others, and designing them together is what actually gives a distributed system its resilience.

FAQ

Q1. How does this differ from retries, and when should I use which? A. Retry with backoff is designed for short-lived transient failures that resolve within hundreds of milliseconds to a few seconds, while a circuit breaker is designed for a dependency that stays down persistently. In practice, use both together: retry an individual call a few times with backoff, but if the breaker is Open, skip retrying entirely and fail immediately.

Q2. How many trial calls should Half-Open allow? A. Resilience4j’s default is 10. Too few, and the decision becomes noisy based on a small, possibly unrepresentative sample; too many, and you risk hammering a dependency that hasn’t actually fully recovered yet. This article’s experiment used 5, and actually experienced two false negatives (trials that reverted to Open because the dependency was still down) before succeeding on the third — a reasonable range to tune based on your traffic volume, as long as it isn’t too small.

Q3. What if there’s no fallback available? A. Even without a fallback, a circuit breaker still delivers value: it stops wasted load on the dependency and stops tying up the caller’s own resources, simply by failing immediately. That said, the user experience degrades to a plain error, so it’s worth considering degraded-mode designs — returning a cached previous value, or disabling part of a feature — for your highest-priority dependencies first.

Q4. Is Hystrix still usable? A. Netflix’s official repository explicitly states, “Hystrix is no longer in active development, and is currently in maintenance mode.”⁵ Netflix itself recommends actively maintained libraries like resilience4j for new projects; whatever the status of existing Hystrix usage, there’s little reason to adopt it fresh today.

Related Articles

• Exponential Backoff and Jitter Explained: The Retry Design Playbook, Verified by Simulation - Covers retry and backoff as a defense against short-lived failures, complementing this article’s circuit breaker approach to sustained failures.
• Modeling Database Connection Pool Sizing with Queueing Theory (M/M/c) - Derives Little’s Law (used in Section 1 of this article) in detail and quantitatively verifies connection pool exhaustion.

For a broader reading list, see 10 Must-Read Technical Books for Engineers .

References