DevOps NetworkDevOpsNetwork
Join Free
Build it yourself

Circuit breaker

mediumRate limiting and resilience Must-do

Problem statement

Implement a circuit breaker that wraps calls to a dependency. When the dependency keeps failing, stop calling it for a while and fail fast instead. Callers get an immediate error rather than waiting on timeouts, and the struggling service gets room to recover.

API (Java: <T> T call(Callable<T> fn) throws Exception, state() returns an enum, CircuitOpenException)

◈ DIAGRAM
class CircuitOpenError(Exception)
CircuitBreaker(failure_threshold: int, reset_timeout: float, clock=time.monotonic)
call(fn) -> fn's result # raises fn's exception, or CircuitOpenError
state() -> "closed" | "open" | "half_open"

States and transitions

  • closed: calls go through. Each exception from fn counts as a failure, and each success resets the count. When failure_threshold failures happen in a row, the breaker trips to open.
  • open: call raises CircuitOpenError without running fn. Once reset_timeout seconds have passed since the breaker opened, it becomes half-open. Compute this lazily in state() and call(), with no timer thread.
  • half_open: exactly one trial call is let through. If it succeeds, the breaker closes and the failure count is zero. If it fails, the breaker opens again and the timeout restarts from now. While the trial is running, other calls are rejected.

Other rules

  • fn's own exception is always re-raised unchanged. The breaker records it but never hides it.
  • failure_threshold < 1 or reset_timeout <= 0 raises ValueError.

In the examples ok returns "ok" and fail raises RuntimeError("boom"). Output lines show the result, error: boom when fn raised, or rejected when the breaker refused the call.

Examples

Example 1

Input: t=0 cb = CircuitBreaker(failure_threshold=2, reset_timeout=5) t=0 cb.call(fail) t=0 cb.state() t=1 cb.call(fail) t=1 cb.state() t=3 cb.call(ok) t=6 cb.state() t=6 cb.call(ok) t=6 cb.state()

Output: error: boom closed error: boom open rejected half_open ok closed

Explanation: Two failures in a row trip the breaker at t=1. At t=3 it is still cooling down, so ok is never called. At t=6, 5 s have passed, and the successful trial closes it.

Example 2

Input: t=0 cb = CircuitBreaker(failure_threshold=3, reset_timeout=5) t=0 cb.call(fail), fail, ok, fail, fail t=0 cb.state() t=0 cb.call(fail) t=0 cb.state() t=10 cb.call(fail) t=10 cb.state() t=12 cb.call(ok) t=15 cb.call(ok)

Output: error: boom error: boom ok error: boom error: boom closed error: boom open error: boom open rejected ok

Explanation: The success in the middle resets the count, so it takes three more failures to trip. The failed trial at t=10 reopens the breaker until t=15.

Hints

Approach

Optimal

Keep four fields: state, failures (consecutive, while closed), opened_at, and trial_running.

  1. state(): if open and now - opened_at >= reset_timeout, switch to half-open. Every other method reads the state through this, so the timeout is honoured without a background thread.
  2. call(fn): reject with CircuitOpenError if open, or if half-open with a trial already running. Otherwise note whether this is the trial and run fn.
  3. On an exception: if this was the trial, trip again, which resets opened_at to now. If closed, increment failures and trip at the threshold. Then re-raise the original exception.
  4. On success: set closed and failures = 0.
  5. A finally clears trial_running, so a trial that raises cannot leave the breaker stuck rejecting forever.

Every operation is O(1). The breaker adds only a few comparisons to each call.

ComplexityTime O(1) per callSpace O(1)
Python
import time
class CircuitOpenError(Exception):
pass
class CircuitBreaker:
CLOSED, OPEN, HALF_OPEN = "closed", "open", "half_open"
def __init__(self, failure_threshold, reset_timeout, clock=time.monotonic):
if failure_threshold < 1 or reset_timeout <= 0:
raise ValueError("failure_threshold must be >= 1 and reset_timeout > 0")
self.threshold = failure_threshold
self.reset_timeout = reset_timeout
self.clock = clock
self._state = self.CLOSED
self._failures = 0 # consecutive failures while closed
self._opened_at = 0.0
self._trial_running = False
def state(self):
if self._state == self.OPEN and self.clock() - self._opened_at >= self.reset_timeout:
self._state = self.HALF_OPEN # cool-down over: allow one trial call
return self._state
def _trip(self):
self._state = self.OPEN
self._opened_at = self.clock()
self._failures = 0
def call(self, fn):
st = self.state()
if st == self.OPEN or (st == self.HALF_OPEN and self._trial_running):
raise CircuitOpenError("circuit open")
self._trial_running = st == self.HALF_OPEN
try:
result = fn()
except Exception:
if st == self.HALF_OPEN:
self._trip() # trial failed: back to open, restart the timer
else:
self._failures += 1
if self._failures >= self.threshold:
self._trip()
raise
finally:
self._trial_running = False
self._state = self.CLOSED # any success closes the circuit
self._failures = 0
return result

Follow-up questions

  • Make it thread-safe. Why must checking trial_running and setting it happen under the same lock?
  • Only count some errors: timeouts and 5xx trip the breaker, 4xx do not.
  • Allow N trial calls in half-open instead of one, and close only if most succeed.

Frequently asked questions

Consecutive failures are simple and fine at low traffic. At high traffic, a dependency failing 40% of requests may never produce 5 failures in a row, yet it is clearly unhealthy. Production breakers usually trip on a failure rate over a sliding window, such as over 50% of the last 20 calls, with a minimum call count so two failures out of three calls at 3 a.m. do not trip it.

In Go it is a struct with a sync.Mutex around the state fields and a method like Execute(func() (T, error)) (T, error) (generics) or func() error, returning a sentinel ErrOpen that callers check with errors.Is. Libraries such as sony/gobreaker follow this shape and add hooks for state changes. In production, export the state as a metric and alert on it, keep one breaker per downstream host or endpoint rather than one global breaker, and pair it with a fallback such as a cached response.

Retries help with short, random failures. A breaker helps with sustained failures, where retries only add load. They work together: a retry loop inside the breaker should treat CircuitOpenError as non-retryable, so it stops as soon as the breaker opens.