Retry with exponential backoff and jitter

Q: Full jitter, equal jitter, or no jitter?

With no jitter, clients that fail together retry together. Full jitter (random(0, cap)) spreads retries the most and is the usual default. Equal jitter (cap/2 + random(0, cap/2)) guarantees a minimum wait, which helps if an immediate retry is known to be pointless. Either way, keep the cap: backoff without a ceiling grows past any useful wait within a few failures.

Q: How would this look in Go, and in production?

In Go the helper takes a context.Context and a func(ctx) error, sleeps with select { case <-time.After(d): case <-ctx.Done(): return ctx.Err() } so a cancelled request stops retrying, and uses errors.As to classify retryable errors. In production also cap total time, not just attempts, honour a server's Retry-After header when present, and add a retry budget, for example at most 10% extra traffic from retries, so a fleet of clients cannot multiply load during an outage.

Q: What should count as retryable?

Timeouts, connection resets, HTTP 429 and 503, and throttling errors from cloud APIs. Not 400, 401, 403 or 404: repeating the same request will fail the same way. Be careful with non-idempotent operations such as "create order": a timeout may mean the first attempt succeeded, so retry only with an idempotency key.

easyRate limiting and resilience Must-do

Write a reusable retry helper for calls that fail transiently: a cloud API returning a throttling error, a DNS lookup timing out, a database failover. Retrying immediately in a tight loop makes an overloaded service worse, so wait longer after each failure and add randomness so thousands of clients do not all retry at the same instant.

API (Java: Predicate<Exception>, DoubleConsumer sleep, DoubleSupplier rand, <T> T call(Callable<T> fn) throws Exception)

◈ DIAGRAM

Retrier(max_attempts: int, base_delay: float, max_delay: float,
        is_retryable=lambda exc: True, sleep=time.sleep, rand=random.random)
call(fn) -> whatever fn returns
delay(retry_number: int) -> float

Rules

call runs fn() up to max_attempts times in total, the first try included. It returns the first successful result.
If fn raises an exception for which is_retryable(exc) is false, re-raise it immediately. A 400 Bad Request will not fix itself.
After the last allowed attempt fails, re-raise that last exception unchanged. Do not wrap it or swallow it.
Before retry number n (0 for the first retry), sleep delay(n), defined with full jitter: rand() * min(max_delay, base_delay * 2**n).
sleep and rand are injected so tests run instantly and deterministically.
max_attempts < 1 raises ValueError.

The examples use base_delay=1, rand always returning 0.5, is_retryable true only for TransientError, and an op that raises the listed errors on its first calls, then returns "ok". Output lines: the result (or the exception raised), the list of sleeps, and how many times op ran.

Examples

Example 1

Input: Retrier(max_attempts=5, base_delay=1, max_delay=8) op fails with: TransientError("timeout"), TransientError("timeout")

Output: ok [0.5, 1.0] 3

Explanation: Caps are 1 and 2 seconds for the first two retries; full jitter with rand = 0.5 halves them.

Example 2

Input: Retrier(max_attempts=4, base_delay=1, max_delay=3) op always fails with: TransientError("timeout")

Output: raised TransientError: timeout [0.5, 1.0, 1.5] 4

Explanation: The third cap would be 4 seconds but max_delay limits it to 3. Four attempts means three sleeps, then the last error is re-raised.

Example 3

Input: Retrier(max_attempts=5, base_delay=1, max_delay=8) op fails with: PermanentError("bad request")

Output: raised PermanentError: bad request [] 1

Explanation: Non-retryable errors are raised on the first attempt with no sleep.

Hints

Approach

Optimal

A single loop over attempts 1..max_attempts:

Call fn(). If it returns, return that value.
On an exception, re-raise it if it is not retryable or if this was the final attempt. A bare raise in Python (or throw e in Java) keeps the original exception and stack trace, which is what the caller's error handling and logs expect.
Otherwise sleep delay(attempt - 1) and loop.

delay(n) doubles the cap each retry (exponential backoff), clamps it at max_delay so a long outage does not produce hour-long waits, and multiplies by a random number in [0, 1) (full jitter). Without jitter, every client that failed at the same moment retries at the same moment, and the recovering service receives synchronised waves of traffic.

Injecting sleep and rand is what makes the class testable. The test replaces them with list.append and a constant.

ComplexityTime O(max_attempts) callsSpace O(1)

Python

import random
import time
 
 
class Retrier:
    def __init__(self, max_attempts, base_delay, max_delay,
                 is_retryable=lambda exc: True, sleep=time.sleep, rand=random.random):
        if max_attempts < 1:
            raise ValueError("max_attempts must be at least 1")
        self.max_attempts = max_attempts
        self.base_delay = base_delay
        self.max_delay = max_delay
        self.is_retryable = is_retryable
        self.sleep = sleep
        self.rand = rand
 
    def delay(self, retry_number):
        """Full jitter: uniform in [0, min(max_delay, base * 2^n))."""
        cap = min(self.max_delay, self.base_delay * 2 ** retry_number)
        return self.rand() * cap
 
    def call(self, fn):
        for attempt in range(1, self.max_attempts + 1):
            try:
                return fn()
            except Exception as exc:
                if not self.is_retryable(exc) or attempt == self.max_attempts:
                    raise                       # original exception, original traceback
                self.sleep(self.delay(attempt - 1))

Follow-up questions

Add an overall deadline: stop retrying once deadline seconds have passed, even if attempts remain.
Combine this with a circuit breaker. Which one wraps the other?
Log each retry with the attempt number and delay without making the helper depend on a logging library.

Frequently asked questions

With no jitter, clients that fail together retry together. Full jitter (random(0, cap)) spreads retries the most and is the usual default. Equal jitter (cap/2 + random(0, cap/2)) guarantees a minimum wait, which helps if an immediate retry is known to be pointless. Either way, keep the cap: backoff without a ceiling grows past any useful wait within a few failures.

In Go the helper takes a context.Context and a func(ctx) error, sleeps with select { case <-time.After(d): case <-ctx.Done(): return ctx.Err() } so a cancelled request stops retrying, and uses errors.As to classify retryable errors. In production also cap total time, not just attempts, honour a server's Retry-After header when present, and add a retry budget, for example at most 10% extra traffic from retries, so a fleet of clients cannot multiply load during an outage.

Timeouts, connection resets, HTTP 429 and 503, and throttling errors from cloud APIs. Not 400, 401, 403 or 404: repeating the same request will fail the same way. Be careful with non-idempotent operations such as "create order": a timeout may mean the first attempt succeeded, so retry only with an idempotency key.

Problem statement