DevOps NetworkDevOpsNetwork
Join Free
Practical infra coding

Retry an HTTP call with exponential backoff

mediumAPIs and HTTP Must-do

Problem statement

A deploy script calls an internal API that sometimes drops connections or returns 503 during a rollout. Write get_with_retry(client, path, ...) that calls client.get(path) and retries the failures that are worth retrying, waiting longer after each one.

Rules:

  • Retry on ConnectionError, TimeoutError, and HTTP status 429, 500, 502, 503, 504.
  • Do not retry any other 4xx. A 404 or 403 will not fix itself; raise HTTPError right away.
  • Any status below 400 is success: return the response.
  • Wait base * 2 ** (attempt - 1) seconds after a failed attempt (0.5, 1, 2, 4 ...), but never more than cap.
  • Do not sleep after the last attempt. After max_attempts failures raise RuntimeError naming the path, the number of attempts and the last reason.
  • Take sleep as a parameter, so tests can pass a fake and run instantly.

There is no network in the exercise. A FakeClient replays a scripted list of outcomes, where each entry is either an exception to raise or a Response(status). Print one line per attempt. Run three scenarios:

  1. ConnectionError("reset by peer"), then 503, then 502, then 200.
  2. A single 404.
  3. Six 503s with max_attempts=6 and cap=3.0.

Examples

Example 1

Input: The three scenarios above, with `base=0.5` and a fake `sleep` that returns immediately.

Output: flaky then ok: attempt 1: ConnectionError: reset by peer, sleeping 0.50s attempt 2: HTTP 503, sleeping 1.00s attempt 3: HTTP 502, sleeping 2.00s attempt 4: 200 ok permanent error: attempt 1: 404, not retryable raised HTTP 404 always down, max_attempts=6, cap=3: attempt 1: HTTP 503, sleeping 0.50s attempt 2: HTTP 503, sleeping 1.00s attempt 3: HTTP 503, sleeping 2.00s attempt 4: HTTP 503, sleeping 3.00s attempt 5: HTTP 503, sleeping 3.00s attempt 6: HTTP 503, giving up raised /v1/deploys: failed after 6 attempts (HTTP 503)

Explanation: Attempt 4 in the last scenario would wait 4 s, but cap=3.0 limits it. There are five sleeps for six attempts.

Hints

Approach

Optimal

Loop over attempt from 1 to max_attempts. Each pass ends in exactly one of four ways:

  1. Success (status < 400): return the response.
  2. Permanent failure (a 4xx that is not 429): raise HTTPError immediately. Retrying a 404 five times only makes the script slower and adds load.
  3. Retryable failure on the last attempt: raise RuntimeError with the path, the attempt count and the last reason, so the log says why it failed, not just that it failed.
  4. Retryable failure otherwise: compute min(cap, base * 2 ** (attempt - 1)), log it, and sleep.

The try/except/else keeps the exception path and the response path separate. Only ConnectionError and TimeoutError are caught. A bare except Exception would also retry bugs like a TypeError in your own code.

sleep is a parameter, so the demo passes a fake that does nothing and the whole run takes milliseconds. A test can also pass a function that records the delays and assert on them.

Jitter. If 200 hosts all fail at the same moment and all retry at exactly 0.5 s, 1 s, 2 s, they hit the recovering service in synchronized waves. Passing jitter=random.Random() switches to "full jitter": each delay becomes a random value between 0 and the computed delay, which spreads the retries out. The demo leaves it off so the output is predictable.

ComplexityTime O(max_attempts)Space O(1)
Python
import time
class Response:
def __init__(self, status, body=None):
self.status = status
self.body = body
class HTTPError(Exception):
def __init__(self, status):
super().__init__(f"HTTP {status}")
self.status = status
class FakeClient:
# Replays a scripted list of outcomes: an exception to raise or a Response.
def __init__(self, script):
self.script = list(script)
def get(self, path):
outcome = self.script.pop(0)
if isinstance(outcome, Exception):
raise outcome
return outcome
RETRYABLE_STATUS = {429, 500, 502, 503, 504}
RETRYABLE_EXC = (ConnectionError, TimeoutError)
def get_with_retry(client, path, max_attempts=5, base=0.5, cap=8.0,
sleep=time.sleep, jitter=None, log=print):
for attempt in range(1, max_attempts + 1):
try:
resp = client.get(path)
except RETRYABLE_EXC as e:
reason = f"{type(e).__name__}: {e}"
else:
if resp.status < 400:
log(f" attempt {attempt}: {resp.status} ok")
return resp
if resp.status not in RETRYABLE_STATUS:
log(f" attempt {attempt}: {resp.status}, not retryable")
raise HTTPError(resp.status)
reason = f"HTTP {resp.status}"
if attempt == max_attempts:
log(f" attempt {attempt}: {reason}, giving up")
raise RuntimeError(f"{path}: failed after {max_attempts} attempts ({reason})")
delay = min(cap, base * 2 ** (attempt - 1))
if jitter is not None: # "full jitter": uniform in [0, delay]
delay = jitter.uniform(0, delay)
log(f" attempt {attempt}: {reason}, sleeping {delay:.2f}s")
sleep(delay)
def fake_sleep(seconds):
pass # the demo does not really wait
print("flaky then ok:")
client = FakeClient([ConnectionError("reset by peer"), Response(503), Response(502), Response(200, "[]")])
get_with_retry(client, "/v1/deploys", sleep=fake_sleep)
print("permanent error:")
try:
get_with_retry(FakeClient([Response(404)]), "/v1/deploys/999", sleep=fake_sleep)
except HTTPError as e:
print(" raised", e)
print("always down, max_attempts=6, cap=3:")
try:
get_with_retry(FakeClient([Response(503)] * 6), "/v1/deploys",
max_attempts=6, cap=3.0, sleep=fake_sleep)
except RuntimeError as e:
print(" raised", e)

Follow-up questions

  • A 429 response includes a Retry-After header. Use it instead of your own delay.
  • Add an overall deadline (for example 30 s total) on top of max_attempts.
  • Turn this into a decorator, @retry(max_attempts=5), that works on any function.

Frequently asked questions

Every automation script that talks to a cloud API, a registry or an internal service will see transient errors. Retrying without backoff turns a brief outage into a flood of requests from every client at once, which can keep the service down longer. Interviewers look for the retryable/non-retryable split, the cap, and jitter.

Only if the operation is idempotent, or the API supports an idempotency key you send with every attempt. Otherwise a request that succeeded on the server but timed out on the way back gets applied twice, for example creating two load balancers. GETs, PUTs and DELETEs are normally safe to retry.

Without a cap, attempt 10 with base 0.5 waits 256 s. A cap keeps the worst case predictable, and together with max_attempts it bounds the total time the script can spend on one call.