DevOps NetworkDevOpsNetwork
Join Free
Practical infra coding

Health-check 100 URLs in parallel with a worker limit

mediumConcurrency basics Must-do

Problem statement

After a deploy you must health-check 100 hosts. One at a time, a few slow or hung hosts make the whole check take minutes. Checking all 100 at once can overload the network or the service you are checking. Run the checks in parallel with at most 10 in flight, a per-request timeout of 0.2 s, and a clean, sorted report.

There is no network. A mock fake_get(url, timeout) stands in for the HTTP call; its behaviour is fixed per host number i (from host-000 to host-099):

  • if i % 23 == 0 the host hangs: the call waits timeout seconds and raises TimeoutError;
  • otherwise it takes (i * 37 % 120) milliseconds, then returns 503 if i % 17 == 0, raises ConnectionRefusedError for i == 42, and returns 200 otherwise.

Requirements:

  • Use concurrent.futures.ThreadPoolExecutor with max_workers=10.
  • A failing or crashing check must never lose the other results.
  • Print failures sorted by host (completion order is random, so the output must not depend on it), then healthy/total.
  • Prove the limit held: track how many checks are running at once with a lock-protected counter, and print whether the peak stayed within 10.

Examples

Example 1

Input: python solution.py

Output: FAIL host-000: TimeoutError: no response in 0.2s FAIL host-017: HTTP 503 FAIL host-023: TimeoutError: no response in 0.2s FAIL host-034: HTTP 503 FAIL host-042: ConnectionRefusedError: connection refused FAIL host-046: TimeoutError: no response in 0.2s FAIL host-051: HTTP 503 FAIL host-068: HTTP 503 FAIL host-069: TimeoutError: no response in 0.2s FAIL host-085: HTTP 503 FAIL host-092: TimeoutError: no response in 0.2s 89/100 healthy peak concurrency <= 10: True finished well under the sequential time: True

Explanation: host-000 matches both rules; the hang is checked first. The parallel run finishes in about a second, against more than 5 s one at a time.

Hints

Approach

Health checks spend almost all their time waiting on the network, so threads are a good fit even with Python's GIL: a thread that is waiting does not hold it.

  1. Bounded pool. ThreadPoolExecutor(max_workers=10) starts at most 10 threads. All 100 tasks are submitted up front; the pool queues them and runs 10 at a time. The limit protects both the machine running the checks and the fleet being checked.
  2. Map futures back to URLs. futures = {pool.submit(check, url): url ...}. as_completed yields futures in the order they finish, which is effectively random, so the dict is how you know which result belongs to which host.
  3. Errors stay per task. check turns expected failures (timeouts, refused connections, bad status) into result strings. An unexpected exception inside check is re-raised by fut.result(), caught in the main loop and recorded as crashed, so one bug cannot lose the other 99 results.
  4. Deterministic output. Results go into a dict and are printed sorted by URL after the pool finishes. Printing from worker threads would interleave lines in a different order on every run.
  5. Proving the limit. in_flight is shared between threads, so it is only changed under a Lock, and the finally block decrements it even when the call raised. peak records the maximum. Without the lock, two threads could read and write the counter at the same time and the number would be wrong.

With 10 workers the total time is roughly the sum of latencies divided by 10, bounded below by the slowest single check.

ComplexityTime O(n * latency / workers)Space O(n)
Python
import threading
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
URLS = [f"http://host-{i:03d}.internal/healthz" for i in range(100)]
MAX_WORKERS = 10
TIMEOUT = 0.2
in_flight = 0
peak = 0
lock = threading.Lock()
def fake_get(url, timeout):
# Mock HTTP call. Behaviour is fixed per host so every run gives the same answer.
i = int(url.split("-")[1][:3])
latency = 5.0 if i % 23 == 0 else (i * 37 % 120) / 1000 # 0-119 ms, or a hang
if latency > timeout:
time.sleep(timeout)
raise TimeoutError(f"no response in {timeout}s")
time.sleep(latency)
if i % 17 == 0:
return 503
if i == 42:
raise ConnectionRefusedError("connection refused")
return 200
def check(url):
global in_flight, peak
with lock:
in_flight += 1
peak = max(peak, in_flight)
try:
status = fake_get(url, timeout=TIMEOUT)
return ("ok" if status == 200 else f"HTTP {status}")
except (TimeoutError, ConnectionError) as e:
return f"{type(e).__name__}: {e}"
finally:
with lock:
in_flight -= 1
start = time.monotonic()
results = {}
with ThreadPoolExecutor(max_workers=MAX_WORKERS) as pool:
futures = {pool.submit(check, url): url for url in URLS}
for fut in as_completed(futures):
url = futures[fut]
try:
results[url] = fut.result()
except Exception as e: # a bug in check() must not lose the other results
results[url] = f"crashed: {e!r}"
elapsed = time.monotonic() - start
failed = {u: r for u, r in results.items() if r != "ok"}
for url in sorted(failed):
print(f"FAIL {url.split('//')[1].split('.')[0]}: {failed[url]}")
print(f"{len(results) - len(failed)}/{len(results)} healthy")
print(f"peak concurrency <= {MAX_WORKERS}: {peak <= MAX_WORKERS}")
print(f"finished well under the sequential time: {elapsed < 3}")

Follow-up questions

  • Add an overall deadline: after 30 s, report every check that has not finished as unknown.
  • Retry each failed host once before reporting it, without blocking the other workers.
  • Rewrite it with asyncio and a Semaphore(10). When would that be the better choice?

Frequently asked questions

Checking many hosts or endpoints quickly is a core SRE and deploy task: smoke tests, fleet audits, canary verification. The interview version checks that you bound concurrency, keep per-task failures isolated, and produce a stable report.

The work is waiting on I/O, not computing. Threads are cheap and release the GIL while blocked on sockets, so they parallelize I/O well. Processes help for CPU-heavy work; for network checks they only add start-up cost and data copying.

Start from what the targets and the network can tolerate, not from CPU count. Ten to a few dozen concurrent HTTP checks is typical for a script. Measure: raising the limit stops helping once you hit bandwidth, file descriptor or target-side rate limits.