DevOps NetworkDevOpsNetwork
Join Free
Practical infra coding

Apply a daily, weekly and monthly backup retention policy

hardFiles and filesystem

Problem statement

A backup directory contains db-YYYYMMDD-HHMM.tar.gz files, created nightly plus the odd manual run. Implement a grandfather-father-son retention policy and print, newest first, which files to keep and why, and which to delete:

  • daily 3: keep the newest backup of each of the 3 most recent days that have a backup.
  • weekly 3: keep the newest backup of each of the 3 most recent ISO weeks that have a backup.
  • monthly 3: keep the newest backup of each of the 3 most recent calendar months that have a backup.

A file is kept if any rule keeps it. Everything else is deleted. Files whose name does not match the pattern, or whose date is invalid, are never touched; list them as ignored. It is a dry run by default. As a safety net, refuse to run if the policy would delete every backup.

Directory listing (dates in September 2026: the 24th is a Thursday, the 21st, 14th and 7th are Mondays)

TEXT
db-20260924-0200.tar.gz
db-20260923-1415.tar.gz (manual run)
db-20260923-0200.tar.gz
db-20260922-0200.tar.gz
db-20260921-0200.tar.gz
db-20260914-0200.tar.gz
db-20260907-0200.tar.gz
db-20260831-0200.tar.gz
db-20260801-0200.tar.gz
db-20260715-0200.tar.gz
db-20260601-0200.tar.gz
db-latest.tar.gz
notes.txt

Examples

Example 1

Input: `python solution.py` (the demo creates these files in a temporary directory and runs a dry run)

Output: KEEP db-20260924-0200.tar.gz daily, weekly, monthly KEEP db-20260923-1415.tar.gz daily DELETE db-20260923-0200.tar.gz KEEP db-20260922-0200.tar.gz daily DELETE db-20260921-0200.tar.gz KEEP db-20260914-0200.tar.gz weekly KEEP db-20260907-0200.tar.gz weekly KEEP db-20260831-0200.tar.gz monthly DELETE db-20260801-0200.tar.gz KEEP db-20260715-0200.tar.gz monthly DELETE db-20260601-0200.tar.gz ignored (name does not match): db-latest.tar.gz, notes.txt

Explanation: Daily keeps the 24th, the 23rd (its newest run, 14:15) and the 22nd. The 21st to 24th are one ISO week, already covered by the 24th, so weekly then takes the 14th and the 7th. Monthly takes the newest in September (24th), August (31st) and July (15th).

Hints

Approach

Optimal

Parse strictly, sort once, then run each rule as a bucket scan.

  1. parse accepts only names that fullmatch the pattern and whose timestamp passes strptime, so db-20260231-... (no 31 February) is ignored rather than guessed. Sort (timestamp, name) newest first.
  2. For each rule, walk the sorted list keeping a set of buckets seen. The first backup met in a new bucket is that bucket's newest: add the rule name to its reasons. Stop when the rule has filled its count.
  3. A backup with no reasons is deleted.
  4. apply refuses to proceed if every backup would be deleted, prints the plan, and calls os.remove only with dry_run=False.

b backups, r rules. Keeping reasons per file makes the output auditable: you can see why a file survived, which is the first question anyone asks when a retention policy surprises them.

ComplexityTime O(b log b + r * b)Space O(b)
Python
import os
import re
import sys
import tempfile
from datetime import datetime
from pathlib import Path
NAME = re.compile(r"db-(\d{8}-\d{4})\.tar\.gz")
RULES = { # rule -> function giving the bucket a backup falls into
"daily": lambda ts: ts.date(),
"weekly": lambda ts: ts.isocalendar()[:2], # (ISO year, ISO week)
"monthly": lambda ts: (ts.year, ts.month),
}
def parse(names):
backups, ignored = [], []
for name in names:
m = NAME.fullmatch(name)
try:
ts = datetime.strptime(m[1], "%Y%m%d-%H%M") if m else None
except ValueError: # e.g. 20260231
ts = None
(backups.append((ts, name)) if ts else ignored.append(name))
backups.sort(reverse=True) # newest first
return backups, sorted(ignored)
def plan(names, keep):
"""keep = {"daily": 3, ...}. Returns ([(name, [reasons])] newest first, ignored)."""
backups, ignored = parse(names)
reasons = {name: [] for _, name in backups}
for rule, count in keep.items():
bucket_of = RULES[rule]
seen = set()
for ts, name in backups: # newest first, so the first hit per bucket is its newest
if len(seen) >= count:
break
bucket = bucket_of(ts)
if bucket not in seen:
seen.add(bucket)
reasons[name].append(rule)
return [(name, reasons[name]) for _, name in backups], ignored
def apply(directory, keep, dry_run=True):
names = [p.name for p in Path(directory).iterdir() if p.is_file()]
decisions, ignored = plan(names, keep)
if decisions and not any(r for _, r in decisions):
sys.exit("refusing to delete every backup; check the policy") # safety net
for name, why in decisions:
if why:
print(f"KEEP {name} {', '.join(why)}")
else:
print(f"DELETE {name}")
if not dry_run:
os.remove(Path(directory, name))
if ignored:
print("ignored (name does not match):", ", ".join(ignored))
SAMPLE = [
"db-20260924-0200.tar.gz", "db-20260923-1415.tar.gz", "db-20260923-0200.tar.gz",
"db-20260922-0200.tar.gz", "db-20260921-0200.tar.gz", "db-20260914-0200.tar.gz",
"db-20260907-0200.tar.gz", "db-20260831-0200.tar.gz", "db-20260801-0200.tar.gz",
"db-20260715-0200.tar.gz", "db-20260601-0200.tar.gz", "db-latest.tar.gz", "notes.txt",
]
if __name__ == "__main__":
policy = {"daily": 3, "weekly": 3, "monthly": 3}
if len(sys.argv) > 1:
apply(sys.argv[1], policy, dry_run="--apply" not in sys.argv)
else:
with tempfile.TemporaryDirectory() as tmp:
for name in SAMPLE:
Path(tmp, name).touch()
apply(tmp, policy) # dry run: prints the plan, deletes nothing

Follow-up questions

  • Add a yearly rule, and a keep-last N rule that ignores buckets.
  • The backups are objects in S3 with a LastModified time instead of dated names. What changes? (Listing is paginated, and deletes are batched.)
  • How would you test the policy across a year boundary and a leap day without waiting for real backups?

Frequently asked questions

With "the last 3 calendar days", a week-long backup outage followed by a cleanup run would keep nothing from the daily rule. Counting buckets that actually contain backups, which is how tools such as restic's forget --keep-daily behave, always keeps the most recent N. It fails safe when backups have stopped, which is exactly when you need the old ones.

Every backup system needs retention, and it is a classic source of off-by-one bugs, time zone bugs and year-boundary bugs that surface months later as "why is there no backup from last month?". Interviewers look for bucket logic, strict parsing, a dry run and a refusal to delete everything.

Yes. A backup at 00:30 local time might be the previous day in UTC, which changes which daily bucket it falls into. Pick one zone, preferably UTC, write it into the file names, and bucket in that zone. This script treats the timestamps in names as naive and consistent, and that assumption should be stated.