DevOps NetworkDevOpsNetwork
Join Free
Practical infra coding

Compare two directory trees like rsync --dry-run

mediumFiles and filesystem

Problem statement

A release directory is built on CI and mirrored to a server. Before syncing, you want the plan: which files must be copied (only in the source), updated (in both, but different content), or deleted (only in the replica). Print one line per action, sorted by path, then a summary with the unchanged count.

Two files are the same only if their bytes are the same. Compare sizes first, because it is free, and hash only when the sizes are equal. bin/app below is the trap: same size in both trees, different content.

If either argument is not a directory, exit with an error. An empty listing would otherwise mean "copy everything" or "delete everything".

Sample trees

Bash
source/ replica/
bin/app "v2 binary" bin/app "v1 binary"
configs/app.yaml "port: 80\n" configs/app.yaml "port: 80\n"
configs/new.yaml "x: 1\n" README "hello"
README "hello" old/legacy.sh "echo hi\n"

Examples

Example 1

Input: `python solution.py` (the demo builds both trees above in a temporary directory)

Output: ~ update bin/app + copy configs/new.yaml - delete old/legacy.sh 1 to copy, 1 to update, 1 to delete, 2 unchanged

Explanation: bin/app is 9 bytes on both sides, so only the hash shows it changed. README and configs/app.yaml are unchanged.

Hints

Approach

Optimal

Listings first, content only when needed.

  1. listing walks a tree, keeps regular files, and maps relpath (converted with as_posix()) to size.
  2. For every path in the sorted union: only in source means copy; only in replica means delete; different sizes mean update without reading anything; equal sizes mean hash both sides, 1 MiB at a time, and compare.
  3. Count actions for the summary.

n is the number of paths. The listings are held in memory, which is fine for millions of files. File contents never are.

ComplexityTime O(n log n + bytes hashed)Space O(n)
Python
import hashlib
import os
import stat
import sys
import tempfile
from pathlib import Path
def sha256(path):
h = hashlib.sha256()
with open(path, "rb") as f:
for block in iter(lambda: f.read(1 << 20), b""):
h.update(block)
return h.digest()
def listing(root):
"""relative posix path -> size, for regular files only."""
files = {}
for dirpath, _dirs, names in os.walk(root, onerror=lambda e: print(f"warning: {e}", file=sys.stderr)):
for name in names:
full = os.path.join(dirpath, name)
try:
st = os.lstat(full)
except OSError:
continue
if stat.S_ISREG(st.st_mode):
files[Path(os.path.relpath(full, root)).as_posix()] = st.st_size
return files
def plan(src, dst):
a, b = listing(src), listing(dst)
actions, unchanged = [], 0
for rel in sorted(a.keys() | b.keys()):
if rel not in b:
actions.append(("+", "copy", rel))
elif rel not in a:
actions.append(("-", "delete", rel))
elif a[rel] != b[rel]:
actions.append(("~", "update", rel)) # different size: no need to read either file
elif sha256(os.path.join(src, rel)) != sha256(os.path.join(dst, rel)):
actions.append(("~", "update", rel)) # same size, different bytes
else:
unchanged += 1
return actions, unchanged
def report(src, dst):
for d in (src, dst):
if not os.path.isdir(d):
sys.exit(f"not a directory: {d}") # an empty listing would mean 'copy everything'
actions, unchanged = plan(src, dst)
for sign, verb, rel in actions:
print(f"{sign} {verb:<7} {rel}")
n = {v: sum(1 for _, verb, _ in actions if verb == v) for v in ("copy", "update", "delete")}
print(f"{n['copy']} to copy, {n['update']} to update, {n['delete']} to delete, {unchanged} unchanged")
SOURCE = {"bin/app": b"v2 binary", "configs/app.yaml": b"port: 80\n",
"configs/new.yaml": b"x: 1\n", "README": b"hello"}
REPLICA = {"bin/app": b"v1 binary", "configs/app.yaml": b"port: 80\n",
"README": b"hello", "old/legacy.sh": b"echo hi\n"}
def build(base, files):
for rel, data in files.items():
p = Path(base, rel)
p.parent.mkdir(parents=True, exist_ok=True)
p.write_bytes(data)
if __name__ == "__main__":
if len(sys.argv) > 2:
report(sys.argv[1], sys.argv[2])
else:
with tempfile.TemporaryDirectory() as src, tempfile.TemporaryDirectory() as dst:
build(src, SOURCE)
build(dst, REPLICA)
report(src, dst)

Follow-up questions

  • Apply the plan: copy with shutil.copy2, delete, then remove directories left empty.
  • The replica is on a remote host. What would you send over the network instead of file contents? (Listings with hashes, or rsync's rolling-checksum blocks.)
  • Also compare file permissions and flag executables that lost +x.

Frequently asked questions

By default rsync's quick check compares size and modification time and only transfers files where either differs. That is fast but can miss a change that keeps both the size and the mtime, which happens with some build tools and touch -r. rsync --checksum hashes like this script does, at the cost of reading every same-size file on both sides.

This script lists only files, so x (a file) becomes a copy and x/y becomes a delete. Applying that plan in the wrong order fails, because you cannot create file x while directory x exists. A real sync must process deletions first or handle the type change explicitly. It is a good edge case to mention without being asked.

Syncing artifacts, detecting config drift on servers and verifying backups are all tree comparisons. The interviewer looks for cheap checks before expensive ones, consistent path normalisation across operating systems, and a dry run before any destructive action.