DevOps NetworkDevOpsNetwork
Join Free
Bash and Linux

Archive Old Logs

mediumfind and xargs

Problem statement

Pack every log file older than 3 days into one dated .tar.gz, check the archive, and delete only the files that are really inside it. This is what log rotation does behind the scenes, and doing it by hand teaches the safe order: archive, verify, then delete.

The script creates these logs and sets their ages:

logs/

TEXT
logs/app-0926.log 5 days old
logs/app-0927.log 4 days old
logs/app-0930.log 1 day old
logs/web access.log 6 days old (the name has a space)
logs/current.log today
  1. List the logs older than 3 days.
  2. Pack exactly those into archive/logs-2026-10-01.tar.gz, safely, even with the space in a name.
  3. Check the archive and list what is inside.
  4. Delete only the files listed in the archive, then list what is left.

Expected output:

TEXT
== logs older than 3 days ==
logs/app-0926.log
logs/app-0927.log
logs/web access.log
== inside the archive (this also checks it is readable) ==
logs/app-0926.log
logs/app-0927.log
logs/web access.log
== logs left ==
logs/app-0930.log
logs/current.log

Hints

Hint 1: find logs -name '*.log' -mtime +3 -print0 | tar --null -czf archive.tar.gz -T - tells tar to read the null-separated file list from the pipe.

Approach

Optimal: find into tar, verify, then delete

Covers: find -mtime -print0, tar --null -T - (GNU), tar -tzf to verify, &&, while IFS= read -r, rm --, dated archive names.

The safe order is archive, verify, delete. If you delete first, or delete even when the archive failed, you lose logs. Each step only runs when the one before it worked:

%%{init: {"flowchart": {"padding": 18, "nodeSpacing": 30, "rankSpacing": 40, "htmlLabels": true}, "themeVariables": {"fontSize": "18px"}}}%% flowchart TB F(["find -mtime +3 -print0"]):::purple --> T(["tar --null -czf -T -"]):::purple T --> A[("logs-2026-10-01.tar.gz")]:::green A --> V{{"tar -tzf reads it?"}}:::yellow V --> D(["delete only the listed files"]):::red V --> K["keep everything"]:::gray classDef blue fill:#dbeafe,stroke:#2563eb,color:#1e3a8a,stroke-width:2px classDef yellow fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:2px classDef green fill:#d1fae5,stroke:#059669,color:#064e3b,stroke-width:2px classDef red fill:#fee2e2,stroke:#dc2626,color:#7f1d1d,stroke-width:2px classDef purple fill:#ede9fe,stroke:#7c3aed,color:#4c1d95,stroke-width:2px classDef gray fill:#f3f4f6,stroke:#6b7280,color:#111827,stroke-width:2px linkStyle default stroke:#94a3b8,stroke-width:2px

Handing find's list to tar. tar can read the list of files to pack from a file with -T, and -T - means "read the list from the pipe". --null says the names are separated by null characters, which matches find -print0, so web access.log arrives as one name. These two flags are GNU tar, the tar on almost every Linux server.

Verifying the archive. tar -tzf archive.tar.gz reads the whole archive and lists it. If the file is cut short or not valid gzip, it fails with an error. Joining with && means the delete step only runs if that check succeeds.

Delete from the archive's list. Running a second find ... -delete sounds simpler, but between the two runs a file can turn 3 days old, and it would be deleted without being archived. Reading the names back from the archive guarantees you only delete what is safely packed:

Part Does
tar -tzf "$archive" prints each path in the archive, one per line
while IFS= read -r f reads each line whole, spaces and backslashes kept
rm -- "$f" deletes it; -- protects names starting with -

Dated names. The archive name holds the date, so each run makes a new file instead of overwriting yesterday's. In a real script you would write logs-$(date +%F).tar.gz; here the date is fixed so the output is the same every time.

Walking through the code. The # Setup: lines only create the logs and fake their ages with touch -d, so skip past them.

  1. The find with -mtime +3 lists three files, including the one with a space.
  2. The find | tar pipeline creates the archive.
  3. tar -tzf ... | sort checks it and shows the contents.
  4. The while read loop deletes those files, and find shows the two recent logs left.
%%{init: {"flowchart": {"padding": 18, "nodeSpacing": 30, "rankSpacing": 40, "htmlLabels": true}, "themeVariables": {"fontSize": "18px"}}}%% flowchart LR subgraph GONE["archived, then deleted"] direction TB G1["app-0926.log"]:::yellow ~~~ G2["app-0927.log"]:::yellow ~~~ G3["web access.log"]:::yellow end subgraph KEEP["still in logs/"] direction TB K1["app-0930.log"]:::green ~~~ K2["current.log"]:::green end GONE ~~~ KEEP classDef blue fill:#dbeafe,stroke:#2563eb,color:#1e3a8a,stroke-width:2px classDef yellow fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:2px classDef green fill:#d1fae5,stroke:#059669,color:#064e3b,stroke-width:2px classDef red fill:#fee2e2,stroke:#dc2626,color:#7f1d1d,stroke-width:2px classDef purple fill:#ede9fe,stroke:#7c3aed,color:#4c1d95,stroke-width:2px classDef gray fill:#f3f4f6,stroke:#6b7280,color:#111827,stroke-width:2px linkStyle default stroke:#94a3b8,stroke-width:2px style GONE fill:transparent,stroke:#d97706,stroke-width:2px style KEEP fill:transparent,stroke:#059669,stroke-width:2px

Edge cases. If nothing is old enough, tar creates an empty archive; add a check like [ -n "$(find logs -name '*.log' -mtime +3 -print -quit)" ] to skip the run. A log a program still writes to should not be deleted under it; real rotation renames it and tells the program to reopen its log, which is what logrotate does.

# Setup: create logs and set their ages with touch -d, in a fresh temporary folder
cd "$(mktemp -d)"
mkdir logs archive
touch -d '5 days ago' logs/app-0926.log
touch -d '4 days ago' logs/app-0927.log
touch -d '1 day ago'  logs/app-0930.log
touch -d '6 days ago' "logs/web access.log"
touch logs/current.log
archive=archive/logs-2026-10-01.tar.gz    # a real script would use $(date +%F)

echo "== logs older than 3 days =="
find logs -name '*.log' -mtime +3 | sort

find logs -name '*.log' -mtime +3 -print0 | tar --null -czf "$archive" -T -

echo "== inside the archive (this also checks it is readable) =="
tar -tzf "$archive" | sort

# Delete only what is really in the archive, and only if the archive reads cleanly
tar -tzf "$archive" > /dev/null &&
  tar -tzf "$archive" | while IFS= read -r f; do rm -- "$f"; done

echo "== logs left =="
find logs -name '*.log' | sort

Interview follow-ups

  • Send the archive to another server before deleting anything.

    Add one more step between verify and delete: copy the archive, then check the copy. For example rsync -a "$archive" backup:/srv/logs/ && ssh backup "tar -tzf /srv/logs/$(basename "$archive") > /dev/null". Only when both succeed, run the delete loop. With && between every step, any failure stops the chain and the local logs stay where they are.

Frequently asked questions

On a real server, you should: logrotate is installed almost everywhere and runs daily from cron or a systemd timer. A short config file tells it which logs to rotate, how often, how many old copies to keep, and to compress them. It also handles the hard part, telling the app to reopen its log file, with postrotate or copytruncate. Writing it by hand, as here, is how you understand what logrotate does and how to debug it.

GNU tar can delete files right after adding them with --remove-files, which looks perfect. The risk is that it removes each file as it goes, so if tar fails part way, for example because the disk fills, some files are gone and the archive is broken. The archive, verify, delete order on this page never leaves you with neither. If you do use --remove-files, write the archive to a different disk from the logs.

Dated names sort in time order, so list them oldest first and drop the newest 7: ls -1 archive/logs-*.tar.gz | head -n -7 (GNU head). Check that list, then delete it with | xargs -r rm --. The names here are made by your own script and have no spaces, so plain ls and xargs are safe. For names you do not control, use find with -print0 instead.