Top IPs in an Access Log
Problem statement
Find the 3 client IPs that sent the most requests, then the 3 that got the most server errors (5xx), and count how many different clients there were. This is the most common live shell task in DevOps and SRE interviews, and a real first step when a site slows down: is one client hammering us, and who is seeing the errors?
access.log
10.0.0.7 - - [01/Oct/2026:09:00:01 +0000] "GET / HTTP/1.1" 200 51210.0.0.2 - - [01/Oct/2026:09:00:02 +0000] "GET /cart HTTP/1.1" 200 1024192.168.1.9 - - [01/Oct/2026:09:00:03 +0000] "POST /pay HTTP/1.1" 502 010.0.0.7 - - [01/Oct/2026:09:00:04 +0000] "POST /pay HTTP/1.1" 500 6410.0.0.5 - - [01/Oct/2026:09:00:05 +0000] "GET / HTTP/1.1" 200 51210.0.0.2 - - [01/Oct/2026:09:00:06 +0000] "GET /cart HTTP/1.1" 503 010.0.0.7 - - [01/Oct/2026:09:00:07 +0000] "GET /cart HTTP/1.1" 200 1024192.168.1.9 - - [01/Oct/2026:09:00:08 +0000] "POST /pay HTTP/1.1" 502 010.0.0.8 - - [01/Oct/2026:09:00:09 +0000] "GET /nope HTTP/1.1" 404 12810.0.0.2 - - [01/Oct/2026:09:00:10 +0000] "GET / HTTP/1.1" 200 51210.0.0.7 - - [01/Oct/2026:09:00:11 +0000] "POST /pay HTTP/1.1" 500 6410.0.0.5 - - [01/Oct/2026:09:00:12 +0000] "GET /cart HTTP/1.1" 200 1024192.168.1.9 - - [01/Oct/2026:09:00:13 +0000] "POST /pay HTTP/1.1" 502 010.0.0.2 - - [01/Oct/2026:09:00:14 +0000] "GET / HTTP/1.1" 200 51210.0.0.7 - - [01/Oct/2026:09:00:15 +0000] "GET / HTTP/1.1" 200 512The client IP is the first field and the status code is the ninth.
- Print the top 3 IPs by number of requests, as
ip count. - Print the top 3 IPs by number of 5xx responses, as
ip count. - Print how many different client IPs there are.
Expected output:
== top 3 IPs by requests ==10.0.0.7 510.0.0.2 4192.168.1.9 3== top 3 IPs by 5xx errors ==192.168.1.9 310.0.0.7 210.0.0.2 1== unique client IPs ==5Hints
awk '{print $1}' | sort | uniq -c | sort -rn | head -n 3.Approach
Optimal: Count and rank pipeline
Covers: awk '{print $1}', awk conditions $9 ~ /^5/, sort, uniq -c, sort -rn, head -n, sort -u, wc -l, cut -d' ' -f1.
One pattern answers most "top N" questions. Top IPs, top URLs, top user agents, top error messages: they are all the same five steps. Learn this shape once and you can answer any of them in an interview without thinking.
15 lines")]:::blue --> X(["extract
awk print $1"]):::purple X --> G(["group
sort"]):::purple G --> C(["count
uniq -c"]):::purple C --> R(["rank
sort -rn"]):::purple R --> H(["cut
head -n 3"]):::purple H --> OUT["10.0.0.7 5
10.0.0.2 4
192.168.1.9 3"]:::green classDef blue fill:#dbeafe,stroke:#2563eb,color:#1e3a8a,stroke-width:2px classDef yellow fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:2px classDef green fill:#d1fae5,stroke:#059669,color:#064e3b,stroke-width:2px classDef red fill:#fee2e2,stroke:#dc2626,color:#7f1d1d,stroke-width:2px classDef purple fill:#ede9fe,stroke:#7c3aed,color:#4c1d95,stroke-width:2px classDef gray fill:#f3f4f6,stroke:#6b7280,color:#111827,stroke-width:2px linkStyle default stroke:#94a3b8,stroke-width:2px
| Step | Command | What the data looks like after it |
|---|---|---|
| extract | awk '{print $1}' |
one IP per line, in log order |
| group | sort |
equal IPs next to each other |
| count | uniq -c |
5 10.0.0.7, one line per IP |
| rank | sort -rn |
biggest count first |
| cut | head -n 3 |
only the top three |
A final awk '{print $2, $1}' swaps the columns into ip count and removes the padding spaces that uniq -c adds.
Filter first, then count. For "who gets the most errors", the steps stay the same, but you keep only the error lines before extracting. awk can do both in one go: awk '$9 ~ /^5/ {print $1}'.
| Part | Means |
|---|---|
$9 |
the ninth field, the status code |
~ |
"matches the pattern" |
/^5/ |
starts with 5, so 500 to 599 |
{print $1} |
for those lines only, print the IP |
You could also write grep '" 5[0-9][0-9] ' before a plain awk '{print $1}', but testing the exact field in awk is safer: a URL or a byte count that happens to contain " 5 cannot fool it.
Counting unique values. "How many different clients?" needs no counts per IP. sort -u keeps one copy of each IP, and wc -l counts the lines that are left.
Walking through the code. The # Setup: lines only create the sample log, so skip past them.
- The first pipeline runs all five steps and prints
10.0.0.7 5,10.0.0.2 4and192.168.1.9 3. - The second keeps only lines whose status starts with 5, then runs the same steps.
192.168.1.9has fewer requests in total but the most errors, which is exactly the kind of thing this check is for. sort -u | wc -lprints5.
Edge cases. If two IPs tie at the edge of the top 3, head keeps whichever one sort put first, so say "top 3, ties broken by IP" and add -k2 if it matters. If fewer than 3 IPs exist, head -n 3 simply prints fewer lines. Behind a load balancer or CDN, field 1 is the balancer's IP, not the client's; see the FAQ.
# Setup: create the sample access log in a fresh temporary folder
cd "$(mktemp -d)"
cat > access.log << 'LOG'
10.0.0.7 - - [01/Oct/2026:09:00:01 +0000] "GET / HTTP/1.1" 200 512
10.0.0.2 - - [01/Oct/2026:09:00:02 +0000] "GET /cart HTTP/1.1" 200 1024
192.168.1.9 - - [01/Oct/2026:09:00:03 +0000] "POST /pay HTTP/1.1" 502 0
10.0.0.7 - - [01/Oct/2026:09:00:04 +0000] "POST /pay HTTP/1.1" 500 64
10.0.0.5 - - [01/Oct/2026:09:00:05 +0000] "GET / HTTP/1.1" 200 512
10.0.0.2 - - [01/Oct/2026:09:00:06 +0000] "GET /cart HTTP/1.1" 503 0
10.0.0.7 - - [01/Oct/2026:09:00:07 +0000] "GET /cart HTTP/1.1" 200 1024
192.168.1.9 - - [01/Oct/2026:09:00:08 +0000] "POST /pay HTTP/1.1" 502 0
10.0.0.8 - - [01/Oct/2026:09:00:09 +0000] "GET /nope HTTP/1.1" 404 128
10.0.0.2 - - [01/Oct/2026:09:00:10 +0000] "GET / HTTP/1.1" 200 512
10.0.0.7 - - [01/Oct/2026:09:00:11 +0000] "POST /pay HTTP/1.1" 500 64
10.0.0.5 - - [01/Oct/2026:09:00:12 +0000] "GET /cart HTTP/1.1" 200 1024
192.168.1.9 - - [01/Oct/2026:09:00:13 +0000] "POST /pay HTTP/1.1" 502 0
10.0.0.2 - - [01/Oct/2026:09:00:14 +0000] "GET / HTTP/1.1" 200 512
10.0.0.7 - - [01/Oct/2026:09:00:15 +0000] "GET / HTTP/1.1" 200 512
LOG
echo "== top 3 IPs by requests =="
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -n 3 | awk '{print $2, $1}'
echo "== top 3 IPs by 5xx errors =="
awk '$9 ~ /^5/ {print $1}' access.log | sort | uniq -c | sort -rn | head -n 3 | awk '{print $2, $1}'
echo "== unique client IPs =="
awk '{print $1}' access.log | sort -u | wc -lRecapThe whole problem in a few lines, for the night before
- Spot it: "top N" of anything in a log: IPs, paths, status codes, user agents
- Idea: extract, then
sort | uniq -c | sort -rn | head -n N; filter with an awk condition first when needed - Cost: O(n log n) for the sort; the one-pass awk counter is O(n) with memory for one counter per distinct value
- Trap:
uniq -cwithoutsortfirst, andsortwithout-non counts
Interview follow-ups
The log is 30 GB and still being written. How do you do this fast?
Avoid sorting every line. Count in one awk pass and only sort the small result:
awk '{c[$1]++} END {for (ip in c) print c[ip], ip}' access.log | sort -rn | head -n 3. awk keeps one counter per distinct IP, which is usually thousands, not billions, so memory stays small and the file is read once. PutLC_ALL=Cin front for extra speed. For a live view, run the same command ontail -n 100000 access.logto look at only the most recent lines.
Frequently asked questions
If the server sits behind a load balancer, reverse proxy or CDN, the connection comes from that machine, so field 1 is its IP for every request. The real client IP is usually passed in a header called X-Forwarded-For, and the web server must be told to log it. In nginx that means adding $http_x_forwarded_for to the log_format. Then pull that field instead of $1. Check this first whenever "top IPs" shows only private addresses like 10.x or 172.16.x.
When fields are separated by single spaces, cut -d' ' -f1 works and is slightly faster than awk. But cut treats every single space as a separator, so two spaces in a row create an empty field and shift every number after it. awk splits on runs of spaces and tabs, so it does not have that problem. awk can also filter in the same step, as in the 5xx list. In interviews, either is fine; say why you chose it.
Filter by time before counting. The time is in field 4, like [01/Oct/2026:09:00:01, so for one hour you can match its start: awk '$4 ~ /01\/Oct\/2026:09:/ {print $1}'. For "the last 60 minutes from now", turning each time into seconds is easier in a small script, or with tail if you know roughly how many lines an hour produces. Many teams send logs to a log system for exactly this kind of question, but the shell version is what interviews ask for.