Count with sort and uniq
Problem statement
Count how many times each HTTP status code appears in an access log, using a pipe of small commands: pull out the status column, sort it, count each group with uniq -c, and rank the groups. "How many 500s did we serve?" is the first question in almost every web incident, and this pipeline answers it in one line.
access.log
10.0.0.1 - - [01/Oct/2026:08:00:01 +0000] "GET / HTTP/1.1" 200 51210.0.0.2 - - [01/Oct/2026:08:00:02 +0000] "GET /cart HTTP/1.1" 200 102410.0.0.3 - - [01/Oct/2026:08:00:03 +0000] "GET /old HTTP/1.1" 301 010.0.0.1 - - [01/Oct/2026:08:00:04 +0000] "GET /missing HTTP/1.1" 404 12810.0.0.4 - - [01/Oct/2026:08:00:05 +0000] "POST /pay HTTP/1.1" 500 6410.0.0.2 - - [01/Oct/2026:08:00:06 +0000] "GET / HTTP/1.1" 200 51210.0.0.5 - - [01/Oct/2026:08:00:07 +0000] "GET /missing HTTP/1.1" 404 12810.0.0.1 - - [01/Oct/2026:08:00:08 +0000] "GET /cart HTTP/1.1" 200 102410.0.0.4 - - [01/Oct/2026:08:00:09 +0000] "POST /pay HTTP/1.1" 500 6410.0.0.6 - - [01/Oct/2026:08:00:10 +0000] "GET /nope HTTP/1.1" 404 12810.0.0.3 - - [01/Oct/2026:08:00:11 +0000] "GET / HTTP/1.1" 200 512Print, in this order:
- What
uniq -cgives when you forget to sort first (the first 4 lines). - The count for each status code.
- The same, most common first.
- The same, as
code count. - The codes that appear exactly once.
Expected output:
== uniq -c WITHOUT sort (wrong: 200 is split up) == 2 200 1 301 1 404 1 500== count per code (sort, then uniq -c) == 5 200 1 301 3 404 2 500== most common first == 5 200 3 404 2 500 1 301== as code and count ==200 5404 3500 2301 1== codes seen exactly once ==301Hints
awk '{print $9}' prints just that column.Approach
Optimal: sort | uniq -c | sort -rn
Covers: pipes |, awk '{print $9}', sort, sort -n, sort -r, sort -k, sort -t, uniq, uniq -c, uniq -d, uniq -u, LC_ALL=C.
A pipe joins small tools into one. a | b sends the output of a into b as its input. Each tool does one small job, and the pipe chains them. Read a pipeline from left to right, like a sentence:
Step 1: pick the column. A web server log line is split by spaces into numbered words, called fields. awk numbers them from $1:
IP"]:::blue LINE --> F4["$4
time"]:::gray LINE --> F7["$7
path"]:::yellow LINE --> F9["$9
status"]:::green LINE --> F10["$10
bytes"]:::purple classDef blue fill:#dbeafe,stroke:#2563eb,color:#1e3a8a,stroke-width:2px classDef yellow fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:2px classDef green fill:#d1fae5,stroke:#059669,color:#064e3b,stroke-width:2px classDef red fill:#fee2e2,stroke:#dc2626,color:#7f1d1d,stroke-width:2px classDef purple fill:#ede9fe,stroke:#7c3aed,color:#4c1d95,stroke-width:2px classDef gray fill:#f3f4f6,stroke:#6b7280,color:#111827,stroke-width:2px linkStyle default stroke:#94a3b8,stroke-width:2px
awk '{print $9}' prints field 9, the status code, for every line. The quoted request "GET / HTTP/1.1" has spaces inside, so it takes three fields ($6, $7, $8). That is why the status lands at $9.
Step 2: why sort must come first. uniq is simple: it looks at each line and the one just before it. When they are equal, it merges them. It never looks further back. So if the same code shows up in two places with something else between them, uniq sees two separate groups. The first block of the output shows this: 200 is counted as 2, and the other three 200s are counted later in separate groups.
wrong"]:::red end subgraph YES["sort, then uniq -c"] direction TB Y1["200 200 200 200 200 301 ..."]:::green --> Y2["5 200
right"]:::green end NO ~~~ YES classDef blue fill:#dbeafe,stroke:#2563eb,color:#1e3a8a,stroke-width:2px classDef yellow fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:2px classDef green fill:#d1fae5,stroke:#059669,color:#064e3b,stroke-width:2px classDef red fill:#fee2e2,stroke:#dc2626,color:#7f1d1d,stroke-width:2px classDef purple fill:#ede9fe,stroke:#7c3aed,color:#4c1d95,stroke-width:2px classDef gray fill:#f3f4f6,stroke:#6b7280,color:#111827,stroke-width:2px linkStyle default stroke:#94a3b8,stroke-width:2px style NO fill:transparent,stroke:#dc2626,stroke-width:2px style YES fill:transparent,stroke:#059669,stroke-width:2px
sort puts all equal lines next to each other, and then uniq -c counts each group correctly.
Step 3: rank the counts. uniq -c puts the count in front of each line, padded with spaces, like 5 200. Sorting those lines needs two flags:
| Flag | Means | Why here |
|---|---|---|
-n |
compare as numbers | without it, 10 sorts before 9, because 1 comes before 9 |
-r |
reverse, biggest first | most common at the top |
Other flags worth knowing.
| Command | Does |
|---|---|
sort -k 2 |
sort by the 2nd field instead of the whole line |
sort -t ',' -k 3 |
fields are separated by commas; sort by the 3rd |
sort -u |
sort and drop duplicates (like sort | uniq) |
uniq -d |
print only lines that appear more than once |
uniq -u |
print only lines that appear exactly once |
uniq -i |
ignore case when comparing |
Walking through the code. The # Setup: lines only create the sample log, so skip past them.
- Without
sort,uniq -csplits the 200s into groups.head -n 4shows the first four lines of that wrong answer. - With
sort, each code appears once, with its full count. Notice that the order is by code, not by count. sort -rnputs the biggest count first.- The last
awk '{print $2, $1}'swaps the two columns. It also drops the padding spaces, because awk splits on runs of spaces. uniq -uprints only codes that appear once:301.
Edge cases. An empty log gives empty output. A broken line with fewer than 9 fields makes $9 empty, which shows up as a count with a blank code. Filter it with awk 'NF >= 10 {print $9}' if your logs can be messy.
# Setup: create the sample access log in a fresh temporary folder
cd "$(mktemp -d)"
cat > access.log << 'LOG'
10.0.0.1 - - [01/Oct/2026:08:00:01 +0000] "GET / HTTP/1.1" 200 512
10.0.0.2 - - [01/Oct/2026:08:00:02 +0000] "GET /cart HTTP/1.1" 200 1024
10.0.0.3 - - [01/Oct/2026:08:00:03 +0000] "GET /old HTTP/1.1" 301 0
10.0.0.1 - - [01/Oct/2026:08:00:04 +0000] "GET /missing HTTP/1.1" 404 128
10.0.0.4 - - [01/Oct/2026:08:00:05 +0000] "POST /pay HTTP/1.1" 500 64
10.0.0.2 - - [01/Oct/2026:08:00:06 +0000] "GET / HTTP/1.1" 200 512
10.0.0.5 - - [01/Oct/2026:08:00:07 +0000] "GET /missing HTTP/1.1" 404 128
10.0.0.1 - - [01/Oct/2026:08:00:08 +0000] "GET /cart HTTP/1.1" 200 1024
10.0.0.4 - - [01/Oct/2026:08:00:09 +0000] "POST /pay HTTP/1.1" 500 64
10.0.0.6 - - [01/Oct/2026:08:00:10 +0000] "GET /nope HTTP/1.1" 404 128
10.0.0.3 - - [01/Oct/2026:08:00:11 +0000] "GET / HTTP/1.1" 200 512
LOG
echo "== uniq -c WITHOUT sort (wrong: 200 is split up) =="
awk '{print $9}' access.log | uniq -c | head -n 4
echo "== count per code (sort, then uniq -c) =="
awk '{print $9}' access.log | sort | uniq -c
echo "== most common first =="
awk '{print $9}' access.log | sort | uniq -c | sort -rn
echo "== as code and count =="
awk '{print $9}' access.log | sort | uniq -c | sort -rn | awk '{print $2, $1}'
echo "== codes seen exactly once =="
awk '{print $9}' access.log | sort | uniq -uInterview follow-ups
The log is 20 GB. Does this still work?
Yes, but
sortis the slow part, because it must see every line before it can print anything. GNUsorthandles files bigger than memory by writing sorted chunks to temporary files and merging them;-S 2Ggives it more memory and-T /bigdiskputs the temporary files somewhere with space.LC_ALL=Cmakes it much faster. Better still, count in one pass with awk and no sort at all:awk '{c[$9]++} END {for (k in c) print c[k], k}' access.log | sort -rn. awk keeps one counter per status code, which is tiny, and only the final few lines are sorted.
Frequently asked questions
sort follows the locale, the language settings of the shell. In en_US.UTF-8, sorting ignores some punctuation and mixes upper and lower case, while in the C locale it compares plain bytes, so B comes before a. Counts and pure numbers usually sort the same way everywhere, but names and paths may not. Put LC_ALL=C in front, like LC_ALL=C sort, when you need the same order on every machine. It is also faster on big files.
Plain sort compares text one character at a time. 10 starts with 1 and 9 starts with 9, and 1 comes first, so 10 goes before 9. -n reads the start of each line as a number and compares the values, so 9 comes before 10. Always use -n for counts, sizes and IDs. For sizes like 4K and 2M, use -h instead, which understands those suffixes.
For dropping duplicates, yes: both leave one copy of each line. sort -u is one step instead of two. They are not the same when you need counts or need to see the duplicates: only uniq has -c, -d and -u. So use sort -u for "list of unique values", and sort | uniq -c for "how many of each".