Total Bytes in a Log
Problem statement
Add up the bytes column of an access log with awk, skipping lines where the size is -, and print the total in bytes and in MiB with two decimals. Bandwidth questions like "how much did we serve in this log?" or "which day was heaviest?" all start with this running total.
access.log
10.0.0.1 - - [01/Oct/2026:10:00:01 +0000] "GET /video.mp4 HTTP/1.1" 200 104857610.0.0.2 - - [01/Oct/2026:10:00:02 +0000] "GET /app.js HTTP/1.1" 200 52428810.0.0.3 - - [01/Oct/2026:10:00:03 +0000] "GET /backup.tar HTTP/1.1" 200 209715210.0.0.4 - - [01/Oct/2026:10:00:04 +0000] "GET /old HTTP/1.1" 304 -10.0.0.1 - - [01/Oct/2026:10:00:05 +0000] "GET /logo.png HTTP/1.1" 200 26214410.0.0.2 - - [01/Oct/2026:10:00:06 +0000] "GET /video.mp4 HTTP/1.1" 200 104857610.0.0.5 - - [01/Oct/2026:10:00:07 +0000] "HEAD / HTTP/1.1" 200 -The size is field 10, and - means "no body was sent".
- Print the total bytes.
- Print the total in MiB (1 MiB = 1,048,576 bytes), with two decimals.
- Print how many lines were counted and how many were skipped.
Expected output:
total bytes: 4980736total MiB: 4.75lines counted: 5, skipped (-): 2Hints
{ sum += $10 } adds each line's size to a running total. Print it once, in an END block.Approach
Optimal: Running total in awk
Covers: awk variables, +=, ++, next, END, printf with %d and %.2f, MiB vs MB.
A running total. Picture a box labelled sum that starts at 0. For every line, awk adds that line's size into the box. After the last line, you open the box and print what is inside.
skipped"]:::gray L4 --> MORE["... more lines ..."]:::blue MORE --> E{{"END"}}:::purple E --> R["print sum
4980736 bytes"]:::green classDef blue fill:#dbeafe,stroke:#2563eb,color:#1e3a8a,stroke-width:2px classDef yellow fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:2px classDef green fill:#d1fae5,stroke:#059669,color:#064e3b,stroke-width:2px classDef red fill:#fee2e2,stroke:#dc2626,color:#7f1d1d,stroke-width:2px classDef purple fill:#ede9fe,stroke:#7c3aed,color:#4c1d95,stroke-width:2px classDef gray fill:#f3f4f6,stroke:#6b7280,color:#111827,stroke-width:2px linkStyle default stroke:#94a3b8,stroke-width:2px
awk makes this easy because variables need no setup. The first time you use sum, awk treats it as 0. sum += $10 means "add field 10 to sum", and count++ means "add 1 to count".
Skipping lines with next. Some lines have - instead of a number, for example a 304 Not Modified or a HEAD request, where no body was sent. awk would treat - as 0, so the total would still be right, but the line count would be wrong. A separate rule handles them first:
$10 == "-" { skipped++; next } count it as skipped, then go to the next line{ sum += $10; counted++ } only reached by lines with a real numbernext stops working on this line and moves straight to the next one, so the second rule never sees - lines.
Printing with printf. print writes values as they are. printf lets you choose the format, like in C:
| Format | Prints | Example |
|---|---|---|
%d |
a whole number | 4980736 |
%.2f |
a number with 2 decimals | 4.75 |
%s |
text | MiB |
\n |
a new line (printf does not add one) |
MiB or MB? A MiB (mebibyte) is 1024 × 1024 = 1,048,576 bytes, which is what ls -lh and du -h use. An MB (megabyte) is 1,000,000 bytes, which is what disk makers and many dashboards use. Say which one you mean, because the difference is about 5%.
Walking through the code. The # Setup: lines only create the sample log, so skip past them.
- The first rule counts and skips the two
-lines. - The second rule adds the other five sizes and counts them.
ENDprints the total with%d, divides by 1048576 and prints it with%.2f, then prints both counts.
The total is 4,980,736 bytes, which is exactly 4.75 MiB.
Edge cases. An empty file prints a total of 0, because awk treats an unused sum as 0 in printf "%d". A malformed line with text in field 10 would be added as 0, so if your logs can be messy, test that the field is a number: $10 ~ /^[0-9]+$/.
# Setup: create the sample access log in a fresh temporary folder
cd "$(mktemp -d)"
cat > access.log << 'LOG'
10.0.0.1 - - [01/Oct/2026:10:00:01 +0000] "GET /video.mp4 HTTP/1.1" 200 1048576
10.0.0.2 - - [01/Oct/2026:10:00:02 +0000] "GET /app.js HTTP/1.1" 200 524288
10.0.0.3 - - [01/Oct/2026:10:00:03 +0000] "GET /backup.tar HTTP/1.1" 200 2097152
10.0.0.4 - - [01/Oct/2026:10:00:04 +0000] "GET /old HTTP/1.1" 304 -
10.0.0.1 - - [01/Oct/2026:10:00:05 +0000] "GET /logo.png HTTP/1.1" 200 262144
10.0.0.2 - - [01/Oct/2026:10:00:06 +0000] "GET /video.mp4 HTTP/1.1" 200 1048576
10.0.0.5 - - [01/Oct/2026:10:00:07 +0000] "HEAD / HTTP/1.1" 200 -
LOG
awk '
$10 == "-" { skipped++; next }
{ sum += $10; counted++ }
END {
printf "total bytes: %d\n", sum
printf "total MiB: %.2f\n", sum / 1048576
printf "lines counted: %d, skipped (-): %d\n", counted, skipped
}
' access.logInterview follow-ups
Which single URL served the most bytes?
Total per URL, then rank:
awk '$10 != "-" { b[$7] += $10 } END { for (u in b) print b[u], u }' access.log | sort -rn | head -n 1.$7is the path, and the arraybkeeps one total per path. Sorting the small result by number puts the biggest first. In this sample,/video.mp4wins with two requests of 1 MiB each.
Frequently asked questions
print turns large numbers into text using a short format, so very big totals can come out in scientific notation. Use printf "%d\n", sum or printf "%.0f\n", sum to print every digit. awk stores numbers as floating point, which is exact for whole numbers up to about 9 thousand million million, so byte totals are safe. For money or anything needing exact decimals, do the maths in whole cents.
Use an array keyed by the day, one total per key. The date is the start of field 4, like [01/Oct/2026:10:00:01, so substr($4, 2, 11) gives 01/Oct/2026. Then { t[substr($4, 2, 11)] += $10 } END { for (d in t) print d, t[d] }. The next page, Average Time per Endpoint, explains arrays step by step.
Yes, but it is clumsier. cut -d' ' -f10 access.log | grep -v '^-$' | paste -sd+ - | bc pulls the column, drops -, joins the numbers with + and lets bc add them. It works, but it is four tools and breaks if fields shift. awk does the pick, filter and sum in one pass, which is why it is the usual answer.