Skip to main content
Bash and Linux

Total Bytes in a Log

easyawk and cut

Problem statement

Add up the bytes column of an access log with awk, skipping lines where the size is -, and print the total in bytes and in MiB with two decimals. Bandwidth questions like "how much did we serve in this log?" or "which day was heaviest?" all start with this running total.

access.log

TEXT
10.0.0.1 - - [01/Oct/2026:10:00:01 +0000] "GET /video.mp4 HTTP/1.1" 200 1048576
10.0.0.2 - - [01/Oct/2026:10:00:02 +0000] "GET /app.js HTTP/1.1" 200 524288
10.0.0.3 - - [01/Oct/2026:10:00:03 +0000] "GET /backup.tar HTTP/1.1" 200 2097152
10.0.0.4 - - [01/Oct/2026:10:00:04 +0000] "GET /old HTTP/1.1" 304 -
10.0.0.1 - - [01/Oct/2026:10:00:05 +0000] "GET /logo.png HTTP/1.1" 200 262144
10.0.0.2 - - [01/Oct/2026:10:00:06 +0000] "GET /video.mp4 HTTP/1.1" 200 1048576
10.0.0.5 - - [01/Oct/2026:10:00:07 +0000] "HEAD / HTTP/1.1" 200 -

The size is field 10, and - means "no body was sent".

  1. Print the total bytes.
  2. Print the total in MiB (1 MiB = 1,048,576 bytes), with two decimals.
  3. Print how many lines were counted and how many were skipped.

Expected output:

TEXT
total bytes: 4980736
total MiB: 4.75
lines counted: 5, skipped (-): 2

Hints

Hint 1: A variable in awk starts at 0, so { sum += $10 } adds each line's size to a running total. Print it once, in an END block.

Approach

Optimal: Running total in awk

Covers: awk variables, +=, ++, next, END, printf with %d and %.2f, MiB vs MB.

A running total. Picture a box labelled sum that starts at 0. For every line, awk adds that line's size into the box. After the last line, you open the box and print what is inside.

%%{init: {"flowchart": {"padding": 18, "nodeSpacing": 30, "rankSpacing": 40, "htmlLabels": true}, "themeVariables": {"fontSize": "18px"}}}%% flowchart TB S(["sum starts at 0"]):::purple --> L1["line 1: + 1048576"]:::blue L1 --> L2["line 2: + 524288"]:::blue L2 --> L4["line 4: size is -
skipped"]:::gray L4 --> MORE["... more lines ..."]:::blue MORE --> E{{"END"}}:::purple E --> R["print sum
4980736 bytes"]:::green classDef blue fill:#dbeafe,stroke:#2563eb,color:#1e3a8a,stroke-width:2px classDef yellow fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:2px classDef green fill:#d1fae5,stroke:#059669,color:#064e3b,stroke-width:2px classDef red fill:#fee2e2,stroke:#dc2626,color:#7f1d1d,stroke-width:2px classDef purple fill:#ede9fe,stroke:#7c3aed,color:#4c1d95,stroke-width:2px classDef gray fill:#f3f4f6,stroke:#6b7280,color:#111827,stroke-width:2px linkStyle default stroke:#94a3b8,stroke-width:2px

awk makes this easy because variables need no setup. The first time you use sum, awk treats it as 0. sum += $10 means "add field 10 to sum", and count++ means "add 1 to count".

Skipping lines with next. Some lines have - instead of a number, for example a 304 Not Modified or a HEAD request, where no body was sent. awk would treat - as 0, so the total would still be right, but the line count would be wrong. A separate rule handles them first:

TEXT
$10 == "-" { skipped++; next } count it as skipped, then go to the next line
{ sum += $10; counted++ } only reached by lines with a real number

next stops working on this line and moves straight to the next one, so the second rule never sees - lines.

Printing with printf. print writes values as they are. printf lets you choose the format, like in C:

Format Prints Example
%d a whole number 4980736
%.2f a number with 2 decimals 4.75
%s text MiB
\n a new line (printf does not add one)

MiB or MB? A MiB (mebibyte) is 1024 × 1024 = 1,048,576 bytes, which is what ls -lh and du -h use. An MB (megabyte) is 1,000,000 bytes, which is what disk makers and many dashboards use. Say which one you mean, because the difference is about 5%.

Walking through the code. The # Setup: lines only create the sample log, so skip past them.

  1. The first rule counts and skips the two - lines.
  2. The second rule adds the other five sizes and counts them.
  3. END prints the total with %d, divides by 1048576 and prints it with %.2f, then prints both counts.

The total is 4,980,736 bytes, which is exactly 4.75 MiB.

Edge cases. An empty file prints a total of 0, because awk treats an unused sum as 0 in printf "%d". A malformed line with text in field 10 would be added as 0, so if your logs can be messy, test that the field is a number: $10 ~ /^[0-9]+$/.

# Setup: create the sample access log in a fresh temporary folder
cd "$(mktemp -d)"
cat > access.log << 'LOG'
10.0.0.1 - - [01/Oct/2026:10:00:01 +0000] "GET /video.mp4 HTTP/1.1" 200 1048576
10.0.0.2 - - [01/Oct/2026:10:00:02 +0000] "GET /app.js HTTP/1.1" 200 524288
10.0.0.3 - - [01/Oct/2026:10:00:03 +0000] "GET /backup.tar HTTP/1.1" 200 2097152
10.0.0.4 - - [01/Oct/2026:10:00:04 +0000] "GET /old HTTP/1.1" 304 -
10.0.0.1 - - [01/Oct/2026:10:00:05 +0000] "GET /logo.png HTTP/1.1" 200 262144
10.0.0.2 - - [01/Oct/2026:10:00:06 +0000] "GET /video.mp4 HTTP/1.1" 200 1048576
10.0.0.5 - - [01/Oct/2026:10:00:07 +0000] "HEAD / HTTP/1.1" 200 -
LOG

awk '
  $10 == "-" { skipped++; next }
  { sum += $10; counted++ }
  END {
    printf "total bytes: %d\n", sum
    printf "total MiB:   %.2f\n", sum / 1048576
    printf "lines counted: %d, skipped (-): %d\n", counted, skipped
  }
' access.log

Interview follow-ups

  • Which single URL served the most bytes?

    Total per URL, then rank: awk '$10 != "-" { b[$7] += $10 } END { for (u in b) print b[u], u }' access.log | sort -rn | head -n 1. $7 is the path, and the array b keeps one total per path. Sorting the small result by number puts the biggest first. In this sample, /video.mp4 wins with two requests of 1 MiB each.

Frequently asked questions

print turns large numbers into text using a short format, so very big totals can come out in scientific notation. Use printf "%d\n", sum or printf "%.0f\n", sum to print every digit. awk stores numbers as floating point, which is exact for whole numbers up to about 9 thousand million million, so byte totals are safe. For money or anything needing exact decimals, do the maths in whole cents.

Use an array keyed by the day, one total per key. The date is the start of field 4, like [01/Oct/2026:10:00:01, so substr($4, 2, 11) gives 01/Oct/2026. Then { t[substr($4, 2, 11)] += $10 } END { for (d in t) print d, t[d] }. The next page, Average Time per Endpoint, explains arrays step by step.

Yes, but it is clumsier. cut -d' ' -f10 access.log | grep -v '^-$' | paste -sd+ - | bc pulls the column, drops -, joins the numbers with + and lets bc add them. It works, but it is four tools and breaks if fields shift. awk does the pick, filter and sum in one pass, which is why it is the usual answer.