Average Time per Endpoint
Problem statement
Work out the average response time for each endpoint in a log, with the number of requests and the slowest one, and list the endpoints from slowest to fastest. Grouping by a key and reporting per group is the most useful awk skill there is: per endpoint, per host, per user, per minute.
requests.log (method, path, status, seconds)
GET /api/cart 200 0.120GET /api/pay 200 0.800GET /health 200 0.002GET /api/cart 200 0.180GET /api/pay 500 1.400GET /api/cart 200 0.150GET /health 200 0.004GET /api/search 200 0.300For each path, print the average time in milliseconds with one decimal, the number of requests, and the slowest request, ordered from the slowest average to the fastest.
Expected output:
/api/pay avg 1100.0 ms n=2 max 1400 ms/api/search avg 300.0 ms n=1 max 300 ms/api/cart avg 150.0 ms n=3 max 180 ms/health avg 3.0 ms n=2 max 4 msHints
s[$2] += $4 adds up the time and n[$2]++ counts the requests for that path.Approach
Optimal: awk arrays, then sort
Covers: awk associative arrays, a[key] += x, a[key]++, for (k in a), keeping a maximum, printf padding, sorting the result.
An array is a set of labelled boxes. In awk, an array does not need numbers as positions. Any text can be the label (the key). s["/api/cart"] is one box, s["/health"] is another. A new box starts at 0 the first time you use it, just like a normal variable.
/api/cart"]):::purple K --> S["s[/api/cart] += 120"]:::blue K --> N["n[/api/cart] += 1"]:::yellow K --> X["mx[/api/cart]
keep the biggest"]:::red classDef blue fill:#dbeafe,stroke:#2563eb,color:#1e3a8a,stroke-width:2px classDef yellow fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:2px classDef green fill:#d1fae5,stroke:#059669,color:#064e3b,stroke-width:2px classDef red fill:#fee2e2,stroke:#dc2626,color:#7f1d1d,stroke-width:2px classDef purple fill:#ede9fe,stroke:#7c3aed,color:#4c1d95,stroke-width:2px classDef gray fill:#f3f4f6,stroke:#6b7280,color:#111827,stroke-width:2px linkStyle default stroke:#94a3b8,stroke-width:2px
For each line, the program does three things with the path in $2:
| Code | Means |
|---|---|
t = $4 * 1000 |
this request's time in milliseconds |
s[$2] += t |
add it to this path's total |
n[$2]++ |
add 1 to this path's count |
if (t > mx[$2]) mx[$2] = t |
remember the biggest time seen for this path |
After the last line, each path has its own total, count and maximum, and the average is just total divided by count.
Looping over the boxes. for (p in s) visits every key in the array s, putting each one in p. The order is not fixed: it can change between awk versions and runs. So the program prints the average first on each line, and sort -k1,1nr puts the slowest first. A final awk formats the columns neatly with printf.
Padding with printf. A number after % sets a width: %-12s prints text in 12 characters, left-aligned (the -), and %7.1f prints a number in 7 characters with 1 decimal, right-aligned. That lines the columns up like a table.
total 450, n 3"]:::blue ~~~ B2["/api/pay
total 2200, n 2"]:::red end BOX --> AVG(["END: total / n"]):::purple --> OUT["/api/pay 1100.0
/api/cart 150.0"]:::green classDef blue fill:#dbeafe,stroke:#2563eb,color:#1e3a8a,stroke-width:2px classDef yellow fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:2px classDef green fill:#d1fae5,stroke:#059669,color:#064e3b,stroke-width:2px classDef red fill:#fee2e2,stroke:#dc2626,color:#7f1d1d,stroke-width:2px classDef purple fill:#ede9fe,stroke:#7c3aed,color:#4c1d95,stroke-width:2px classDef gray fill:#f3f4f6,stroke:#6b7280,color:#111827,stroke-width:2px linkStyle default stroke:#94a3b8,stroke-width:2px style BOX fill:transparent,stroke:#6b7280,stroke-width:2px
Walking through the code. The # Setup: lines only create the sample log, so skip past them.
- The first awk builds three arrays as it reads, then in
ENDprintsaverage path count maxfor each path. sort -k1,1nrsorts by the first field as a number, biggest first.- The last awk reorders and pads the fields for reading.
/api/pay is slowest at 1100.0 ms, because one of its two requests took 1.4 seconds.
Edge cases. A path with one request has an average equal to its maximum. A line with a missing time adds 0 and still counts, which pulls the average down, so skip such lines with a test like NF == 4. Times in the sample are seconds; check your log format, because some servers log milliseconds or microseconds.
# Setup: create the sample request log in a fresh temporary folder
cd "$(mktemp -d)"
cat > requests.log << 'LOG'
GET /api/cart 200 0.120
GET /api/pay 200 0.800
GET /health 200 0.002
GET /api/cart 200 0.180
GET /api/pay 500 1.400
GET /api/cart 200 0.150
GET /health 200 0.004
GET /api/search 200 0.300
LOG
awk '
{
t = $4 * 1000 # seconds to milliseconds
s[$2] += t # total time per path
n[$2]++ # requests per path
if (t > mx[$2]) mx[$2] = t # slowest request per path
}
END {
for (p in s) printf "%.1f %s %d %.0f\n", s[p] / n[p], p, n[p], mx[p]
}
' requests.log |
sort -k1,1nr |
awk '{ printf "%-12s avg %7.1f ms n=%d max %.0f ms\n", $2, $1, $3, $4 }'RecapThe whole problem in a few lines, for the night before
- Spot it: "per X, show the average / total / max of Y"
- Idea: awk arrays keyed by X:
s[$2] += t; n[$2]++, then inENDprints[k] / n[k] - Cost: one pass, O(n) time, memory for one entry per distinct key
- Trap: expecting
for (k in a)to come out in order; always sort the result
Interview follow-ups
Report the p95 time per endpoint instead of the max.
Sort all lines by path and then by time, so each path's times are in order:
sort -k2,2 -k4,4n requests.log. Then let awk collect each path's times into a list and, when the path changes, pick the value at positionceil(0.95 * count). For large logs this is the job of a metrics system, but the sort-then-pick method is a good interview answer because it shows you know why the average is not enough.
Frequently asked questions
An average hides the slow requests that users actually notice. Nine requests at 100 ms and one at 5 seconds average about 600 ms, which looks fine, but one user in ten waited 5 seconds. That is why monitoring uses percentiles: p95 is the time that 95% of requests beat. Reporting the max, as this page does, is a cheap first warning; a p95 needs the sorted list of times per endpoint.
awk arrays are hash tables, so for (k in a) visits keys in an order that depends on the awk version and on the keys themselves. Never rely on it. Print the value you want to sort by as the first field and pipe into sort, as here. In gawk you can also set PROCINFO["sorted_in"], but that does not work in mawk, the default awk on Ubuntu.
Join them into one key: s[$2 " " $3] += $4. Each combination, like /api/pay 500, gets its own box. When you print, the key already contains both parts with a space, so it splits back into two columns for sort or cut. awk also has a built-in separator for this, s[$2, $3], but a plain space is easier to read in the output.