DevOps NetworkDevOpsNetwork
Join Free
Practical infra coding

Count errors per minute, including quiet minutes

easyLog parsing and aggregation

Problem statement

An application writes one line per event: date, time, level, component and message. For a dashboard you need the number of ERROR lines in each minute, from the first minute in the log to the last.

A minute with no errors must still appear with a count of 0. Otherwise a graph built from your output draws a straight line across the gap and hides the quiet period. Lines whose timestamp cannot be parsed are skipped and reported on stderr.

app.log

TEXT
2026-09-20 10:00:05 INFO api request served
2026-09-20 10:00:17 ERROR api upstream timeout
2026-09-20 10:00:41 ERROR api upstream timeout
2026-09-20 10:01:02 INFO api request served
2026-09-20 10:02:10 ERROR db connection reset
2026-09-20 10:02:11 ERROR db connection reset
2026-09-20 10:02:59 ERROR api upstream timeout
2026-09-20 10:03:30 WARN api retry budget low

Print one line per minute: YYYY-MM-DD HH:MM count.

Examples

Example 1

Input: python solution.py app.log

Output: 2026-09-20 10:00 2 2026-09-20 10:01 0 2026-09-20 10:02 3 2026-09-20 10:03 0

Explanation: 10:01 and 10:03 have log lines but no errors, so they are printed with 0. The range runs from the first minute in the log to the last, not from the first error to the last.

Hints

Approach

Optimal

Make one pass over the lines, then one pass over the minute range.

  1. Split each line into at most four parts: date, time, level, rest. If the timestamp does not parse, count the line as skipped.
  2. Truncate the timestamp to the minute. Update first and last with min/max instead of assuming the file is sorted; multi-threaded loggers often write lines slightly out of order.
  3. If the level is ERROR, increment counts[minute].
  4. Walk from first to last one minute at a time and emit (minute, counts[minute]).

n is the number of lines and m the number of minutes covered. Memory only depends on the time span, so a day of logs needs at most 1,440 counters however many lines it has.

ComplexityTime O(n + m)Space O(m)
Python
import sys
from collections import Counter
from datetime import datetime, timedelta
def errors_per_minute(lines):
counts = Counter()
first = last = None
skipped = 0
for line in lines:
parts = line.split(maxsplit=3)
if not parts:
continue # blank line
try:
ts = datetime.strptime(f"{parts[0]} {parts[1]}", "%Y-%m-%d %H:%M:%S")
level = parts[2]
except (IndexError, ValueError):
skipped += 1
continue
minute = ts.replace(second=0)
# min/max rather than "first line wins": lines can arrive slightly out of order
first = minute if first is None else min(first, minute)
last = minute if last is None else max(last, minute)
if level == "ERROR":
counts[minute] += 1
rows = []
minute = first
while minute is not None and minute <= last:
rows.append((minute, counts[minute])) # Counter returns 0 for quiet minutes
minute += timedelta(minutes=1)
return rows, skipped
def main(path):
with open(path, encoding="utf-8", errors="replace") as f:
rows, skipped = errors_per_minute(f)
for minute, n in rows:
print(f"{minute:%Y-%m-%d %H:%M} {n}")
if skipped:
print(f"skipped {skipped} unparseable line(s)", file=sys.stderr)
if __name__ == "__main__":
main(sys.argv[1] if len(sys.argv) > 1 else "app.log")

Follow-up questions

  • Group by component as well, so you get errors per minute per component.
  • The log spans a daylight-saving change in local time. What goes wrong, and why should logs be written in UTC?
  • Make it streaming: print each minute as soon as it is complete instead of at the end. What do you do about late, out-of-order lines?

Frequently asked questions

Slicing works for well-formed lines and is faster. But 25:61 would pass as a valid minute, and you could not step to the next minute to fill gaps. Parsing into a datetime validates the timestamp and gives you arithmetic. For very large files you can slice first and parse only the distinct minute strings.

Turning raw logs into a time series is the first step of most incident timelines and of every log-based alert. The zero-filling detail shows whether you think about how the output will be used, not only whether the counts are right.