DevOps NetworkDevOpsNetwork
Join Free
Bash and Linux

Redirect Output and Errors

easyShell basics and files Must-do

Problem statement

Send a command's normal output and its error messages where you want them: into separate files, into one file, into nothing, or to the screen and a file at the same time. You need this in every script and cron job, so that errors are logged, noise is hidden, and results can be saved.

The script defines a fake health check called check_hosts. It writes two good lines to normal output and one error line to the error stream:

check_hosts

TEXT
ok: web-1 (normal output)
error: db-1 is down (error output)
ok: web-2 (normal output)

Do these steps in order:

  1. Save the good lines to ok.txt and the error to errors.txt, then print both files.
  2. Save everything into one file, all.txt, then print it.
  3. Add the line ok: web-3 to the end of ok.txt without wiping it, then print it.
  4. Throw the errors away and count only the good lines.
  5. Show the good lines on screen and save them to copy.txt at the same time.
  6. Show the common order mistake: write the redirects in the wrong order and see where the error goes.

Expected output:

TEXT
== ok.txt ==
ok: web-1
ok: web-2
== errors.txt ==
error: db-1 is down
== all.txt ==
ok: web-1
error: db-1 is down
ok: web-2
== ok.txt after >> ==
ok: web-1
ok: web-2
ok: web-3
== good lines ==
2
== tee ==
ok: web-1
ok: web-2
copy.txt has 2 lines
== order trap (this error line was meant for the file) ==
error: db-1 is down
== trap.txt ==
ok: web-1
ok: web-2

Hints

Hint 1: Every command has two output streams: number 1 for normal output and number 2 for errors. > redirects 1, and 2> redirects 2.

Approach

Optimal: Streams and redirects

Covers: >, >>, <, 2>, 2>&1, &>, /dev/null, |, tee, >&2, stdin, stdout, stderr.

Every command has three streams. Think of them as three pipes attached to the program. Each one has a number, called a file descriptor:

Number Name Default
0 stdin (standard input) your keyboard
1 stdout (standard output) your screen
2 stderr (standard error) your screen

Normal results go to stdout. Error messages go to stderr. Both show on your screen by default, so they look the same, but they are separate. That separation is what lets you save results and errors to different places.

%%{init: {"flowchart": {"padding": 18, "nodeSpacing": 30, "rankSpacing": 40, "htmlLabels": true}, "themeVariables": {"fontSize": "18px"}}}%% flowchart TB IN["stdin 0
keyboard"]:::gray --> P{{"check_hosts"}}:::blue P --> O(["stdout 1"]):::green P --> E(["stderr 2"]):::red O --> F1[("ok.txt")]:::green E --> F2[("errors.txt")]:::red classDef blue fill:#dbeafe,stroke:#2563eb,color:#1e3a8a,stroke-width:2px classDef yellow fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:2px classDef green fill:#d1fae5,stroke:#059669,color:#064e3b,stroke-width:2px classDef red fill:#fee2e2,stroke:#dc2626,color:#7f1d1d,stroke-width:2px classDef purple fill:#ede9fe,stroke:#7c3aed,color:#4c1d95,stroke-width:2px classDef gray fill:#f3f4f6,stroke:#6b7280,color:#111827,stroke-width:2px linkStyle default stroke:#94a3b8,stroke-width:2px

The redirects you will use most:

You write It does
cmd > file stdout into file, replacing what was there
cmd >> file stdout added to the end of file
cmd 2> file stderr into file
cmd > file 2>&1 both streams into file
cmd &> file the same, shorter (bash only)
cmd 2> /dev/null throw errors away
cmd < file read stdin from file
cmd1 | cmd2 stdout of cmd1 becomes stdin of cmd2
cmd | tee file show stdout on screen and save it to file

/dev/null is a special file that swallows everything written to it. It is the bin for output you do not want.

Why the order matters. The shell reads redirects from left to right, and 2>&1 copies wherever stream 1 points at that moment.

  • cmd > file 2>&1: first stream 1 goes to file, then stream 2 copies stream 1, so it also goes to file. Both end up in the file.
  • cmd 2>&1 > file: first stream 2 copies stream 1, which is still the screen. Then stream 1 moves to file. Errors stay on the screen.
%%{init: {"flowchart": {"padding": 18, "nodeSpacing": 30, "rankSpacing": 40, "htmlLabels": true}, "themeVariables": {"fontSize": "18px"}}}%% flowchart LR subgraph RIGHT["Right: > file 2>&1"] direction TB R1["1 goes to the file"]:::green --> R2["2 copies 1
also the file"]:::green end subgraph WRONG["Wrong: 2>&1 > file"] direction TB W1["2 copies 1
still the screen"]:::red --> W2["then 1 goes
to the file"]:::red end RIGHT ~~~ WRONG classDef blue fill:#dbeafe,stroke:#2563eb,color:#1e3a8a,stroke-width:2px classDef yellow fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:2px classDef green fill:#d1fae5,stroke:#059669,color:#064e3b,stroke-width:2px classDef red fill:#fee2e2,stroke:#dc2626,color:#7f1d1d,stroke-width:2px classDef purple fill:#ede9fe,stroke:#7c3aed,color:#4c1d95,stroke-width:2px classDef gray fill:#f3f4f6,stroke:#6b7280,color:#111827,stroke-width:2px linkStyle default stroke:#94a3b8,stroke-width:2px style RIGHT fill:transparent,stroke:#059669,stroke-width:2px style WRONG fill:transparent,stroke:#dc2626,stroke-width:2px

Step 6 of the code shows this mistake on purpose: the error line appears in the output instead of in trap.txt.

Pipes carry only stdout. In cmd | wc -l, only stdout goes into wc. Errors skip the pipe and go straight to the screen. That is why step 4 uses 2>/dev/null: it removes the error so only good lines are counted. To send both into a pipe, write cmd 2>&1 | wc -l.

Walking through the code. The # Setup: lines only create the fake check, so skip past them. Inside it, >&2 sends the error line to stream 2, which is how your own scripts should print errors too.

  1. > ok.txt 2> errors.txt splits the two streams into two files.
  2. > all.txt 2>&1 puts both into one file, in the order they were written.
  3. >> appends instead of replacing.
  4. 2>/dev/null | wc -l drops the error and counts the 2 good lines.
  5. | tee copy.txt prints the lines and writes them to a file at the same time.
  6. 2>&1 > trap.txt is in the wrong order, so the error line goes to the screen.

Edge cases. > empties the file before the command even starts, so sort file > file leaves you with an empty file. Write to a new file and rename it instead. If a command prints nothing, > still creates an empty file.

# Setup: a fake check that writes good news to stdout and bad news to stderr
cd "$(mktemp -d)"
check_hosts() {
  echo "ok: web-1"
  echo "error: db-1 is down" >&2      # >&2 sends this line to stderr
  echo "ok: web-2"
}

# 1. Split the two streams into two files
check_hosts > ok.txt 2> errors.txt
echo "== ok.txt =="
cat ok.txt
echo "== errors.txt =="
cat errors.txt

# 2. Put both streams into one file
check_hosts > all.txt 2>&1
echo "== all.txt =="
cat all.txt

# 3. Add a line to the end instead of replacing the file
echo "ok: web-3" >> ok.txt
echo "== ok.txt after >> =="
cat ok.txt

# 4. Throw errors away and count only the good lines
echo "== good lines =="
check_hosts 2>/dev/null | wc -l

# 5. See the output AND save it with tee
echo "== tee =="
check_hosts 2>/dev/null | tee copy.txt
echo "copy.txt has $(wc -l < copy.txt) lines"

# 6. The order trap: 2>&1 BEFORE > file sends errors to the screen, not the file
echo "== order trap (this error line was meant for the file) =="
check_hosts 2>&1 > trap.txt
echo "== trap.txt =="
cat trap.txt
RecapThe whole problem in a few lines, for the night before
  • Spot it: "save the output", "log the errors", "hide the noise", "count only the results"
  • Idea: stream 1 is stdout, stream 2 is stderr; > replaces, >> appends, 2>&1 joins them, /dev/null throws away
  • Cost: no extra cost, redirects only change where bytes go
  • Trap: 2>&1 > file leaves errors on the screen; write > file 2>&1

Interview follow-ups

  • How do you save errors to a log and still see them on screen?

    Send stderr through tee. A plain pipe only carries stdout, so use bash's process substitution: cmd 2> >(tee errors.log >&2). That sends stream 2 into tee, which writes errors.log and prints the lines back to stderr. A simpler form for both streams is cmd 2>&1 | tee all.log, which shows and saves everything together. Add -a to tee to append instead of replacing the log.

Frequently asked questions

The shell sets up redirects before the command runs. > data.txt empties the file first, and only then does sort start, so it reads an empty file and writes nothing. The same happens with any command that reads and writes the same file. Write to a new file and then rename it: sort data.txt > data.tmp && mv data.tmp data.txt. Some commands have their own safe option, like sort -o data.txt data.txt or sed -i.

sudo only runs echo as root. The redirect > is done by your own shell, which is not root, before sudo even starts. So your shell is the one trying to open /etc/file, and it is refused. Use echo text | sudo tee /etc/file instead: tee runs as root and does the writing. Add > /dev/null after it if you do not want to see the text on screen, and use tee -a to append.

Send them to stderr with >&2, like echo "config file missing" >&2. Then a user who runs myscript > result.txt still sees the error on screen instead of finding it buried in the result file. It also means a pipe like myscript | sort sorts only real results. Pair this with a non-zero exit code, for example exit 1, so other scripts can tell it failed.