Shell Redirection and Pipes Cheat Sheet

shell_redirection_and_pipes_cheat_sheet.md

Shell Redirection and Pipes Cheat Sheet

This cheat sheet covers:

>
>>
2>
2>&1
|
tee
<(process substitution)
>(process substitution)

These work in shells like bash and zsh.

---

1. Standard output and standard error

Most shell commands have three default streams:

Stream	Number	Meaning
stdin	0	Input to a command
stdout	1	Normal output
stderr	2	Error output

Example:

ls file-that-exists missing-file

This may produce:

file-that-exists
ls: cannot access 'missing-file': No such file or directory

The filename goes to stdout.
The error message goes to stderr.

---

2. Redirect stdout with >

> sends normal output to a file.

ls > files.txt

How it works:

• ls produces normal output.
• > redirects stdout into files.txt.
• Nothing appears in the terminal.
• If files.txt already exists, it is overwritten.

Example:

echo "hello" > message.txt

This writes:

hello

to message.txt.

---

Overwrite warning

echo "new content" > notes.txt

If notes.txt already contained something, it is replaced.

Before:

old content

After:

new content

---

Redirect command output to a log file

date > run.log

How it works:

• date prints the current date and time.
• > writes that output to run.log.

---

3. Append stdout with >>

>> appends normal output to a file.

echo "another line" >> notes.txt

How it works:

• echo produces normal output.
• >> adds it to the end of notes.txt.
• Existing content is preserved.

Example:

echo "first" > log.txt
echo "second" >> log.txt
echo "third" >> log.txt

Now log.txt contains:

first
second
third

---

Append command output to a log

date >> run.log

How it works:

• Adds the current date to the end of run.log.
• Does not erase previous log entries.

Useful in scripts:

echo "Backup started" >> backup.log
tar -czf backup.tar.gz project/ >> backup.log
echo "Backup finished" >> backup.log

This records normal output from the script in backup.log.

---

4. Redirect stderr with 2>

2> redirects error output.

ls missing-file 2> error.log

How it works:

• ls missing-file produces an error.
• 2> redirects stderr to error.log.
• The error does not appear in the terminal.

Example output in error.log:

ls: cannot access 'missing-file': No such file or directory

---

stdout still appears normally

ls existing-file missing-file 2> error.log

How it works:

• Normal output for existing-file still appears in the terminal.
• Error output for missing-file goes to error.log.

So you may see:

existing-file

And error.log gets:

ls: cannot access 'missing-file': No such file or directory

---

Append stderr with 2>>

ls missing-file 2>> error.log

How it works:

• Appends error output to error.log.
• Does not overwrite the file.

Useful for long-running scripts:

./deploy.sh 2>> deploy-errors.log

This saves only errors from deploy.sh.

---

5. Redirect stdout and stderr separately

command > output.log 2> error.log

How it works:

• > sends stdout to output.log.
• 2> sends stderr to error.log.

Example:

ls file1 missing-file > output.log 2> error.log

If file1 exists and missing-file does not:

output.log contains:

file1

error.log contains:

ls: cannot access 'missing-file': No such file or directory

---

6. Redirect stderr to stdout with 2>&1

2>&1 means:

send stderr to wherever stdout is currently going

This is commonly used to combine normal output and errors.

command > combined.log 2>&1

How it works:

• > combined.log sends stdout to combined.log.
• 2>&1 sends stderr to the same place as stdout.
• Both normal output and errors go into combined.log.

Example:

ls file1 missing-file > combined.log 2>&1

combined.log may contain:

file1
ls: cannot access 'missing-file': No such file or directory

---

Order matters

This works:

command > combined.log 2>&1

This is different:

command 2>&1 > combined.log

Why?

In this version:

command 2>&1 > combined.log

The shell does this from left to right:

1. 2>&1 sends stderr to wherever stdout currently goes.
2. At that moment, stdout is still the terminal.
3. > combined.log then sends stdout to the file.
4. stderr still goes to the terminal.

So:

command > combined.log 2>&1

means both stdout and stderr go to the file.

But:

command 2>&1 > combined.log

means stdout goes to the file, stderr stays on the terminal.

---

7. Modern shortcut: &>

In bash, this redirects both stdout and stderr:

command &> combined.log

Equivalent to:

command > combined.log 2>&1

Append both stdout and stderr:

command &>> combined.log

Equivalent to:

command >> combined.log 2>&1

Example:

make &> build.log

How it works:

• Saves normal compiler output.
• Saves compiler errors.
• Terminal stays clean.

---

8. Discard output with /dev/null

/dev/null is a special file that throws away anything written to it.

Discard stdout

command > /dev/null

How it works:

• Normal output is discarded.
• Errors still appear.

Example:

curl https://example.com > /dev/null

The downloaded HTML is discarded.

---

Discard stderr

command 2> /dev/null

How it works:

• Error messages are discarded.
• Normal output still appears.

Example:

find / -name notes.txt 2> /dev/null

How it works:

• find / searches the whole filesystem.
• Some directories may give permission errors.
• 2> /dev/null hides those errors.
• Matching filenames still appear.

---

Discard everything

command > /dev/null 2>&1

or:

command &> /dev/null

How it works:

• stdout is discarded.
• stderr is sent to stdout’s destination.
• Both are thrown away.

Example:

grep -q "hello" file.txt > /dev/null 2>&1

Usually grep -q already suppresses output, but this pattern is common for quiet checks.

---

9. Pipes with |

A pipe sends stdout from one command into stdin of another command.

command1 | command2

How it works:

• command1 produces output.
• | passes that output to command2.
• command2 reads it as input.

Example:

ls | grep ".txt"

How it works:

• ls lists files.
• grep ".txt" filters only lines containing .txt.

---

Count matching files

ls | grep ".txt" | wc -l

How it works:

1. ls lists files.
2. grep ".txt" keeps only lines containing .txt.
3. wc -l counts the remaining lines.

---

Search running processes

ps aux | grep nginx

How it works:

• ps aux lists running processes.
• grep nginx filters lines containing nginx.

Better version:

ps aux | grep "[n]ginx"

Why?

• grep nginx may also match the grep nginx process itself.
• grep "[n]ginx" matches nginx, but the grep command line does not literally contain the string nginx.

---

Pipe into less

dmesg | less

How it works:

• dmesg prints kernel messages.
• less lets you scroll through them.

Useful with long output:

journalctl -xe | less

---

Pipe into sort and uniq

cat names.txt | sort | uniq

How it works:

1. cat names.txt prints the file.
2. sort sorts the lines.
3. uniq removes adjacent duplicate lines.

Better version:

sort names.txt | uniq

Even better:

sort -u names.txt

---

Count repeated lines

sort access.log | uniq -c | sort -nr

How it works:

1. sort access.log groups identical lines together.
2. uniq -c counts each group.
3. sort -nr sorts numerically in reverse order.

---

10. Important pipe detail: only stdout is piped

By default, | pipes stdout, not stderr.

command | grep "text"

How it works:

• stdout from command goes to grep.
• stderr from command still appears on the terminal.

Example:

ls existing-file missing-file | grep file

How it works:

• existing-file goes through the pipe to grep.
• The error for missing-file bypasses the pipe and appears on the terminal.

---

Pipe both stdout and stderr

command 2>&1 | grep "error"

How it works:

• 2>&1 sends stderr to stdout.
• | pipes the combined stream to grep.

Example:

make 2>&1 | grep -i "error"

How it works:

• make produces normal output and errors.
• 2>&1 combines both streams.
• grep -i "error" filters lines containing error, ignoring case.

---

Bash shortcut: |&

In bash and zsh:

command |& grep "error"

Equivalent to:

command 2>&1 | grep "error"

Example:

make |& grep -i "warning"

How it works:

• Sends both stdout and stderr from make into grep.
• Shows only lines containing warning.

---

11. tee

tee writes input to both the terminal and a file.

command | tee file.txt

How it works:

• command produces output.
• tee file.txt displays it on the terminal.
• It also saves it to file.txt.

Example:

ls | tee files.txt

How it works:

• Shows the file list in your terminal.
• Saves the same list to files.txt.

---

tee overwrites by default

echo "hello" | tee message.txt

How it works:

• Prints hello to the terminal.
• Writes hello to message.txt.
• If message.txt exists, it is overwritten.

---

Append with tee -a

echo "another line" | tee -a message.txt

How it works:

• Prints another line to the terminal.
• Appends it to message.txt.
• Existing content is preserved.

---

Save command output while still seeing it

make | tee build.log

How it works:

• You see the build output live.
• A copy is saved to build.log.

But this only captures stdout.

To capture stdout and stderr:

make 2>&1 | tee build.log

or:

make |& tee build.log

How it works:

• Both normal output and errors are combined.
• tee shows them in the terminal.
• tee saves them to build.log.

---

Use tee with sudo

This often fails:

sudo echo "hello" > /etc/example.conf

Why?

• sudo echo "hello" runs echo as root.
• But > is handled by your current shell, not by sudo.
• Your non-root shell tries to write to /etc/example.conf.
• Permission denied.

Correct version:

echo "hello" | sudo tee /etc/example.conf

How it works:

• echo "hello" runs normally.
• The output is piped to sudo tee.
• tee runs as root and writes to /etc/example.conf.

Hide the terminal copy:

echo "hello" | sudo tee /etc/example.conf > /dev/null

How it works:

• tee writes to the protected file.
• > /dev/null discards the copy that tee would print to the terminal.

---

Append to a protected file

echo "new setting" | sudo tee -a /etc/example.conf

How it works:

• -a tells tee to append.
• sudo tee has permission to write to the protected file.

---

12. Redirect input with <

< sends a file into a command’s stdin.

command < file.txt

Example:

wc -l < file.txt

How it works:

• file.txt is used as input to wc -l.
• wc -l counts lines.
• Because the file is redirected rather than passed as an argument, the output is just the number.

Compare:

wc -l file.txt

Output:

12 file.txt

With redirection:

wc -l < file.txt

Output:

12

---

Read from a file in a loop

while read line; do
  echo "Line: $line"
done < names.txt

How it works:

• names.txt is redirected into the while loop.
• read line reads one line at a time.
• The loop prints each line with a prefix.

Safer version:

while IFS= read -r line; do
  echo "Line: $line"
done < names.txt

Why?

• IFS= preserves leading and trailing whitespace.
• -r prevents backslash escapes from being interpreted.

---

13. Here strings with <<<

A here string sends a short string into stdin.

command <<< "some input"

Example:

grep "cat" <<< "the cat sat"

How it works:

• The string the cat sat is sent into grep.
• grep "cat" searches it.

Output:

the cat sat

Another example:

wc -w <<< "hello world from shell"

How it works:

• Sends the string into wc -w.
• wc -w counts words.

Output:

4

---

14. Here documents with <<

A here document sends multiple lines into stdin.

command << EOF
line one
line two
EOF

Example:

cat << EOF
hello
world
EOF

How it works:

• Everything between the first EOF and the final EOF is sent to cat.
• cat prints it.

Output:

hello
world

---

Create a file with a here document

cat > config.txt << EOF
host=localhost
port=8080
debug=true
EOF

How it works:

• The here document provides input to cat.
• > config.txt redirects cat output into config.txt.
• The file is created with those three lines.

---

Prevent variable expansion

cat << 'EOF'
The variable is $HOME
EOF

How it works:

• Quoting 'EOF' prevents shell expansion.
• $HOME stays literal.

Output:

The variable is $HOME

Without quotes:

cat << EOF
The variable is $HOME
EOF

The shell expands $HOME first.

Output might be:

The variable is /home/lee

---

15. Process substitution: <(...)

Process substitution lets a command treat another command’s output like a file.

command <(other-command)

This is very useful for commands that expect filenames instead of piped input.

---

Compare output from two commands

diff <(ls dir1) <(ls dir2)

How it works:

• ls dir1 produces a list of files.
• ls dir2 produces another list.
• <(...) makes each output look like a temporary file.
• diff compares those two temporary-looking files.

---

Compare sorted files without creating temp files

diff <(sort file1.txt) <(sort file2.txt)

How it works:

• sort file1.txt produces sorted output.
• sort file2.txt produces sorted output.
• diff compares the sorted results.
• No separate sorted files are created.

---

Use with comm

comm compares two sorted files.

comm -12 <(sort list1.txt) <(sort list2.txt)

How it works:

• sort list1.txt sorts the first list.
• sort list2.txt sorts the second list.
• comm -12 shows only lines common to both.
• Process substitution avoids temporary files.

---

Find files in one list but not another

comm -23 <(sort old.txt) <(sort new.txt)

How it works:

• comm needs sorted input.
• -23 suppresses columns 2 and 3.
• It shows lines only found in old.txt.

---

Use process substitution with grep -f

grep -f <(printf "error\nfatal\npanic\n") app.log

How it works:

• printf creates a list of grep patterns.
• <(...) turns that list into a file-like input.
• grep -f reads patterns from that file-like input.
• app.log is searched for error, fatal, or panic.

---

16. Process substitution: >(...)

Output process substitution sends output into another command as if it were writing to a file.

command > >(other-command)

This is less common than <(...), but powerful.

---

Send output through a command

echo "hello world" > >(tr a-z A-Z)

How it works:

• echo writes output.
• > >(...) redirects that output into tr.
• tr a-z A-Z converts lowercase to uppercase.

Output:

HELLO WORLD

---

Split stdout and stderr into separate processors

command > >(tee stdout.log) 2> >(tee stderr.log >&2)

How it works:

• stdout goes to tee stdout.log.
• stderr goes to tee stderr.log.
• >&2 sends stderr output back to the terminal as stderr.
• Both streams are logged separately while still appearing on screen.

Example:

make > >(tee build-output.log) 2> >(tee build-errors.log >&2)

How it works:

• Normal build output is saved in build-output.log.
• Build errors are saved in build-errors.log.
• You still see both in the terminal.

---

17. Combining pipes and redirection

Pipe filtered output into a file

ps aux | grep nginx > nginx-processes.txt

How it works:

1. ps aux lists processes.
2. grep nginx filters nginx lines.
3. > saves the filtered output to nginx-processes.txt.

---

Save errors from the first command

find / -name "*.conf" 2> errors.log | grep nginx

How it works:

• find / -name "*.conf" searches for config files.
• Permission errors from find go to errors.log.
• Successful results go through the pipe.
• grep nginx filters the successful results.

---

Pipe both output and errors

find / -name "*.conf" 2>&1 | grep denied

How it works:

• find produces normal results and permission errors.
• 2>&1 combines stderr with stdout.
• grep denied filters both streams.

---

Save everything and view errors

make 2>&1 | tee build.log | grep -i error

How it works:

1. make 2>&1 combines stdout and stderr.
2. tee build.log saves the full build output and passes it on.
3. grep -i error displays only lines containing error.

Result:

• build.log contains everything.
• Terminal displays only error lines.

---

18. Common recipes

Save normal output

command > output.txt

Writes stdout to output.txt.

---

Append normal output

command >> output.txt

Adds stdout to the end of output.txt.

---

Save errors only

command 2> errors.txt

Writes stderr to errors.txt.

---

Save stdout and stderr together

command > all.log 2>&1

Writes both stdout and stderr to all.log.

---

Append stdout and stderr together

command >> all.log 2>&1

Appends both stdout and stderr to all.log.

---

Hide normal output but show errors

command > /dev/null

Useful when you only care about failures.

---

Hide errors but show normal output

command 2> /dev/null

Useful when permission errors are noisy.

---

Hide everything

command > /dev/null 2>&1

Runs silently.

---

View and save output

command | tee output.txt

Shows stdout and saves it.

---

View and append output

command | tee -a output.txt

Shows stdout and appends it.

---

View and save stdout plus stderr

command 2>&1 | tee output.txt

Shows and saves both stdout and stderr.

---

Pipe stderr too

command |& grep pattern

Filters both stdout and stderr through grep.

---

Compare two command outputs

diff <(command1) <(command2)

Compares command outputs without temporary files.

---

19. Mental model

Read redirections from left to right.

command > file 2>&1

Means:

1. Send stdout to file.
2. Send stderr to wherever stdout now goes.
3. Result: both go to file.

But:

command 2>&1 > file

Means:

1. Send stderr to where stdout currently goes: the terminal.
2. Send stdout to file.
3. Result: stdout goes to file, stderr goes to terminal.

---

20. Quick reference table

Syntax	Meaning
command > file	Write stdout to file, overwriting
command >> file	Append stdout to file
command 2> file	Write stderr to file, overwriting
command 2>> file	Append stderr to file
command > file 2>&1	Write stdout and stderr to same file
command &> file	Bash shortcut for stdout and stderr to file
command < file	Use file as stdin
command1 | command2	Pipe stdout from command1 into command2
command 2>&1 | command2	Pipe stdout and stderr into command2
command |& command2	Bash shortcut for piping stdout and stderr
command | tee file	Show output and save to file
command | tee -a file	Show output and append to file
<(command)	Treat command output as a file
>(command)	Redirect output into a command
/dev/null	Throw output away

---

21. Best examples to memorize

command > output.txt

Save normal output.

command >> output.txt

Append normal output.

command 2> errors.txt

Save errors.

command > output.txt 2> errors.txt

Save output and errors separately.

command > all.log 2>&1

Save everything.

command 2>&1 | tee all.log

Show everything and save everything.

command | grep pattern

Filter normal output.

command |& grep pattern

Filter normal output and errors.

diff <(sort a.txt) <(sort b.txt)

Compare transformed command outputs without temporary files.