Mastering Stdin, Stdout, and Stderr in Bash

As a Linux power user, understanding standard streams is crucial for harnessing the full power of the shell. In this comprehensive guide, we will dive into bash‘s stdin, stdout, and stderr streams, how they work, and how to leverage them for building robust shell scripts and command-line workflows.

An Introduction to Stdin, Stdout, and Stderr

When a program runs in bash, three default streams are opened:

• Stdin: The standard input stream, file descriptor 0. This is how data can be fed into a program for processing.
• Stdout: The standard output stream, file descriptor 1. A program writes normal results to this stream.
• Stderr: The standard error stream, file descriptor 2. Errors and diagnostics are sent to this stream.

Having separate error and output streams is extremely useful. It allows successful program output to be redirected or piped separately from error messages.

Redirecting Streams

The stdin, stdout, and stderr streams can be gracefully managed in bash by redirecting them to and from files.

For example, to redirect stdout to a file:

$ program_to_run > output.log

To redirect stdin from a file:

$ sort < input.txt

Stderr can also be redirected in a similar fashion:

$ faulty_program 2> errors.log

Let‘s see a real example:

$ find . -name "*.py" > python_files.txt 2> find_errors.log

Here we redirect the successfully found python files to python_files.txt while any errors are written to find_errors.log

Piping Streams Between Programs

We can also pipe the stdout stream directly into another program‘s stdin stream with the pipe | operator:

$ cat files.txt | grep -i error | wc -l

This powerful concept is fundamental to the Linux philosophy of small modular programs working together via streams.

Managing Errors and Output

In some cases, we may wish to throw away all errors and standard output, and just receive an exit code. The /dev/null special file is useful here:

$ program_to_test > /dev/null 2>&1
$ echo $? # Print exit code

We redirect both streams into the void, while still being able to check the final exit code, 0 for success and non-zero on failure.

To keep both output and error streams combined, they can be merged:

$ noisy_program > combined_output.log 2>&1

This merges stderr with stdout for capturing all program text outputs to a single log file.

Stdin, Stdout, Stderr, and Background Processes

When processes are started in the background with &, their stdin stream is redirected from /dev/null. This is to prevent awaiting keyboard input from locking up backgrounded tasks.

Background processes have their stdout and stderr streams redirected to the terminal by default. This allows output from long-running commands to be monitored at the terminal.

In bash, streams and redirection offer an incredibly flexible system for controlling where data flows to and from. Mastering stdin, stdout and stderr is a key skill on the path to Linux scripting mastery.