Advanced Bash Scripting
Streams
Pipes
In Bash shell scripting, pipes allow you to direct the output of one command into another, enabling powerful workflows without intermediate files. This guide covers two primary pipe types:
Pipe Type | Symbol/Command | Description |
---|---|---|
Named Pipe | mkfifo / FIFO | Creates a persistent file endpoint for inter-process communication. |
Anonymous Pipe | ` | ` |
Named Pipes (FIFOs)
Note
Named pipes, also known as FIFOs, are special files you create with mkfifo
. They let processes communicate through a named file path. Here, <
and >
redirect standard input and output to or from file resources, sometimes referred to as file-based piping.
Below is an example that uses both input and output redirection:
Suppose abc.txt
contains five unsorted letters. You can sort them and save the results:
$ sort < abc.txt > abc_sorted.txt
$ cat abc_sorted.txt
a
b
c
d
e
< abc.txt
feeds the contents ofabc.txt
intosort
.> abc_sorted.txt
writes the sorted output toabc_sorted.txt
.cat abc_sorted.txt
displays the sorted list.
Anonymous Pipes
Anonymous pipes use the |
symbol to pass commands’ output directly as input to the next command, without creating files.
Think of a pipeline like an assembly line:
- The first command produces data.
- Each subsequent command processes the incoming data and passes it along.
- The final output appears in your terminal.
Example: Filtering and Sorting a List
Given animals.txt
containing unsorted names (with duplicates):
$ cat animals.txt
Frog
Cheetah
Elephant
Giraffe
Antelope
Bear
Deer
Iguana
Jaguar
Hippopotamus
Ostrich
Frog
Cheetah
Elephant
Giraffe
Antelope
Bear
Deer
Iguana
Jaguar
Hippopotamus
Step 1: Filter names containing “a” (case-insensitive):
$ cat animals.txt | grep -i "a"
Cheetah
Elephant
Giraffe
Antelope
Bear
Iguana
Jaguar
Hippopotamus
Cheetah
Elephant
Giraffe
Antelope
Bear
Iguana
Jaguar
Hippopotamus
Step 2: Further filter for “o” and sort alphabetically:
$ cat animals.txt | grep -i "o" | sort
Antelope
Antelope
Frog
Frog
Hippopotamus
Hippopotamus
Ostrich
cat animals.txt
reads the list.grep -i "o"
selects lines with “o” (case-insensitive).sort
orders the filtered names.
You can extend pipelines by appending more commands after each |
, letting each stage transform the data in sequence.
Handling Errors in Pipelines
By default, pipelines only pass standard output (stdout) to the next command. Errors (stderr) are displayed directly on the terminal, and the pipeline continues.
$ ls -z | echo "Hello world"
Hello world
ls: unknown option -- z
Try 'ls --help' for more information
Warning
Errors are not piped between commands. Use redirection like 2>&1
if you need to capture stderr in your pipeline.
Links and References
- Bash Reference Manual: Pipelines and I/O Redirection
- Linux mkfifo Command
- GNU grep Documentation
- Sorting Text Files with sort
Watch Video
Watch video content