Advanced Bash Scripting

Streams

Pipes

In Bash shell scripting, pipes allow you to direct the output of one command into another, enabling powerful workflows without intermediate files. This guide covers two primary pipe types:

Pipe TypeSymbol/CommandDescription
Named Pipemkfifo / FIFOCreates a persistent file endpoint for inter-process communication.
Anonymous Pipe``

Named Pipes (FIFOs)

Note

Named pipes, also known as FIFOs, are special files you create with mkfifo. They let processes communicate through a named file path. Here, < and > redirect standard input and output to or from file resources, sometimes referred to as file-based piping.

Below is an example that uses both input and output redirection:

Suppose abc.txt contains five unsorted letters. You can sort them and save the results:

$ sort < abc.txt > abc_sorted.txt
$ cat abc_sorted.txt
a
b
c
d
e
  • < abc.txt feeds the contents of abc.txt into sort.
  • > abc_sorted.txt writes the sorted output to abc_sorted.txt.
  • cat abc_sorted.txt displays the sorted list.

Anonymous Pipes

Anonymous pipes use the | symbol to pass commands’ output directly as input to the next command, without creating files.

The image is a diagram explaining "Anonymous Pipe," which helps pass output from one place to another.

The image shows a dark interface with the title "Pipes" at the top. It features a checkmark labeled "Pipe Input" and a vertical line symbol, possibly representing a pipe character.

Think of a pipeline like an assembly line:

  1. The first command produces data.
  2. Each subsequent command processes the incoming data and passes it along.
  3. The final output appears in your terminal.

The image illustrates the concept of pipes in computing, showing how the output of one process (command1) is used as input for another process (command2), ultimately displaying the result on a screen.

Example: Filtering and Sorting a List

Given animals.txt containing unsorted names (with duplicates):

$ cat animals.txt
Frog
Cheetah
Elephant
Giraffe
Antelope
Bear
Deer
Iguana
Jaguar
Hippopotamus
Ostrich
Frog
Cheetah
Elephant
Giraffe
Antelope
Bear
Deer
Iguana
Jaguar
Hippopotamus

Step 1: Filter names containing “a” (case-insensitive):

$ cat animals.txt | grep -i "a"
Cheetah
Elephant
Giraffe
Antelope
Bear
Iguana
Jaguar
Hippopotamus
Cheetah
Elephant
Giraffe
Antelope
Bear
Iguana
Jaguar
Hippopotamus

Step 2: Further filter for “o” and sort alphabetically:

$ cat animals.txt | grep -i "o" | sort
Antelope
Antelope
Frog
Frog
Hippopotamus
Hippopotamus
Ostrich
  • cat animals.txt reads the list.
  • grep -i "o" selects lines with “o” (case-insensitive).
  • sort orders the filtered names.

You can extend pipelines by appending more commands after each |, letting each stage transform the data in sequence.

Handling Errors in Pipelines

By default, pipelines only pass standard output (stdout) to the next command. Errors (stderr) are displayed directly on the terminal, and the pipeline continues.

$ ls -z | echo "Hello world"
Hello world
ls: unknown option -- z
Try 'ls --help' for more information

Warning

Errors are not piped between commands. Use redirection like 2>&1 if you need to capture stderr in your pipeline.

Watch Video

Watch video content

Previous
Heredocs