Advanced Bash Scripting
sed
Introduction
This guide explores sed, the powerful stream editor used for text processing and pattern-based transformations on the command line. You’ll learn how sed combines pattern matching with built-in editing capabilities—making it a versatile tool for everything from simple substitutions to complex in-place file edits.
Comparing grep, awk, and sed
Before diving into sed, it helps to see how it relates to other common utilities:
# List disk usage
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 7.7G 2.9G 4.9G 38% /
devtmpfs 486M 0 486M 0% /dev
...
# grep for "root"
$ df -h | grep "root"
/dev/root 7.7G 2.9G 4.9G 38% /
# Print second line with awk
$ df -h | awk 'NR == 2 { print }'
/dev/root 7.7G 2.9G 4.9G 38% /
Tool | Strengths | Limitations | Example |
---|---|---|---|
grep | Fast plain-text pattern matching | No built-in substitution | grep "root" disk-usage.txt |
awk | Column/record data processing | Complex syntax for edits | awk 'NR==2' disk-usage.txt |
sed | Stream editing, inline substitutions | Regex-based commands | sed 's/foo/bar/g' file.txt |
sed Basics: Substitution Command
The core sed substitution syntax is:
sed 's/<pattern>/<replacement>/<flags>' [file]
- s: substitute command
- pattern: a regular expression to match
- replacement: text to replace each match
- flags: modifiers (e.g.,
g
for global replacement on each line)
Example: Replace every sample
with are
in poem.txt:
$ sed 's/sample/are/g' poem.txt
Roses are red,
Violets are blue,
Sugar are sweet,
And so are you.
Breakdown:
s
: substitution/sample/
: text to find/are/
: replacement textg
: apply to all matches on each line
In this tutorial, we’ll start with basic operations such as printing specific lines, then move on to advanced techniques like in-place editing and generating custom output.
GNU vs. BSD sed
sed comes in two main flavors:
- GNU sed: Default on most Linux distributions, supports extended features
- BSD sed: Bundled with macOS, lacks some GNU-specific options
Most basic sed commands work across both implementations, but when you need advanced flags (e.g., -r
for extended regex), GNU sed is the recommended choice.
Note
On macOS, install GNU sed via Homebrew:
brew install gnu-sed
Then use gsed
instead of sed
for full compatibility.
What's Next
- Printing and deleting lines
- Addressing ranges and patterns
- In-place (
-i
) file edits - Advanced scripting with semicolons and labels
For more details, see the GNU sed Manual.
Watch Video
Watch video content