DispersedNetUNIX Shell Programming

Lesson 1

Using sed in UNIX Shell Scripts

Objective: Recognize when to use sed for repeatable text transformations.

The sed stream editor applies editing instructions to text without an interactive editing session. It is useful when a transformation must be repeatable: select records, remove unwanted lines, replace text, or filter another command’s output. The same script can run on a small practice file and later on a larger input stream.

What you will learn

  • Write a quoted sed script and supply an input file.
  • Select lines by number, range, or regular expression.
  • Use d, p, and s to delete, print, and substitute.
  • Reuse matched text with & and run multiple instructions in order.
  • Combine sed with other commands in a pipeline.

Use sed for simple, line-oriented, one-pass text rewrites that are scriptable and repeatable.
Recognize these signals:

  • Pattern substitution (s/old/new/g), deletion (/pattern/d), or insertion on whole lines or streams
  • Editing files in place (-i) or filtering pipelines without loading everything into memory
  • Batch/repeatable transforms where the unit is the line, not fields/columns
  • No need for arithmetic, state across lines, or structured records
Prefer something else when:
  • Working with columns/fields, sums, or logic → awk
  • Pure searching → grep
  • Complex multi-line or programmatic transforms → Perl/Python
Classic one-liner form: `sed 's/pattern/replacement/g' file` or in a pipeline.

Start with output, not an overwritten file

sed 's/foo/bar/g' notes.txt

This prints transformed text to standard output. It replaces every non-overlapping occurrence of the pattern foo on each line, including occurrences inside longer words. It does not require a whole-word match. The examples in this module leave the input unchanged unless a separate output or replacement step is explicitly shown.

Some implementations offer an in-place option, usually -i. Its syntax and backup behavior differ across GNU and BSD implementations. We use portable basic regular expressions and explicit output files instead. The examples have been checked with GNU sed 4.9; consult your system manual for implementation-specific options.

A shared practice file

Lesson 3 introduces petchip, a small historical teaching dataset. Its old dates and names are sample values, not current customer records. Work in a directory you own and keep an untouched copy so each lesson starts from the same input. You need a shell and sed, not root privileges.

Use sed for line-oriented transformations. Use an interactive editor for composing prose, a structured parser for JSON or HTML, and awk when the main task is field-based calculations or reporting. Choosing the tool that matches the data makes the result easier to verify.