sed is a useful command-line program with a horrible syntax. It is a stream editor. This means it operates on text flowing through it. Words are poured into one end, sed does its thing and the edited text spills out the other.

It’s most useful inside shell scripts when text needs to be changed, but there’s no file to open. The stream of text can be from the output of other programs.

Before going to far, it would be useful to show some syntax examples. Please sit down, Regular Expressions are involved.

sed 's/ /_/g' -e 's/,/ #/g'`
sed 's/\[tags \(.*\)]/\1/'
sed '1,3 d'

That’s not gibberish. It’s sed commands. I said the syntax was horrible. Learning sed is like learning how to drive a manual transmission car. It might not be used everyday, but it could come in handy.

Helper programs

The most important thing to know about sed is that it doesn’t care about files or folders. It only operates on text fed to its standard input (stdin). This means the text must be supplied by a program that writes text to its standard output (stdout) then send to sed. The usual helper programs are echo and cat.

Echo

Echo simply copies its stdin to stdout. The input can be a string literal, or text stored in a variable.

echo "This is text, which I typed."
myvar="This is more text, being stored in a variable."
echo "$myvar"

The quotes are important, as they keep all the text together.

Cat

Cat takes the contents of a file and writes it to stdout. The name comes from “concatenate” because it accepts multiple files and joins them together as one output stream.

cat text.txt
cat one.txt two.txt three.txt

Once we have a stream of text flowing from either echo or cat, the pipe command | will send it to sed.

Substitution

A simple sed command is to replace a lower case letter with it’s uppercase version. The basic syntax for a substitution (find and replace) is to use the s operator. The text to find is between the first and second slashes, and the replacement is between the second and third.

echo "this is sparta." | sed 's/s/S/'
thiS is sparta.

Notice that only the first ’s’ was replaced. Sed is a bit conservative, and needs to be told to replace all. This is done by adding the g operator to the end of the expression.

echo "this is sparta." | sed 's/s/S/g'
thiS iS Sparta.

For basic substitutions, this is all that’s needed. Regular expressions (RE) come in when patterns need to be matched.

To test this on a file, create a file named sed.txt and save it to the desktop. Cat can help.

cat > ~/Desktop/sed.txt  # press enter, and type:
one
two
three
four
potato

Then press control-D to tell cat the input is finished and to save the file.

To change every ‘o’ to ‘#’ try this:

cat ~/Desktop/sed.txt | sed 's/o/#/g'

The results should look like this:

#ne
tw#
three
f#ur
p#tat#

Use cat to read the file without sed, and see no changes were made:

cat ~/Desktop/sed.txt
one
two
three
four
potato

Sed operated on the output of cat, not the original file. The output from sed can be save to a file using the redirect operators, > and >>. The stream can also be piped to other programs with |.

I’m going to end part one here. I’ve shown how sed gets input, and the basic substitution syntax. In part two, I plan to introduce the line operators.

Further reading: Wikipedia and the sed manual page.