Linux · text processing

sed

find and replace, without an editor

sed — stream editor — applies an editing command to every line of its input. Its substitute command, s/old/new/, is find-and-replace from the command line, and with -i it changes the file itself.

s/old/new/g — substitute, the pattern, the replacement, and g for every occurrence on the line.

Replace

s/old/new/g

g means all on the line

By default

Prints, changes nothing

the file is untouched

Edit the file

-i

in place

Delete lines

/pattern/d

or by number: 5d

At the terminal

Read the session first, then the notes: each numbered line in the output is answered by the note with the same number.

user@netflow — bashsed

$ sed 's/http:/https:/g' links.txt

1https://example.com/a

https://example.com/b

2$ sed -n '2p' links.txt

http://example.com/b

3$ sed -i.bak 's/^#Port 22/Port 2222/' sshd_config

4$ sed '/^#/d;/^$/d' sshd_config | head -2

Port 2222

PermitRootLogin no

Illustrative session · names, sizes and dates are examples, and every address is from a documentation or private range

Reading the output

What the numbered lines are telling you

Every match replaced, in the output

The result is printed. links.txt on disk still says http: — nothing was written.

Printing one line

-n stops sed printing everything, and 2p prints line 2 only. It shows the file is unchanged.

-i edits in place, .bak keeps a copy

The file is rewritten, and the original is saved as sshd_config.bak. ^ anchors the pattern to the start of the line.

Deleting lines

Two commands separated by a semicolon: delete lines starting with #, and delete empty lines. What is left is the active configuration.

Switches

A switch changes how the command behaves. These are the ones used day to day; man sed lists the rest. Single-letter switches can usually be joined, so -l -a may be written -la.

-i: Edit the file in place. -i.bak also keeps a backup.
-n: Do not print each line automatically; print only what p asks for.
-e SCRIPT: Add another editing command.
-E: Use extended regular expressions.
-f FILE: Read the editing commands from a file.

Variations worth knowing

The same tool, pointed at a narrower question. These four cover most of what it is reached for day to day.

sed -n '10,20p' file.txt

Prints lines 10 to 20 only.

sed 's/ *$//' file.txt

Removes trailing spaces from every line.

sed -i 's/\r$//' script.sh

Removes Windows line endings from a script.

sed 's|/var/www|/srv/www|g' config

Uses | as the delimiter, so paths need no escaping.

In cybersecurity

sed -i rewrites a file with no confirmation and no undo.

Always keep a backup suffix

-i.bak costs nothing and saves the original. Run the command without -i first to see the result.

A pattern can match more than you meant

A dot matches any character and unanchored text matches anywhere in a line. Anchor with ^ and $, and escape the dots.

Changing configuration in bulk

sed is how hardening scripts flip settings across many files. Check the result with grep afterwards — a pattern that matched nothing fails silently.

Filed under

Text processing

Read alongside

awkProcesses text column by column: prints, filters and totals fields from each line.grepSearches files for lines that match a pattern, and prints the lines it finds.