Using sed to print and replace text

sed is a stream editor that reads and manipulates text using functions you specify in your command. sed understands regular expressions, which makes it a powerful toy.

The basics

sed can be a handy tool if you want to simply print specific lines in a file. For instance, you can use sed to print lines 21 to 32 of a wp-config.php file:

$ sed -n 21,32p wp-config.php 
// ** MySQL settings - You can get this info from your web host ** //
/** The name of the database for WordPress */
define( 'DB_NAME', 'example_db' );

/** MySQL database username */
define( 'DB_USER', 'example_user' );

/** MySQL database password */
define( 'DB_PASSWORD', '?43 Rft>N>)8V@"D7.<I' );

/** MySQL hostname */
define( 'DB_HOST', 'localhost' );

The -n option prevents sed from printing every line in the file and 21,32p prints lines 21 to 32.

Our aim with the above command is to get the database connection information. We therefore really just want the lines that start with “define”. To get that information we could pipe the output to grep. The below grep command prints all lines that start with (^) the string “define”:

$ sed -n 21,32p wp-config.php | grep ^define wp-config.php
define( 'DB_NAME', 'example_db' );
define( 'DB_USER', 'example_user' );
define( 'DB_PASSWORD', '?43 Rft>N>)8V@"D7.<I' );
define( 'DB_HOST', 'localhost' );

You can print the same lines using only sed with multiple -e options. Here, we are printing lines 23, 26, 29 and 32:

$ sed -n -e 23p -e 26p -e 29p -e 32p wp-config.php
define( 'DB_NAME', 'example_db' );
define( 'DB_USER', 'example_user' );
define( 'DB_PASSWORD', '?43 Rft>N>)8V@"D7.<I' );
define( 'DB_HOST', 'localhost' );

It is of course also possible to just use grep. However, if you just look for lines that start with “define” and include the string “DB_” then you will get a few extra lines:

$ grep ^define.*DB_ wp-config.php
define( 'DB_NAME', 'example_db' );
define( 'DB_USER', 'example_user' );
define( 'DB_PASSWORD', '?43 Rft>N>)8V@"D7.<I' );
define( 'DB_HOST', 'localhost' );
define( 'DB_CHARSET', 'utf8' );
define( 'DB_COLLATE', '' );

Replacing text

sed is commonly used to replace text. You may develop websites on your localhost, where the path to files is different than the path on the live server. For instance, on your localhost your document root may include the website’s domain name:

$ cat /var/www/html/example.net/_incs/header.html
<header id="header">
<h1 id="mast"><a href="/example.net/">Examples Rock!</a></h1>
</header>

Here, the document root is /example.net/. On the live server the document root is likely to be /. Before uploading files to the live server all instances of /example.net/ may therefore need to be changed to /. sed can do this without having to open the file in an editor.

The basic command for substituting text is as follows:

sed 's/old_text/new_text/g' file_name

The s tells sed that we want to substitute text and the g at the end of the command is short for “global”. With the global option sed will replace all instances of old_text with new_text on a line. Without the option only the first instance on a line would be replaced.

To replace the string /example.net with nothing we can use the following command:

$ sed 's/\/example.net//g' /var/www/html/example.net/_incs/header.html
<header id="header">
<h1 id="mast"><a href="/">Examples Rock!</a></h1>
</header>

Note that we escaped the forward stroke (/) in /example.net with a backward stroke (\). This is because by default sed uses forward strokes to delimit the different parts of the command. That can get really ugly:

$ sed 's/\/example.net\//\//g' /var/www/html/example.net/_incs/header.html
<header id="header">
<h1 id="mast"><a href="/">Examples Rock!</a></h1>
</header>

Luckily, we don’t have to use strokes as the delimiter. You can specify any other character, such as the pipe symbol (|). As you can see, that makes the command much more readable:

$ sed 's|/example.net/|/|g' /var/www/html/example.net/_incs/header.html
<header id="header">
<h1 id="mast"><a href="/">Examples Rock!</a></h1>
</header>

In-place editing

So far, our changes have not been saved. We asked sed to replace a string and the editor dived into the file we specified. It looked at every line, made the substitution we wanted, and printed the lines to the screen. The original file hasn’t changed though:

$ cat /var/www/html/example.net/_incs/header.html
<header id="header">
<h1 id="mast"><a href="/example.net/">Examples Rock!</a></h1>
</header>

To edit files ‘in place’ you need to add the -i option.

$ sed -i 's|/example.net/|/|g' /var/www/html/example.net/_incs/header.html
$ cat /var/www/html/example.net/_incs/header.html
<header id="header">
<h1 id="mast"><a href="/">Examples Rock!</a></h1>
</header>

Back-ups

If you give the -i option a suffix you will get two versions of the file: before the original file is overwritten sed will save a copy of the original file using the suffix you specify:

$ sed -i.BAK 's|/example.net/|/|g' /var/www/html/example.net/_incs/header.html
$ cat /var/www/html/example.net/_incs/header.html
<header id="header">
<h1 id="mast"><a href="/">Examples Rock!</a></h1>
</header>

$ cat /var/www/html/example.net/_incs/header.html.BAK
<header id="header">
<h1 id="mast"><a href="/example.net/">Examples Rock!</a></h1>
</header>

find, xargs and sed

That’s all very dandy, you might say, but wouldn’t it be easier to just open the file in a normal editor and do a search and replace? Which is true if you need to replace a string in just one or two files. But what if you need to replace the same string in dozens or hundreds of files?

Combining sed with other utilities will save the day. To continue with our example, if you want to replace the string /example.net/ with just a single stroke throughout the /var/www/html/example/ directory then you can combine sed with find and xargs:

find /var/www/html/example/ -name "*.html" -print0 | xargs -0 sed -i 's|/example.net/|/|g'

This commands first finds all files containing the string .html. The output of the find command is then piped to xargs which executes our sed command. The -print0 and -0 options are used to deal with file names that contain spaces. Of course, HTML files names shouldn’t contain spaces, but it is always better to be safe than sorry.

More information

This article has really just scratched the surface of what sed can do. It is an incredibly flexible and powerful tool. If you are going spend lots of time working with text files on the command line then it is worth learning more about sed. We quite like Bruce Barnett’s introduction to sed and Tutorial Point’s sed guide.