Using sed to print and replace text

Last updated: 29 April 2021

sed is a stream editor that reads and manipulates text using functions you specify in your command. sed understands regular expressions, which makes it a powerful toy.

The basics

sed can be a handy tool if you want to print specific lines in a file. For instance, you can use sed to print lines 21 to 32 of a wp-config.php file:

$ sed -n 21,32p wp-config.php
// ** MySQL settings - You can get this info from your web host ** //
/** The name of the database for WordPress */
define( 'DB_NAME', 'example_db' );

/** MySQL database username */
define( 'DB_USER', 'example_user' );

/** MySQL database password */
define( 'DB_PASSWORD', '?43 Rft>N>)8V@"D7.<I' );

/** MySQL hostname */
define( 'DB_HOST', 'localhost' );

The -n option prevents sed from printing every line in the file and 21,32p prints lines 21 to 32.

Our aim with the above command is to get the database connection information. We therefore really just want the lines that start with “define”. To get that information we could pipe the output to grep. The below command prints all lines that start with (^) the string “define”:

$ sed -n 21,32p wp-config.php | grep ^define wp-config.php
define( 'DB_NAME', 'example_db' );
define( 'DB_USER', 'example_user' );
define( 'DB_PASSWORD', '?43 Rft>N>)8V@"D7.<I' );
define( 'DB_HOST', 'localhost' );

You can print the same lines using only sed with multiple -e options. Here, we are printing lines 23, 26, 29 and 32:

$ sed -n -e 23p -e 26p -e 29p -e 32p wp-config.php
define( 'DB_NAME', 'example_db' );
define( 'DB_USER', 'example_user' );
define( 'DB_PASSWORD', '?43 Rft>N>)8V@"D7.<I' );
define( 'DB_HOST', 'localhost' );

It is of course also possible to just use grep. However, if you just look for lines that start with “define” (and perhaps include the string “DB_”) then you get a few extra lines:

$ grep ^define.*DB_ wp-config.php
define( 'DB_NAME', 'example_db' );
define( 'DB_USER', 'example_user' );
define( 'DB_PASSWORD', '?43 Rft>N>)8V@"D7.<I' );
define( 'DB_HOST', 'localhost' );
define( 'DB_CHARSET', 'utf8' );
define( 'DB_COLLATE', '' );

Replacing text

sed is commonly used to replace text. You may develop websites on your localhost, where the path to files is different than the path on the live server. For instance, on your localhost your document root perhaps includes the website’s domain name:

$ cat /var/www/html/example.net/_incs/header.html
<header id="header">
<h1 id="mast"><a href="/example.net/">Examples Rock!</a></h1>
</header>

Here, the document root is /example.net/. On the live server the document root is likely to be /. Before uploading files to the live server all instances of /example.net/ therefore need to be changed to /. sed can do this for you. The basic command for substituting text is as follows:

sed 's/old_text/new_text/g' file_name

The s tells sed that we want to substitute text and the g at the end of the command is short for “global”. With the global option sed replaces all instances of old_text with new_text on a line. Without the option only the first instance on a line is replaced.

To replace the string /example.net/ with / we can use the following command:

$ sed 's/\/example.net\//\//g' /var/www/html/example.net/_incs/header.html
<header id="header">
<h1 id="mast"><a href="/">Examples Rock!</a></h1>
</header>

Note that we escaped the forward strokes (/) in the strings with backward strokes (\). That is necessary, as sed uses forward strokes to delimit the different parts of the command. All the escaped strokes make the command difficult to read. Luckily, you don’t have to use strokes as the delimiter. You can specify any other character, such as the pipe symbol (|). As you can see, that makes the command much more readable:

$ sed 's|/example.net/|/|g' /var/www/html/example.net/_incs/header.html
<header id="header">
<h1 id="mast"><a href="/">Examples Rock!</a></h1>
</header>

In-place editing

So far, our changes have not been saved. We asked sed to replace a string and the editor dived into the file we specified. It looked at every line in the file, made the substitution we wanted, and printed the lines to the screen. The original file hasn’t changed though:

$ cat /var/www/html/example.net/_incs/header.html
<header id="header">
<h1 id="mast"><a href="/example.net/">Examples Rock!</a></h1>
</header>

To edit files ‘in place’ you need to add the -i option:

$ sed -i 's|/example.net/|/|g' /var/www/html/example.net/_incs/header.html
$ cat /var/www/html/example.net/_incs/header.html
<header id="header">
<h1 id="mast"><a href="/">Examples Rock!</a></h1>
</header>

Backups

If you give the -i option a suffix you get two versions of the file: before the original file is overwritten sed saves a copy of the original file using the suffix you specified:

$ sed -i.BAK 's|/example.net/|/|g' /var/www/html/example.net/_incs/header.html
$ cat /var/www/html/example.net/_incs/header.html
<header id="header">
<h1 id="mast"><a href="/">Examples Rock!</a></h1>
</header>

$ cat /var/www/html/example.net/_incs/header.html.BAK
<header id="header">
<h1 id="mast"><a href="/example.net/">Examples Rock!</a></h1>
</header>

find, xargs and sed

That’s all very dandy, you might say, but wouldn’t it be easier to just open the file in a normal editor and do a search and replace? Which is true if you need to replace a string in just one or two files. But what if you need to replace the same string in dozens or hundreds of files?

Combining sed with other utilities will save the day. To continue with our example, if you want to replace the string /example.net/ with just a single stroke throughout the /var/www/html/example/ directory then you can combine sed with find and xargs:

$ find /var/www/html/example/ -name "*.html" -print0 \
| xargs -0 sed -i 's|/example.net/|/|g'

This commands first finds all files containing the string .html. The output of the find command is then piped to xargs which executes our sed command. The -print0 and -0 options are used to deal with file names that contain spaces. Of course, HTML files names shouldn’t contain spaces, but it is always better to be safe than sorry.

More information

This article has really just scratched the surface of what sed can do. It is an incredibly flexible and powerful tool. If you are going spend lots of time working with text files on the command line then it is worth learning more about sed. We quite like Bruce Barnett’s introduction to sed and Tutorial Point’s sed guide.