rsync utility is used to copy files. It can copy files locally as well as from one host to another. You can also use
rsync to make backups.
rsync is almost always run with one or more options. The most common options are
-v. We will briefly look at these options before we get to some examples.
-a option is a combination of a bunch of other options:
-r Recurse into directories -l Copy symlinks as symlinks -p Preserve file permissions -t Preserve modification times -g Preserve group -o Preserve owner -D Preserve device and special files
This is a convenient shortcut but you don’t necessary want to use all these options. For instance, if you are syncing files to a remote system you may not want to preserve the group and owner data, as the same user and group might not exist on the remote system.
Another commonly used option is
--verbose). As the name suggest, this option makes
rsync produce lots of output – it prints all the files that are being synced and a summary of the amount of data transferred. You typically want to use this option, as it tells you what
rsync is doing.
To demonstrate how to use
rsync we will copy an entire WordPress site to a different directory. This is something you may want to do after you have installed WordPress via Softaculous. By default Softaculous installs WordPress in a subdirectory with the name wp. Often that is useful, as the install will be separate from any existing files in the public_html directory. However, it does mean that you have to move the files if you prefer to have the install in the public_html directory.
You can sync all the files from the public_html/wp directory to the public_html directory in seconds:
$ rsync -av /home/example/public_html/wp/ /home/example/public_html/ sending incremental file list ./ .htaccess index.php ... sent 44,901,625 bytes received 36,723 bytes 29,958,898.67 bytes/sec total size is 44,762,896 speedup is 1.00
As we added the
-v option the output tells us which files were copied and how much data was transferred. Copying about 45MB took about 1.5 seconds.
It is worth mentioning that the data has been copied rather than moved. In other words, the public_html/wp directory still contains all the original files. If you just want to move files it makes more sense to use the
mv utility instead. There are two things to be aware of though. The first is that
mv doesn’t move dotfiles by default:
$ mv /home/example/public_html/wp/ /home/example/public_html/ $ ls -1A /home/example/public_html/wp/ .htaccess
In the above example we moved all the files, but the .htaccess file is still sitting in the wp subdirectory. In Bash, you can set the
dotglob option to get
mv to also move hidden files:
$ shopt -s dotglob nullglob $ mv /home/example/public_html/wp/* /home/example/public_html/ $ ls -lA wp/ $
A second thing to note is that
mv doesn’t preserve the SELinux context of files. If SELinux is enforced on your server then it is easier to use
rsync, which does preserve the context.
It is very important to be aware of the function of the trailing slash in the source. To illustrate the point, let’s create two directories (source and destination) and create a file named testfile in the source directory:
$ mkdir source destination $ touch source/testfile
If you want to copy all files in the source directory to the destination directory then the source should include the trailing slash, like so:
$ rsync -av source/ destination/
The trailing slash instructs
rsync to copy all files (and directories) in the source directory. Can you guess what happens if you omit the trailing slash?
$ rsync -a source destination $ tree destination/ destination/ └── source └── testfile 1 directory, 1 file
Instead of copying the test file it copied the entire source directory to the destination directory. It is easy to see how you can very quickly do a lot of damage with
rsync if you get the trailing slash wrong.
If you are not quite sure about an
rsync command then you can use the
rsync will go through the motions but not actually copy any files. You can then check the output to see if your command works as it should:
$ rsync -avn source destination sending incremental file list source/ source/testfile sent 108 bytes received 23 bytes 262.00 bytes/sec total size is 0 speedup is 0.00 (DRY RUN)
The main thing to note in the above output is that
rsync is copying the source/ directory, which is not what we wanted.
To copy files to a remote server you use pretty much the same syntax. In the below example we are syncing files from the /srv/www/public_html directory on our local computer to the example.net server:
$ rsync -avz /srv/www/public_html/ email@example.com:/home/example/transfers/ sending incremental file list ./ .htaccess index.php license.txt ... sent 12,344,200 bytes received 36,504 bytes 575,846.70 bytes/sec total size is 44,750,608 speedup is 3.61
We added one new option:
--compress) compresses file data during the transfer. This is particularly useful when you sync files to and from a remote server, as this will always be slower than syncing files locally.
To connect to the remote server you use the username and hostname, just like you do when you connect to a remote server via SSH. You can specify the directory to which you want to sync the files by adding a colon (
:) followed by the path to the destination directory. In this case the files were synced to the /home/example/transfers directory on the remote server.
The opposite, syncing files from a remote server to your local computer, is of course also possible. You can simply reverse the source and destination, as shown in the below example:
$ rsync -avz firstname.lastname@example.org:/home/example/transfers/ /srv/www/public_html/
--exclude option lets you exclude a directory. This can be useful if you are developing websites locally and have, say, a directory for backups in your document root. Using
--exclude you can copy all the files, except for the backups directory:
$ rsync -avn --exclude 'backup' /srv/www/public_html/ email@example.com:/var/www/html/ | grep -c ^backup 0
Notice that we used the verbose option (
-v) and did a dry run (
-n). The output lists all the files that would normally be copied. As that list is very long we piped the output to
grep -c ^backup, which gives us a count of the number of lines starting with the string backup. There were zero lines, which means that our command does indeed exclude the backup directory.
You can exclude files in exactly the same way. And if you need to exclude more than one file or directory you can simply use multiple
As a bonus tip, you can also use wild cards (“globs”). For instance, you may have two directories that are not part of the website itself and which should never be copied. Let’s say you got a __backups directory for backups and a __resources directory for assets. As both directories start with two underscores you can exclude both at the same time:
$ rsync -avn --exclude '__*' /srv/www/public_html/ firstname.lastname@example.org:/var/www/html/ | grep -c ^__ 0
The asterisk in
--exclude '__*' matches both __backups and __resources. In the same way you can exclude specific file types, such as .tar.gz or .zip files:
$ rsync -avn /srv/www/public_html/ email@example.com:/var/www/html/ \ | grep -c ".*tar.gz$" 21 $ rsync -avn --exclude '*.tar.gz' /srv/www/public_html/ firstname.lastname@example.org:/var/www/html/ \ | grep -c ".*tar.gz$" 0
--delete option deletes files from the destination that are not in the source. In other words, the file foo.txt is deleted if it exists in the destination but not in the source.
This option is typically used for non-incremental backups. For instance, here we are copying the directory /src/www/ to an external backup drive:
$ rsync -av --delete /srv/www/ /run/media/example/ext_drive/backup/
After the sync the two directories are identical. All files are copied from /src/www/ to the backup drive and any files that exist on the backup drive but not in /srv/www/ directory are deleted.
As an aside, strictly speaking it is not true that all the files in the /src/www/ directory are copied. Any files that already exist on the backup drive are skipped. If you want you can force
rsync to overwrite files that already exist at the destination using the
rsync has an impressive amount of options. This article aimed to give you an idea of how you can use the tool. To learn more, it is worth reading the man page and search for more thorough tutorials, such as the one written by Computer Hope.
Also, the best way to learn about utilities such as
rsync is to use them. To demonstrate the importance of the trailing slash in the source directory we created two test directories and a test file. Rather than just reading about the utility you can create simple test cases like that yourself.