rsync: the MacGyver tool for making backups on *nix based operating systems
Posted by jpluimers on 2014/04/23
I’ve been using rsync as my MacGyver style backup-tool of choice on *nix systems and sometimes even Windows systems.
It works both locally and remotely, is simple to setup and over the years has gathered a lot of nifty functionality.
It is way easier to backup using rsync one directory to another than using tar (tar has the benefit of putting everything in one archive though) using a command like this:
rsync -aiv /path/to/source/directory /path/to/destination/directoryFor remote copies, I usually add replace
-aivwith-aivzor with-aiz.Given the ubiquitous of hard disk space, I hardly compress or archive directory trees for archival purposes any more.
For an introduction of basic functionality read Everything Linux – A Tutorial on Using Rsync. An article from 1999 that is still very valid now.
Besides my praise for rsync, there are a few small things I want to mention in this article:
- Sometimes more is less. Recently someone asked me how to force rsync not to keep the time stamps of files.
He wasn’t the first to ask.
The solution is simple: since the-aoption archive option implies-t, so the solution is to expand-ainto its parts-rlptgoD, then remove the-tfrom that. - The –link-dest=DIR option was added in about 2004 (later: no, link-dest was added to rsync 2.5.6 in januari 2003), which allows you to do incremental backups.
Though link-dest still does not work for individual files, it works great for directories.
Lots and lots of people have written scripts for making incremental backups, even for Mac. - Do not underestimate the -i option (–itemize-changes) which makes reading the output a lot easier.
The rsync man page explains it towards the bottom, but the explanation at understanding the output of rsync itemize changes is much more readable. - Tools have been written around rsync in order to facilitate making backups.
For instance rsnapshot can be used to make incremental snapshots with defined rotating/retention periods. Very similar to Time Machine on Mac OS X, sans the UI. Thanks Erik Hollensbe for pointing me at this.
The –itemize changes flag:
<br />snip:/etc/apache2 # rsync -avi vhosts.d.backup/ vhosts.d.backup2/<br />sending incremental file list<br />created directory vhosts.d.backup2<br />cd+++++++++ ./<br />>f+++++++++ pluimers.com.conf<br />>f+++++++++ vhost-ssl.template<br />>f+++++++++ vhost.template<br /><br />sent 14042 bytes received 72 bytes 28228.00 bytes/sec<br />total size is 13816 speedup is 0.98<br />snip:/etc/apache2 # rsync -avi vhosts.d.backup/ vhosts.d.backup2/<br />sending incremental file list<br /><br />sent 94 bytes received 12 bytes 212.00 bytes/sec<br />total size is 13816 speedup is 130.34<br />
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| snip:/etc/apache2 # rsync -avi vhosts.d.backup/ vhosts.d.backup2/ | |
| sending incremental file list | |
| created directory vhosts.d.backup2 | |
| cd+++++++++ ./ | |
| >f+++++++++ pluimers.com.conf | |
| >f+++++++++ vhost-ssl.template | |
| >f+++++++++ vhost.template | |
| sent 14042 bytes received 72 bytes 28228.00 bytes/sec | |
| total size is 13816 speedup is 0.98 | |
| snip:/etc/apache2 # rsync -avi vhosts.d.backup/ vhosts.d.backup2/ | |
| sending incremental file list | |
| sent 94 bytes received 12 bytes 212.00 bytes/sec | |
| total size is 13816 speedup is 130.34 | |
–jeroen
The Wiert Corner - irregular stream of stuff 
jpluimers said
Thomas Mueller7:46 AM+1 on G+: https://plus.google.com/u/0/+JeroenPluimers/posts/BNf1VtpiGD5
I’m using dirvish at work, which is based on rsync. Works like a charm and due to to the history it keeps and the automatic deduplication, it’s really nice to use. We’re using two external hard drives which get switched daily. One then is replaced and archived every month, so we also get a longtime archive. So far we didn’t get a single failure on the archived disks. The oldest ones are now 5 years old.
http://www.dirvish.org