The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,446 other followers

rsync: the MacGyver tool for making backups on *nix based operating systems

Posted by jpluimers on 2014/04/23

I’ve been using rsync as my MacGyver style backup-tool of choice on *nix systems and sometimes even Windows systems.

It works both locally and remotely, is simple to setup and over the years has gathered a lot of nifty functionality.

It is way easier to backup using rsync one directory to another than using tar (tar has the benefit of putting everything in one archive though) using a command like this:

rsync -aiv /path/to/source/directory /path/to/destination/directory

For remote copies, I usually add replace -aiv with -aivz or with -aiz.

Given the ubiquitous of hard disk space, I hardly compress or archive directory trees for archival purposes any more.

For an introduction of basic functionality read Everything Linux – A Tutorial on Using Rsync. An article from 1999 that is still very valid now.

Besides my praise for rsync, there are a few small things I want to mention in this article:

  1. Sometimes more is less. Recently someone asked me how to force rsync not to keep the time stamps of files.
    He wasn’t the first to ask.
    The solution is simple: since the -a option archive option implies -t, so the solution is to expand -a into its parts -rlptgoD, then remove the -t from that.
  2. The –link-dest=DIR option was added in about 2004 (later: no, link-dest was added to rsync 2.5.6 in januari 2003), which allows you to do incremental backups.
    Though link-dest still does not work for individual files, it works great for directories.
    Lots and lots of people have written scripts for making incremental backups, even for Mac.
  3. Do not underestimate the -i option (–itemize-changes) which makes reading the output a lot easier.
    The rsync man page explains it towards the bottom, but the explanation at understanding the output of rsync itemize changes is much more readable.
  4. Tools have been written around rsync in order to facilitate making backups.
    For instance rsnapshot can be used to make incremental snapshots with defined rotating/retention periods. Very similar to Time Machine on Mac OS X, sans the UI. Thanks Erik Hollensbe for pointing me at this.

The –itemize changes flag:

<br />snip:/etc/apache2 # rsync -avi vhosts.d.backup/ vhosts.d.backup2/<br />sending incremental file list<br />created directory vhosts.d.backup2<br />cd+++++++++ ./<br />>f+++++++++ pluimers.com.conf<br />>f+++++++++ vhost-ssl.template<br />>f+++++++++ vhost.template<br /><br />sent 14042 bytes  received 72 bytes  28228.00 bytes/sec<br />total size is 13816  speedup is 0.98<br />snip:/etc/apache2 # rsync -avi vhosts.d.backup/ vhosts.d.backup2/<br />sending incremental file list<br /><br />sent 94 bytes  received 12 bytes  212.00 bytes/sec<br />total size is 13816  speedup is 130.34<br />

–jeroen

One Response to “rsync: the MacGyver tool for making backups on *nix based operating systems”

  1. jpluimers said

    Thomas Mueller7:46 AM+1 on G+: https://plus.google.com/u/0/+JeroenPluimers/posts/BNf1VtpiGD5

    I’m using dirvish at work, which is based on rsync. Works like a charm and due to to the history it keeps and the automatic deduplication, it’s really nice to use. We’re using two external hard drives which get switched daily. One then is replaced and archived every month, so we also get a longtime archive. So far we didn’t get a single failure on the archived disks. The oldest ones are now 5 years old.

    http://www.dirvish.org

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

 
%d bloggers like this: