Posted by Jeroen Pluimers on 2014/04/23
I’ve been using rsync as my MacGyver style backup-tool of choice on *nix systems and sometimes even Windows systems.
It works both locally and remotely, is simple to setup and over the years has gathered a lot of nifty functionality.
It is way easier to backup using rsync one directory to another than using tar (tar has the benefit of putting everything in one archive though) using a command like this:
rsync -aiv /path/to/source/directory /path/to/destination/directory
For remote copies, I usually add replace
-aivz or with
Given the ubiquitous of hard disk space, I hardly compress or archive directory trees for archival purposes any more.
For an introduction of basic functionality read Everything Linux – A Tutorial on Using Rsync. An article from 1999 that is still very valid now.
Besides my praise for rsync, there are a few small things I want to mention in this article:
- Sometimes more is less. Recently someone asked me how to force rsync not to keep the time stamps of files.
He wasn’t the first to ask.
The solution is simple: since the
-a option archive option implies
-t, so the solution is to expand
-a into its parts
-rlptgoD, then remove the
-t from that.
- The –link-dest=DIR option was added in about 2004 (later: no, link-dest was added to rsync 2.5.6 in januari 2003), which allows you to do incremental backups. Read the rest of this entry »
Posted in *nix, bash, Cygwin, Development, Linux, Power User, Scripting, Software Development, SuSE Linux | 1 Comment »
Posted by Jeroen Pluimers on 2014/04/13
A while ago, I wrote about getting rsync on ESXi: ESXi 5.1 and rsync – damiendebin.net.
Now I needed 7zip on ESXi to make sure I could test unpack some 7zip archives.
This turned out much easier than I thought, thanks to 7Zip for ESXI | Vladimir Lukianov: Заметки who pointed me to the P7ZIP project. P7ZIP actually created three things:
- p7zip (a POSIX 7zip),
- J7ZIP (a Java port of 7zip)
- java_lzma (the Java port of the 7zip lzma SDK which had the first implementation of lzma).
Here are the full steps to get 7zip on ESXi 5.x:
Read the rest of this entry »
Posted in *nix, *nix-tools, ESXi4, ESXi5, ESXi5.1, ESXi5.5, Linux, Power User, SuSE Linux, VMware, VMware ESXi | Tagged: 7z, 7zip | 2 Comments »
Posted by Jeroen Pluimers on 2014/04/09
A long time ago I asked this OpenSuSE/Linux question: How can a partition be full if du does not show it is? – Linux on Super User.
With help of the OpenSuSE forums, I did figure out the source of the problem and solution, but I totally forgot to blog about it.
So below it is, just in case SuperUser ever shuts down, or the StackOverflow moderators are taking over SuperUser as well.
But first the comments in the questions about where I found the source and solution:
I found it through the openSUSE forums: it uses btrfs and snapshots. So the snapshots take up a lot of space. And I need to find out a way to delete old snapshots. forums.opensuse.org/english/get-technical-help-here/…
I think I found it: nrtm.org/index.php/2012/03/13/…
Read the rest of this entry »
Posted in *nix, *nix-tools, Linux, Power User, SuSE Linux | Tagged: snapshots | Leave a Comment »
Posted by Jeroen Pluimers on 2014/03/30
Read this very nice post on Nex7′s Blog: ECC vs non-ECC RAM: The Great Debate.
There is no debate. Use ECC dude.
Use ECC especially for server side things (storage, virtualization, databases, etc) where you employ some kind of redundancy/correction in the storage (ZFS, RAID, etc) side of things.
And think about using ECC for the rest of your stuff, especially when things stay in memory for a longer period of time (in-memory processing of data can speed up things a lot, but also increase the risk).
There is no debate here. None.
if you think non-ECC RAM can compete with ECC RAM, you are mistaken. If you think there’s a risk/reward analysis here, you’re correct. The risk is not gigantic, and there’s a real cost to alleviating that risk. You have to decide if that cost is worth alleviating that risk.
If you believe there’s a risk/reward plan where you can take the reward and apply to to mitigate the risk, you are back to being mistaken. The only benefit of non-ECC RAM (and thus the only reward in its choice over ECC RAM) is it will make the solution cheaper. There is not, however, any way (that I’ve heard of, yet) you can use the cost savings to mitigate the risk using non-ECC RAM will introduce.
If you choose to use non-ECC RAM, you open yourself up to a new vector for data corruption/loss/downtime/errors/etc,
one that could (rarely) even cause you to lose your entire filesystem, and one ZFS does not (cannot) resolve for you. Indeed, one it likely can’t even see at all. If you choose to employ non-ECC RAM, or are forced to do so because of circumstance or environmental constraint, that’s potentially understandable (and even acceptable) – but do not then attempt to validate or explain away that choice with pseudoscience or downplaying the risk you’ve added. You are using an inferior solution with an extra vector for data corruption/loss that ECC RAM solutions simply do not have. It is that simple.
Hint 3: There’s a reason we’re so gung-ho about using ECC RAM for ZFS, and it’s not just because we’re paranoid about data loss (which goes hand in hand with being a ZFS zealot, really). It is because you likely don’t realize how at risk you are. Due to the nature of how ZFS handles writes, your incoming (write) data is at risk of RAM-related bit errors for likely significantly longer than traditional storage solutions or alternative filesystems. 5, 10, 30, 60 or more seconds in a state where it is at risk.
Read the rest of this entry »
Posted in *nix, ECC memory, Endian, ESXi4, ESXi5, ESXi5.1, ESXi5.5, Hardware, Hyper-V, Linux, Memory, Power User, SuSE Linux, VMware, VMware ESXi, Windows, Windows 7, Windows 8, Windows 8.1, Windows Server 2008, Windows Server 2008 R2 | Tagged: ECC RAM, ZFS | Leave a Comment »
Posted by Jeroen Pluimers on 2014/03/29
Thanks Robert Gamble, ephemient and Jonathan Leffler. Be sure to read the top two answers and comments for full details.
Until now, I always used $* to pass on arguments from *nux shells (bash, sh, ash, etc.). Works on ESXi as well. But that is not the correct way to do.
But “$@” is the correct way:
- Use “$@” to represent all the arguments:
for var in "$@"
- As a shortcut,
for var; do ...; done means
for var in "$@"; do ...; done
- Basic thesis: “$@” is correct, and $* (unquoted) is almost always wrong. This is because “$@” works fine when arguments contain spaces, and works the same as $* when they don’t. In some circumstances, “$*” is OK too, but “$@” usually (but not always) works in the same places. Unquoted, $@ and $* are equivalent (and almost always wrong).
This next to the following construct makes file processing in *nix a breeze:
for filename in *.7z; do if 7za t $filename 2>&1 > /dev/null; then echo $filename passed; else echo $filename failed; fi; done
via: command line – How to iterate over arguments in bash script – Stack Overflow.
Posted in *nix, bash, Cygwin, Development, ESXi4, ESXi5, ESXi5.1, ESXi5.5, Linux, Power User, Scripting, Software Development, SuSE Linux, VMware ESXi | Leave a Comment »