The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,862 other subscribers

Archive for the ‘Linux’ Category

rsync: the MacGyver tool for making backups on *nix based operating systems

Posted by jpluimers on 2014/04/23

I’ve been using rsync as my MacGyver style backup-tool of choice on *nix systems and sometimes even Windows systems.

It works both locally and remotely, is simple to setup and over the years has gathered a lot of nifty functionality.

It is way easier to backup using rsync one directory to another than using tar (tar has the benefit of putting everything in one archive though) using a command like this:

rsync -aiv /path/to/source/directory /path/to/destination/directory

For remote copies, I usually add replace -aiv with -aivz or with -aiz.

Given the ubiquitous of hard disk space, I hardly compress or archive directory trees for archival purposes any more.

For an introduction of basic functionality read Everything Linux – A Tutorial on Using Rsync. An article from 1999 that is still very valid now.

Besides my praise for rsync, there are a few small things I want to mention in this article:

  1. Sometimes more is less. Recently someone asked me how to force rsync not to keep the time stamps of files.
    He wasn’t the first to ask.
    The solution is simple: since the -a option archive option implies -t, so the solution is to expand -a into its parts -rlptgoD, then remove the -t from that.
  2. The –link-dest=DIR option was added in about 2004 (later: no, link-dest was added to rsync 2.5.6 in januari 2003), which allows you to do incremental backups. Read the rest of this entry »

Posted in *nix, Development, Linux, Power User, Scripting, Software Development, SuSE Linux | 1 Comment »

7zip on ESXi through p7zip

Posted by jpluimers on 2014/04/13

A while ago, I wrote about getting rsync on ESXi: ESXi 5.1 and rsync – damiendebin.net.

Now I needed [WayBack7zip on ESXi to make sure I could test unpack some 7zip archives.

This turned out much easier than I thought, thanks to [WayBack7Zip for ESXI | Vladimir Lukianov: Заметки who pointed me to the [WayBackP7ZIP project. P7ZIP actually created three things:

  • p7zip (a POSIX 7zip),
  • J7ZIP (a Java port of 7zip)
  • java_lzma (the Java port of the [WayBack7zip lzma SDK which had the first implementation of [WayBack] lzma).

Here are the full steps to get 7zip on ESXi 5.x:

Read the rest of this entry »

Posted in *nix, *nix-tools, ESXi4, ESXi5, ESXi5.1, ESXi5.5, Linux, Power User, SuSE Linux, VMware, VMware ESXi | Tagged: , | 3 Comments »

How can a partition be full if du does not show it is? (via: linux – Super User) #OpenSuSE #btrfs #snapshots

Posted by jpluimers on 2014/04/09

A long time ago I asked this OpenSuSE/Linux question: How can a partition be full if du does not show it is? – Linux on Super User.

With help of the OpenSuSE forums, I did figure out the source of the problem and solution, but I totally forgot to blog about it.

So below it is, just in case SuperUser ever shuts down, or the StackOverflow moderators are taking over SuperUser as well.

But first the comments in the questions about where I found the source and solution:

I found it through the openSUSE forums: it uses btrfs and snapshots. So the snapshots take up a lot of space. And I need to find out a way to delete old snapshots. forums.opensuse.org/english/get-technical-help-here/…

and

I think I found it: nrtm.org/index.php/2012/03/13/…

I wasn’t alone, so here are some more useful links and links from people asking for help:

–jeroen

PS: here is my SE question on it: linux – How can a partition be full if du does not show it is? – Super User.

Read the rest of this entry »

Posted in *nix, *nix-tools, Linux, openSuSE, Power User, SuSE Linux | Tagged: | 1 Comment »

Heartbleed: Serious OpenSSL zero day vulnerability revealed | ZDNet

Posted by jpluimers on 2014/04/08

The fixed OpenSSL 1.01g is already available in source and for many platforms.

When do they become available anyone using OpenSSL 1.01 or 1.02 must deploy the patched version as fast as possible.

You also need to have all your certificates re-issued.

During the vulnerability period, your private keys may have been exposed, and there is no way to tell that they were not exposed.

Note the official binaries for Win32 1.01g are not available for yet (expect them soon), but the Indy team made Win32 and Win64 versions available.

Note that OpenSuSE did a backport of the patch to 1.01e for 12.3 and 13.1. Older openSuSE versions do not have updates for this issue, but you want to upgrade anything lower than 0.98 as they contain serious other vulnerabilities.

–jeroen

via

Posted in *nix, Delphi, Delphi 2006, Delphi 2007, Delphi 2009, Delphi 2010, Delphi 6, Delphi 7, Delphi XE, Delphi XE2, Delphi XE3, Delphi XE4, Delphi XE5, Development, Linux, OpenSSL, openSuSE, Power User, Security, Software Development, SuSE Linux | 7 Comments »

ECC vs non-ECC RAM: The Great Debate (via: Nex7’s Blog). Use the ECC dude.

Posted by jpluimers on 2014/03/30

Read this very nice post on Nex7’s Blog: ECC vs non-ECC RAM: The Great Debate.

There is no debate. Use ECC dude.

Use ECC especially for server side things (storage, virtualization, databases, etc) where you employ some kind of redundancy/correction in the storage (ZFS, RAID, etc) side of things.

And think about using ECC for the rest of your stuff, especially when things stay in memory for a longer period of time (in-memory processing of data can speed up things a lot, but also increase the risk).

Summary:

There is no debate here. None.

[…]

if you think non-ECC RAM can compete with ECC RAM, you are mistaken. If you think there’s a risk/reward analysis here, you’re correct. The risk is not gigantic, and there’s a real cost to alleviating that risk. You have to decide if that cost is worth alleviating that risk.

[…]

If you believe there’s a risk/reward plan where you can take the reward and apply to to mitigate the risk, you are back to being mistaken. The only benefit of non-ECC RAM (and thus the only reward in its choice over ECC RAM) is it will make the solution cheaper. There is not, however, any way (that I’ve heard of, yet) you can use the cost savings to mitigate the risk using non-ECC RAM will introduce.

[…]

If you choose to use non-ECC RAM, you open yourself up to a new vector for data corruption/loss/downtime/errors/etc,

one that could (rarely) even cause you to lose your entire filesystem, and one ZFS does not (cannot) resolve for you. Indeed, one it likely can’t even see at all. If you choose to employ non-ECC RAM, or are forced to do so because of circumstance or environmental constraint, that’s potentially understandable (and even acceptable) – but do not then attempt to validate or explain away that choice with pseudoscience or downplaying the risk you’ve added. You are using an inferior solution with an extra vector for data corruption/loss that ECC RAM solutions simply do not have. It is that simple.

[…]

Hint 3: There’s a reason we’re so gung-ho about using ECC RAM for ZFS, and it’s not just because we’re paranoid about data loss (which goes hand in hand with being a ZFS zealot, really). It is because you likely don’t realize how at risk you are. Due to the nature of how ZFS handles writes, your incoming (write) data is at risk of RAM-related bit errors for likely significantly longer than traditional storage solutions or alternative filesystems. 5, 10, 30, 60 or more seconds in a state where it is at risk.

Read the rest of this entry »

Posted in *nix, ECC memory, Endian, ESXi4, ESXi5, ESXi5.1, ESXi5.5, Hardware, Hyper-V, Linux, Memory, Power User, SuSE Linux, VMware, VMware ESXi, Windows, Windows 7, Windows 8, Windows 8.1, Windows Server 2008, Windows Server 2008 R2 | Tagged: , | Leave a Comment »

*nux: “$@” is how to iterate over arguments in bash script (via: command line – Stack Overflow)

Posted by jpluimers on 2014/03/29

Thanks Robert Gamble, ephemient and Jonathan Leffler. Be sure to read the top two answers and comments for full details.

Until now, I always used $* to pass on arguments from *nux shells (bash, sh, ash, etc.). Works on ESXi as well. But that is not the correct way to do.

But “$@” is the correct way:

  • Use “$@” to represent all the arguments:

for var in "$@"
do
echo "$var"
done

  • As a shortcut, for var; do ...; done means for var in "$@"; do ...; done
  • Basic thesis: “$@” is correct, and $* (unquoted) is almost always wrong. This is because “$@” works fine when arguments contain spaces, and works the same as $* when they don’t. In some circumstances, “$*” is OK too, but “$@” usually (but not always) works in the same places. Unquoted, $@ and $* are equivalent (and almost always wrong).

This next to the following construct makes file processing in *nix a breeze:

for filename in *.7z; do if 7za t $filename 2>&1 > /dev/null; then echo $filename passed; else echo $filename failed; fi; done

–jeroen

via: command line – How to iterate over arguments in bash script – Stack Overflow.

Posted in *nix, bash, Cygwin, Development, ESXi4, ESXi5, ESXi5.1, ESXi5.5, Linux, Power User, Scripting, Software Development, SuSE Linux, VMware ESXi | Leave a Comment »

osx – Midnight Commander (MC) installer for Mac OS X – Super User

Posted by jpluimers on 2014/03/28

Must have on a Mac when you use terminal; either installation way is fine. I got mc from louise, as that was the first one I found.

osx – Midnight Commander (MC) installer for Mac OS X – Super User.

–jeroen

Posted in Apple, Linux, Mac, Mac OS X / OS X / MacOS, Mac OS X 10.4 Tiger, Mac OS X 10.5 Leopard, Mac OS X 10.6 Snow Leopard, Mac OS X 10.7 Lion, OS X 10.8 Mountain Lion, Power User | Leave a Comment »

Some links on ESXi disaster recovery and configuration backup

Posted by jpluimers on 2014/03/27

Just in case the shit ever hits the fan:

Background information:

Notes:

bootbank, altbootbank, backup.sh, auto-backup.sh

Adding your own software to ESXi:

–jeroen

Posted in *nix, ESXi4, ESXi5, ESXi5.1, ESXi5.5, Linux, openSuSE, Power User, SuSE Linux, Virtualization, VMware, VMware ESXi | Tagged: , , | Leave a Comment »

vi intro — the cheat sheet method (via: IBM developerworks)

Posted by jpluimers on 2014/03/27

IBM isn’t all about dry corporate stuff and sometimes hard to read redbook documentation (:

I love the way they lead you do build your own VI cheat sheet step by step in vi intro — the cheat sheet method.

It is basically a vi tutorial that helps you to build up your own cheat sheet.

–jeroen Read the rest of this entry »

Posted in *nix, Cygwin, Endian, ESXi4, ESXi5, ESXi5.1, ESXi5.5, Linux, Power User, SuSE Linux, vi, VMware, VMware ESXi | Leave a Comment »

*nix: recursively listing “hidden” files from the current directory

Posted by jpluimers on 2014/03/25

As a follow up on my recent rsync on ESXi 5.1 post, as – when rsync in ESXi terminates the hard way because of a lost SSH connection – rsync can leave “hidden” files behind.

A small script that recursively shows the hidden files (those starting with a dot) starting from the current directory:

find . -iname ".*"

More of those (including deleting them, filtering for only files or only directories, etc) are at Linux / UNIX: Bash Find And Delete All Hidden Files Directories.

Note: don’t try to outsmart using something like piping through grep "\/\." as that will also match files who’s parent directories are hidden.

–jeroen

via:

Posted in *nix, Apple, bash, Development, ESXi4, ESXi5, ESXi5.1, ESXi5.5, Linux, Mac, Mac OS X / OS X / MacOS, Mac OS X 10.4 Tiger, Mac OS X 10.5 Leopard, Mac OS X 10.6 Snow Leopard, Mac OS X 10.7 Lion, MacBook, MacBook Retina, MacBook-Air, MacBook-Pro, OS X 10.8 Mountain Lion, OS X 10.9 Mavericks, Power User, Scripting, Software Development, SuSE Linux, VMware, VMware ESXi | Tagged: , | Leave a Comment »