The Wiert Corner – irregular stream of stuff

This post is a reminder to self to really work on rsync backup of your ESXi box: How to make a statically linked rsync binary.

I’m still catching up on things that happened during my (recovery) of all the cancer treatments in the 2019-2022 time frame.

Which is why I missed two things (actually three if you count the long list of rsync changes up to the current version).

The first is to install the static rsync 3.1.3 binary that runs on ESXi (at least versions 5 through 7, likely 8 as well) on some of my various ESXi boxes.

This is how you get it:

1. [Wayback/Archive] noelmartinon/vmtools: Tools for VMware ESXi to use in ESXi
   

   The tools included in the VMtools suite are:

   • ovf-export 1.1.0 – export/backup one or more virtual machines to OVF format
   • ovf-deploy 1.1.0 – deploy/restore a virtual machine from OVF template
   • rsync-static 3.1.3 – a pre-build static rsync version (see usage in https://rsync.samba.org/)

   …

   Install ‘rsync’ (optional)

   1. Get rsync

   This command lines must be execute from a linux console and NOT in ESXi!

   • From vmtools site:

     cd /tmp
     wget https://github.com/noelmartinon/vmtools/master/bin/rsync-static
     mv rsync-static rsync
     chmod 775 rsync
     scp -p rsync root@ESXi_IP:/usr/bin/
     

   • From source (Centos 3.9 required):

     # In CentOS 3.9 (http://vault.centos.org/3.9/isos/i386/CentOS-3.9-server-i386.iso):
     sed -i "s/centos\.org/hmc\.edu/g" /etc/yum.conf
     rpm --import http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-3
     yum groupinstall "Development Tools"
     yum install gcc zlib-devel popt-devel glibc-static popt-static -y
     wget http://rsync.samba.org/ftp/rsync/rsync-3.1.3.tar.gz
     gunzip rsync-3.1.3.tar.gz
     tar -xvf rsync-3.1.3.tar
     cd rsync-3.1.3/
     sed -i "s/<zlib.h>/<zlib.h>\n\n#ifndef Z_INSERT_ONLY\n#define Z_INSERT_ONLY Z_SYNC_FLUSH\n#endif/g" token.c
     ./configure
     make CFLAGS="-static" EXEEXT="-static"
     strip rsync-static
     scp -p rsync-static root@ESXi_IP:/usr/bin/rsync
        
     # /!\ Trying under Debian and other CentOS build OK but got 'segmentation fault' in esxi 
     # sudo apt install libpopt-dev zlibc
     # cd /tmp
     # wget http://rsync.samba.org/ftp/rsync/rsync-3.1.3.tar.gz
     # tar -xvf rsync-3.1.3.tar.gz
     # cd rsync-3.1.3/
     # sed -i "s/<zlib.h>/<zlib.h>\n\n#ifndef Z_INSERT_ONLY\n#define Z_INSERT_ONLY Z_SYNC_FLUSH\n#endif/g" token.c
     # ./configure
     # make CFLAGS="-static" EXEEXT="-static"
     # strip rsync-static
     

2. [Wayback/Archive] vmtools/bin at master · noelmartinon/vmtools
3. [Wayback/Archive] vmtools/bin/rsync-static at master · noelmartinon/vmtools
4. [Wayback] raw.githubusercontent.com/noelmartinon/vmtools/master/bin/rsync-static which you can download using wget https://raw.githubusercontent.com/noelmartinon/vmtools/master/bin/rsync-static -O rsync-static-3.1.3 (if your ESXi is new enough, see ESXi 6.7 and up: embedded busybox wget understands https (finally!))

I also missed the availability of a docker container that should automagically build that binary:

• [Wayback/Archive] rsync backup of your ESXi box: How to make a statically linked rsync binary: comment
  

  Gorka Benguria said
  2022/07/18 at 08:43
  Hi I have manage to create the static rsync for esxi 7.0 using docker, I link here the github repo where I upload the docker file that creates the compiled rsync in case is useful for someone. github.com/gbenguria/rsync-esxi-docker-compiler

• [Wayback/Archive] gbenguria/rsync-esxi-docker-compiler: dockerfile that generated a static rsync binary to be used in esxi 7.0
  

  in order to generate the statically compiled rsync for esxi 7.0 do the following.

  cd this_repo
  echo to create the image that compiles rsync statically
  docker build . -t rsync-esxi-compiler
  echo to start the container in order to be able to copy the rsync binary
  docker run -d --name rsync-esxi-compiler rsync-esxi-compiler /bin/bash -c "while true; do sleep 30; done;"
  docker cp rsync-esxi-compiler:/rsync/rsync ~/.
  echo to check that effectivelly is a statically compiled binary
  ldd ~/rsync
  echo to copy to the target esxi in a folder that belongs to the path
  chmod 555 ~/rsync
  scp ~/rsync root@esxi1.somedomain:/bin/rsync
  echo to use it to create send a file 
  while ( ! rsync -av --progress --bwlimit=20480 --inplace --partial --append ubuntu-22.04-live-server-amd64.iso root@esxi1.somedomain:/targetfolder ); do sleep 10; done

• [Wayback/Archive] “not a dynamic executable” · Issue #1 · gbenguria/rsync-esxi-docker-compiler (20230203):
  

  Just a heads up, it doesn’t seem to compile properly anymore.

  I’m not really seeing any errors in the Docker Build other than these warnings:

  /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libcrypto.a(fips.o): In function `verify_checksums':
  (.text+0x4b9): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
  uidlist.o: In function `getallgroups':
  uidlist.c:(.text+0x15d2): warning: Using 'getgrouplist' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
  main.o: In function `become_copy_as_user':
  main.c:(.text+0x749): warning: Using 'initgroups' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
  uidlist.o: In function `gid_to_group':
  uidlist.c:(.text+0x31b): warning: Using 'getgrgid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
  uidlist.o: In function `group_to_gid':
  uidlist.c:(.text+0x4d6): warning: Using 'getgrnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
  uidlist.o: In function `user_to_uid':
  uidlist.c:(.text+0x3fa): warning: Using 'getpwnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
  main.o: In function `become_copy_as_user':
  main.c:(.text+0x663): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
  socket.o: In function `try_bind_local':
  socket.c:(.text+0x4c6): warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
  access.o: In function `match_hostname':
  access.c:(.text+0x169): warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

• On the above warnings about statically linked functions:
  • [Wayback/Archive] warning: Using ‘dlopen’ in statically linked applications requires at runtime the shared libraries from the glibc version used for linking – Google Search
  • [Wayback/Archive] I want static link crypto.a but has ” Using ‘dlopen’ in statically linked applications requires at runtime the shared libraries from the glibc version used for linking” · Issue #19035 · openssl/openssl
  • [Wayback/Archive] Solved: Using ‘dlopen’ in statically linked applications requires at runtime the shared libraries from the g – Intel Community
  • [Wayback/Archive] “rsync” warning: Using ‘dlopen’ in statically linked applications requires at runtime the shared libraries from the glibc version used for linking – Google Search
  • [Wayback/Archive] 617756 – sys-fs/lvm2-2.02.171::gentoo failed (compile phase): undefined reference to `nearbyint’
  • [Wayback/Archive] glibc-static-2.17-322.el7_9.x86_64 RPM
  • [Wayback/Archive] linux – Linking static libraries instead of shared using g++ – Stack Overflow

Older steps

[Wayback/Archive] How to make a statically linked Rsync Binary – neddix.com wiki

This document describes how to make a statically linked rsync binary as it is used on page Backup of ESXi Virtual Machines using Affa.

• download the source from rsync.samba.org
• unpack the tarball

tar xzf rsync-3.0.4.tar.gz
cd rsync-3.0.4

• build the statically linked binary

make CFLAGS="-static" EXEEXT="-static"
strip rsync-static

rsync changes

Home page at the time of writing had [Wayback/Archive] rsync: Rsync version 3.3.0pre1 released.

Browsing up to the newest version from the [Wayback/Archive] NEWS for rsync 3.1.3 I noticed a few things:

• there is a md2man (which actually is a symlink to md-convert) converting markdown man pages to html and optionally nroff:
  1. [Wayback/Archive] WayneD/rsync: An open source utility that provides fast incremental file transfer. It also has useful features for backup and restore operations among many other use cases.
  2. [Wayback/Archive] rsync/md2man at master · WayneD/rsync
  3. [Wayback/Archive] rsync/md-convert at master · WayneD/rsync
• Improved sparse file handling and configuration files.
• Memory management has much improved, as are documentation, version handling, security and error handling.
• Optional support for the most recent releases of [Wayback/Archive] Cyan4973/xxHash: Extremely fast non-cryptographic hash algorithm
• At least these CVE IDs fixed (prepended by rsync versions that fix them):
  • 3.2.0 [Wayback/Archive] CVE-2016-9840 : inftrees.c in zlib 1.2.8 might allow context-dependent attackers to have unspecified impact by leveraging improper point
  • 3.2.0 [Wayback/Archive] CVE-2016-9841 : inffast.c in zlib 1.2.8 might allow context-dependent attackers to have unspecified impact by leveraging improper pointe
  • 3.2.0 [Wayback/Archive] CVE-2016-9842 : The inflateMark function in inflate.c in zlib 1.2.8 might allow context-dependent attackers to have unspecified impact v
  • 3.2.0 [Wayback/Archive] CVE-2016-9843 : The crc32_big function in crc32.c in zlib 1.2.8 might allow context-dependent attackers to have unspecified impact via v
  • 3.2.4 [Wayback/Archive] CVE-2018-25032 : zlib before 1.2.12 allows memory corruption when deflating (i.e., when compressing) if the input has many distant matche
  • 3.2.5 [Wayback/Archive] CVE-2022-29154 : An issue was discovered in rsync before 3.2.5 that allows malicious remote servers to write arbitrary files inside the d
  • 3.2.5 [Wayback/Archive] CVE-2022-37434 : zlib through 1.2.12 has a heap-based buffer over-read or buffer overflow in inflate in inflate.c via a large gzip header

The CVE IDs are even more important to me than other bug fixes in order to get a more recent rsync static binary.

Query

Actually I ran [Wayback/Archive] “esxi” “rsync 3.1.3” – Google Search with all the version numbers up to 3.3.0.

Blog post

ESXi: rsync over ssh between two ESXi machines (and overcoming a potential “sh: rsync: not found”)

–jeroen