ESXi: storing an ISO 8601 time-stamped backup tarball locally
Posted by jpluimers on 2022/01/25
In Determining the ESXi installation type (2014558) | VMware KB, I also showed how to backup the configuration and download it.
Sometimes you want an ISO 8601 time-stamped local tarball just in case you want to revert to it at a later stage.
First a small recap on how to get the tarball, download location and temporary location in the first place (it will be automatically deleted from the temporary location):
# vim-cmd hostsvc/firmware/sync_config # vim-cmd hostsvc/firmware/backup_config Bundle can be downloaded at : http://*/downloads/52aa233b-5db4-2298-5e1b-f510b2cd149f/configBundle-ESXi-X10SRH-CF.tgz # find /scratch/downloads/ -name *.tgz /scratch/downloads/52aa233b-5db4-2298-5e1b-f510b2cd149f/configBundle-ESXi-X10SRH-CF.tgz
Goal is to get the download filename and save it to a different folder and embed the ISO 8601 timestamp in the filename.
Like many scripts, sed and regular expressions come to the rescue once more, just like in ESXi ash/dash/busybox shell getting current timestamp in UTC ISO8601 format without colons or dashes (which we will need anyway because of the ISO 8601 time stamp, and a bit of fiddling at regex101.com/r/NyrzKF
# SCRATCH_CONFIG_BUNDLE_NAME=$(vim-cmd hostsvc/firmware/backup_config | sed -n -E -e "s/^(Bundle can be downloaded at : http://*)(/downloads/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}/configBundle-.+?)(.tgz)$//scratch23/p") # echo "SCRATCH_CONFIG_BUNDLE_NAME: '${SCRATCH_CONFIG_BUNDLE_NAME}'" SCRATCH_CONFIG_BUNDLE_NAME: '/scratch/downloads/5271677d-97db-30dc-673d-b99e61bed251/configBundle-ESXi-X10SRH-CF.tgz' # date --utc -I'seconds' --reference "${SCRATCH_CONFIG_BUNDLE_NAME}" 2021-05-09T17:44:42UTC
Note:
- In the
sedregular, expression,[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}matches a GUID. - In the
datecommand, the--referenceparameter must be last. - If you get the error below, then you ran too many
backup_configcommands in succession:
(vim.fault.TooManyWrites) { faultCause = (vmodl.MethodFault) null, faultMessage = msg = "Received SOAP response fault from []: syncConfiguration fault.TooManyWrites.summary" }You can see this in the
hostd.log, which on my system is in/scratch/log/hostd.logwhere it says I can retry in 2031 seconds (slightly more than half an hour):2021-05-09T19:34:13.420Z verbose hostd[A703B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab3] RecordOp ASSIGN: latestEvent, ha-eventmgr. Applied change to temp map. 2021-05-09T19:34:13.420Z info hostd[A703B70] [Originator@6876 sub=Vimsvc.ha-eventmgr opID=vim-cmd-27-2ab3] Event 196 : User root@127.0.0.1 logged in as VMware-client/6.5.0 2021-05-09T19:34:13.420Z verbose hostd[A703B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab3] RecordOp ADD: sessionList["52f6d64e-0a35-c1d7-de97-624d234bc2a7"], ha-sessionmgr. Applied change to temp map. 2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Vimsvc.TaskManager opID=vim-cmd-27-2ab5 user=root] Task Created : haTask--vim.host.FirmwareSystem.syncConfiguration-114207804 2021-05-09T19:34:13.423Z verbose hostd[9AC1B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab5 user=root] RecordOp ADD: recentTask["haTask--vim.host.FirmwareSystem.syncConfiguration-114207804"], ha-taskmgr. Applied change to temp map. 2021-05-09T19:34:13.423Z verbose hostd[9AC1B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab5 user=root] RecordOp ASSIGN: info, haTask--vim.host.FirmwareSystem.syncConfiguration-114207804. Applied change to temp map. 2021-05-09T19:34:13.423Z error hostd[9AC1B70] [Originator@6876 sub=Hostsvc.FirmwareSystem opID=vim-cmd-27-2ab5 user=root] Failed to sync configuration. Too many writes. Next sync possible in 2031 sec. 2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Default opID=vim-cmd-27-2ab5 user=root] AdapterServer caught exception: vim.fault.TooManyWrites 2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Vimsvc.TaskManager opID=vim-cmd-27-2ab5 user=root] Task Completed : haTask--vim.host.FirmwareSystem.syncConfiguration-114207804 Status error 2021-05-09T19:34:13.423Z verbose hostd[9AC1B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab5 user=root] RecordOp ASSIGN: info, haTask--vim.host.FirmwareSystem.syncConfiguration-114207804. Applied change to temp map. 2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Solo.Vmomi opID=vim-cmd-27-2ab5 user=root] Activation [N5Vmomi10ActivationE:0x0987e6e0] : Invoke done [syncConfiguration] on [vim.host.FirmwareSystem:ha-firmwareSystem] 2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Solo.Vmomi opID=vim-cmd-27-2ab5 user=root] Throw vim.fault.TooManyWrites 2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Solo.Vmomi opID=vim-cmd-27-2ab5 user=root] Result: --> (vim.fault.TooManyWrites) { --> faultCause = (vmodl.MethodFault) null, --> faultMessage = --> msg = "" --> }
Not few people have bumped into this, the only other I could find through [Wayback] “vim.fault.TooManyWrites” “syncConfiguration” – Google Search is [Archive.is] mal wieder purple Screen – VMware-Forum.
Figuring out the various parts of the SCRATCH_CONFIG_BUNDLE_NAME: '/scratch/downloads/5271677d-97db-30dc-673d-b99e61bed251/configBundle-ESXi-X10SRH-CF.tgz' is like at regex101.com/r/J4yU72, regex101.com/r/uID9xs and regex101.com/r/o8a4Am:
CONFIG_BUNDLE_DIRECTORY_NAME=$(echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\1/p") CONFIG_BUNDLE_FILE_NAME=$( echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\2/p") CONFIG_BUNDLE_DOT_EXTENSION=$( echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\3/p") echo "CONFIG_BUNDLE_DIRECTORY_NAME: '${CONFIG_BUNDLE_DIRECTORY_NAME}'" echo "CONFIG_BUNDLE_FILE_NAME: '${CONFIG_BUNDLE_FILE_NAME}'" echo "CONFIG_BUNDLE_DOT_EXTENSION: '${CONFIG_BUNDLE_DOT_EXTENSION}'"
Output is like this:
SCRATCH_CONFIG_BUNDLE_NAME: '/scratch/downloads/528f9f5a-0123-f022-2b4d-a5c2e595c51a/configBundle-ESXi-X10SRH-CF.tgz' CONFIG_BUNDLE_DIRECTORY_NAME: '/scratch/downloads/528f9f5a-0123-f022-2b4d-a5c2e595c51a/' CONFIG_BUNDLE_FILE_NAME: 'configBundle-ESXi-X10SRH-CF' CONFIG_BUNDLE_DOT_EXTENSION: '.tgz'
Full backup-config-to-ESXi_configuration_backup-directory.sh script:
#!/bin/sh # https://wiert.me/2022/01/25/esxi-storing-an-iso-8601-time-stamped-backup-tarball-locally/ # Absolute path to this script, e.g. /home/user/bin/foo.sh echo "Command: '$0'" SCRIPT_FILENAME=$(readlink -f "$0") # Absolute path this script is in, thus /home/user/bin SCRIPT_PATH=$(dirname "${SCRIPT_FILENAME}") echo "SCRIPT_PATH: '${SCRIPT_PATH}'" PARENT_PATH=$(dirname "${SCRIPT_PATH}") echo "PARENT_PATH: '${PARENT_PATH}'" BACKUP_PATH="$PARENT_PATH/ESXi_configuration_backup" # echo "BACKUP_PATH: '${BACKUP_PATH}'" # https://www.cyberciti.biz/faq/howto-check-if-a-directory-exists-in-a-bash-shellscript/ if [ -d "${BACKUP_PATH}" ] then echo "BACKUP_PATH: '${BACKUP_PATH}' exists." else echo "BACKUP_PATH: '${BACKUP_PATH}' does not exist; creating." mkdir "${BACKUP_PATH}" fi # short hostname (the fqdn would include the domain name) SHORT_HOSTNAME=$(hostname -s) echo "SHORT_HOSTNAME: '$SHORT_HOSTNAME'" vim-cmd hostsvc/firmware/sync_config SCRATCH_CONFIG_BUNDLE_NAME=$(vim-cmd hostsvc/firmware/backup_config | sed -n -E -e "s/^(Bundle can be downloaded at : http:\/\/\*)(\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/configBundle-.+?)(\.tgz)$/\/scratch\2\3/p") echo "SCRATCH_CONFIG_BUNDLE_NAME: '${SCRATCH_CONFIG_BUNDLE_NAME}'" # ls -Alh ${SCRATCH_CONFIG_BUNDLE_NAME} # https://wiert.me/2022/01/17/esxi-ash-dash-busybox-shell-getting-current-timestamp-in-utc-iso8601-format/ UTC_TIMESTAMP=$(date --utc -I'seconds' --reference "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/^([[:digit:]]{4})\-([[:digit:]]{2})\-([[:digit:]]{2}T[[:digit:]]{2})\:([[:digit:]]{2})\:([[:digit:]]{2})UTC$/\1\2\3\4\5Z/p") echo "UTC_TIMESTAMP: '${UTC_TIMESTAMP}'" # https://stackoverflow.com/questions/13055889/sed-with-literal-string-not-input-file/37682812#37682812 CONFIG_BUNDLE_DIRECTORY_NAME=$(echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\1/p") CONFIG_BUNDLE_FILE_NAME=$( echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\2/p") CONFIG_BUNDLE_DOT_EXTENSION=$( echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\3/p") echo "CONFIG_BUNDLE_DIRECTORY_NAME: '${CONFIG_BUNDLE_DIRECTORY_NAME}'" echo "CONFIG_BUNDLE_FILE_NAME: '${CONFIG_BUNDLE_FILE_NAME}'" echo "CONFIG_BUNDLE_DOT_EXTENSION: '${CONFIG_BUNDLE_DOT_EXTENSION}'" CONFIG_TARBALL_NAME="${BACKUP_PATH}/${CONFIG_BUNDLE_FILE_NAME}.${UTC_TIMESTAMP}${CONFIG_BUNDLE_DOT_EXTENSION}" echo "CONFIG_TARBALL_NAME: '${CONFIG_TARBALL_NAME}'" rsync --times ${SCRATCH_CONFIG_BUNDLE_NAME} ${CONFIG_TARBALL_NAME}
This post was in part inspired by [Wayback] How to Back Up and Restore VMware ESXi Host Configuration, which also adds a crontab entry to automate the configuration backup, and explains how to restore a configuration backup.
–-jeroen
The Wiert Corner - irregular stream of stuff 
Leave a Reply