The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 2,573 other followers

Archive for the ‘Power User’ Category

ESXi: reloading all virtual machines from their (potentially) vmx files

Posted by jpluimers on 2022/01/26

I had to reload all registered virtual machine configurations after replacing an NVMe adapter in one of my ESXi hosts (see the posts next week).

The below script is based on this series of April last year on listing, stopping, starting, hibernating and waking up VMs, especially vim-cmd-list-all-VMs.sh:

  1. VMware ESXi console: viewing all VMs, suspending and waking them up: part 1 on Busybox, sed and regular expressions (especially capture groups)
  2. VMware ESXi console: viewing all VMs, suspending and waking them up: part 2 on vim-cmd vmsvc commands, especially the on the ones accepting a vmid parameter
  3. VMware ESXi console: viewing all VMs, suspending and waking them up: part 3 digging deeper into the various vim-cmd vmsvc/power.* commands
  4. VMware ESXi console: viewing all VMs, suspending and waking them up: part 4 on amending vim-cmd vmsvc/getallvms with information from vim-cmd vmsvc/power.getstate and vim-cmd vmsvc/get.config
  5. VMware ESXi console: viewing all VMs, suspending and waking them up: part 5 on applying the various vim-cmd vmsvc/power.* commands on a list of relevant virtual machines

Here is the script vim-cmd-reload-all-VM-vmx-configurations.sh:

#!/bin/sh
vmids=`vim-cmd vmsvc/getallvms | sed -n -E -e "s/^([[:digit:]]+)\s+((\S.+\S)?)\s+(\[\S+\])\s+(.+\.vmx)\s+(\S+)\s+(vmx-[[:digit:]]+)\s*?((\S.+)?)$/\1/p"`
for vmid in ${vmids} ; do                                       
    powerState=`vim-cmd vmsvc/power.getstate ${vmid} | sed '1d'`                                                                                    
    name=`vim-cmd vmsvc/get.config ${vmid} | sed -n -E -e '/\(vim.vm.ConfigInfo\) \{/,/files = \(vim.vm.FileInfo\) \{/ s/^ +name = "(.*)",.*?/\1/p'`                         
    vmPathName=`vim-cmd vmsvc/get.config ${vmid} | sed -n -E -e '/files = \(vim.vm.FileInfo\) \{/,/tools = \(vim.vm.ToolsConfigInfo\) \{/ s/^ +vmPathName = "(.*)",.*?/\1/p'`
    echo "Reloading vmx of VM with id ${vmid} which has power state ${powerState} (name = ${name}; vmPathName = ${vmPathName})."
    vim-cmd vmsvc/reload ${vmid}
done
exit 0

Note that the script can throw errors, for instance the below one:

# vim-cmd vmsvc/reload 27
(vmodl.fault.SystemError) {
   faultCause = (vmodl.MethodFault) null, 
   faultMessage = , 
   reason = "Invalid fault"
   msg = "Received SOAP response fault from []: reload
vim.fault.InvalidState"
}

This particular error means that the ESXi system itself is in an invalid state (as opposed to the virtual machine being an invalid state). In this case, it can be thrown if the ESXi system is in Maintenance Mode: in that case, it is not allowed to reload any .vmx virtual machine configurations (it only shows the configuration at the time that hostd got started).

There are two ways to solve this:

  1. The blunt way: restart the hostd as this will re-read the .vmx files
  2. The not so blunt way: bring the server out of maintenance mode, then rerun the vim-cmd vmsvc/reload command.

On the many places that can throw vim.fault.InvalidState

In this case, the reason for vim.fault.InvalidState was that the server was in maintenance mode. However, be aware that there are way more places that can throw vim.fault.InvalidState.

Via [Wayback/Archive.is] 1-day-SAP-custom/InvalidState.rst at master · bcstryker/1-day-SAP-custom and [Wayback/Archive.is] bcstryker/1-day-SAP-custom, I found [Wayback/Archive.is] InvalidState – vSphere Web Services API – VMware {code} that lists about 200 API calls for this seemingly simple fault:

Fault Description

An InvalidState fault is thrown if the operation failed due to the current state of the system.

For example, at [Wayback] Failed to deploy VM: vim.fault.InvalidState – VMware Technology Network VMTN the cause of vim.fault.InvalidState was deploying an .ova file over a slow WiFi connection.

On VMware services and their names

The VMware service names are very well explained in the [Wayback] what is difference between hostd, Vpxa and vpxd. – VMware Technology Network VMTN answer by [Wayback] About AndreTheGiant – VMware Technology Network VMTN

  • hostd is the daemon for direct VIC connection (when you use VIC to connect to your ESX).
  • vpx is the original name of vCenter Center, so:
    • vpxa is the VC agent (ESX side)
    • vpxd is the VC daemon (VC side)

Since ESXi 5.0, the VIC (VMware Infrastructure Client) got gradually replaced by the vSphere Web Client which is serviced by the hostd as well, see:

–jeroen

Posted in Power User, VMware, VMware ESXi, Virtualization, ESXi6, ESXi6.5, ESXi6.7, ESXi7 | Leave a Comment »

Good scientific abstract template

Posted by jpluimers on 2022/01/25

Good Abstracts: A Template
Fabrizio Gilardi*
February 24, 2021
Abstract
Everyone agrees that this issue is really important. But we do not know much about
this specific question, although it matters a great deal, for these reasons. We approach the
problem from this perspective. Our research design focuses on these cases and relies on
these data, which we analyze using this method. Results show what we have learned about
the question. They have these broader implications.
*Department of Political Science, University of Zurich (https://fabriziogilardi.org/).
1

Source: [Wayback] fabriziogilardi.org/resources/papers/good-abstracts.pdf

Via:

–jeroen

Posted in LifeHacker, Power User | Leave a Comment »

ESXi: storing an ISO 8601 time-stamped backup tarball locally

Posted by jpluimers on 2022/01/25

In Determining the ESXi installation type (2014558) | VMware KB, I also showed how to backup the configuration and download it.

Sometimes you want an ISO 8601 time-stamped local tarball just in case you want to revert to it at a later stage.

First a small recap on how to get the tarball, download location and temporary location in the first place (it will be automatically deleted from the temporary location):

# vim-cmd hostsvc/firmware/sync_config
# vim-cmd hostsvc/firmware/backup_config
Bundle can be downloaded at : http://*/downloads/52aa233b-5db4-2298-5e1b-f510b2cd149f/configBundle-ESXi-X10SRH-CF.tgz
# find /scratch/downloads/ -name *.tgz
/scratch/downloads/52aa233b-5db4-2298-5e1b-f510b2cd149f/configBundle-ESXi-X10SRH-CF.tgz

Goal is to get the download filename and save it to a different folder and embed the ISO 8601 timestamp in the filename.

Like many scripts, sed and regular expressions come to the rescue once more, just like in ESXi ash/dash/busybox shell getting current timestamp in UTC ISO8601 format without colons or dashes (which we will need anyway because of the ISO 8601 time stamp, and a bit of fiddling at regex101.com/r/NyrzKF

# SCRATCH_CONFIG_BUNDLE_NAME=$(vim-cmd hostsvc/firmware/backup_config | sed -n -E -e "s/^(Bundle can be downloaded at : http://*)(/downloads/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}/configBundle-.+?)(.tgz)$//scratch23/p")
# echo "SCRATCH_CONFIG_BUNDLE_NAME: '${SCRATCH_CONFIG_BUNDLE_NAME}'"
SCRATCH_CONFIG_BUNDLE_NAME: '/scratch/downloads/5271677d-97db-30dc-673d-b99e61bed251/configBundle-ESXi-X10SRH-CF.tgz'
# date --utc -I'seconds' --reference "${SCRATCH_CONFIG_BUNDLE_NAME}"
2021-05-09T17:44:42UTC

Note:

  • In the sed regular, expression, [[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12} matches a GUID.
  • In the date command, the --reference parameter must be last.
  • If you get the error below, then you ran too many backup_config commands in succession:
    (vim.fault.TooManyWrites) {
       faultCause = (vmodl.MethodFault) null, 
       faultMessage = 
       msg = "Received SOAP response fault from []: syncConfiguration
    fault.TooManyWrites.summary"
    }

    You can see this in the hostd.log, which on my system is in /scratch/log/hostd.log where it says I can retry in 2031 seconds (slightly more than half an hour):

    2021-05-09T19:34:13.420Z verbose hostd[A703B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab3] RecordOp ASSIGN: latestEvent, ha-eventmgr. Applied change to temp map.
    2021-05-09T19:34:13.420Z info hostd[A703B70] [Originator@6876 sub=Vimsvc.ha-eventmgr opID=vim-cmd-27-2ab3] Event 196 : User root@127.0.0.1 logged in as VMware-client/6.5.0
    2021-05-09T19:34:13.420Z verbose hostd[A703B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab3] RecordOp ADD: sessionList["52f6d64e-0a35-c1d7-de97-624d234bc2a7"], ha-sessionmgr. Applied change to temp map.
    2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Vimsvc.TaskManager opID=vim-cmd-27-2ab5 user=root] Task Created : haTask--vim.host.FirmwareSystem.syncConfiguration-114207804
    2021-05-09T19:34:13.423Z verbose hostd[9AC1B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab5 user=root] RecordOp ADD: recentTask["haTask--vim.host.FirmwareSystem.syncConfiguration-114207804"], ha-taskmgr. Applied change to temp map.
    2021-05-09T19:34:13.423Z verbose hostd[9AC1B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab5 user=root] RecordOp ASSIGN: info, haTask--vim.host.FirmwareSystem.syncConfiguration-114207804. Applied change to temp map.
    2021-05-09T19:34:13.423Z error hostd[9AC1B70] [Originator@6876 sub=Hostsvc.FirmwareSystem opID=vim-cmd-27-2ab5 user=root] Failed to sync configuration. Too many writes. Next sync possible in 2031 sec.
    2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Default opID=vim-cmd-27-2ab5 user=root] AdapterServer caught exception: vim.fault.TooManyWrites
    2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Vimsvc.TaskManager opID=vim-cmd-27-2ab5 user=root] Task Completed : haTask--vim.host.FirmwareSystem.syncConfiguration-114207804 Status error
    2021-05-09T19:34:13.423Z verbose hostd[9AC1B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab5 user=root] RecordOp ASSIGN: info, haTask--vim.host.FirmwareSystem.syncConfiguration-114207804. Applied change to temp map.
    2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Solo.Vmomi opID=vim-cmd-27-2ab5 user=root] Activation [N5Vmomi10ActivationE:0x0987e6e0] : Invoke done [syncConfiguration] on [vim.host.FirmwareSystem:ha-firmwareSystem]
    2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Solo.Vmomi opID=vim-cmd-27-2ab5 user=root] Throw vim.fault.TooManyWrites
    2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Solo.Vmomi opID=vim-cmd-27-2ab5 user=root] Result:
    --> (vim.fault.TooManyWrites) {
    -->    faultCause = (vmodl.MethodFault) null,
    -->    faultMessage = 
    -->    msg = ""
    --> }
    

Not few people have bumped into this, the only other I could find through [Wayback] “vim.fault.TooManyWrites” “syncConfiguration” – Google Search is [Archive.is] mal wieder purple Screen – VMware-Forum.

Figuring out the various parts of the SCRATCH_CONFIG_BUNDLE_NAME: '/scratch/downloads/5271677d-97db-30dc-673d-b99e61bed251/configBundle-ESXi-X10SRH-CF.tgz' is like at regex101.com/r/J4yU72, regex101.com/r/uID9xs and regex101.com/r/o8a4Am:

CONFIG_BUNDLE_DIRECTORY_NAME=$(echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\1/p")
CONFIG_BUNDLE_FILE_NAME=$(     echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\2/p")
CONFIG_BUNDLE_DOT_EXTENSION=$( echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\3/p")
echo "CONFIG_BUNDLE_DIRECTORY_NAME: '${CONFIG_BUNDLE_DIRECTORY_NAME}'"
echo "CONFIG_BUNDLE_FILE_NAME:      '${CONFIG_BUNDLE_FILE_NAME}'"
echo "CONFIG_BUNDLE_DOT_EXTENSION:  '${CONFIG_BUNDLE_DOT_EXTENSION}'"

Output is like this:

SCRATCH_CONFIG_BUNDLE_NAME:   '/scratch/downloads/528f9f5a-0123-f022-2b4d-a5c2e595c51a/configBundle-ESXi-X10SRH-CF.tgz'
CONFIG_BUNDLE_DIRECTORY_NAME: '/scratch/downloads/528f9f5a-0123-f022-2b4d-a5c2e595c51a/'
CONFIG_BUNDLE_FILE_NAME:      'configBundle-ESXi-X10SRH-CF'
CONFIG_BUNDLE_DOT_EXTENSION:  '.tgz'

Full backup-config-to-ESXi_configuration_backup-directory.sh script:

Read the rest of this entry »

Posted in *nix, *nix-tools, ash/dash, ash/dash development, cron/crontab, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

Some links on Excel format strings

Posted by jpluimers on 2022/01/25

In Get Formatted Value of Cell in Excel – Stack Overflow, I focused on the =TEXT function, then indicated I would look into Excel format strings later.

Below are just a few links and a very short description as hopefully later I will have more time to dig into this.

The basic format is this (where all bits other than Format1 are optional):

[Locale]Format1;Format2;Format3;Format4

For now this is for my link archive:

  • [Wayback] Excel number format strings (e.g. “@”, “$-409]d-mmm-yy;@”) – Complete reference availability? – Stack Overflow has two answers of which the second one is deleted, but very useful:
    1. Here’s a list of the Number Format Codes for Excel.
      • Second option for link: Ecma-376 4th Edition Part 1, Section 18.8.31
    2. If you are looking for the international number codes try at http://office.microsoft.com/en-gb/excel-help/creating-international-number-formats-HA001034635.aspx
  • The HA001034635 link has disappeared, which is a pity as it explains the localisation; luckily two forms of the URL have been saved where [Locale] is constructed like this example (all Locale digits are hexadecimal):

    [$-24050412]m/d/yy

    In the preceding example:

    24  is the numeral shape component (Korean 1).
    05  is the calendar type component (Korean (Tangun era)).
    0412  is the locale and language designation component (Korean).

    1. [Wayback] Archive.is: Creating international number formats – Excel … HA001034635.aspx
    2. [Wayback] Archive.is: Creating international number formats – Excel – Office.com … HA001034635.aspx?redir=0
  • More [Locale] information is explained in [Wayback] What does the 130000 in Excel locale code [$-130000] mean? – Stack Overflow by [Wayback] User IrwinAllen13 – Stack Overflow and has more calendar formats than the above Microsoft links.
  • [Wayback] Number format codes – Excel for Mac has the base and sort of explains what the @ is for: the content of the cell.

    First the base:

    When you create custom number formats, you can specify up to four sections of format code. These sections of code define the formats for

    1. positive numbers,
    2. negative numbers,
    3. zero values, and
    4. text,

    in that order. The sections of code must be separated by semicolons (;).

    The following example shows the four types of format code sections.

    Four types of format code sections

    Callout 1 Format for positive numbers

    Callout 2 Format for negative numbers

    Callout 3  Format for zeros

    Callout 4  Format for text

    Then on on specifying less than 4 sections:

    • If you specify only one section of format code, the code in that section is used for all numbers.
    • If you specify two sections of format code, the first section of code is used for positive numbers and zeros, and the second section of code is used for negative numbers.
    • When you skip code sections in your number format, you must include a semicolon for each of the missing sections of code.
    • You can use the ampersand (&) text operator to join, or concatenate, two values.

    Then on the @ sign (which is under “Text and spacing”):

    To create a number format that includes text that is typed in a cell, insert an “at” sign (@) in the text section of the number format code section at the point where you want the typed text to be displayed in the cell.

    For example, to include text before the text that’s typed in the cell, enter “gross receipts for “@ in the text section of the number format code.

    Finally it explains how to colorise the formatting or hide particular values under “Decimal places, spaces, colors, and conditions”:

    The color code must be the first item in the code section.

    [Black] [Blue] [Cyan] [Green] [Magenta] [Red] [White] [Yellow]

    Hiding various values:

    To hide Use this code
    Zero values 0;–0;;@
    All values ;;; (three semicolons)
  • [Wayback] ECMA-376 – Ecma International where Ecma-376 4th Edition Part 1, Section 18.8.31 is supposed to have format strings (will dig into this later)

    Office Open XML file formats – This Standard defines Office Open XML’s vocabularies and document representation and packaging

    1. [Wayback] ECMA-376, Part 1
    2. Not sure where Part 2 is
    3. [Wayback] ECMA-376, Part 3
    4. [Wayback] ECMA-376, Part 4
    5. Not sure where Part 5 is

Below a few screenshots while experimenting.

The first one shows the formulas, the second one the content. The Date value cells shows the values when entered as shown; the Date formatted cells are all formatted with yyyy-mm-dd;@ formatting.

I still need to figure out why using a =TEXT function shows #VALUE! whereas using cell formatting just a bunch of ############### (15 times a #, not sure if that is always the same number).

–jeroen

Posted in Development, Excel, Office, Office Development, Power User, Software Development | Leave a Comment »

Medische gegevens | LUMC

Posted by jpluimers on 2022/01/24

For my link archive:

Problem was that the privacy statement was not on a separate page during DigiD logon: you had to copy/paste to word in order to save it.

–jeroen

Posted in About, GDPR/DS-GVO/AVG, LifeHacker, Personal, Power User, Privacy | Leave a Comment »

 
%d bloggers like this: