January 2026
M	T	W	T	F	S	S
	1	2	3	4
5	6	7	8	9	10	11
12	13	14	15	16	17	18
19	20	21	22	23	24	25
26	27	28	29	30	31

Archive for the ‘ash/dash development’ Category

ESXi: getting and setting the host name, domain and fqdn

Posted by jpluimers on 2022/01/19

A few links and notes:

[Wayback] Changing the hostname of an ESX or ESXi host (1010821)
Run these commands to change the hostname in ESXi 5.x, ESXi 6.x,ESXi 7.x, using the command line:
- esxcli system hostname set –host=hostname
- esxcli system hostname set –fqdn=fqdn
[Wayback] ESX Host appears as localhost.localdomain in VMware Infrastructure/vSphere client (2009720)

Cause

The name resolution parameters were not properly configured during the installation of the ESX host.
[Wayback] Domain repoint for embedded vCenter Server fails with error: “domain_consolidator Failed to set machine id” (71020)

This issue is caused by a mismatch between the FQDN that was configured as the PNID during the vCenter Server deployment and the hostname that is currently configured.

I had a mismatch happen because of the second entry: a host configured in a different domain than it was deployed to.

Here are the commands to list and change the hosts name, domain and fqdn:

Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

ESXi: persistent files you can edit to apply settings during boot

Posted by jpluimers on 2022/01/18

Since ESXi boots from RAM, most files in /etc are not persisted after modification.

The files that are persisted, are only persisted once every hour by auto-backup.sh, so better run auto-backup.sh by hand if you want to reboot after changing them.

The auto-backup.sh script is ran every hour at 1 minute past the hour as per below crontab.

Default ESXi crontab in /var/spool/cron/crontabs/root:
#min hour day mon dow command
1    1    *   *   *   /sbin/tmpwatch.py
1    *    *   *   *   /sbin/auto-backup.sh
0    *    *   *   *   /usr/lib/vmware/vmksummary/log-heartbeat.py
*/5  *    *   *   *   /bin/hostd-probe.sh ++group=host/vim/vmvisor/hostd-probe/stats/sh
00   1    *   *   *   localcli storage core device purge
Schedules deciphered via [Wayback] Crontab.guru – The cron schedule expression editor:
[Wayback] Every day at 01:01: “This module removes stale temporary files”
1    1    *   *   *   /sbin/tmpwatch.py
[Wayback] Every hour at *:01: saves backup to /bootbank/state.tgz.
1    *    *   *   *   /sbin/auto-backup.sh
[Wayback] Every hour at *:00 logs heartbeat messages to /var/log/vmksummary.log like 2021-02-23T19:00:02Z heartbeat: up 577d2h37m16s, 9 VMs; [[2802426 vmx 4194304kB] [6176344 vmx 4194304kB] [68997 vmx 8388608kB]] [[2802426 vmx 0%max] [6176344 vmx 0%max] [68997 vmx 0%max]]
0    *    *   *   *   /usr/lib/vmware/vmksummary/log-heartbeat.py
[Wayback] Every 5th minute logs to /var/log/hostd-probe.log.
*/5  *    *   *   *   /bin/hostd-probe.sh ++group=host/vim/vmvisor/hostd-probe/stats/sh
[Wayback] Every day at 01:00: Removes storage devices which have not been seen in some time interval.
00   1    *   *   *   localcli storage core device purge
Note that localcli commands are the same as esxcli; for esxcli, a running hostd is required; localcli can run without hostd. See:

[Wayback] VCDX #200 Blog of one VMware Infrastructure Designer: ESXi localcli

[Wayback/Archive.is] Overview of vSphere Command-Line Interfaces – Getting Started with vSphere Command-Line Interfaces – VMware {code}

localcli commands

Set of commands for use with VMware Technical Support. localcli commands are equivalent to ESXCLI commands, but bypass the hoste daemon ( hostd). The localcli commands are only for situations when hostd is unavailable and cannot be restarted. After you run a localcli command, you must restart hostd. Run ESXCLI commands after the restart.

Warning: If you use a localcli command, an inconsistent system state and potential failure can result.

[Wayback] VMware ESXi 6.5 – ESXCLI Command Reference | virten.net

–jeroen

Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, Scripting, Software Development, Virtualization, VMware | Leave a Comment »

ESXi ash/dash/busybox shell getting current timestamp in UTC ISO8601 format without colons or dashes

Posted by jpluimers on 2022/01/17

On VMware ESXi, with the Busybox ash/dash shell, I wanted to get the current UTC timestamp in ISO 8601 format without dashes (-) or especially colons (:) and plus-signs (+) you have to back-slash escape colons or double quote parameters, which is often can be a pain).

[Wayback] How to Manipulate Filenames Having Spaces and Special Characters in Linux describes the pain handling special characters:

In this article, we will see how to create, copy, move and delete filenames that starts with spaces and special characters (say #, *, &, =, etc.) in Linux.
[Wayback] filenames – What characters are safe in cross-platform file names for Linux, Windows and OS-X – Super User
…
- []()^ #%&!@:+={}'~ and [`] all have special meanings in many shells, and are annoying to work around, and so should be avoided. They also tend to look horrible in URLs.
…

This is why we can’t have good things: Getting the UTC 8610 timestamp was far less easy than I hoped for.

First of all, Busybox only allows for a precision of seconds, not milliseconds, and the specification format needs better documentation as per [Wayback] embedded linux – How to get ISO8601 seconds format from “date” in busybox? – Stack Overflow:

Read the rest of this entry »

Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, Scripting, sed, sed script, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

Busybox ash/dash – Hexadecimal To Decimal in Shell Script (via Stack Overflow)

Posted by jpluimers on 2021/12/21

This works fine on “BusyBox v1.29.3 (2019-05-21 15:22:06 PDT) multi-call binary.” that is included with VMware ESXi 6.5 update 3:

[Wayback] bash – Hexadecimal To Decimal in Shell Script – Stack Overflow

Dealing with a very lightweight embedded version of busybox on Linux means many of the traditional commands are not available (bc, printf, dc, perl, python)
echo $((0x2f))
47

hexNum=2f
echo $((0x${hexNum}))
47
Credit to [Wayback] Peter Leung for this solution.

–jeroen

Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, Power User, Scripting, Software Development | Leave a Comment »

On my list of things to try: Python with ESXi

Posted by jpluimers on 2021/10/28

After doing a lot of – historically grown – dash scripting for ESXi, I found out there is Python available on ESXi:

Python 3.5.10 on VMware ESXi 6.7.0 build-17700523 (VMware ESXi 6.7.0 Update 3)
Python 3.5.6 on VMware ESXi 6.5.0 build-13932383 (VMware ESXi 6.5.0 Update 3)
VMware 7: to be determined.

Yes I know that Python 3.5 is end-of-life (and 3.5.10 was the latest version), but it is a lot better than shell scripts.

So now some links for my list of things to try in order to use Python for scripting ESXi operations:

[Wayback] virtualization – Accessing vSphere ESX/ESXi server through python – Server Fault answer by [Wayback] Yohann:

Take a look at the pyVmomi project which is the official Python SDK for the VMware vSphere API that allows you to manage ESX, ESXi and vCenter.

Moreover, the pyvmomi-community-samples repository contains lot of examples.
[Wayback] vmware – List all vm of ESXI, vsphere with python – Stack Overflow

Also, checkout the pyvmomi Community Samples repo. It has all kinds of great samples, including one that could fit this request: github.com/vmware/pyvmomi-community-samples/blob/master/samples/… – [Wayback] Kyle Ruddy
[Wayback] Get All VMs – Samples – VMware {code}
[Wayback] pyvmomi-community-samples

Welcome to the pyvmomi-community-samples project

This is a place for anyone to learn about working with the vSphere Management SDK’s native python binding library pyVmomi. Samples here are contributed by developers like you and curated by your fellow developers. The quality and validity of the samples will vary, however, if you see a problem report it! If you can solve someone’s problem, contribute a fix! If you need a new sample ask for one!
[Wayback] vmware/pyvmomi: VMware vSphere API Python Bindings

pyVmomi is the Python SDK for the VMware vSphere API that allows you to manage ESX, ESXi, and vCenter.
[Wayback] pyvmomi-community-samples/samples at master · vmware/pyvmomi-community-samples
Executable Samples

This directory contains a collection of executable sample scripts that use the pyVmomi library. There is a tools directory that holds a collection of tools and a tests directory that holds the tests for those tools.

Quality and License

Scripts are provided as-is by numerous contributors and the status of any sample at any point is subject to change. The project is intended as a collaborative exercise in community learning and may not contain best practice methods.

All samples revert to the license of the project and all ownership reverts to the community project.

Contribution Notes
- If a script is in this directory, it is an executable sample.
- conform to pep8
- avoid using any special tools beyond pyVmomi.
- do not extend the pyVmomi API in this project we have two separate projects dedicated to that.
- tests are appreciated but optional because of this sample quality must be manually assessed bug reports and fixes are much appreciated.
- A reviewer will pull a new sample for testing and will attempt to run the sample. If the reviewer can do this, the sample can be merged
Getting Help
- Question can be opened as issues at https://github.com/vmware/pyvmomi-community-samples/issues
- The IRC forum #pyvmomi is for developers working with pyVmomi
of which there are many, for instance:
- [Wayback] pyvmomi-community-samples/getallvms.py at master · vmware/pyvmomi-community-samples
- [Wayback] pyvmomi-community-samples/virtual_machine_device_info.py at master · vmware/pyvmomi-community-samples
- [Wayback] pyvmomi-community-samples/samples/tools at master · vmware/pyvmomi-community-samples
  
  Any reusable classes or methods that you develop can be included in packages under this directory. When adding a reusable component we highly recommend that you include tests.
  
  Only some samples will make use of reusable components. This is where to put those.
[Wayback] How did I get started with the vSphere Python SDK “pyvmomi” on ubuntu distro? | vThinkBeyondVM
[Wayback] pyVmomi tutorial : How to get all the core vCenter server inventory objects and play around? | vThinkBeyondVM

–jeroen

Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, Power User, Python, Scripting, Software Development | Leave a Comment »

Some bash parameter propagation links that hopefully will work with ash/dash too

Posted by jpluimers on 2021/10/27

For my link archive; I started with [Wayback] dash get all parameters quoted – Google Search:

[Wayback] Handling positional parameters [Bash Hackers Wiki] has this handy table:

Parameter(s) Description

$0 the first positional parameter, equivalent to argv[0] in C, see the first argument

$FUNCNAME the function name (attention: inside a function, $0 is still the $0 of the shell, not the function name)

$1 … $9 the argument list elements from 1 to 9

${10} … ${N} the argument list elements beyond 9 (note the parameter expansion syntax!)

$* all positional parameters except $0, see mass usage

$@ all positional parameters except $0, see mass usage

$# the number of arguments, not counting $0

These positional parameters reflect exactly what was given to the script when it was called.

Option-switch parsing (e.g. -h for displaying help) is not performed at this point.

See also the dictionary entry for “parameter”.

[Wayback] Handling positional parameters: mass usage [Bash Hackers Wiki] has this summary table:

Syntax Effective result

$* $1 $2 $3 … ${N}

$@ $1 $2 $3 … ${N}

"$*" "$1c$2c$3c…c${N}"

"$@" "$1" "$2" "$3" … "${N}"
[Wayback] Parameter expansion [Bash Hackers Wiki]
The elaborate answer to [Wayback] How to pass all arguments passed to my bash script to a function of mine? – Stack Overflow by [Wayback] User Gordon Davisson – Stack Overflow of which the first part is:
The $@ variable expands to all command-line parameters separated by spaces. Here is an example.
```
abc "$@"
```
When using $@, you should (almost) always put it in double-quotes to avoid misparsing of arguments containing spaces or wildcards (see below). This works for multiple arguments. It is also portable to all POSIX-compliant shells.

It is also worth nothing that $0 (generally the script’s name or path) is not in $@.

The Bash Reference Manual Special Parameters Section says that $@ expands to the positional parameters starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word. That is "$@" is equivalent to "$1" "$2" "$3"....
Last part of the [Wayback] Accessing bash command line args $@ vs $* – Stack Overflow answer by [Wayback] Serge Stroobrandt:

If the arguments are to be stored in a script variable and the arguments are expected to contain spaces, I wholeheartedly recommend employing a "$*" trick with the input field separator set to tab IFS=$'\t' [Wayback].

and comment by Serge as well:

Here is [Wayback] an example, which includes quoted input. The input also matters!

Parameter(s)	Description
`$0`	the first positional parameter, equivalent to `argv[0]` in C, see the first argument
`$FUNCNAME`	the function name (attention: inside a function, `$0` is still the `$0` of the shell, not the function name)
`$1 … $9`	the argument list elements from 1 to 9
`${10} … ${N}`	the argument list elements beyond 9 (note the parameter expansion syntax!)
`$*`	all positional parameters except `$0`, see mass usage
`$@`	all positional parameters except `$0`, see mass usage
`$#`	the number of arguments, not counting `$0`

Syntax	Effective result
`$*`	`$1 $2 $3 … ${N}`
`$@`	`$1 $2 $3 … ${N}`
`"$*"`	`"$1c$2c$3c…c${N}"`
`"$@"`	`"$1" "$2" "$3" … "${N}"`

[Wayback] Propagate all arguments in a bash shell script – Stack Overflow (thanks [Wayback] Sdaz MacSkibbons for the answer and [Wayback] Ilan Schemoul to make it understandable by applying proper naming of the script files):

Use "$@" instead of plain $@ if you actually wish your parameters to be passed the same.

Observe:

$ cat no_quotes.sh
#!/bin/bash
echo_args.sh $@

$ cat quotes.sh
#!/bin/bash
echo_args.sh "$@"

$ cat echo_args.sh
#!/bin/bash
echo Received: $1
echo Received: $2
echo Received: $3
echo Received: $4

$ ./no_quotes.sh first second
Received: first
Received: second
Received:
Received:

$ ./no_quotes.sh "one quoted arg"
Received: one
Received: quoted
Received: arg
Received:

$ ./quotes.sh first second
Received: first
Received: second
Received:
Received:

$ ./quotes.sh "one quoted arg"
Received: one quoted arg
Received:
Received:
Received:

–jeroen

Posted in *nix, *nix-tools, ash/dash, ash/dash development, bash, bash, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

ESXi: listing virtual machines with their IP addresses

Posted by jpluimers on 2021/10/26

This is sort of a follow-up on VMware ESXi console: viewing all VMs, suspending and waking them up: part 4 which already gave part of the configuration details of all the configured VMs.

Back then, we ended with this:

List the `vmid` values, power status and name of all VMs

Back to the listing script vim-cmd-list-all-VMs.sh:

#!/bin/sh
# https://wiert.me/2021/04/29/vmware-esxi-console-viewing-all-vms-suspending-and-waking-them-up-part-4/
vmids=`vim-cmd vmsvc/getallvms | sed -n -E -e "s/^([[:digit:]]+)\s+((\S.+\S)?)\s+(\[\S+\])\s+(.+\.vmx)\s+(\S+)\s+(vmx-[[:digit:]]+)\s*?((\S.+)?)$/\1/p"`
for vmid in ${vmids} ; do
    powerState=`vim-cmd vmsvc/power.getstate ${vmid} | sed '1d'`
    name=`vim-cmd vmsvc/get.config ${vmid} | sed -n -E -e '/\(vim.vm.ConfigInfo\) \{/,/files = \(vim.vm.FileInfo\) \{/ s/^ +name = "(.*)",.*?/\1/p'`
    vmPathName=`vim-cmd vmsvc/get.config ${vmid} | sed -n -E -e '/files = \(vim.vm.FileInfo\) \{/,/tools = \(vim.vm.ToolsConfigInfo\) \{/ s/^ +vmPathName = "(.*)",.*?/\1/p'`
    echo "VM with id ${vmid} has power state ${powerState} (name = ${name}; vmPathName = ${vmPathName})."
done

It uses vim-cmd vmsvc/getallvms, vim-cmd vmsvc/power.getstate and vim-cmd vmsvc/get.config with some sed and a for loop from dash to generate a nice list of information.

A long time ago, I already figured out that vim-cmd vmsvc/get.guest # gives all guest information including network information for a running VM that has either VMware Tools or open-vm-tools running (see VMware ESXi console: viewing all VMs, suspending and waking them up: part 3 for the difference between these two tools).

A full output of a sample VM is below the signature.

There are a few places that have the LAN ipAddress. For now, I choose to use only the IPv4 main address from ipAddress, which is in between (vim.vm.GuestInfo) { and net = (vim.vm.GuestInfo.NicInfo) [.

I modified the above script to become this:

Read the rest of this entry »

Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, find, Power User, Scripting, sed, sed script, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

ESXi shell: appending the parent directory of a script to the path and starting a new shell, even if the script is symlinked

Posted by jpluimers on 2021/10/26

I needed a way to append the directory of a script to the path as all my tool scripts are in there, and I did not want to modify any profile scripts as these might be modified during ESXi upgrade.

First you need the full script filename through readlink then toe parent directory name through dirname:

- [Wayback] string – bash : Bad Substitution – Stack Overflow
```
me@pc:~$ readlink -f $(which sh)
/bin/dash
```
  By [Wayback] Vanni Totaro.
- [Wayback] shell – Getting parent’s directory name by piping the results of dirname to basename in a Bash script – Stack Overflow
```
# echo 'test/90_2a5/Windows' | xargs dirname | xargs
basename90_2a5
```
  By [Wayback] Rafael.

Note there might be dragons with more symlinks or different shells:

I created the script below. It is not perfect, but for my situation it gets the job done.

If you do not start a new shell, then the export is lost as a new dash shell process is started for each script that runs from the terminal or console.

# cat /opt/bin/append-script-directory-to-path-and-start-new-shell.sh
#!/bin/sh
# Absolute path to this script, e.g. /home/user/bin/foo.sh
# echo "'$0'"
SCRIPT=$(readlink -f "$0")
# Absolute path this script is in, thus /home/user/bin
SCRIPTPATH=$(dirname "$SCRIPT")
# echo Appending to $PATH: $SCRIPTPATH
export PATH=$PATH:$SCRIPTPATH
sh

–jeroen

“fixing” ESXi “rsync error: error allocating core memory buffers (code 22) at util2.c(106) [sender=3.1.2]”

Posted by jpluimers on 2021/08/30

Reminder to self: create a static ESXi binary for a recent rsync release.

Quite a few people have bumped into rsync erroring out with “large” sets of files (where large can be as low as ~1000), like for instance Tj commenting on my post “ESXi 5.1 and rsync – damiendebin.net.”:

ERROR: out of memory in receive_sums [sender] │······
rsync error: error allocating core memory buffers (code 22) at util2.c(102) [sender=3.1.1] │······
rsync: [generator] write error: Broken pipe (32) │······

I bumped into this myself as well, even when updating from rsync 3.1.0 to 3.1.2.

There are various static rsync for ESXi around. Just a few of them for completeness:

3.1.0: ESXi 5.1 and rsync – damiendebin.net. (how to download the build by Damien Debin from http://damiendebin.net and how to create the right XML firewall settings from his [Wayback] gist)

3.1.2: [Wayback] DOWNLOADS – bachmann-lan.de via [Wayback] VMware ESXi 5.1 rsync 3.0.9 statically linked binary erstellen – bachmann-lan.de

3.1.3: [Wayback/Archive.is] noelmartinon/vmtools: Tools for VMware ESXi to use in ESXi which pointed at the wrong binary link, so I archived the right one in the Wayback machine.

There is also 3.0.9 (via [Wayback] VMware ESXi 5.1 rsync 3.0.9 statically linked binary erstellen – bachmann-lan.de), but it has a VMFS bug ([Wayback] 8177 – Problems with big sparsed files) as per [Wayback] ESXi 5.1 and rsync – damiendebin.net.)

The good news is that it is fixed in 3.2.2 as a user-configurable setting, but since there is no ESXi build yet (see reminder above)…

Anyway: [Wayback] 12769 – error allocating core memory buffers (code 22) depending on source file system

Wayne Davison 2020-06-26 03:56:35 UTC

I fixed the allocation args to be size_t values (and improved a bunch of allocation error checking while I was at it).

I then added an option that lets you override this allocation sanity-check value. The default is still 1G per allocation, but you can now specify a much larger value (up to "--max-alloc=8192P-1").

If you want to make a larger value the default for your copies, export RSYNC_MAX_ALLOC in the environment with the size value of your choice.

Committed for release in 3.2.2.

This is what happens with 3.1.2 and 3.1.3:

time rsync -aiv --info=progress2 --progress --partial --existing --inplace /vmfs/volumes/Samsung850-2TB-S3D4NX0HA01043L/ Samsung850-2TB-S3D4NX0HA01043L/
sending incremental file list
              0   0%    0.00kB/s    0:00:00 (xfr#0, ir-chk=1000/1259)
ERROR: out of memory in flist_expand [sender]
rsync error: error allocating core memory buffers (code 22) at util2.c(106) [sender=3.1.2]
Command exited with non-zero status 22
real    0m 0.87s
user    0m 0.10s
sys 0m 0.00s
time rsync -aiv --info=progress2 --progress --partial --ignore-existing --sparse /vmfs/volumes/Samsung850-2TB-S3D4NX0HA01043L/ Samsung850-2TB-S3D4NX0HA01043L/
sending incremental file list
              0   0%    0.00kB/s    0:00:00 (xfr#0, ir-chk=1000/1259)
ERROR: out of memory in flist_expand [sender]
rsync error: error allocating core memory buffers (code 22) at util2.c(106) [sender=3.1.2]
Command exited with non-zero status 22
real    0m 0.28s
user    0m 0.12s
sys 0m 0.00s
Finished

I was lucky that [Wayback] “rsync error: error allocating core memory buffers” protocol version “3.1.2” – Google Search got me a result so quickly: add a --protocol-29 and you are set.

The first result (Wayback has the results reversed from what got) didn’t fix it. The second did.

[Wayback] 225761 – net/rsync long path causes buffer overflow (update to 3.1.3)
[Wayback/Archive.is] AIX Open Source – IBM Power Systems Community: rsync out of memory

As a work around, I added “--protocol=29” to one of our servers that was consistently failing with “ERROR: out of memory in flist_expand [receiver]” “rsync error: error allocating core memory buffers (code 22) at util2.c(105) [receiver=3.1.3]” in rsync-3.1.3-2.ppc

I read the man page and started experimenting with the protocol version until I lowered it enough to get it to work consistently.

The problem might be that running on the ESXi gives you limited memory, but then some 10k files should not use more than like half a megabyte of memory.

Sometime I will dig deeper into the protocol version differences, for now a list of files I think will be relevant for that (mainly look for protocol_version):

Some web pages mentioning the --protocol option and might give me more insight in the protocol differences:

With --protocol=29, time estimation is way off, but there are no errors:

time rsync -aiv --info=progress2 --progress --partial --existing --inplace --protocol=29 /vmfs/volumes/Samsung850-2TB-S3D4NX0HA01043L/ Samsung850-2TB-S3D4NX0HA01043L/
building file list ... 
9059 files to consider
.d..t...... isos/
         27,593   0%    0.00kB/s    0:00:06 (xfr#1, to-chk=0/9059)   

sent 212,594 bytes  received 268 bytes  20,272.57 bytes/sec
total size is 3,055,677,645,398  speedup is 14,355,204.99
real    0m 13.31s
user    0m 1.35s
sys 0m 0.00s

time /vmfs/volumes/5791a3e1-0b9368de-4965-0cc47aaa9742/local-bin/rsync -aiv --info=progress2 --progress --partial --ignore-existing --sparse --protocol=29 /vmfs/volumes/Samsung850-2TB-S3D4NX0HA01043L/ Samsung850-2TB-S3D4NX0HA01043L/
building file list ... 
9059 files to consider
>f+++++++++ isos/EN-Windows-XP-SP3-VL.iso
...
cd+++++++++ ESXi65.filesystem-root/usr/share/
216,868,164,639   7%   40.64MB/s    1:24:48 (xfr#2571, to-chk=0/9059)   

sent 216,894,938,870 bytes  received 57,858 bytes  42,582,702.80 bytes/sec
total size is 3,055,677,645,398  speedup is 14.09
real    1h 24m 58s
user    34m 5.59s
sys 0m 0.00s
Finished

Even not on ESXi, there were just a few people bumping into this, so I wonder why there are so few matches on [Wayback] “ERROR: out of memory in flist_expand [sender]” “sender=3.1” – Google Search:

RHEL rsync.x86_64 3.1.2-6.el7_6.1 [Wayback] 1751822 – rsync failing with out of memory in flist_expand (Redhat denied fix effort because “Red Hat Enterprise Linux version 7 entered the Maintenance Support 1 Phase in August 2019”)
Cygwin rsync 3.0.9:
- [Wayback] Richard Ivarson – Re: rsync ERROR: out of memory in flist_expand [sender]
- [Wayback] Bruno Galindro da Co – rsync ERROR: out of memory in flist_expand [sender]

–jeroen

Posted in *nix, *nix-tools, ash/dash development, Development, ESXi5, ESXi5.1, ESXi5.5, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, rsync, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

ESXi: where are my log files actually stored? Actually, most of them are in `/scratch/log` which points to a hidden `.locker` directory in a datastore.

Posted by jpluimers on 2021/05/05

A summary of the full gist at [Wayback/Archive.is] ESXi-where-are-my-log-files-stored.txt:

# ls -al / /var/ /var/log/ /var/run/ /scratch/ /scratch/log/ | grep "/\|log\|-\>"
/:
...
lrwxrwxrwx    1 root     root            57 Apr  4 18:16 scratch -> /vmfs/volumes/5ce2d440-72311161-75c5-0025907d9d5c/.locker
...
/scratch/:
drwxr-xr-x    1 root     root        106496 Apr 10 08:40 log
/scratch/log/:
-rw-------    1 root     root           411 Apr  4 18:20 Xorg.log
...
-rw-------    1 root     root         78835 Apr  4 10:30 syslog.0.gz
-rw-------    1 root     root         61136 Mar 18 15:05 syslog.1.gz
-rw-------    1 root     root         60589 Feb 24 00:30 syslog.2.gz
-rw-------    1 root     root         60373 Feb  1 08:01 syslog.3.gz
-rw-------    1 root     root         60203 Jan  9 15:50 syslog.4.gz
-rw-------    1 root     root         59889 Dec 17 23:20 syslog.5.gz
-rw-------    1 root     root         60398 Nov 25 06:50 syslog.6.gz
-rw-------    1 root     root         60563 Nov  2 14:25 syslog.7.gz
-rw-------    1 root     root        531794 Apr 10 09:35 syslog.log
...
-rw-------    1 root     root        157255 Apr  4 18:17 vvold.log
/var/:
...
drwxr-xr-x    1 root     root           512 Apr  5 19:19 log
...
/var/log/:
-rw-r--r--    1 root     root           416 Apr  4 18:16 .vmsyslogd.err
...
-rw-r--r--    1 root     root         38069 Apr  4 18:20 configRP.log
-rw-r--r--    1 root     root             0 Apr  4 18:16 cryptoloader.log
...
-rw-r--r--    1 root     root            87 Apr  5 21:57 esxcli.log
...
-rw-------    1 root     root          3350 Apr  4 18:16 init.log
-rw-r--r--    1 root     root           966 Apr  4 18:16 iofilter-init.log
...
-rw-r--r--    1 root     root         21769 Apr  4 18:16 jumpstart-esxcli-stdout.log
-rw-r--r--    1 root     root         18857 Apr  4 18:16 jumpstart-native-stdout.log
-rw-r--r--    1 root     root         10837 Apr  4 18:16 jumpstart-stdout.log
-rw-r--r--    1 root     root             0 Apr  4 18:16 kickstart.log
...
-rw-------    1 root     root         10916 Apr  4 18:16 sysboot.log
...
-rw-------    1 root     root            64 Apr 10 09:13 tallylog
...
/var/run/:
lrwxrwxrwx    1 root     root            12 Apr  4 18:16 log -> /scratch/log

Almost all log files (most from /var/log and all from /var/run/log) are actually persistently stored in /scratch/log and survive reboots. Just a few are non-persistent.
/var/log/syslog is being archived as .gz files (compressed by gzip).
syslog is special: the location can be configured, and even be external: [Wayback] Configuring syslog on ESXi (2003322)

VMware vSphere ESXi 5.0 and higher hosts run a Syslog service (vmsyslogd) that provides a standard mechanism for logging messages from the VMkernel and other system components. By default in ESXi, these logs are placed on a local scratch volume or a ramdisk. To preserve the logs further, ESXi can be configured to place these logs to an alternate storage location on disk and to send the logs across the network to a Syslog server.
A summary of some of the above log files is at [Wayback] ESXi Log File Locations

–jeroen

Posted in *nix, *nix-tools, ash/dash, ash/dash development, BusyBox, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, gzip, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

« Previous Entries

Next Entries »

	Lars Fosdal on Security alarm provider Woonve…
	Thomas Mueller on Question got closed in May 202…
	Thaddy de Koning on Formulier voor bewindvoerders…
	Thaddy de Koning on Formulier voor bewindvoerders…
	Thaddy de Koning on Formulier voor bewindvoerders…

The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

Subscribe

Archives

Recent Comments

Recent Posts

Blog Stats

Meta title

Tag Cloud Title

Top Clicks

Top Posts

My badges

Twitter Updates

My Flickr Stream

Pages

All categories

Email Subscription

Archive for the ‘ash/dash development’ Category

ESXi: getting and setting the host name, domain and fqdn

Cause

ESXi: persistent files you can edit to apply settings during boot

ESXi ash/dash/busybox shell getting current timestamp in UTC ISO8601 format without colons or dashes

Busybox ash/dash – Hexadecimal To Decimal in Shell Script (via Stack Overflow)

On my list of things to try: Python with ESXi

Welcome to the pyvmomi-community-samples project

Executable Samples

Quality and License

Contribution Notes

Getting Help

Some bash parameter propagation links that hopefully will work with ash/dash too

ESXi: listing virtual machines with their IP addresses

List the `vmid` values, power status and name of all VMs

ESXi shell: appending the parent directory of a script to the path and starting a new shell, even if the script is symlinked

“fixing” ESXi “rsync error: error allocating core memory buffers (code 22) at util2.c(106) [sender=3.1.2]”

ESXi: where are my log files actually stored? Actually, most of them are in `/scratch/log` which points to a hidden `.locker` directory in a datastore.

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

Subscribe

Archives

Recent Comments

Recent Posts

Blog Stats

Meta title

Tag Cloud Title

Top Clicks

Top Posts

My badges

My Flickr Stream

Pages

All categories

Email Subscription

Archive for the ‘ash/dash development’ Category

Cause

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Welcome to the pyvmomi-community-samples project

Executable Samples

Quality and License

Contribution Notes

Getting Help

Rate this:

Share this:

Rate this:

Share this:

List the vmid values, power status and name of all VMs

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

List the `vmid` values, power status and name of all VMs