The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,861 other subscribers

Archive for the ‘VMware ESXi’ Category

Downloading files with wget on ESXi · random writes

Posted by jpluimers on 2021/05/31

Reminder to self to check if wget on ESXi now finally supports https downloading: [WayBack] Downloading files with wget on ESXi · random writes.

In the mean time, ESXi 6.7 Update 2 and up seems to support this; so the below workaround might only be needed for ESXi 6.7 update 1 and below.

[WayBack] VMware ESXi: help downloading large ISO – Server Fault

I will likely not do this, as by now all my ESXi boxes should have been recent enough.

I will keep the article because of the BusyBox commands section below.

If so, I might finally try and write a Python wrapper for this, as I know that Python 3 on ESXi supports https, but the ESXi BusyBox does not have a built-in cURL.

Some links and notes I might need by then:

BusyBox commands

Another cool thing in the above blog post is that it shows how to dump the BusyBox built in commands.

I ran it for ESXi 6.7 with a slight trick to get the full path (using back-ticks and escaped back-ticks) and content.

Since ESXi is BusyBox based, the commands that are in /bin are not actually binaries, but each command is a symlink to the BusyBox binary. BusyBox then knows the original name of the command, so it can deduct what part to execute. This makes for a very space efficient storage scheme.

The various bits of the tricks to get the location of the BusyBox binary, so the --list parameter can be passed to it:

  • The which wget gives the full path of wget.
  • The ls -l `which wget` shows the full path of wget and the symlink target (but there is no way for ls to only show the symlink target).
  • The readlink -f `which wget` shows the full path of where /bin/wget points to: the BusyBox binary.

The main trick consists of backtick evalution, and knowing that ls cannot get you just the symlink target, but readlink can:

Now the back-tick escapes, because you cannot nest back-ticks:

  • The `readlink -f \`which wget\`` executes the BusyBox binary without arguments.
  • The `readlink -f \`which wget\`` --list executes the BusyBox binary with the --list parameter.

Note I do not like the cat --help (see [WayBack] How do I check busybox version (from busybox)? – Unix & Linux Stack Exchange) way of getting the BusyBox version, as that gets echoed to stderr.

This is the output:

Read the rest of this entry »

Posted in *nix, *nix-tools, cURL, ESXi6, ESXi6.5, ESXi6.7, Power User, Virtualization, VMware ESXi, wget | Leave a Comment »

“FIPS mode initialized” when you ssh out of an ESXi box

Posted by jpluimers on 2021/05/28

The once per console/shell logon output of FIPS mode initialized to stderr when you ssh out of an ESXi box seems to be something new since ESXi 6.7.

Since I hardly do this, it took a while to reproduce and track back the version where it was introduced and to realise why it is on stderr.

stderr in retrospect is logical: if you need to parse stdout of a job running across an ssh channel, you do not want it to get interfered with “side channel” output, hence stderr.

For a longer explanation see, for instance [WayBack] ssh “FIPS mode initialized” message to stderr – Why? – Unix and Linux | DSLReports Forums:

Keep in mind that “ssh” is used to transport a stream, as with “rsync”. What you put on “stdout” becomes part of the stream. That’s why this sort of informational message needs to go to “stderr”.

Parsing is hard, so bugs like [WayBack] Git fetcher fails on machine with FIPS enabled machines · Issue #3664 · inspec/inspec · GitHub got [WayBack] fixed in [WayBack] pull request like [WayBack] not parsing stderr, but checking for exitstatus.

Stock OpenSSH portable does not contain FIPS support

Finding back when and how FIPS support for OpenSSH was introduced provide a bit harder than I hoped for.

It appears that stock [WayBack] OpenSSH: Portable Release does not support FIPS. But there are patches on top of these files:

Many (most?) Linux distributions include a patched version like [WayBack] ssh.c in openssh located at /openssh-5.9p1 (git://pkgs.fedoraproject.org/openssh).

They integrate the patches like [WayBack] File openssh.spec of Package openssh – openSUSE Build Service.

Patches for instance look like [WayBack] openssh/openssh-5.3p1-fips.patch at master · gooselinux/openssh · GitHub which is more than a decade old (see the 2009 message [WayBack] rpms/openssh/devel openssh-5.3p1-fips.patch, NONE, 1.1 openssh-5.3p1-mls.patch, NONE, 1.1 openssh-5.3p1-nss-keys.patch, NONE, 1.1 openssh-5.3p1-selabel.patch, NONE, 1.1 openssh-5.3p1-skip-initial.patch, NONE, 1.1 .cvsignore, 1.24, 1.25 openssh.spec, 1.170, 1.171 sources, 1.24, 1.25 openssh-3.8.1p1-krb5-config.patch, 1.1, NONE openssh-4.7p1-audit.patch, 1.2, NONE openssh-5.1p1-mls.patch, 1.1, NONE openssh-5.1p1-skip-initial.patch, 1.1, NONE openssh-5.2p1-fips.patch, 1.6, NONE openssh-5.2p1-nss-keys.patch, 1.3, NONE openssh-5.2p1-selabel.patch, 1.2, NONE).

The patches seem to originate at the (now defunct) WayBack Index of /export/openssh of http://openssl.com/export/openssh/ .

In the end I found [WayBack] Mailing List Archive: OpenSSH FIPS 140-2 support using OpenSSL FIPS modules? having these quotes:

vanilla OpenSSH doesn’t support running OpenSSL in FIPS-140 mode. Some
downstream providers patch OpenSSH they deliver with their distributions
with changes to enable FIPS-140 mode.

[WayBack] Secure Shell and FIPS 140-2 – Managing Secure Shell Access in Oracle® Solaris 11.4 explains a bit of background of them.

ESXi 6.7

Binary searching for the version where this was introduced could have been a lot shorter if I had done a “FIPS mode initialized” “ESXi” – Google Search, resulting in for instance:

The final two links made me discover XSIBackup

They see be one of the few (only one?!) free backup solutions for the bare ESXi:

In addition, they have a binary for rsync version 3.1.0: [WayBack] 33HOPS | Rsync for VMWare Backup, so lees need to go to Source: ESXi 5.1 and rsync – damiendebin.net

jeroen

Posted in *nix, *nix-tools, ESXi6.5, ESXi6.7, Power User, ssh/sshd, Virtualization, VMware, VMware ESXi | Leave a Comment »

Forgot the ESXi root password? No problems, here are 4 ways to reset it! – VMWARE BLOG

Posted by jpluimers on 2021/05/24

I only needed one of the standalone ways for the many ways in [WayBack] Forgot the ESXi root password? No problems, here are 4 ways to reset it! – VMWARE BLOG

Passwords are the things people tend to forget. Well, ESXi root passwords are not an exception either! Without the root password, you lose control over your hosts, so it’s good to know how to reset it. Well, resetting an ESXi host password is the thing I gonna talk about in this article.

Resetting root password on the standalone ESXi hosts

Now, as we know how to reset the password with vCenter, let’s look at some tough cases. Let’s say, you don’t have vCenter installed on the host. Once again, I do not want to re-install the server OS as VMware says. Seriously, that’s not fun! Let’s look at something more interesting instead. Well, let’s say, what about changing the password right on the node itself?

Before I start, I’d like to mention that you won’t be able to trick ESXi security and change the root password on the node without shutting it down. This means that you, like it or not, do need to shut down each VM from the inside! If you screw things up, you won’t be able to start VMs without ESXi re-installation.

Also, you need the boot the CD image. I used Ubuntu GNOME in this article. Find out how to create a boot CD and download Ubuntu GNOME here. You also need Rufus to write the boot CD image on the flash drive.

C:\21a983d22b51938355d6c52e7f69741e

So, you need to boot from the flash disk, mount the required ESXi datastore, unpack the archive, and edit the file with passwords. Next, you upload the file back into the initial directory, and, after rebooting the host, you can access the it without the password.

Editing the “shadow” file

What’s “shadow” is?

For safety concerns, ESXi keeps passwords encrypted in some file… whatever, here’s how you still can reset the password. According to some unofficial sources, this file is called “shadow”. You can find it in one of those booting volumes in the /etc directory. Before the host boots, /etc is in the local.tgz archive. Here’s the path: /etc => local.tgz => state.tgz. You can find it in one of those booting volumes in the /etcdirectory. Before the host boots, /etc is in the local.tgz archive. Here’s the path: state.tgz => local.tgz => /etc.

Here’s how the disk is formatted in ESXi 6.0 or higher:

Volume name What it is for? Volume size in my case
/dev/sda1 Starts the system 4 MB
/dev/sda2: /scratch: System volume that is created while installing ESXi on the over-5 GB disk. 4 GB
/dev/sda3: VMFS datastore: Represents all the remaining disk space
/dev/sda5: /bootbank: The ESXi image 250 MB
/dev/sda6: /altrbootbank: The older system version image. You’ll see it as an empty volume if you have never updated the system 250 MB
/dev/sda7: vmkDiagnostic (the first volume) Keeps the core dump 110 MB
/dev/sda8: /store VMware Tools image 286 MB
/dev/sda9: vmkDiagnostic (the second volume) Keeps all the information connected with vSAN diagnostics. You can observe this volume only in over-8 GB datastores 2.5 GB

Among of all those volumes, we need only the /bootbank one as it keeps the ESXi archive. In this way, “shadow” should be somewhere there.

Chasing the “shadow”

So, let’s boot the host from the flash disk first and start the terminal.

Run the following cmdlet to acquire root privileges:

# sudo su

Next, deploy the command below to look through the sda directory.

# fdisk –l | grep /dev/sda*

C:\c7eb70e4332b280e897bc91da2843eb5

Well, it seems that we need that 250 MB /dev/sda5 directory. Create the mnt directory.

# mkdir /mnt/sda5

Create the directory for the temporary files now.

# mkdir /temp

And, mount the /dev/sda5 directory using the cmdlet below.

# mount /dev/sda5 /mnt/sda5

Now, look for that state.tgz archive I was talking above.

# ls -l /mnt/sda5/state.tgz

Extract both state.tgz and local.tgz. Here are the commands you can use for that purpose:

# tar -xf /mnt/sda5/state.tgz –C /temp/

# tar -xf /temp/local.tgz –C /temp/

Once you are done with unpacking, get rid of those old archives with the cmdlet below:

# rm /temp/local.tgz

Now, you are ready to do some magic with “shadow”. Open the file, edit it, and close it. As simple as it! To double-check the changes, open the file one more time.

# vi /temp/etc/shadow

Actually, here’s how “shadow” looks like inside. See, it contains all users’ passwords.

C:\5cfa53db6df27f3419c38304e61a1937

To reset the password, just delete everything between the double colons. Remember, everything is encrypted? That’s why passwords look that weird.

C:\569ce0a0bd6088cfe538f3b76c1872b3

# vi /temp/etc/shadow

Next, go to the work directory.

# cd /temp

Now, add the “shadow” back to the archive.

# tar -czf local.tgz etc

# tar -czf state.tgz local.tgz

Move the new archive to the initial directory.

# mv state.tgz /mnt/sda5/

Unmount the /sda5 disk with the cmdlet below:

# umount /mnt/sda5

And, eventually reboot the host.

# reboot

Well, to make the stuff I’ve just written above more reader-friendly, here’re all commands you need to deploy step-by-step.

C:\786a70bf9387ec447bd86ea06e01bd12

Well, you are almost there. Reboot the server now, and try accessing the host without any password. Well, check out what I’ve got.

C:\67ddfd5b95a9399d71561e4f7e82fe71

Now, select Configure Password, and type a new password in the self-titled field.

C:\659a2f378848ab4f9e11135e321968d9

Ok, this time, please write the root password, or just try no to forget it!

Replace one “shadow” with another

There’s another way to reset the ESXi root password using “shadow”. Actually, that’s nothing more than a variation of the method I described above.

So, another thing you can do to reset the ESXi password is just using another host “shadow” file! Yes, you can just copy the “shadow” file from another host with the known root password to the one more flask disk. To get the file with passwords from another host, you need WinSCP. The utility is available here. The nice thing is that you can retrieve that file from the host with the unknown ESXi root password without even shutting it down.

C:\c538c5686ddc4ba551ea1f5237280e1b

Next, call the terminal with the Ubuntu GNOME and reset the password.

Update user privileges to root first. You can run the following command for that purpose:

# sudo su

Now, let’s see what you have on the disk.

# fdisk –l | grep sd 

Create two temporary volumes afterward.

# mkdir /mnt/sda5

# mkdir /mnt/sdb1

Mount the ESXi disk and flash disk where the “shadow” resides using the following cmdlet.

# mount /dev/sda5 /mnt/sda5

# mount /dev/sdb1 /mnt/sdb1

Now, create the temporary volume for further work with archives.

# mkdir /temp

Create the volume where you are going to keep the state.tgz copy just in case something goes wrong.

# mkdir /mnt/sdb1/save

Find the necessary file in the archive.

# ls -l /mnt/sda5/state.tgz

Copy the archive.

# cp /mnt/sda5/state.tgz /mnt/sdb1/save

Run the following command to double-check whether the file has been copied:

# ls -l /mnt/sdb1/save

Extract state.tgz using the cmdlet below:

# tar -xf /mnt/sda5/state.tgz –C /temp/

Find the temp file.

# ls –l /temp

Extract local.tgz.

# tar -xf /temp/local.tgz –C /temp/

Make sure that you extracted the /etc directory.

# ls –l /temp

C:\8b102fd08f266e9fca099d664a77e2c6

Now, delete the local.tgz volume to ensure that it won’t be included into the new archive by accident.

# rm /temp/local.tgz

Find “shadow” in the /etc directory.

# ls -l /temp/etc

Replace the original “shadow” with the one from the host with known root password. Type the following cmdlet:

# cp /mnt/sdb1/shadow /temp/etc

C:\8045c097389c9a0cbc8a78ed1e5805fe

Now, deploy the following command to open the file and look through the saved credentials.

# vi /temp/etc/shadow

If you do not want some users to access the host, go ahead and just remove them from the listing! Here, I removed Test from the users that can access the host. Wait, why did I delete only Test? At this point, I’d like to warn you against deleting any users you are not familiar with. In my case, all users except Test are system ones. If you delete any of those guys, you may destabilize the OS!

C:\91a5a7a5552948a084c9c8bbbd4c4d1c

Here’s how the “shadow”: file looks like once the unnecessary user.

C:\601a3512f8477b298365221f92dcfed7

Check whether all changes have been applied.

# vi /temp/etc/shadow

Type the following line to navigate to the /temp directory.

# cd /temp

Archive the /etc directory.

# tar -czf local.tgz etc

Check whether archiving has run smoothly.

# ls -l /temp/

Now, create the state.tgz volume.

# tar -czf state.tgz local.tgz

Again, check whether the volume has been created.

# ls -l /temp/

Move the archive to the working ESXi directory.

# mv state.tgz /mnt/sda5/

Check the result one more time.

# ls -l /mnt/sda5/

Unmount the sda5 directory.

# umount /mnt/sda5

Eventually, reboot the host.

# reboot

Enjoy! If everything is done right, you can access the host with the known password. Well, to make everything more or less convenient here’s the entire set of commands I used for this method.

C:\aa3e81917d7434ea1863f161d7985514

If the host starts acting weird after reboot, there’s still a copy of the initial state.tgz. Well, it should be. You can mount both /sda5 and /sdb1 and retrieve the original state.tgz using the following cmdlet… and try again!

# cp /mnt/sdb1/save/state.tgz /mnt/sda5/

–jeroen

Posted in Power User, Virtualization, VMware, VMware ESXi | Leave a Comment »

How to Copy files between ESXi hosts using SCP Command

Posted by jpluimers on 2021/05/21

Derived the bits below from [WayBack] How to Copy files between ESXi hosts using SCP Command.

Recursive copy from a remote machine to an existing local directory:

scp -rp root@192.168.71.97://vmfs/volumes/EVO860_500GB/VM1/ /vmfs/volumes/EVO860_250GB/VM2/

After this you need to edit the .vmxf files in the VM2 directory to ensure these are not duplicates.

One thing to remember is that you need the current host to allow the SSH client in the firewall, which is disabled by default:

After enabling:

Be really careful with the -3 option to scp; it allows you to transfer from one remote machine to another remote machine, but when using keyboard-interactive, you have a high change to lock-out your accounts: SSH will try to keyboard-interactive to both hosts at the same time.

If you lock-out root, then you have to go through the local DCUI console (use ALT-F2 to go there), then reset the root account failure count using pam_tally2 --user root --reset.

So this can be bad:

scp -3 -rp root@192.168.71.97://vmfs/volumes/EVO860_500GB/VM1/ root@192.168.71.91://vmfs/volumes/EVO860_250GB/VM2/

This works, but assumes the SSH client is enabled from the first host:

scp -rp root@192.168.71.97://vmfs/volumes/EVO860_500GB/VM1/ root@192.168.71.91://vmfs/volumes/EVO860_250GB/VM2/

See these links:

 

[root@ESXi-X9SRI-F:~] esxcli network firewall get
   Default Action: DROP
   Enabled: true
   Loaded: true
[root@ESXi-X9SRI-F:~] esxcli network firewall ruleset list --ruleset-id sshClient
Name       Enabled
---------  -------
sshClient    false
[root@ESXi-X9SRI-F:~] esxcli network firewall ruleset set --ruleset-id sshClient --enabled true
[root@ESXi-X9SRI-F:~] esxcli network firewall ruleset list --ruleset-id sshClient
Name       Enabled
---------  -------
sshClient     true
[root@ESXi-X9SRI-F:~] esxcli network firewall ruleset set --ruleset-id sshClient --enabled false
[root@ESXi-X9SRI-F:~] esxcli network firewall ruleset list --ruleset-id sshClient
Name       Enabled
---------  -------
sshClient    false

–jeroen

Posted in *nix, *nix-tools, ESXi6, ESXi6.5, ESXi6.7, Power User, ssh/sshd, Virtualization, VMware, VMware ESXi | Leave a Comment »

ESXi 6.x download URL

Posted by jpluimers on 2021/05/17

Many of the ESXi download URLs get you to my-vmware in places that indicate you do not have a license.

This seems to be the only link that consistently gets you the license and downloads: my.vmware.com/en/group/vmware/evalcenter?p=free-esxi6.

For instance, the ones below with

–jeroen

Posted in ESXi6, ESXi6.5, ESXi6.7, Power User, Virtualization, VMware, VMware ESXi | Leave a Comment »

ESXi: wrong IPv4 address after moving the ESXi boot USB stick and SSD devices to an identical motherboard with different MAC addresses

Posted by jpluimers on 2021/05/17

A while ago, I wanted to move the ESXi USB stick and SSD devices to another machine with identical motherboard as it had a larger physical case more suited for expansion.

To my surprise, the management network stayed at the same IPv4 address, despite it being being from a DHCP pool, and the new MAC addresses having different IPv4 addresses assigned in the pool (I run a kind of static dynamic address system where the DHCP server has the correct mapping between MAC and IPv4 addresses).

This appears to be a known issue: by default, ESXi copies the MAC address to the vmknic of the management network instead of following hardware changes. You can see this in the screenshot showing the right physical MAC, but the wrong virtual MAC:

The fix is actually quite simple:

After this, you have to perform a reboot for the new setting to take effect.

When booting is done, the virtual MAC has been copied from the physical MAC:

Note I did not have to fiddle with /etc/vmware/esx.conf as VirtuallyVTrue had to.

For more information, see these links (I copied the content of the final link below the footer as it cannot be saved in the WayBack or Archive.is archives):

–jeroen

Read the rest of this entry »

Posted in ESXi6.5, ESXi6.7, Power User, Virtualization, VMware, VMware ESXi | Leave a Comment »

VMware Standalone Converter: more recent versions do not support older operating systems

Posted by jpluimers on 2021/05/14

It pays to keep several versions of VMware Standalone Converter at hand as newer supporters do support newer operating systems and ESXi versions, but do not support older operating systems.

Hopefully the free StarWind V2V converter that does also support P2V has a broader support for older versions.

Some relevant links:

–jeroen

[WayBack] Free Tools VMware VMware vSphere goodies and freebies. VMware Monitoring tools, backup. Those tools are Free to use in production environment, no time limit

[WayBack] Cool Free VPN Server Software SoftEther VPN | ESX Virtualization

Posted in Power User, Virtualization, VMware, VMware Converter, VMware ESXi | Leave a Comment »

NFS server on Windows

Posted by jpluimers on 2021/05/14

One way to access files from ESXi is over NFS shares.

Out of the box, Windows Server is the only edition that provides NFS server capability, but desktop editions only have an NFS client.

There are some commercial and open sources implementations though, of which [WayBack] GitHub – winnfsd/winnfsd seems the best maintained open source one.

In case I ever need NFS server support, I need to check out these links:

–jeroen

Posted in *nix, Power User, Virtualization, VMware, VMware ESXi, Windows | Leave a Comment »

Solved: Very slow speed on SSD |VMware Communities (via “Building a lab with ESXI and Vagrant – DarthSidious”)

Posted by jpluimers on 2021/05/11

Via [WayBack] Building a lab with ESXI and Vagrant – DarthSidious while researching the possibility of running Vagrant (software) – Wikipedia on VMware ESXi – Wikipedia for building and distributing development environments:

[WayBack] Solved: Very slow speed on SSD |VMware Communities “solution” that seems to work for ESXi 6.5 and 6.7:

ESXi 6.5 includes a new native driver (vmw_ahci) for SATA AHCI controllers, but that introduces performance problems with a lot of controllers and/or disks.

Try to disable the native driver and revert to the older sata-ahci driver by running

esxcli system module set --enabled=false --module=vmw_ahci

in an ESXi shell.

Reboot the host to make the change effective.

which solves it for some who now get much faster results:

Your suggestion worked for me, now i am getting avg speed 250Mbps from SATA III SSD .

ssd.jpg

Hope will get the full I/Ops from SSD.

However:

One issue I still have is that my 4 port Syba PCIe controller card now vanishes after disabling vmw_ahci and I am restricted to using the SATA ports on the motherboard.

and you need backups:

WARNING: Doing this at least for me erases all the VMs on the aforementioned drive. Migrate as needed.

There was no response for a more permanent fix:

What is the permanent fix for this issue, should we expect a corrected native driver from VMware, or will this require a firmware upgrade on the part of the drive vendors?

and there seem to be other bottle-necks:

tried the command on a 6.7.

Deploying an OVA and I am getting 22.82….

I have a Samsung 860 EVO mSATA 1Tb SSD.

i re-enabled it, I got max 11.81.

Kind of crappy either way. Not SSD speeds IMO.

–jeroen

 

Posted in Development, ESXi6.5, ESXi6.7, Power User, Software Development, Testing, Virtualization, VMware, VMware ESXi | Leave a Comment »

Alternatives to VMware ESXi: working around “[Errno 28] No space left on device” when updating (especially when booting from USB-stick)

Posted by jpluimers on 2021/05/07

Yesterday I talked about VMware ESXi: working around “[Errno 28] No space left on device” when updating (especially when booting from USB-stick).

There are some alternative workarounds mentioned on the interwebz. Below are a the ones I found. I discuss which ones won’t work, and why I dislike others.

Alternative workarounds that failed

Configuring host-swap

This was suggested by:

Host swap was already configured, and it still failed.

Just in case you ever want to configure host swap, it is under an URL like https://esxi67.example.org/ui/#/host/manage/system/swap and looks like this:

ESXi 6.7: configuring host swap

ESXi 6.7: configuring host swap

You get there by:

  1. logging on to the web UI
  2. clicking Host
  3. clicking Manage under Host
  4. clicking Swap under the System tab
  5. clicking Edit settings when you want to change them
    ESXi 6.7: edit host swap settings

    ESXi 6.7: edit host swap settings

More information about host swap:

Alternative workarounds I like less

Below are a few alternative workarounds. I will include them as they gained me more knowledge, but I will also describe why I like them less.

  • [Wayback] ESXI 6.7 update: No space left on device | eknori.de after explaining that directing the swap space to a datastore fails, also mentions alternative this:

    Unfortunately, in this situation, host swap already was enabled.

    There is though, a workaround. You can use an image that doesn’t have the tools vib included with this command:

    esxcli software profile update -p ESXi-6.7.0-20190802001-no-tools -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

    You can then manually install the troublesome vib (if you have a need for tools) with this command:

    esxcli software vib install -v https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/tools-light/VMware_locker_tools-light_10.3.10.12406962-14141615.vib

    I had to edit it as the post itself shows the filename as

    vmw-depot
     -index.xml

    Yup: bitching again, as markup issues make code unreliable. It also allows me to explain why I do not like the solution, which is because of two reasons:

    1. It doesn’t explain why this solution works and if it is future proof. Does a future upgrade that includes changed VMware_locker_tools-light also fail? If it does not fail, does it update the VMware_locker_tools-light?
    2. It does not explain how to get the path of https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/tools-light/VMware_locker_tools-light_10.3.10.12406962-14141615.vib.  I did some mor research on this, and it is actually pretty straightforward: the [Wayback] VMware ESXi 6.7 Patch History has it in the table
      ESXi-6.7.0-20210304001-standard patch table

      ESXi-6.7.0-20210304001-standard patch table

      The “Version” link for “tools-light” [Wayback] 11.2.5.17337674-17700514 actually links to the VMware_locker_tools-light_11.1.1.16303738-16701467.vib file.

  • [Wayback] ESXi 6.7.0 – [Errno 28] No space left on devicevibs = VMware_locker_tools-light_11.1.1.16303738-16701467 does not explain where to get the VMware_locker_tools-light_11.1.1.16303738-16701467.vib link from, does not have the code formatted as such (so I did that below), but does actually answers part of the above questions, but not if a future upgrade will also fail. In short: re-running the upgrade after manually installing the VMware_locker_tools-light_11.1.1.16303738-16701467.vib will succeed:

    Unfortunately swap was already enabled to I had to manually install the tools-light with this command:

    esxcli software vib install -v https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/tools-light/VMware_locker_tools-light_11.1.1.16303738-16701467.vib

    Then re-ran the upgrade and it was successful.

  • [Wayback] ESXi 6 Update error – No Space left on device /locker which suggests to delete “find all big files in /locker and remove it”. I think that is a bad idea, as the /locker directory is maintained by your ESXi system and you should not remove any big file without knowing if it is relied upon by ESXi.
  • While updating VMware ESXi servers, VMware vSphere users may encounter the “No space left on device” error that pops up while executing “esxcli software vib update” command. Interestingly, the problem occurs even though disks are doing well and have enough free space and df -h command proves that.[Wayback] No Space Left on Device? Updating VMware ESXi | StarWind Blog has bad code markup, but explains
    • how to get disk usage with df -h where the vfat volumes usually indicate the ones on USB or SD-card media.
    • that hardly the number of inodes is a problem, and that stat -f / can help you figure out if that is the case on the volume where the upgrade files are stored
    • how to find large files not in data stores; I have changed added -h to the ls command so it becomes human readable:
      find / -path "/vmfs" -prune -o -type f -size +50000k -exec ls -lh '{}' \;
    • suggests how to put the swap space on a data store (which doesn’t work on ESXi 6.7 systems any more)
  •  

    [Wayback] Intel NUC Kit NUC5i3RYH met ESXi 6.0 updaten naar 6.7 – Gahan Zwart’s Blog

    • I like the upgrade copying the ISO to an USB stick with Rufus
    • I do not like the VMware_locker_tools-light... intermediate step, as the last step (download the full depot to a datastore, then update from there)
    • ESXi 7.0 has the same ErrNo 28 update problem as ESXi 6.7 and 6.5, so I will default to the depot download.

–jeroen

Posted in ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, Virtualization, VMware, VMware ESXi | Leave a Comment »