The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,651 other followers

Archive for the ‘ESXi5.1’ Category

Keeping your root visorfs clean: point the path to your own binaries stored on a vmfs volume

Posted by jpluimers on 2019/04/26

Some interesting commands derived from [WayBackESXi/ESX error: No free space left on device (1007638) | VMware KB:

  • finding large files:
    find / -path "/vmfs" -prune -o -type f -size +50000k -exec ls -lh '{}' \;
  • finding space on the root file system (which is not listed in df -h):
    stat -f /

This was in the process of trying to keep my local binaries out of [WayBackVisorFS: A Special-purpose File System for Efficient Handling of System Images – VMware Labs as it is inherently small in size (both total size and number of inodes) as it is a RAM disk based file system.

Based on that, at [WayBackTrouble shooting – esx.problem.visorfs.ramdisk.full – DefinIT I found this even more useful statement vdf -h | grep "%\|Ramdisk" which shows the exact usage of what’s in this filesystem. Example output on one of my systems:

# vdf -h | grep "%\|Ramdisk"
Ramdisk                   Size      Used Available Use% Mounted on
root                       32M        1M       30M   6% --
etc                        28M      184K       27M   0% --
opt                        32M        0B       32M   0% --
var                        48M      352K       47M   0% --
tmp                       256M        4K      255M   0% --
iofilters                  32M        0B       32M   0% --
hostdstats                678M        4M      673M   0% --

The easiest is not to store them in the root file system at all, but then you need to alter the default path:

# echo $PATH
/bin:/sbin

Since my local binaries are at /vmfs/volumes/Samsung512NVME/local-bin/, I wanted to persist this path change:

export PATH=$PATH:/vmfs/volumes/Samsung512NVME/local-bin/

Basically you can do this with any current directory on your system: export PATH=$PATH:`pwd`

The easiest way to persist that path is to ensure you can shoehorn the effect in a file that gets started during bootup.

The standard – but unsupported – way to do that is shown for instance by:

Final solution

So, edit vi /etc/rc.local.d/local.sh, then shutdown all your VMs and reboot the system to verify the effects. However inserting that export isn’t enough. This is the line you need to add before the exit 0:

sed -i -e 's!PATH=/bin:/sbin!PATH=/bin:/sbin:/vmfs/volumes/Samsung512NVME/local-bin/!' /etc/profile

Related

  • [WayBack] ESXi: Aliases definieren › /dev/blog/ID10T
  • [Archive.is] Solved: How to keep a .profile in / of ESXi? |VMware Communities

    ESXi does not remove that file on boot-up, it simply does not save it. ESXi runs from memory. So if you created some file (i.e. /.profile) it is only in “memory-disk”, not in disk-image which is loaded again at the next boot-up.

    Either create custom vib and install it as every other, or use rc.local which is persistent (any changes you make to this file survive boot-up). You can create & save that file somewhere else and use rc.local to copy it to /, or use rc.local with shell commands to create .profile at every boot-up.

    Wait a minute, you are using ESXi 6.0, right? I’m not sure if there is /etc/rc.local, but it used to be in 5.0/5.5…

    I edited /etc/rc.local.d/local.sh to copy the file that I keep on one of my datastores. After reboot, it worked fine. (So, yes, this does work in 6.0.) Thanks!

    I’m glad it worked for you. BTW you are right: instead of single file /etc/rc.local (as in 5.0) there is now the whole sub-dir /etc/rc.local.d/ but functionality is the same…

–jeroen

Posted in ESXi5, ESXi5.1, ESXi5.5, ESXi6, ESXi6.5, Power User, Virtualization, VMware, VMware ESXi | Leave a Comment »

ESXi 6.5: mount a datastore that does not automount; esxcfg-volume to the rescue

Posted by jpluimers on 2019/03/29

I had a 1.5 TB SATA disk with VMFS5 created on ESXi 5.1 that would not want to mount on ESXi 6.5 automatically, not even after a rescan, or fresh boot, so I did this:

[root@ESXi-X10SRH-CF:~] esxcfg-volume --help
esxcfg-volume
-l|--list                                List all volumes which have been
                                         detected as snapshots/replicas.
-m|--mount              Mount a snapshot/replica volume, if 
                                         its original copy is not online.
-u|--umount             Umount a snapshot/replica volume.
-r|--resignature        Resignature a snapshot/replica volume.
-M|--persistent-mount   Mount a snapshot/replica volume
                                         persistently, if its original copy is
                                         not online.
-U|--upgrade            Upgrade a VMFS3 volume to VMFS5.
-h|--help                                Show this message.
[root@ESXi-X10SRH-CF:~] esxcfg-volume --list
Scanning for VMFS-3/VMFS-5 host activity (512 bytes/HB, 2048 HBs).
VMFS UUID/label: 59a5306c-a8793061-4a23-001f29022aed/ST1500LM0032D9YH148-backup
Can mount: Yes
Can resignature: Yes
Extent name: naa.5000c5002dba6642:1 range: 0 - 1430527 (MB)

Scanning for VMFS-3/VMFS-5 host activity (512 bytes/HB, 2048 HBs).
VMFS UUID/label: 532cd010-6e8c01d1-45be-001f29022aed/Raid6SATA
Can mount: Yes
Can resignature: Yes
Extent name: naa.600605b00aa054a0ff000021022683ae:1 range: 0 - 1830143 (MB)

[root@ESXi-X10SRH-CF:~] esxcfg-volume -m 532cd010-6e8c01d1-45be-001f29022aed/Raid6SATA
No matching volume 532cd010-6e8c01d1-45be-001f29022aed/Raid6SATA found!
[root@ESXi-X10SRH-CF:~] esxcfg-volume --mount 532cd010-6e8c01d1-45be-001f29022aed
Mounting volume 532cd010-6e8c01d1-45be-001f29022aed

Based on: [WayBackMount VMFS Datastore – via GUI or via CLI [Guide] – ESX Virtualization

–jeroen

Posted in ESXi5.1, ESXi6.5, Power User, Virtualization, VMware, VMware ESXi | Leave a Comment »

ssh from Mac OS X to ESXi: “WARNING: terminal is not fully functional”

Posted by jpluimers on 2019/03/29

When connecting from my Mac to my ESXi rig, some commands (especially less) show this output:

WARNING: terminal is not fully functional

So I created this alias to connect from my Mac to the internal address of my ESXi rig:

alias ssh-esxi-X10SRH-CF-internal='TERM=xterm ssh -p 22 root@192.168.71.91'

The trick is the bold part: TERM=xterm (which you can also replace by export TERM=xterm; if you want future ssh sessions to use the same [wayback] TERM setting).

The reason is that the Mac defines the TERM variable as containing xterm-256 which is defined on the Mac itself, but ESXi has a hard time coping with it.

Some Mac OS and Xcode combinations had a problem with xterm-256 not being present ([WayBackmacos – Terminal strangeness after installing Xcode on Lion – Super User), but this isn’t the case on my system:

$ ls -alh `find /usr/share/terminfo | grep 'xterm-256color'`
-rw-r--r-- 1 root wheel 3.2K Jul 30 2016 /usr/share/terminfo/78/xterm-256color

On the Mac you really want to use xterm-256color as it looks way better than xterm-color or xterm: [WayBacklinux – What is the difference between xterm-color & xterm-256color? – Stack Overflow (thanks [WayBack] Chris Page!)

It seems I already did something similar on ESXi itself to get esxtop working: ESXi: when esxtop shows garbage. That was on the ESXi side and works as well for this problem too.

However, it is a bit harder to have a script run during ESXi boot time that sets this, so it is easier to fix this on the Mac side.

It works for all OS X and ESXi versions I’ve tested so far.

–jeroen

Posted in *nix, Apple, ESXi5.1, ESXi5.5, ESXi6, ESXi6.5, iMac, Mac, Mac OS X / OS X / MacOS, Mac OS X 10.4 Tiger, Mac OS X 10.5 Leopard, Mac OS X 10.6 Snow Leopard, Mac OS X 10.7 Lion, MacBook, MacBook Retina, MacBook-Air, MacBook-Pro, MacMini, macOS 10.12 Sierra, OS X 10.10 Yosemite, OS X 10.11 El Capitan, OS X 10.8 Mountain Lion, OS X 10.9 Mavericks, Power User, Virtualization, VMware, VMware ESXi | Leave a Comment »

Using the partedUtil command line utility on ESXi and ESX (1036609) | VMware KB

Posted by jpluimers on 2019/03/13

Reminder to self when checking “new” disks to see what partitions they contain before formatting them as VMFS.

There is a truckload information at [WayBackUsing the partedUtil command line utility on ESXi and ESX (1036609) | VMware KB.

A few tips; example output is further below:

  • Disks are listed under /vmfs/devices/disks/ where there are two entries per device: a path leading to the device, and a link to that path which starts with vml. which I filter out with grep.
  • If a disk under under /vmfs/devices/disks/ ends with :# where # is a number, then it is a partition
  • Just skip partedUtil get as partedUtil getptblwill give you exactly the same information,
    • plus an extra initial line indicating what kind of partition table it is. KB 1036609 has a longer list, but these are the ones you usually see:
      • unknown: the disk has no partition table yet (usually), or the type of partition table cannot be determined (hardly)
      • gpt: there is a GUID Partition Table
      • msdos: there is a Master Boot Record partition table
    • on ESXi 6.x two extra columns listing the partition GUID and partition type description
  • The output of partedUtil is unformatted, which means it is easy to parse, but hard to read for humans. You can pipe through sed 's/ /\t/g' (as there is no tr on the ESXi busybox)

Some more background reading

On scripting:

  • The shell is sh (always been there)
  • There is Python (ESXi 5.1 has Python 2.7.8; ESXi 6.5 has Python 3.5.3; it has likely been available in earlier versions too).

On device names:

On errors:

    1. ~ # find /vmfs/devices/disks/ | grep T1500LM0032D9YH148
      /vmfs/devices/disks/t10.ATA_____ST1500LM0032D9YH148__________________________________Z110C4Q0a
      ~ # partedUtil getptbl /vmfs/devices/disks/t10.ATA_____ST1500LM0032D9YH148__________________________________Z110C4Q0
      unknown
      182401 255 63 2930277168
      ~ # 

I know of three VMFS types:

  • VMFS-3: Supported in ESXi 3.X, 4.X, 5.x & 6.x; deprecated as of 6.0 (cannot be created as of 6.0), has quite some limitations.
  • VMFS-4: got never released.
  • VMFS-5: Can be converted from VMFS-3
  • VMFS-6: Cannot be converted from other VMFS types

Some interesting links about the various VMFS types:

Busybox has been updated over time:

 

Examples and output

Example outputs on one of my systems, of which I stripped most of the disks as they’re not really relevant here.

[root@ESXi-X10SRH-CF:~] ls -1 /vmfs/devices/disks/ | grep -v '^vml\.'
naa.5000c50087762d1b
...
naa.600605b00aa054a0ff000021022683ae
naa.600605b00aa054a0ff000021022683ae:1
...
t10.ATA_____ST1500LM0032D9YH148__________________________________Z110C4Q0
t10.ATA_____Samsung_SSD_850_PRO_2TB_________________S2KMNCAGB04321L_____
t10.ATA_____Samsung_SSD_850_PRO_2TB_________________S2KMNCAGB04321L_____:1
...
t10.SanDisk00Ultra_Fit000000000000004C530001240406103372
t10.SanDisk00Ultra_Fit000000000000004C530001240406103372:1
t10.SanDisk00Ultra_Fit000000000000004C530001240406103372:5
t10.SanDisk00Ultra_Fit000000000000004C530001240406103372:6
t10.SanDisk00Ultra_Fit000000000000004C530001240406103372:7
t10.SanDisk00Ultra_Fit000000000000004C530001240406103372:8
t10.SanDisk00Ultra_Fit000000000000004C530001240406103372:9
[root@ESXi-X10SRH-CF:~] partedUtil getptbl /vmfs/devices/disks/t10.ATA_____ST1500LM0032D9YH148__________________________________Z110C4Q0 
unknown
182401 255 63 2930277168
[root@ESXi-X10SRH-CF:~] partedUtil get /vmfs/devices/disks/t10.ATA_____ST1500LM0032D9YH148__________________________________Z110C4Q0 
182401 255 63 2930277168
[root@ESXi-X10SRH-CF:~] partedUtil get /vmfs/devices/disks/t10.ATA_____SAMSUNG_MZHPV512HDGL2D00000______________S1X1NYAGB09589______
62260 255 63 1000215216
1 2048 1000214527 0 0
[root@ESXi-X10SRH-CF:~] partedUtil getptbl /vmfs/devices/disks/t10.ATA_____SAMSUNG_MZHPV512HDGL2D00000______________S1X1NYAGB09589______
gpt
62260 255 63 1000215216
1 2048 1000214527 AA31E02A400F11DB9590000C2911D1B8 vmfs 0
[root@ESXi-X10SRH-CF:~] partedUtil getptbl /vmfs/devices/disks/t10.SanDisk00Ultra_Fit000000000000004C530001240406103372
gpt
3738 255 63 60062500
1 64 8191 C12A7328F81F11D2BA4B00A0C93EC93B systemPartition 128
5 8224 520191 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
6 520224 1032191 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
7 1032224 1257471 9D27538040AD11DBBF97000C2911D1B8 vmkDiagnostic 0
8 1257504 1843199 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
9 1843200 7086079 9D27538040AD11DBBF97000C2911D1B8 vmkDiagnostic 0
[root@ESXi-X10SRH-CF:~] partedUtil get /vmfs/devices/disks/t10.SanDisk00Ultra_Fit000000000000004C530001240406103372
3738 255 63 60062500
1 64 8191 0 128
5 8224 520191 0 0
6 520224 1032191 0 0
7 1032224 1257471 0 0
8 1257504 1843199 0 0
9 1843200 7086079 0 0
[root@ESXi-X10SRH-CF:~] partedUtil getptbl /vmfs/devices/disks/t10.SanDisk00Ultra_Fit000000000000004C530001240406103372 | sed 's/ /\t/g'
gpt
3738    255 63  60062500
1   64  8191    C12A7328F81F11D2BA4B00A0C93EC93B    systemPartition 128
5   8224    520191  EBD0A0A2B9E5443387C068B6B72699C7    linuxNative 0
6   520224  1032191 EBD0A0A2B9E5443387C068B6B72699C7    linuxNative 0
7   1032224 1257471 9D27538040AD11DBBF97000C2911D1B8    vmkDiagnostic   0
8   1257504 1843199 EBD0A0A2B9E5443387C068B6B72699C7    linuxNative 0
9   1843200 7086079 9D27538040AD11DBBF97000C2911D1B8    vmkDiagnostic   0
[root@ESXi-X10SRH-CF:~] partedUtil get /vmfs/devices/disks/t10.SanDisk00Ultra_Fit000000000000004C530001240406103372 | sed 's/ /\t/g'
3738    255 63  60062500
1   64  8191    0   128
5   8224    520191  0   0
6   520224  1032191 0   0
7   1032224 1257471 0   0
8   1257504 1843199 0   0
9   1843200 7086079 0   0
[root@ESXi-X10SRH-CF:~] partedUtil --help
Usage: 
 Get Partitions : get  
 Set Partitions : set  ["partNum startSector endSector type attr"]* 
 Delete Partition : delete  
 Resize Partition : resize    
 Get Partitions : getptbl  
 Set Partitions : setptbl   ["partNum startSector endSector type/guid attr"]* 
 Fix Partition Table : fix  
 Create New Label (all existing data will be lost): mklabel   
 Show commonly used partition type guids : showGuids 
 Get usable first and last sectors : getUsableSectors  
 Fix GPT Table interactively : fixGpt  

[root@ESXi-X10SRH-CF:~] partedUtil showGuids
 Partition Type       GUID
 vmfs                 AA31E02A400F11DB9590000C2911D1B8
 vmkDiagnostic        9D27538040AD11DBBF97000C2911D1B8
 vsan                 381CFCCC728811E092EE000C2911D0B2
 virsto               77719A0CA4A011E3A47E000C29745A24
 VMware Reserved      9198EFFC31C011DB8F78000C2911D1B8
 Basic Data           EBD0A0A2B9E5443387C068B6B72699C7
 Linux Swap           0657FD6DA4AB43C484E50933C84B4F4F
 Linux Lvm            E6D6D379F50744C2A23C238F2A3DF928
 Linux Raid           A19D880F05FC4D3BA006743F0F84911E
 Efi System           C12A7328F81F11D2BA4B00A0C93EC93B
 Microsoft Reserved   E3C9E3160B5C4DB8817DF92DF00215AE
 Unused Entry         00000000000000000000000000000000
[root@ESXi-X10SRH-CF:~] cat /local/bin/what-is-my-shell.sh 
if test -n "$ZSH_VERSION"; then
  PROFILE_SHELL=zsh
elif test -n "$BASH_VERSION"; then
  PROFILE_SHELL=bash
elif test -n "$KSH_VERSION"; then
  PROFILE_SHELL=ksh
elif test -n "$FCEDIT"; then
  PROFILE_SHELL=ksh
elif test -n "$PS3"; then
  PROFILE_SHELL=unknown
else
  PROFILE_SHELL=sh
fi
echo $PROFILE_SHELL
echo $SHELL
[root@ESXi-X10SRH-CF:~] /local/bin/what-is-my-shell.sh 
sh
/bin/sh
[root@ESXi-X10SRH-CF:~] python --version
Python 3.5.3
[root@ESXi-X10SRH-CF:~] 

–jeroen

Posted in Development, ESXi4, ESXi5, ESXi5.1, ESXi5.5, ESXi6, ESXi6.5, Power User, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

What directories to backup on ESXi apart from the VMs? – Server Fault

Posted by jpluimers on 2017/08/18

What directories to backup on ESXi apart from the VMs? – Server Fault [WayBack]:

You may want to look at purpose-built VMware backup tools. You will be very disappointed if you treat an ESXi host like a Linux/Unix server.

Use the VMware backup commands [WayBack] from a separate station, and you’ll be left with a nice configuration tarball. I would find another approach for the actual VM backups.

Edit: Host-based example

vim-cmd hostsvc/firmware/sync_config
vim-cmd hostsvc/firmware/backup_config

This stores the config in /scratch/downloads:

# vim-cmd hostsvc/firmware/backup_config
Bundle can be downloaded at : http://ip.of.esxi.host/downloads/52fd67ba-2fdf-9876-6651-46c3da638f1a/configBundle-centaur.ewwhite.net.tgz

Also see: http://www.virtuallyghetto.com/2013/02/how-to-backup-restore-free-esxi-host.html [WayBack]

–jeroen

Posted in ESXi5, ESXi5.1, ESXi5.5, Power User, Virtualization, VMware, VMware ESXi | Leave a Comment »

 
%d bloggers like this: