Archive for the ‘ash/dash development’ Category
Posted by jpluimers on 2026/10/20
I have been using rsync for years now to sync directories around on VMware ESXi systems with great success.
Usually when upgrading to new physical machines, I synced the things to migrate to a big disk, then physically mounted that disk in a new machine.
Not this time: because of an unscheduled event I had to bring up the target machine in a different place before I could assemble the data on the big disk.
But since there was a network tunnel between the networks of both ESXi machines, I wanted to transfer over ssh: a thing I had done numerous times between various Linux and unix machines.
The cool thing is that rsync supports remote connections over ssh out of the box. Between ESXi machines, I bumped into this error though:
sh: rsync: not found
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.2]
Command exited with non-zero status 12
The last three lines are collateral of the first line, which I queried [Wayback/Archive] “sh: rsync: not found” – Google Search. Further below I will go deeper into the rabbit-hole I took, but the actual solution was to Read the rest of this entry »
Posted in *nix, *nix-tools, ash/dash, ash/dash development, BusyBox, Development, ESXi6, ESXi6.5, ESXi6.7, man/manual pages, OpenSSH, Power User, rsync, Scripting, Software Development, SSH, ssh/sshd, Virtualization, VMware, VMware ESXi | Leave a Comment »
Posted by jpluimers on 2026/08/18
Where in ESXi 6.x you could change /etc/rc.local in ESXi 7 and up tightened security and now use /etc/rc.local.d/local.sh [Wayback/Archive] virtualization – ESXi 7.X file permissions – how to buypass new security measures – Server Fault
Notes:
- some ESXi (for instance when booting from PXE do not have persistent storage)
- be sure to always backup an ESXi configuration after changing it
Via [Wayback/Archive] “esxi 7” run your own shell scripts – Google Search
–jeroen
Posted in ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, ESXi8, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »
Posted by jpluimers on 2026/01/15
On OpenWRT GL.iNET based devices, the WireGuard client does not restart upon reboot, even if it was started before rebooting.
Hopefully the /usr/bin/wireguard_watchdog script will help with this as others indicates it should.
My first try was no succes, but since it is supposed to run from cron it does no output. The script on GL-SFT1200 firmware version 3.215, script /usr/bin/wireguard_watchdog is different from the one in the OpenWRT repository, so it needs some investigation.
Some links for checking this out:
Read the rest of this entry »
Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, GL-AR300M, GL.iNet, GL.iNET GL-SFT1200, OpenWRT, Power User, Scripting, Software Development | Leave a Comment »
Posted by jpluimers on 2024/02/22
Cool: [Wayback] ShellCheck – shell script analysis tool
ShellCheck finds bugs in your shell scripts
It needs a shebang at the start of a script (like #!/usr/bin/env bash) to recognise the kind of shell, then does amazing analysis.
It is open source at [Wayback/Archive.is] koalaman/shellcheck: ShellCheck, a static analysis tool for shell scripts with excellent documentation including explaining screenshots like
It’s not just available on-line or on the command-line, but also integrates with many code editors (like [Wayback/Archive.is] ShellCheck – Visual Studio Marketplace: Integrates ShellCheck into VS Code, a linter for Shell scripts.) and CI/CD pipelines.
Via: [Wayback] bash – error conditional binary operator expected in compound branch – Unix & Linux Stack Exchange (thanks [Wayback] Cyrus!)
–jeroen
Posted in *nix, *nix-tools, ash/dash, ash/dash development, bash, bash, Development, Power User, Scripting, Software Development | Leave a Comment »
Posted by jpluimers on 2024/01/10
I bumped into [Wayback/Archive] Script to rename a virtual machine in ESXi 6: gist.github.com/Gremgoll/8bd91258d71fe895c0d416e2543ca2dc.
Then I forked it with the intent to make usage more clear, as the current usage on my ESXi rig (which has mos VMs not in the root of data stores) is like the bold italic portion here:
[root@X9SRI3F-ESXi:/vmfs/volumes/608be754-f21556ad-1082-0025907d9d5c/VM] vm-rename.sh NVMe980PRO_1TB/VM X9SRI-3F-W10P-NL-OFFICE X9SRI-3F-W10P-NL X9SRI-3F-W10P-NL-OFFICE
VOLNAME=NVMe980PRO_1TB/VM
DIRNAME=X9SRI-3F-W10P-NL-OFFICE
OLDNAME=X9SRI-3F-W10P-NL
NEWNAME=X9SRI-3F-W10P-NL-OFFICE
VM_DIRPATH=/vmfs/volumes/NVMe980PRO_1TB/VM/X9SRI-3F-W10P-NL-OFFICE
NW_DIRPATH=/vmfs/volumes/NVMe980PRO_1TB/VM/X9SRI-3F-W10P-NL-OFFICE
Failed to rename './X9SRI-3F-W10P-NL-flat.vmdk' to './X9SRI-3F-W10P-NL-OFFICE-flat.vmdk': The file specified is not a virtual disk (15)
renaming ./X9SRI-3F-W10P-NL-41260b40.vmem to ./X9SRI-3F-W10P-NL-OFFICE-41260b40.vmem
renaming ./X9SRI-3F-W10P-NL-41260b40.vmss to ./X9SRI-3F-W10P-NL-OFFICE-41260b40.vmss
renaming ./X9SRI-3F-W10P-NL-6a4b8f29.hlog to ./X9SRI-3F-W10P-NL-OFFICE-6a4b8f29.hlog
renaming ./X9SRI-3F-W10P-NL.nvram to ./X9SRI-3F-W10P-NL-OFFICE.nvram
renaming ./X9SRI-3F-W10P-NL.vmsd to ./X9SRI-3F-W10P-NL-OFFICE.vmsd
renaming ./X9SRI-3F-W10P-NL.vmx to ./X9SRI-3F-W10P-NL-OFFICE.vmx
renaming ./X9SRI-3F-W10P-NL.vmx.backup to ./X9SRI-3F-W10P-NL-OFFICE.vmx.backup
renaming ./X9SRI-3F-W10P-NL.vmxf to ./X9SRI-3F-W10P-NL-OFFICE.vmxf
renaming ./vmx-X9SRI-3F-W10P-NL-1093012288-1.vswp to ./vmx-X9SRI-3F-W10P-NL-OFFICE-1093012288-1.vswp
All Done. You now need to register X9SRI-3F-W10P-NL-OFFICE to the inventory.
There also was a renaming bug (see the italic line).
I managed to fix both that and added more documentation plus output.
Repositories and code
- Original:
- Fork with added VSAN functionality (see [Wayback/Archive] Revisions · Script to rename a virtual machine in ESXi 6):
- My fork of 2:
- My fix:
Code before modifying: [Wayback/Archive] gist.githubusercontent.com/jpluimers/fcc601dd41ac89f601a5174be92c841c/raw/e3683fbb6bdf1e73d65d2b784027c70cf42a5512/vm-rename
Code after modifying: [Wayback/Archive] raw.githubusercontent.com/jpluimers/vm-rename/master/vm-rename.sh.
Yes, that is not in a gist any more (see below why), it is now part of [Wayback/Archive] jpluimers/vm-rename: Script to rename a virtual machine in ESXi 6; fork from https://gist.github.com/jpluimers/fcc601dd41ac89f601a5174be92c841c as [Wayback/Archive] vm-rename/vm-rename.sh at master · jpluimers/vm-rename.
The reason was
Yesterday I described the workaround in Fork Gist to Repo on GitHub – Stack Overflow.
Queries
I found the original via [Wayback/Archive] script esxi rename vmx vmxf and related files – Google Search.
My start was [Wayback/Archive] script esxi duplicate vm and rename vmdk – Google Search which found:
- [Wayback/Archive] Renaming a virtual machine and its files in VMware ESXi (1029513) which has a PowerCLI script (that requires Windows to run PowerCLI on)
- [Wayback/Archive] Solved: How to clone a VM on a ESXI 6.5 server? – VMware Technology Network VMTN which pointed me to the next Google Search result :
- [Wayback/Archive] VMware esxi – Script to clone a VM without vSphere or vCenter. | GrangerX which I put in this gist:
- [Wayback/Archive] Script to create Linked Clones on ESXi | RedNectar’s Blog which is great, but way to complicate for my use case. It references a few other cool scripts though, and shows how to use Resource Pools in ESXi (which the Web-UI does not seem to support):
- [Wayback/Archive] [script] Cloning VMs using ESXi shell/admin console – Virtualizing Unraid – Unraid
- [Wayback/Archive] Script to clone a VM with free VMware ESXi – Rob Pomeroy (again: PowerCLI)
–jeroen
Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, Power User, PowerCLI, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »
Posted by jpluimers on 2023/12/28
This is a sort of follow-up on ESXi: listing virtual machines with their IP addresses where we ended with this:
I modified the above script to become this:
#!/bin/sh
vmids=`vim-cmd vmsvc/getallvms | sed -n -E -e "s/^([[:digit:]]+)s+((S.+S)?)s+([S+])s+(.+.vmx)s+(S+)s+(vmx-[[:digit:]]+)s*?((S.+)?)$/1/p"`
for vmid in ${vmids} ; do
# powerState values:
# Powered off
# Powered on
# Suspended
powerState=`vim-cmd vmsvc/power.getstate ${vmid} | sed '1d'`
name=`vim-cmd vmsvc/get.config ${vmid} | sed -n -E -e '/(vim.vm.ConfigInfo) {/,/files = (vim.vm.FileInfo) {/ s/^ +name = "(.*)",.*?/1/p'`
vmPathName=`vim-cmd vmsvc/get.config ${vmid} | sed -n -E -e '/files = (vim.vm.FileInfo) {/,/tools = (vim.vm.ToolsConfigInfo) {/ s/^ +vmPathName = "(.*)",.*?/1/p'`
# For now, I choose to use only the IPv4 main address from ipAddress, which is in between (vim.vm.GuestInfo) { and net = (vim.vm.GuestInfo.NicInfo) [.
ipAddress=`vim-cmd vmsvc/get.guest ${vmid} | sed -n -E -e '/(vim.vm.GuestInfo) {/,/net = (vim.vm.GuestInfo.NicInfo) [/ s/^ +ipAddress = "(.*)",.*?/1/p'`
printf "VM with id %3s has power state %-11s and IPv4=%-15s (name = ${name}; vmPathName = ${vmPathName}).n" "${vmid}" "${powerState}" "${ipAddress}"
done
Now the script grew even larger in to vim-cmd-list-all-VMs-with-IPv4-MAC-uuid.sh:
Read the rest of this entry »
Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »
Posted by jpluimers on 2022/08/16
A while back Kristian Köhntopp (isotopp) wrote a blog post after quite a Twitter argument where he poses against using git empty commits. I’m with Kris: don’t use them for anything, especially not for kicking off your CI/CD.
Basically his blog post is all about avoiding to think you have a golden hammer, and avoid falling for the Law of the instrument – Wikipedia.
Originally, Abraham Maslow said in 1966:
“I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.”
For me this has all to do with preventing technical debt: find the right tool to kick your CI/CD pipeline after part of that chain somehow malfunctioned is way better than polluting the commit history with empty commits.
His blog post: [Wayback/Archive.is] Empty commits and other wrong tools for the job | Die wunderbare Welt von Isotopp
The most important bit in it:
And since we are talking about CI/CD pipelines: Don’t YAML them. Don’t JSON them. Don’t XML them.
…
Programming in any of these three is wrong use of tooling, and you should not do it.
- YAML, JSON and XML are for declarative things.
- Python, Go and Rust are for procedural things.
- Bash is for interactive use only.
Use the proper tooling for the job. Be an engineer.
This very much reminds me of an Entwickler Konferenz keynote a long time ago, where Neal Ford made the point that most software engineers act very much unlike what is expected from traditional engineering way of operating where the engineer is both responsible and liable for his actions.
The start of the Twitter thread: [Archive.is] Kristian Köhntopp on Twitter: “A lot of people right now that git is an API and triggering CI/CD pipelines with empty commits replaces the equivalent of a Kubernetes controller for their fragile pile of bash in git triggers. This is broken and begets more brokenness. Evidence:… “
The tweet that started the subtweet: [Archive.is] Florian Haas on Twitter: “(For anyone wondering, what’s nice about this one is it works in any CI. So you don’t have to remember how to manually kick off a GitLab CI pipeline or GitHub Action or Zuul job, you just push an empty commit and off you go.)”
Other relevant tweets:
Yes, you want to avoid shell too (anything like for instance sh, ash, dash, bash or zsh), but you have to know it (and understand why to avoid it) as often it is the only interactive way to access systems from the console.
And of course Kris also wrote a big document on that too, which is available as full PDF (Wayback), full HTML (Wayback) and chaptered HTML Die UNIX Shell /bin/sh.
But more importantly, Kris wrote [Wayback/Archive.is] Using Python to bash | Die wunderbare Welt von Isotopp which is about using Python to do things you might be tempted to do in the shell. It quotes
Shell is a thing you want to understand and then not use, because you learned to understand it.
which is from the German post in thread [Wayback/Archive.is] Bashprogrammierung, wo gehts am besten los which quotes Kris’ 1998 message:
From kris Tue Sep 1 11:26:12 1998
From: kris
Newsgroups: de.comp.os.unix.misc
Subject: Re: Shell-Frage, find, xargs, kopieren von vielen Dateien
References: <6seh24$q9a$2...@nz12.rz.uni-karlsruhe.de>
From: kr...@koehntopp.de (Kristian Koehntopp)
Alignment: chaotic/neutral
X-Copyright: (C) Copyright 1987-1998 Kristian Koehntopp -- All rights
reserved.
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Marc.Hab...@gmx.de (Marc Haber) writes:
>mir ist das ganze Zeug mit der Shell, find, xargs und Konsorten noch
>reichlich verschlüsselt.
http://www.koehntopp.de/kris/artikel/unix/shellprogrammierung/
>xargs hin oder sollte ich besser ein Perlskript schreiben?
Verwende Perl. Shell will man koennen, dann aber nicht verwenden.
Kristian
–jeroen
Posted in *nix, *nix-tools, ash/dash, ash/dash development, bash, bash, Conference Topics, Conferences, Continuous Integration, Development, DVCS - Distributed Version Control, Event, git, Power User, Scripting, sh, Sh Shell, Software Development, Source Code Management, Technical Debt | Leave a Comment »
Posted by jpluimers on 2022/04/13
I say to people: only use shell interactively, don’t write scripts. Never. Not one.
But Kris, they ask, why so radical?
Because of this:
is the literal English Google Translation of the German text
Ich sage den Leuten: benutzt Shell nur interaktiv, schreibt keine Scripte. Nie. Nicht eines.
Aber Kris, fragen sie, wieso so Radikal?
Deswegen:
then links to [Wayback/Archive] Jan Schaumann on Twitter: “TIL zgrep(1) is a shell script. BSD basically does “zcat | grep”, but GNU does “gzip -dc | sed”. How did I learn that? The fun way! CVE-2022-1271, arbitrary-file-write and code execution vulnerability in GNU zgrep / gzip. …”:
Read the rest of this entry »
Posted in *nix, *nix-tools, Apple, ash/dash, ash/dash development, bash, bash, BSD, Development, Mac, Mac OS X / OS X / MacOS, Power User, Scripting, Software Development | Leave a Comment »
Posted by jpluimers on 2022/04/12
Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi5, ESXi5.1, ESXi5.5, ESXi6, ESXi6.5, ESXi6.7, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »
Posted by jpluimers on 2022/03/17
Note before you think about putting stuff in /etc/rc.local.d/local.sh: that script will not be executed when UEFI booting.
In a very lightweight Busybox system, I wanted to modify some configuration files automatically using fragments stored in template files.
The system has diff, but no patch.
The basic idea is to use sed to insert the template files into certain spots of the configuration file when certain marker texts are not present. So I want the opposite of [Wayback] Hey Stephen Wood: Try patch instead of sed in shell scripts.
Basically the idea is a poor-man’s patch, described in Too bad: ESXi busybox has diff, but not patch « The Wiert Corner – irregular stream of stuff.
Some links that might help me with this:
One alternative would have been to use ed (which is part of the normal Busybox), but ESXi Busybox omits ed like it omits patch.
Too bad that sed commands are too different from ed commands, as I could have used diff -e on another system based on ideas here:
I might give it one more go, as vi is sort of derived from ed via ex (see vi: Creation – Wikipedia), which means that vi “colon mode” (officially command mode: [Wayback] Vim documentation: cmdline) is very similar to ed.
Another alternative would be awk, but I have done so little work with it awk, that I’m hesitating to use a new tool. Some links:
And finally, ash could be used:
The kind of modifications I am after
Below are a few links with examples of the kind of modifications I want to make. Most patch just /etc/rc.local.d/local.sh, but some others introduce other changes as well.
Note that especially with networking settings, local.sh commands might not have any effect (for instance when having slow DHCP or other network issues), see for instance [Wayback/Archive.is] I’m running ESXi 5.5 and my persistent route in local.sh is not taking effect after boot. : vmware.
There is a very convoluted way around using local.sh by using the VIB authoring tool as described in [Wayback] How to create persistent firewall rules on ESXi. It requires lowering the software acceptance level to Community Supported (esxcli software acceptance set --level=CommunitySupported), which gives you a hard time installing ESXi updates.
I got that VIB idea from [Wayback] Solved: Re: Persistent firewall rule – VMware Technology Network VMTN, as:
The local.sh file gets overwritten often with upgrades so it would mean another step during the process.
From the same thread comes [Wayback] Solved: Re: Persistent firewall rule – VMware Technology Network VMTN
set the sticky bit on your separate xml-file – then it will be backed up and persist through reboot: chmod +t
run backup manually before the first reboot: /sbin/auto-backup.sh because backup runs only once per hour
Within vSphere, one could use [Wayback] Configure ESXi Hosts with Host Profiles, but a standalone ESXi box is not part of vSphere, so that won’t work.
ESXi 7 and up
ESXi 7 makes the above harder as for instance user root cannot change file rights any more, so eventually I might revert to a VM that auto-boots when ESXi comes up, then patches the right files in place over PowerCLI (read-only) or SSH.
Need to give this some thought later:
–jeroen
Posted in *nix, *nix-tools, ash/dash, ash/dash development, Awk, BusyBox, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, PowerCLI, Scripting, sed, sed script, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »
