Some notes on replacing parts of a text file with template text using sed on a Busybox system
Posted by jpluimers on 2022/03/17
Note before you think about putting stuff in
/etc/rc.local.d/local.sh: that script will not be executed when UEFI booting.
In a very lightweight Busybox system, I wanted to modify some configuration files automatically using fragments stored in template files.
The system has diff, but no patch.
The basic idea is to use sed to insert the template files into certain spots of the configuration file when certain marker texts are not present. So I want the opposite of [Wayback] Hey Stephen Wood: Try patch instead of sed in shell scripts.
Basically the idea is a poor-man’s patch, described in Too bad: ESXi busybox has diff, but not patch « The Wiert Corner – irregular stream of stuff.
Some links that might help me with this:
- [Wayback] regex – Using sed to delete all lines between two matching patterns – Stack Overflow (thanks users [Wayback] Lri and [Wayback] Akito)
- [Wayback/Archive.is] akito-libbash/setup.bash at dd91364083f13d1132d68489172bbce664b9c9c0 · theAkito/akito-libbash an actual example close to what I am after (it appends, I want to keep the end of the file in tact).
- [Wayback] bash – removing lines between two patterns (not inclusive) with sed – Stack Overflow
- [Wayback] shell – How to append multiple lines to a file – Unix & Linux Stack Exchange
- [Wayback] text processing – How to insert the content of a file into another file before a pattern (marker)? – Unix & Linux Stack Exchange
One alternative would have been to use ed (which is part of the normal Busybox), but ESXi Busybox omits ed like it omits patch.
Too bad that sed commands are too different from ed commands, as I could have used diff -e on another system based on ideas here:
- [Wayback] How can I create a sed command line from diff? – Super User
- [Wayback] Using diff -e Option to Create a Baseline diff File – Alvin Bunk
I might give it one more go, as vi is sort of derived from ed via ex (see vi: Creation – Wikipedia), which means that vi “colon mode” (officially command mode: [Wayback] Vim documentation: cmdline) is very similar to ed.
Another alternative would be awk, but I have done so little work with it awk, that I’m hesitating to use a new tool. Some links:
And finally, ash could be used:
- [Wayback] text processing – Script matching literal pattern over multiple lines? – Unix & Linux Stack Exchange
The kind of modifications I am after
Below are a few links with examples of the kind of modifications I want to make. Most patch just /etc/rc.local.d/local.sh, but some others introduce other changes as well.
- [Wayback] Shell script to automatically power on a specific VM which is powered off – VirtuallyVTrue
- [Wayback] Enable the SSH shell permanently in VMWare ESXi 6.7.0 and above
- [Wayback] Executing Commands During Boot Up In ESXi 5.1
- [Wayback] How to clone ESXi setup – VMware Technology Network VMTN
- [Wayback] Article Detail: Many current.png.xxxx on the /tmp on ESXi host causes hostd to crash and disconnect (2031839)
- [Wayback] ESXi booting faster than your SAN
- [Wayback] Shell script to automatically power on a specific VM which is powered off – VirtuallyVTrue
- [Wayback/Archive.is] Help getting GhettoVCB & cron jobs working on ESXi 6. : vmware
- [Wayback/Archive.is] Want to modify Roots CronTab : vmware
- [Wayback/Archive.is] Script Share: ESXi 6.7 re-scan iSCSI after FreeNAS VM has booted : freenas
- [Archive.is] Wayback: VMware KB: Changing the port used by SSH on an ESXi 5.0 host
- [Wayback/Archive.is] Creating custom firewall rules in VMware ESXi 5.x (2008226)
- [Wayback] Solved: Re: Persistent firewall rule – VMware Technology Network VMTN
- [Wayback] Execute ESXCLI commands during ESXi startup – The Virtualist
Note that especially with networking settings, local.sh commands might not have any effect (for instance when having slow DHCP or other network issues), see for instance [Wayback/Archive.is] I’m running ESXi 5.5 and my persistent route in local.sh is not taking effect after boot. : vmware.
There is a very convoluted way around using local.sh by using the VIB authoring tool as described in [Wayback] How to create persistent firewall rules on ESXi. It requires lowering the software acceptance level to Community Supported (esxcli software acceptance set --level=CommunitySupported), which gives you a hard time installing ESXi updates.
I got that VIB idea from [Wayback] Solved: Re: Persistent firewall rule – VMware Technology Network VMTN, as:
The
local.shfile gets overwritten often with upgrades so it would mean another step during the process.
From the same thread comes [Wayback] Solved: Re: Persistent firewall rule – VMware Technology Network VMTN
set the sticky bit on your separate xml-file – then it will be backed up and persist through reboot:
chmod +trun backup manually before the first reboot:
/sbin/auto-backup.shbecause backup runs only once per hour
Within vSphere, one could use [Wayback] Configure ESXi Hosts with Host Profiles, but a standalone ESXi box is not part of vSphere, so that won’t work.
ESXi 7 and up
ESXi 7 makes the above harder as for instance user root cannot change file rights any more, so eventually I might revert to a VM that auto-boots when ESXi comes up, then patches the right files in place over PowerCLI (read-only) or SSH.
Need to give this some thought later:
- [Wayback] virtualization – ESXi 7.X file permissions – how to buypass new security measures – Server Fault
- [Archive.is] The root account can no longer change permissions or executable files in ESXi 7.0.x (78689)
- [Wayback/Archive.is] PowerCLI, ESXi 6.5 free and Start-VM: Current license or ESXi version prohibits execution of the requested operation : homelab
PowerCLI commands on the free version are limited to commands that are “read-only”, so you can only find out information rather than perform actions, if that makes sense.
So you can find out if a VM is turned on, find it’s uptime, but you can not turn it on or reboot it
This presumably is to prevent automation without a license…
Having a quick look at the VMware KB, it looks like PowerCLI was limited on the free version to “read-only” operations from 5.0 (I.e. when it went from ESX to ESXI) along with vCLI and vSphere-Perl, so for some time from the looks of it
–jeroen
The Wiert Corner - irregular stream of stuff 
Leave a comment