The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now20140424-VMware-Fusion-6.0.3.-no-reclaimable
    More Photos

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,862 other subscribers

Archive for the ‘*nix-tools’ Category

« Previous Entries
Next Entries »

Preventing to eject/unmount a MacOS drive (opposite of figuring out what prevents the unmount)

Posted by jpluimers on 2024/02/01

Not long after Figuring out which processes are preventing to eject/unmount my MacOS Time Machine backup USB drive, I wanted to do the opposite: prevent /Volumes/Sandisk1TB from being ejected, as this is the “built-in” MicroSD card I use to store large or infrequently used files on (ISO and other disk images, drivers, hardware and software documentation, stuff to be installed on a fresh machine).

The opposite is straightforward: have a process keep at least one handle open on the Volume as per [Wayback] macos – How do I not accidentally eject external drives? – Ask Different (thanks [Wayback] kLy, [Wayback] dan and [Wayback] gerlos):

If your important external drive is mounted on the following mount point:

/Volumes/important_disk

Then you can protect it against an accidental removal by locking this mount point as opened. For this one very simple method consists in opening Terminal and doing this basic command:

$ cd /Volumes/important_disk

To get rid of this locking, you might type within the same Terminal window:

$ cd /

or you might as well just close this Terminal window ($ exit, or ⌃+D, or ⌘+W).

…

An even more elegant way to do it is open a screen session (just type screen in Terminal) and open the mount point from that session. This way you can even close Terminal, since the session will keep running in the background, until you reattach it and stop it (so there’s no need to keep a window open if you don’t need it). I guess you can even create an Automator action for it. For tips on screen see: [Wayback] kinnetica.com/2011/05/29/using-screen-on-mac-os-x

–jeroen

Posted in *nix, *nix-tools, Apple, Mac OS X / OS X / MacOS, Power User, screen, Terminal | Leave a Comment »

Script to rename a virtual machine in ESXi 6

Posted by jpluimers on 2024/01/10

I bumped into [Wayback/Archive] Script to rename a virtual machine in ESXi 6: gist.github.com/Gremgoll/8bd91258d71fe895c0d416e2543ca2dc.

Then I forked it with the intent to make usage more clear, as the current usage on my ESXi rig (which has mos VMs not in the root of data stores) is like the bold italic portion here:

[root@X9SRI3F-ESXi:/vmfs/volumes/608be754-f21556ad-1082-0025907d9d5c/VM] vm-rename.sh NVMe980PRO_1TB/VM X9SRI-3F-W10P-NL-OFFICE X9SRI-3F-W10P-NL X9SRI-3F-W10P-NL-OFFICE
VOLNAME=NVMe980PRO_1TB/VM
DIRNAME=X9SRI-3F-W10P-NL-OFFICE
OLDNAME=X9SRI-3F-W10P-NL
NEWNAME=X9SRI-3F-W10P-NL-OFFICE
VM_DIRPATH=/vmfs/volumes/NVMe980PRO_1TB/VM/X9SRI-3F-W10P-NL-OFFICE
NW_DIRPATH=/vmfs/volumes/NVMe980PRO_1TB/VM/X9SRI-3F-W10P-NL-OFFICE
Failed to rename './X9SRI-3F-W10P-NL-flat.vmdk' to './X9SRI-3F-W10P-NL-OFFICE-flat.vmdk': The file specified is not a virtual disk (15)
renaming ./X9SRI-3F-W10P-NL-41260b40.vmem to ./X9SRI-3F-W10P-NL-OFFICE-41260b40.vmem
renaming ./X9SRI-3F-W10P-NL-41260b40.vmss to ./X9SRI-3F-W10P-NL-OFFICE-41260b40.vmss
renaming ./X9SRI-3F-W10P-NL-6a4b8f29.hlog to ./X9SRI-3F-W10P-NL-OFFICE-6a4b8f29.hlog
renaming ./X9SRI-3F-W10P-NL.nvram to ./X9SRI-3F-W10P-NL-OFFICE.nvram
renaming ./X9SRI-3F-W10P-NL.vmsd to ./X9SRI-3F-W10P-NL-OFFICE.vmsd
renaming ./X9SRI-3F-W10P-NL.vmx to ./X9SRI-3F-W10P-NL-OFFICE.vmx
renaming ./X9SRI-3F-W10P-NL.vmx.backup to ./X9SRI-3F-W10P-NL-OFFICE.vmx.backup
renaming ./X9SRI-3F-W10P-NL.vmxf to ./X9SRI-3F-W10P-NL-OFFICE.vmxf
renaming ./vmx-X9SRI-3F-W10P-NL-1093012288-1.vswp to ./vmx-X9SRI-3F-W10P-NL-OFFICE-1093012288-1.vswp
All Done. You now need to register X9SRI-3F-W10P-NL-OFFICE to the inventory.

There also was a renaming bug (see the italic line).

I managed to fix both that and added more documentation plus output.

Repositories and code

  1. Original:
  2. Fork with added VSAN functionality (see [Wayback/Archive] Revisions · Script to rename a virtual machine in ESXi 6):
  3. My fork of 2:
  4. My fix:

Code before modifying: [Wayback/Archive] gist.githubusercontent.com/jpluimers/fcc601dd41ac89f601a5174be92c841c/raw/e3683fbb6bdf1e73d65d2b784027c70cf42a5512/vm-rename

Code after modifying: [Wayback/Archive] raw.githubusercontent.com/jpluimers/vm-rename/master/vm-rename.sh.

Yes, that is not in a gist any more (see below why), it is now part of [Wayback/Archive] jpluimers/vm-rename: Script to rename a virtual machine in ESXi 6; fork from https://gist.github.com/jpluimers/fcc601dd41ac89f601a5174be92c841c as [Wayback/Archive] vm-rename/vm-rename.sh at master · jpluimers/vm-rename.

The reason was

Yesterday I described the workaround in Fork Gist to Repo on GitHub – Stack Overflow.

Queries

I found the original via [Wayback/Archive] script esxi rename vmx vmxf and related files – Google Search.

My start was [Wayback/Archive] script esxi duplicate vm and rename vmdk – Google Search which found:

  1. [Wayback/Archive] Renaming a virtual machine and its files in VMware ESXi (1029513) which has a PowerCLI script (that requires Windows to run PowerCLI on)
  2. [Wayback/Archive] Solved: How to clone a VM on a ESXI 6.5 server? – VMware Technology Network VMTN which pointed me to the next Google Search result :
  3. [Wayback/Archive] VMware esxi – Script to clone a VM without vSphere or vCenter. | GrangerX which I put in this gist:
  4. [Wayback/Archive] Script to create Linked Clones on ESXi | RedNectar’s Blog which is great, but way to complicate for my use case. It references a few other cool scripts though, and shows how to use Resource Pools in ESXi (which the Web-UI does not seem to support):
  5. [Wayback/Archive] [script] Cloning VMs using ESXi shell/admin console – Virtualizing Unraid – Unraid
  6. [Wayback/Archive] Script to clone a VM with free VMware ESXi – Rob Pomeroy (again: PowerCLI)

–jeroen

Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, Power User, PowerCLI, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

VMware ESXi: shell script to get uuid.bios and all Ethernet generatedAddress MAC address values for all VMs

Posted by jpluimers on 2023/12/28

This is a sort of follow-up on ESXi: listing virtual machines with their IP addresses where we ended with this:

I modified the above script to become this:

#!/bin/sh
vmids=`vim-cmd vmsvc/getallvms | sed -n -E -e "s/^([[:digit:]]+)s+((S.+S)?)s+([S+])s+(.+.vmx)s+(S+)s+(vmx-[[:digit:]]+)s*?((S.+)?)$/1/p"`
for vmid in ${vmids} ; do
    # powerState values:
    #   Powered off
    #   Powered on
    #   Suspended
    powerState=`vim-cmd vmsvc/power.getstate ${vmid} | sed '1d'`
    name=`vim-cmd vmsvc/get.config ${vmid} | sed -n -E -e '/(vim.vm.ConfigInfo) {/,/files = (vim.vm.FileInfo) {/ s/^ +name = "(.*)",.*?/1/p'`
    vmPathName=`vim-cmd vmsvc/get.config ${vmid} | sed -n -E -e '/files = (vim.vm.FileInfo) {/,/tools = (vim.vm.ToolsConfigInfo) {/ s/^ +vmPathName = "(.*)",.*?/1/p'`
    # For now, I choose to use only the IPv4 main address from ipAddress, which is in between (vim.vm.GuestInfo) { and net = (vim.vm.GuestInfo.NicInfo) [.
    ipAddress=`vim-cmd vmsvc/get.guest ${vmid} | sed -n -E -e '/(vim.vm.GuestInfo) {/,/net = (vim.vm.GuestInfo.NicInfo) [/ s/^ +ipAddress = "(.*)",.*?/1/p'`
    printf "VM with id %3s has power state %-11s and IPv4=%-15s (name = ${name}; vmPathName = ${vmPathName}).n" "${vmid}" "${powerState}" "${ipAddress}"
done

Now the script grew even larger in to vim-cmd-list-all-VMs-with-IPv4-MAC-uuid.sh:

Read the rest of this entry »

Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

Guess the maximum DNS Response Size… (by Jan Schaumann)

Posted by jpluimers on 2023/12/26

Every once in a while Jan Schaumann writes a long Twitter thread and saves it in a blog post. Always good ways to learn. This time it was no different: [Wayback/Archive] DNS Response Size started with

Read the rest of this entry »

Posted in Communications Development, Development, DNS, Internet, Internet protocol suite, IPv4, IPv6, Power User, TCP, tcpdump, UDP, Wireshark | Leave a Comment »

Hello “SMTP Smuggling” information released days before the Holiday season to open source SMTP server teams

Posted by jpluimers on 2023/12/24

Jan Wildeboer was mad for good reasons, though the open source projects didn’t yet seem to publicly have show their real madness, just bits like [Wayback/Archive] oss-security – Re: Re: New SMTP smuggling attack:

I'm a little confused by sec-consult's process here. They identify a
problem affecting various pieces of software including some very widely
deployed open source software, go to the trouble of doing a coordinated
disclosure, but only do that with...looking at their timeline... gmx,
microsoft and cisco?

“SMTP Smuggling” is bad, and big open source SMTP server projects like exim, postfix and sendmail needed to assess and fix/prevent the issue on very short notice: effectively confronting them with a zero-day less than a week between the information got released and the Holiday season.

That gives “deploy on Fridays” a totally different dimension.

How bad? Well, it already managed to reach this Newline – Wikipedia entry:

The standard Internet Message Format[26] for email states: “CR and LF MUST only occur together as CRLF; they MUST NOT appear independently in the body”. Differences between SMTP implementations in how they treat bare LF and/or bare CF characters have led to so-called SMTP smuggling attacks[27].

The crux of the problem is very well described by the “Postfix: SMTP Smuggling” link below: recommended reading, and the middle of [Wayback/Archive] SMTP Smuggling – Spoofing Emails Worldwide | Hacker News

…

TLDR: In the SMTP protocol, the end of the payload (email message) is indicated by a line consisting of a single dot. The line endings normally have to be CRLF, but some MTAs also accept just LF before and/or after the dot. This allows SMTP commands that follow an LF-delimited dot line to be “tunneled” through a first MTA (which requires CRLF and thus considers the commands to be part of the email message) to a second MTA (which accepts LF and thus processes the commands as real commands). For the second MTA, the commands appear to come from the first MTA, hence this allows sending any email that the first MTA is authorized to send. That is, emails from arbitrary senders under the domains associated with the first MTA can be spoofed.

…

Here are some links to keep you busy the next hours/days/weeks:

And the toots linking to background information:

Read the rest of this entry »

Posted in *nix, *nix-tools, Communications Development, Development, exim mail, Internet protocol suite, postfix, Power User, Python, Scripting, sendmail, SMTP, Software Development | Leave a Comment »

Avoid VirtualBox; use Hyper-V or VMware in stead

Posted by jpluimers on 2023/11/10

A while ago, Jilles found out why not to use VirtualBox: [Wayback/Archive] Jilles🏳️‍🌈 on Twitter: “@jpluimers Ik wil op basis van wat de Arch community schreeuwt; “Virtualbox is stom, als je geen hyper-v gebruikt vraag je om problemen”, HYPER-V maar gaan proberen.” / Twitter

The biggest problem is that VirtualBox seems to be developed ant tested for the happy path, not the failing path.

Which means that when you use it for less common scenarios, it will often fail in mysterious ways.

Back in Running ArchiveTeam Warrior version 3.2 on ESXi, I already mentioned this:

Totally agreeing with Kristian Kohntopp, I do not understand why people use VirtualBox at all: I just run in too much issues like [Archive.is] Kristian Köhntopp on Twitter: “Hint: Wenn die Installation einer Linux-Distro in Virtualbox mit wechselnden, unbekannten Fehlern scheitert, hilft es, stattdessen einmal VMware Workstation oder kvm zu probieren. In meinem Fall hat es dann *jedes* *einzelne* *Mal* mit *demselben* Iso geklappt.”.

Read the rest of this entry »

Posted in *nix, *nix-tools, ArchiveTeamWarrior, Hyper-V, InternetArchive, Linux, Power User, VirtualBox, Virtualization, VMware, WayBack machine, Windows, Windows 10, Windows 11 | 1 Comment »

GitHub – kellyjonbrazil/jc: CLI tool and python library that converts the output of popular command-line tools and file-types to JSON or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts.

Posted by jpluimers on 2023/10/17

I already knew about jq and jo (output: the echo for JSON), but not yet about jc for JSON.

Like jq is for querying (the sed for JSON), jc tries to be the universal parser of common command-line tools into JSON:

[Wayback/Archive] kellyjonbrazil/jc: CLI tool and python library that converts the output of popular command-line tools and file-types to JSON or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts.

Via [Wayback/Archive] Kris on Twitter: ““Jc” verwandelt den Output von üblichen Linux Kommandozeilen Tools in JSON. “Jc” kann auch als Import in Python Programmen verwendet werden und mit subprocess kombiniert werden. Danke, ⁦@janwalzer⁩ und ⁦@the_mutax⁩ für den Tip. “

A few weeks later though, Kris discovered that command-line tools aren’t that portable in their output format: [Wayback/Archive] Kris on Twitter: “As much as I love the idea of @kellyjonbrazil’s jc, this is not a winnable game. sigh Probably still better than a self-cooked parser, but the slightest bit of extra makes it catch fire in multiple possible ways …”

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, Development, JavaScript/ECMAScript, jo, jq, JSON, man/manual pages, mankier, Power User, Python, Scripting, Software Development | Leave a Comment »

Ookla speedtest CLI for Windows has some undocumented arguments to accept license and GDPR

Posted by jpluimers on 2023/10/11

I had speedtest-cli running on MacOS and various Linux machines, but not yet on Windows (see for instance my post Ubuntu: Fixing the myserious “Failed to stop apt-daily.timer: Connection timed out”).

[Wayback/Archive] Install and Test Internet Speed with Speedtest CLI Command Line – NEXTOFWINDOWS.COM reminded me there is a Speedtest CLI for Windows download at at [Wayback/Archive] Speedtest CLI: Internet speed test for the command line, but I am a an automation/scripting/devops person, so luckily there are also [Wayback/Archive] Chocolatey Software | Speedtest by Ookla (don’t get [Wayback/Archive] Ookla.Speedtest download, as that is the GUI version).

Both the Chocolatey and winget packages are named the same, so that is quite confusing. This is how I have set them apart:

Read the rest of this entry »

Posted in *nix, *nix-tools, Batch-Files, Chocolatey, DevOps, GDPR/DS-GVO/AVG, Internet, ISP, KPN, Notepad++, Power User, Privacy, Scripting, SpeedTest, Windows, xs4all | 2 Comments »

Getting a local VM disk to OVH (or most any VM hoster) is as easy as 1, 2, 3, 4, 5 (dd, gzip ssh, gunzip, dd)

Posted by jpluimers on 2023/09/29

This is why I love nx:

Actually, saving the OVH image first is another 1, 2, 3, 4 (ssh, dd, gzip, dd).

Speed wasn’t bad BTW: [Wayback/Archive] Jilles🏳️‍🌈 on Twitter: “Done 167772160+0 records in 167772160+0 records out 85899345920 bytes (86 GB, 80 GiB) copied, 2481.09 s, 34.6 MB/s [root@arch ~]#” / Twitter

–jeroen

Posted in *nix, *nix-tools, dd, Power User | Leave a Comment »

Viewing email in Linux using postfix’s mailq and postcat | Jeff Geerling

Posted by jpluimers on 2023/09/22

Since mail mostly “works” I use these below commands only very little and tend to forget them.

Luckily they were documented at [Wayback/Archive] Viewing email in Linux using postfix’s mailq and postcat | Jeff Geerling

Here are the most common commands I use when either developing or troubleshooting email in production:
  • mailq – print a list of all queued mail
  • postcat -vq [message-id] – print a particular message, by ID (you can see the ID along in mailq‘s output)
  • postqueue -f – process the queued mail immediately
  • postsuper -d ALL – delete ALL queued mail (use with caution—but handy if you have a mail send going awry!)

Via [Wayback/Archive] postfix process queue – Google Search two interesting answers (thanks [Wayback/A] Nicolas for asking):

Read the rest of this entry »

Posted in *nix, *nix-tools, postfix, Power User | Leave a Comment »

« Previous Entries
Next Entries »