The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now20140424-VMware-Fusion-6.0.3.-no-reclaimable
    More Photos

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,839 other subscribers

Archive for the ‘*nix-tools’ Category

« Previous Entries
Next Entries »

Listing information on all active interfaces on MacOS part 1: getting the active interface names

Posted by jpluimers on 2021/07/29

Listing Listing information on all active interfaces on MacOS is a process involving multiple pieces, which then can be combined together.

Listing all active interfaces try 1

This involves both the -l (list with optional criteria) and -u parameter (the up criterion) as per excerpts from the [Archive.is] ifconfig(8) [osx man page] / [WayBack] ifconfig Man Page – macOS – SS64.com:

NAME
     ifconfig -- configure network interface parameters

SYNOPSIS
     ...
     ifconfig -l [-d] [-u] [address_family]
     ...

DESCRIPTION
     The ifconfig utility is used to assign an address to a network interface and/or configure network interface parameters.

     The following options are available:

     ...

     address_family
             Specify the address family which affects interpretation of the remaining parameters.  Since an interface can receive transmissions
             in differing protocols with different naming schemes, specifying the address family is recommended.  The address or protocol fami-
             lies currently supported are ``inet'', ``inet6'', and ``link''.  The default is ``inet''.  ``ether'' and ``lladdr'' are synonyms
             for ``link''.

     ...

     The -l flag may be used to list all available interfaces on the system, with no other additional information.  Use of this flag is mutually
     exclusive with all other flags and commands, except for -d (only list interfaces that are down) and -u (only list interfaces that are up).

Example:

ifconfig -l -u

Each interface on one line:

ifconfig -l -u | xargs -n1 echo

The problem is that on my system, it also lists bridges as active, whereas they are not:

# ifconfig -l -u | xargs -n1 echo
lo0
en1
en2
en0
p2p0
awdl0
bridge0
utun0
en10

# ifconfig bridge0
bridge0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=63<RXCSUM,TXCSUM,TSO4,TSO6>
    ether 6a:00:02:9a:23:f0 
    Configuration:
        id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0
        maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200
        root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0
        ipfilter disabled flags 0x2
    member: en1 flags=3<LEARNING,DISCOVER>
            ifmaxaddr 0 port 5 priority 0 path cost 0
    member: en2 flags=3<LEARNING,DISCOVER>
            ifmaxaddr 0 port 6 priority 0 path cost 0
    Address cache:
    nd6 options=201<PERFORMNUD,DAD>
    media: 
    status: inactive

So this is where the MacOS and BSD documentation is inaccurate.

Interface types

The above interfaces are many more than just ethernet or WiFi interfaces; there is a list at [WayBack] macos – What are en0, en1, p2p, and so on, that are displayed after executing ifconfig? – Stack Overflow by [WayBack] mcint:

In arbitrary order of my familarity / widespread relevance:

lo0 is loopback.

en0 at one point “ethernet”, now is WiFi (and I have no idea what extra en1 or en2 are used for).

fw0 is the FireWire network interface.

stf0 is an IPv6 to IPv4 tunnel interface to support the transition from IPv4 to the IPv6 standard.

gif0 is a more generic tunneling interface [46]-to-[46].

awdl0 is Apple Wireless Direct Link

p2p0 is related to AWDL features. Either as an old version, or virtual interface with different semantics than awdl.

…

many VPNs will add additional devices, often “utun#” or “utap#” following TUN/TAP (L3/L2)virtual networking devices.

More on AWDL at [WayBack] ios – What is AWDL (Apple Wireless Direct Link) and how does it work? – Stack Overflow.

Listing all active interfaces try 2

Read the rest of this entry »

Posted in *nix, *nix-tools, Apple, bash, Development, ifconfig, Mac OS X / OS X / MacOS, Power User, Scripting, Software Development | Leave a Comment »

-r argument to pipe (no argument for MacOS)- If no input is given to xargs, don’t let xargs run the utility – Unix & Linux Stack Exchange

Posted by jpluimers on 2021/07/28

TL;DR

There is a non-standard -r option to xargs that allows it to skip executing when there are no arguments at all.

On some operating systems, the -r is default.

MacOS has no -r, but does not execute xargs if there are no arguments given.

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, Development, Power User, Scripting, Software Development, xargs | Leave a Comment »

Listing information on all active interfaces on MacOS part 2: adding DHCP/BOOTP and routing details

Posted by jpluimers on 2021/07/27

This is a continuation of yesterdays

Listing information on all active interfaces on MacOS part 1: getting the active interface names.

It is based on ideas in these StackExchange posts:

I threw most of the implementation details in the ideas away, as they were way to much based on empirical trial and error, than proper research.

So I tried doing the research and came up with the things below.

Getting the IPv4 address and DHCP/BOOTP information of a NIC

By using the ipconfig command, you can get specific details for a NIC like an IPv4 (with the getifaddr) or DHCP (with the getpacket option to get the latest DHCP packet):

for i in $(ifconfig -l -u); do if ifconfig $i | grep -q "status: active" ; then echo $i; fi; done | xargs -n1 -I_nic_ sh -c 'echo "_nic_: $(ipconfig getifaddr _nic_)"'

or DHCP/BOOTP:

for i in $(ifconfig -l -u); do if ifconfig $i | grep -q "status: active" ; then echo $i; fi; done | xargs -n1 -I_nic_ sh -c 'echo "_nic_: $(ipconfig getpacket _nic_)"'

The latter returns a very long list, which I wanted to shorten into a more readable format.

ipconfig syntax

You can find more information in the [Archive.is] ipconfig(8) [osx man page] / [WayBack] ipconfig Man Page – macOS – SS64.com excerpt:

Read the rest of this entry »

Posted in *nix, *nix-tools, Apple, bash, Development, DNS, ifconfig, Mac OS X / OS X / MacOS, Power User, Scripting, Software Development | Leave a Comment »

Filezilla: figuring out the cause of “Connection timed out after 20 seconds of inactivity”

Posted by jpluimers on 2021/07/21

On one of my Raspberry Pi boxes, somehow I could not access files over SFTP (SSH File Transfer Protocol) via FileZilla.

I would consistently get this error:

"Connection timed out after 20 seconds of inactivity"

Figuring the exact cause took a while.

TL;DR: SFTP uses an interactive non-login shell, then interprets the output from that shell. For that kind of shell, ensure few or none scripts run that output text.

These links finally got me to the cause

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, Communications Development, Conference Topics, Conferences, Development, Event, Internet protocol suite, Power User, Scripting, SFTP, Software Development, SSH, TCP | Leave a Comment »

windows – Is there any sed like utility for cmd.exe? – Stack Overflow

Posted by jpluimers on 2021/07/19

[WayBack] windows – Is there any sed like utility for cmd.exe? – Stack Overflow

TL;DR: many people suggest to use PowerShell, but there is GNU sed in Chocolatey

The chocolatey part:

The PowerShell part: read the other answers from the above question.

–jeroen

Posted in *nix, *nix-tools, CommandLine, Power User, PowerShell, RegEx, sed, Windows | Leave a Comment »

Obtaining system information from SMB – Nmap: Network Exploration and Security Auditing Cookbook – Second Edition

Posted by jpluimers on 2021/07/09

Based on

This scans the 192.168.1.0/24 network for SMB capable machines, and extracts information from them:

nmap -p139,445 --script smb-os-discovery 192.168.1.0/24

Note that experimenting this, I found out that nmap is also available on Chocolatey: [WayBack] Chocolatey Gallery | Nmap 7.70 (heck, since 2016, no less!).

I was hoping I wrote a little batch file around this, called find-smb-hosts.on.192.168.1.network.bat, because net view is working not so well on Windows 10 any more, but that failed, so here is the batch file:

@echo off
:: only works from older versions than Windows 10
:: the delay is caused by the "net view" scanning the network
:: the first for calls ping with the hostname
:: the second for gets the IP and hostname without waiting for a ping result
for /f "usebackq tokens=1* delims=\ " %%m in (`net view ^| findstr "\\"`) do (
  for /f "usebackq tokens=2,3 delims=[] " %%h in (`ping -4 %%m -n 1 -w 1 ^| grep Pinging`) do (
    echo %%i    %%h
  )
)
goto :eof

:: output of the first for without filtering (no starting newline):
:: Server Name            Remark
::
:: -------------------------------------------------------------------------------
:: \\REVUE                Samba 4.7.3-git.30.54c196e5d35SUSE-oS15.5-x86_64
:: \\VCS-CI
:: The command completed successfully.

:: output of the second for without filtering (including the starting newline):
::
:: Pinging revue [192.168.1.62] with 32 bytes of data:
:: Reply from 192.168.1.62: bytes=32 time<1ms TTL=64
::
:: Ping statistics for 192.168.1.62:
::     Packets: Sent = 1, Received = 1, Lost = 0 (0% loss),
:: Approximate round trip times in milli-seconds:
::     Minimum = 0ms, Maximum = 0ms, Average = 0ms

The above batch file delivered many more results than this line:

nmap -p139,445 --script smb-os-discovery 192.168.71.1/24 | grep -w "\(report\|Computer name\)"

–jeroen

Posted in *nix, *nix-tools, nmap, Power User | Leave a Comment »

How to rename a VM through SSH on ESXi ? |VMware Communities

Posted by jpluimers on 2021/06/28

From [WayBack] How to rename a VM through SSH on ESXi ? |VMware Communities (numbering and code highlighting mine):

Kindly find the below:

  1. Backup the virtual machine
  2. Power down the virtual machine
  3. Remove the virtual machine from the vSphere host inventory
  4. Open an SSH console session to the vSphere host
  5. Navigate to the storage directory containing the virtual machine: For example: cd /vmfs/volumes/<datastore_name>/<original_vmname>
  6. Rename the primary .vmdk configuration files: vmkfstools -E "<original_vmname>.vmdk" "<new_vmname>.vmdk"
  7. Rename the .vmx configuration file: mv "original_vmname.vmx" "new_vmname.vmx"
  8. Edit the virtual machine .vmx configuration file (Be sure to properly update the directory and file name of the .vswp swap file reference): vi "new_vmname.vmx"
  9. Rename any remaining files in the virtual machine’s folder as needed:
    1. Rename the .vmxf configuration file: mv "original_vmname.vmxf" "new_vmname.vmxf"
    2. Rename the .nvram configuration file: mv "original_vmname.nvram" "new_vmname.nvram"
    3. Rename the .vsd configuration file: mv "original_vmname.vsd" "new_vmname.vmsd"
  10. Rename the virtual machine folder: Move up one directory level to the parent folder ( cd .. ) then rename the virtual machine directory: mv "original_directory" "new_directory"
  11. Add the newly-named virtual machine to the host’s inventory (the newly renamed .vmx configuration file)
  12. Power on the newly renamed virtual machine
  13. Answer “I moved it” to the virtual machine question prompt (not “I copied it”)
  14. Review the virtual machine and all files/folders to make sure it is named as desired and functioning properly

Note: There are other methods to allow for renaming, but this method is fairly quick and easy. It should work on all editions of vSphere from free to Enterprise Plus.

The “Answer question” prompt where you should selected “I moved it”:

->

Prompt with symlink names in the path

On a site note, I need to figure uit how to set the ESXi shell prompt to show the current path like pwd does (with symlink names in it instead of the followed symlink targets):

[root@ESXi-X9SRI-3F:~] cd /vmfs/volumes/EVO860_250GB/
[root@ESXi-X9SRI-3F:/vmfs/volumes/5c9bd516-ef1f6d4c-f1b1-0025907d9d5c] pwd
/vmfs/volumes/EVO860_250GB

The ESXi shell is based on busybox, in fact it uses the ash variety:

[root@ESXi-X9SRI-3F:/vmfs/volumes/5c9bd516-ef1f6d4c-f1b1-0025907d9d5c] `readlink -f \`which readlink\`` | grep ^BusyBox
BusyBox v1.29.3 (2018-11-02 15:37:50 PDT) multi-call binary.
BusyBox is copyrighted by many authors between 1998-2015.
[root@ESXi-X9SRI-3F:/vmfs/volumes/5c9bd516-ef1f6d4c-f1b1-0025907d9d5c] type chdir
chdir is a shell builtin

This seemed to work fine:

[root@ESXi-X9SRI-3F:/vmfs/volumes/5c9bd516-ef1f6d4c-f1b1-0025907d9d5c] PS1="[\u@\h:`pwd`] "
[root@ESXi-X9SRI-3F:/vmfs/volumes/EVO860_250GB]

But in faxt fails, as it only takes a pwd value once, and not every time the prompt is evaluated:

[root@ESXi-X9SRI-3F:/vmfs/volumes/EVO860_250GB] cd ..
[root@ESXi-X9SRI-3F:/vmfs/volumes/EVO860_250GB] pwd
/vmfs/volumes
[root@ESXi-X9SRI-3F:/vmfs/volumes/EVO860_250GB]

So I need to re-visit these links:

–jeroen

Posted in *nix, *nix-tools, BusyBox, ESXi6, ESXi6.5, ESXi6.7, Power User, Virtualization, VMware, VMware ESXi | Leave a Comment »

Hopefully datendomina (@sys_adm_ama) has followed up with some cool vi tips…

Posted by jpluimers on 2021/06/28

I hope that datendomina (@sys_adm_ama) has followed up with some cool vi/vim tips.

Though I can do basic editing (far more than quit-without-saving) and know about he various mode, I still feel not proficient.

[WayBack] Jeroen Pluimers on Twitter: “LOL! Boy was I glad that after finding my way in Ed and sed on SunOS in the 1980s, I discovered vi. Still not proficient in it (and I probably never will). However, knowing some of the basics allowed me to visually edit any file on any Unix like system. That’s still gold to me.…”

It also made me discover [WayBack] ed(1) Conference (@ed1conf) | Twitter.

One important tip:

[WayBack] Kristian Köhntopp on Twitter: “vi movement Kommandos haben System. Erkenne und lerne das System. hjkl + prefix+hjkl, Marken, prefix+jump to mark und so weiter. Und bleibe von den verblödeten Plugins weg. Die braucht kein Mensch und machen vim nur langsam im Start und kompliziert.”

The original thread, which I hoped would get longer: [WayBack] Thread by @sys_adm_ama: “Ich lerne jetzt vi(m). Klingt beknackt, oder?se […]”

Ich lerne jetzt vi(m).

Klingt beknackt, oder? Aber ich hab überlegt: ich möchte effizienter werden, meinen Kram stressfreier bewältigen. Und ich finde, es bietet sich an da an Baustellen anzusetzen, die täglich relevant sind. Und vi(m) nutze ich in der Tat täglich.

1/

Aber auch wenn ich ihn nutze und über den »Hilfe, wie komm ich aus dem Editor wieder raus?!«-Witz nur sehr müde die Augen rolle gehe ich davon aus, dass ich nicht mal einen Bruchteil der Möglichkeiten ausschöpfe, die er bietet (1. Release 1976, älter als ich!). Das ist spannend.
Das ist jetzt meine Mini-Challenge, auf die ich jeden Tag eine Viertelstunde verwenden will: wenn ich eine Funktion brauche mich nicht mehr drum herum zu hacken, sondern recherchieren wie es richtig geht und das dann gefälligst auch verinnerlichen. Mal sehen, ob das so klappt

3/

In dem Zuge will ich auch wieder verstärkt (neo)mutt in Verbindung mit vim nutzen – das wäre ein wunderbares tägliches Training 😎 Mails schreiben muss man irgendwie immer.

neomutt bietet leider auch einen Eimer voll Funktionen, die ich noch nicht ordentlich nutze. Gnah.

4/

An euren Replies erkenne ich, dass das mit dem »sich die Kürzel merken« echt heikel zu sein scheint 🤔 Wie handhabt ihr das im Alltag? Einfach ein paar Basics wie :u und CTRL+r und gut ist? Ich bin neugierig. Oder nutzt ihr alle nano? (Ich glaub, dann muss ich entfolgen) 😂
Nur als kleinen Zwischenstand: nach dem Lesen eurer Antworten schließe ich, dass ich mich mit meinem Kenntnisstand nicht verstecken muss 😂 Da hat mir der virtuelle Schulterblick schon weiter geholfen.

/5
vi(m), weil ich Admin bin und dieses Tool auf jedem System und ohne X-Geraffels üblicherweise vorfinde (wenn auch bei neueren Installationen dieser absurde „visual mode“ der Default ist 😳). Auch unter (Open)Solaris, IRIX, was weiß ich.

/6

Eben hab ich das Buch von @MasteringVim aus der Packstation gezogen (extrem vielversprechend!) – und klar, ich werde berichten 😎 Ich bin sehr gespannt.

/7ed

–jeroen

Read the rest of this entry »

Posted in *nix, *nix-tools, Development, ed, Power User, Scripting, sed, sed script, Software Development, vi/vim | Leave a Comment »

Downloading files with wget on ESXi · random writes

Posted by jpluimers on 2021/05/31

Reminder to self to check if wget on ESXi now finally supports https downloading: [WayBack] Downloading files with wget on ESXi · random writes.

In the mean time, ESXi 6.7 Update 2 and up seems to support this; so the below workaround might only be needed for ESXi 6.7 update 1 and below.

[WayBack] VMware ESXi: help downloading large ISO – Server Fault

I will likely not do this, as by now all my ESXi boxes should have been recent enough.

I will keep the article because of the BusyBox commands section below.

If so, I might finally try and write a Python wrapper for this, as I know that Python 3 on ESXi supports https, but the ESXi BusyBox does not have a built-in cURL.

Some links and notes I might need by then:

BusyBox commands

Another cool thing in the above blog post is that it shows how to dump the BusyBox built in commands.

I ran it for ESXi 6.7 with a slight trick to get the full path (using back-ticks and escaped back-ticks) and content.

Since ESXi is BusyBox based, the commands that are in /bin are not actually binaries, but each command is a symlink to the BusyBox binary. BusyBox then knows the original name of the command, so it can deduct what part to execute. This makes for a very space efficient storage scheme.

The various bits of the tricks to get the location of the BusyBox binary, so the --list parameter can be passed to it:

  • The which wget gives the full path of wget.
  • The ls -l `which wget` shows the full path of wget and the symlink target (but there is no way for ls to only show the symlink target).
  • The readlink -f `which wget` shows the full path of where /bin/wget points to: the BusyBox binary.

The main trick consists of backtick evalution, and knowing that ls cannot get you just the symlink target, but readlink can:

Now the back-tick escapes, because you cannot nest back-ticks:

  • The `readlink -f \`which wget\`` executes the BusyBox binary without arguments.
  • The `readlink -f \`which wget\`` --list executes the BusyBox binary with the --list parameter.

Note I do not like the cat --help (see [WayBack] How do I check busybox version (from busybox)? – Unix & Linux Stack Exchange) way of getting the BusyBox version, as that gets echoed to stderr.

This is the output:

Read the rest of this entry »

Posted in *nix, *nix-tools, cURL, ESXi6, ESXi6.5, ESXi6.7, Power User, Virtualization, VMware ESXi, wget | Leave a Comment »

“FIPS mode initialized” when you ssh out of an ESXi box

Posted by jpluimers on 2021/05/28

The once per console/shell logon output of FIPS mode initialized to stderr when you ssh out of an ESXi box seems to be something new since ESXi 6.7.

Since I hardly do this, it took a while to reproduce and track back the version where it was introduced and to realise why it is on stderr.

stderr in retrospect is logical: if you need to parse stdout of a job running across an ssh channel, you do not want it to get interfered with “side channel” output, hence stderr.

For a longer explanation see, for instance [WayBack] ssh “FIPS mode initialized” message to stderr – Why? – Unix and Linux | DSLReports Forums:

Keep in mind that “ssh” is used to transport a stream, as with “rsync”. What you put on “stdout” becomes part of the stream. That’s why this sort of informational message needs to go to “stderr”.

Parsing is hard, so bugs like [WayBack] Git fetcher fails on machine with FIPS enabled machines · Issue #3664 · inspec/inspec · GitHub got [WayBack] fixed in [WayBack] pull request like [WayBack] not parsing stderr, but checking for exitstatus.

Stock OpenSSH portable does not contain FIPS support

Finding back when and how FIPS support for OpenSSH was introduced provide a bit harder than I hoped for.

It appears that stock [WayBack] OpenSSH: Portable Release does not support FIPS. But there are patches on top of these files:

Many (most?) Linux distributions include a patched version like [WayBack] ssh.c in openssh located at /openssh-5.9p1 (git://pkgs.fedoraproject.org/openssh).

They integrate the patches like [WayBack] File openssh.spec of Package openssh – openSUSE Build Service.

Patches for instance look like [WayBack] openssh/openssh-5.3p1-fips.patch at master · gooselinux/openssh · GitHub which is more than a decade old (see the 2009 message [WayBack] rpms/openssh/devel openssh-5.3p1-fips.patch, NONE, 1.1 openssh-5.3p1-mls.patch, NONE, 1.1 openssh-5.3p1-nss-keys.patch, NONE, 1.1 openssh-5.3p1-selabel.patch, NONE, 1.1 openssh-5.3p1-skip-initial.patch, NONE, 1.1 .cvsignore, 1.24, 1.25 openssh.spec, 1.170, 1.171 sources, 1.24, 1.25 openssh-3.8.1p1-krb5-config.patch, 1.1, NONE openssh-4.7p1-audit.patch, 1.2, NONE openssh-5.1p1-mls.patch, 1.1, NONE openssh-5.1p1-skip-initial.patch, 1.1, NONE openssh-5.2p1-fips.patch, 1.6, NONE openssh-5.2p1-nss-keys.patch, 1.3, NONE openssh-5.2p1-selabel.patch, 1.2, NONE).

The patches seem to originate at the (now defunct) WayBack Index of /export/openssh of http://openssl.com/export/openssh/ .

In the end I found [WayBack] Mailing List Archive: OpenSSH FIPS 140-2 support using OpenSSL FIPS modules? having these quotes:

vanilla OpenSSH doesn’t support running OpenSSL in FIPS-140 mode. Some
downstream providers patch OpenSSH they deliver with their distributions
with changes to enable FIPS-140 mode.

[WayBack] Secure Shell and FIPS 140-2 – Managing Secure Shell Access in Oracle® Solaris 11.4 explains a bit of background of them.

ESXi 6.7

Binary searching for the version where this was introduced could have been a lot shorter if I had done a “FIPS mode initialized” “ESXi” – Google Search, resulting in for instance:

The final two links made me discover XSIBackup

They see be one of the few (only one?!) free backup solutions for the bare ESXi:

In addition, they have a binary for rsync version 3.1.0: [WayBack] 33HOPS | Rsync for VMWare Backup, so lees need to go to Source: ESXi 5.1 and rsync – damiendebin.net

––jeroen

Posted in *nix, *nix-tools, ESXi6.5, ESXi6.7, Power User, ssh/sshd, Virtualization, VMware, VMware ESXi | Leave a Comment »

« Previous Entries
Next Entries »