The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,862 other subscribers

Archive for the ‘Communications Development’ Category

Tricks used by software developers to https://127.0.0.1

Posted by jpluimers on 2021/09/07

Long interesting thread at [WayBack] Thread by @sleevi_: “@SwiftOnSecurity So, some history: It used to be folks would get certs for “localhost”, just like they would from “webmail”, despite no CA e […]”

In  2019, applications were still using tricks (including shipping private keys!) to “securely” access https://127.0.0.1 on some port.

This should have stopped in 2015, but hadn’t. I wonder how bad it still is today.

Related:

Read the rest of this entry »

Posted in Communications Development, Development, DNS, HTTP, Internet, Power User, Software Development, TCP, TLS | Leave a Comment »

For my link archive: DNS over https

Posted by jpluimers on 2021/09/02

DNS over HTTPS

For my link archive:

JSON DNS output

Some DNS over HTTSP providers support dns-json, which Cloudflare delivers non-pretty printed.

Read the rest of this entry »

Posted in Cloud, Cloudflare, Communications Development, Development, DNS, Encryption, HTTP, https, HTTPS/TLS security, Infrastructure, Internet, Internet protocol suite, Power User, Security, Software Development, TCP, TLS | Leave a Comment »

Windows chocolatey Wireshark install: ensure you install nmap too, so you have a pcap interface for capturing!

Posted by jpluimers on 2021/08/19

Wireshark is indispensable when doing network communications development or DevOps.

This is my choco-install-network-tools.bat batch file to install Wireshark and the pcap dependency which nmap provides:

choco install --yes nmap
:: wireshark requires a pcap for capturing; nmap comes with npcap which fulfills this dependency
:: see:
:: - https://chocolatey.org/packages/wireshark
:: - https://chocolatey.org/packages/win10pcap
:: - https://chocolatey.org/packages/WinPcap
:: - https://chocolatey.org/packages/nmap
choco install --yes wireshark

Yes, I know: Windows Subsystem for Linux could have an easier installation, but the above:

See:

–jeroen

Posted in *nix, *nix-tools, Communications Development, Development, Internet protocol suite, nmap, Power User, Software Development, Windows, Windows 10, Windows 8.1, WSL Windows Subsystem for Linux | Leave a Comment »

Auto connect SSH without autossh?

Posted by jpluimers on 2021/08/10

Hopefully an example ssh config will follow.

[WayBack] Jeroen Pluimers on Twitter: “Would you mind sharing a trimmed down version of your ~/.ssh/config file? The bits from your posts are a bit fragmented now, so I’ve lost the overview (:”

–jeroen

Read the rest of this entry »

Posted in *nix, Communications Development, Development, Internet protocol suite, Power User, SSH, ssh/sshd, TCP | Leave a Comment »

On my list of things to try: Amazon SES for outbound/inbound email handling

Posted by jpluimers on 2021/08/10

SES mail servers at the time of writing

*n*x:

# nslookup -type=TXT amazonses.com | grep "v=spf1"
amazonses.com   text = "v=spf1 ip4:199.255.192.0/22 ip4:199.127.232.0/22 ip4:54.240.0.0/18 ip4:69.169.224.0/20 ip4:76.223.180.0/23 ip4:76.223.188.0/24 ip4:76.223.189.0/24 ip4:76.223.190.0/24 -all"I

Windows

C:\>nslookup -type=TXT amazonses.com | find "v=spf1"
Non-authoritative answer:
        "v=spf1 ip4:199.255.192.0/22 ip4:199.127.232.0/22 ip4:54.240.0.0/18 ip4:69.169.224.0/20 ip4:76.223.180.0/23 ip4:76.223.188.0/24 ip4:76.223.189.0/24 ip4:76.223.190.0/24 -all"

These addresses use a compact CIDR notation to denote ranges of networks containing ranges of network IPv4 addresses.

CIRD processing to sendmail access file

(this is linux sendmail only)

Converting the nslookup outout to a CIDR based sendmail /etc/mail/access excerpt goes via a pipe sequence of multiple sed commands:

# nslookup -type=TXT amazonses.com | grep "v=spf1" | sed 's/\(^.*"v=spf1 ip4:\| -all"$\)//g' | sed 's/\ ip4:/\n/g' | xargs -I {} sh -c "prips {} | sed 's/$/\tRELAY/g'"
199.255.192.0   RELAY
199.255.192.1   RELAY
...
76.223.190.254  RELAY
76.223.190.255  RELAY

What happens here is this:

  1. Filter out only spf1 records using grep.
  2. Remove the head (.*v=spf1 ip4:) and tail ( -all") of the output, see [WayBack] use of alternation “|” in sed’s regex – Super User.
  3. Replaces all ip4: with newlines (so the output get split over multiple lines), see [WayBack] linux – splitting single line into multiple line in numbering format using awk – Stack Overflow.
  4. Convert the CIDR notation to individual IP addresses (as sendmail cannot handle CIDR),
    1. This uses a combination of xargs with the  sh trick to split the CIDR list into separate arguments, and prips (which prints the IP addresses for a CIDR); see:
    2. Alternatively, use
  5. Replaces all end-of-line anchor ($) with a tab followed by RELAY, see

You can append the output of this command to /etc/mail/access, then re-generate /etc/mail/access.db and restart sendmail; see for instance [WayBack] sendmail access.db by example | LinuxWebLog.com.

Without the xargs, the output would look like this:

# nslookup -type=TXT amazonses.com | grep "v=spf1" | sed 's/\(^.*"v=spf1 ip4:\| -all"$\)//g' | sed 's/\ ip4:/\n/g'
199.255.192.0/22
199.127.232.0/22
54.240.0.0/18
69.169.224.0/20
76.223.180.0/23
76.223.188.0/24
76.223.189.0/24
76.223.190.0/24

Via

–jeroen

Posted in *nix, *nix-tools, Amazon SES, Amazon.com/.de/.fr/.uk/..., Cloud, Communications Development, Development, Infrastructure, Internet protocol suite, Power User, sendmail, SMTP, Software Development | Leave a Comment »

autossh on Windows from a service: automatically starting a tunnel no matter anyone being logged on

Posted by jpluimers on 2021/08/09

There is an autossh binary for Windows available on GitHub: [WayBack] GitHub – jazzl0ver/autossh: Windows binary for autossh v1.4c.

Combined with NSSM (which for instance you can install through [WayBack] Chocolatey Software | NSSM – the Non-Sucking Service Manager) you can not only automatically build and maintain an SSH connection, but also ensure the autossh process is up and running as a service without the need for an active logon.

This allows for SSH based tunnels from and to your Windows system.

For this usage scenario, there is no need for these tools any more:

Future research:

One time steps

These are in part based on:

1. Download autoSSH

Download the most recent [WayBack] Releases · jazzl0ver/autossh · GitHub  (see below for updates).

I used the 1.4g version: [WayBack] autossh.exe, then put on my Windows PATH.

2. Install NSSM

Since it is on chocolatey ([WayBack] Chocolatey Software | NSSM – the Non-Sucking Service Manager 2.24.101.20180116), this will suffice:

choco install --yes nssm

3 .Prepare remote computer so it allows enough SSH retries

Check the value of MaxAuthTries in /etc/ssh/sshd_config.

# grep MaxAuthTries /etc/ssh/sshd_config MaxAuthTries 1

The value needs to be at least 3 or higher for ssh-copy-id to work properly.

When changing the value, be sure to restart the sshd daemon.

Without a low value of MaxAuthTries in /etc/ssh/sshd_config, ssh-copy-id will give an error ERROR: Received disconnect from myRemoteComputer port 2222:2: Too many authentication failures.

See also these link via [WayBack ]“INFO: attempting to log in with the new key(s), to filter out any that are already installed” “Too many authentication failures” – Google Search:

4. Temporarily allow the remote account to perform interctive logon

Temporarily change the user shell to /bin/bash to allow [WayBack] ssh-copy-id to work at all.

This is explained in more detail by [WayBack] shell – ssh dissable login, but allow copy-id – Server Fault.

5. Generate public and private key pairs

You need an ssh public and private key, then transfer this to your Windows client. You can for instance use these as a base:

For instance (where myLocalUser is the local user generate the key-pair for for, and myRemoteUser plus myRemoteComputer is the remote user and computer you want to autossh to):

  • ssh-keygen -t rsa -b 4096 -f %UserProfile%\.ssh\id_rsa_myLocalUser@%ComputerName%_autossh_myRemoteUser@myRemoteComputer
  • ssh-keygen -t ed25519 -f %UserProfile%\.ssh\id_ed25519_myLocalUser@%ComputerName%_autossh_myRemoteUser@myRemoteComputer

6. install git (for ssh-copy-id and bash)

Since git includes ssh-copy-id (which you need in the next step, it is at %Program Files%\Git\usr\bin\ssh-copy-id) and git is on chocolatey ([WayBack] Chocolatey Software | Git (Install) 2.23.0):

choco install --yes git.install --params "/GitAndUnixToolsOnPath /NoGitLfs /SChannel /NoAutoCrlf /WindowsTerminal"

7. Copy the public parts of the generated key pairs to the remote account on the remote machine

Use bash with ssh-copy-id to transfer the generated public keys to a remote system (replace 2222 with the SSH port number on the remote computer; often it is just 22):

pushd %UserProfile%\.ssh
bash -c "ssh-copy-id -i %UserProfile%\.ssh\id_rsa_myLocalUser@%ComputerName%_autossh_myRemoteUser@myRemoteComputer -p 2222 myRemoteUser@myRemoteComputer"
bash -c "ssh-copy-id -i %UserProfile%\.ssh\id_ed25519_myLocalUser_%ComputerName%_autossh_myRemoteUser@myRemoteComputer -p 2222 myRemoteUser@myRemoteComputer"
popd

This sounds overly complicated, but is the only way to incorporate the environment variables.

8. Test with ssh, then with autossh

These two ssh commands should succeed; choose the one for which you prefer the rsa or ed25519 algorithm.

  • ssh -i %UserProfile%\.ssh\id_rsa_myLocalUser@%ComputerName%_autossh_myRemoteUser@myRemoteComputer -p 2222 myRemoteUser@myRemoteComputer
  • ssh -i %UserProfile%\.ssh\id_ed25519_myLocalUser_%ComputerName%_autossh_myRemoteUser@myRemoteComputer -p 2222 myRemoteUser@myRemoteComputer

After this, try with autossh:

  • autossh -M 0 -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -i %UserProfile%\.ssh\id_rsa_myLocalUser@%ComputerName%_autossh_myRemoteUser@myRemoteComputer -p 2222 myRemoteUser@myRemoteComputer
  • autossh -M 0 -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -i %UserProfile%\.ssh\id_ed25519_myLocalUser_%ComputerName%_autossh_myRemoteUser@myRemoteComputer -p 2222 myRemoteUser@myRemoteComputer

This disables the autossh port monitoring (the -M 0 option, but uses a combination of interval/count-max from ssh itself to monitor the connection (the -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" options).

Note that there is no default monitoring port, as it can be any one: [WayBack] linux – What is the default monitoring port for autossh? – Super User

9. Install autossh as a service

a

Steps

  1. a
  2. b
  3. c
  4. d
  5. e

SSH logon

Depending on which algorithm you like most, use either of the below 2 (replace 2222 with the SSH port number on the remote computer; often it is just 22):

  • ssh -i %UserProfile%\.ssh\id_rsa_myLocalUser@%ComputerName%_autossh_myRemoteUser@myRemoteComputer -p 2222 myRemoteUser@myRemoteComputer
  • ssh -i %UserProfile%\.ssh\id_ed25519_myLocalUser_%ComputerName%_autossh_myRemoteUser@myRemoteComputer -p 2222 myRemoteUser@myRemoteComputer

 

C:\Users\jeroenp>ssh-keygen -t ed25519 -f %UserProfile%\.ssh\id_ed25519_myUser_%ComputerName%_autossh_revue
Generating public/private ed25519 key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in C:\Users\jeroenp\.ssh\id_ed25519_myUser_D10U003_autossh_revue.
Your public key has been saved in C:\Users\jeroenp\.ssh\id_ed25519_myUser_D10U003_autossh_revue.pub.
The key fingerprint is:
SHA256:6qjzXhQtZpTzU6aryHMYuwVs5b4a/2COKxFGFQj0Eg4 jeroenp@D10U003
The key's randomart image is:
+--[ED25519 256]--+
|E+ oo...         |
|o =  .o.  o      |
| + .  *o.+       |
|  +. = o+        |
| . .+ o So       |
|  ...+ ..        |
|   o.=B.         |
|  o *@oo         |
|  .*O*=..        |
+----[SHA256]-----+

C:\Users\jeroenp>ssh-keygen -t rsa -b 4096 -f %UserProfile%\.ssh\id_rsa_myUser_%ComputerName%_autossh_revue
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in C:\Users\jeroenp\.ssh\id_rsa_myUser_D10U003_autossh_revue.
Your public key has been saved in C:\Users\jeroenp\.ssh\id_rsa_myUser_D10U003_autossh_revue.pub.
The key fingerprint is:
SHA256:WaWRoAnr4OuXAnc+MekpbdnNto71SgdMykp7XqylQr8 jeroenp@D10U003
The key's randomart image is:
+---[RSA 4096]----+
|    .   .....    |
|     o o  .+     |
|  . . o . o      |
| . o . + o       |
|  . o.o S        |
|. .o*o . .       |
| o.*oO.o* .      |
| .o %ooO+o       |
|  .= oE++o.      |
+----[SHA256]-----+

NSSM

NSSM is really cool to run any application as a service: [WayBack] NSSM – the Non-Sucking Service Manager

nssm is a service helper which doesn’t suck. srvany and other service helper programs suck because they don’t handle failure of the application running as a service. If you use such a program you may see a service listed as started when in fact the application has died. nssm monitors the running service and will restart it if it dies. With nssm you know that if a service says it’s running, it really is. Alternatively, if your application is well-behaved you can configure nssm to absolve all responsibility for restarting it and let Windows take care of recovery actions.

nssm logs its progress to the system Event Log so you can get some idea of why an application isn’t behaving as it should.

nssm also features a graphical service installation and removal facility. Prior to version 2.19 it did suck. Now it’s quite a bit better.

After installing, everything is command-line based (I cut away some blank lines for readability):

C:\bin\bin>nssm --help
NSSM: The non-sucking service manager
Version 2.24-101-g897c7ad 64-bit, 2017-04-26
Usage: nssm [ ...]

To show service installation GUI:

        nssm install []

To install a service without confirmation:

        nssm install   [ ...]

To show service editing GUI:

        nssm edit 

To retrieve or edit service parameters directly:

        nssm dump 
        nssm get   []
        nssm set   [] 
        nssm reset   []

To show service removal GUI:

        nssm remove []

To remove a service without confirmation:

        nssm remove  confirm

To manage a service:

        nssm start 
        nssm stop 
        nssm restart 
        nssm status 
        nssm statuscode 
        nssm rotate 
        nssm processes 

Windows binary autossh version

If it is behind on [WayBack] autossh (see version history at [WayBack] autossh/CHANGES.txt), then just ask for a new version; usually it gets built and released quickly: [WayBack] Any plans for 1.4g? · Issue #3 · jazzl0ver/autossh · GitHub

[WayBack] Releases · jazzl0ver/autossh · GitHub  at the time of writing:

–jeroen

Posted in *nix, *nix-tools, Communications Development, Development, Internet protocol suite, Power User, SSH, TCP | Leave a Comment »

Firefox: disable DNS over HTTPS (which they call TTR)

Posted by jpluimers on 2021/08/03

There are many reasons to disable DNS over HTTPS (DoH), of which enough are discussed in the links below.

Disabling DoH always talks about setting TTR (the abbreviation Mozilla uses for it) to 5 (like [WayBack] Thread by @isotopp: “Firefox is about to break DNS by enabling DNS-over-HTTP by default […]”), but hardly ever explains the meaning of 5, or any other potential values.

After some searching, I found [WayBack] Firefox disable trr | Knowledge Base:

  • 0: Off by default
  • 1: Firefox chooses faster
  • 2: TRR default w/DNS fallback
  • 3: TRR only mode
  • 5: Disabled

I imagine the setting we’re all looking for is: user_pref(“network.trr.mode”, 5); (emphasis mine)

It pointed me to [WayBack] Trusted Recursive Resolver – MozillaWiki:

Read the rest of this entry »

Posted in Cloud, Cloudflare, Communications Development, Development, DNS, Firefox, Infrastructure, Internet protocol suite, Power User, TCP, Web Browsers | Leave a Comment »

Filezilla: figuring out the cause of “Connection timed out after 20 seconds of inactivity”

Posted by jpluimers on 2021/07/21

On one of my Raspberry Pi boxes, somehow I could not access files over SFTP (SSH File Transfer Protocol) via FileZilla.

I would consistently get this error:

"Connection timed out after 20 seconds of inactivity"

Figuring the exact cause took a while.

TL;DR: SFTP uses an interactive non-login shell, then interprets the output from that shell. For that kind of shell, ensure few or none scripts run that output text.

These links finally got me to the cause

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, Communications Development, Conference Topics, Conferences, Development, Event, Internet protocol suite, Power User, Scripting, SFTP, Software Development, SSH, TCP | Leave a Comment »

Many http headers via 🔎Julia Evans🔍 on Twitter: “some security headers… “

Posted by jpluimers on 2021/07/20

An image on CORS will follow; likely more on related topics too. [WayBack] 🔎Julia Evans🔍 on Twitter: “some security headers… “ about:

Interesting comments in the thread.

More to follow: [Archive.is] 🔎Julia Evans🔍 on Twitter: “going to talk about CORS headers on a different page because that’s a Whole Thing but i’d love to know what else I left out / got wrong here :)” including these:

Read the rest of this entry »

Posted in Communications Development, Development, Encryption, HTTP, https, HTTPS/TLS security, Internet protocol suite, Power User, Security, TCP | Leave a Comment »

email file decoding: Encode/Decode Quoted Printable – Webatic

Posted by jpluimers on 2021/03/26

For my link archive: [WayBack] Encode/Decode Quoted Printable – Webatic.

It did a splendid job at decoding email files in MIME format Quoted-printable.

–jeroen

Posted in *nix, *nix-tools, Communications Development, Development, eMail, Encoding, Internet, Internet protocol suite, Power User, sendmail, SMTP, SocialMedia, Software Development | Leave a Comment »