The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,862 other subscribers

Archive for the ‘Internet protocol suite’ Category

GitHub – TimeToogo/tunshell: Remote shell into ephemeral environments 🐚 🦀

Posted by jpluimers on 2021/11/25

Cool: [Wayback/Archive.is] GitHub – TimeToogo/tunshell: Remote shell into ephemeral environments 🐚 🦀

Via: [Archive.is] Jan Schaumann on Twitter: “This looks neat: on-demand remote shell into ephemeral environments, e.g. CI/CD pipeline container. Both sides fetch a client, use rendezvous server to negotiate session info, then establish connection or fall back to proxy through rendezvous. “

Read the rest of this entry »

Posted in Communications Development, Development, DevOps, HTTP, Infrastructure, Internet protocol suite, Power User, Software Development, TCP, WebSockets | Leave a Comment »

Some scripts and tips for easing the maintenance of a postfix based SMTP system

Posted by jpluimers on 2021/11/24

A few scripts and tips I found Googling around.

Deleting queued messages by regular expression pattern

I have seen the below delete-from-mailq.pl script numerous time, usually without any attribution (for instance [Wayback] Postfix Flush the Mail Queue – nixCraft and  [Wayback] postfix-delete.pl – Following script deletes all mail from the mailq which matches the regular expression specified as the first argument · GitHub).

The earliest version I could find was in [Wayback] ‘Re: delete messages from mailq’ – MARC by [Wayback] ‘Ralf Hildebrandt ‘ posts – MARC:

--- snip ---
#!/usr/bin/perl

$REGEXP = shift || die "no email-adress given (regexp-style, e.g. bl.*\@yahoo.com)!";

@data = qx</usr/sbin/postqueue -p>;
for (@data) {
  if (/^(\w+)\*?\s/) {
     $queue_id = $1;
  }
  if($queue_id) {
    if (/$REGEXP/i) {
      $Q{$queue_id} = 1;
      $queue_id = "";
    }
  }
}
                                
#open(POSTSUPER,"|cat") || die "couldn't open postsuper" ;
open(POSTSUPER,"|postsuper -d -") || die "couldn't open postsuper" ;

foreach (keys %Q) {
  print POSTSUPER "$_\n";
};
close(POSTSUPER);
--- snip ---

And then use:
% delete-from-mailq "^test"

 

Tips

[Wayback] How do I check the postfix queue size? – Server Fault

Lots of great answers and pointers to useful guides/software there.

qstat

[Wayback] Postfix Bottleneck Analysis points to [Wayback] Postfix manual – qshape(1): qshape - Print Postfix queue domain and age distribution, then explains about different scenarion and queues:

postqueue

postqueue -p | tail -n 1

Last line in the postqueue -p shows how many requests and size:

-- 317788 Kbytes in 11860 Requests.

View queues size

I tried finding the original posting of the below script, but could not. If you find it, please let me know.

#!/usr/bin/env perl

# postfix queue/s size
# author: 
# source: http://tech.groups.yahoo.com/group/postfix-users/message/255133

use strict;
use warnings;
use Symbol;
sub count {
        my ($dir) = @_;
        my $dh = gensym();
        my $c = 0;
        opendir($dh, $dir) or die "$0: opendir: $dir: $!\n";
        while (my $f = readdir($dh)) {
                if ($f =~ m{^[A-F0-9]{5,}$}) {
                        ++$c;
                } elsif ($f =~ m{^[A-F0-9]$}) {
                        $c += count("$dir/$f");
                }
        }
        closedir($dh) or die "closedir: $dir: $!\n";
        return $c;
}
my $qdir = `postconf -h queue_directory`;
chomp($qdir);
chdir($qdir) or die "$0: chdir: $qdir: $!\n";
printf "Incoming: %d\n", count("incoming");
printf "Active: %d\n", count("active");
printf "Deferred: %d\n", count("deferred");
printf "Bounced: %d\n", count("bounce");
printf "Hold: %d\n", count("hold");
printf "Corrupt: %d\n", count("corrupt");

Various commands

[Wayback] Inspecting Postfix’s email queue – Tech-G explaining about:

  • mailq
  • postqueue -p
  • postcat -vq XXXXXXXXXX (where XXXXXXXXXX is the message ID)
  • postqueue -f / postfix flush
  • postsuper -d to delete messages

More of these in [Wayback] Postfix Mail Queue Management – Linux Hint and [Wayback] Postfix Bottleneck Analysis: queues.

Makefile

Based on [Wayback] Using “make” for Postfix file maintenance

MAPS = relays.db aliases.db transport.db relocated.db \
        virtual.db sender_checks.db rejected_recips.db \
        helo_access.db

all : $(MAPS)

aliases.db : aliases
        newaliases

%.db : %
        postmap $*

This is my Makefile that runs fine on Tumbleweed (note: all 8-space indents are TAB characters):

MAPS =  /etc/aliases.db \
        transport.db \
        virtual.db \
        helo_access.db \
        canonical.db \
        sasl_passwd.db \
        relocated.db \
        relay.db \
        access.db \
        relay_ccerts.db \
        sender_canonical.db

all : $(MAPS)

aliases.db : aliases
        @echo "Rebuilding $@."
        newaliases

%.db : %
        @echo "Rebuilding $@."
        postmap $*

In the future, I might try [Wayback] Makefile.postfix · GitHub, though I think it is convoluted:


## Postfix: Makefile to update *.db files
POSTCONF= /usr/sbin/postconf
POSTMAP= /usr/sbin/postmap
default: postmap
postmap: Makefile.postmap
@echo 'Updating database files …'
$(MAKE) -f Makefile.postmap
Makefile.postmap: main.cf
@echo 'Updating $@ …'
@set -e; \
rm -f $@.$$$$.tmp; \
echo 'POSTMAP=$(POSTMAP)' >>$@.$$$$.tmp; \
echo 'postmap::' >>$@.$$$$.tmp; \
config_directory="$(PWD)"; \
{ $(POSTCONF) -c $(PWD) || kill $$$$; } \
|tr ' ' '\n' \
|sed -n \
-e 's/,$$//' \
-e 's#^hash:\$$config_directory/##p' \
-e 's#^hash:'"$$config_directory/##p" \
|sort -u \
|while read mapfile; do \
echo "postmap:: $$mapfile.db" >>$@.$$$$.tmp; \
echo "$$mapfile.db: $$mapfile" >>$@.$$$$.tmp; \
echo " \$$(POSTMAP) $$<" >>$@.$$$$.tmp; \
done; \
mv $@.$$$$.tmp $@

 

 

[Wayback] Ralf Hildebrandt

Ralf Hildebrandt is an active and well-known figure in the Postfix community. He’s a systems engineer for T-NetPro, a German telecommunications company and has spoken about Postfix at industry conferences and contributes regularly to a number of open source mailing lists.

Co-author of this book: [Wayback: Book of Postfix State-of-the-Art Message Transport ISBN 9781593270018] (which used to have its own site: [Wayback: The Book of Postfix]

Book of Postfix

State-of-the-Art Message Transport

By Patrick KoetterRalf Hildebrandt

Publisher: No Starch PressRelease Date: March 2005Pages: 496

Best practices for Postfix–the popular alternative to Sendmail. Developed with security and speed in mind, Postfix has become a popular alternative to Sendmail and comes preinstalled in many Linux distributions as the default mailer. The Book of Postfix is a complete guide to Postfix whether used at home, as a mailrelay or virus-scanning gateway, or as a company mailserver. Practical examples show how to deal with daily challenges like protecting mail users from spam and viruses, managing multiple domains, and offering roaming access.

This is a great review of the book: [Wayback] The Book of Postfix (Ralf Hildebrandt, Patrick Koetter)

Related

For my postfix studies… « The Wiert Corner – irregular stream of stuff

–jeroen

 

Posted in *nix, *nix-tools, bash, Communications Development, Development, Internet protocol suite, Makefile, postfix, Power User, Scripting, SMTP, Software Development | Leave a Comment »

Random User Generator | Home

Posted by jpluimers on 2021/11/23

Cool tool for when you ever need random users to test a system [Wayback] Random User Generator | Home:

Random user generator is a FREE API for generating placeholder user information. Get profile photos, names, and more. It’s like Lorem Ipsum, for people.

This was used when extracting Parler data to substantiate evidence around the 20210106 USA Capitol riots.

You can even use a simple HTTP GET like [Wayback] randomuser.me/api and get a JSON result like this.

{"results":[{"gender":"female","name":{"title":"Miss","first":"Malou","last":"Mortensen"},"location":{"street":{"number":2669,"name":"Lyngbyvej"},"city":"Sundby","state":"Syddanmark","country":"Denmark","postcode":48047,"coordinates":{"latitude":"-35.1307","longitude":"113.7480"},"timezone":{"offset":"+1:00","description":"Brussels, Copenhagen, Madrid, Paris"}},"email":"malou.mortensen@example.com","login":{"uuid":"981747de-66fe-40b0-87ea-adfe403fe1be","username":"purpleostrich871","password":"sweets","salt":"x86aQbIB","md5":"55497ac53530b428f98b9d36267ceeef","sha1":"358b94ffabe7d827c34da15791e5d6717c594428","sha256":"6e357e887877e29b7e6d53073f648174382c53c24f83479e25fed9c82075ed32"},"dob":{"date":"1995-06-05T04:50:35.145Z","age":26},"registered":{"date":"2018-07-21T00:59:50.523Z","age":3},"phone":"02990797","cell":"94800012","id":{"name":"CPR","value":"050695-9954"},"picture":{"large":"https://randomuser.me/api/portraits/women/27.jpg","medium":"https://randomuser.me/api/portraits/med/women/27.jpg","thumbnail":"https://randomuser.me/api/portraits/thumb/women/27.jpg"},"nat":"DK"}],"info":{"seed":"8971869bb62b73d7","results":1,"page":1,"version":"1.3"}}

Via:

–jeroen

Read the rest of this entry »

Posted in Communications Development, Development, HTTP, Internet protocol suite, JavaScript/ECMAScript, JSON, Python, REST, Scripting, Software Development, TCP | Leave a Comment »

🔎Julia Evans🔍 on Twitter: “ten questions about UDP”

Posted by jpluimers on 2021/11/10

From a while back, but still relevant when you learn all your life:

[Archive.is] 🔎Julia Evans🔍 on Twitter: “ten questions about UDP: “

[WayBack] UDP questions

Hello! Here are some questions & answers. The goal isn’t to get all the questions “right”. Instead, the goal is to learn something! If you find a topic you’re interested in learning more about, I’d encourage you to look it up and learn more

–jeroen

Read the rest of this entry »

Posted in Communications Development, Development, Internet protocol suite, Software Development, UDP | Leave a Comment »

Some links on SMTP tar-pit to lessen SPAM

Posted by jpluimers on 2021/11/03

Some links for my archive; note that pure tar-pits by now are also hampering large email sender services like SendGrid, Mailgun and Amazon SES.

So the below links are for educational and historic purposes only.

I assembled these links because out of a sudden, Ring 2FA verification emails could not be delivered any more.

Ring 2FA came mandatory towards the end of February 2020.

Some links on that:

Sendmail timeouts:

–jeroen

Read the rest of this entry »

Posted in *nix, Communications Development, Development, HIS Host Integration Services, Internet protocol suite, Power User, SMTP | Leave a Comment »

Windows: unblocking SMB/NetBIOS/CIFS/File-and-Printer-sharing traffic from other subnets

Posted by jpluimers on 2021/10/29

If you enable File and Printer sharing on Windows, by default the firewall only enables it on private networks for the local subnet as remote address (for domain networks, it allows “Any”) as seen on the picture below.

When your network consists of multiple subnets, for instance when it is large, or multiple sites are connected via site-to-site VPN (often called LAN-to-LAN VPN) solutions, then these subnets cannot access each others files or printers.

Realising these default blocks, they are easy to resolve as explained in for instance [WayBack] Windows firewall blocking network shares through VPN server – Server Fault by [WayBack] Brian:

I realize this is almost three years late, but I just spent today fighting with the same problem. I did get it working, so I figured I’d share. Note that I’m using a Windows 7 PC as the file server; other versions might need slightly different configuration.

In the “Windows Firewall with Advance Security”, there are several “File and Printer Sharing” rules:

  • File and Printer Sharing (NB-Datagram-In)
  • File and Printer Sharing (NB-Name-In)
  • File and Printer Sharing (NB-Session-In)
  • File and Printer Sharing (SMB-In)

(There are additional rules, but I didn’t care about printer sharing. The same changes would apply if you want those.)

File and Printer Sharing appears to default to “Local subnet” only. You’ll need to add the subnet of your VPN clients.

Modify each of those rules as follows:

  1. Open the Properties dialog for the rule.
  2. Navigate to the Scope tab.
  3. In the Remote IP address section, the “These IP addresses” radio button should be selected.
  4. Click “Add…” next to the list of addresses. By default, only “Local subnet” is in the list.
  5. In the “This IP address or subnet:” field, enter the subnet assigned to your VPN clients (this is probably 192.168.1.0/24 in the OP, but if not, it’s the subnet assigned to the VPN adapter on the client side), then click OK.
  6. If you’re also using IPv6, add the VPN client IPv6 subnet as well.

That was enough for me to access file shares over the VPN.

(If you want to do it manually, you need to open TCP ports 139 and 445, and UDP ports 137 and 138, in the file server’s firewall.)

Hopefully I will find some time in the future to automate this using PowerShell, as netsh names are localised do hard to make universal.

These links might help me with that:

Read the rest of this entry »

Posted in Communications Development, Development, Internet protocol suite, Power User, SMB, TCP, Windows | Leave a Comment »

Filippo Valsorda on Twitter: “whoami.filippo.io , the SSH server that knows who you are … Try it out! $ ssh http://whoami.filippo.io”

Posted by jpluimers on 2021/10/20

[Archive.is] Filippo Valsorda on Twitter: “whoami.filippo.io , the SSH server that knows who you are, got some newly refreshed intel! Try it out! $ ssh whoami.filippo.io

The server itself has some HTML with information too whoami.filippo.io redirecting to [WayBack] ssh whoami.filippo.io (source code is at [WayBack] GitHub – FiloSottile/whoami.filippo.io: A ssh server that knows who you are. $ ssh whoami.filippo.io).

It’s a cool open source server written in Golang, that gets all your public ssh keys (ssh automatically transmits those) and tries to map them back to a GitHub account.

In addition it shows you some potential vulnerabilities of your ssh client.

Note that in October 2020, it was temporarily down, but it will be up again: [Archive.is] Filippo Valsorda 💉💉 on Twitter: “Yeah I’m planning to but I can’t give you an ETA I’m afraid. A few weeks, maybe?… “

Thread comments

Some interesting comments in the thread:

Related: [WayBack] Auditing GitHub users’ SSH key quality

Stop presenting public keys

[WayBack] GitHub – FiloSottile/whoami.filippo.io: A ssh server that knows who you are. $ ssh whoami.filippo.io: How do I stop passing public keys

How do I stop it?

If this behavior is problematic for you, you can tell ssh not to present your public keys to the server by default.

Add these lines at the end of your ~/.ssh/config (after other “Host” directives)

Host *
    PubkeyAuthentication no
    IdentitiesOnly yes

And then specify what keys should be used for each host

Host example.com
    PubkeyAuthentication yes
    IdentityFile ~/.ssh/id_rsa
    # IdentitiesOnly yes # Enable ssh-agent (PKCS11 etc.) keys

If you want you can use different keys so that they can’t be linked together

Host github.com
    PubkeyAuthentication yes
    IdentityFile ~/.ssh/github_id_rsa

–jeroen

Read the rest of this entry »

Posted in *nix, *nix-tools, Communications Development, Development, Go (golang), Internet protocol suite, Power User, Software Development, SSH, ssh/sshd, TCP | Leave a Comment »

The browser wars that started on iOS (forcing Safari) and Android (forcing Chrome) now are continued on Windows 11 (forcing Edge)

Posted by jpluimers on 2021/10/05

Via:

 

Posted in Awareness, Development, HTTP, Internet protocol suite, Software Development, TCP, TLS, URI, Web Development | Leave a Comment »

Solved: ‘Answering Yes to “You have an older version of PackageManagement known to cause issues with the PowerShell extension. Would you like to update PackageManagement (You will need to restart the PowerShell extension after)?” hung my Visual Studio Code.…’

Posted by jpluimers on 2021/10/04

From a while back: [Archive.is] Jeroen Wiert Pluimers on Twitter: ‘Answering Yes to “You have an older version of PackageManagement known to cause issues with the PowerShell extension. Would you like to update PackageManagement (You will need to restart the PowerShell extension after)?” hung my Visual Studio Code.… ‘

After clicking “Yes”, the the only thing visible was this notification that had an ever running “progress bar”:

Notifications - Powershell - Source: Powershell (Extension)

Notifications – Powershell – Source: Powershell (Extension)

The first part of the solution was relatively simple: restart Visual Studio code, then the original notification showed, and after clicking “Yes”, the “Panel” (you can toggle it with Ctrl+J) showed the “Terminal” output (yes, I was working on [Wayback/Archive.is] PowerShell script for sending Wake-on-LAN magic packets to given machine hardware MAC address, more about that later):

Read the rest of this entry »

Posted in .NET, Communications Development, Development, Encryption, HTTP, HTTPS/TLS security, Internet protocol suite, Power User, Security, Software Development, TCP, Visual Studio and tools, vscode Visual Studio Code, Windows, Windows 10 | Leave a Comment »

One of the Let’s Encrypt’s Root Certificates expired today (and their corresponding intermediate yesterday); how is your infrastructure doing?

Posted by jpluimers on 2021/09/30

Last weekend I published 5 days before the Let’s Encrypt’s Root Certificate is expiring!

It basically was a post trying to amplify the [Wayback/Archive.isLet’s Encrypt’s Root Certificate is expiring! message by [WaybackScott Helme .

Yesterday and today, he is maintaining a Twitter thread on things that have broken.

Quite a few things have, including some versions of curl, on which a lot of infrastructure relies (the certificate for it got fixed later on 20120930), see:

Two important starting points in his thread:

  1. [Archive.is] Scott Helme on Twitter: “🚨🚨🚨 5 minutes until the Let’s Encrypt R3 intermediate expires 🚨🚨🚨 29 September 2021 19:21:40 UTC”
  2. [Archive.is] Scott Helme on Twitter: “🚨🚨🚨 30 minute warning 🚨🚨🚨 IdentTrust DST Root CA X3 Expires: Sep 30 14:01:15 2021 UTC… “

If you want to check from one of your own clients, try [Archive.is] Scott Helme on Twitter: “I’ve created a test site to help identify issues with clients. If you can connect to https://t.co/bXHsnlRk8D then your client can handle being served the expired R3 Intermediate in the server chain!… “

[Wayback/Archive.is] https://expired-r3-test.scotthelme.co.uk/

Note that neither SSLabs, nor Cencys, nor CertCheckkerApp do show the expired certificate, only the new one:

Yes, I know the pluimers.com web server is rated B from a TLS perspective. Will be working on it, but I’m still recovering from rectum cancer treatments, and have an almost 1.5 year backlog to get through.

–jeroen

Posted in Communications Development, Development, Encryption, HTTP, https, HTTPS/TLS security, Internet protocol suite, Let's Encrypt (letsencrypt/certbot), Power User, Security, Software Development, TCP, TLS, Uncategorized, Web Development | Leave a Comment »