The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now20140424-VMware-Fusion-6.0.3.-no-reclaimable
    More Photos

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,859 other subscribers

Archive for the ‘SMTP’ Category

« Previous Entries

Some notes on running Postfix inside a docker container

Posted by jpluimers on 2025/11/14

The plan was to run a Postfix secondary MX inside a docker container.

Below are many links that might help me to get that going.

For now, I think this is the shortlist of solutions to try:

  • Docker Mailserver
  • Mailcow
  • Mailu

Read the rest of this entry »

Posted in *nix, *nix-tools, Cloud, Communications Development, Containers, Development, Docker, Infrastructure, Internet protocol suite, Kubernetes (k8n), postfix, Power User, SMTP | Tagged: , , , , , , | Leave a Comment »

roehling/postsrsd: Postfix Sender Rewriting Scheme daemon

Posted by jpluimers on 2025/11/12

This finally made me perform the (long overdue) migration from Sendmail to Postfix:

The Sender Rewriting Scheme (SRS) is a technique to forward mails from domains which deploy the Sender Policy Framework (SPF) to prohibit other Mail Transfer Agents (MTAs) from sending mails on their behalf. With SRS, an MTA can circumvent SPF restrictions by replacing the envelope sender with a temporary email address from one of their own domains. This temporary address is bound to the original sender and only valid for a certain amount of time, which prevents abuse by spammers.

[Wayback/Archive] roehling/postsrsd: Postfix Sender Rewriting Scheme daemon

Via a long queste to figure out why Gmail every now and then bounces forwarded messages because of Sender Policy Framework (SPF). Below are a few of the links that brought me here in mostly reverse order, but first some links that should help me further on the topic of Postfix virtual aliases.

The sendmail setup had some features not covered below (like a catch-all forward for email to addresses virtual domains not covered by a virtual alias) which I hopefully can cover later.

One thing learned both over the past decades and related postfix material: use separate servers or containers for each of your functions. So do not mix web-hosting, outgoing mail, incoming mail, fail2ban and others on the same server.

Links:

Read the rest of this entry »

Posted in *nix, *nix-tools, Communications Development, Development, Internet protocol suite, postfix, Power User, sendmail, SMTP | Tagged: | Leave a Comment »

Attempting to stop Microsoft users sending ‘reactions’ to email from me by adding a postfix header

Posted by jpluimers on 2024/09/27

If you do not want Outlook kinds of clients spamming you, then add this header to your email messages above the Content-Type header (see [Wayback/Archive] The Message Content-Type in MIME)

x-ms-reactions: disallow

[Wayback/Archive] Attempting to stop Microsoft users sending ‘reactions’ to email from me by adding a postfix header

Via [Wayback/Archive] Kris: “x-ms-reactions: disallow http…” – chaos.social

x-ms-reactions: disallow

neilzone.co.uk/2024/07/attempt

Eine kleine Mailserver Config verhindert, daß Outlook Volldeppen meine Mailbox mit Likes spammen.

Sehr gut.

--jeroen

Posted in *nix, *nix-tools, Communications Development, Development, Internet protocol suite, Office, Outlook, postfix, Power User, SMTP | Leave a Comment »

Hello “SMTP Smuggling” information released days before the Holiday season to open source SMTP server teams

Posted by jpluimers on 2023/12/24

Jan Wildeboer was mad for good reasons, though the open source projects didn’t yet seem to publicly have show their real madness, just bits like [Wayback/Archive] oss-security – Re: Re: New SMTP smuggling attack:

I'm a little confused by sec-consult's process here. They identify a
problem affecting various pieces of software including some very widely
deployed open source software, go to the trouble of doing a coordinated
disclosure, but only do that with...looking at their timeline... gmx,
microsoft and cisco?

“SMTP Smuggling” is bad, and big open source SMTP server projects like exim, postfix and sendmail needed to assess and fix/prevent the issue on very short notice: effectively confronting them with a zero-day less than a week between the information got released and the Holiday season.

That gives “deploy on Fridays” a totally different dimension.

How bad? Well, it already managed to reach this Newline – Wikipedia entry:

The standard Internet Message Format[26] for email states: “CR and LF MUST only occur together as CRLF; they MUST NOT appear independently in the body”. Differences between SMTP implementations in how they treat bare LF and/or bare CF characters have led to so-called SMTP smuggling attacks[27].

The crux of the problem is very well described by the “Postfix: SMTP Smuggling” link below: recommended reading, and the middle of [Wayback/Archive] SMTP Smuggling – Spoofing Emails Worldwide | Hacker News

…

TLDR: In the SMTP protocol, the end of the payload (email message) is indicated by a line consisting of a single dot. The line endings normally have to be CRLF, but some MTAs also accept just LF before and/or after the dot. This allows SMTP commands that follow an LF-delimited dot line to be “tunneled” through a first MTA (which requires CRLF and thus considers the commands to be part of the email message) to a second MTA (which accepts LF and thus processes the commands as real commands). For the second MTA, the commands appear to come from the first MTA, hence this allows sending any email that the first MTA is authorized to send. That is, emails from arbitrary senders under the domains associated with the first MTA can be spoofed.

…

Here are some links to keep you busy the next hours/days/weeks:

And the toots linking to background information:

Read the rest of this entry »

Posted in *nix, *nix-tools, Communications Development, Development, exim mail, Internet protocol suite, postfix, Power User, Python, Scripting, sendmail, SMTP, Software Development | Leave a Comment »

Email Handling and vBulletin Cloud – vBulletin Community Forum

Posted by jpluimers on 2023/08/25

For my link archive: [Wayback/Archive] Email Handling and vBulletin Cloud – vBulletin Community Forum.

  • Asking your end users to white list your email address and the Sendgrid IP (167.89.58.99) can help alleviate the issues.

I didn’t know the above but bumped into an issue because I didn’t know a supplier had moved to vBulletin Cloud, my account password stopped being accepted and my account password reset messages would not arrive.

So I wrote this as part of a mail to sort this out, and it was confirmed to be correct:

Then I re-checked a few connection refusals that appeared close to the password reset tries. Not sure if this a pattern, but a few of them had this:
2022-02-20T19:41:42.999415+01:00 snap sendmail[24314]: NOQUEUE: connect from o1678958x99.outbound-mail.sendgrid.net [167.89.58.99]
2022-02-20T19:41:43.015958+01:00 snap sendmail[24314]: NOQUEUE: dns 99.58.89.167.bl.spamcop.net. => 127.0.0.2
2022-02-20T19:41:43.016442+01:00 snap sendmail[24314]: ruleset=check_relay, arg1=o1678958x99.outbound-mail.sendgrid.net, arg2=127.0.0.2, relay=o1678958x99.outbound-mail.sendgrid.net [167.89.58.99], reject=553 5.3.0 Spam blocked see: http://spamcop.net/bl.shtml?167.89.58.99
2022-02-20T19:42:29.527789+01:00 snap sendmail[23814]: 21KIeTfo023814: engine10.uptimerobot.com [69.162.124.231] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
From the linked page I got to https://www.spamcop.net/w3m?action=blcheck&ip=167.89.58.99 indicating

167.89.58.99 listed in bl.spamcop.net (127.0.0.2)

If there are no reports of ongoing objectionable email from this system it will be delisted automatically in approximately 13 hours.

Causes of listing
  • System has sent mail to SpamCop spam traps in the past week (spam traps are secret, no reports or evidence are provided by SpamCop)

Express-delisting is not available

Listing History

In the past 44.4 days, it has been listed 12 times for a total of 13.3 days

Can you check if the forum software uses sendgrid?

The confirmation linked to the first post in this blog entry on how to whitelist the SendGrid outgoing IP-address.

One thing I wonder: why does SendGrid use a single outgoing IP-address? If it gets blacklisted, many of their clients have problems.

Anyway: before adding the entry to my whitelist, the problem had resolved itself, and the blacklist entries were done:

Related: [Wayback/Archive] Forum Move – Scooter Forums

We’ve moved our forums to vBulletin Cloud.

New forum URL: https://forum.scootersoftware.com/

Links to the old forum will be redirected to the new URL.

If you notice any problems after the move, please let us know.

[Wayback/Archive] Forums – Scooter Forums

–jeroen

Posted in Communications Development, Development, eMail, GMail, Google, Internet protocol suite, Power User, SMTP, SocialMedia | Leave a Comment »

Cryptosense Discovery

Posted by jpluimers on 2022/03/15

This is cool: [Wayback] Cryptosense Discovery:

Free tool that discovers security configuration errors in SSH and TLS servers and explains how to fix them. Supports STARTTLS and can also scan HTTPS, POP3, IMAP and SMTP servers.

It gives you a list of servers a target domain uses (for purposes like web, email, etc) that can have external encryption enabled, then allows you to test these.

The list by default has only servers within that target domain enabled, but you can optionally include other servers (for instance if a domain uses a third party for their SMTP handling).

Basically it is the web-counterpart of a tool like testssl.sh (which I have written about before).

Found while checking out how to test the MX security of a domain using [Wayback] testssl.sh as I forgot the syntax, which in retrospect is dead easy as per [Wayback] tls – How to use testssl.sh on an SMTP server? – Information Security Stack Exchange (thanks [Wayback] Z.T.!):

…

testssl.sh --mx <domain name>

works fine.

testssl.sh -t smtp <ip>:25

and

testssl.sh -t smtp <ip>:587

also work fine.

Note that not specifying the port assumes port 443, despite specifying protocol smtp. That doesn’t work.

…

Also, you might try discovery.cryptosense.com which does the same thing only better

That website is made by the cool people at [Wayback] Cryptosense.

Both are a lot easier than the alternatives described in [Wayback] Blog · How to test SMTP servers using the command-line · Halon MTA: using nslookup and dig for determining the affected hosts, using nc or telnet for testing basic connectivity, using [Wayback/Archive.is] openssl s_client to test TLS, and [Wayback/Archive.is] smtpping for measuring throughput.

In addition to the above tools mentioned in the blog, I’ve also used sendEmail (note case sensitivity), ehlo-size, and swaks.

This is what I tested:

–jeroen

Posted in *nix, *nix-tools, Awk, bash, bash, Communications Development, Development, DNS, Encryption, grep, HTTPS/TLS security, Internet, Internet protocol suite, Power User, Scripting, Security, SMTP, Software Development, SSH, ssh/sshd, TCP, testssl.sh, TLS | Leave a Comment »

Which SMTP Port Should I Use? Learn Ports 25, 465, & 587 (and unofficial port 2525) | Mailgun

Posted by jpluimers on 2021/11/30

When trying to deliver mail, it is important to know which protocols and ports you can use.

On smtp, smtp-submission, smtps (ports 25, 587 and 465) and unofficial port 2525 (which Maingun maps to `smtp-submission): [Wayback] Which SMTP Port Should I Use? Learn Ports 25, 465, & 587 | Mailgun

Quote on why smtps port 465 is hardly used:

Port 465:

IANA has reassigned a new service to this port, and it should no longer be used for SMTP communications.

However, because it was once recognized by IANA as valid, there may be legacy systems that are only capable of using this connection method. Typically, you will use this port only if your application demands it. A quick Google search, and you’ll find many consumer Inbox Service Providers’ (ISPs) articles that suggest port 465 as the recommended setup. However, we do not recommend it, as it is not RFC compliant.

–jeroen

Posted in Communications Development, Development, Internet protocol suite, SMTP, Software Development, TLS, Web Development | Leave a Comment »

Some scripts and tips for easing the maintenance of a postfix based SMTP system

Posted by jpluimers on 2021/11/24

A few scripts and tips I found Googling around.

Deleting queued messages by regular expression pattern

I have seen the below delete-from-mailq.pl script numerous time, usually without any attribution (for instance [Wayback] Postfix Flush the Mail Queue – nixCraft and  [Wayback] postfix-delete.pl – Following script deletes all mail from the mailq which matches the regular expression specified as the first argument · GitHub).

The earliest version I could find was in [Wayback] ‘Re: delete messages from mailq’ – MARC by [Wayback] ‘Ralf Hildebrandt ‘ posts – MARC:

--- snip ---
#!/usr/bin/perl

$REGEXP = shift || die "no email-adress given (regexp-style, e.g. bl.*\@yahoo.com)!";

@data = qx</usr/sbin/postqueue -p>;
for (@data) {
  if (/^(\w+)\*?\s/) {
     $queue_id = $1;
  }
  if($queue_id) {
    if (/$REGEXP/i) {
      $Q{$queue_id} = 1;
      $queue_id = "";
    }
  }
}
                                
#open(POSTSUPER,"|cat") || die "couldn't open postsuper" ;
open(POSTSUPER,"|postsuper -d -") || die "couldn't open postsuper" ;

foreach (keys %Q) {
  print POSTSUPER "$_\n";
};
close(POSTSUPER);
--- snip ---

And then use:
% delete-from-mailq "^test"

 

Tips

[Wayback] How do I check the postfix queue size? – Server Fault

Lots of great answers and pointers to useful guides/software there.

qstat

[Wayback] Postfix Bottleneck Analysis points to [Wayback] Postfix manual – qshape(1): qshape - Print Postfix queue domain and age distribution, then explains about different scenarion and queues:

postqueue

postqueue -p | tail -n 1

Last line in the postqueue -p shows how many requests and size:

-- 317788 Kbytes in 11860 Requests.

View queues size

I tried finding the original posting of the below script, but could not. If you find it, please let me know.

#!/usr/bin/env perl

# postfix queue/s size
# author: 
# source: http://tech.groups.yahoo.com/group/postfix-users/message/255133

use strict;
use warnings;
use Symbol;
sub count {
        my ($dir) = @_;
        my $dh = gensym();
        my $c = 0;
        opendir($dh, $dir) or die "$0: opendir: $dir: $!\n";
        while (my $f = readdir($dh)) {
                if ($f =~ m{^[A-F0-9]{5,}$}) {
                        ++$c;
                } elsif ($f =~ m{^[A-F0-9]$}) {
                        $c += count("$dir/$f");
                }
        }
        closedir($dh) or die "closedir: $dir: $!\n";
        return $c;
}
my $qdir = `postconf -h queue_directory`;
chomp($qdir);
chdir($qdir) or die "$0: chdir: $qdir: $!\n";
printf "Incoming: %d\n", count("incoming");
printf "Active: %d\n", count("active");
printf "Deferred: %d\n", count("deferred");
printf "Bounced: %d\n", count("bounce");
printf "Hold: %d\n", count("hold");
printf "Corrupt: %d\n", count("corrupt");

Various commands

[Wayback] Inspecting Postfix’s email queue – Tech-G explaining about:

  • mailq
  • postqueue -p
  • postcat -vq XXXXXXXXXX (where XXXXXXXXXX is the message ID)
  • postqueue -f / postfix flush
  • postsuper -d to delete messages

More of these in [Wayback] Postfix Mail Queue Management – Linux Hint and [Wayback] Postfix Bottleneck Analysis: queues.

Makefile

Based on [Wayback] Using “make” for Postfix file maintenance

MAPS = relays.db aliases.db transport.db relocated.db \
        virtual.db sender_checks.db rejected_recips.db \
        helo_access.db

all : $(MAPS)

aliases.db : aliases
        newaliases

%.db : %
        postmap $*

This is my Makefile that runs fine on Tumbleweed (note: all 8-space indents are TAB characters):

MAPS =  /etc/aliases.db \
        transport.db \
        virtual.db \
        helo_access.db \
        canonical.db \
        sasl_passwd.db \
        relocated.db \
        relay.db \
        access.db \
        relay_ccerts.db \
        sender_canonical.db

all : $(MAPS)

aliases.db : aliases
        @echo "Rebuilding $@."
        newaliases

%.db : %
        @echo "Rebuilding $@."
        postmap $*

In the future, I might try [Wayback] Makefile.postfix · GitHub, though I think it is convoluted:


## Postfix: Makefile to update *.db files
POSTCONF= /usr/sbin/postconf
POSTMAP= /usr/sbin/postmap
default: postmap
postmap: Makefile.postmap
@echo 'Updating database files …'
$(MAKE) -f Makefile.postmap
Makefile.postmap: main.cf
@echo 'Updating $@ …'
@set -e; \
rm -f $@.$$$$.tmp; \
echo 'POSTMAP=$(POSTMAP)' >>$@.$$$$.tmp; \
echo 'postmap::' >>$@.$$$$.tmp; \
config_directory="$(PWD)"; \
{ $(POSTCONF) -c $(PWD) || kill $$$$; } \
|tr ' ' '\n' \
|sed -n \
-e 's/,$$//' \
-e 's#^hash:\$$config_directory/##p' \
-e 's#^hash:'"$$config_directory/##p" \
|sort -u \
|while read mapfile; do \
echo "postmap:: $$mapfile.db" >>$@.$$$$.tmp; \
echo "$$mapfile.db: $$mapfile" >>$@.$$$$.tmp; \
echo " \$$(POSTMAP) $$<" >>$@.$$$$.tmp; \
done; \
mv $@.$$$$.tmp $@

view raw

Makefile.postfix

hosted with ❤ by GitHub

 

 

[Wayback] Ralf Hildebrandt

Ralf Hildebrandt is an active and well-known figure in the Postfix community. He’s a systems engineer for T-NetPro, a German telecommunications company and has spoken about Postfix at industry conferences and contributes regularly to a number of open source mailing lists.

Co-author of this book: [Wayback: Book of Postfix State-of-the-Art Message Transport ISBN 9781593270018] (which used to have its own site: [Wayback: The Book of Postfix]

Book of Postfix

State-of-the-Art Message Transport

By Patrick Koetter, Ralf Hildebrandt

Publisher: No Starch PressRelease Date: March 2005Pages: 496

Best practices for Postfix–the popular alternative to Sendmail. Developed with security and speed in mind, Postfix has become a popular alternative to Sendmail and comes preinstalled in many Linux distributions as the default mailer. The Book of Postfix is a complete guide to Postfix whether used at home, as a mailrelay or virus-scanning gateway, or as a company mailserver. Practical examples show how to deal with daily challenges like protecting mail users from spam and viruses, managing multiple domains, and offering roaming access.

This is a great review of the book: [Wayback] The Book of Postfix (Ralf Hildebrandt, Patrick Koetter)

Related

For my postfix studies… « The Wiert Corner – irregular stream of stuff

–jeroen

 

Posted in *nix, *nix-tools, bash, Communications Development, Development, Internet protocol suite, Makefile, postfix, Power User, Scripting, SMTP, Software Development | Leave a Comment »

Some links on SMTP tar-pit to lessen SPAM

Posted by jpluimers on 2021/11/03

Some links for my archive; note that pure tar-pits by now are also hampering large email sender services like SendGrid, Mailgun and Amazon SES.

So the below links are for educational and historic purposes only.

I assembled these links because out of a sudden, Ring 2FA verification emails could not be delivered any more.

Ring 2FA came mandatory towards the end of February 2020.

Some links on that:

Sendmail timeouts:

–jeroen

Read the rest of this entry »

Posted in *nix, Communications Development, Development, HIS Host Integration Services, Internet protocol suite, Power User, SMTP | Leave a Comment »

On my list of things to try: Amazon SES for outbound/inbound email handling

Posted by jpluimers on 2021/08/10

SES mail servers at the time of writing

*n*x:

# nslookup -type=TXT amazonses.com | grep "v=spf1"
amazonses.com   text = "v=spf1 ip4:199.255.192.0/22 ip4:199.127.232.0/22 ip4:54.240.0.0/18 ip4:69.169.224.0/20 ip4:76.223.180.0/23 ip4:76.223.188.0/24 ip4:76.223.189.0/24 ip4:76.223.190.0/24 -all"I

Windows

C:\>nslookup -type=TXT amazonses.com | find "v=spf1"
Non-authoritative answer:
        "v=spf1 ip4:199.255.192.0/22 ip4:199.127.232.0/22 ip4:54.240.0.0/18 ip4:69.169.224.0/20 ip4:76.223.180.0/23 ip4:76.223.188.0/24 ip4:76.223.189.0/24 ip4:76.223.190.0/24 -all"

These addresses use a compact CIDR notation to denote ranges of networks containing ranges of network IPv4 addresses.

CIRD processing to sendmail access file

(this is linux sendmail only)

Converting the nslookup outout to a CIDR based sendmail /etc/mail/access excerpt goes via a pipe sequence of multiple sed commands:

# nslookup -type=TXT amazonses.com | grep "v=spf1" | sed 's/\(^.*"v=spf1 ip4:\| -all"$\)//g' | sed 's/\ ip4:/\n/g' | xargs -I {} sh -c "prips {} | sed 's/$/\tRELAY/g'"
199.255.192.0   RELAY
199.255.192.1   RELAY
...
76.223.190.254  RELAY
76.223.190.255  RELAY

What happens here is this:

  1. Filter out only spf1 records using grep.
  2. Remove the head (.*v=spf1 ip4:) and tail ( -all") of the output, see [WayBack] use of alternation “|” in sed’s regex – Super User.
  3. Replaces all ip4: with newlines (so the output get split over multiple lines), see [WayBack] linux – splitting single line into multiple line in numbering format using awk – Stack Overflow.
  4. Convert the CIDR notation to individual IP addresses (as sendmail cannot handle CIDR),
    1. This uses a combination of xargs with the  sh trick to split the CIDR list into separate arguments, and prips (which prints the IP addresses for a CIDR); see:
    2. Alternatively, use
  5. Replaces all end-of-line anchor ($) with a tab followed by RELAY, see

You can append the output of this command to /etc/mail/access, then re-generate /etc/mail/access.db and restart sendmail; see for instance [WayBack] sendmail access.db by example | LinuxWebLog.com.

Without the xargs, the output would look like this:

# nslookup -type=TXT amazonses.com | grep "v=spf1" | sed 's/\(^.*"v=spf1 ip4:\| -all"$\)//g' | sed 's/\ ip4:/\n/g'
199.255.192.0/22
199.127.232.0/22
54.240.0.0/18
69.169.224.0/20
76.223.180.0/23
76.223.188.0/24
76.223.189.0/24
76.223.190.0/24

Via

–jeroen

Posted in *nix, *nix-tools, Amazon SES, Amazon.com/.de/.fr/.uk/..., Cloud, Communications Development, Development, Infrastructure, Internet protocol suite, Power User, sendmail, SMTP, Software Development | Leave a Comment »

« Previous Entries