The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,862 other subscribers

Archive for the ‘*nix’ Category

gpg creation and sign Gino’s Key ($1785651) · Snippets · GitLab

Posted by jpluimers on 2020/12/04

Boy it was a long time ago that I did anything with gpg. Here is how to generate and sign keys.

[WayBack] gpg creation and sign Gino’s Key ($1785651) · Snippets · GitLab

And here to check your email confguration:

[WayBack] Home – dmarcian Founded in 2012 by the primary author of the DMARC specification, dmarcian is dedicated to upgrading the entire world’s email by making DMARC accessible to all. dmarcian brings together thousands of senders, vendors, and operators in a common effort to build DMARC into the email ecosystem.

–jeroen

Posted in *nix, Power User, Security | Leave a Comment »

Forcing the queue on individual email message

Posted by jpluimers on 2020/11/23

I tried the trick from [WayBack] Forcing the queue on individual email message. Despite what the linked post says, on my system, the email is not sent, but bounces, and the queue files get lost, so there is no way to get to the original content without a proper back-up mechanism (which – for transient data like queued email messages – often is not implemented).

The reason is the -oTnow parameter. It is kind of documented, but none of that is returned via the “sendmail” “-oTnow” – Google Search.

Before explaining the cause and workaround deeper, these are the relevant documentation parts:

I could not find the 8.15 operations guide as PDF, but the version 8.12 HTML edition is mostly the same as the PDF (Despite the time span between those releases spanning 14 years, parameters and configuration options didn’t change a whole lot), so as the HTML is easier to quote, here we go:

The arguments; -q, -o versus -O options

[WayBack] www.sendmail.org/~ca/email/doc8.12/op-sh-7.html “ACKNOWLEDGEMENTS” explains the basic arguments as well:

Arguments must be presented with flags before addresses. The flags are:

-oxvalue
Set option x to the specified value. These options are described in Section 5.6.
-Ooption=value
Set option to the specified value (for long form option names). These options are described in Section 5.6.

-q[!]Xstring
Run the queue once, limiting the jobs to those matching Xstring. The key letter X can be I to limit based on queue identifier, R to limit based on recipient, or S to limit based on sender. A particular queued job is accepted if one of the corresponding addresses contains the indicated string. The optional ! character negates the condition tested. Multiple -qX flags are permitted, with items with the same key letter or'ed together, and items with different key letters and'ed together.

[WayBackwww.sendmail.org/~ca/email/doc8.12/op-sh-5.html “THE WHOLE SCOOP ON THE CONFIGURATION FILE” goes into more detail:

O — Set Option

There are a number of global options that can be set from a configuration file. Options are represented by full words; some are also representable as single characters for back compatibility. The syntax of this line is:

O option = value

This sets option option to be value. Note that there must be a space between the letter `O’ and the name of the option. An older version is:

O ovalue

where the option o is a single character. Depending on the option, value may be a string, an integer, a boolean (with legal values tTf, or F; the default is TRUE), or a time interval.

then lists the relevant options for us:

QueueTimeout=timeout
[T] A synonym for Timeout.queuereturn. Use that form instead of the QueueTimeout form.

Timeout.type=timeout
[r; subsumes old T option as well] Set timeout values. For more information, see section

The -oT parameter, or T option sets timeouts

[WayBack] www.sendmail.org/~ca/email/doc8.12/op-sh-4.html “TUNING” has a whole section on

Timeouts

All time intervals are set using a scaled syntax. For example, 10m represents ten minutes, whereas 2h30m represents two and a half hours. The full set of scales is:

s seconds
m minutes
h hours
d days
w weeks

There is also a special timeout value now, and timeouts can be defined in various levels, including message timeouts:

Message timeouts

After sitting in the queue for a few days, an undeliverable message will time out. This is to insure that at least the sender is aware of the inability to send a message. The timeout is typically set to five days. It is sometimes considered convenient to also send a warning message if the message is in the queue longer than a few hours (assuming you normally have good connectivity; if your messages normally took several hours to send you wouldn’t want to do this because it wouldn’t be an unusual event). These timeouts are set using the Timeout.queuereturn and Timeout.queuewarn options in the configuration file (previously both were set using the T option).

If the message is submitted using the NOTIFY SMTP extension, warning messages will only be sent if NOTIFY=DELAY is specified. The queuereturn and queuewarn timeouts can be further qualified with a tag based on the Precedence: field in the message; they must be one of urgent (indicating a positive non-zero precedence) normal (indicating a zero precedence), or non-urgent (indicating negative precedences). For example, setting Timeout.queuewarn.urgent=1h sets the warning timeout for urgent messages only to one hour. The default if no precedence is indicated is to set the timeout for all precedences. The value “now” can be used for -O Timeout.queuereturn to return entries immediately during a queue run, e.g., to bounce messages independent of their time in the queue.

Since these options are global, and since you cannot know a priori how long another host outside your domain will be down, a five day timeout is recommended. This allows a recipient to fix the problem even if it occurs at the beginning of a long weekend. RFC 1123 section 5.3.1.1 says that this parameter should be “at least 4-5 days”.

The Timeout.queuewarn value can be piggybacked on the T option by indicating a time after which a warning message should be sent; the two timeouts are separated by a slash. For example, the line

OT5d/4h

causes email to fail after five days, but a warning message will be sent after four hours. This should be large enough that the message will have been tried several times.

So the -oT is the T option, which is shorthand for -oTimeout.queuereturn. (well, actually you can combine Timeout.queuewarn using a slash).

Specifying now gives you this behaviour: The value “now” can be used for -O Timeout.queuereturn to return entries immediately during a queue run, e.g., to bounce messages independent of their time in the queue.

Of course that bounces, invalidating the below -oTnow trick.

The -qI parameter, or I queue flag for message identifier

You can also limit the jobs to those with a particular queue identifier, recipient, sender, or queue group using one of the queue modifiers. For example, -qRberkeley restricts the queue run to jobs that have the string berkeley somewhere in one of the recipient addresses. Similarly, -qSstring limits the run to particular senders, -qIstring limits it to particular queue identifiers, and -qGstring limits it to a particular queue group. You may also place an ! before the I or R or S to indicate that jobs are limited to not including a particular queue identifier, recipient or sender. For example, -q!Rseattle limits the queue run to jobs that do not have the stringseattle somewhere in one of the recipient addresses. Should you need to terminate the queue jobs currently active then a SIGTERM to the parent of the process (or processes) will cleanly stop the jobs.

So the -qI flag followed by an identifier only processes that message.

Sending to a host that got marked as timeout: the Timeout.hoststatus option

When sending just one message, you definitely want to disregard any cached host timeout status: you want to retry the message now, not skip it just because the host had a timeout a while ago.

This is where the -OTimeout.hoststatus=0m comes in: it sets the cache to zero minutes.

Below are the relevant bits of documentation.

[WayBack] cf/m4/README.txt:

confTO_HOSTSTATUS  Timeout.hoststatus
                    [30m] How long information about host
                    statuses will be maintained before it
                    is considered stale and the host should
                    be retried.  This applies both within
                    a single queue run and to persistent
                    information (see below).
...
confHOST_STATUS_DIRECTORY HostStatusDirectory
                    [undefined] If set, host status is kept
                    on disk between sendmail runs in the
                    named directory tree.  This need not be
                    a full pathname, in which case it is
                    interpreted relative to the queue
                    directory.

[WayBack] chapter 4 “TUNING”

Read timeouts

Timeouts all have option names Timeout.suboption. Most of these control SMTP operations. The recognized suboptions, their default values, and the minimum values allowed by RFC 2821 section 4.5.3.2 (or RFC 1123 section 5.3.2) are:

hoststatus
How long status information about a host (e.g., host down) will be cached before it is considered stale [30m, unspecified].

A trick that really works

With the above information,

sendmail -v -OTimeout.hoststatus=0m -qImessage_identifier

So it is now a bash function for me: sendmail-retry-one-messae-by-queue-ID:

function sendmail-retry-one-messae-by-queue-ID() {
  sendmail -v -OTimeout.hoststatus=0m -qI$1                     
}

Back to the original “trick” (that makes you loose the message)

I made it into a function:

function sendmail-bounce-one-message-by-queue-ID() {
  sendmail -oTnow -qI$1
}

From [WayBack] Forcing the queue on individual email message:

comp.mail.sendmail

Paul Aviles wrote:

> I know about sendmail -q, but is is possible to individually kill a 
> pending queued email or force it with or without an NDA?

This is simple. Use the mailq command to find the local queue ID of the message. It will look something like:

      k5PBASdd010482

You can then flush the message with:

      sendmail -qIyourqueueID -oTnow

The “I” flag to the -q flag specifies which queue ID to use. The -oTnow flag sets the queue timeout to “now”. Note sendmail will still try to deliver the message once when you do this. If this is not OK, then blow the message away with:

      rm /var/spool/mqueue/*yourqueueID

But the timeout is much recommended over blowing the message away.

Just another “Harker’s Helpful Hints”

RLH

–jeroen

Posted in *nix, *nix-tools, Power User, sendmail | Leave a Comment »

OpenCandy – Wikipedia

Posted by jpluimers on 2020/11/06

Hmm, one of my machines contained OpenCandy – Wikipedia as found by Malwarebytes (software) – Wikipedia:

Tracking back the installation, revealed it came with ImgBurn 2.5.8.0, which is now on my black-list.

In my case this was how to remove it:

rd /s /q %AppData%\OpenCandy

This is not universal; you might need to take additional measures like in [WayBack] How to Remove PUP.Optional.OpenCandy (Removal Guide).

I use this batch-file to get the most recent Malwarebytes and Chameleon:

get-malware-bytes.bat 

:: redirects to something like wget https://data-cdn.mbamupdates.com/web/mb3-setup-consumer/mb3-setup-consumer-3.6.1.2711-1.0.508-1.0.8211.exe
wget --content-disposition https://downloads.malwarebytes.com/file/mb3/
:: redirects to something like wget https://data-cdn.mbamupdates.com/web/mbam-chameleon-3.1.33.0.zip
wget --content-disposition https://downloads.malwarebytes.com/file/chameleon/

If you do not have wget on your system, then try this PowerShell alternative (which does not show progress) via [WayBack] Windows batch file file download from a URL – Stack Overflow

:: in case you do not have wget:
powershell -Command "(New-Object Net.WebClient).DownloadFile('https://downloads.malwarebytes.com/file/mb3/', 'mb3.exe')"
powershell -Command "(New-Object Net.WebClient).DownloadFile('https://downloads.malwarebytes.com/file/chameleon/', 'chameleon.exe')"
:: note these do not show progress!
:: https://stackoverflow.com/questions/4619088/windows-batch-file-file-download-from-a-url

Related: [WayBackJeroen Pluimers on Twitter: “What if the most recent @Malwarebytes on a Windows 8.1 x64 VM (all patches installed) on ESXi backed by NVME hangs for hours on one file with hardly any CPU usage? Screenshots of mbam.exe, mbamservice.exe and mbamtray.exe thread usage below.

–jeroen

Posted in *nix, *nix-tools, LifeHacker, Power User, Security, wget, Windows | Leave a Comment »

How To Use Journalctl to View and Manipulate Systemd Logs | DigitalOcean

Posted by jpluimers on 2020/11/05

For my link archive because of the tips: [WayBack] How To Use Journalctl to View and Manipulate Systemd Logs | DigitalOcean

–jeroen

Posted in *nix, *nix-tools, journalctl and journald, Linux, Power User | Leave a Comment »

Linux Find Out What Process Are Using Swap Space – nixCraft

Posted by jpluimers on 2020/10/26

Some interesting tips at [WayBack] Linux Find Out What Process Are Using Swap Space – nixCraft

  • pidof
  • pgrep
  • grep –color VmSwap /proc/####/status
  • awk the same information
  • doing the same in loops
  • smem
  • top

Via: [WayBack] Want to find out what process are using swap space? Useful to debug and optimize your app. … – nixCraft – Google+

–jeroen

Read the rest of this entry »

Posted in *nix, Linux, Power User | Leave a Comment »

Some postfix notes

Posted by jpluimers on 2020/10/15

Postfix has documentation on primary MX and secondary MX, but not on tertiary MX.

If the primary MX is down, you have a series of secondary MX and tertiary MX that configured the same way, MX DNS priority for primary, the series of secondary MX and tertiary MX have increasing numbers, and the primary MX goes down, then senders can get “too many hops” as secondary and tertiary MX are looping.

I had a hard time finding a good and easy solution as these queries do not return many meaningful results:

Here are some links that helped getting this solved:

  • [WayBack] Postfix Frequently Asked Questions: What does “Error: too many hops” mean?

    Short answer: this message means that mail is probably looping. If you see this after you turned on Postfix content filtering, then you have made a mistake that causes mail to be filtered repeatedly. This is cured by appropriate use of content_filter=header_checks=, and body_checks=.

    Long answer: the message has too many Received: message headers. A received header is added whenever Postfix (or any MTA) receives a message. A large number of Received: message headers is an indication that mail is looping around.

    Side comment: email uses the opposite of the technique that is used to avoid IP forwarding loops. With IP, the sender sets a TTL (time to live) field in the IP header. The field is decremented by each router. When the TTL reaches zero the packet is discarded and an ICMP error message is returned to the sender.

  • [WayBack] Error: too many hops (in reply to end of DATA command) · Issue #713 · mail-in-a-box/mailinabox · GitHub

    In case you or anyone else was/is wondering about the mydestination = localhost thing, the reason it has to be set to just localhost is because MIAB uses Postfix’s “virtual domain hosting” (http://www.postfix.org/VIRTUAL_README.html) support. Per the documentation for mydestination at http://www.postfix.org/postconf.5.html#mydestination:

    Do not specify the names of virtual domains – those domains are specified elsewhere. See VIRTUAL_README for more information.

    (in the context of MIAB every domain is a virtual domain).

In my case a series of these:

Received: from mwgp.xs4all.nl (mwgp.xs4all.nl [80.101.239.92])
    by fiber24315337242.heldenvannu.net (Postfix) with ESMTP id 26395200FE
    for <jeroen@pluimers.com>; Fri, 29 Jun 2018 11:01:02 +0200 (CEST)
Received: from fiber24315337242.heldenvannu.net (unknown [37.153.243.246])
    by mwgp.xs4all.nl (Postfix) with ESMTP id 077A5E937
    for <jeroen@pluimers.com>; Fri, 29 Jun 2018 11:01:02 +0200 (CEST)

Specifying the transport will likely help me solve this problem:

This all came down to editing /etc/postfix/transport adding lines for each relayed domain like this one:

example.org    smtp:[mx-a-record.example.org]

Lines like it direct to use the smtp transport and use a specific host (normally, the relay transport is being used).

After this:

# postmap /etc/postfix/transport
# rcpostfix reload

I choose not to configure [WayBack] Postfix Configuration Parameters: relay_recipient_maps, but might if I had an automated way of replicating lists of valid (and invalid) users.

Another option was confirmed at [WayBack] Software-update: Postfix 3.4.0 / 3.3.3 / 3.2.8 / 3.1.11 / 3.0.15 – Computer – Downloads – Tweakers by [WayBack] menocchio. Thanks!

Dat is volgens mij eenvoudig op te lossen met relay_transport of transport_maps. Zie ook: Postfix transport table format.

Daarmee dwing je de secondary servers de mail altijd af te willen leveren bij de primary server (en dus niet bij een andere secondary). En als de primary niet online is, dan wacht ie netjes tot dat wel het geval is :-)

Bijvoorbeeld:
relay_transport = smtp:[primarymx.domain.tld]

Likely relevant: [WayBack] The Book of Postfix

Maybe relevant in the future:

Found on my hunt for the above:

Try not to make typo’s: [WayBack] postfix appears not finding MX records or host names from DNS

Interesting thought, but not sure how smart SPAM bots are now: [Archive.is] Spam relaying through secondary MX… – Google Groups

To archive this:

  1. Rename from
  2. To
  3. Then save in Archive.is

–jeroen

Posted in *nix, Communications Development, Development, DevOps, DNS, etckeeper, Infrastructure, Internet, Internet protocol suite, Linux, Power User, SMTP | Leave a Comment »

Logging request body in HAProxy · GitHub

Posted by jpluimers on 2020/10/05

I will likely need this one day: [WayBack] Logging request body in HAProxy · GitHub.

Related:

–jeroen

Read the rest of this entry »

Posted in *nix, Development, HAProxy, Power User, Software Development | Leave a Comment »

Rebooting a Linux server unattended – twm’s blog

Posted by jpluimers on 2020/09/21

[WayBack] Rebooting a Linux server unattended – twm’s blog:

/sbin/shutdown -r now

Simple, but I keep forgetting where Linux has short/long command options and short/long verbs.

–jeroen

Posted in *nix, *nix-tools, Debian, Linux, OpenShift, openSuSE, Power User, Raspbian, RedHat, SuSE Linux, Tumbleweed, Ubuntu | Leave a Comment »

On my list of things to figure out: how to force fsck with opensuse Tumbleweed on Raspberry Pi 3

Posted by jpluimers on 2020/09/11

Hopefully the below links will eventually get me further in figuring out how to force fsck with opensuse Tumbleweed on Raspberry Pi 3.

For now, I just reinstalled an SD card (my Mac didn’t support the file systems and I did not have time and equipment with me to mount it to another opensuse based device).

–jeroen

Posted in *nix, Linux, openSuSE, Power User, SuSE Linux, Tumbleweed | Leave a Comment »

I need to dig into IPP / driverless printer confuguration

Posted by jpluimers on 2020/09/07

It looks like I need to learn about IPP and driverless based on [WayBack] TIL that your Linux desktop can probably use your somewhat recently made printer, efficiently, with all major features exposed, without needing to download a ton of vendor shitware, without needing to find a PPD file in the depths of hell, without needing to pay extra for explicit PostScript 3 support, and without needing to accept that it will do 0.2 instead of 20 pages per minute because the in-printer PostScript rasterizer runs on a Z80…. – Maik Zumstrull – Google+.

So here are some links:

Via: [WayBack] TIL that your Linux desktop can probably use your somewhat recently made prin… – Kristian Köhntopp – G+

When adding my printer in the Chrome tool, it can properly detect it:

Printer information
Printer make/model: OKI-MC342-36855D
Printer state: idle
Accepting jobs: true
IPP server version: 1.1
Supports PDF natively: true
Supports PWG raster: false
Supports Postscript: true
Supports Unirast: true
Supports application/octet-stream: true
CUPS server: No
Compatability report: PASS Printer should be compatible – try printing

–jeroen

Posted in *nix, *nix-tools, Hardware, OKI C332, OKI Printers, Power User, Printers | 2 Comments »