Cleaning up bounces from /var/spool/mqueue using qtool
Posted by jpluimers on 2018/11/26
Part of my /var/spool/mqueue consist of administrative bounces to mail domains that fail for a long time.
First a few queries to filter the messages I want to move (the -h suppresses filename so you can aggregate with sort and uniq):
grep -h "MDeferred: Connection" /tmp/mqueue-junk/qf* | sort | uniq -c
It gives results like this:
...
56 MDeferred: Connection refused by static.vnpt.vn.
...
1 MDeferred: Connection reset by cleanfreshliving.com.
...
10 MDeferred: Connection timed out with netflix.ssl.com.
...
After blacklisting those domains, I’ve used qtool.pl to cleanup the mail queue.
qtool.pl
As qtool.pl does not have “dry run” or log options, it’s best to test expressions on a copy of your mail queue first. I’ve made copies in /tmp/mqueue for this.
The query expression language on qtool.pl is complicated to get right: the documentation talks about using %msg which in fact is $msg and there is no official documentation on the mapping of qf files in the mqueue directory to expressions used in qtool.pl.
Luckily that mapping is in qtool.pl itself as explained by www.the-art-of-web.com/system/sendmail-qtool/#section_2. A recent source is at github.com/freebsd/freebsd/blob/master/contrib/sendmail/contrib/qtool.pl where I copied the fragment further below from.
Now just see these commands:
./contrib/qtool.pl -C /etc/sendmail.cf -e '$msg{message}[0] =~ /Deferred: Connection refused by/' /tmp/mqueue-junk/ /tmp/mqueue/
and
./contrib/qtool.pl -C /etc/sendmail.cf -e '$msg{num_delivery_attempts} > 100' /tmp/mqueue-junk/ /tmp/mqueue/
Since there are two M lines per qf file, you have to index the {message} part. There is no need for that with the {num_delivery_attempts}.
Because of the =~ operator, the match expressions are of [WayBack] perlre – perldoc.perl.org: Perl regular expressions.
If you run this on the live /var/spool/mqueue directory, then you can get errors like this which means you should retry after a few minutes (or run with sendmail disabled):
Could not obtain fcntl lock on '/var/spool/mqueue//qfv4H9jv7M007291': Resource temporarily unavailable.
1
Could not obtain fcntl lock on '/var/spool/mqueue//qfv5DB2NkJ024360': Resource temporarily unavailable.
1
Note that the searching for Mhost map: lookup \(.*\): deferredfails, so I write this little script that shows which commands are going to be executed and how to execute them:
grep -l "^Mhost map: lookup \(.*\): deferred$" /var/spool/mqueue/qf* | xargs -n1 -I {} echo "./contrib/qtool.pl -C /etc/sendmail.cf /var/spool/mqueue-junk/ {}" grep -l "^Mhost map: lookup \(.*\): deferred$" /var/spool/mqueue/qf* | xargs -n1 -I {} ./contrib/qtool.pl -C /etc/sendmail.cf /var/spool/mqueue-junk/ {}
It executes the qtool.pl once per grep output line.
References I’ve used to get to the above:
- [WayBack] Ubuntu Manpage: qtool – manipulate sendmail queues
- [WayBack] Analysing mailq and the mqueue directory < System | The Art of Web
- [WayBack] Using qtool.pl to manage sendmail queues < System | The Art of Web
- [WayBack] Viewing the mail queue
- [WayBack] Sendmail: Clear / Delete / Flush Mail Queue – nixCraft
- [WayBack] linux – Remove messages from sendmail queue matching subject – Server Fault
- [WayBack] centos – Native sendmail commands to operate the mail queue? – Unix & Linux Stack Exchange
Mapping between qf and qtool.pl
my %parse_table =
(
'A' => 'auth',
'B' => 'body_type',
'C' => 'controlling_user',
'D' => 'data_file_name',
'd' => 'data_file_directory',
'E' => 'error_recipient',
'F' => 'flags',
'H' => 'parse_header',
'I' => 'inode_number',
'K' => 'next_delivery_time',
'L' => 'content-length',
'M' => 'message',
'N' => 'num_delivery_attempts',
'P' => 'priority',
'Q' => 'original_recipient',
'R' => 'recipient',
'q' => 'quarantine_reason',
'r' => 'final_recipient',
'S' => 'sender',
'T' => 'creation_time',
'V' => 'version',
'Y' => 'current_delay',
'Z' => 'envid',
'!' => 'deliver_by',
'$' => 'macro'
);
–jeroen
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/sh | |
| # This script will prompt for sender domain and then purge the mail queue | |
| # Requires qtool.pl | |
| echo "Type the sender domain you would like to purge, followed by [ENTER]:" | |
| read domain | |
| echo "Got it. I'll purge all messages from $domain" | |
| QIDS="$(mailq | grep -B1 $domain | grep '^[a-z]' | awk '{print $1}' | sed 's/\*$//')" | |
| for q in $QIDS | |
| do | |
| /usr/local/bin/qtool.pl -C /etc/mail/sendmail.cf -d /var/spool/mqueue/$q | |
| done | |
The Wiert Corner - irregular stream of stuff 
Leave a Reply