I encountered this issue one Saturday afternoon, on which I happened to be working. Being a Saturday, I expected a relatively quiet day at the office. This was why, I was initially surprised when our monitoring software reported that there were more than 3500 mails present in one of our servers (say, vm.domain.com) mail queue. Within half an hour, the count had increased to approximately 5000 mails. There goes my quiet Saturday. Sigh.

Before proceeding, I would like to point out that all our servers run on CentOS and use Postfix.

Postfix/SASL troubleshooting

One possible reason for the huge number of mails in the queue could have been a receiving mail server that was down. This would have caused mails to be sent repeatedly and resulted in the mail queue being clogged. We have scripts that check for this sort of thing and delete the mails that are addressed to the problematic mail server. But on this occasion, the scripts did not return any hits.

So now that the usual solution had failed, the next step was to check the mail queue status. This can be done for postfix by using the mailq command.

062A7226A8 734 Sat Mar 1 01:27:59 root@vm.domain.com (delivery
temporarily suspended: SASL authentication failed; cannot 
authenticate to server smtp.mailserver.com[xx.xx.xx.xx$ 
info@efg.com-- 4813 Kbytes in 4797 Requests.

As seen in the above example, all the mails in the queue had failed with the error message SASL authentication failed. Following this, I had a peek inside the server’s mail log file (maillog literally):

Mar 1 01:00:10 vm09 postfix/smtp[4897]: warning: 
SASL authentication failure: No worthy mechs found 
Mar 1 01:00:10 vm09 postfix/smtp[4897]: 3B11A2033E: 
SASL authentication failed; cannot authenticate to server 
smtp.mailserver.com

Lo and behold, maillog also thought that something was wrong with the way SASL was set up in the server. Having made my initial diagnosis, I started verifying postfix configurations that were required for the proper functioning of SASL.

Note: postconf is the Postfix configuration utility. Syntax is postconf
<configuration_parameter> When used with the option “-d”, returns the default values of the specified parameter. The option “-n”, returns the value assigned in main.cf (postfix configuration file). This value has more precedence than the default value. The option “-e”, allows you to modify the value of the parameters from the command line. This value will be reset once postfix is restarted.

The following configurations were checked:

smtp_sasl_auth_enable = yes
smtp_sasl_mechanism_filter =
smtp_sasl_password_maps = static:user@domain.com: 
smtp_sasl_security_options = noanonymous

None of these settings seemed to be the cause of the issue. I had hit my second dead end.

I decided to look at the issue from a different perspective. For postfix, SASL may be implemented using either Cyrus or Dovecot plug-ins.

> postconf -A
cyrus
...
> postconf -d smtp_sasl_type
smtp_sasl_type = cyrus

From this, I could conclude that Cyrus was selected. Next, I set out to check whether the configurations of Cyrus were apt.

Note: The file smptd.conf specifies the method of authentication for Postfix/SASL

> cat /etc/sasl2/smtpd.conf
pwcheck_method: saslauthd
mech_list: plain login

saslauthd can use only either PLAIN or LOGIN methods. These settings are default and are fine. Screeeech! That ladies and gentlemen was dead end number three.

By now you may be wondering (like I was at the time), whether I would ever get to the bottom of the issue. Well, I did and here’s how.

Going back to the Cyrus configuration file smtpd.conf, it stated that the mechanisms to be used for SASL authentication was either PLAIN or LOGIN. But for these authentication schemes to work, an additional package cyrus-sasl-plain has to be installed.

I checked the list of installed packages and as suspected, the package was missing.

> rpm -qa | grep cyrus
cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64
cyrus-sasl-2.1.23-13.el6_3.1.x86_64

I installed the package using yum and the mail queue cleared itself within a minute.

> yum install cyrus-sasl-plain
...
> rpm -qi cyrus-sasl-plain-2.1.23-13.el6_3.1.x86_64
.

Summary : PLAIN and LOGIN authentication support for Cyrus SASL
Description :The cyrus-sasl-plain package contains the Cyrus SASL plugins which supportPLAIN and LOGIN authentication schemes.

Well what can I say? It’s the last thing that I would have thought off. Next time, it will be the second last.

About the author:

Paul Junior Fernandez is a Systems Engineer at Bobcares.com. He’s a technology buff and likes to spend his free time catching up with friends on and off social media.