Re: [postmaster@chiark.greenend.org.uk: [postmaster@sources.redhat.com] Excessive retries by your mail system]

[Ian Lance Taylor <ian@airs.com>]

> From: Ian Jackson as chiark postmaster <postmaster@chiark.greenend.org.uk>
> Subject: [postmaster@sources.redhat.com] Excessive retries by your mail system
> To: postmaster@sources.redhat.com
> Date: Wed, 13 Aug 2003 15:10:22 +0100
> 
> 
> In the hour 1220-1320 UTC today (which I'm picking as a convenient
> example), sources.redhat.com made 114 connections to my system
> chiark.greenend.org.uk, 102 of which were rejected by my system with a
> 421 banner message (due to your system's excessive use of concurrent
> SMTP sessions and its excessive history of SMTP errors when talking to
> mine).
> 
> That's an average of one failed connection attempt every 35 seconds.
> This is grossly excessive.  It's much faster than the retry rates
> recommended in RFC1123 (Host Requirements).  It is also a much faster
> retry rate than I have configured my system to permit to a single
> calling site.
> 
> You are triggering capacity reservation and rate-limiting mechanisms
> which are intended to cope with denial-of-service attacks and to slow
> down spammers.  As a result the real mail which ought to be flowing
> from your system to mine (various mailing lists hosted on
> sources.redhat.com) is suffering delays.
> 
> Please could you reconfigure your system to retry much less often.
> See RFC1123 s5.3.1.  Your system appears to be in violation (for
> example) of the following paragraph, for example:
> 
>       The sender MUST delay retrying a particular destination
>       after one attempt has failed.  In general, the retry
>       interval SHOULD be at least 30 minutes; however, more
>       sophisticated and variable strategies will be beneficial
>       when the sender-SMTP can determine the reason for non-
>       delivery.
> 
> When you've made your system stop hammering mine so much, the mail
> should start flowing normally within an hour or two.

sources.redhat.com runs qmail.  qmail does implement the restriction
above.  However, since sources.redhat.com sends so many messages to so
many failing hosts, and since qmail records a fixed number of hosts
which have failed and updates the list in a circular fashion, it is
possible that the list of hosts is being overwritten such that qmail
is trying your system when it normally would not.

That said, the fact is that sources.redhat.com generates a great deal
of mail, and if people on your system are signed up to several of the
high volume mailing lists, those mailing lists will consistently
trigger rate-limiting mechanisms.  We've seen this before on other
systems.  The effect is that the e-mail traffic from
sources.redhat.com will never flow smoothly in the presence of rate
limitations, and, indeed, sources.redhat.com will start bouncing
e-mail messages sent to your users, and will eventually simply remove
them from the mailing lists.

I'm afraid there is no particularly good way around this.  In general,
high volume mailing lists and rate limitations are not compatible.
You should either make an exception to your rate limiting code for
sources.redhat.com, or you should prohibit your users from signing up
for sources.redhat.com mailing lists (which include gcc.gnu.org
mailing lists).

Ian