Re: [Fwd: Apologies for multiple messages (Please Help!)]

Re: [Fwd: Apologies for multiple messages (Please Help!)]

[Herb Maeder]

On 24 Oct 2008 18:28:24 BST, Dave Korn wrote:
> > (I've got a hunch that the root cause may be some miscommunication between
> > the servers due to timeouts or something, but I have no way to verify
> > that).
> 
>   You don't have access to the maeder.org server?

I have access to maeder.org, which is a very limited (and shared) vps at 
verio, but only as a user.  

I do not have access to any system log files or the mail queue.  And
surprisingly enough, the first level tech support guys also had limited
access, but I think I'm talking to the right level of people now.

> > How to stop the still repeating evil message?  I still haven't a clue...
> 
>   Those headers you showed us are completely conclusive:
> 
> > >From message 1:
> >   Received: from maeder.org (HELO maeder.org) (192.220.73.146)     by
> sourceware.org (qpsmtpd/0.31) with ESMTP; Thu, 23 Oct 2008 20:43:33 +0000
> >   Received: (qmail 86029 invoked by uid 18834); 23 Oct 2008 20:43:31 -0000
> > 
> > >From message 2:
> >   Received: from maeder.org (HELO maeder.org) (192.220.73.146)     by
> sourceware.org (qpsmtpd/0.31) with ESMTP; Thu, 23 Oct 2008 20:50:13 +0000
> >   Received: (qmail 86029 invoked by uid 18834); 23 Oct 2008 20:43:31 -0000
> > 
> > >From message 3:
> >   Received: from maeder.org (HELO maeder.org) (192.220.73.146)     by
> sourceware.org (qpsmtpd/0.31) with ESMTP; Thu, 23 Oct 2008 21:10:13 +0000
> >   Received: (qmail 86029 invoked by uid 18834); 23 Oct 2008 20:43:31 -0000
> 
> 
>   What this shows is that maeder.org (192.220.73.146) is repeatedly contacting
> sourceware.org to deliver the mail.  You sent the mail once only - the
> repeated (qmail invoked by uid) lines all have the same stamp; your local MTA
> accepted and spooled it, and is trying to send it to sourceware.org.  Because
> it has no msgid, sourceware.org thinks it's a new message every time.  The
> problem is caused by the fact that sourceware.org accepts the post, but
> maeder.org thinks it hasn't, and retries a while later (looks like a slowly
> increasing backoff is involved).
> 
>   You need to look at the mail logs on maeder.org and see what it thinks has
> happened during the conversation at sourceware.org.  It may be that
> sourceware.org is bogusly both accepting the mail and yet indicating an error
> code to maeder.org, or it may be that maeder.org is misinterpreting the status
> code from sourceware.org and only thinking it means "not accepted" when it in
> fact means something like "accepted with warnings".
> 
>   I don't have access to the sourceware logs, but you may be able to read
> /var/log/maillog on maeder.org even if you only have standard user-level
> access, it's quite common not to need root access just to read the logs.

Thanks for the insight and confirmation of my suspicions as to what was
going on (and for giving me some good ammo for dealing with my isp).

They were able to look at my mail queue on maeder.org and make sure it was
all clear, I have no outgoing messages in the queue.  They also cannot
find any indication that the was sent multiple times.  But unfortunately,
another message got sent to the list while they I was on the phone with
them, so clearly the problem is not fixed.  

I am now back to believing that I cannot stop the message from being sent
from my end (that's not so say that the problem is not really at
maeder.org, but I am stuck again).  I really need to get in touch with
someone that can take a look at the equivalent logs and queues on the
sourceware.org side to see what is going on there.

I'm happy to have "my people" talk to "their people" in order to sort it
out if needed.

I sent mail to postmaster AT sourceware.org, as Larry suggested, but have
not heard from anyone yet.

Herb.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: [Fwd: Apologies for multiple messages (Please Help!)]

[Dave Korn]

Herb Maeder wrote on 24 October 2008 18:05:

> I have a theory as to what's going on...
> 
> I believe that adding the Message-ID field is the responsibility of the
> MUA.  

  Yep.

> But some mail servers add a Message-ID if it is missing.

  Yep.  In my case, it gets added at my local server when I receive the post
from sourceware.org.

> I don't think that the missing Message-ID is the root cause of the mail
> duplication problem.  But I do believe that the missing Message-ID causes
> the workaround to the problem to fail.

  Yep and yep.

>  I'm guessing that the
> sourceware.org server may detect and prevent duplicated messages based on
> the Message-ID field.

  Yep, this is a standard dup-filtering strategy for MTAs.

>  But since there wasn't one in my message, that
> mechanism did not work.

  Indeed it could not.

> (I've got a hunch that the root cause may be some miscommunication between
> the servers due to timeouts or something, but I have no way to verify
> that).

  You don't have access to the maeder.org server?

> In any case, if my guess about the misssing Message-ID is correct, then
> I believe that a few of things should be done:
> 
>   1) I should make sure that my MUA generates the Message-ID field on
>      outgoing messages.  I believe that I have fixed this now (and this
>      message should have a "@maeder.org" Message-ID).

  Confirmed.

>   2) If the cygwin mail server is truly depending on Message-ID to stop
>      this message duplication, it should somehow ensure that it is always
>      there.  One option would be to prevent posts to the list that are
>      missing the Message-ID field (with some reasonable return error
>      message to the sender).  NOTE: I'm not sure if generating a
>      Message-ID field on the cygwin mail server would work (that might
>      just add a unique Message-ID on each duplicated message, I'm not
> sure). 

  Well, it might be possible to make a msgid based on a hash of the body of
the email.  But that would have the consequence of making it easier to impose
a computational-load DoS on the machine by sending it lots of spam with no
msgid headers.  Probably an outright rejection would be best.

>   3) Determine and fix the root of the duplication problem (though I'm not
>      sure how feasible this would really be).

> How to stop the still repeating evil message?  I still haven't a clue...

  Those headers you showed us are completely conclusive:

> >From message 1:
>   Received: from maeder.org (HELO maeder.org) (192.220.73.146)     by
sourceware.org (qpsmtpd/0.31) with ESMTP; Thu, 23 Oct 2008 20:43:33 +0000
>   Received: (qmail 86029 invoked by uid 18834); 23 Oct 2008 20:43:31 -0000
> 
> >From message 2:
>   Received: from maeder.org (HELO maeder.org) (192.220.73.146)     by
sourceware.org (qpsmtpd/0.31) with ESMTP; Thu, 23 Oct 2008 20:50:13 +0000
>   Received: (qmail 86029 invoked by uid 18834); 23 Oct 2008 20:43:31 -0000
> 
> >From message 3:
>   Received: from maeder.org (HELO maeder.org) (192.220.73.146)     by
sourceware.org (qpsmtpd/0.31) with ESMTP; Thu, 23 Oct 2008 21:10:13 +0000
>   Received: (qmail 86029 invoked by uid 18834); 23 Oct 2008 20:43:31 -0000


  What this shows is that maeder.org (192.220.73.146) is repeatedly contacting
sourceware.org to deliver the mail.  You sent the mail once only - the
repeated (qmail invoked by uid) lines all have the same stamp; your local MTA
accepted and spooled it, and is trying to send it to sourceware.org.  Because
it has no msgid, sourceware.org thinks it's a new message every time.  The
problem is caused by the fact that sourceware.org accepts the post, but
maeder.org thinks it hasn't, and retries a while later (looks like a slowly
increasing backoff is involved).

  You need to look at the mail logs on maeder.org and see what it thinks has
happened during the conversation at sourceware.org.  It may be that
sourceware.org is bogusly both accepting the mail and yet indicating an error
code to maeder.org, or it may be that maeder.org is misinterpreting the status
code from sourceware.org and only thinking it means "not accepted" when it in
fact means something like "accepted with warnings".

  I don't have access to the sourceware logs, but you may be able to read
/var/log/maillog on maeder.org even if you only have standard user-level
access, it's quite common not to need root access just to read the logs.


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [Fwd: Apologies for multiple messages (Please Help!)]

[Herb Maeder]

On 24 Oct 2008 10:30:08 +0200, Corinna Vinschen wrote:
> we have a strange mail duplication here on the cygwin ML.  The
> mail with Message ID <0ML4cO-1Kt7yy1h2s-000U3k@mx.kundenserver.de>
> is duplicated over and over again.  Could something on sourceware
> be the culprit?

Now that is interesting!  I see the message come back to me with no
Message-ID in the headers (sample included below, with '@' and '.'
replaced to protect the innocent).

I have a theory as to what's going on...

I believe that adding the Message-ID field is the responsibility of the
MUA.  But some mail servers add a Message-ID if it is missing.  I just
verified that my MUA was not creating the Message-ID field, and I'm
guessing that one got added somewhere along the way to Corinna's mailbox.

I don't think that the missing Message-ID is the root cause of the mail
duplication problem.  But I do believe that the missing Message-ID causes
the workaround to the problem to fail.  I'm guessing that the
sourceware.org server may detect and prevent duplicated messages based on 
the Message-ID field.  But since there wasn't one in my message, that
mechanism did not work.

(I've got a hunch that the root cause may be some miscommunication between
the servers due to timeouts or something, but I have no way to verify
that).

In any case, if my guess about the misssing Message-ID is correct, then
I believe that a few of things should be done:

  1) I should make sure that my MUA generates the Message-ID field on
     outgoing messages.  I believe that I have fixed this now (and this
     message should have a "@maeder.org" Message-ID).

  2) If the cygwin mail server is truly depending on Message-ID to stop
     this message duplication, it should somehow ensure that it is always
     there.  One option would be to prevent posts to the list that are 
     missing the Message-ID field (with some reasonable return error
     message to the sender).  NOTE: I'm not sure if generating a Message-ID 
     field on the cygwin mail server would work (that might just add 
     a unique Message-ID on each duplicated message, I'm not sure).

  3) Determine and fix the root of the duplication problem (though I'm not 
     sure how feasible this would really be).


I think most modern MUA's add the Message-ID field, but it is not a strict
requirement.  Implementing something for 2) above may help avoid the
duplication problem for mail sent from misconfigured and/or old school mail
clients (nmh in my case).

How to stop the still repeating evil message?  I still haven't a clue...

Herb.



From cygwin-return-145237-maeder-cygml=maeder DOT org AT cygwin DOT com Thu Oct 23 20:44:41 2008
Return-Path: <cygwin-return-145237-maeder-cygml=maeder DOT org AT cygwin DOT com>
Delivered-To: maeder-cygml AT maeder DOT org
Received: (qmail 87667 invoked by uid 18834); 23 Oct 2008 20:44:41 -0000
Received: from unknown (HELO sourceware DOT org) ([209.132.176.174])
          (envelope-sender <cygwin-return-145237-maeder-cygml=maeder DOT org AT cygwin DOT com>)
          by 192.220.73.146 (qmail-ldap-1.03) with SMTP
          for <maeder-cygml AT maeder DOT org>; 23 Oct 2008 20:44:41 -0000
Received: (qmail 31157 invoked by alias); 23 Oct 2008 20:44:40 -0000
Received: (qmail 31140 invoked by uid 22791); 23 Oct 2008 20:44:38 -0000
X-Spam-Check-By: sourceware DOT org
Received: from maeder DOT org (HELO maeder DOT org) (192.220.73.146)     by sourceware DOT org (qpsmtpd/0.31) with ESMTP; Thu, 23 Oct 2008 20:43:33 +0000
Received: (qmail 86029 invoked by uid 18834); 23 Oct 2008 20:43:31 -0000
Received: from unknown (HELO maeder DOT org) ([127.0.0.1])           (envelope-sender <maeder-cygml AT maeder DOT org>)           by 127.0.0.1 (qmail-ldap-1.03) with SMTP           for <cygwin AT cygwin DOT com>; 23 Oct 2008 20:43:31 -0000
To: cygwin AT cygwin DOT com
From: Herb Maeder <maeder-cygml AT maeder DOT org>
In-reply-to: "Manning, Sid" <sidneym AT qualcomm DOT com> 's message of                  Mon, 20 Oct 2008 11:53:19 PDT.
Subject: Re: Compile time Local Cygwin vs. VMware session on same system
Date: Thu, 23 Oct 2008 13:43:30 -0700
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
List-Id: <cygwin.cygwin DOT com>
List-Unsubscribe: <mailto:cygwin-unsubscribe-maeder-cygml=maeder DOT org AT cygwin DOT com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware DOT org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware DOT org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Status: RO
X-Status: 
X-Keywords:                 
X-UID: 4142

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

[Fwd: Apologies for multiple messages (Please Help!)]

[Corinna Vinschen]

Hi overseers,

we have a strange mail duplication here on the cygwin ML.  The
mail with Message ID <0ML4cO-1Kt7yy1h2s-000U3k@mx.kundenserver.de>
is duplicated over and over again.  Could something on sourceware
be the culprit?


Thanks for looking,
Corinna

----- Forwarded message from Herb Maeder -----

> Date: Thu, 23 Oct 2008 19:49:09 -0700
> From: Herb Maeder
> Subject: Apologies for multiple messages (Please Help!)
> To: cygwin@cygwin.com
> 
> Apologies to the list for getting the same message sent over and over to
> the list from my account.  I can't yet explain what is happening, but I can 
> assure you that I only sent the message once (and in no way differently
> than my other posts to the list).
> 
> I've been on the phone with my ISP for the better part of the last 4 hours
> trying to figure out what is going on, to no avail.  Messages keep getting
> sent to the list, even though my account was deactivated and every process
> connected with it killed.  Unsubscribing from the list didn't help either,
> messages still got through.  Bottom line, I really don't know how I can 
> stop the madness, so I need some help.
> 
> >From the headers it seems to show my domain keeps sending the message to
> sourceware.org, my ISP assures me that their maillog only shows the
> message being sent once.
> 
> I am pulling my hair out trying to figure out what is going on.  Is there
> any chance that the problem could be on the sourceware.org side of things?
> I'm beginning to suspect that, but I cannot confirm.
> 
> Please correct me if I am wrong, but this type of thing has happened to
> the list before, right?  I seem to remember it happening to someone else
> a while back, but I cannot find it in the archives at the moment.
> 
> Can someone please tell me how best to get in touch with the list
> administrator to try and figure out what is going on?  They contact me
> directly at maeder-cygml AT maeder DOT org.  
> 
> Or at the bare minimum disable any more mail from my account to the list.
> The list has been tortured enough....
> 
> Herb.
> 
> 
> P.S.  I'm hoping like hell that this message doesn't loop...
> 
> P.P.S.  Here is what I'm seeing in the headers in case anyone can enlighten
>         me as what is going on.
> 
> 
> >From message 1:
>   Received: from maeder.org (HELO maeder.org) (192.220.73.146)     by sourceware.org (qpsmtpd/0.31) with ESMTP; Thu, 23 Oct 2008 20:43:33 +0000
>   Received: (qmail 86029 invoked by uid 18834); 23 Oct 2008 20:43:31 -0000
> 
> >From message 2:
>   Received: from maeder.org (HELO maeder.org) (192.220.73.146)     by sourceware.org (qpsmtpd/0.31) with ESMTP; Thu, 23 Oct 2008 20:50:13 +0000
>   Received: (qmail 86029 invoked by uid 18834); 23 Oct 2008 20:43:31 -0000
> 
> >From message 3:
>   Received: from maeder.org (HELO maeder.org) (192.220.73.146)     by sourceware.org (qpsmtpd/0.31) with ESMTP; Thu, 23 Oct 2008 21:10:13 +0000
>   Received: (qmail 86029 invoked by uid 18834); 23 Oct 2008 20:43:31 -0000
> 
> 
> The key thing to note is that the time stamp on the second line of each
> message is exactly the same (i.e. I sent the message only once and it
> fired off only one qmail pid).  The first line is when the message
> subsequently gets received by sourceware.org, the timestamps for each new
> message are different.
> 
> At first, it would appear that maeder.org is sending the message to
> sourceware.org repeatedly.  But since the messsage must be coming from
> sourceware.org, there is also the possibility that sourceware.org only
> received the message once, but is passing the same message on at whacky 
> 7-40 minute intervals.  Having someone look at the mail logs on 
> sourceware.org would certainly help.
> 
> 
> --
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> Problem reports:       http://cygwin.com/problems.html
> Documentation:         http://cygwin.com/docs.html
> FAQ:                   http://cygwin.com/faq/

----- End forwarded message -----

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/