Problems with email delivery from sourceware?

Problems with email delivery from sourceware?

[Eli Zaretskii]

Is (or was?) something wrong with email delivery from sourceware?  I'm
receiving only now some emails that were sent to the list 4 days ago.
The headers clearly show that the email messages were stuck on
sourceware.org for all those 4 days.

Does anyone know what is going on?

Sorry for slightly off-topic post.

Re: Problems with email delivery from sourceware?

[Andreas Schwab]

On Jun 15 2021, Eli Zaretskii via Gdb-patches wrote:

> Is (or was?) something wrong with email delivery from sourceware?  I'm
> receiving only now some emails that were sent to the list 4 days ago.

Any example?

> The headers clearly show that the email messages were stuck on
> sourceware.org for all those 4 days.

Are you sure they were stuck @sourceware.org, and not just delayed by
the next hop?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: Problems with email delivery from sourceware?

[Eli Zaretskii]

[-- Attachment #1: Type: text/plain, Size: 661 bytes --]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: Eli Zaretskii <eliz@gnu.org>
> Date: Tue, 15 Jun 2021 18:33:07 +0200
> 
> On Jun 15 2021, Eli Zaretskii via Gdb-patches wrote:
> 
> > Is (or was?) something wrong with email delivery from sourceware?  I'm
> > receiving only now some emails that were sent to the list 4 days ago.
> 
> Any example?

I attach one such email below.

> > The headers clearly show that the email messages were stuck on
> > sourceware.org for all those 4 days.
> 
> Are you sure they were stuck @sourceware.org, and not just delayed by
> the next hop?

I could be misinterpreting the headers, so please see for yourself.

Thanks.


[-- Attachment #2: mdelay --]
[-- Type: application/octet-stream, Size: 6179 bytes --]

From gdb-patches-bounces+eliz=gnu.org@sourceware.org Tue Jun 15 11:37:35 2021
Return-path: <gdb-patches-bounces+eliz=gnu.org@sourceware.org>
Envelope-to: eliz@gnu.org
Delivery-date: Tue, 15 Jun 2021 11:37:35 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:51778)
	by fencepost.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
	(Exim 4.90_1)
	(envelope-from <gdb-patches-bounces+eliz=gnu.org@sourceware.org>)
	id 1ltB8B-0000As-Oc
	for eliz@gnu.org; Tue, 15 Jun 2021 11:37:35 -0400
Received: from ip-8-43-85-97.sourceware.org ([8.43.85.97]:48510 helo=sourceware.org)
	by eggs.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <gdb-patches-bounces+eliz=gnu.org@sourceware.org>)
	id 1ltB86-00053K-RV
	for eliz@gnu.org; Tue, 15 Jun 2021 11:37:35 -0400
Received: from server2.sourceware.org (localhost [IPv6:::1])
	by sourceware.org (Postfix) with ESMTP id 3AD6A3850424
	for <eliz@gnu.org>; Fri, 11 Jun 2021 21:08:20 +0000 (GMT)
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 3AD6A3850424
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org;
	s=default; t=1623445700;
	bh=bqagaYzNe7yfpxBnIioviIuHUp+IdkVzKG8KrBnq75M=;
	h=Date:To:Subject:In-Reply-To:References:List-Id:List-Unsubscribe:
	 List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	 From;
	b=dkIPrvk1t15MoU2EvtqF8KlTqDD4+CBG07JnaV7pTHoYdGMfflmLmXs02eA6XBDM3
	 G/kJ9u0Y8d9gvuBJNZUax/hKlLjRzcZ24ZLV7SIy3XmiEFKUB+592RF1h+kj9BiFd7
	 gK6wAYLxR85N0KWVLHyomXMo4IXvqv2sIbCCAnOU=
X-Original-To: gdb-patches@sourceware.org
Delivered-To: gdb-patches@sourceware.org
Received: from us-smtp-delivery-124.mimecast.com
 (us-smtp-delivery-124.mimecast.com [170.10.133.124])
 by sourceware.org (Postfix) with ESMTP id D1E88385501D
 for <gdb-patches@sourceware.org>; Fri, 11 Jun 2021 21:07:53 +0000 (GMT)
DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org D1E88385501D
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-508-3-XJ14BJOKiOtkR3axoq7w-1; Fri, 11 Jun 2021 17:07:50 -0400
X-MC-Unique: 3-XJ14BJOKiOtkR3axoq7w-1
Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com
 [10.5.11.11])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D16351850600;
 Fri, 11 Jun 2021 21:07:49 +0000 (UTC)
Received: from f33-m1.lan (ovpn-113-207.phx2.redhat.com [10.3.113.207])
 by smtp.corp.redhat.com (Postfix) with ESMTPS id B57A66064B;
 Fri, 11 Jun 2021 21:07:49 +0000 (UTC)
Date: Fri, 11 Jun 2021 14:07:48 -0700
To: Simon Marchi <simon.marchi@polymtl.ca>
Subject: Re: [PATCH 4/4] mi-sym-info.exp: Increase timeout for
 114-symbol-info-functions
Message-ID: <20210611140748.2b1e0146@f33-m1.lan>
In-Reply-To: <299cb515-c9ad-0b45-8885-c950fbdf0774@polymtl.ca>
References: <20210610172641.1052704-1-kevinb@redhat.com>
 <20210610172641.1052704-5-kevinb@redhat.com>
 <299cb515-c9ad-0b45-8885-c950fbdf0774@polymtl.ca>
Organization: Red Hat
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Spam-Status: No, score=-6.6 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH,
 DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_LOW,
 RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS,
 TXREP autolearn=ham autolearn_force=no version=3.4.2
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on
 server2.sourceware.org
X-BeenThere: gdb-patches@sourceware.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Gdb-patches mailing list <gdb-patches.sourceware.org>
List-Unsubscribe: <https://sourceware.org/mailman/options/gdb-patches>,
 <mailto:gdb-patches-request@sourceware.org?subject=unsubscribe>
List-Archive: <https://sourceware.org/pipermail/gdb-patches/>
List-Post: <mailto:gdb-patches@sourceware.org>
List-Help: <mailto:gdb-patches-request@sourceware.org?subject=help>
List-Subscribe: <https://sourceware.org/mailman/listinfo/gdb-patches>,
 <mailto:gdb-patches-request@sourceware.org?subject=subscribe>
From: Kevin Buettner via Gdb-patches <gdb-patches@sourceware.org>
Reply-To: Kevin Buettner <kevinb@redhat.com>
Cc: gdb-patches@sourceware.org
Errors-To: gdb-patches-bounces+eliz=gnu.org@sourceware.org
Sender: "Gdb-patches" <gdb-patches-bounces+eliz=gnu.org@sourceware.org>
Received-SPF: pass client-ip=8.43.85.97; envelope-from=gdb-patches-bounces+eliz=gnu.org@sourceware.org; helo=sourceware.org
X-Spam_score_int: -43
X-Spam_score: -4.4
X-Spam_bar: ----
X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DKIM_VALID_EF=-0.1,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RDNS_DYNAMIC=0.982,SPF_HELO_PASS=-0.001,SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no
X-Spam_action: no action
X-RMAIL-ATTRIBUTES: -D------

On Fri, 11 Jun 2021 16:11:31 -0400
Simon Marchi <simon.marchi@polymtl.ca> wrote:

> On 2021-06-10 1:26 p.m., Kevin Buettner via Gdb-patches wrote:
> > Loading libc.so's symbols increased the amount of time needed for
> > 114-symbol-info-function to fetch symbols, causing a timeout during my
> > testing.  I enclosed the entire block with a "with_timeout_factor 4",
> > which fixes the problem for me.  (Using 2 also fixed it for me, but it
> > might not be enough when running this test on slower machines.)
> > 
> > gdb/testsuite/ChangeLog:
> > 
> > 	* gdb.mi/mi-sym-info.exp (114-symbol-info-function test): Increase
> > 	timeout.  
> 
> This LGTM - as long as you can confirm that this is a case of "GDB
> thinking for a long time before outputting something", 

Yes, confirmed.

> and not a case of
> "there's a lot of output and we are trying to chew too much in a single
> match".  But it looks like the test is already written to consume little
> chunks at a time, since:
> 
>     https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=2d61316c32a9fa3e14786c3312d9ca87c9298db5
> 

Agreed.

Kevin

Re: Problems with email delivery from sourceware?

[Lancelot SIX]

Hi Eli,

From what I can see, I received this message on time. It reached my mail
server on “Fri, 11 Jun 2021 21:08:41 +0000 (UTC)”.

The delay you had was before the mail could reached eggs.gnu.org.  Any
chance this host had issues during last week?

Lancelot.

On Tue, Jun 15, 2021 at 07:52:58PM +0300, Eli Zaretskii via Gdb-patches wrote:
> > From: Andreas Schwab <schwab@linux-m68k.org>
> > Cc: Eli Zaretskii <eliz@gnu.org>
> > Date: Tue, 15 Jun 2021 18:33:07 +0200
> > 
> > On Jun 15 2021, Eli Zaretskii via Gdb-patches wrote:
> > 
> > > Is (or was?) something wrong with email delivery from sourceware?  I'm
> > > receiving only now some emails that were sent to the list 4 days ago.
> > 
> > Any example?
> 
> I attach one such email below.
> 
> > > The headers clearly show that the email messages were stuck on
> > > sourceware.org for all those 4 days.
> > 
> > Are you sure they were stuck @sourceware.org, and not just delayed by
> > the next hop?
> 
> I could be misinterpreting the headers, so please see for yourself.
> 
> Thanks.
> 

Re: Problems with email delivery from sourceware?

[Eli Zaretskii]

> Date: Tue, 15 Jun 2021 18:14:10 +0100
> From: Lancelot SIX <lsix@lancelotsix.com>
> Cc: Andreas Schwab <schwab@linux-m68k.org>, gdb-patches@sourceware.org
> 
> Hi Eli,
> 
> >From what I can see, I received this message on time. It reached my mail
> server on “Fri, 11 Jun 2021 21:08:41 +0000 (UTC)”.
> 
> The delay you had was before the mail could reached eggs.gnu.org.  Any
> chance this host had issues during last week?

The headers say:

  Received: from ip-8-43-85-97.sourceware.org ([8.43.85.97]:48510 helo=sourceware.org)
	  by eggs.gnu.org with esmtp (Exim 4.90_1)
	  (envelope-from <gdb-patches-bounces+eliz=gnu.org@sourceware.org>)
	  id 1ltB86-00053K-RV
	  for eliz@gnu.org; Tue, 15 Jun 2021 11:37:35 -0400

Doesn't this mean that the email was delivered to eggs.gnu.org by
ip-8-43-85-97.sourceware.org on June 15th?

Anyway, I received email from other sources via eggs.gnu.org during
all these days.  So I have no reasons to believe the problem was
there.

But if I was the only one who experienced these strange delays, then I
don't want to waste bandwidth on these issues anymore.  Thanks for
taking the time to look at the message I sent as an example.

Re: Problems with email delivery from sourceware?

[Andreas Schwab]

On Jun 15 2021, Eli Zaretskii wrote:

> The headers say:
>
>   Received: from ip-8-43-85-97.sourceware.org ([8.43.85.97]:48510 helo=sourceware.org)
> 	  by eggs.gnu.org with esmtp (Exim 4.90_1)
> 	  (envelope-from <gdb-patches-bounces+eliz=gnu.org@sourceware.org>)
> 	  id 1ltB86-00053K-RV
> 	  for eliz@gnu.org; Tue, 15 Jun 2021 11:37:35 -0400
>
> Doesn't this mean that the email was delivered to eggs.gnu.org by
> ip-8-43-85-97.sourceware.org on June 15th?

That is true, but the delay is more likely caused by eggs being unable
to receive mail for 4 days.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: Problems with email delivery from sourceware?

[Florian Weimer]

* Eli Zaretskii via Gdb-patches:

> But if I was the only one who experienced these strange delays, then I
> don't want to waste bandwidth on these issues anymore.  Thanks for
> taking the time to look at the message I sent as an example.

I encountered similar delays just before VERP was enabled for
sourceware.org.  Apparently it was not possible to figure out the cause
of the mail delays based on the available Postfix logs.  As I result, I
don't know if it was Red Hat's email infrastructure or sourceware that
caused the delay.  Like in your case, the Received: lines only showed
that the hand-off between the two domains did not happen in a timely
fashion.

Thanks,
Florian

Re: Problems with email delivery from sourceware?

[Eli Zaretskii]

> Date: Tue, 15 Jun 2021 19:12:10 +0300
> From: Eli Zaretskii via Gdb-patches <gdb-patches@sourceware.org>
> 
> Is (or was?) something wrong with email delivery from sourceware?  I'm
> receiving only now some emails that were sent to the list 4 days ago.
> The headers clearly show that the email messages were stuck on
> sourceware.org for all those 4 days.
> 
> Does anyone know what is going on?

It looks like the issue is still going on: this message, which I sent
2 days ago (and several others with the same Subject) only now got
back to me.

Who is overseeing the email delivery on sourceware.org?  I'd like to
write to those persons, instead of spamming this list.

Re: Problems with email delivery from sourceware?

[Pedro Alves]

On 2021-06-17 7:48 a.m., Eli Zaretskii via Gdb-patches wrote:
>> Date: Tue, 15 Jun 2021 19:12:10 +0300
>> From: Eli Zaretskii via Gdb-patches <gdb-patches@sourceware.org>
>>
>> Is (or was?) something wrong with email delivery from sourceware?  I'm
>> receiving only now some emails that were sent to the list 4 days ago.
>> The headers clearly show that the email messages were stuck on
>> sourceware.org for all those 4 days.
>>
>> Does anyone know what is going on?
> 
> It looks like the issue is still going on: this message, which I sent
> 2 days ago (and several others with the same Subject) only now got
> back to me.
> 
> Who is overseeing the email delivery on sourceware.org?  I'd like to
> write to those persons, instead of spamming this list.
> 

I'd suggest reaching out to overseers at sourceware.org.

Re: Problems with email delivery from sourceware?

[Eli Zaretskii]

> From: Florian Weimer <fweimer@redhat.com>
> Cc: Lancelot SIX <lsix@lancelotsix.com>,  Eli Zaretskii <eliz@gnu.org>,
>   schwab@linux-m68k.org
> Date: Tue, 15 Jun 2021 22:32:48 +0200
> 
> * Eli Zaretskii via Gdb-patches:
> 
> > But if I was the only one who experienced these strange delays, then I
> > don't want to waste bandwidth on these issues anymore.  Thanks for
> > taking the time to look at the message I sent as an example.
> 
> I encountered similar delays just before VERP was enabled for
> sourceware.org.  Apparently it was not possible to figure out the cause
> of the mail delays based on the available Postfix logs.  As I result, I
> don't know if it was Red Hat's email infrastructure or sourceware that
> caused the delay.  Like in your case, the Received: lines only showed
> that the hand-off between the two domains did not happen in a timely
> fashion.

Thanks.

FYI: I asked the gnu.org sysadmins, and they replied that something
went wrong with sourceware.org's mail server, which caused the email
from there to trigger the fail2ban rule on gnu.org mail servers.  The
solution was to unban and whitelist sourceware.org.

So yes, both sourceware.org and gnu.org mail delivery were parts of
the puzzle here.