Re: Can we please have the old mailing list back

Re: Can we please have the old mailing list back

[Bernd Edlinger]

On 4/2/20 11:01 AM, Richard Sandiford wrote:
> Bernd Edlinger <bernd.edlinger@hotmail.de> writes:
>> On 4/1/20 8:51 AM, Bernd Edlinger wrote:
>>> On 3/26/20 4:27 PM, Bernd Edlinger wrote:
>>>> On 3/26/20 4:16 PM, Christopher Faylor wrote:
>>>>>
>>>>> marc.info is an independent site that is not associated with
>>>>> sourceware.org.  We don't control it.  If you have questions about their
>>>>> site then ask them.
>>>>>
>>>>> The mailing list software is all easily discernible by investigating
>>>>> email headers and via google but someone else answered your questions
>>>>> later in this thread.
>>>>>
>>>>
>>>> But don't you think that we change something in 6.3 to make them break.
>>>> like no longer sending updates, or something?
>>>>
>>>> Don't you have any idea what changed on our side?
>>>>
>>>> I mean what should I tell them they should do to fix that?????
>>>>
>>>>
>>>
>>> Ah, marc.info is fixed, it turned out that the messages were just Quarantined
>>> because due to the change in the ip adresses, mailing software etc.
>>> marc.info was under the impression that all these messages were just spam.
>>>
>>> That is what they told me:
>>>
>>> "For lists that often get spammed, we set up some silent header-checks
>>> so that mails that don't look like they came from the real listserver
>>> get quarrantined, and don't appear when viewing that list.
>>>
>>> Well, that can break when mailing list software changes - like when they
>>> switched away from ezmlm to Mailman.
>>>
>>> I've updated our filter check and un-quarrantined about 4500 mails to
>>> various gcc- lists that landed there this month."
>>>
>>> So indeed all our mailing list message are again on marc.info,
>>> I think when it can handle lkml it can handle gcc-patches as well.
>>>
>>> Many Thanks go to Hank Leininger who does a gread job with marc.info.
>>>
>>>
>>> Bernd.
>>>
>>
>> PS: I have a discovered a very serious problem with the mailing lists
>> that must be fixed by our overseers.
>>
>> That is the scubbed attachments.
>>
>> As an example please look at this one:
>> https://marc.info/?l=gdb-patches&m=158571308379946&w=2
>>
>>
>> you see this:
>>
>> -------------- next part --------------
>> A non-text attachment was scrubbed...
>> Name: 0001-Fix-range-end-handling-of-inlined-subroutines.patch
>> Type: text/x-patch
>> Size: 10992 bytes
>> Desc: not available
>> URL: <http://sourceware.org/pipermail/gdb-patches/attachments/20200313/5158bb87/attachment.bin>
>>
>> So there are two serious problems here:
>>
>> 1. there is a single point of failure, if sourceware.org goes down the attachment is lost.
>>
>> 2. since the url is http: a man in the middle can impersonate sourceware.org and give you a
>> virus instead of my patch file.
>> It does not help that sourceware.org redirects the download to https://sourceware.org/pipermail/gdb-patches/attachments/20200313/5158bb87/attachment.bin
>> an attacker will not be so polite to do that.
>>
>>
>> @overseeers: PLEASE STOP IMMEDIATELY THAT SCRUBBING
>>
>> can you act now, or do you need a CVE number first ?
> 
> The overseers are reachable on:
> 
>   https://sourceware.org/mailman/listinfo/overseers
> 
> Please keep the tone civil.  I hope we never see the day where the GCC/
> sourceware lists have to have a code of conduct, but if we did, I think
> some of the messages on this thread would have breached it.
> 
> Thanks,
> Richard
> 
Thanks, for reminding me.

I do personally full-heatedly apologize, and regret what I said above.

I am sorry if I made you feel bad.  That was not the true intention of what
I said.


I asked Hank Leininger for clarification how mark.info subscribes the mails,
and what data he gets exactly from us.

I am still waiting for his response, and let you know what he says.

In the meantime, culd you please change http: to https:


Thanks
Bernd.

Re: Can we please have the old mailing list back

[Richard Sandiford]

Bernd Edlinger <bernd.edlinger@hotmail.de> writes:
> On 4/2/20 11:01 AM, Richard Sandiford wrote:
>> Bernd Edlinger <bernd.edlinger@hotmail.de> writes:
>>> On 4/1/20 8:51 AM, Bernd Edlinger wrote:
>>>> On 3/26/20 4:27 PM, Bernd Edlinger wrote:
>>>>> On 3/26/20 4:16 PM, Christopher Faylor wrote:
>>>>>>
>>>>>> marc.info is an independent site that is not associated with
>>>>>> sourceware.org.  We don't control it.  If you have questions about their
>>>>>> site then ask them.
>>>>>>
>>>>>> The mailing list software is all easily discernible by investigating
>>>>>> email headers and via google but someone else answered your questions
>>>>>> later in this thread.
>>>>>>
>>>>>
>>>>> But don't you think that we change something in 6.3 to make them break.
>>>>> like no longer sending updates, or something?
>>>>>
>>>>> Don't you have any idea what changed on our side?
>>>>>
>>>>> I mean what should I tell them they should do to fix that?????
>>>>>
>>>>>
>>>>
>>>> Ah, marc.info is fixed, it turned out that the messages were just Quarantined
>>>> because due to the change in the ip adresses, mailing software etc.
>>>> marc.info was under the impression that all these messages were just spam.
>>>>
>>>> That is what they told me:
>>>>
>>>> "For lists that often get spammed, we set up some silent header-checks
>>>> so that mails that don't look like they came from the real listserver
>>>> get quarrantined, and don't appear when viewing that list.
>>>>
>>>> Well, that can break when mailing list software changes - like when they
>>>> switched away from ezmlm to Mailman.
>>>>
>>>> I've updated our filter check and un-quarrantined about 4500 mails to
>>>> various gcc- lists that landed there this month."
>>>>
>>>> So indeed all our mailing list message are again on marc.info,
>>>> I think when it can handle lkml it can handle gcc-patches as well.
>>>>
>>>> Many Thanks go to Hank Leininger who does a gread job with marc.info.
>>>>
>>>>
>>>> Bernd.
>>>>
>>>
>>> PS: I have a discovered a very serious problem with the mailing lists
>>> that must be fixed by our overseers.
>>>
>>> That is the scubbed attachments.
>>>
>>> As an example please look at this one:
>>> https://marc.info/?l=gdb-patches&m=158571308379946&w=2
>>>
>>>
>>> you see this:
>>>
>>> -------------- next part --------------
>>> A non-text attachment was scrubbed...
>>> Name: 0001-Fix-range-end-handling-of-inlined-subroutines.patch
>>> Type: text/x-patch
>>> Size: 10992 bytes
>>> Desc: not available
>>> URL: <http://sourceware.org/pipermail/gdb-patches/attachments/20200313/5158bb87/attachment.bin>
>>>
>>> So there are two serious problems here:
>>>
>>> 1. there is a single point of failure, if sourceware.org goes down the attachment is lost.
>>>
>>> 2. since the url is http: a man in the middle can impersonate sourceware.org and give you a
>>> virus instead of my patch file.
>>> It does not help that sourceware.org redirects the download to https://sourceware.org/pipermail/gdb-patches/attachments/20200313/5158bb87/attachment.bin
>>> an attacker will not be so polite to do that.
>>>
>>>
>>> @overseeers: PLEASE STOP IMMEDIATELY THAT SCRUBBING
>>>
>>> can you act now, or do you need a CVE number first ?
>> 
>> The overseers are reachable on:
>> 
>>   https://sourceware.org/mailman/listinfo/overseers
>> 
>> Please keep the tone civil.  I hope we never see the day where the GCC/
>> sourceware lists have to have a code of conduct, but if we did, I think
>> some of the messages on this thread would have breached it.
>> 
>> Thanks,
>> Richard
>> 
> Thanks, for reminding me.
>
> I do personally full-heatedly apologize, and regret what I said above.
>
> I am sorry if I made you feel bad.  That was not the true intention of what
> I said.
>
>
> I asked Hank Leininger for clarification how mark.info subscribes the mails,
> and what data he gets exactly from us.
>
> I am still waiting for his response, and let you know what he says.
>
> In the meantime, culd you please change http: to https:

Just in case: I'm not actually an overseer myself, but I can see how
my message could give that impression.  I think the request would be
better sent to the overseers list, if you haven't already

Sorry for the confusion :-)

Richard

Re: Can we please have the old mailing list back

[Bernd Edlinger]

On 4/2/20 11:48 AM, Richard Sandiford wrote:
> Bernd Edlinger <bernd.edlinger@hotmail.de> writes:
>> On 4/2/20 11:01 AM, Richard Sandiford wrote:
>>> Bernd Edlinger <bernd.edlinger@hotmail.de> writes:
>>>> On 4/1/20 8:51 AM, Bernd Edlinger wrote:
>>>>> On 3/26/20 4:27 PM, Bernd Edlinger wrote:
>>>>>> On 3/26/20 4:16 PM, Christopher Faylor wrote:
>>>>>>>
>>>>>>> marc.info is an independent site that is not associated with
>>>>>>> sourceware.org.  We don't control it.  If you have questions about their
>>>>>>> site then ask them.
>>>>>>>
>>>>>>> The mailing list software is all easily discernible by investigating
>>>>>>> email headers and via google but someone else answered your questions
>>>>>>> later in this thread.
>>>>>>>
>>>>>>
>>>>>> But don't you think that we change something in 6.3 to make them break.
>>>>>> like no longer sending updates, or something?
>>>>>>
>>>>>> Don't you have any idea what changed on our side?
>>>>>>
>>>>>> I mean what should I tell them they should do to fix that?????
>>>>>>
>>>>>>
>>>>>
>>>>> Ah, marc.info is fixed, it turned out that the messages were just Quarantined
>>>>> because due to the change in the ip adresses, mailing software etc.
>>>>> marc.info was under the impression that all these messages were just spam.
>>>>>
>>>>> That is what they told me:
>>>>>
>>>>> "For lists that often get spammed, we set up some silent header-checks
>>>>> so that mails that don't look like they came from the real listserver
>>>>> get quarrantined, and don't appear when viewing that list.
>>>>>
>>>>> Well, that can break when mailing list software changes - like when they
>>>>> switched away from ezmlm to Mailman.
>>>>>
>>>>> I've updated our filter check and un-quarrantined about 4500 mails to
>>>>> various gcc- lists that landed there this month."
>>>>>
>>>>> So indeed all our mailing list message are again on marc.info,
>>>>> I think when it can handle lkml it can handle gcc-patches as well.
>>>>>
>>>>> Many Thanks go to Hank Leininger who does a gread job with marc.info.
>>>>>
>>>>>
>>>>> Bernd.
>>>>>
>>>>
>>>> PS: I have a discovered a very serious problem with the mailing lists
>>>> that must be fixed by our overseers.
>>>>
>>>> That is the scubbed attachments.
>>>>
>>>> As an example please look at this one:
>>>> https://marc.info/?l=gdb-patches&m=158571308379946&w=2
>>>>
>>>>
>>>> you see this:
>>>>
>>>> -------------- next part --------------
>>>> A non-text attachment was scrubbed...
>>>> Name: 0001-Fix-range-end-handling-of-inlined-subroutines.patch
>>>> Type: text/x-patch
>>>> Size: 10992 bytes
>>>> Desc: not available
>>>> URL: <http://sourceware.org/pipermail/gdb-patches/attachments/20200313/5158bb87/attachment.bin>
>>>>
>>>> So there are two serious problems here:
>>>>
>>>> 1. there is a single point of failure, if sourceware.org goes down the attachment is lost.
>>>>
>>>> 2. since the url is http: a man in the middle can impersonate sourceware.org and give you a
>>>> virus instead of my patch file.
>>>> It does not help that sourceware.org redirects the download to https://sourceware.org/pipermail/gdb-patches/attachments/20200313/5158bb87/attachment.bin
>>>> an attacker will not be so polite to do that.
>>>>
>>>>
>>>> @overseeers: PLEASE STOP IMMEDIATELY THAT SCRUBBING
>>>>
>>>> can you act now, or do you need a CVE number first ?
>>>
>>> The overseers are reachable on:
>>>
>>>   https://sourceware.org/mailman/listinfo/overseers
>>>
>>> Please keep the tone civil.  I hope we never see the day where the GCC/
>>> sourceware lists have to have a code of conduct, but if we did, I think
>>> some of the messages on this thread would have breached it.
>>>
>>> Thanks,
>>> Richard
>>>
>> Thanks, for reminding me.
>>
>> I do personally full-heatedly apologize, and regret what I said above.
>>
>> I am sorry if I made you feel bad.  That was not the true intention of what
>> I said.
>>
>>
>> I asked Hank Leininger for clarification how mark.info subscribes the mails,
>> and what data he gets exactly from us.
>>
>> I am still waiting for his response, and let you know what he says.
>>
>> In the meantime, culd you please change http: to https:
> 
> Just in case: I'm not actually an overseer myself, but I can see how
> my message could give that impression.  I think the request would be
> better sent to the overseers list, if you haven't already
> 
> Sorry for the confusion :-)
> 
> Richard
> 

No problem, I know who you are.

I did add CC: overseeers@sourceware.org this time.
Every time I hit reply-to-all the overseeer are not in the 
list I use thunderbird here, and someting on their e-mail address
is so bogus that thunderbird does not want to send them mails.

By the way instead of using the "please use the list"
in angle brackets, which I never look at, I look only
at the name.  I would suggest you just use your name
and overseeers@sourceware.org if you prefer, or something
that is quarantined, until you can look at it if it is
a personal mail or something of general interest.

But if I write a mail I spend a lot of time for it,
and if it bounces, that work is lost.  This makes
a bad start, and the first impression usually decides
a lot in our lives.


Thanks
Bernd.

Re: Can we please have the old mailing list back

[Christopher Faylor]

[gcc mailing list removed]
On Thu, Apr 02, 2020 at 12:01:59PM +0200, Bernd Edlinger wrote:
>On 4/2/20 11:48 AM, Richard Sandiford wrote:
>> Sorry for the confusion :-)
>
>No problem, I know who you are.

So do I.  Hi Richard!

>But if I write a mail I spend a lot of time for it, and if it bounces,
>that work is lost.  This makes a bad start, and the first impression
>usually decides a lot in our lives.

Thunderbird, and most other email clients, have a "Sent" folder so this
is just ridiculous.

I'm not going to respond to most of the other points.  Having a public
discussion about someone's inconvenience with my email address is silly.

I will reiterate the that if there are issues with marc.info, or any
other non-sourceware.org mailing list archiver they need to be addressed
with the non-sourceware.org entity.

If there are issues with email received from sourceware.org/gcc.gnu.org
like attachments looking funny, then, of course, we will want to hear
about them.  We care if email doesn't show up in our subscribers inbox
or if the sourceware.org/gcc.gnu.org archives are incorrect.

We don't care if attachments look wrong on sites over which we have no
control.  We don't care if email isn't showing up on other archiving
sites.  We would certainly listen to concerns from the owners of other
sites but there are no guarantees that we would change anything to make
their lives easier.  That's not how this works.

Your valid point about http vs. https has been corrected.  Future
attachments will show https.  We'll schedule a mailing list archive
rebuild at some point to fix the rest. Thanks for  bringing that issue
to our attention.

Re: Can we please have the old mailing list back

[Bernd Edlinger]

On 4/2/20 6:00 PM, Christopher Faylor wrote:
> [gcc mailing list removed]
> On Thu, Apr 02, 2020 at 12:01:59PM +0200, Bernd Edlinger wrote:
>> On 4/2/20 11:48 AM, Richard Sandiford wrote:
>>> Sorry for the confusion :-)
>>
>> No problem, I know who you are.
> 
> So do I.  Hi Richard!
> 
>> But if I write a mail I spend a lot of time for it, and if it bounces,
>> that work is lost.  This makes a bad start, and the first impression
>> usually decides a lot in our lives.
> 
> Thunderbird, and most other email clients, have a "Sent" folder so this
> is just ridiculous.
> 
> I'm not going to respond to most of the other points.  Having a public
> discussion about someone's inconvenience with my email address is silly.
> 

how about also apologizing for the silly-word here?


> I will reiterate the that if there are issues with marc.info, or any
> other non-sourceware.org mailing list archiver they need to be addressed
> with the non-sourceware.org entity.
> 
> If there are issues with email received from sourceware.org/gcc.gnu.org
> like attachments looking funny, then, of course, we will want to hear
> about them.  We care if email doesn't show up in our subscribers inbox
> or if the sourceware.org/gcc.gnu.org archives are incorrect.
> 
> We don't care if attachments look wrong on sites over which we have no
> control.  We don't care if email isn't showing up on other archiving
> sites.  We would certainly listen to concerns from the owners of other
> sites but there are no guarantees that we would change anything to make
> their lives easier.  That's not how this works.
> 
> Your valid point about http vs. https has been corrected.  Future
> attachments will show https.  We'll schedule a mailing list archive
> rebuild at some point to fix the rest. Thanks for  bringing that issue
> to our attention.
> 

Re: Can we please have the old mailing list back

[Gerald Pfeifer]

On Thu, 2 Apr 2020, Bernd Edlinger via Overseers wrote:
>> I'm not going to respond to most of the other points.  Having a public
>> discussion about someone's inconvenience with my email address is silly.
> how about also apologizing for the silly-word here?

May I suggest to take a deep breath?

This thread started on the wrong foot, and while I think I understand 
where you came from, Bernd, I also believe some of your statements 
came across stronger and more emotional than you may have intended.

Many of us here are volunteers, doing this on our own time, and all
of us want to improve GCC and the infrastructure (gcc.gnu.org and
otherwise) - in our different ways.

I, for one, appreciate the work you have been doing and equally how
Christopher and others have been taking care of our infrastructure.

Thank you!
Gerald

Re: Can we please have the old mailing list back

[Christopher Faylor]

On Thu, Apr 02, 2020 at 10:34:28PM +0200, Gerald Pfeifer wrote:
>I, for one, appreciate the work you have been doing and equally how
>Christopher and others have been taking care of our infrastructure.

I'm happy to help.

cgf

Re: Can we please have the old mailing list back

[Bernd Edlinger]

On 4/2/20 10:47 PM, Christopher Faylor wrote:
> On Thu, Apr 02, 2020 at 10:34:28PM +0200, Gerald Pfeifer wrote:
>> I, for one, appreciate the work you have been doing and equally how
>> Christopher and others have been taking care of our infrastructure.
> 
> I'm happy to help.
> 

me too.

Bernd.

Re: Can we please have the old mailing list back

[Bernd Edlinger]

On 4/2/20 10:34 PM, Gerald Pfeifer wrote:
> On Thu, 2 Apr 2020, Bernd Edlinger via Overseers wrote:
>>> I'm not going to respond to most of the other points.  Having a public
>>> discussion about someone's inconvenience with my email address is silly.
>> how about also apologizing for the silly-word here?
> 
> May I suggest to take a deep breath?
> 
> This thread started on the wrong foot, and while I think I understand 
> where you came from, Bernd, I also believe some of your statements 
> came across stronger and more emotional than you may have intended.
> 
> Many of us here are volunteers, doing this on our own time, and all
> of us want to improve GCC and the infrastructure (gcc.gnu.org and
> otherwise) - in our different ways.
> 
> I, for one, appreciate the work you have been doing and equally how
> Christopher and others have been taking care of our infrastructure.
> 

Thanks Gerald, and once again apologies Christopher.
FYI: We are here trapped in our home, and cannot leave when we want.

And by the way where is the new data center located, and how do you
expect to be affected by corona?

What I said to Christopher is how I feel, but I believe everybody
feels a bit depressed, when mails bounce.  That can easily avoided,
but my recommendations to avoid that was ignored.  And the s-word was
used, which makes me feel kind of really sorry.

BTW: meanwhile I got a feed-back from Hank Leininger of marc.info.

It turns out, that the mail I observed on gdb-patches was not
received in the normal way, but from the web-gui, they crawled that
that back-filling, I may have misunderstood how that happens.
So the issue will fix itself, more or less, once they subscribed
the gdb-patches archive, that archive was just added yesterday on my
request, apparently.

I quote from Hank's mail which I just received:

> 
> Aha, OK this is because of where/when/how we got that data - it's one
> that we downloaded from an existing archive and bulk-inserted.
> 
> Pipermail has two ways it can publish old list traffic:
> 
> 1) One single raw-ish .mbox file, fairly unmolested. In my experience
>    almost no Pipermail admins enable this any more (spam concerns?
>    resource concerns? removed from recent Pipermail versions by default?
>    I do not know).
> 
> 2) A .txt(.gz)? file for each month. These are less raw - they have
>    attachments stripped out and replaced with a URL as you saw; they
>    also cook some mail headers, maybe do some s/@/ at / mangling, etc.
> 
> Since in practice we can never get #1 any more, we do #2.
> 
> But we don't try to follow and fetch each attachment-link and then fuss
> with it until it looks like a regular attachment again.  I agree with
> you it's a shame to not have it all in one place. MARC is only an
> unofficial archive (for most projects anyway), but still it would be
> nice to be complete. But I don't have the bandwidth to implement the
> attachment snarfing and re-attaching.
> 
> And it'd indeed be smarter for sourceware's pipermail to output https://
> versions of those links; it appears the server does support that fine.
> It may be either an overlooked default setting, or a deliberate
> resource-constraint choice.
> 
> MARC's policy on various things (like message-removal or -editing) is
> usually to follow the lead of the official list archives/admins - so if
> they switch their config to publishing https:// versions, I'll make an
> effort to go through our sourceware.org-origin mailing list archives and
> convert attachment URLs from http:// to https://. (That will be a little
> bit painful to implement though, so I might not be fast.)
> 
>> My question, did I understand right, that you just subscribe to a mailing
>> list, then receive the mails and store them on your hard drive(s) ?
>>
>> So what you show as raw message is 1:1 what you recived from sourceware.org ?
> 
> Yup, once we are subscribed, when we get new messages we'll preserve
> them entirely[*].
> 
> So:
> 
> - Messages with inline patches that were never attachments, we've got.
> 
> - Messages sent once we've subscribed that have attachments, we've got,
>   for example:
> 
>     https://marc.info/?l=gdb-patches&m=158584046107542&w=2
> 
> - Messages we bulk-inserted that had stripped attachments, we have only
>   what was in the .txt(.gz)? version of the message.
> 
> [*] For some version of "entirely". We don't preserve all mail headers
>     for instance, only the ones I thought were worth keeping back in
>     1997 and a few added since then 


One final question, can we provide them with
"1) One single raw-ish .mbox file, fairly unmolested."
with the data they need to fill the gaps in the history?


Thanks
Bernd.