Re: text/x-* attachments strippe

Re: text/x-* attachments strippe

[Gerald Pfeifer]

On Mon, 9 Mar 2020, Thomas König wrote:
> I also seem to have missed all discussion on this change (if there was 
> anything). I do not understand why such a huge change was implemented 
> that way, and who did this. Perhaos the person(s) responsible could 
> speak up about this.

Let's be careful - most people working on this are volunteers, and 
it's great that they took care and spent evenings and weekends.

Could this have gone a bit smoother? Yes.  More collaborative? Maybe.

But it's been old system and quite an upgrade, so changes (including
some inconvenient ones) are to be expected.  I have found and reported
and (with the little I can) helped address some issues and will continue
to do so -- and am confident this is heading in the right direction.

Gerald

Re: text/x-* attachments strippe

[Frank Ch. Eigler]

Hi -

Thanks for the kind words.

> Could this have gone a bit smoother? Yes.  More collaborative? Maybe.

We tried: the plan to migrate to mailman was included by reference
from the systemwide announcement blast two weeks ago:
https://sourceware.org/sourceware-wiki/MigrationStatus/

We continue to welcome advice & help.

- FChE

Re: text/x-* attachments stripped (was: Re: gcc ML archive: text/x-patch attachments no longer shown inline

[Nathan Sidwell]

On 3/9/20 9:57 AM, Thomas König wrote:
> Hi,
> 
> I concur with what Jakub wrote. The new web interface is much less useful than the old one; a severe regression for developers, so to speak.

OMG I've just looked.  It's awful.  Sorry, but No.

For example 
https://gcc.gnu.org/pipermail/gcc-patches/2020-March/date.html just 
gives a list of emails, no dates shown.  There's no indication what the 
ordering is -- and apparently it is not most recent first.

nathan

-- 
Nathan Sidwell

Re: text/x-* attachments stripped (was: Re: gcc ML archive: text/x-patch attachments no longer shown inline

[Andreas Schwab]

On Mär 09 2020, Nathan Sidwell wrote:

> For example https://gcc.gnu.org/pipermail/gcc-patches/2020-March/date.html
> just gives a list of emails, no dates shown.  There's no indication what
> the ordering is -- and apparently it is not most recent first.

Heading says:

Starting: Sun Mar 1 01:37:00 GMT 2020
Ending: Mon Mar 9 16:56:12 GMT 2020

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: text/x-* attachments stripped (was: Re: gcc ML archive: text/x-patch attachments no longer shown inline

[Jonathan Wakely]

On Mon, 9 Mar 2020 at 16:58, Nathan Sidwell wrote:
>
> On 3/9/20 9:57 AM, Thomas König wrote:
> > Hi,
> >
> > I concur with what Jakub wrote. The new web interface is much less useful than the old one; a severe regression for developers, so to speak.
>
> OMG I've just looked.  It's awful.  Sorry, but No.
>
> For example
> https://gcc.gnu.org/pipermail/gcc-patches/2020-March/date.html just
> gives a list of emails, no dates shown.  There's no indication what the
> ordering is -- and apparently it is not most recent first.

Oldest first. Add #end to jump to the newest.

(That's a workaround for now, I'm still not fond of the new format)

Re: text/x-* attachments stripped

[Nathan Sidwell]

On 3/9/20 1:07 PM, Jonathan Wakely wrote:
> On Mon, 9 Mar 2020 at 16:58, Nathan Sidwell wrote:
>>
>> On 3/9/20 9:57 AM, Thomas König wrote:
>>> Hi,
>>>
>>> I concur with what Jakub wrote. The new web interface is much less useful than the old one; a severe regression for developers, so to speak.
>>
>> OMG I've just looked.  It's awful.  Sorry, but No.
>>
>> For example
>> https://gcc.gnu.org/pipermail/gcc-patches/2020-March/date.html just
>> gives a list of emails, no dates shown.  There's no indication what the
>> ordering is -- and apparently it is not most recent first.
> 
> Oldest first. Add #end to jump to the newest.

You're joking, right?

nathan

-- 
Nathan Sidwell

Re: text/x-* attachments stripped (was: Re: gcc ML archive: text/x-patch attachments no longer shown inline (was:Re: Mailing list stripping off attachments))

[Thomas König]

Hi,

Some comments.

Generally, I found the old format to be very good for navigating, and I
would like to have the new one match the old one as closely as possible.

> 1) the by date monthly list of mails used to be ordered newest to oldest
> mails first, now it is oldest to newest, so when dealing with new stuff one
> has to always scroll dow
Also important if one just wants to search; it is often the newest
(or newer) message that is of interest.

> 2) the dates and times of mails used to be shown (date as a section in the
> list, times in the left column), now there is nothing, so without clicking
> something it is hard to guess how exactly old it is

I concur, that is irritating. You can guess, but you don't get the same
sort of what's happening just from a glance.

> 3) the columns were nicer (date, subject left justified, email right
> justified, now there are no columns)

This does make it harder to see what's what, true.

> 4) some headers were shown, now there is nothing

Yep.

> 5) emails used to be sanitized against harvesters, now they aren't

Surely, there is a feature for this?

> 6) there used to be a Raw text URL to grab the raw email, now there is nothing

This is something that I have rarely used.

As far as the advantages go: A per-thread view is nice, but I don't
think having it outweighs the disadvantages above.

 From my personal preference, having 1), 2) and 3) (basically, let the
new interface look like the old one) would be great.

Regards

	Thomas

Re: text/x-* attachments stripped (was: Re: gcc ML archive: text/x-patch attachments no longer shown inline (was:Re: Mailing list stripping off attachments))

[Joseph Myers]

On Mon, 9 Mar 2020, Jakub Jelinek via Overseers wrote:

> 5) emails used to be sanitized against harvesters, now they aren't

The pipermail munging feature was unusably bad (it messed up Texinfo diffs 
very badly, including in the mbox version of the archive, e.g. "+@node" at 
the start of a line was interpreted as an email address).

I'm very doubtful that munging that produces human-readable results 
actually does anything against harvesters.  Spammers happily send mail to 
addresses that don't exist at all (taken from message-ids, using random 
names as local-parts, etc.); I expect they'll happily try generating 
addresses from " at ".

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: text/x-* attachments stripped (was: Re: gcc ML archive: text/x-patch attachments no longer shown inline (was:Re: Mailing list stripping off attachments))

[Jonathan Wakely]

On Mon, 9 Mar 2020 at 19:57, Thomas König wrote:
> As far as the advantages go: A per-thread view is nice, but I don't
> think having it outweighs the disadvantages above.

We always had a threaded view:
https://gcc.gnu.org/legacy-ml/gcc-bugs/2020-03/threads.html
It just wasn't the default:
https://gcc.gnu.org/legacy-ml/gcc-bugs/2020-03/

Re: text/x-* attachments stripped (was: Re: gcc ML archive: text/x-patch attachments no longer shown inline (was:Re: Mailing list stripping off attachments))

[Jonathan Wakely]

[-- Attachment #1: Type: text/plain, Size: 632 bytes --]

On Mon, 9 Mar 2020 at 10:28, Jakub Jelinek wrote:
> 6) there used to be a Raw text URL to grab the raw email, now there is nothing

Based on info from #overseers ...

While you can't download the raw text of an individual email now, you
can get the entire month's mail in a compressed archive, from
https://gcc.gnu.org/pipermail/gcc-bugs/

Also, in each mail in the web archives the sender's address at the top
of the page is a hyperlink which includes the message-id, allowing you
to reply to it and preserve threading.
I've adapted the script I was using to fetch the "raw text" mail and
reply to it, the new version is attached.

[-- Attachment #2: get_gcc_mail --]
[-- Type: application/octet-stream, Size: 674 bytes --]

#!/bin/sh
# Download mail from GCC list archive.

if [[ $* == '' || $* == '--help' ]]
then
  echo "Usage: $0 https://gcc.gnu.org/ml/list/YYYY-MM/msgNNNNN.html" >&2
  test $# -eq 1
  exit
fi

urldecode() {
  awk -niord '{printf RT?$0chr("0x"substr(RT,2)):$0}' RS=%..
}

set -xe
draft=`mktemp /dev/shm/gcc-patches.XXXXXX`
for i in "$@"
do
  list=${i#*gcc.gnu.org/pipermail/}
  list=${list%%/*}
  echo "CC: $list@gcc.gnu.org" > $draft
  curl --silent --fail "$i" | sed -n '/LINK REL="made"/{s/^.*="mailto:/To: /;s/?Subject=Re:%20/%0aSubject: /;s/&In-Reply-To=/%0aIn-Reply-To: /;s/">$/%0a%0a/p}' | urldecode >> $draft
  mutt -H $draft -- "$(awk '/^To:/{print $2}' $draft)"
done

Re: text/x-* attachments stripped (was: Re: gcc ML archive: text/x-patch attachments no longer shown inline (was:Re: Mailing list stripping off attachments))

[Martin Liška]

On 3/9/20 11:25 AM, Jakub Jelinek wrote:
> On Mon, Mar 09, 2020 at 10:46:31AM +0100, Tobias Burnus wrote:
>> Hi Thomas, hi Overseers
>>
>> I can confirm that those are stripped off!
>>
>> I did sent an email with three attachments:
>> * test.txt (text/plain)
>> * test.diff (text/x-diff)
>> * the company's disclaimer
>>
>> The attachment with 'text/x-diff' MIME was removed :-(
>> See: https://gcc.gnu.org/pipermail/fortran/current/054078.html
> 
> A different mail archiver is now used it seems.
> For the mails before the sourceware move, one can access the old one too,
> e.g.
> https://gcc.gnu.org/legacy-ml/gcc-bugs/2020-03/
> is the old one vs.
> https://gcc.gnu.org/pipermail/gcc-bugs/2020-March/
> I've been using the gcc-bugs mail archive all the time in the past, but I'm
> afraid pipermail at least in current configuration is significant step back
> and I'll likely just use my mailbox from now on.
> Some reasons:
> 1) the by date monthly list of mails used to be ordered newest to oldest
> mails first, now it is oldest to newest, so when dealing with new stuff one
> has to always scroll down
> 2) the dates and times of mails used to be shown (date as a section in the
> list, times in the left column), now there is nothing, so without clicking
> something it is hard to guess how exactly old it is
> 3) the columns were nicer (date, subject left justified, email right
> justified, now there are no columns)
> 4) some headers were shown, now there is nothing
> 5) emails used to be sanitized against harvesters, now they aren't
> 6) there used to be a Raw text URL to grab the raw email, now there is nothing
> 
> 	Jakub
> 

Hello.

I agree with Jakub that the listed feature were nicer about the previous mail list
archiver. On the other hand, I agree that we want to use something more recent that
is support and under some development. That said, we did we decide to use mailman-2.1
which is a legacy release that can be shortly out of support? Have you consider
using version 3.3.0?

I'm willing to customize the mail archiver, it should be quite simple.
Would it be possible to apply local patches for a RHEL package that's install
on the system?

Thanks,
Martin

mailman customization

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 515 bytes --]

Hello.

I believe we can quite easily customize mailman 2.1 to match our needs.
The biggest challenge I see is a proper testing as I don't see it easy
to set up a local mailman instance. I've got a patch that changes:

- by date sorting will be done in reverse order
- default link of e.g. https://gcc.gnu.org/pipermail/gcc-patches/2020-April/ will
   point to sorting by date
- email date is added to the listing

Further changes would be possible but I'll need a cooperation from oversees people.

Thanks,
Martin

[-- Attachment #2: mailman-improvement.patch --]
[-- Type: text/x-patch, Size: 1191 bytes --]

diff --git a/Mailman/Archiver/HyperArch.py b/Mailman/Archiver/HyperArch.py
index 4469193..2e186ff 100644
--- a/Mailman/Archiver/HyperArch.py
+++ b/Mailman/Archiver/HyperArch.py
@@ -637,7 +637,7 @@ class HyperArchive(pipermail.T):
     FILEMODE = 0660
 
     VERBOSE = 0
-    DEFAULTINDEX = 'thread'
+    DEFAULTINDEX = 'date'
     ARCHIVE_PERIOD = 'month'
 
     THREADLAZY = 0
diff --git a/Mailman/Archiver/HyperDatabase.py b/Mailman/Archiver/HyperDatabase.py
index 2475d47..3566425 100644
--- a/Mailman/Archiver/HyperDatabase.py
+++ b/Mailman/Archiver/HyperDatabase.py
@@ -71,7 +71,7 @@ class DumbBTree:
     def __sort(self, dirty=None):
         if self.__dirty == 1 or dirty:
             self.sorted = self.dict.keys()
-            self.sorted.sort()
+            self.sorted.sort(reverse = self.path.endswith('date'))
             self.__dirty = 0
 
     def lock(self):
diff --git a/templates/en/archidxentry.html b/templates/en/archidxentry.html
index f9bb57a..365e836 100644
--- a/templates/en/archidxentry.html
+++ b/templates/en/archidxentry.html
@@ -1,3 +1,5 @@
+<I>%(datestr)s
+</I>
 <LI><A HREF="%(filename)s">%(subject)s
 </A><A NAME="%(sequence)i">&nbsp;</A>
 <I>%(author)s

Re: mailman customization

[Frank Ch. Eigler]

Hi -

> I believe we can quite easily customize mailman 2.1 to match our needs.
> The biggest challenge I see is a proper testing as I don't see it easy
> to set up a local mailman instance. I've got a patch that changes:

I suppose we can do some local RPM respins - as long as these changes
are small and rare.  Even with a deadish upstream, distro reporting
would be nice, at least at the centos/fedora point (?), as a reference
place to stash the patch and get us a bug#.

- FChE

Re: mailman customization

[Martin Liška]

On 4/3/20 5:54 PM, Frank Ch. Eigler wrote:
> Hi -
> 
>> I believe we can quite easily customize mailman 2.1 to match our needs.
>> The biggest challenge I see is a proper testing as I don't see it easy
>> to set up a local mailman instance. I've got a patch that changes:
> 
> I suppose we can do some local RPM respins - as long as these changes
> are small and rare.

That would be great. Should I create a git repo where we'll stack these changes?

>  Even with a deadish upstream, distro reporting
> would be nice, at least at the centos/fedora point (?), as a reference
> place to stash the patch and get us a bug#.

Can you please do it for me as I don't have any experience with Fedora
packaging?

Thank you,
Martin

> 
> - FChE
> 

Re: mailman customization

[Christopher Faylor]

On Fri, Apr 03, 2020 at 05:58:51PM +0200, Martin Liška wrote:
>On 4/3/20 5:54 PM, Frank Ch. Eigler wrote:
>>Hi -
>>
>>>I believe we can quite easily customize mailman 2.1 to match our needs.
>>>The biggest challenge I see is a proper testing as I don't see it easy
>>>to set up a local mailman instance. I've got a patch that changes:
>>
>>I suppose we can do some local RPM respins - as long as these changes
>>are small and rare.
>
>That would be great. Should I create a git repo where we'll stack these changes?
>
>> Even with a deadish upstream, distro reporting
>>would be nice, at least at the centos/fedora point (?), as a reference
>>place to stash the patch and get us a bug#.
>
>Can you please do it for me as I don't have any experience with Fedora
>packaging?

If you're volunteering to maintain your patch, perhaps you should try
learning how to do that?

Re: mailman customization

[Christopher Faylor]

On Fri, Apr 03, 2020 at 11:54:20AM -0400, Frank Ch. Eigler wrote:
>> I believe we can quite easily customize mailman 2.1 to match our needs.
>> The biggest challenge I see is a proper testing as I don't see it easy
>> to set up a local mailman instance. I've got a patch that changes:
>
>I suppose we can do some local RPM respins - as long as these changes
>are small and rare.  Even with a deadish upstream, distro reporting
>would be nice, at least at the centos/fedora point (?), as a reference
>place to stash the patch and get us a bug#.

I don't think most of the patch would be acceptable upstream since it
changes default behavior without any way to revert it.