[Bug Infrastructure/29713] New: Please make libc-alpha mailing list compatible with “git am”

[Bug Infrastructure/29713] New: Please make libc-alpha mailing list compatible with “git am”

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

            Bug ID: 29713
           Summary: Please make libc-alpha mailing list compatible with
                    “git am”
           Product: sourceware
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Infrastructure
          Assignee: overseers at sourceware dot org
          Reporter: fweimer at redhat dot com
  Target Milestone: ---

The From: rewriting breaks “git am”, which makes patch-based contribution
workflows unnecessarily difficult. Other Mailman 2 mailing lists (e.g., at
gnu.org or mail.openjdk.org) do not have this problem.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

Frank Ch. Eigler <fche at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |WAITING
                 CC|                            |fche at redhat dot com

--- Comment #1 from Frank Ch. Eigler <fche at redhat dot com> ---
From: rewriting from mailman is done to ensure high reputation for our mail
server, so other mailing systems continue to gladly receive email from us and
reduce the likelihood of false spam/rejection.

Raw messages are available via:

https://inbox.sourceware.org/libc-alpha/

and are suitable for use by "git am".

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

Mark Wielaard <mark at klomp dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |iank at fsf dot org,
                   |                            |mark at klomp dot org

--- Comment #2 from Mark Wielaard <mark at klomp dot org> ---
I would also really like to see this for elfutils-devel.

If we don't modify the message to invalidate the original DKIM signature we
don't have to do From rewriting. But that does mean making sure we don't do
things like To/CC rewriting (the no-duplicates setting), Subject rewriting,
adding footers, HTML scrubbing, etc.

lists.gnu.org also has a mode which allows this. Ian (added to CC) might have
the full setup (they use exim instead of postfix, but should be mostly
identical to our mailman setup otherwise). You can setup your list with
"unmodified messages" (which has the above restrictions) or with From rewriting
(if you use any of the message modification settings).

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #3 from Frank Ch. Eigler <fche at redhat dot com> ---

> If we don't modify the message to invalidate the original DKIM signature we
> don't have to do From rewriting. [...]

That still breaks SPF, which some mailing systems use to detect fraudulent
senders.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fweimer at redhat dot com

--- Comment #4 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Frank Ch. Eigler from comment #3)
> > If we don't modify the message to invalidate the original DKIM signature we
> > don't have to do From rewriting. [...]
> 
> That still breaks SPF, which some mailing systems use to detect fraudulent
> senders.

Mailman puts its bounce processor address into the SMTP envelope MAIL FROM:
(sometimes also called return path), and that's what's being checked by SPF.
There is no way around that, obviously.

The request here is about the user-visible From: header, not the envelope
sender.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #5 from Frank Ch. Eigler <fche at redhat dot com> ---
> The request here is about the user-visible From: header, not the envelope
> sender.

OK, but DMARC then adds From: header alignment checking.

Anyway, mailing list admins are already able to change the relevant mailman2
setting for their own lists.  I would advise against that in isolation
("from_is_list" change "Munge From" to "No"), because other mailing list
processing may well corrupt whatever remaining authentication info is
in the messages.  This would result in drops or rejections & DMARC reports.

There may be recipes for additional mailman2 options to reliably minimize
those risks, and will let you know when/if we find them.

(Note that google-groups does the same From: munging for reflected traffic
for DMARC participant domains.)

In the mean time, the raw mailing list traffic may be processed via
imap, nntp, or http via inbox.sourceware.org.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[iank at fsf dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #6 from iank at fsf dot org ---
I admin the GNU lists. Here is some info about how to accomplish this:

The mailman docs imply that it's From: rewriting is needed for DMARC
compliance, but that isn't the case. Instead, you just ensure that dkim
signatures are not broken for that list. The mailman options needed are
to make the mailman list have no subject modification or footer
addition, and then set dmarc_moderation_action to accept. In mailman web
interface, that under privacy, sender filter.  Also to fix where mailman
breaks dkim signatures by removing CCs using no duplicates user setting,
either use new mailman https://launchpad.net/mailman/+milestone/2.1.30
and set appropriate DEFAULT_DROP_CC described there, or a lightly
patched older mailman.  We use a patch,
https://launchpadlibrarian.net/444501204/nodup-dkim.patch, which is
linked from the relevant bug. After applying the patch, compile mailman
with "pycompile -p mailman /usr/lib/mailman/Mailman -V 2.4-", then
restart. There are a few other rare non-default settings that would
break all dkim signatures. Just send a test message to make sure you
don't have these.


Then, there is the rare case someone sends from a domain with DMARC set
to reject or quarantine and a missing or broken dkim signature.  In that
case, the sender has sent a bad message. These are quite rare, and rare
enough that other big free software lists I've looked at ignore them and
let them get rejected by any recipient which does dmarc validation. On
GNU lists, I decided that wasn't good enough because if the sender just
made a mistake, or just had a misconfiguration, those rejections just go
into the list server mta log and no one notices them, and then only some
subscribers will get their message, leading to confusion. It is better
to either reject the message from the poster, telling them the problem
in the reject message, or be even nicer and do from rewriting in the mta
because their message may still be useful even with a rewritten
from. Then, if the rewriting causes a problem, a human can tell them to
fix the issue on their end. That is what we do, it is documented here,
https://wiki.debian.org/Exim#For_running_a_mailing_list_and_ensuring_all_sent_mail_is_DMARC_compliant
. Sourcware uses postfix, so that is not a drop in solution, and is
probably much easier to setup rejecting of the message.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[iank at fsf dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #7 from iank at fsf dot org ---
The other main setting is from_is_list in the main list settings, of
course that needs to be set to no.

Minor correction to my previous comment, I was talking about
lists.gnu.org, of course some GNU lists exist elsewhere like here.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[cgf at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

Christopher G. Faylor <cgf at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |cgf at sourceware dot org

--- Comment #8 from Christopher G. Faylor <cgf at sourceware dot org> ---
I can take a look at this in the next few days, if no one beats me to it.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[iank at fsf dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #9 from iank at fsf dot org ---
Also note: the compile command I mentioned has a path specific to a
debian based mailman packages.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[cgf at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

Christopher G. Faylor <cgf at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |NEW

--- Comment #10 from Christopher G. Faylor <cgf at sourceware dot org> ---
Actually, except for the patching issue, it looks like mailman list
administrators can make the changes themselves.  Is anyone going to try this?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #11 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Christopher G. Faylor from comment #10)
> Actually, except for the patching issue, it looks like mailman list
> administrators can make the changes themselves.  Is anyone going to try this?

Looking at https://launchpadlibrarian.net/444501204/nodup-dkim.patch, I think
this patch is absolutely required: Like many Mailman 2 installations,
sourceware defaults “Avoid duplicate copies of messages?” to Yes, and users can
edit anyway. As a result, most subscribers will receive rewritten messages
which fail DKIM checks. I believe this patch disables that.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

Frank Ch. Eigler <fche at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |WAITING

--- Comment #12 from Frank Ch. Eigler <fche at redhat dot com> ---
I rebuilt the mailman 2.1.29 package from rhel8-modules with that patch, and
installed as mailman-3:2.1.29-12.el8.sourceware.3.x86_64.  We may have to do it
again later, so left /root/rpmbuild with all the artifact to redo this work
later, should later distro updates nuke this patch.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #13 from Mark Wielaard <mark at klomp dot org> ---
I tried setting up test-list@sourceware.org according to these settings. But
the dmarc_moderation_action for privacy/sender cannot be set to "Accept" it
will revert to "Munge From" and an error is generated:
"Error: dmarc_moderation_action must be >= the configured default value."

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[iank at fsf dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #14 from iank at fsf dot org ---
Yes, I forgot about this. Mailman refuses to
let list owners change the setting to what you are
trying if you've set a site-wide/system setting. To be
specific, on debian based systems there is

/usr/lib/mailman/Mailman/Defaults.py

which are the default mailman settings, it has

DEFAULT_DMARC_MODERATION_ACTION = 0

And the system mailman config file overrides the defaults. 
On debian based systems, that file is 

/etc/mailman/mm_cfg.py

I think the issue is that you've set DEFAULT_DMARC_MODERATION_ACTION to
something other than 0 in your system config.
So, remove it it to 0 to overcome this issue.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[iank at fsf dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #15 from iank at fsf dot org ---
correction: remove it or set it to 0

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #16 from Frank Ch. Eigler <fche at redhat dot com> ---
(In reply to iank from comment #15)
> correction: remove it or set it to 0

I set /etc/mailman/mm_cfg.py DEFAULT_DMARC_MODERATION_ACTION to 0.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #17 from Mark Wielaard <mark at klomp dot org> ---
I believe test-list@sourceware.org is now setup so that it keeps dkim
signatures valid when sent through that list keeping the From header. But it
would be good if people could test it by subscribing and/or sending with
addresses that use a strict dmarc policy to see if we do get unexpected
rewrites which cause bounces.

https://sourceware.org/mailman/listinfo/test-list

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #18 from Mark Wielaard <mark at klomp dot org> ---
(In reply to iank from comment #6)
> There are a few other rare non-default settings that would
> break all dkim signatures.

One of these is stripping text/html attachments, which is done by default on
our install under Content filtering, filter_mime_types. For test-list I set
filter_content to No.

It would be good if there was a way to reject emails containing text/html.

Also public-inbox won't archive them (which is already a problem, because inbox
sees the messages before the text/html part is stripped).

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[joseph at codesourcery dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #19 from joseph at codesourcery dot com <joseph at codesourcery dot com> ---
On Mon, 31 Oct 2022, mark at klomp dot org via Overseers wrote:

> It would be good if there was a way to reject emails containing text/html.

At least for some lists, accepting them is deliberate, to avoid losing 
contributions by placing requirement on email formatting that are 
problematic for many users nowadays.

> Also public-inbox won't archive them (which is already a problem, 
> because inbox sees the messages before the text/html part is stripped).

That's clearly a bug in our public-inbox configuration.  Since we're 
deliberate accepting text/html we should set up our archiving to handle 
messages using it.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #20 from Mark Wielaard <mark at klomp dot org> ---
(In reply to joseph@codesourcery.com from comment #19)
> On Mon, 31 Oct 2022, mark at klomp dot org via Overseers wrote:
> 
> > It would be good if there was a way to reject emails containing text/html.
> 
> At least for some lists, accepting them is deliberate, to avoid losing 
> contributions by placing requirement on email formatting that are 
> problematic for many users nowadays.

Sad, but true. So if we want to accept them and keep DKIM signatures correct
then we'll need to turn off content filtering, because stripping text/html
parts or turning them into plain text will alter the message body.

> > Also public-inbox won't archive them (which is already a problem, 
> > because inbox sees the messages before the text/html part is stripped).
> 
> That's clearly a bug in our public-inbox configuration.  Since we're 
> deliberate accepting text/html we should set up our archiving to handle 
> messages using it.

Lets open a separate bug for that. This is a design choice of public-inbox:

  To discourage phishing, trackers, exploits and other nuisances,
  only plain-text emails are allowed and HTML is rejected by default.
  This improves accessibility, and saves bandwidth and storage
  as mail is archived forever.

We have other archives of course, the messages are in the mailman archives, but
it would be good to see what we can do for the public-inbox archives. Will
discuss with the upstream project.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #21 from Mark Wielaard <mark at klomp dot org> ---
(In reply to Mark Wielaard from comment #17)
> I believe test-list@sourceware.org is now setup so that it keeps dkim
> signatures valid when sent through that list keeping the From header. But it
> would be good if people could test it by subscribing and/or sending with
> addresses that use a strict dmarc policy to see if we do get unexpected
> rewrites which cause bounces.
> 
> https://sourceware.org/mailman/listinfo/test-list

So it almost works, but almost of course is not good enough because it means
some messages do arrive with broken dkim verification.

Specifically it seems that the Content-Transfer-Encoding of attachments are
changed from 8bit to quoted-printable for the text/html and to base64 for
text/plain parts.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[iank at fsf dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #22 from iank at fsf dot org ---
I looked closer at list settings I did on lists.gnu.org and the
I found one more:

Content Filter, filter_content = No

How is that set on the test list?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #23 from Mark Wielaard <mark at klomp dot org> ---
filter_content is set to No.

But looking around it seems 8bit content transfer encodings are problematic
with DKIM signing in general. The spec says before signing a message 8bit
content should be transformed to a 7bit encoding so the signature survives if
any transport doesn't support the 8bit encoding. So it could be said this is
just an issue with the mimecast dkim signing in the above case, since it
doesn't know that the full transport to the end user is 8bit clean, it should
have transformed to a 7bit encoding itself before signing.

There is a postfix setting that seems to try to keep the 8bit content transfer
encoding even if a transport doesn't explicitly say it supports 8bit encodings:

http://www.postfix.org/postconf.5.html#disable_mime_output_conversion

   Disable the conversion of 8BITMIME format to 7BIT format. Mime output
conversion is needed when the destination does not advertise 8BITMIME support. 

This is recommended for postfix filters and milters to avoid breaking DKIM etc.
signatures:
https://www.postfix.org/MILTER_README.html#workarounds
https://www.postfix.org/FILTER_README.html#advanced_filter

So we probably just need to set disable_mime_output_conversion=yes and hope
that any further transports do actually support 8BITMIME.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #24 from Frank Ch. Eigler <fche at redhat dot com> ---
added that flag to /etc/pstfix/main.cf

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[iank at fsf dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #25 from iank at fsf dot org ---
> almost of course is not good enough because it means some messages do
> arrive with broken dkim verification

It is ok that some messages come in with broken dkim signatures. Those
domains are not strict DMARC domains.

> 8bit content transfer encodings are problematic with DKIM signing in general

I don't think so. https://cr.yp.to/smtp/8bitmime.html was written in the
90s and suggests 7 bit only MTA's are not a thing that exist on the
internet anymore, so the DKIM spec suggesting conversion is not based on
reality, it is only there to conform to really old RFCs.

Exim doesn't support any conversion to 7 bit. The setting you added to
postfix sounds right.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #26 from Mark Wielaard <mark at klomp dot org> ---
(In reply to iank from comment #25)
> > 8bit content transfer encodings are problematic with DKIM signing in general
> 
> I don't think so. https://cr.yp.to/smtp/8bitmime.html was written in the
> 90s and suggests 7 bit only MTA's are not a thing that exist on the
> internet anymore, so the DKIM spec suggesting conversion is not based on
> reality, it is only there to conform to really old RFCs.
> 
> Exim doesn't support any conversion to 7 bit. The setting you added to
> postfix sounds right.

The setting does indeed help, no more conversions of the attachments. It is
slightly odd that this isn't the default of postfix. By default postfix goes
out of its way to "downgrade" things to 7bit encodings. But that does indeed
not seem to be needed.

Everything looks correct now, unless someone with the nodups flag is on CC
(then the CC is altered and the DKIM signature fails).

So the last remaining issue is the AvoidDuplicates. It looks like the
mailman-nodup-dkim.patch from Comment 11 isn't yet installed. At least our
install of /usr/lib/mailman/Mailman/Handlers/AvoidDuplicates.py isn't patched.

I might misunderstand Comment 12. Do we have the package, but it isn't
installed yet?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #27 from Frank Ch. Eigler <fche at redhat dot com> ---
> It looks like the mailman-nodup-dkim.patch from Comment 11 isn't yet installed.

It was installed.  A later distro update package replaced it.  This is the
problem with de facto forking the package with a local patch.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #28 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Frank Ch. Eigler from comment #27)
> > It looks like the mailman-nodup-dkim.patch from Comment 11 isn't yet installed.
> 
> It was installed.  A later distro update package replaced it.  This is the
> problem with de facto forking the package with a local patch.

Would it make sense to get this patch applied in the Red Hat Enterprise Linux
mailman package?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #29 from Mark Wielaard <mark at klomp dot org> ---
(In reply to Florian Weimer from comment #28)
> Would it make sense to get this patch applied in the Red Hat Enterprise
> Linux mailman package?

O, yes, that would be great. I assumed it was an orphaned EPEL package. But if
it is a RHEL package then having this patch applied there would be very good.

For now it has been fixed by using a "higher" version for the local build
mailman-2.1.29-12.sourceware.el8.3.x86_64

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #30 from Florian Weimer <fweimer at redhat dot com> ---
I filed a bug: https://bugzilla.redhat.com/show_bug.cgi?id=2140631

It may make sense to open a Customer Portal ticket as well (and link it from
the bug) if you have a supported subscription. Thanks.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #31 from Mark Wielaard <mark at klomp dot org> ---
We really need some people with strict dmarc policies and/or who use dkim
signatures to subscribe to the test-list to see if all settings are correct
now.

One remaining problem is what I believe Ian described in comment #6, people who
sign dkim messages in a way that we cannot avoid to break. e.g. when people
include List-* or Sender headers in their dkim signature. I am unsure what to
do about this. Should we reject such messages? It seems that is signaling they
don't want their messages on a mailinglist or forwarded by a different sender
because they "protected" those headers. But maybe it isn't done deliberately?

Frank your domain, elastic.org, seems to be configured to sign (empty) list
headers
(Sender:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive)
on all your messages. How would you like us to handle that?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #32 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Mark Wielaard from comment #31)
> We really need some people with strict dmarc policies and/or who use dkim
> signatures to subscribe to the test-list to see if all settings are correct
> now.

Fangrui Song sent a couple of test messages:

https://sourceware.org/pipermail/test-list/2022/thread.html

They made it through correctly and back into Gmail, as far as I can see. I
believe google.com has a rather restricted DMARC policy, and Gmail enforces it.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #33 from Frank Ch. Eigler <fche at redhat dot com> ---
(In reply to Mark Wielaard from comment #31)
> Frank your domain, elastic.org, seems to be configured to sign (empty) list
> headers
> (Sender:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-
> Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-
> Owner:List-Archive) on all your messages. How would you like us to handle
> that?

This must be the default for the exim MTA in use here.  It goes to show how
fragile this sort of forwarding is - not sure you or I can do anything about
it.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #34 from Mark Wielaard <mark at klomp dot org> ---
(In reply to Frank Ch. Eigler from comment #33)
> (In reply to Mark Wielaard from comment #31)
> > Frank your domain, elastic.org, seems to be configured to sign (empty) list
> > headers
> > (Sender:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-
> > Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-
> > Owner:List-Archive) on all your messages. How would you like us to handle
> > that?
> 
> This must be the default for the exim MTA in use here.  It goes to show how
> fragile this sort of forwarding is - not sure you or I can do anything about
> it.

This is somewhat disappointing since this seems against the spec. These headers
shouldn't be included by default if they aren't set on the original outgoing
message. Is there a page we can point people to that explains why these
defaults are wrong and how to properly setup the exim MTA to do dkim signing?

If I understand Ian's comment #6 these misconfigured dkim defaults are not
common and we could just ignore them. People will just get periodically
unsubscribed till they fix their dkim setup to not include empty Sender or
List-* headers.

But it would be nice to apply From rewriting to help such misconfigurations.
There is a recipy for Exim to do that in comment #6. I don't know if that can
easily translate to postfix.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #35 from Frank Ch. Eigler <fche at redhat dot com> ---
> This is somewhat disappointing since this seems against the spec. These
> headers shouldn't be included by default if they aren't set on the original
> outgoing message.

The RFC4871 standard allows more and non-existent headers to be signed.  I
suppose one argument is that signing non-existent headers protects email
against later insertion of such headers.  Characterizing this default as
"wrong" or a "misconfiguration" in need of a "fix" is a little too far.  They
are inconvenient for the efforts associated with this PR, that's all.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #36 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Frank Ch. Eigler from comment #35)
> > This is somewhat disappointing since this seems against the spec. These
> > headers shouldn't be included by default if they aren't set on the original
> > outgoing message.
> 
> The RFC4871 standard allows more and non-existent headers to be signed.  I
> suppose one argument is that signing non-existent headers protects email
> against later insertion of such headers.  Characterizing this default as
> "wrong" or a "misconfiguration" in need of a "fix" is a little too far. 
> They are inconvenient for the efforts associated with this PR, that's all.

Asserting the non-existence of these headers is what breaks DKIM signatures
with mailing lists. I reported this upstream-upstream here:

  Signed header defaults break mailing lists
  <https://github.com/duncanthrax/pdkim/issues/4>

If Exim (or pdkim really) are unwilling to fix this, Mailman (or something in
front of Mailman) needs to be taught to strip such anti-mailing-list DKIM
signatures.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #37 from Mark Wielaard <mark at klomp dot org> ---
(In reply to Florian Weimer from comment #36)
> Asserting the non-existence of these headers is what breaks DKIM signatures
> with mailing lists. I reported this upstream-upstream here:
> 
>   Signed header defaults break mailing lists
>   <https://github.com/duncanthrax/pdkim/issues/4>

Thanks. There is also a Exim and Debian bug for this:
https://bugs.exim.org/show_bug.cgi?id=2394
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=939808

It looks like a workaround is to not rely on the default exim setting of the
dkim sign headers, but explicitly set it to:

DKIM_SIGN_HEADERS =
From:From:=Sender:Reply-To:Subject:Subject:Date:To:Cc:Message-ID:In-Reply-To:References:MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID:Content-Description:=Resent-Date:=Resent-From:=Resent-Sender:=Resent-To:=Resent-Cc:=Resent-Message-ID:=List-Id:=List-Help:=List-Unsubscribe:=List-Subscribe:=List-Post:=List-Owner:=List-Archive"

> If Exim (or pdkim really) are unwilling to fix this, Mailman (or something
> in front of Mailman) needs to be taught to strip such anti-mailing-list DKIM
> signatures.

The problem with getting this fixed seems to be that some people are convinced
dmarc and dkim will just break mailinglists always, so they don't see a point
in fixing the defaults.

Stripping can only be done if the dmarc policy for the domain doesn't require
DKIM signatures, otherwise we will either have to bounce them before they reach
the list or do From munging anyway (as suggested in comment #6).

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[iank at fsf dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #38 from iank at fsf dot org ---
Just general fyi: My exim configuration is
DKIM_SIGN_HEADERS = mime-version:in-reply-to:references:from:date:subject:to
I got that from looking at the headers that gmail signs.
What you posted seems excessive, but should still work.

I think exim basically tries to follow the RFCs in it's defaults
even when they don't make any practical sense, and the best change
you might get exim upstream to accept is to add a note to the
documentation.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #39 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Mark Wielaard from comment #37)
> (In reply to Florian Weimer from comment #36)
> > Asserting the non-existence of these headers is what breaks DKIM signatures
> > with mailing lists. I reported this upstream-upstream here:
> > 
> >   Signed header defaults break mailing lists
> >   <https://github.com/duncanthrax/pdkim/issues/4>
> 
> Thanks. There is also a Exim and Debian bug for this:
> https://bugs.exim.org/show_bug.cgi?id=2394
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=939808
> 
> It looks like a workaround is to not rely on the default exim setting of the
> dkim sign headers, but explicitly set it to:
> 
> DKIM_SIGN_HEADERS =
> From:From:=Sender:Reply-To:Subject:Subject:Date:To:Cc:Message-ID:In-Reply-To:
> References:MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID:
> Content-Description:=Resent-Date:=Resent-From:=Resent-Sender:=Resent-To:
> =Resent-Cc:=Resent-Message-ID:=List-Id:=List-Help:=List-Unsubscribe:=List-
> Subscribe:=List-Post:=List-Owner:=List-Archive"

“=” for the mailing list headers disables nested mailing lists (or forwarding
mail to lists with “S D r” in Gnus, “b” in mutt). It should be much more
compatible with the default.

> > If Exim (or pdkim really) are unwilling to fix this, Mailman (or something
> > in front of Mailman) needs to be taught to strip such anti-mailing-list DKIM
> > signatures.
> 
> The problem with getting this fixed seems to be that some people are
> convinced dmarc and dkim will just break mailinglists always, so they don't
> see a point in fixing the defaults.

Some of them even have the power to change the defaults. It might be tricky to
convince them to make the change that proves them wrong.

> Stripping can only be done if the dmarc policy for the domain doesn't
> require DKIM signatures, otherwise we will either have to bounce them before
> they reach the list or do From munging anyway (as suggested in comment #6).

Right, it's non-trivial logic, and not going to be implemented for Mailman 2.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #40 from Mark Wielaard <mark at klomp dot org> ---
Note that c8s seems to have been updated to mailman 2.1.30
https://gitlab.com/redhat/centos-stream/rpms/mailman/-/commits/stream-mailman-2.1-rhel-8.9.0
https://bugzilla.redhat.com/show_bug.cgi?id=2140631

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #41 from Mark Wielaard <mark at klomp dot org> ---
(In reply to Mark Wielaard from comment #40)
> Note that c8s seems to have been updated to mailman 2.1.30
> https://gitlab.com/redhat/centos-stream/rpms/mailman/-/commits/stream-
> mailman-2.1-rhel-8.9.0
> https://bugzilla.redhat.com/show_bug.cgi?id=2140631

Package available at
https://kojihub.stream.centos.org/koji/buildinfo?buildID=35377

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #42 from Mark Wielaard <mark at klomp dot org> ---
(In reply to Mark Wielaard from comment #41)
> (In reply to Mark Wielaard from comment #40)
> > Note that c8s seems to have been updated to mailman 2.1.30
> > https://gitlab.com/redhat/centos-stream/rpms/mailman/-/commits/stream-
> > mailman-2.1-rhel-8.9.0
> > https://bugzilla.redhat.com/show_bug.cgi?id=2140631
> 
> Package available at
> https://kojihub.stream.centos.org/koji/buildinfo?buildID=35377

I went through the diff and it contains the patch we use in
mailman-2.1.29-12.sourceware.el8.3.x86_64 It also contains a simpler way to
prevent dropping CCs:

+# The process which avoids sending a list copy of a message to a member who
+# is also directly addressed in To: or Cc: can drop the address from Cc: to
+# avoid growing a long Cc: list in long threads.  This can be undesirable as
+# it can break DKIM signatures and possibly cause confusion.  To avoid changes
+# to Cc: headers, set the list's drop_cc to No.
+DEFAULT_DROP_CC = Yes

We should set that to No.

There is also a new dmarc_moderation_addresses:

+            ('dmarc_moderation_addresses', mm_cfg.EmailListEx, (10, WIDTH), 1,
+             _("""List of addresses (or regexps) whose posts should always
apply
+             <a href="?VARHELP=privacy/sender/dmarc_moderation_action"
+             >dmarc_moderation_action</a> 
+             regardless of any domain specific DMARC Policy."""),
+
+             _("""Postings from any of these addresses will automatically
+             apply any DMARC action mitigation.  This can be utilized to
+             automatically wrap or munge postings from known addresses or
+             domains that might have policies rejecting external mail From:
+             themselves.

We could use this for addresses that add any list headers to their own DKIM
signatures.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #43 from Mark Wielaard <mark at klomp dot org> ---
We now run 2.1.30 with DEFAULT_DROP_CC = No

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #44 from Mark Wielaard <mark at klomp dot org> ---
I analyzed the ~7000 messages from this year to libc-alpha.
Only one legitimate email had signed (empty) List headers (the other was spam).
The email is from a one time contributor.

patchwork did capture the patch:
https://patchwork.sourceware.org/project/glibc/patch/20230119034258.15881-1-kilobyte@angband.pl/
But it was never reviewed (maybe because people had bounced that email?)

Anyway, my conclusion is that in general libc-alpha contributors don't use dkim
signing of list headers.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #45 from Mark Wielaard <mark at klomp dot org> ---
So to make this work we have to make sure none of the list settings change the
email body, subject or any other header that is likely covered by a dkim
signature. Which means the following settings (see also Comment #6, Comment #7,
Comment #18 and Comment 22):

- subject_prefix (general): (empty)
- from_is_list (general): No
- anonymous_list (general): No
- first_strip_reply_to (general): No
- reply_goes_to_list (general): Poster
- reply_to_address (general): (empty)
- include_sender_header (general): No
- drop_cc (general): No
- msg_header (nondigest): (empty)
- msg_footer (nondigest): (empty)
- scrub_nondigest (nondigest): No
- dmarc_moderation_action (privacy): Accept
- filter_content (contentfilter): No

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

Mark Wielaard <mark at klomp dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|---                         |FIXED

--- Comment #46 from Mark Wielaard <mark at klomp dot org> ---
This is now used by libc-alpha, gcc-patches, libstdc++, jit, fortran, gcc-rust,
newlib, gdb-patches, binutils, elfutils, libabigail.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29713] Please make libc-alpha mailing list compatible with “git am”

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29713

--- Comment #47 from Florian Weimer <fweimer at redhat dot com> ---
Thank you so much!

-- 
You are receiving this mail because:
You are the assignee for the bug.