* Not usable email content encoding @ 2020-03-16 13:45 Martin Liška 2020-03-16 13:47 ` Richard Earnshaw ` (3 more replies) 0 siblings, 4 replies; 76+ messages in thread From: Martin Liška @ 2020-03-16 13:45 UTC (permalink / raw) To: overseers; +Cc: GCC Development Hello. I noticed some emails reaching gcc-patches@gcc.gnu.org use the following quoting: ``` case UNGT_EXPR: case UNGE_EXPR: case UNEQ_EXPR: + case MEM_REF: /* Binary operations evaluating both arguments (increment and =09 decrement are binary internally in GCC). */ orig_op0 =3D op0 =3D TREE_OPERAND (expr, 0); @@ -435,6 +436,14 @@ c_fully_fold_internal (tree expr, bool i =09 || TREE_CODE (TREE_TYPE (orig_op0)) =3D=3D FIXED_POINT_TYPE) =09 && TREE_CODE (TREE_TYPE (orig_op1)) =3D=3D INTEGER_TYPE) =09warn_for_div_by_zero (loc, op1); + if (code =3D=3D MEM_REF +=09 && ret !=3D expr +=09 && TREE_CODE (ret) =3D=3D MEM_REF) +=09{ +=09 TREE_READONLY (ret) =3D TREE_READONLY (expr); +=09 TREE_SIDE_EFFECTS (ret) =3D TREE_SIDE_EFFECTS (expr); +=09 TREE_THIS_VOLATILE (ret) =3D TREE_THIS_VOLATILE (expr); +=09} ``` It's probably related to the following email tag: Content-Transfer-Encoding: quoted-printable The format is problematic when copying a patch. Email example: https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542053.html Thank you, Martin ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-16 13:45 Not usable email content encoding Martin Liška @ 2020-03-16 13:47 ` Richard Earnshaw 2020-03-16 13:52 ` Martin Liška 2020-03-16 13:51 ` Florian Weimer ` (2 subsequent siblings) 3 siblings, 1 reply; 76+ messages in thread From: Richard Earnshaw @ 2020-03-16 13:47 UTC (permalink / raw) To: Martin Liška, overseers; +Cc: GCC Development On 16/03/2020 13:45, Martin Liška wrote: > Hello. > > I noticed some emails reaching gcc-patches@gcc.gnu.org use the following > quoting: > > ``` > case UNGT_EXPR: > case UNGE_EXPR: > case UNEQ_EXPR: > + case MEM_REF: > /* Binary operations evaluating both arguments (increment and > =09 decrement are binary internally in GCC). */ > orig_op0 =3D op0 =3D TREE_OPERAND (expr, 0); > @@ -435,6 +436,14 @@ c_fully_fold_internal (tree expr, bool i > =09 || TREE_CODE (TREE_TYPE (orig_op0)) =3D=3D FIXED_POINT_TYPE) > =09 && TREE_CODE (TREE_TYPE (orig_op1)) =3D=3D INTEGER_TYPE) > =09warn_for_div_by_zero (loc, op1); > + if (code =3D=3D MEM_REF > +=09 && ret !=3D expr > +=09 && TREE_CODE (ret) =3D=3D MEM_REF) > +=09{ > +=09 TREE_READONLY (ret) =3D TREE_READONLY (expr); > +=09 TREE_SIDE_EFFECTS (ret) =3D TREE_SIDE_EFFECTS (expr); > +=09 TREE_THIS_VOLATILE (ret) =3D TREE_THIS_VOLATILE (expr); > +=09} > ``` > > It's probably related to the following email tag: > Content-Transfer-Encoding: quoted-printable > > The format is problematic when copying a patch. > Email example: > https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542053.html > > Thank you, > Martin This isn't new. It's usually the sender that controls that, though it's often automatic in their email client. R. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-16 13:47 ` Richard Earnshaw @ 2020-03-16 13:52 ` Martin Liška 0 siblings, 0 replies; 76+ messages in thread From: Martin Liška @ 2020-03-16 13:52 UTC (permalink / raw) To: Richard Earnshaw, overseers; +Cc: GCC Development On 3/16/20 2:47 PM, Richard Earnshaw wrote: > This isn't new. It's usually the sender that controls that, though it's often automatic in their email client. Jakub told me that he sees: Ten mail byl: Content-Type: text/plain; charset=us-ascii Content-Disposition: inline while the email reaching the mailing list has: Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Martin ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-16 13:45 Not usable email content encoding Martin Liška 2020-03-16 13:47 ` Richard Earnshaw @ 2020-03-16 13:51 ` Florian Weimer 2020-03-16 13:52 ` Frank Ch. Eigler 2020-03-16 13:54 ` Alexander Monakov 3 siblings, 0 replies; 76+ messages in thread From: Florian Weimer @ 2020-03-16 13:51 UTC (permalink / raw) To: Martin Liška; +Cc: overseers, GCC Development * Martin Liška: > It's probably related to the following email tag: > Content-Transfer-Encoding: quoted-printable > > The format is problematic when copying a patch. > Email example: > https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542053.html This is a known issue in the redhat.com email infrastructure. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-16 13:45 Not usable email content encoding Martin Liška 2020-03-16 13:47 ` Richard Earnshaw 2020-03-16 13:51 ` Florian Weimer @ 2020-03-16 13:52 ` Frank Ch. Eigler 2020-03-16 13:54 ` Alexander Monakov 3 siblings, 0 replies; 76+ messages in thread From: Frank Ch. Eigler @ 2020-03-16 13:52 UTC (permalink / raw) To: Overseers mailing list; +Cc: overseers, Martin Liška, GCC Development Hi - > I noticed some emails reaching gcc-patches@gcc.gnu.org use the following quoting: > It's probably related to the following email tag: > Content-Transfer-Encoding: quoted-printable This is not something that the mailing list system does. This content-transfer-encoding comes from the original email (as evident in the /var/lib/mailman/archives/private/gcc-patches.mbox/gcc-patches.mbox file). - FChE ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-16 13:45 Not usable email content encoding Martin Liška ` (2 preceding siblings ...) 2020-03-16 13:52 ` Frank Ch. Eigler @ 2020-03-16 13:54 ` Alexander Monakov 2020-03-16 14:11 ` Martin Liška 2020-03-17 19:46 ` Segher Boessenkool 3 siblings, 2 replies; 76+ messages in thread From: Alexander Monakov @ 2020-03-16 13:54 UTC (permalink / raw) To: Martin Liška; +Cc: overseers, GCC Development On Mon, 16 Mar 2020, Martin Liška wrote: > It's probably related to the following email tag: > Content-Transfer-Encoding: quoted-printable > > The format is problematic when copying a patch. > Email example: > https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542053.html I'm surprised it's an issue for you: normally your email client would transform quoted-printable and copying would do the right thing (i.e. select actual patch contents, without whitespace munging). Are you trying to copy from the raw message representation? Alexander ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-16 13:54 ` Alexander Monakov @ 2020-03-16 14:11 ` Martin Liška 2020-03-16 14:18 ` Jonathan Wakely 2020-03-17 19:46 ` Segher Boessenkool 1 sibling, 1 reply; 76+ messages in thread From: Martin Liška @ 2020-03-16 14:11 UTC (permalink / raw) To: Alexander Monakov; +Cc: overseers, GCC Development On 3/16/20 2:54 PM, Alexander Monakov wrote: > Are you trying to copy from the raw message representation? Exactly. Martin ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-16 14:11 ` Martin Liška @ 2020-03-16 14:18 ` Jonathan Wakely 0 siblings, 0 replies; 76+ messages in thread From: Jonathan Wakely @ 2020-03-16 14:18 UTC (permalink / raw) To: Martin Liška; +Cc: Alexander Monakov, overseers, GCC Development On Mon, 16 Mar 2020 at 14:13, Martin Liška wrote: > > On 3/16/20 2:54 PM, Alexander Monakov wrote: > > Are you trying to copy from the raw message representation? > > Exactly. That's never been reliable. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-16 13:54 ` Alexander Monakov 2020-03-16 14:11 ` Martin Liška @ 2020-03-17 19:46 ` Segher Boessenkool 2020-03-17 19:51 ` Frank Ch. Eigler 2020-03-21 22:29 ` Maciej W. Rozycki 1 sibling, 2 replies; 76+ messages in thread From: Segher Boessenkool @ 2020-03-17 19:46 UTC (permalink / raw) To: Alexander Monakov; +Cc: Martin Liška, overseers, GCC Development On Mon, Mar 16, 2020 at 04:54:51PM +0300, Alexander Monakov via Gcc wrote: > On Mon, 16 Mar 2020, Martin Liška wrote: > > > It's probably related to the following email tag: > > Content-Transfer-Encoding: quoted-printable > > > > The format is problematic when copying a patch. > > Email example: > > https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542053.html > > I'm surprised it's an issue for you: normally your email client > would transform quoted-printable and copying would do the right thing > (i.e. select actual patch contents, without whitespace munging). > > Are you trying to copy from the raw message representation? Everyone trying to work with a patch (instead of just the email) always is working with the raw message. Just patch < mbox or git-am mbox for example. https://gcc.gnu.org/contribute.html says It is strongly discouraged to post patches as MIME parts of type application/whatever, disposition attachment or encoded as base64 or quoted-printable. (which many people still do not follow, making reviewing their patches much harder than needed). Segher ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-17 19:46 ` Segher Boessenkool @ 2020-03-17 19:51 ` Frank Ch. Eigler 2020-03-17 21:45 ` Segher Boessenkool 2020-03-18 10:38 ` Florian Weimer 2020-03-21 22:29 ` Maciej W. Rozycki 1 sibling, 2 replies; 76+ messages in thread From: Frank Ch. Eigler @ 2020-03-17 19:51 UTC (permalink / raw) To: Overseers mailing list Cc: Alexander Monakov, Segher Boessenkool, overseers, GCC Development, Martin Liška Hi - > > Are you trying to copy from the raw message representation? > > Everyone trying to work with a patch (instead of just the email) always > is working with the raw message. Just patch < mbox or git-am mbox > for example. > > https://gcc.gnu.org/contribute.html says > It is strongly discouraged to post patches as MIME parts of type > application/whatever, disposition attachment or encoded as base64 or > quoted-printable. > > (which many people still do not follow, making reviewing their patches > much harder than needed). The key here is to realize that the raw message is not what you get back from the mailing list reflector, and also not the raw message that was sent by the sender. In this day of mta intermediaries, proxies, reflectors, it may be time to revisit that suggestion. Attachments seem to survive unmolested better than mail email bodies, for example. And MUAs can undo some of the unexpected transformations in the main body. - FChE ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-17 19:51 ` Frank Ch. Eigler @ 2020-03-17 21:45 ` Segher Boessenkool 2020-03-18 10:38 ` Florian Weimer 1 sibling, 0 replies; 76+ messages in thread From: Segher Boessenkool @ 2020-03-17 21:45 UTC (permalink / raw) To: Frank Ch. Eigler Cc: Overseers mailing list, overseers, GCC Development, Alexander Monakov Hi! On Tue, Mar 17, 2020 at 03:51:58PM -0400, Frank Ch. Eigler via Gcc wrote: > > > Are you trying to copy from the raw message representation? > > > > Everyone trying to work with a patch (instead of just the email) always > > is working with the raw message. Just patch < mbox or git-am mbox > > for example. > > > > https://gcc.gnu.org/contribute.html says > > It is strongly discouraged to post patches as MIME parts of type > > application/whatever, disposition attachment or encoded as base64 or > > quoted-printable. > > > > (which many people still do not follow, making reviewing their patches > > much harder than needed). > > The key here is to realize that the raw message is not what you get > back from the mailing list reflector, Really? It does not change any of this, afaik? It never was like that before, and if it now is, that makes this whole mailman experiment a failure, imnsho. > and also not the raw message > that was sent by the sender. In this day of mta intermediaries, > proxies, reflectors, it may be time to revisit that suggestion. Not really, no. Any encoding is bad for most workflows. MIME types application/* or */x-* do not belong on mailing lists, by definition. And attachments are (or should be!) just a last resort option to get your patches through unmolested (and even then it does not always work for everyone). Of course most people use attachments with a bad MIME type or disposition, making it harder to work with still. > Attachments seem to survive unmolested better than mail email bodies, > for example. And MUAs can undo some of the unexpected transformations > in the main body. Yes, but not most. People doing Free Software just need to learn how to use the various tools involved correctly and effectively. Email is one such tool. There is no better one yet (all of the other code review options and discussion platforms have much bigger downsides than upsides). We have been using this since forever, too, and we know how it works (*including* this encoding/disposition/etc. problem, which has been around for decades). Segher ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-17 19:51 ` Frank Ch. Eigler 2020-03-17 21:45 ` Segher Boessenkool @ 2020-03-18 10:38 ` Florian Weimer 2020-03-18 11:01 ` Frank Ch. Eigler 1 sibling, 1 reply; 76+ messages in thread From: Florian Weimer @ 2020-03-18 10:38 UTC (permalink / raw) To: Frank Ch. Eigler via Gcc Cc: Overseers mailing list, Frank Ch. Eigler, overseers, Alexander Monakov, Segher Boessenkool * Frank Ch. Eigler via Gcc: >> > Are you trying to copy from the raw message representation? >> >> Everyone trying to work with a patch (instead of just the email) always >> is working with the raw message. Just patch < mbox or git-am mbox >> for example. >> >> https://gcc.gnu.org/contribute.html says >> It is strongly discouraged to post patches as MIME parts of type >> application/whatever, disposition attachment or encoded as base64 or >> quoted-printable. >> >> (which many people still do not follow, making reviewing their patches >> much harder than needed). > > The key here is to realize that the raw message is not what you get > back from the mailing list reflector, and also not the raw message > that was sent by the sender. In this day of mta intermediaries, > proxies, reflectors, it may be time to revisit that suggestion. But these largely are new problems. It used to work flawlessly. Patch reencoding problems go back to the redhat.com changes last November (I understand the responsible vendor is working on a fix, but I'm not up-to-date on the current developments). Since the sourceware.org Mailman migration, the From: header is being rewritten, without any compelling reason. I certainly do not do any DMARC checking here, so the rewriting does not benefit me. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 10:38 ` Florian Weimer @ 2020-03-18 11:01 ` Frank Ch. Eigler 2020-03-18 11:39 ` Florian Weimer 2020-03-18 14:19 ` Michael Matz 0 siblings, 2 replies; 76+ messages in thread From: Frank Ch. Eigler @ 2020-03-18 11:01 UTC (permalink / raw) To: Overseers mailing list Cc: Frank Ch. Eigler via Gcc, Florian Weimer, overseers, Alexander Monakov, Segher Boessenkool Hi - > > The key here is to realize that the raw message is not what you get > > back from the mailing list reflector, and also not the raw message > > that was sent by the sender. In this day of mta intermediaries, > > proxies, reflectors, it may be time to revisit that suggestion. > > But these largely are new problems. It used to work flawlessly. I understand that's frustrating. But these workflows were counting on literally unspecified behaviours not changing, or outright standards violations continuing. > Patch reencoding problems go back to the redhat.com changes last > November (I understand the responsible vendor is working on a fix, > but I'm not up-to-date on the current developments). This one is a standards-compliant reencoding. Even if mimecast (?) stops doing it, we can't be sure nothing else will. > Since the sourceware.org Mailman migration, the From: header is being > rewritten, without any compelling reason. I certainly do not do any > DMARC checking here, so the rewriting does not benefit me. It benefits you because more and more email services are rejecting or interfering with mail that is not clean enough. If you want to receive mail reliably, or send and have confidence that it is received, clean mail benefits you. - FChE ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 11:01 ` Frank Ch. Eigler @ 2020-03-18 11:39 ` Florian Weimer 2020-03-18 14:19 ` Michael Matz 1 sibling, 0 replies; 76+ messages in thread From: Florian Weimer @ 2020-03-18 11:39 UTC (permalink / raw) To: Frank Ch. Eigler Cc: Overseers mailing list, Frank Ch. Eigler via Gcc, overseers, Alexander Monakov, Segher Boessenkool * Frank Ch. Eigler: > Hi - > >> > The key here is to realize that the raw message is not what you get >> > back from the mailing list reflector, and also not the raw message >> > that was sent by the sender. In this day of mta intermediaries, >> > proxies, reflectors, it may be time to revisit that suggestion. >> >> But these largely are new problems. It used to work flawlessly. > > I understand that's frustrating. But these workflows were counting on > literally unspecified behaviours not changing, or outright standards > violations continuing. Delivery of each individual message is unspecified as well, but we still count it to happen. I'm sorry, but I think this argument sounds a bit vacuous. With our own infrastructure, we should be able to get it to behave in the way we need. >> Patch reencoding problems go back to the redhat.com changes last >> November (I understand the responsible vendor is working on a fix, >> but I'm not up-to-date on the current developments). > > This one is a standards-compliant reencoding. Even if mimecast (?) > stops doing it, we can't be sure nothing else will. But it's rather unusual in the RFC 822 world, especially with the decline of sendmail and its peculiar 8BITMIME handling. I understand that you get a lot of this in the corporate mail world originally influenced by X.400, but such messages are rarely handled by sourceware these days, probably because people rather use Gmail than wrestle with their inadequate corporate email. >> Since the sourceware.org Mailman migration, the From: header is being >> rewritten, without any compelling reason. I certainly do not do any >> DMARC checking here, so the rewriting does not benefit me. > > It benefits you because more and more email services are rejecting or > interfering with mail that is not clean enough. If you want to > receive mail reliably, or send and have confidence that it is > received, clean mail benefits you. It's definitely not cleaner after Mailman has applied its destructive header rewriting. It just replaces one address spoofing with another. What are the plans for Mailman on sourceware? Will it be replaced soon with something else, given that the software stack it runs on is effectively EOL? It should not be too hard to add a configuration option where subscribers can opt out of header rewriting, but with Mailman's upstream status, I'm not sure if that's a worthwhile effort. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 11:01 ` Frank Ch. Eigler 2020-03-18 11:39 ` Florian Weimer @ 2020-03-18 14:19 ` Michael Matz 2020-03-18 14:22 ` Frank Ch. Eigler 1 sibling, 1 reply; 76+ messages in thread From: Michael Matz @ 2020-03-18 14:19 UTC (permalink / raw) To: Frank Ch. Eigler Cc: Overseers mailing list, overseers, Frank Ch. Eigler via Gcc, Alexander Monakov, Florian Weimer, Segher Boessenkool Hi, On Wed, 18 Mar 2020, Frank Ch. Eigler via Gcc wrote: > > > The key here is to realize that the raw message is not what you get > > > back from the mailing list reflector, and also not the raw message > > > that was sent by the sender. In this day of mta intermediaries, > > > proxies, reflectors, it may be time to revisit that suggestion. > > > > But these largely are new problems. It used to work flawlessly. > > I understand that's frustrating. But these workflows were counting on > literally unspecified behaviours not changing, or outright standards > violations continuing. Wut? How is "not mangle the mail body" in any way violating standards? You're talking about rewriting or adding headers (where the former is Real Bad, no matter what DMARC wants to impose), but the suggestion is based on not rewriting the body. If the body (including attachtments) is rewritten any way then that simply is a bug. > > Patch reencoding problems go back to the redhat.com changes last > > November (I understand the responsible vendor is working on a fix, > > but I'm not up-to-date on the current developments). > > This one is a standards-compliant reencoding. Even if mimecast (?) > stops doing it, we can't be sure nothing else will. > > > Since the sourceware.org Mailman migration, the From: header is being > > rewritten, without any compelling reason. I certainly do not do any > > DMARC checking here, so the rewriting does not benefit me. > > It benefits you because more and more email services are rejecting or > interfering with mail that is not clean enough. If you want to > receive mail reliably, or send and have confidence that it is > received, clean mail benefits you. Depends on your definition of "clean". If by that you mean rewriting mail bodies then I'm not sure what to say. Ciao, Michael. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 14:19 ` Michael Matz @ 2020-03-18 14:22 ` Frank Ch. Eigler 2020-03-18 15:15 ` Bernd Schmidt 0 siblings, 1 reply; 76+ messages in thread From: Frank Ch. Eigler @ 2020-03-18 14:22 UTC (permalink / raw) To: Overseers mailing list Cc: Frank Ch. Eigler, Michael Matz, overseers, Frank Ch. Eigler via Gcc, Segher Boessenkool, Alexander Monakov, Florian Weimer Hi - > [...] You're talking about rewriting or adding headers (where the > former is Real Bad, no matter what DMARC wants to impose), but the > suggestion is based on not rewriting the body. If the body > (including attachtments) is rewritten any way then that simply is a > bug. [...] We're mixing two things. The From: header rewriting for DMARC participants is something sourceware is doing now. The Content-Transfer-Encoding: change is done by intermediate MTA's whose identity is unknown. (I don't believe this behaviour is forbidden by RFCs, but even if it were, we may have no way of fixing the mystery MTA.) - FChE ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 14:22 ` Frank Ch. Eigler @ 2020-03-18 15:15 ` Bernd Schmidt 2020-03-18 16:22 ` Frank Ch. Eigler 0 siblings, 1 reply; 76+ messages in thread From: Bernd Schmidt @ 2020-03-18 15:15 UTC (permalink / raw) To: Frank Ch. Eigler, Overseers mailing list Cc: overseers, Frank Ch. Eigler via Gcc, Alexander Monakov, Segher Boessenkool, Frank Ch. Eigler, Florian Weimer On 3/18/20 3:22 PM, Frank Ch. Eigler via Gcc wrote: > The From: header rewriting for DMARC participants is something sourceware > is doing now. Out of curiousity, is this rewriting you are talking about the cause for a lot of mails showing up as "From: GCC List" rather than their real senders? This has become very annoying recently. Bernd ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 15:15 ` Bernd Schmidt @ 2020-03-18 16:22 ` Frank Ch. Eigler 2020-03-18 16:42 ` Michael Matz 2020-04-06 20:58 ` Maciej W. Rozycki 0 siblings, 2 replies; 76+ messages in thread From: Frank Ch. Eigler @ 2020-03-18 16:22 UTC (permalink / raw) To: Bernd Schmidt Cc: Overseers mailing list, overseers, Frank Ch. Eigler via Gcc, Alexander Monakov, Segher Boessenkool, Frank Ch. Eigler, Florian Weimer Hi - > > The From: header rewriting for DMARC participants is something sourceware > > is doing now. > > Out of curiousity, is this rewriting you are talking about the cause for a > lot of mails showing up as "From: GCC List" rather than their real senders? > This has become very annoying recently. Yes, for emails from domains with declared interest in email cleanliness, via DMARC records in DNS. We have observed mail -blocked- at third parties, even just days ago, when we failed to sufficiently authenticate outgoing reflected emails. AIUI, all this effort is driven by wanting to defeat not just spammers but also real security problems like phishing enabled by forgery, including specifically the From: header. - FChE ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 16:22 ` Frank Ch. Eigler @ 2020-03-18 16:42 ` Michael Matz 2020-03-18 17:36 ` Christopher Faylor 2020-03-18 21:52 ` Jim Wilson 2020-04-06 20:58 ` Maciej W. Rozycki 1 sibling, 2 replies; 76+ messages in thread From: Michael Matz @ 2020-03-18 16:42 UTC (permalink / raw) To: Frank Ch. Eigler Cc: Bernd Schmidt, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Florian Weimer Hello, On Wed, 18 Mar 2020, Frank Ch. Eigler via Gcc wrote: > > > The From: header rewriting for DMARC participants is something sourceware > > > is doing now. > > > > Out of curiousity, is this rewriting you are talking about the cause for a > > lot of mails showing up as "From: GCC List" rather than their real senders? > > This has become very annoying recently. > > Yes, for emails from domains with declared interest in email > cleanliness, via DMARC records in DNS. We have observed mail > -blocked- at third parties, even just days ago, when we failed to > sufficiently authenticate outgoing reflected emails. Was this blocking also a problem before mailman (i.e. two weeks ago)? Why did nobody scream for not having received mail? Or why is it blocked now, but wasn't before? Can it be made so again, like it was with ezmlm? (And DMARCs requirement of having to rewrite From: headers should make it clear to everyone that it's stupid). Ciao, Michael. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 16:42 ` Michael Matz @ 2020-03-18 17:36 ` Christopher Faylor 2020-03-18 21:52 ` Jim Wilson 1 sibling, 0 replies; 76+ messages in thread From: Christopher Faylor @ 2020-03-18 17:36 UTC (permalink / raw) To: Overseers mailing list Cc: Segher Boessenkool, Alexander Monakov, Bernd Schmidt, Florian Weimer On Wed, Mar 18, 2020 at 04:42:13PM +0000, Michael Matz wrote: >On Wed, 18 Mar 2020, Frank Ch. Eigler via Gcc wrote: >> Yes, for emails from domains with declared interest in email >> cleanliness, via DMARC records in DNS. We have observed mail >> -blocked- at third parties, even just days ago, when we failed to >> sufficiently authenticate outgoing reflected emails. > >Was this blocking also a problem before mailman (i.e. two weeks ago)? >Why did nobody scream for not having received mail? Or why is it blocked >now, but wasn't before? Can it be made so again, like it was with ezmlm? We *were* seeing an increased number of bounces because qmail wasn't doing the right thing. People were complaining to postmaster that they hadn't received email. I was dreading having to figure out what DMARC issues they were having with ezmlm/qmail. So, it's nice for us admin volunteers to have a standard system to use that doesn't require searching for patches or workarounds non-standard systems like ezmlm/qmail. >(And DMARCs requirement of having to rewrite From: headers should make it >clear to everyone that it's stupid). I hate the rewriting quite a bit. But, if major players like gmail have adopted DMARC then there really isn't much we can do except to play along. It is galling, but the alternative is telling people not to use gmail or yahoo. We obviously can't do that. I'm not sure that everyone understands that on the old server we were using an ancient, hacked email system of qmail and ezmlm. I'd added unofficial (if official even means anything) patches and work arounds to allow us to continue to communicate with, e.g., yahoo and qmail but they never worked 100% right and, as mentioned, we were starting to see bounces and complaints. Even the old system munged From addresses, too. We decided for server2 to standardize on postfix + mailman because both were available on RHEL. Using postfix + mailman means that if anything happens to fche or cgf, it should be possible for some hypothetical volunteer to figure out how email works on sourceware. There is at least one site that offers qmail and ezmlm packages but the support seeemed somewhat iffy. And, if we had used those instead, and they had updated their support for DMARC/DKIM/SPF then we'd be in the same boat anyway. mailman is supposed to only munge From addresses from domains that enforce DMARC but we're finding that some domains don't properly advertise themselves and email is not going through to them. So, it is looking like we may have to turn it on universally. The bottom line is that mailman tries harder to make sure that mail gets through than the kludge I added to qmail ever did so, in theory, more mail should get through once we finish tweaking. FWIW, I rewrote my local email delivery agent so that it it puts most of the From addresses back the way they were before. It catches most of the munging so that I don't even notice it anymore. cgf ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 16:42 ` Michael Matz 2020-03-18 17:36 ` Christopher Faylor @ 2020-03-18 21:52 ` Jim Wilson 2020-03-18 22:29 ` Jonathan Wakely ` (2 more replies) 1 sibling, 3 replies; 76+ messages in thread From: Jim Wilson @ 2020-03-18 21:52 UTC (permalink / raw) To: Michael Matz Cc: Frank Ch. Eigler, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Florian Weimer I'm one of the old timers that likes our current work flow, but even I think that we are risking our future by staying with antiquated tools. One of the first things I need to teach new people is now to use email "properly". It is a barrier to entry for new contributors, since our requirements aren't how the rest of the world uses email anymore. LLVM has phabricator. Some git based projects are using gerrit. Github and gitlab are useful services. We need to think about setting up easier ways for people to submit patches, rather than trying to fix all of the MUAs and MTAs in the world. Jim ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 21:52 ` Jim Wilson @ 2020-03-18 22:29 ` Jonathan Wakely 2020-03-18 22:34 ` Jonathan Wakely ` (2 more replies) 2020-03-18 22:33 ` Frank Ch. Eigler 2020-03-19 0:32 ` Segher Boessenkool 2 siblings, 3 replies; 76+ messages in thread From: Jonathan Wakely @ 2020-03-18 22:29 UTC (permalink / raw) To: Jim Wilson Cc: Michael Matz, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Florian Weimer On Wed, 18 Mar 2020 at 21:54, Jim Wilson wrote: > > I'm one of the old timers that likes our current work flow, but even I > think that we are risking our future by staying with antiquated tools. > One of the first things I need to teach new people is now to use email > "properly". It is a barrier to entry for new contributors, since our > requirements aren't how the rest of the world uses email anymore. > LLVM has phabricator. Which is horrible. > Some git based projects are using gerrit. Which I looked into previously and decided I didn't like it. If I recall correctly, gerrit has to "own" the repo, and so it's only possible to commit to the repo by pushing to gerrit first, then getting the patch approved. That is fine for write-after-approval, but adds a step for maintainers who can approve their own changes. I think it also only very recently gained the ability to group a series of patches together, as it wants a single commit per review. > Github and gitlab are useful services. We need to think about setting > up easier ways for people to submit patches, rather than trying to fix > all of the MUAs and MTAs in the world. There's also https://pagure.io/pagure which is Free Software. I think it would be great if one of those forge services could be set up to run *in parallel* to our existing workflow, so that we can accept pull requests there without forcing all work to go through it (like gerrit). New contributors who are used to the GitHub model could submit pull requests there, and maintainers could merge them into the repo with the click of a button. For significant changes it's more likely maintainers would pull the branch into their local repo, test it, and then push manually. But if we had a CI service testing pull requests (like Travis CI does for GitHub) and a pull request was shown to introduce no regressions then it could be merged with a single click. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 22:29 ` Jonathan Wakely @ 2020-03-18 22:34 ` Jonathan Wakely 2020-03-18 22:44 ` Frank Ch. Eigler 2020-03-18 22:45 ` Joseph Myers 2020-03-19 13:05 ` Tom Tromey 2 siblings, 1 reply; 76+ messages in thread From: Jonathan Wakely @ 2020-03-18 22:34 UTC (permalink / raw) To: gcc; +Cc: overseers N.B. the CC list has got too big and is causing posts to this thread to be held for moderator approval. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 22:34 ` Jonathan Wakely @ 2020-03-18 22:44 ` Frank Ch. Eigler 2020-03-19 3:30 ` Christopher Faylor 0 siblings, 1 reply; 76+ messages in thread From: Frank Ch. Eigler @ 2020-03-18 22:44 UTC (permalink / raw) To: Overseers mailing list; +Cc: gcc, Jonathan Wakely, overseers Hi - > N.B. the CC list has got too big and is causing posts to this thread > to be held for moderator approval. Ah, can cycle through the lists and raise that limit. The default 10 is too low. - FChE ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 22:44 ` Frank Ch. Eigler @ 2020-03-19 3:30 ` Christopher Faylor 2020-03-19 3:32 ` Christopher Faylor 0 siblings, 1 reply; 76+ messages in thread From: Christopher Faylor @ 2020-03-19 3:30 UTC (permalink / raw) To: Overseers mailing list; +Cc: overseers, gcc, Jonathan Wakely On Wed, Mar 18, 2020 at 06:44:15PM -0400, Frank Ch. Eigler wrote: >> N.B. the CC list has got too big and is causing posts to this thread >> to be held for moderator approval. > >Ah, can cycle through the lists and raise that limit. >The default 10 is too low. Didn't you have to lower that limit for outpost, fche? I believe it used to be 10 for server1 too, though, fwiw. cgf ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-19 3:30 ` Christopher Faylor @ 2020-03-19 3:32 ` Christopher Faylor 0 siblings, 0 replies; 76+ messages in thread From: Christopher Faylor @ 2020-03-19 3:32 UTC (permalink / raw) To: overseers, gcc, Jonathan Wakely On Wed, Mar 18, 2020 at 11:30:22PM -0400, Christopher Faylor wrote: >On Wed, Mar 18, 2020 at 06:44:15PM -0400, Frank Ch. Eigler wrote: >>> N.B. the CC list has got too big and is causing posts to this thread >>> to be held for moderator approval. >> >>Ah, can cycle through the lists and raise that limit. >>The default 10 is too low. > >Didn't you have to lower that limit for outpost, fche? > >I believe it used to be 10 for server1 too, though, fwiw. Another annoying thing about mailman is this double inclusion of the same address in the Cc. cgf ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 22:29 ` Jonathan Wakely 2020-03-18 22:34 ` Jonathan Wakely @ 2020-03-18 22:45 ` Joseph Myers 2020-03-18 23:02 ` Jonathan Wakely 2020-03-19 13:05 ` Tom Tromey 2 siblings, 1 reply; 76+ messages in thread From: Joseph Myers @ 2020-03-18 22:45 UTC (permalink / raw) To: Jonathan Wakely; +Cc: overseers, Frank Ch. Eigler via Gcc On Wed, 18 Mar 2020, Jonathan Wakely via Gcc wrote: > > Some git based projects are using gerrit. > > Which I looked into previously and decided I didn't like it. If I > recall correctly, gerrit has to "own" the repo, and so it's only The glibc experiment with gerrit worked without it owning the repo. There were a few issues with email notifications that were addressed by local patches to gerrit, and a few other such issues with email interaction that didn't get addressed but looked like they could have been - but nothing for which addressing it seemed unacceptable upstream or that looked fatal to using gerrit without it owning the repo and (given some fixes) working reasonably well with email (similarly well to Bugzilla, say - sending email notifications to a list with sensible content, doing something sensible with email replies). -- Joseph S. Myers joseph@codesourcery.com ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 22:45 ` Joseph Myers @ 2020-03-18 23:02 ` Jonathan Wakely 0 siblings, 0 replies; 76+ messages in thread From: Jonathan Wakely @ 2020-03-18 23:02 UTC (permalink / raw) To: Joseph Myers; +Cc: overseers, Frank Ch. Eigler via Gcc On Wed, 18 Mar 2020 at 22:45, Joseph Myers wrote: > > On Wed, 18 Mar 2020, Jonathan Wakely via Gcc wrote: > > > > Some git based projects are using gerrit. > > > > Which I looked into previously and decided I didn't like it. If I > > recall correctly, gerrit has to "own" the repo, and so it's only > > The glibc experiment with gerrit worked without it owning the repo. Ah, good to know, thanks. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 22:29 ` Jonathan Wakely 2020-03-18 22:34 ` Jonathan Wakely 2020-03-18 22:45 ` Joseph Myers @ 2020-03-19 13:05 ` Tom Tromey 2020-03-19 13:26 ` Florian Weimer 2 siblings, 1 reply; 76+ messages in thread From: Tom Tromey @ 2020-03-19 13:05 UTC (permalink / raw) To: Jonathan Wakely via Gcc Cc: Jim Wilson, Jonathan Wakely, overseers, Overseers mailing list, Alexander Monakov, Segher Boessenkool, Frank Ch. Eigler, Frank Ch. Eigler, Florian Weimer >>>>> "Jonathan" == Jonathan Wakely via Gcc <gcc@gcc.gnu.org> writes: [gerrit] Jonathan> I think it also only very recently gained the ability to group a Jonathan> series of patches together, as it wants a single commit per review. We tried gerrit for gdb for a while, and in the end decided to drop it. The main issue for us is that gerrit's support for patch series is poor. In particular, it doesn't have any way to provide a cover letter (like git send-email --compose), but in gdb we rely on these to provide an introduction to the series -- to help justify the series overall and orient the reviewers. Here's the gerrit bug: https://bugs.chromium.org/p/gerrit/issues/detail?id=924 Based on this I think we all assumed that the situation wouldn't improve. Also, gerrit was pretty bad about threading messages, so it became quite hard to follow progress in email (but following all patches in the web interface is very difficult, a problem shared by all these web UIs). Phabricator, IME, is even worse. Last I used it, it had extremely bad support for patch series, to the extent that Mozilla had to write a tool wrapping Phabricator to make it workable. In gdb we've also considered using an updated patchworks -- with a gerrit-like commit hook it would be possible to automatically close patches when they land, which is patchworks' biggest weakness. (In gdb land we're more concerned with tracking unreviewed patches than with online patch review.) However, this probably would not be a good match for the new From munging, because it would mean extra (forgettable) steps when trying to apply patches from the patchworks repository. TL;DR we're doomed, Tom ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-19 13:05 ` Tom Tromey @ 2020-03-19 13:26 ` Florian Weimer 2020-03-19 13:41 ` Richard Biener 0 siblings, 1 reply; 76+ messages in thread From: Florian Weimer @ 2020-03-19 13:26 UTC (permalink / raw) To: Tom Tromey Cc: Jonathan Wakely via Gcc, Jim Wilson, Jonathan Wakely, overseers, Overseers mailing list, Alexander Monakov, Segher Boessenkool, Frank Ch. Eigler, Frank Ch. Eigler * Tom Tromey: > Also, gerrit was pretty bad about threading messages, so it became quite > hard to follow progress in email (but following all patches in the web > interface is very difficult, a problem shared by all these web UIs). What I found most disappointing was that the web interface doesn't guide you to the next reasonable step for your reviews and patches, like showing comments which need addressing. Tagging messages in an email client for later action actually works better than that, I think. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-19 13:26 ` Florian Weimer @ 2020-03-19 13:41 ` Richard Biener 2020-03-19 13:58 ` Florian Weimer 2020-03-19 18:08 ` Segher Boessenkool 0 siblings, 2 replies; 76+ messages in thread From: Richard Biener @ 2020-03-19 13:41 UTC (permalink / raw) To: Florian Weimer Cc: Tom Tromey, overseers, Jonathan Wakely via Gcc, Segher Boessenkool, Alexander Monakov, Overseers mailing list, Frank Ch. Eigler, Frank Ch. Eigler On Thu, Mar 19, 2020 at 2:28 PM Florian Weimer <fw@deneb.enyo.de> wrote: > > * Tom Tromey: > > > Also, gerrit was pretty bad about threading messages, so it became quite > > hard to follow progress in email (but following all patches in the web > > interface is very difficult, a problem shared by all these web UIs). > > What I found most disappointing was that the web interface doesn't > guide you to the next reasonable step for your reviews and patches, > like showing comments which need addressing. Tagging messages in an > email client for later action actually works better than that, I think. I guess if anything we'd want something git-centric now like github or gitlab pull requests & reviews. The only complication is approval then which would still mean manual steps. Patch review would also not be publicly visible and archived(?) so both chiming in late after visible progress and archeology would be harder. I think following all patch reviews by clicking on websites rather than watching gcc-patches is impractical. Richard. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-19 13:41 ` Richard Biener @ 2020-03-19 13:58 ` Florian Weimer 2020-03-19 18:17 ` Segher Boessenkool 2020-04-23 0:54 ` Tamar Christina 2020-03-19 18:08 ` Segher Boessenkool 1 sibling, 2 replies; 76+ messages in thread From: Florian Weimer @ 2020-03-19 13:58 UTC (permalink / raw) To: Richard Biener Cc: Tom Tromey, overseers, Jonathan Wakely via Gcc, Segher Boessenkool, Alexander Monakov, Overseers mailing list, Frank Ch. Eigler, Frank Ch. Eigler * Richard Biener: > On Thu, Mar 19, 2020 at 2:28 PM Florian Weimer <fw@deneb.enyo.de> wrote: >> >> * Tom Tromey: >> >> > Also, gerrit was pretty bad about threading messages, so it became quite >> > hard to follow progress in email (but following all patches in the web >> > interface is very difficult, a problem shared by all these web UIs). >> >> What I found most disappointing was that the web interface doesn't >> guide you to the next reasonable step for your reviews and patches, >> like showing comments which need addressing. Tagging messages in an >> email client for later action actually works better than that, I think. > > I guess if anything we'd want something git-centric now like github > or gitlab pull requests & reviews. The only complication is approval > then which would still mean manual steps. Gitlab has a “merge if CI passes” button, I think. There are similar third-party solutions for Github. > Patch review would also not be publicly visible and archived(?) so > both chiming in late after visible progress and archeology would be > harder. The comments could be archived on a mailing list. But there is only some threading. Maybe this can be compensated to some extent by producing smaller patches, so that there's less reason for mega-review-threads. I don't know if Gitlab supports patch reviews, or if you only can review all the squashed changes in a merge request. Github and Gerrit support comments on individual patches, but Pagure does not, so this is not something that has universal support in all such tools. (I would consider such a feature critical for glibc development.) I haven't figured out yet under what circumstances Github links commits to pull requests in the web UI. I'm not sure if this relationship is available outside the web UI. (Obviously, this is missing from the current gcc-patches feed as well.) ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-19 13:58 ` Florian Weimer @ 2020-03-19 18:17 ` Segher Boessenkool 2020-04-23 0:54 ` Tamar Christina 1 sibling, 0 replies; 76+ messages in thread From: Segher Boessenkool @ 2020-03-19 18:17 UTC (permalink / raw) To: Florian Weimer Cc: Richard Biener, Tom Tromey, overseers, Jonathan Wakely via Gcc, Alexander Monakov, Overseers mailing list, Frank Ch. Eigler, Frank Ch. Eigler On Thu, Mar 19, 2020 at 02:58:55PM +0100, Florian Weimer wrote: > * Richard Biener: > > I guess if anything we'd want something git-centric now like github > > or gitlab pull requests & reviews. The only complication is approval > > then which would still mean manual steps. > > Gitlab has a “merge if CI passes” button, I think. There are similar > third-party solutions for Github. To actually commit patches is even more trivial than it was with SVN. This is not the hard step. "Merge if CI passes"... Well, it would be nice if we *had* some CI system :-) (But commits should *not* depend on that, and conversely, you should be able to run something on the CI without committing it, as well.) > The comments could be archived on a mailing list. But there is only > some threading. Maybe this can be compensated to some extent by > producing smaller patches, so that there's less reason for > mega-review-threads. We often have discussions on gcc-patches@ (instead of on gcc@). This is mostly natural, and artificially moving that to another ML because some tool is lousy is making new problems, not solving any. Segher ^ permalink raw reply [flat|nested] 76+ messages in thread
* RE: Not usable email content encoding 2020-03-19 13:58 ` Florian Weimer 2020-03-19 18:17 ` Segher Boessenkool @ 2020-04-23 0:54 ` Tamar Christina 2020-04-23 5:47 ` Florian Weimer 2020-04-23 11:46 ` Segher Boessenkool 1 sibling, 2 replies; 76+ messages in thread From: Tamar Christina @ 2020-04-23 0:54 UTC (permalink / raw) To: Florian Weimer, Richard Biener Cc: overseers, Jonathan Wakely via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Tom Tromey > -----Original Message----- > From: Gcc <gcc-bounces@gcc.gnu.org> On Behalf Of Florian Weimer > Sent: Thursday, March 19, 2020 1:59 PM > To: Richard Biener <richard.guenther@gmail.com> > Cc: overseers@gcc.gnu.org; Jonathan Wakely via Gcc <gcc@gcc.gnu.org>; > Overseers mailing list <overseers@sourceware.org>; Segher Boessenkool > <segher@kernel.crashing.org>; Alexander Monakov <amonakov@ispras.ru>; > Frank Ch. Eigler <fche@elastic.org>; Frank Ch. Eigler <fche@redhat.com>; > Tom Tromey <tom@tromey.com> > Subject: Re: Not usable email content encoding > > * Richard Biener: > > > On Thu, Mar 19, 2020 at 2:28 PM Florian Weimer <fw@deneb.enyo.de> > wrote: > >> > >> * Tom Tromey: > >> > >> > Also, gerrit was pretty bad about threading messages, so it became > >> > quite hard to follow progress in email (but following all patches > >> > in the web interface is very difficult, a problem shared by all these web > UIs). > >> > >> What I found most disappointing was that the web interface doesn't > >> guide you to the next reasonable step for your reviews and patches, > >> like showing comments which need addressing. Tagging messages in an > >> email client for later action actually works better than that, I think. > > > > I guess if anything we'd want something git-centric now like github or > > gitlab pull requests & reviews. The only complication is approval > > then which would still mean manual steps. > > Gitlab has a “merge if CI passes” button, I think. There are similar third-party > solutions for Github. A bit late to the party, but this really doesn't work that well because until recent version of gitlab there was no fairness guarantee. another patch could be approved after mine (with hours in between because of CI) and yet still get merged first causing my own patch to no longer apply, you'd rebase and roll the dice again. To fix this they added merge trains https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/ but trains for GCC Will likely be very short because of Changelog conflicts. So I don't think an automated merge workflow would work for projects where every single commit changes the same files. For these workflows to work as well you'd need to have a policy that every single commit go through CI, as a non-green master could flush your entire train if it's rebased on a bad commit. > > > Patch review would also not be publicly visible and archived(?) so > > both chiming in late after visible progress and archeology would be > > harder. > > The comments could be archived on a mailing list. But there is only some > threading. Maybe this can be compensated to some extent by producing > smaller patches, so that there's less reason for mega-review-threads. > > I don't know if Gitlab supports patch reviews, or if you only can review all the > squashed changes in a merge request. Github and Gerrit support comments > on individual patches, but Pagure does not, so this is not something that has > universal support in all such tools. (I would consider such a feature critical for > glibc development.) > > I haven't figured out yet under what circumstances Github links commits to > pull requests in the web UI. I'm not sure if this relationship is available > outside the web UI. (Obviously, this is missing from the current gcc-patches > feed as well.) ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-23 0:54 ` Tamar Christina @ 2020-04-23 5:47 ` Florian Weimer 2020-04-23 7:34 ` Richard Biener ` (2 more replies) 2020-04-23 11:46 ` Segher Boessenkool 1 sibling, 3 replies; 76+ messages in thread From: Florian Weimer @ 2020-04-23 5:47 UTC (permalink / raw) To: Tamar Christina Cc: Richard Biener, overseers, Jonathan Wakely via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Tom Tromey * Tamar Christina: > A bit late to the party, but this really doesn't work that well > because until recent version of gitlab there was no fairness > guarantee. another patch could be approved after mine (with hours > in between because of CI) and yet still get merged first causing my > own patch to no longer apply, you'd rebase and roll the dice again. > To fix this they added merge trains > https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/ > > but trains for GCC Will likely be very short because of Changelog > conflicts. So I don't think an automated merge workflow would work > for projects where every single commit changes the same files. I had not thought about that. Does Gitlab support pluggable merge helpers? The gnulib changelog auto-merger did a great job when we were still writing changelogs for glibc. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-23 5:47 ` Florian Weimer @ 2020-04-23 7:34 ` Richard Biener [not found] ` <CAH6eHdShSXwc4K7ar9HWgHu12tDQe-s2LifdvmVp8VytwYV-ew@mail.gmail.com> 2020-04-23 9:55 ` Tamar Christina 2 siblings, 0 replies; 76+ messages in thread From: Richard Biener @ 2020-04-23 7:34 UTC (permalink / raw) To: Florian Weimer Cc: Tamar Christina, overseers, Jonathan Wakely via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Tom Tromey On Thu, Apr 23, 2020 at 7:47 AM Florian Weimer <fw@deneb.enyo.de> wrote: > > * Tamar Christina: > > > A bit late to the party, but this really doesn't work that well > > because until recent version of gitlab there was no fairness > > guarantee. another patch could be approved after mine (with hours > > in between because of CI) and yet still get merged first causing my > > own patch to no longer apply, you'd rebase and roll the dice again. > > To fix this they added merge trains > > https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/ > > > > but trains for GCC Will likely be very short because of Changelog > > conflicts. So I don't think an automated merge workflow would work > > for projects where every single commit changes the same files. > > I had not thought about that. > > Does Gitlab support pluggable merge helpers? The gnulib changelog > auto-merger did a great job when we were still writing changelogs for > glibc. Btw, I encourage everybody trying to experiment with CI to set it up for release branches first because of the lower check-in count. Richard. ^ permalink raw reply [flat|nested] 76+ messages in thread
[parent not found: <CAH6eHdShSXwc4K7ar9HWgHu12tDQe-s2LifdvmVp8VytwYV-ew@mail.gmail.com>]
* Re: Not usable email content encoding [not found] ` <CAH6eHdShSXwc4K7ar9HWgHu12tDQe-s2LifdvmVp8VytwYV-ew@mail.gmail.com> @ 2020-04-23 8:39 ` Jakub Jelinek 2020-04-23 15:32 ` Jeff Law 1 sibling, 0 replies; 76+ messages in thread From: Jakub Jelinek @ 2020-04-23 8:39 UTC (permalink / raw) To: Jonathan Wakely Cc: Florian Weimer, overseers, Jonathan Wakely via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Tom Tromey On Thu, Apr 23, 2020 at 09:34:20AM +0100, Jonathan Wakely via Gcc wrote: > I've been having problems with it recently, e.g. > https://gcc.gnu.org/g:e76100ced607218a3bf had to fix a changelog, > because https://gcc.gnu.org/g:d76925e46fad09fc9be67 put my changelog > entry in the wrong place in gcc/testsuite/Changelog, as a result of a > rebase using merge-changelog. > > Maybe we should follow glibc and get rid of ChangeLog files before > trying to use automated CI and Git workflows. I thought we are leaning towards generating the ChangeLog files from commit messages rather than getting rid of them, but we need a reliable script to find the ChangeLog snippet in the commit message and split it into the separate ChangeLog files and reliable checker that will refuse commits with badly formatted entries in there, because when it comes from commit messages, we won't be able to fix mistakes in it later. Jakub ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding [not found] ` <CAH6eHdShSXwc4K7ar9HWgHu12tDQe-s2LifdvmVp8VytwYV-ew@mail.gmail.com> 2020-04-23 8:39 ` Jakub Jelinek @ 2020-04-23 15:32 ` Jeff Law 1 sibling, 0 replies; 76+ messages in thread From: Jeff Law @ 2020-04-23 15:32 UTC (permalink / raw) To: Jonathan Wakely, Florian Weimer Cc: overseers, Jonathan Wakely via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Tom Tromey On Thu, 2020-04-23 at 09:34 +0100, Jonathan Wakely via Gcc wrote: > On Thu, 23 Apr 2020 at 06:49, Florian Weimer wrote: > > * Tamar Christina: > > > > > A bit late to the party, but this really doesn't work that well > > > because until recent version of gitlab there was no fairness > > > guarantee. another patch could be approved after mine (with hours > > > in between because of CI) and yet still get merged first causing my > > > own patch to no longer apply, you'd rebase and roll the dice again. > > > To fix this they added merge trains > > > https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/ > > > > > > but trains for GCC Will likely be very short because of Changelog > > > conflicts. So I don't think an automated merge workflow would work > > > for projects where every single commit changes the same files. > > > > I had not thought about that. > > > > Does Gitlab support pluggable merge helpers? The gnulib changelog > > auto-merger did a great job when we were still writing changelogs for > > glibc. > > I've been having problems with it recently, e.g. > https://gcc.gnu.org/g:e76100ced607218a3bf had to fix a changelog, > because https://gcc.gnu.org/g:d76925e46fad09fc9be67 put my changelog > entry in the wrong place in gcc/testsuite/Changelog, as a result of a > rebase using merge-changelog. > > Maybe we should follow glibc and get rid of ChangeLog files before > trying to use automated CI and Git workflows. This is precisely why I want to get rid of ChangeLogs and instead generate them from the VCS as part of the release process. Ultimately I want to be able to use workflows where I can push a button on something that's gone through a CI cycle or "git am" and not have to go back and fix things *by hand*. What we do with ChangeLogs is (*&@#$*)(*&^ insane from an efficient workflow standpoint. jeff ^ permalink raw reply [flat|nested] 76+ messages in thread
* RE: Not usable email content encoding 2020-04-23 5:47 ` Florian Weimer 2020-04-23 7:34 ` Richard Biener [not found] ` <CAH6eHdShSXwc4K7ar9HWgHu12tDQe-s2LifdvmVp8VytwYV-ew@mail.gmail.com> @ 2020-04-23 9:55 ` Tamar Christina 2 siblings, 0 replies; 76+ messages in thread From: Tamar Christina @ 2020-04-23 9:55 UTC (permalink / raw) To: Florian Weimer Cc: Richard Biener, overseers, Jonathan Wakely via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Tom Tromey > -----Original Message----- > From: Florian Weimer <fw@deneb.enyo.de> > > * Tamar Christina: > > > A bit late to the party, but this really doesn't work that well > > because until recent version of gitlab there was no fairness > > guarantee. another patch could be approved after mine (with hours in > > between because of CI) and yet still get merged first causing my own > > patch to no longer apply, you'd rebase and roll the dice again. > > To fix this they added merge trains > > > https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_me > > rged_results/merge_trains/ > > > > but trains for GCC Will likely be very short because of Changelog > > conflicts. So I don't think an automated merge workflow would work > > for projects where every single commit changes the same files. > > I had not thought about that. > > Does Gitlab support pluggable merge helpers? The gnulib changelog auto- > merger did a great job when we were still writing changelogs for glibc. I'm not sure and couldn't find a clear answer from a look in the docs.. But I believe it's using git for the rebase so might be able to do it with githooks? It does have some built in conflicts resolution methods but with some surprising limitations: --- The file, with conflict markers added, is not over 200 KB in size --- And also --- Note: GitLab resolves conflicts by creating a merge commit in the source branch that is not automatically merged into the target branch. This allows the merge commit to be reviewed and tested before the changes are merged, preventing unintended changes entering the target branch without review or breaking the build. --- So you probably don't want it to be aware of the conflict.. Cheers, Tamar ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-23 0:54 ` Tamar Christina 2020-04-23 5:47 ` Florian Weimer @ 2020-04-23 11:46 ` Segher Boessenkool [not found] ` <CAH6eHdTHsZWO9vuycKm22dDbSGSfzcrKbzEpb6vNdb-zXCiKjQ@mail.gmail.com> 2020-04-23 15:33 ` Jeff Law 1 sibling, 2 replies; 76+ messages in thread From: Segher Boessenkool @ 2020-04-23 11:46 UTC (permalink / raw) To: Tamar Christina Cc: Florian Weimer, Richard Biener, overseers, Jonathan Wakely via Gcc, Overseers mailing list, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Tom Tromey Hi! On Thu, Apr 23, 2020 at 12:54:04AM +0000, Tamar Christina wrote: > but trains for GCC Will likely be very short because of Changelog conflicts. Why that? Your patches should *not* touch changelog files, ever. For CI it is probably easiest if the CI never gets to see the changelog at all, and that isn't hard to do ;-) Segher ^ permalink raw reply [flat|nested] 76+ messages in thread
[parent not found: <CAH6eHdTHsZWO9vuycKm22dDbSGSfzcrKbzEpb6vNdb-zXCiKjQ@mail.gmail.com>]
* Re: Not usable email content encoding [not found] ` <CAH6eHdTHsZWO9vuycKm22dDbSGSfzcrKbzEpb6vNdb-zXCiKjQ@mail.gmail.com> @ 2020-04-23 12:30 ` Segher Boessenkool 0 siblings, 0 replies; 76+ messages in thread From: Segher Boessenkool @ 2020-04-23 12:30 UTC (permalink / raw) To: Jonathan Wakely Cc: Tamar Christina, overseers, Jonathan Wakely via Gcc, Overseers mailing list, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Florian Weimer, Tom Tromey On Thu, Apr 23, 2020 at 12:56:20PM +0100, Jonathan Wakely wrote: > On Thu, 23 Apr 2020 at 12:47, Segher Boessenkool > <segher@kernel.crashing.org> wrote: > > On Thu, Apr 23, 2020 at 12:54:04AM +0000, Tamar Christina wrote: > > > but trains for GCC Will likely be very short because of Changelog conflicts. > > > > Why that? Your patches should *not* touch changelog files, ever. > > > > For CI it is probably easiest if the CI never gets to see the changelog > > at all, and that isn't hard to do ;-) > > For the actual CI testing, yes. But if you want an automated "merge > after CI testing passes" then what you merge needs to add a ChangeLog > entry. So either we auto-generate the entry when doing the auto-merge, > or the ChangeLog entry is already there. Why would we make it part of the patch for that? That is backwards :-) If it isn't part of the patch, no matter how we do it I don't see how it would cause conflicts? Segher ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-23 11:46 ` Segher Boessenkool [not found] ` <CAH6eHdTHsZWO9vuycKm22dDbSGSfzcrKbzEpb6vNdb-zXCiKjQ@mail.gmail.com> @ 2020-04-23 15:33 ` Jeff Law 2020-04-23 16:57 ` Frank Ch. Eigler 2020-04-23 20:27 ` Segher Boessenkool 1 sibling, 2 replies; 76+ messages in thread From: Jeff Law @ 2020-04-23 15:33 UTC (permalink / raw) To: Segher Boessenkool, Tamar Christina Cc: overseers, Jonathan Wakely via Gcc, Overseers mailing list, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Florian Weimer, Tom Tromey On Thu, 2020-04-23 at 06:46 -0500, Segher Boessenkool wrote: > Hi! > > On Thu, Apr 23, 2020 at 12:54:04AM +0000, Tamar Christina wrote: > > but trains for GCC Will likely be very short because of Changelog conflicts. > > Why that? Your patches should *not* touch changelog files, ever. > > For CI it is probably easiest if the CI never gets to see the changelog > at all, and that isn't hard to do ;-) *Nothing* should touch changelog files :-) They should be generated from the VCS. IMHO of course. That implies that the patch includes a proper ChangeLog entry in the commit message. It also implies that if we muck one up, then we live with it rather than going back and rewriting them. jeff ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-23 15:33 ` Jeff Law @ 2020-04-23 16:57 ` Frank Ch. Eigler 2020-04-23 20:32 ` Segher Boessenkool 2020-04-23 20:27 ` Segher Boessenkool 1 sibling, 1 reply; 76+ messages in thread From: Frank Ch. Eigler @ 2020-04-23 16:57 UTC (permalink / raw) To: Jeff Law Cc: Segher Boessenkool, Tamar Christina, overseers, Jonathan Wakely via Gcc, Overseers mailing list, Alexander Monakov, Frank Ch. Eigler, Florian Weimer, Tom Tromey Hi - > *Nothing* should touch changelog files :-) They should be generated from the > VCS. IMHO of course. IMHO: the VCS should be the changelog. - FChE ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-23 16:57 ` Frank Ch. Eigler @ 2020-04-23 20:32 ` Segher Boessenkool 0 siblings, 0 replies; 76+ messages in thread From: Segher Boessenkool @ 2020-04-23 20:32 UTC (permalink / raw) To: Frank Ch. Eigler Cc: Jeff Law, Tamar Christina, overseers, Jonathan Wakely via Gcc, Overseers mailing list, Alexander Monakov, Frank Ch. Eigler, Florian Weimer, Tom Tromey Hi! On Thu, Apr 23, 2020 at 12:57:43PM -0400, Frank Ch. Eigler wrote: > > *Nothing* should touch changelog files :-) They should be generated from the > > VCS. IMHO of course. > > IMHO: the VCS should be the changelog. The VCS shows what changed. The changelog shows what was meant to be changed. The difference is very helpful for reviewers, and for post-mortem purposes, etc. This can be more than compensated for if submitters would write better patch descriptions, of course. But will that happen? Segher ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-23 15:33 ` Jeff Law 2020-04-23 16:57 ` Frank Ch. Eigler @ 2020-04-23 20:27 ` Segher Boessenkool 2020-04-23 21:14 ` Tom Tromey 1 sibling, 1 reply; 76+ messages in thread From: Segher Boessenkool @ 2020-04-23 20:27 UTC (permalink / raw) To: Jeff Law Cc: Tamar Christina, overseers, Jonathan Wakely via Gcc, Overseers mailing list, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Florian Weimer, Tom Tromey On Thu, Apr 23, 2020 at 09:33:47AM -0600, Jeff Law wrote: > On Thu, 2020-04-23 at 06:46 -0500, Segher Boessenkool wrote: > > Hi! > > > > On Thu, Apr 23, 2020 at 12:54:04AM +0000, Tamar Christina wrote: > > > but trains for GCC Will likely be very short because of Changelog conflicts. > > > > Why that? Your patches should *not* touch changelog files, ever. > > > > For CI it is probably easiest if the CI never gets to see the changelog > > at all, and that isn't hard to do ;-) > *Nothing* should touch changelog files :-) They should be generated from the > VCS. IMHO of course. My point was that this should *never* be part of patches, already. > That implies that the patch includes a proper ChangeLog entry in the commit > message. It also implies that if we muck one up, then we live with it rather > than going back and rewriting them. There also is the middle ground where we still have changelog files, but normal checkins automatically update those from the commit message. This is an intermediate step, but also makes it possible (and even easy) to fix up mistakes. Segher ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-23 20:27 ` Segher Boessenkool @ 2020-04-23 21:14 ` Tom Tromey 2020-04-24 15:48 ` Thomas Koenig 2020-05-01 15:51 ` Jeff Law 0 siblings, 2 replies; 76+ messages in thread From: Tom Tromey @ 2020-04-23 21:14 UTC (permalink / raw) To: Segher Boessenkool Cc: Jeff Law, Tamar Christina, overseers, Jonathan Wakely via Gcc, Overseers mailing list, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Florian Weimer, Tom Tromey >>>>> "Segher" == Segher Boessenkool <segher@kernel.crashing.org> writes: Segher> My point was that this should *never* be part of patches, already. FWIW, I use a few scripts so that I can keep ChangeLogs as files. That's what I do when working on gdb. https://github.com/tromey/git-gnu-changelog This is easier on the whole, IME, because it means there is no extra manual step before pushing. Of course, better would be to remove ChangeLogs entirely (including not putting anything like them into a commit message), because they are largely not useful and are just make-work. Again IMNSHO -- I know there are some folks who read them, but I basically never have since switching to git. Tom ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-23 21:14 ` Tom Tromey @ 2020-04-24 15:48 ` Thomas Koenig 2020-04-24 17:09 ` Segher Boessenkool 2020-05-01 15:51 ` Jeff Law 1 sibling, 1 reply; 76+ messages in thread From: Thomas Koenig @ 2020-04-24 15:48 UTC (permalink / raw) To: Tom Tromey, Segher Boessenkool Cc: overseers, Jonathan Wakely via Gcc, Overseers mailing list, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Florian Weimer > Of course, better would be to remove ChangeLogs entirely (including not > putting anything like them into a commit message), because they are > largely not useful and are just make-work. I disagree. I find them quite useful. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-24 15:48 ` Thomas Koenig @ 2020-04-24 17:09 ` Segher Boessenkool 0 siblings, 0 replies; 76+ messages in thread From: Segher Boessenkool @ 2020-04-24 17:09 UTC (permalink / raw) To: Thomas Koenig Cc: Tom Tromey, overseers, Jonathan Wakely via Gcc, Overseers mailing list, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Florian Weimer Hi! On Fri, Apr 24, 2020 at 05:48:38PM +0200, Thomas Koenig wrote: > >Of course, better would be to remove ChangeLogs entirely (including not > >putting anything like them into a commit message), because they are > >largely not useful and are just make-work. > > I disagree. I find them quite useful. For what? And, can that be provided some other way? Segher ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-23 21:14 ` Tom Tromey 2020-04-24 15:48 ` Thomas Koenig @ 2020-05-01 15:51 ` Jeff Law 2020-05-02 12:50 ` Segher Boessenkool 1 sibling, 1 reply; 76+ messages in thread From: Jeff Law @ 2020-05-01 15:51 UTC (permalink / raw) To: Tom Tromey, Segher Boessenkool Cc: Tamar Christina, overseers, Jonathan Wakely via Gcc, Overseers mailing list, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Florian Weimer On Thu, 2020-04-23 at 15:14 -0600, Tom Tromey wrote: > > > > > > "Segher" == Segher Boessenkool <segher@kernel.crashing.org> writes: > > Segher> My point was that this should *never* be part of patches, already. > > FWIW, I use a few scripts so that I can keep ChangeLogs as files. > That's what I do when working on gdb. > > https://github.com/tromey/git-gnu-changelog > > This is easier on the whole, IME, because it means there is no extra > manual step before pushing. Right. And that's really my goal here -- eliminate the manual steps. Ideally I want to be able to git am; git push on a good patch. Manual steps for good patches need to be excised from the workflow. The ChangeLog file is a major problem in that regard. > > Of course, better would be to remove ChangeLogs entirely (including not > putting anything like them into a commit message), because they are > largely not useful and are just make-work. Again IMNSHO -- I know there > are some folks who read them, but I basically never have since switching > to git. I read them less and less. At this point I think I read them to see if a particular patch in my queue has already been applied. Otherwise I'm using the git info. jeff ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-05-01 15:51 ` Jeff Law @ 2020-05-02 12:50 ` Segher Boessenkool 0 siblings, 0 replies; 76+ messages in thread From: Segher Boessenkool @ 2020-05-02 12:50 UTC (permalink / raw) To: Jeff Law Cc: Tom Tromey, Tamar Christina, overseers, Jonathan Wakely via Gcc, Overseers mailing list, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Florian Weimer On Fri, May 01, 2020 at 09:51:53AM -0600, Jeff Law wrote: > On Thu, 2020-04-23 at 15:14 -0600, Tom Tromey wrote: > > > > > > > "Segher" == Segher Boessenkool <segher@kernel.crashing.org> writes: > > > > Segher> My point was that this should *never* be part of patches, already. > > > > FWIW, I use a few scripts so that I can keep ChangeLogs as files. > > That's what I do when working on gdb. > > > > https://github.com/tromey/git-gnu-changelog > > > > This is easier on the whole, IME, because it means there is no extra > > manual step before pushing. > Right. And that's really my goal here -- eliminate the manual steps. Ideally I > want to be able to git am; git push on a good patch. Manual steps for good > patches need to be excised from the workflow. The ChangeLog file is a major > problem in that regard. I still almost always write the changelog not before sending a patch series to the mailing list. This is good *anyway*, it isn't a bad idea to look at your creation from a high level first, then; and it is also not a bad idea to look at the details, to see if you missed something. A "self-review", if you want. This of course also completely side-steps the issues with keeping the changelog up-to-date that so many people struggle with (that probably is what made me start doing this, heh). (I also have the changelogs in files of course, that way: the email series I send out I keep as files). > > Of course, better would be to remove ChangeLogs entirely (including not > > putting anything like them into a commit message), because they are > > largely not useful and are just make-work. Again IMNSHO -- I know there > > are some folks who read them, but I basically never have since switching > > to git. > I read them less and less. At this point I think I read them to see if a > particular patch in my queue has already been applied. Otherwise I'm using the > git info. It helps reviewing tremendously. Having very good patch factoring would help; having very good commit messages would help. Together they are much better than a changelog is. Without either (i.e., *the current status quo*), there is real value lost. And making it easier to shove in garbage is not an advantage :-/ Segher ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-19 13:41 ` Richard Biener 2020-03-19 13:58 ` Florian Weimer @ 2020-03-19 18:08 ` Segher Boessenkool 1 sibling, 0 replies; 76+ messages in thread From: Segher Boessenkool @ 2020-03-19 18:08 UTC (permalink / raw) To: Richard Biener Cc: Florian Weimer, Tom Tromey, overseers, Jonathan Wakely via Gcc, Alexander Monakov, Overseers mailing list, Frank Ch. Eigler, Frank Ch. Eigler On Thu, Mar 19, 2020 at 02:41:05PM +0100, Richard Biener wrote: > I guess if anything we'd want something git-centric now like github > or gitlab pull requests & reviews. The only complication is approval > then which would still mean manual steps. Patch review would also not > be publicly visible and archived(?) so both chiming in late after visible > progress and archeology would be harder. I think following all > patch reviews by clicking on websites rather than watching gcc-patches > is impractical. patchwork used to work great as an archive (it runs for GCC for almost ten years now). It is not meant as a patch review system: instead, it is meant as something for (kernel) maintainers where they can pick up patches to apply (which isn't our workflow: people apply their own patches, in general). It is also useful to find neglected patches, but that only can work if someone (or a group of people) does the work for that. This doesn't work for us. *Used to* work great... With the From: munging, it now has become quite hard (and very annoying) to navigate, and even just to read :-( Segher ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 21:52 ` Jim Wilson 2020-03-18 22:29 ` Jonathan Wakely @ 2020-03-18 22:33 ` Frank Ch. Eigler 2020-03-19 0:17 ` Segher Boessenkool 2020-03-19 0:32 ` Segher Boessenkool 2 siblings, 1 reply; 76+ messages in thread From: Frank Ch. Eigler @ 2020-03-18 22:33 UTC (permalink / raw) To: Jim Wilson Cc: Michael Matz, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Florian Weimer Hi, Jim - > [gerrit etc.] Good points. > [...] We need to think about setting up easier ways for people to > submit patches, rather than trying to fix all of the MUAs and MTAs > in the world. Another related point. We are comingling email as a communication medium AND a commit transport medium. For the former, as in patch review / RFC, one may not require a form of the patch that is finally committable to master, so the exact From: etc. may not matter. For the latter, attachments are more bullet-proof. - FChE ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 22:33 ` Frank Ch. Eigler @ 2020-03-19 0:17 ` Segher Boessenkool 0 siblings, 0 replies; 76+ messages in thread From: Segher Boessenkool @ 2020-03-19 0:17 UTC (permalink / raw) To: Frank Ch. Eigler Cc: Jim Wilson, Michael Matz, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Alexander Monakov, Frank Ch. Eigler, Florian Weimer Hi! On Wed, Mar 18, 2020 at 06:33:08PM -0400, Frank Ch. Eigler wrote: > > [...] We need to think about setting up easier ways for people to > > submit patches, rather than trying to fix all of the MUAs and MTAs > > in the world. > > Another related point. We are comingling email as a communication > medium AND a commit transport medium. For the former, as in patch > review / RFC, one may not require a form of the patch that is finally > committable to master, so the exact From: etc. may not matter. But OTOH, it is extremely valuable to review the commit message at the same time as the patch. Which we now *can*, for contributors who follow a more "git-like" workflow. > For the latter, attachments are more bullet-proof. Disregarding binary attachments, which are unworkable with many tools (and which are disallowed on gcc-patches for that reason), is this really true? Do some MUAs (or MTAs) mess up only the first body part they encounter? Or what? Segher ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 21:52 ` Jim Wilson 2020-03-18 22:29 ` Jonathan Wakely 2020-03-18 22:33 ` Frank Ch. Eigler @ 2020-03-19 0:32 ` Segher Boessenkool 2 siblings, 0 replies; 76+ messages in thread From: Segher Boessenkool @ 2020-03-19 0:32 UTC (permalink / raw) To: Jim Wilson Cc: Michael Matz, Frank Ch. Eigler, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Alexander Monakov, Frank Ch. Eigler, Florian Weimer Hi! On Wed, Mar 18, 2020 at 02:52:14PM -0700, Jim Wilson wrote: > I'm one of the old timers that likes our current work flow, but even I > think that we are risking our future by staying with antiquated tools. It's not ancient tools, it is low-requirement generic tools, and everyone can use that to build a workflow that works for him/herself. > One of the first things I need to teach new people is now to use email > "properly". It is a barrier to entry for new contributors, since our > requirements aren't how the rest of the world uses email anymore. Knowing how to use email properly is a very useful life skill. A large part of the Free Software (and open source) world still uses email as primary communication medium, too. Also you might want to read https://lwn.net/Articles/702177/ for a different viewpoint. > LLVM has phabricator. Some git based projects are using gerrit. > Github and gitlab are useful services. We need to think about setting > up easier ways for people to submit patches, rather than trying to fix > all of the MUAs and MTAs in the world. People should not just send patches: they need to explain why it would be a good idea to include that patch, which requires a lot of talking *about* the patch. Email is a difficult medium for that, but it is still _much_ better than any of the code review website things, imo. The key point is that email is completely free-form, I think? Segher ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-18 16:22 ` Frank Ch. Eigler 2020-03-18 16:42 ` Michael Matz @ 2020-04-06 20:58 ` Maciej W. Rozycki 2020-04-06 21:09 ` Frank Ch. Eigler 2020-04-06 23:15 ` Segher Boessenkool 1 sibling, 2 replies; 76+ messages in thread From: Maciej W. Rozycki @ 2020-04-06 20:58 UTC (permalink / raw) To: Frank Ch. Eigler Cc: Bernd Schmidt, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Florian Weimer On Wed, 18 Mar 2020, Frank Ch. Eigler via Gcc wrote: > > Out of curiousity, is this rewriting you are talking about the cause for a > > lot of mails showing up as "From: GCC List" rather than their real senders? > > This has become very annoying recently. > > Yes, for emails from domains with declared interest in email > cleanliness, via DMARC records in DNS. We have observed mail > -blocked- at third parties, even just days ago, when we failed to > sufficiently authenticate outgoing reflected emails. > > AIUI, all this effort is driven by wanting to defeat not just spammers > but also real security problems like phishing enabled by forgery, > including specifically the From: header. And it has actually broken GCC's patchwork: <http://patchwork.ozlabs.org/project/gcc/list/>, which I used to use to track my patches. Now I cannot do that anymore as patches submitted from my WDC address are marked as coming from <gcc-patches@gcc.gnu.org>, which obviously means they are not attributed to me. I am fairly sure it breaks `git am' too, requiring a `From' override in the change description for author attribution in patch application to work reliably (I tend to work on my outbox when applying my own patches, so I avoid this issue, but I am sure the issue will hit someone sooner or later). And of course I cannot use the `macro@' pattern anymore to select mailing list messages in my inbox that I sent myself. Frankly it's the least of the annoyances, but still, and they all add up. This is all silly, requiring the SMTP envelope sender to match the `From' header breaks even the most basic e-mail mechanisms like the use of a `.forward' file. Can we please do something about it? Is functional regression the price we absolutely need to pay for progress? How come the Linux kernel people who do e-mail patch management on a vastly larger scale than we do, both in terms of traffic and the number of mailing list subscribers, can get away without all these odd quirks in their list server management? Perhaps it would be good asking them how they handle their stuff? Maciej ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-06 20:58 ` Maciej W. Rozycki @ 2020-04-06 21:09 ` Frank Ch. Eigler 2020-04-06 21:59 ` Maciej W. Rozycki 2020-04-06 23:15 ` Segher Boessenkool 1 sibling, 1 reply; 76+ messages in thread From: Frank Ch. Eigler @ 2020-04-06 21:09 UTC (permalink / raw) To: Maciej W. Rozycki Cc: Bernd Schmidt, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Florian Weimer Hi - > [...] > I am fairly sure it breaks `git am' too, requiring a `From' override in > the change description for author attribution in patch application to work > reliably (I tend to work on my outbox when applying my own patches, so I > avoid this issue, but I am sure the issue will hit someone sooner or > later). That part is at least pretty easy: use git format-patch --from "Real Name git <you@email.com>" which will then force a second fake From: header into the body of the commit email, where git-am can find it. > This is all silly, requiring the SMTP envelope sender to match the `From' > header breaks even the most basic e-mail mechanisms like the use of a > `.forward' file. [...] Unfortunately naive .forward based forwarding looks exactly like faked or spam email to a third party MTAs. > How come the Linux kernel people who do e-mail patch management on a > vastly larger scale than we do, both in terms of traffic and the > number of mailing list subscribers, can get away without all these > odd quirks in their list server management? [...] I'm not sure, but their mails tend to be laden with a vast number of Cc:'s, which bypass mailing list reflectors. - FChE ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-06 21:09 ` Frank Ch. Eigler @ 2020-04-06 21:59 ` Maciej W. Rozycki [not found] ` <CAH6eHdTqJEM90d9+nNWrAa_zZrQQha79oMbnbVsRt_2Zd90PdA@mail.gmail.com> 0 siblings, 1 reply; 76+ messages in thread From: Maciej W. Rozycki @ 2020-04-06 21:59 UTC (permalink / raw) To: Frank Ch. Eigler Cc: Bernd Schmidt, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Florian Weimer Hi Frank, > > I am fairly sure it breaks `git am' too, requiring a `From' override in > > the change description for author attribution in patch application to work > > reliably (I tend to work on my outbox when applying my own patches, so I > > avoid this issue, but I am sure the issue will hit someone sooner or > > later). > > That part is at least pretty easy: use > git format-patch --from "Real Name git <you@email.com>" > which will then force a second fake From: header into the body of > the commit email, where git-am can find it. Sure, there are various ways to deal with that, both on the sender's and the receiver's side, however all require a manual intervention and are therefore prone to a human error. Which will obviously happen sooner or later. > > This is all silly, requiring the SMTP envelope sender to match the `From' > > header breaks even the most basic e-mail mechanisms like the use of a > > `.forward' file. [...] > > Unfortunately naive .forward based forwarding looks exactly like faked > or spam email to a third party MTAs. And can certainly score a positive though not a definite rating in spam qualification. I don't think we ought to encourage bad IT management practices by trying to adapt to them too hard and hurting ourselves (our workflow) in the process. > > How come the Linux kernel people who do e-mail patch management on a > > vastly larger scale than we do, both in terms of traffic and the > > number of mailing list subscribers, can get away without all these > > odd quirks in their list server management? [...] > > I'm not sure, but their mails tend to be laden with a vast number of > Cc:'s, which bypass mailing list reflectors. Surely all the bots receive messages via a mailing list subscription though. Cc's are solely for maintainers, reviewers or other explicitly interested parties. Some maintainers actually require submissions to come via the relevant mailing list rather directly to them. Somehow it works. Thank you for your input regardless! Maciej ^ permalink raw reply [flat|nested] 76+ messages in thread
[parent not found: <CAH6eHdTqJEM90d9+nNWrAa_zZrQQha79oMbnbVsRt_2Zd90PdA@mail.gmail.com>]
* Re: Not usable email content encoding [not found] ` <CAH6eHdTqJEM90d9+nNWrAa_zZrQQha79oMbnbVsRt_2Zd90PdA@mail.gmail.com> @ 2020-04-07 9:53 ` Florian Weimer 2020-04-08 12:38 ` Mark Wielaard 2020-04-07 15:13 ` Michael Matz 1 sibling, 1 reply; 76+ messages in thread From: Florian Weimer @ 2020-04-07 9:53 UTC (permalink / raw) To: Jonathan Wakely Cc: Maciej W. Rozycki, Frank Ch. Eigler, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler * Jonathan Wakely: > On Mon, 6 Apr 2020 at 23:00, Maciej W. Rozycki via Gcc <gcc@gcc.gnu.org> wrote: >> And can certainly score a positive though not a definite rating in spam >> qualification. I don't think we ought to encourage bad IT management >> practices by trying to adapt to them too hard and hurting ourselves (our >> workflow) in the process. > > What you call "bad IT management practices" includes how Gmail works, > which a HUGE number of people use. > > A number of lists I'm on switched to our current style of minging a > year or two ago, because gmail users were not receiving mail, because > gmail was rejecting the mail. Gmail can drop mail for any reason. It's totally opaque, so it's a poor benchmark for any mailing list configuration changes because it's very hard to tell if a particular change is effective or not. Many mailing lists have *not* made such changes and continue to work just fine in the face of restrictive DMARC sender policies and enforcement at the recipient. In general, mail drop rates due to DMARC seem to increase in these two cases if the original From: header is preserved: * The sender (i.e., the domain mentioned in the From: header) publishes a restrictive DMARC policy and the mailing list strips the DKIM signature. * The sender signs parts of the message that the mailing list alters, and the mailing list does not strip the DKIM signature. If neither scenario applies, it's safe to pass through the message without munging. The mailing list software can detect this and restricting the From: header munging to those cases. I doubt Mailman 2.x can do this, so it is simply a poor choice as mailing list software at this point. A possible workaround which works in almost all cases would be to make sure that Mailman performs as little message munging as possible: Do not strip MIME multiparts, do not rewrite the subject line to add prefixes, do not rewrite To: and Cc: lines, and so on. With such a configuration, the message would have to assert in a DKIM signature that it does not have a List-Id (or similar) header, but that this is really a fringe case. I believe the OpenJDK lists use such a workaround. Microsoft and Google participate there (as far as I know, both microsoft.com and google.com have restrictive sender policies), and their messages are only dropped if Mailman filters out a text/html multipart, therefore editing the MIME headers. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-07 9:53 ` Florian Weimer @ 2020-04-08 12:38 ` Mark Wielaard 2020-04-08 13:50 ` Michael Matz 0 siblings, 1 reply; 76+ messages in thread From: Mark Wielaard @ 2020-04-08 12:38 UTC (permalink / raw) To: Overseers mailing list, Jonathan Wakely Cc: Florian Weimer, overseers, Frank Ch. Eigler via Gcc, Segher Boessenkool, Alexander Monakov, Maciej W. Rozycki Hi, On Tue, 2020-04-07 at 11:53 +0200, Florian Weimer via Overseers wrote: > Gmail can drop mail for any reason. It's totally opaque, so it's a > poor benchmark for any mailing list configuration changes because it's > very hard to tell if a particular change is effective or not. > > Many mailing lists have *not* made such changes and continue to work > just fine in the face of restrictive DMARC sender policies and > enforcement at the recipient. > > In general, mail drop rates due to DMARC seem to increase in these two > cases if the original From: header is preserved: > > * The sender (i.e., the domain mentioned in the From: header) > publishes a restrictive DMARC policy and the mailing list strips the > DKIM signature. > > * The sender signs parts of the message that the mailing list alters, > and the mailing list does not strip the DKIM signature. > > If neither scenario applies, it's safe to pass through the message > without munging. The mailing list software can detect this and > restricting the From: header munging to those cases. > > I doubt Mailman 2.x can do this, so it is simply a poor choice as > mailing list software at this point. Earlier versions of Mainman2 had some issues which might accidentally change some headers. But the latest fixes make this possible. It is how the FSF handles DMARC for various GNU mailinglists (by NOT modifying the headers and body and passing through the DKIM signatures): https://lists.gnu.org/archive/html/savannah-hackers-public/2019-06/msg00018.html Cheers, Mark ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-08 12:38 ` Mark Wielaard @ 2020-04-08 13:50 ` Michael Matz 2020-04-08 21:15 ` Segher Boessenkool 0 siblings, 1 reply; 76+ messages in thread From: Michael Matz @ 2020-04-08 13:50 UTC (permalink / raw) To: Overseers mailing list Cc: Mark Wielaard, Jonathan Wakely, overseers, Frank Ch. Eigler via Gcc, Segher Boessenkool, Alexander Monakov, Florian Weimer, Maciej W. Rozycki Hello, On Wed, 8 Apr 2020, Mark Wielaard wrote: > On Tue, 2020-04-07 at 11:53 +0200, Florian Weimer via Overseers wrote: > > Gmail can drop mail for any reason. It's totally opaque, so it's a > > poor benchmark for any mailing list configuration changes because it's > > very hard to tell if a particular change is effective or not. > > > > Many mailing lists have *not* made such changes and continue to work > > just fine in the face of restrictive DMARC sender policies and > > enforcement at the recipient. > > > > In general, mail drop rates due to DMARC seem to increase in these two > > cases if the original From: header is preserved: > > > > * The sender (i.e., the domain mentioned in the From: header) > > publishes a restrictive DMARC policy and the mailing list strips the > > DKIM signature. > > > > * The sender signs parts of the message that the mailing list alters, > > and the mailing list does not strip the DKIM signature. > > > > If neither scenario applies, it's safe to pass through the message > > without munging. The mailing list software can detect this and > > restricting the From: header munging to those cases. > > > > I doubt Mailman 2.x can do this, so it is simply a poor choice as > > mailing list software at this point. > > Earlier versions of Mainman2 had some issues which might accidentally > change some headers. But the latest fixes make this possible. It is how > the FSF handles DMARC for various GNU mailinglists (by NOT modifying the > headers and body and passing through the DKIM signatures): > https://lists.gnu.org/archive/html/savannah-hackers-public/2019-06/msg00018.html Oh, that would be nice to have at sourceware.org. Please? :-) Ciao, Michael. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-08 13:50 ` Michael Matz @ 2020-04-08 21:15 ` Segher Boessenkool 2020-04-13 19:47 ` Christopher Faylor 0 siblings, 1 reply; 76+ messages in thread From: Segher Boessenkool @ 2020-04-08 21:15 UTC (permalink / raw) To: Michael Matz Cc: Overseers mailing list, Mark Wielaard, Jonathan Wakely, overseers, Frank Ch. Eigler via Gcc, Alexander Monakov, Florian Weimer, Maciej W. Rozycki Hi! On Wed, Apr 08, 2020 at 01:50:51PM +0000, Michael Matz wrote: > On Wed, 8 Apr 2020, Mark Wielaard wrote: > > Earlier versions of Mainman2 had some issues which might accidentally > > change some headers. But the latest fixes make this possible. It is how > > the FSF handles DMARC for various GNU mailinglists (by NOT modifying the > > headers and body and passing through the DKIM signatures): > > https://lists.gnu.org/archive/html/savannah-hackers-public/2019-06/msg00018.html > > Oh, that would be nice to have at sourceware.org. Please? :-) Yes, please please please, can we have this? Segher ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-08 21:15 ` Segher Boessenkool @ 2020-04-13 19:47 ` Christopher Faylor 2020-04-13 22:53 ` Michael Matz 0 siblings, 1 reply; 76+ messages in thread From: Christopher Faylor @ 2020-04-13 19:47 UTC (permalink / raw) To: Segher Boessenkool Cc: Michael Matz, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Mark Wielaard, Alexander Monakov, Jonathan Wakely, Florian Weimer, Maciej W. Rozycki On Wed, Apr 08, 2020 at 04:15:27PM -0500, Segher Boessenkool wrote: >On Wed, Apr 08, 2020 at 01:50:51PM +0000, Michael Matz wrote: >>On Wed, 8 Apr 2020, Mark Wielaard wrote: >>>Earlier versions of Mainman2 had some issues which might accidentally >>>change some headers. But the latest fixes make this possible. It is >>>how the FSF handles DMARC for various GNU mailinglists (by NOT >>>modifying the headers and body and passing through the DKIM >>>signatures): >>>https://lists.gnu.org/archive/html/savannah-hackers-public/2019-06/msg00018.html >> >>Oh, that would be nice to have at sourceware.org. Please? :-) > >Yes, please please please, can we have this? In case it isn't obvious, we are already running the latest available version of mailman 2. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-13 19:47 ` Christopher Faylor @ 2020-04-13 22:53 ` Michael Matz 2020-04-14 21:06 ` Christopher Faylor 0 siblings, 1 reply; 76+ messages in thread From: Michael Matz @ 2020-04-13 22:53 UTC (permalink / raw) To: overseers Cc: Segher Boessenkool, Frank Ch. Eigler via Gcc, Overseers mailing list, Mark Wielaard, Alexander Monakov, Jonathan Wakely, Florian Weimer, Maciej W. Rozycki Hello, On Mon, 13 Apr 2020, Christopher Faylor wrote: > On Wed, Apr 08, 2020 at 04:15:27PM -0500, Segher Boessenkool wrote: > >On Wed, Apr 08, 2020 at 01:50:51PM +0000, Michael Matz wrote: > >>On Wed, 8 Apr 2020, Mark Wielaard wrote: > >>>Earlier versions of Mainman2 had some issues which might accidentally > >>>change some headers. But the latest fixes make this possible. It is > >>>how the FSF handles DMARC for various GNU mailinglists (by NOT > >>>modifying the headers and body and passing through the DKIM > >>>signatures): > >>>https://lists.gnu.org/archive/html/savannah-hackers-public/2019-06/msg00018.html > >> > >>Oh, that would be nice to have at sourceware.org. Please? :-) > > > >Yes, please please please, can we have this? > > In case it isn't obvious, we are already running the latest available > version of mailman 2. I think that means that dmarc_moderation_action: "Munge From" can simply be switched off then (at least I don't see which other headers e.g. gcc@ is rewriting that would cause DMARC to scream; and if there are any, then it would be better to disable those as well. Same with any potential body rewriting that might still happen). I would offer help testing that this doesn't cause delivery issues, e.g. on some test email list, but it seems none of my domains is DMARC-infected :-/ Ciao, Michael. P.S: I wonder btw. why the From munging is enabled also for p=none domains like redhat.com. The RFC says this is to be used for gathering DMARC feedback, not requiring any specific action for the mail text itself on the sender or receiver. But an answer to this would be moot with the above non-munging of From. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-13 22:53 ` Michael Matz @ 2020-04-14 21:06 ` Christopher Faylor 2020-04-14 21:38 ` Maciej W. Rozycki 0 siblings, 1 reply; 76+ messages in thread From: Christopher Faylor @ 2020-04-14 21:06 UTC (permalink / raw) To: Michael Matz Cc: Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Mark Wielaard, Alexander Monakov, Jonathan Wakely, Florian Weimer, Maciej W. Rozycki On Mon, Apr 13, 2020 at 10:53:06PM +0000, Michael Matz wrote: >On Mon, 13 Apr 2020, Christopher Faylor wrote: >>On Wed, Apr 08, 2020 at 04:15:27PM -0500, Segher Boessenkool wrote: >>>On Wed, Apr 08, 2020 at 01:50:51PM +0000, Michael Matz wrote: >>>>On Wed, 8 Apr 2020, Mark Wielaard wrote: >>>>>Earlier versions of Mainman2 had some issues which might accidentally >>>>>change some headers. But the latest fixes make this possible. It is >>>>>how the FSF handles DMARC for various GNU mailinglists (by NOT >>>>>modifying the headers and body and passing through the DKIM >>>>>signatures): >>>>>https://lists.gnu.org/archive/html/savannah-hackers-public/2019-06/msg00018.html >>>> >>>>Oh, that would be nice to have at sourceware.org. Please? :-) >>> >>>Yes, please please please, can we have this? >> >>In case it isn't obvious, we are already running the latest available >>version of mailman 2. > >I think that means that dmarc_moderation_action: "Munge From" can >simply be switched off then (at least I don't see which other headers >e.g. gcc@ is rewriting that would cause DMARC to scream; and if there >are any, then it would be better to disable those as well. Same with >any potential body rewriting that might still happen). It can't be switched off if we strip html attachments. >I would offer help testing that this doesn't cause delivery issues, e.g. >on some test email list, but it seems none of my domains is DMARC-infected >:-/ Nevertheless, offers of help are appreciated (and rare). :-) cgf ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-14 21:06 ` Christopher Faylor @ 2020-04-14 21:38 ` Maciej W. Rozycki 2020-04-15 2:59 ` Christopher Faylor 0 siblings, 1 reply; 76+ messages in thread From: Maciej W. Rozycki @ 2020-04-14 21:38 UTC (permalink / raw) Cc: Michael Matz, Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Mark Wielaard, Alexander Monakov, Jonathan Wakely, Florian Weimer On Tue, 14 Apr 2020, Christopher Faylor wrote: > >I think that means that dmarc_moderation_action: "Munge From" can > >simply be switched off then (at least I don't see which other headers > >e.g. gcc@ is rewriting that would cause DMARC to scream; and if there > >are any, then it would be better to disable those as well. Same with > >any potential body rewriting that might still happen). > > It can't be switched off if we strip html attachments. Based on current usage would instead plain rejecting e-mail with any HTML attachments be a viable way to move forward? > >I would offer help testing that this doesn't cause delivery issues, e.g. > >on some test email list, but it seems none of my domains is DMARC-infected > >:-/ > > Nevertheless, offers of help are appreciated (and rare). :-) I'm in a `p=none' domain at this e-mail address, so technically I would say it has been quite reasonably configured and therefore none of our people should have to choose between suffering from `From' munging or persuading the postmasters to change the configuration or switching to another MTA for mailing list submissions. Therefore I'd be happy to participate in testing a solution for disabling `From' munging for `p=none' domains. Maciej ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-14 21:38 ` Maciej W. Rozycki @ 2020-04-15 2:59 ` Christopher Faylor 0 siblings, 0 replies; 76+ messages in thread From: Christopher Faylor @ 2020-04-15 2:59 UTC (permalink / raw) To: Maciej W. Rozycki Cc: Frank Ch. Eigler via Gcc, Overseers mailing list, Michael Matz, Segher Boessenkool, Mark Wielaard, Alexander Monakov, Jonathan Wakely, Florian Weimer On Tue, Apr 14, 2020 at 10:38:12PM +0100, Maciej W. Rozycki wrote: >Therefore I'd be happy to participate in testing a solution for >disabling `From' munging for `p=none' domains. I can't speak for fche but I have a limited amount of time for any computer activity right now. A pinched nerve in my neck makes typing difficult. However, as a first step, please subscribe to test-list@sourceware.org. If we do any tinkering it will be there. I'll try to play with settings in the next few days. I think we can bounce email which seems to be html. As I'm sure you all know, we used to do that on server1. Then we switched to stripping the html attachment if there was a text component to the email. At the time, I don't think I had the slightest notion that there would be ramifications for DMARC. Is this something that needs to be voted on? I don't care either way. I think fche would probably not mind letting html through. Do other mailing lists have policies about this? Is it a given that all of the technical mailing lists would be ok with sending text-only? I know it greatly confuses cygwin users but many of them are not that technical. cgf ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding [not found] ` <CAH6eHdTqJEM90d9+nNWrAa_zZrQQha79oMbnbVsRt_2Zd90PdA@mail.gmail.com> 2020-04-07 9:53 ` Florian Weimer @ 2020-04-07 15:13 ` Michael Matz 2020-04-07 15:22 ` Frank Ch. Eigler 2020-04-07 16:08 ` Christopher Faylor 1 sibling, 2 replies; 76+ messages in thread From: Michael Matz @ 2020-04-07 15:13 UTC (permalink / raw) To: Jonathan Wakely Cc: Maciej W. Rozycki, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Frank Ch. Eigler, Florian Weimer Hello, On Tue, 7 Apr 2020, Jonathan Wakely via Gcc wrote: > On Mon, 6 Apr 2020 at 23:00, Maciej W. Rozycki via Gcc <gcc@gcc.gnu.org> wrote: > > And can certainly score a positive though not a definite rating in spam > > qualification. I don't think we ought to encourage bad IT management > > practices by trying to adapt to them too hard and hurting ourselves (our > > workflow) in the process. > > What you call "bad IT management practices" includes how Gmail works, > which a HUGE number of people use. > > A number of lists I'm on switched to our current style of minging a > year or two ago, because gmail users were not receiving mail, because > gmail was rejecting the mail. I find that unconvincing, because even googlegroup email lists don't mangle From: from sender domains that are now mangled by sourceware.org :-/ Can we please switch it off? It's not like we really had a problem before the switch to mailman. Ciao, Michael. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-07 15:13 ` Michael Matz @ 2020-04-07 15:22 ` Frank Ch. Eigler 2020-04-07 15:56 ` Michael Matz 2020-04-07 16:08 ` Christopher Faylor 1 sibling, 1 reply; 76+ messages in thread From: Frank Ch. Eigler @ 2020-04-07 15:22 UTC (permalink / raw) To: Michael Matz Cc: Jonathan Wakely, Maciej W. Rozycki, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Florian Weimer Hi - > > A number of lists I'm on switched to our current style of minging a > > year or two ago, because gmail users were not receiving mail, because > > gmail was rejecting the mail. > > I find that unconvincing, because even googlegroup email lists don't > mangle From: from sender domains that are now mangled by sourceware.org > :-/ It turns out receiving mail FROM google-groups mail is itself sometimes at risk because it fails to do this From: mangling, and its ARC/DKIM re-signature of mail requires even more software to process and bless. (Its current behaviour on some groups-gmail lists I'm on are DMARC non-compliant.) > Can we please switch it off? It's not like we really had a problem > before the switch to mailman. We have offered some first-hand evidence that there were problems, just they were worked on by people in the background. - FChE ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-07 15:22 ` Frank Ch. Eigler @ 2020-04-07 15:56 ` Michael Matz 2020-04-07 16:09 ` Christopher Faylor 0 siblings, 1 reply; 76+ messages in thread From: Michael Matz @ 2020-04-07 15:56 UTC (permalink / raw) To: Frank Ch. Eigler Cc: Jonathan Wakely, Maciej W. Rozycki, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Frank Ch. Eigler, Florian Weimer Hello, On Tue, 7 Apr 2020, Frank Ch. Eigler wrote: > > I find that unconvincing, because even googlegroup email lists don't > > mangle From: from sender domains that are now mangled by sourceware.org > > :-/ > > It turns out receiving mail FROM google-groups mail is itself > sometimes at risk because it fails to do this From: mangling, and its > ARC/DKIM re-signature of mail requires even more software to process > and bless. (Its current behaviour on some groups-gmail lists I'm on > are DMARC non-compliant.) In a way that's amusing and just reinforces my p.o.v. that DMARC is bollocks. > > Can we please switch it off? It's not like we really had a problem > > before the switch to mailman. > > We have offered some first-hand evidence that there were problems, > just they were worked on by people in the background. Okay, now the question is, are those problems offsetting the current problems? IMHO they don't, but of course I'm heavily biased, not having had those old problems :) Ciao, Michael. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-07 15:56 ` Michael Matz @ 2020-04-07 16:09 ` Christopher Faylor 2020-04-07 17:08 ` Maciej W. Rozycki 0 siblings, 1 reply; 76+ messages in thread From: Christopher Faylor @ 2020-04-07 16:09 UTC (permalink / raw) To: Michael Matz Cc: Frank Ch. Eigler, Frank Ch. Eigler via Gcc, Jonathan Wakely, Segher Boessenkool, Alexander Monakov, Overseers mailing list, Florian Weimer, Maciej W. Rozycki On Tue, Apr 07, 2020 at 03:56:09PM +0000, Michael Matz wrote: >In a way that's amusing and just reinforces my p.o.v. that DMARC is >bollocks. Not that it means anything but I agree 100%. It's like whoever made the "standard" just said "to hell with mailing lists". ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-07 16:09 ` Christopher Faylor @ 2020-04-07 17:08 ` Maciej W. Rozycki 2020-04-09 18:19 ` Florian Weimer 0 siblings, 1 reply; 76+ messages in thread From: Maciej W. Rozycki @ 2020-04-07 17:08 UTC (permalink / raw) Cc: Michael Matz, Frank Ch. Eigler, Frank Ch. Eigler via Gcc, Jonathan Wakely, Segher Boessenkool, Alexander Monakov, Overseers mailing list, Florian Weimer On Tue, 7 Apr 2020, Christopher Faylor wrote: > >In a way that's amusing and just reinforces my p.o.v. that DMARC is > >bollocks. > > Not that it means anything but I agree 100%. > > It's like whoever made the "standard" just said "to hell with mailing > lists". Maybe they just didn't know of their existence. Seriously. I have actually been more and more astonished recently with the discovery of people living in some kind of a parallel reality of Internet standards and practices, who never heard of solutions and habits I have known since forever and considered obvious. Especially as we now have an entirely new generation of IT people who never actually saw the Internet like it was back in 1990s and earlier on, simply because it was before they were born. Consequently things that used to work since forever start breaking and people you report such breakage to have no clue as to what you're talking about. Maciej ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-07 17:08 ` Maciej W. Rozycki @ 2020-04-09 18:19 ` Florian Weimer 0 siblings, 0 replies; 76+ messages in thread From: Florian Weimer @ 2020-04-09 18:19 UTC (permalink / raw) To: Maciej W. Rozycki Cc: Michael Matz, Frank Ch. Eigler, Frank Ch. Eigler via Gcc, Jonathan Wakely, Segher Boessenkool, Alexander Monakov, Overseers mailing list * Maciej W. Rozycki: > On Tue, 7 Apr 2020, Christopher Faylor wrote: > >> >In a way that's amusing and just reinforces my p.o.v. that DMARC is >> >bollocks. >> >> Not that it means anything but I agree 100%. >> >> It's like whoever made the "standard" just said "to hell with mailing >> lists". > > Maybe they just didn't know of their existence. Seriously. I think that's extremely unlikely. If you do not do header munging in the mailing list software, things will work. That suggests to me that the people who worked on DKIM and DMARC had this use case in mind. But of course if you add subject prefixes, rewrite attachments (and thus the message body), or enable the Mailman duplicate suppression feature (which edits To:/Cc: lists of the messages posted to the list), then things will break. But that's not required to run a mailing list. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-07 15:13 ` Michael Matz 2020-04-07 15:22 ` Frank Ch. Eigler @ 2020-04-07 16:08 ` Christopher Faylor 2020-04-07 16:58 ` Maciej W. Rozycki 1 sibling, 1 reply; 76+ messages in thread From: Christopher Faylor @ 2020-04-07 16:08 UTC (permalink / raw) To: Michael Matz Cc: Jonathan Wakely, Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Florian Weimer, Maciej W. Rozycki On Tue, Apr 07, 2020 at 03:13:53PM +0000, Michael Matz wrote: >Can we please switch it off? It's not like we really had a problem before >the switch to mailman. You can't really make statements like this which imply that you are aware of "problems" on sourceware when you're not a sourceware administrator. We actually were munging the From on the old server and we were getting an increasing number of complaints to postmaster that people weren't receiving mail. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-07 16:08 ` Christopher Faylor @ 2020-04-07 16:58 ` Maciej W. Rozycki 0 siblings, 0 replies; 76+ messages in thread From: Maciej W. Rozycki @ 2020-04-07 16:58 UTC (permalink / raw) Cc: Michael Matz, Jonathan Wakely, Frank Ch. Eigler via Gcc, Overseers mailing list, Segher Boessenkool, Alexander Monakov, Florian Weimer On Tue, 7 Apr 2020, Christopher Faylor wrote: > >Can we please switch it off? It's not like we really had a problem before > >the switch to mailman. > > You can't really make statements like this which imply that you are > aware of "problems" on sourceware when you're not a sourceware > administrator. > > We actually were munging the From on the old server and we were getting > an increasing number of complaints to postmaster that people weren't > receiving mail. Hmm, methinks these really ought to be bounced to the postmaster of the broken receiving site. It is not that sourceware has been actually doing anything specific to make the recipient's MTA refuse e-mail. Even if some spam prevention has been installed to block some kind of e-mail, whoever requested some kind of e-mail traffic need to have a way to whitelist the traffic requested, and I believe all list server software provides means for the recipient to identify the source, such as a `List-ID' header or suchlike. Maciej ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-04-06 20:58 ` Maciej W. Rozycki 2020-04-06 21:09 ` Frank Ch. Eigler @ 2020-04-06 23:15 ` Segher Boessenkool 1 sibling, 0 replies; 76+ messages in thread From: Segher Boessenkool @ 2020-04-06 23:15 UTC (permalink / raw) To: Maciej W. Rozycki Cc: Frank Ch. Eigler, Bernd Schmidt, overseers, Frank Ch. Eigler via Gcc, Overseers mailing list, Alexander Monakov, Frank Ch. Eigler, Florian Weimer Hi! On Mon, Apr 06, 2020 at 09:58:27PM +0100, Maciej W. Rozycki wrote: > On Wed, 18 Mar 2020, Frank Ch. Eigler via Gcc wrote: > > > Out of curiousity, is this rewriting you are talking about the cause for a > > > lot of mails showing up as "From: GCC List" rather than their real senders? > > > This has become very annoying recently. > > > > Yes, for emails from domains with declared interest in email > > cleanliness, via DMARC records in DNS. We have observed mail > > -blocked- at third parties, even just days ago, when we failed to > > sufficiently authenticate outgoing reflected emails. > > > > AIUI, all this effort is driven by wanting to defeat not just spammers > > but also real security problems like phishing enabled by forgery, > > including specifically the From: header. > > And it has actually broken GCC's patchwork: > <http://patchwork.ozlabs.org/project/gcc/list/>, which I used to use to > track my patches. Yes, I noticed (I run that patchwork). > Now I cannot do that anymore as patches submitted from > my WDC address are marked as coming from <gcc-patches@gcc.gnu.org>, which > obviously means they are not attributed to me. Yup. > I am fairly sure it breaks `git am' too, requiring a `From' override in > the change description for author attribution in patch application to work > reliably (I tend to work on my outbox when applying my own patches, so I > avoid this issue, but I am sure the issue will hit someone sooner or > later). Yup. > And of course I cannot use the `macro@' pattern anymore to select mailing > list messages in my inbox that I sent myself. Frankly it's the least of > the annoyances, but still, and they all add up. It would break even the built-in mutt "you sent this yourself" detection. > This is all silly, requiring the SMTP envelope sender to match the `From' > header breaks even the most basic e-mail mechanisms like the use of a > `.forward' file. This even *never* is true for my mail setups, and never has been, and it all works fine. > Can we please do something about it? Is functional > regression the price we absolutely need to pay for progress? > > How come the Linux kernel people who do e-mail patch management on a > vastly larger scale than we do, both in terms of traffic and the number of > mailing list subscribers, can get away without all these odd quirks in > their list server management? Perhaps it would be good asking them how > they handle their stuff? If someone cannot connect to them because they have a broken mail setup, the kernel people say "your problem, get a working mail setup", and that works fine so far (99.999% of people *can* get working mail, just not always company mail). Segher ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Not usable email content encoding 2020-03-17 19:46 ` Segher Boessenkool 2020-03-17 19:51 ` Frank Ch. Eigler @ 2020-03-21 22:29 ` Maciej W. Rozycki 1 sibling, 0 replies; 76+ messages in thread From: Maciej W. Rozycki @ 2020-03-21 22:29 UTC (permalink / raw) To: Segher Boessenkool; +Cc: Alexander Monakov, overseers, GCC Development On Tue, 17 Mar 2020, Segher Boessenkool wrote: > > I'm surprised it's an issue for you: normally your email client > > would transform quoted-printable and copying would do the right thing > > (i.e. select actual patch contents, without whitespace munging). > > > > Are you trying to copy from the raw message representation? > > Everyone trying to work with a patch (instead of just the email) always > is working with the raw message. Just patch < mbox or git-am mbox > for example. Well, not everyone obviously as I don't, so please be careful speaking quantifiers. I apply inline patches with `git am -' by piping messages or attachments to the command directly from my e-mail client (with its `Pipe' command), so I don't care how any message or attachment has been encoded for transport as long as the patch hasn't come out damaged once decoded. I'd expect any sane e-mail client to support a similar command if it were to be used for software development the way we do it, though I realise that regrettably some may not be as capable. Maciej ^ permalink raw reply [flat|nested] 76+ messages in thread
end of thread, other threads:[~2020-05-02 12:50 UTC | newest] Thread overview: 76+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-03-16 13:45 Not usable email content encoding Martin Liška 2020-03-16 13:47 ` Richard Earnshaw 2020-03-16 13:52 ` Martin Liška 2020-03-16 13:51 ` Florian Weimer 2020-03-16 13:52 ` Frank Ch. Eigler 2020-03-16 13:54 ` Alexander Monakov 2020-03-16 14:11 ` Martin Liška 2020-03-16 14:18 ` Jonathan Wakely 2020-03-17 19:46 ` Segher Boessenkool 2020-03-17 19:51 ` Frank Ch. Eigler 2020-03-17 21:45 ` Segher Boessenkool 2020-03-18 10:38 ` Florian Weimer 2020-03-18 11:01 ` Frank Ch. Eigler 2020-03-18 11:39 ` Florian Weimer 2020-03-18 14:19 ` Michael Matz 2020-03-18 14:22 ` Frank Ch. Eigler 2020-03-18 15:15 ` Bernd Schmidt 2020-03-18 16:22 ` Frank Ch. Eigler 2020-03-18 16:42 ` Michael Matz 2020-03-18 17:36 ` Christopher Faylor 2020-03-18 21:52 ` Jim Wilson 2020-03-18 22:29 ` Jonathan Wakely 2020-03-18 22:34 ` Jonathan Wakely 2020-03-18 22:44 ` Frank Ch. Eigler 2020-03-19 3:30 ` Christopher Faylor 2020-03-19 3:32 ` Christopher Faylor 2020-03-18 22:45 ` Joseph Myers 2020-03-18 23:02 ` Jonathan Wakely 2020-03-19 13:05 ` Tom Tromey 2020-03-19 13:26 ` Florian Weimer 2020-03-19 13:41 ` Richard Biener 2020-03-19 13:58 ` Florian Weimer 2020-03-19 18:17 ` Segher Boessenkool 2020-04-23 0:54 ` Tamar Christina 2020-04-23 5:47 ` Florian Weimer 2020-04-23 7:34 ` Richard Biener [not found] ` <CAH6eHdShSXwc4K7ar9HWgHu12tDQe-s2LifdvmVp8VytwYV-ew@mail.gmail.com> 2020-04-23 8:39 ` Jakub Jelinek 2020-04-23 15:32 ` Jeff Law 2020-04-23 9:55 ` Tamar Christina 2020-04-23 11:46 ` Segher Boessenkool [not found] ` <CAH6eHdTHsZWO9vuycKm22dDbSGSfzcrKbzEpb6vNdb-zXCiKjQ@mail.gmail.com> 2020-04-23 12:30 ` Segher Boessenkool 2020-04-23 15:33 ` Jeff Law 2020-04-23 16:57 ` Frank Ch. Eigler 2020-04-23 20:32 ` Segher Boessenkool 2020-04-23 20:27 ` Segher Boessenkool 2020-04-23 21:14 ` Tom Tromey 2020-04-24 15:48 ` Thomas Koenig 2020-04-24 17:09 ` Segher Boessenkool 2020-05-01 15:51 ` Jeff Law 2020-05-02 12:50 ` Segher Boessenkool 2020-03-19 18:08 ` Segher Boessenkool 2020-03-18 22:33 ` Frank Ch. Eigler 2020-03-19 0:17 ` Segher Boessenkool 2020-03-19 0:32 ` Segher Boessenkool 2020-04-06 20:58 ` Maciej W. Rozycki 2020-04-06 21:09 ` Frank Ch. Eigler 2020-04-06 21:59 ` Maciej W. Rozycki [not found] ` <CAH6eHdTqJEM90d9+nNWrAa_zZrQQha79oMbnbVsRt_2Zd90PdA@mail.gmail.com> 2020-04-07 9:53 ` Florian Weimer 2020-04-08 12:38 ` Mark Wielaard 2020-04-08 13:50 ` Michael Matz 2020-04-08 21:15 ` Segher Boessenkool 2020-04-13 19:47 ` Christopher Faylor 2020-04-13 22:53 ` Michael Matz 2020-04-14 21:06 ` Christopher Faylor 2020-04-14 21:38 ` Maciej W. Rozycki 2020-04-15 2:59 ` Christopher Faylor 2020-04-07 15:13 ` Michael Matz 2020-04-07 15:22 ` Frank Ch. Eigler 2020-04-07 15:56 ` Michael Matz 2020-04-07 16:09 ` Christopher Faylor 2020-04-07 17:08 ` Maciej W. Rozycki 2020-04-09 18:19 ` Florian Weimer 2020-04-07 16:08 ` Christopher Faylor 2020-04-07 16:58 ` Maciej W. Rozycki 2020-04-06 23:15 ` Segher Boessenkool 2020-03-21 22:29 ` Maciej W. Rozycki
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).