[RFC] Auto-posting committed patches to this list?

[RFC] Auto-posting committed patches to this list?

[Siddhesh Poyarekar]

Hi,

There are quite a few patches that get pushed to git with minor 
modifications; it's a convenient workflow hack since it allows 
committers to push trivial changes when they don't really need 
additional review.

That however breaks patchwork patch detection since the checksums 
against the diffs don't match.  There are bugs in patchwork (or to see 
it another way, bugs in email clients) that prevent this too, but 
trivial changes are a pretty big percentage.

I have a wrapper script that, whenever it updates my repo, checks for 
the patchwork patch for each commit and updates its state.  I currently 
do it on need basis, but I could make it a cron job.  How annoying would 
it be for folks here if the script, upon not finding a suitable patch, 
email it to the list as [committed]?

As a bargain chip, one would no longer have to send an email whenever 
they push trivial changes to the repo; my sync would do it for them.

This is not just to make the patchwork backlog cleaner, but also to make 
sure that the community is aware of the final version of the patch that 
went in; they're easier to correlate in the same list since they'd 
typically have the same subject line.

This should eventually become a post-receive hook in the upstream repo, 
but I want to see if this workflow actually works for us before 
proposing this deeper change.

Thoughts?

Siddhesh

Re: [RFC] Auto-posting committed patches to this list?

[DJ Delorie]

How much of the problem space would be solved if we only emailed commit
"patches" directly to the patchwork system?  And how difficult would it
be to get patchwork to properly match independent commit mails to
existing patch series?

Re: [RFC] Auto-posting committed patches to this list?

[Siddhesh Poyarekar]

On 3/11/21 10:03 AM, DJ Delorie wrote:
> 
> How much of the problem space would be solved if we only emailed commit
> "patches" directly to the patchwork system?  And how difficult would it
> be to get patchwork to properly match independent commit mails to
> existing patch series?
> 
Currently the matching logic depends on a checksum based on the diff 
part of the patch with the diff part of the commit.  There are some 
limitations in patchwork email parsing because of which on occasion it 
doesn't read the diff correctly, but it largely works.  In general, 
patches sent using git-send-email are parsed correctly and should result 
in correct matching.

There are two problems I'm trying to solve, one is of 1:1 correspondence 
between patchwork state and repo state and that will be solved by 
feeding commit patches directly to patchwork.

The other problem is visibility of commits that don't match mailing list 
content.  We allow, as part of policy, trivial changes to patches before 
they're committed but ideally one should always post the final version 
before commit.  We enforce this ideal case for patches that only contain 
trivial changes.  Having commits that don't match any existing patches 
in patchwork be posted on the mailing list for visibility plugs this 
gap, giving us confidence that every commit that went into the repo was 
at at least one point advertised on the list.

Siddhesh

Re: [RFC] Auto-posting committed patches to this list?

[DJ Delorie]

Siddhesh Poyarekar <siddhesh@gotplt.org> writes:
> There are two problems I'm trying to solve, one is of 1:1 correspondence 
> between patchwork state and repo state and that will be solved by 
> feeding commit patches directly to patchwork.

On this one, I wonder how well patchwork can guess which patch series
corresponds to a commit that doesn't exactly match any of them.

Re: [RFC] Auto-posting committed patches to this list?

[Siddhesh Poyarekar]

On 3/11/21 10:07 PM, DJ Delorie wrote:
> Siddhesh Poyarekar <siddhesh@gotplt.org> writes:
>> There are two problems I'm trying to solve, one is of 1:1 correspondence
>> between patchwork state and repo state and that will be solved by
>> feeding commit patches directly to patchwork.
> 
> On this one, I wonder how well patchwork can guess which patch series
> corresponds to a commit that doesn't exactly match any of them.

It doesn't at the moment, but that's an orthogonal issue and can only be 
solved with a move to something like gitlab.

Siddhesh

Re: [RFC] Auto-posting committed patches to this list?

[Paul Zimmermann]

       Dear Siddhesh,

do you have an idea by how much the traffic on the list would increase?
If only by a few percents, that would be fine with me.

Best regards,
Paul

Re: [RFC] Auto-posting committed patches to this list?

[Siddhesh Poyarekar]

On 3/15/21 8:29 PM, Paul Zimmermann wrote:
>         Dear Siddhesh,
> 
> do you have an idea by how much the traffic on the list would increase?
> If only by a few percents, that would be fine with me.

Shouldn't be a lot, about 70% of the patches get committed as is, so 
it's only about 30% that bleed through.  It should be way under 10 
emails a week given the frequency of our commits.

Siddhesh

Re: [RFC] Auto-posting committed patches to this list?

[Carlos O'Donell]

On 3/9/21 1:28 PM, Siddhesh Poyarekar wrote:
> There are quite a few patches that get pushed to git with minor
> modifications; it's a convenient workflow hack since it allows
> committers to push trivial changes when they don't really need
> additional review.

Agreed.

We often say "OK with this change..." and it would be lame to force
those people to repost vN just to go ahead and commit it right after
as accepted and then have to find it in patchwork and mark it committed.
 
> That however breaks patchwork patch detection since the checksums
> against the diffs don't match.  There are bugs in patchwork (or to
> see it another way, bugs in email clients) that prevent this too, but
> trivial changes are a pretty big percentage.

It technically doesn't break patchwork patch detection.

*Technically* it is a distinct new version of the patch.

The old version, even in your new method, still needs to be marked as
SUPERSEDED or DROPPED?

So we'll never clean the old versions out unless people maintain the
queues clean, or we notice it in the weekly review and remove it.

> I have a wrapper script that, whenever it updates my repo, checks for
> the patchwork patch for each commit and updates its state.  I
> currently do it on need basis, but I could make it a cron job.  How
> annoying would it be for folks here if the script, upon not finding a
> suitable patch, email it to the list as [committed]?

Not annoying at all for me.
 
> As a bargain chip, one would no longer have to send an email whenever
> they push trivial changes to the repo; my sync would do it for them.

Exactly. Less work is better.
 
> This is not just to make the patchwork backlog cleaner, but also to
> make sure that the community is aware of the final version of the
> patch that went in; they're easier to correlate in the same list
> since they'd typically have the same subject line.

How will it make the patchwork backlog cleaner?

> This should eventually become a post-receive hook in the upstream
> repo, but I want to see if this workflow actually works for us before
> proposing this deeper change.

Agreed.

-- 
Cheers,
Carlos.

Re: [RFC] Auto-posting committed patches to this list?

[Siddhesh Poyarekar]

On 3/16/21 7:54 AM, Carlos O'Donell wrote:
>> This is not just to make the patchwork backlog cleaner, but also to
>> make sure that the community is aware of the final version of the
>> patch that went in; they're easier to correlate in the same list
>> since they'd typically have the same subject line.
> 
> How will it make the patchwork backlog cleaner?

This action in itself won't, but it will enable other actions later on. 
  Once we have 1:1 correspondence between patchwork and the repo, we 
could then have more aggressive actions that mark patches with similar 
subject lines (especially ones that are in Changes Requested or some 
other non-terminal state) from the author as Superseded, automating at 
least some of the developer workflow.

I'll set up a cron job to send out these patches first and we can then 
build on it.

Siddhesh

Re: [RFC] Auto-posting committed patches to this list?

[Carlos O'Donell]

On 3/15/21 11:52 PM, Siddhesh Poyarekar wrote:
> On 3/16/21 7:54 AM, Carlos O'Donell wrote:
>>> This is not just to make the patchwork backlog cleaner, but also
>>> to make sure that the community is aware of the final version of
>>> the patch that went in; they're easier to correlate in the same
>>> list since they'd typically have the same subject line.
>> 
>> How will it make the patchwork backlog cleaner?
> 
> This action in itself won't, but it will enable other actions later
> on.  Once we have 1:1 correspondence between patchwork and the repo,
> we could then have more aggressive actions that mark patches with
> similar subject lines (especially ones that are in Changes Requested
> or some other non-terminal state) from the author as Superseded,
> automating at least some of the developer workflow.

Perfect. Completely understood.

> I'll set up a cron job to send out these patches first and we can
> then build on it.

Thanks!

-- 
Cheers,
Carlos.