Git ChangeLog policy for GCC Testsuite inquiry

Git ChangeLog policy for GCC Testsuite inquiry

[David Edelsohn]

>> > On 1/24/20 8:45 AM, David Edelsohn wrote:
>> > > There is no ChangeLog entry for the testsuite changes.
>> >
>> > I don't believe in ChangeLog entries for testcases, but I'll add one for
>> > the target-supports.exp change, thanks.
>>
>> Is this a general policy change that we want to make?  Current we
>> still have gcc/testsuite/ChangeLog and developers are updating that
>> file.
>
> I would support formalizing that as policy; currently there is no policy.
>
> https://gcc.gnu.org/codingconventions.html#ChangeLogs
>
> "There is no established convention on when ChangeLog entries are to be made for testsuite changes."

Do we want to continue with ChangeLog entries for testsuite changes or
only rely on Git log?

Thanks, David

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Jeff Law]

On Fri, 2020-01-24 at 13:49 -0500, David Edelsohn wrote:
> > > > On 1/24/20 8:45 AM, David Edelsohn wrote:
> > > > > There is no ChangeLog entry for the testsuite changes.
> > > > 
> > > > I don't believe in ChangeLog entries for testcases, but I'll add one for
> > > > the target-supports.exp change, thanks.
> > > 
> > > Is this a general policy change that we want to make?  Current we
> > > still have gcc/testsuite/ChangeLog and developers are updating that
> > > file.
> > 
> > I would support formalizing that as policy; currently there is no policy.
> > 
> > https://gcc.gnu.org/codingconventions.html#ChangeLogs
> > 
> > "There is no established convention on when ChangeLog entries are to be made for testsuite changes."
> 
> Do we want to continue with ChangeLog entries for testsuite changes or
> only rely on Git log?
I strongly prefer to move towards relying on the git log.

jeff

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Eric Botcazou]

> I strongly prefer to move towards relying on the git log.

In my experience the output of git log is a total mess so cannot replace 
ChangeLogs.  But we can well decide to drop ChangeLog for the testsuite.

-- 
Eric Botcazou

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Florian Weimer]

* Eric Botcazou:

>> I strongly prefer to move towards relying on the git log.
>
> In my experience the output of git log is a total mess so cannot replace 
> ChangeLogs.

That's fixable if the commit message is part of the patch review
(just like the source code comments).

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Jeff Law]

On Fri, 2020-01-24 at 20:32 +0100, Eric Botcazou wrote:
> > I strongly prefer to move towards relying on the git log.
> 
> In my experience the output of git log is a total mess so cannot replace 
> ChangeLogs.  But we can well decide to drop ChangeLog for the testsuite.
Well, glibc has moved to extracting them from git, building policies
and scripts around that.  I'm pretty sure other significant projecs are
also extracting their ChangeLogs from git.

We could do the same, selecting some magic date as the cutover point
after which future ChangeLogs are extracted from GIT.  In fact, that's
precisely what I'd like to see us do.

jeff

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Jakub Jelinek]

On Fri, Jan 24, 2020 at 02:36:31PM -0700, Jeff Law wrote:
> On Fri, 2020-01-24 at 20:32 +0100, Eric Botcazou wrote:
> > > I strongly prefer to move towards relying on the git log.
> > 
> > In my experience the output of git log is a total mess so cannot replace 
> > ChangeLogs.  But we can well decide to drop ChangeLog for the testsuite.
> Well, glibc has moved to extracting them from git, building policies
> and scripts around that.  I'm pretty sure other significant projecs are
> also extracting their ChangeLogs from git.
> 
> We could do the same, selecting some magic date as the cutover point
> after which future ChangeLogs are extracted from GIT.  In fact, that's
> precisely what I'd like to see us do.

We don't have a tool that can do it, not even get the boilerplate right.
Yes, mklog helps, but it very often gets stuff wrong.  Not to mention that
the text what actually changed can't be generated very easily.

	Jakub

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Paul Smith]

On Fri, 2020-01-24 at 22:45 +0100, Jakub Jelinek wrote:
> > > In my experience the output of git log is a total mess so cannot
> > > replace ChangeLogs.  But we can well decide to drop ChangeLog for
> > > the testsuite.
> > 
> > Well, glibc has moved to extracting them from git, building
> > policies and scripts around that.  I'm pretty sure other
> > significant projecs are also extracting their ChangeLogs from git.
> > 
> > We could do the same, selecting some magic date as the cutover
> > point after which future ChangeLogs are extracted from GIT.  In
> > fact, that's precisely what I'd like to see us do.
> 
> We don't have a tool that can do it, not even get the boilerplate
> right. Yes, mklog helps, but it very often gets stuff wrong.  Not to
> mention that the text what actually changed can't be generated very
> easily.

I don't know if it counts as a significant project, but GNU make has
been doing this for years.

What I did was take the existing ChangeLogs and rename them to
ChangeLog.1 or whatever, then started with a new ChangeLog generated
from scratch from Git messages.

I use the gnulib build-aux/gitlog-to-changelog script to do it.  It
requires a little bit of discipline to get right; in particular you
have to remember that the Git commit message will be indented 8 spaces
in the ChangeLog, so you have to be careful that your commit messages
wrap at char 70 (or less) in your Git commit.

If you have Git hooks you could enforce a bit of formatting; for
example any line not indented by space must be <=70 chars long; this
allows people to use long lines for formatted content if they indent it
with a space or two.

Otherwise, it's the same as writing the ChangeLog and you only have to
do it once.

Just to note, the above script simply transcribes the commit message
into ChangeLog format.  It does NOT try to auto-generate ChangeLog-
style content (files that changed, functions, etc.) from the Git diff
or whatever.

There are a few special tokens you can add to your Git commit message
that get reformated to special changelog tokens like "(tiny change)"
etc.

As mentioned previously, it's very important that the commit message be
provided as part of the code review, and it is very much fair game for
review comments.  This is common practice, and a good idea because bad
commit messages are always a bummer, ChangeLog or not.

Re: Git ChangeLog policy for GCC Testsuite inquiry

[H.J. Lu]

On Fri, Jan 24, 2020 at 2:39 PM Paul Smith <paul@mad-scientist.net> wrote:
>
> On Fri, 2020-01-24 at 22:45 +0100, Jakub Jelinek wrote:
> > > > In my experience the output of git log is a total mess so cannot
> > > > replace ChangeLogs.  But we can well decide to drop ChangeLog for
> > > > the testsuite.
> > >
> > > Well, glibc has moved to extracting them from git, building
> > > policies and scripts around that.  I'm pretty sure other
> > > significant projecs are also extracting their ChangeLogs from git.
> > >
> > > We could do the same, selecting some magic date as the cutover
> > > point after which future ChangeLogs are extracted from GIT.  In
> > > fact, that's precisely what I'd like to see us do.
> >
> > We don't have a tool that can do it, not even get the boilerplate
> > right. Yes, mklog helps, but it very often gets stuff wrong.  Not to
> > mention that the text what actually changed can't be generated very
> > easily.
>
> I don't know if it counts as a significant project, but GNU make has
> been doing this for years.
>
> What I did was take the existing ChangeLogs and rename them to
> ChangeLog.1 or whatever, then started with a new ChangeLog generated
> from scratch from Git messages.
>
> I use the gnulib build-aux/gitlog-to-changelog script to do it.  It
> requires a little bit of discipline to get right; in particular you
> have to remember that the Git commit message will be indented 8 spaces
> in the ChangeLog, so you have to be careful that your commit messages
> wrap at char 70 (or less) in your Git commit.
>
> If you have Git hooks you could enforce a bit of formatting; for
> example any line not indented by space must be <=70 chars long; this
> allows people to use long lines for formatted content if they indent it
> with a space or two.
>
> Otherwise, it's the same as writing the ChangeLog and you only have to
> do it once.
>
> Just to note, the above script simply transcribes the commit message
> into ChangeLog format.  It does NOT try to auto-generate ChangeLog-
> style content (files that changed, functions, etc.) from the Git diff
> or whatever.
>
> There are a few special tokens you can add to your Git commit message
> that get reformated to special changelog tokens like "(tiny change)"
> etc.
>
> As mentioned previously, it's very important that the commit message be
> provided as part of the code review, and it is very much fair game for
> review comments.  This is common practice, and a good idea because bad
> commit messages are always a bummer, ChangeLog or not.
>

Libgcrypt includes ChangeLog entries in git commit messages:

http://git.gnupg.org/cgi-bin/gitweb.cgi?p=libgcrypt.git

In each patch, commit log starts with ChangeLog entries without leading
TABs followed by separator line with -- and then commit message.   They
have a script to extract ChangeLog for release.

--
H.J.

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Richard Sandiford]

"H.J. Lu" <hjl.tools@gmail.com> writes:
> On Fri, Jan 24, 2020 at 2:39 PM Paul Smith <paul@mad-scientist.net> wrote:
>>
>> On Fri, 2020-01-24 at 22:45 +0100, Jakub Jelinek wrote:
>> > > > In my experience the output of git log is a total mess so cannot
>> > > > replace ChangeLogs.  But we can well decide to drop ChangeLog for
>> > > > the testsuite.
>> > >
>> > > Well, glibc has moved to extracting them from git, building
>> > > policies and scripts around that.  I'm pretty sure other
>> > > significant projecs are also extracting their ChangeLogs from git.
>> > >
>> > > We could do the same, selecting some magic date as the cutover
>> > > point after which future ChangeLogs are extracted from GIT.  In
>> > > fact, that's precisely what I'd like to see us do.
>> >
>> > We don't have a tool that can do it, not even get the boilerplate
>> > right. Yes, mklog helps, but it very often gets stuff wrong.  Not to
>> > mention that the text what actually changed can't be generated very
>> > easily.
>>
>> I don't know if it counts as a significant project, but GNU make has
>> been doing this for years.
>>
>> What I did was take the existing ChangeLogs and rename them to
>> ChangeLog.1 or whatever, then started with a new ChangeLog generated
>> from scratch from Git messages.
>>
>> I use the gnulib build-aux/gitlog-to-changelog script to do it.  It
>> requires a little bit of discipline to get right; in particular you
>> have to remember that the Git commit message will be indented 8 spaces
>> in the ChangeLog, so you have to be careful that your commit messages
>> wrap at char 70 (or less) in your Git commit.
>>
>> If you have Git hooks you could enforce a bit of formatting; for
>> example any line not indented by space must be <=70 chars long; this
>> allows people to use long lines for formatted content if they indent it
>> with a space or two.
>>
>> Otherwise, it's the same as writing the ChangeLog and you only have to
>> do it once.
>>
>> Just to note, the above script simply transcribes the commit message
>> into ChangeLog format.  It does NOT try to auto-generate ChangeLog-
>> style content (files that changed, functions, etc.) from the Git diff
>> or whatever.
>>
>> There are a few special tokens you can add to your Git commit message
>> that get reformated to special changelog tokens like "(tiny change)"
>> etc.
>>
>> As mentioned previously, it's very important that the commit message be
>> provided as part of the code review, and it is very much fair game for
>> review comments.  This is common practice, and a good idea because bad
>> commit messages are always a bummer, ChangeLog or not.
>>
>
> Libgcrypt includes ChangeLog entries in git commit messages:
>
> http://git.gnupg.org/cgi-bin/gitweb.cgi?p=libgcrypt.git
>
> In each patch, commit log starts with ChangeLog entries without leading
> TABs followed by separator line with -- and then commit message.   They
> have a script to extract ChangeLog for release.

How many people would we be catering for by generating changelogs at
release time though?  It seems too low-level to be useful to users,
and people wanting to track gcc development history at the source level
would surely be better off using git (which e.g. makes it much easier to
track changes to particular pieces of code).

Perhaps there are practical or policy reasons for not requiring everyone
who wants to track gcc development history to build or install git.
But if so, why not just include the output of "git log", with whatever
options seem best?  (Probably --stat at least, to show the affected files.)

Like with the svn-to-git conversion, the less we change the way the
history is presented, the less chance there is of something going wrong.
And the idea is that git log should be informative enough for upstream
developers, so surely it should be enough for others too.

Thanks,
Richard

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Jeff Law]

On Mon, 2020-02-03 at 18:55 +0000, Richard Sandiford wrote:
> "H.J. Lu" <hjl.tools@gmail.com> writes:
> > On Fri, Jan 24, 2020 at 2:39 PM Paul Smith <paul@mad-scientist.net> wrote:
> > > On Fri, 2020-01-24 at 22:45 +0100, Jakub Jelinek wrote:
> > > > > > In my experience the output of git log is a total mess so cannot
> > > > > > replace ChangeLogs.  But we can well decide to drop ChangeLog for
> > > > > > the testsuite.
> > > > > 
> > > > > Well, glibc has moved to extracting them from git, building
> > > > > policies and scripts around that.  I'm pretty sure other
> > > > > significant projecs are also extracting their ChangeLogs from git.
> > > > > 
> > > > > We could do the same, selecting some magic date as the cutover
> > > > > point after which future ChangeLogs are extracted from GIT.  In
> > > > > fact, that's precisely what I'd like to see us do.
> > > > 
> > > > We don't have a tool that can do it, not even get the boilerplate
> > > > right. Yes, mklog helps, but it very often gets stuff wrong.  Not to
> > > > mention that the text what actually changed can't be generated very
> > > > easily.
> > > 
> > > I don't know if it counts as a significant project, but GNU make has
> > > been doing this for years.
> > > 
> > > What I did was take the existing ChangeLogs and rename them to
> > > ChangeLog.1 or whatever, then started with a new ChangeLog generated
> > > from scratch from Git messages.
> > > 
> > > I use the gnulib build-aux/gitlog-to-changelog script to do it.  It
> > > requires a little bit of discipline to get right; in particular you
> > > have to remember that the Git commit message will be indented 8 spaces
> > > in the ChangeLog, so you have to be careful that your commit messages
> > > wrap at char 70 (or less) in your Git commit.
> > > 
> > > If you have Git hooks you could enforce a bit of formatting; for
> > > example any line not indented by space must be <=70 chars long; this
> > > allows people to use long lines for formatted content if they indent it
> > > with a space or two.
> > > 
> > > Otherwise, it's the same as writing the ChangeLog and you only have to
> > > do it once.
> > > 
> > > Just to note, the above script simply transcribes the commit message
> > > into ChangeLog format.  It does NOT try to auto-generate ChangeLog-
> > > style content (files that changed, functions, etc.) from the Git diff
> > > or whatever.
> > > 
> > > There are a few special tokens you can add to your Git commit message
> > > that get reformated to special changelog tokens like "(tiny change)"
> > > etc.
> > > 
> > > As mentioned previously, it's very important that the commit message be
> > > provided as part of the code review, and it is very much fair game for
> > > review comments.  This is common practice, and a good idea because bad
> > > commit messages are always a bummer, ChangeLog or not.
> > > 
> > 
> > Libgcrypt includes ChangeLog entries in git commit messages:
> > 
> > http://git.gnupg.org/cgi-bin/gitweb.cgi?p=libgcrypt.git
> > 
> > In each patch, commit log starts with ChangeLog entries without leading
> > TABs followed by separator line with -- and then commit message.   They
> > have a script to extract ChangeLog for release.
> 
> How many people would we be catering for by generating changelogs at
> release time though?  It seems too low-level to be useful to users,
> and people wanting to track gcc development history at the source level
> would surely be better off using git (which e.g. makes it much easier to
> track changes to particular pieces of code).
> 
> Perhaps there are practical or policy reasons for not requiring everyone
> who wants to track gcc development history to build or install git.
> But if so, why not just include the output of "git log", with whatever
> options seem best?  (Probably --stat at least, to show the affected files.)
> 
> Like with the svn-to-git conversion, the less we change the way the
> history is presented, the less chance there is of something going wrong.
> And the idea is that git log should be informative enough for upstream
> developers, so surely it should be enough for others too.
I believe the ChangeLog is primarily a FSF requirement, hence
generating it from the SCM at release time seems reasonable.

ANd yes, even though I have been a regular ChangeLog user, I rely more
and more on the git log these days.

jeff

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Segher Boessenkool]

On Mon, Feb 03, 2020 at 01:24:04PM -0700, Jeff Law wrote:
> ANd yes, even though I have been a regular ChangeLog user, I rely more
> and more on the git log these days.

As a reviewer, the changelog is priceless still.  We shouldn't drop the
changelog before people write *good* commit messages (and we are still
quite far from that goal).


Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Richard Sandiford]

Segher Boessenkool <segher@kernel.crashing.org> writes:
> On Mon, Feb 03, 2020 at 01:24:04PM -0700, Jeff Law wrote:
>> ANd yes, even though I have been a regular ChangeLog user, I rely more
>> and more on the git log these days.
>
> As a reviewer, the changelog is priceless still.  We shouldn't drop the
> changelog before people write *good* commit messages (and we are still
> quite far from that goal).

But that was the point of what others said upthread: now that the patch
description is part of the commit message, we should review them for
whether they're a good description.  The patch description should be
as much a part of the review as the patch itself.

So IMO the right response to poor patch descriptions is to ask for
a better one.  The goal should be "have good patch descriptions" not
"make all patch descriptions use format X".  The traditional changelog
format should be one acceptable way of writing good patch descriptions,
but not the only acceptable way.

Sometimes the traditional changelog is better than a plain English
description (the SVE PCS work being one recent example from my POV).
But for modern VCSes there's IMO no benefit to a changelog like:

2017-09-05  Richard Sandiford  <richard.sandiford@linaro.org>

	* builtins.c (expand_builtin_powi): Use int_mode_for_size.
	(get_builtin_sync_mode): Likewise.
	(expand_ifn_atomic_compare_exchange): Likewise.
	(expand_builtin_atomic_clear): Likewise.
	(expand_builtin_atomic_test_and_set): Likewise.
	(fold_builtin_atomic_always_lock_free): Likewise.
	* calls.c (compute_argument_addresses): Likewise.
	(emit_library_call_value_1): Likewise.
	(store_one_arg): Likewise.
	* combine.c (combine_instructions): Likewise.
	* config/aarch64/aarch64.c (aarch64_function_value): Likewise.
	* config/arm/arm.c (arm_function_value): Likewise.
	(aapcs_allocate_return_reg): Likewise.
	* config/c6x/c6x.c (c6x_expand_movmem): Likewise.
	* config/i386/i386.c (construct_container): Likewise.
	(ix86_gimplify_va_arg): Likewise.
	(ix86_expand_sse_cmp): Likewise.
	(emit_memmov): Likewise.
	(emit_memset): Likewise.
	(expand_small_movmem_or_setmem): Likewise.
	(ix86_expand_pextr): Likewise.
	(ix86_expand_pinsr): Likewise.
	* config/lm32/lm32.c (lm32_block_move_inline): Likewise.
	* config/microblaze/microblaze.c (microblaze_block_move_straight):
	Likewise.
	* config/mips/mips.c (mips_function_value_1) Likewise.
	(mips_block_move_straight): Likewise.
	(mips_expand_ins_as_unaligned_store): Likewise.
	* config/powerpcspe/powerpcspe.c
	(rs6000_darwin64_record_arg_advance_flush): Likewise.
	(rs6000_darwin64_record_arg_flush): Likewise.
	* config/rs6000/rs6000.c
	(rs6000_darwin64_record_arg_advance_flush): Likewise.
	(rs6000_darwin64_record_arg_flush): Likewise.
	* config/sparc/sparc.c (sparc_function_arg_1): Likewise.
	(sparc_function_value_1): Likewise.
	* config/spu/spu.c (adjust_operand): Likewise.
	(spu_emit_branch_or_set): Likewise.
	(arith_immediate_p): Likewise.
	* emit-rtl.c (gen_lowpart_common): Likewise.
	* expr.c (expand_expr_real_1): Likewise.
	* function.c (assign_parm_setup_block): Likewise.
	* gimple-ssa-store-merging.c (encode_tree_to_bitpos): Likewise.
	* reload1.c (alter_reg): Likewise.
	* stor-layout.c (mode_for_vector): Likewise.
	(layout_type): Likewise.

(choosing one of mine to avoid picking on someone else).  I find
reviewing changelogs like this an extra burden rather than a help,
since reviewers are expected to make sure that every affected function
is listed, every sentence ends with a ".", the correct amount of spacing
is used, etc.

So I hope eventually we can be flexible and just use the changelog
format as one optional way of achieving a goal, rather than being
mandatory for every non-testsuite patch.

Richard

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Segher Boessenkool]

On Thu, Feb 06, 2020 at 08:51:42AM +0000, Richard Sandiford wrote:
> Segher Boessenkool <segher@kernel.crashing.org> writes:
> > On Mon, Feb 03, 2020 at 01:24:04PM -0700, Jeff Law wrote:
> >> ANd yes, even though I have been a regular ChangeLog user, I rely more
> >> and more on the git log these days.
> >
> > As a reviewer, the changelog is priceless still.  We shouldn't drop the
> > changelog before people write *good* commit messages (and we are still
> > quite far from that goal).
> 
> But that was the point of what others said upthread: now that the patch
> description is part of the commit message, we should review them for
> whether they're a good description.  The patch description should be
> as much a part of the review as the patch itself.
> 
> So IMO the right response to poor patch descriptions is to ask for
> a better one.

Certainly.  But I don't think we should drop changelogs until we are
there, or have confidence we'll get there very soon.

> The goal should be "have good patch descriptions" not
> "make all patch descriptions use format X".

If you read the "subject" discussion, you know I *very* much agree with
that.

> The traditional changelog
> format should be one acceptable way of writing good patch descriptions,
> but not the only acceptable way.

A changelog is not an acceptable patch description (except in very
trivial cases, fixing a typo in docs for example).  It says only *what*
changed, nothing more.  We *do* need that info, but we need more of
course.

> Sometimes the traditional changelog is better than a plain English
> description (the SVE PCS work being one recent example from my POV).
> But for modern VCSes there's IMO no benefit to a changelog like:
> 
> 2017-09-05  Richard Sandiford  <richard.sandiford@linaro.org>
> 
> 	* builtins.c (expand_builtin_powi): Use int_mode_for_size.

"Use"?  For what?  What does it do with it, or is it used instead of
seomething else, is that what this patch does?  The changelog should
say, but doesn't.

> 	(get_builtin_sync_mode): Likewise.
> 	(expand_ifn_atomic_compare_exchange): Likewise.
> 	(expand_builtin_atomic_clear): Likewise.
> 	(expand_builtin_atomic_test_and_set): Likewise.
> 	(fold_builtin_atomic_always_lock_free): Likewise.

You don't need to point out every function you did the same change in.
It's helpful when it is just a handful, or ten or so, but no one will
read a list of eighty.  And it is not required.

> (choosing one of mine to avoid picking on someone else).  I find
> reviewing changelogs like this an extra burden rather than a help,

Yes.  It's just something more to delete in the reply.  The tiresome
part is scanning it to see if there is something else in there.

> since reviewers are expected to make sure that every affected function
> is listed,

No, that is not required.  Every change should be noted, but you can
just say

	* builtins.c: Use int_mode_for_size instead of mode_for_size.
	* combine.c: Likewise.

etc.

> every sentence ends with a ".", the correct amount of spacing
> is used, etc.

Your editor can do that for you, but it is automatic as well: things
just look wrong if they are formatted differently.  In that sense,
consistent formatting is a great help to any reader of the code,
reviewer or not.

This is not specific to changelogs at all, of course.

> So I hope eventually we can be flexible and just use the changelog
> format as one optional way of achieving a goal, rather than being
> mandatory for every non-testsuite patch.

Yes.  Eventually.  People first need to learn to write commit messages
that are not even less helpful than the changelogs they write.

Writing a commit message should not be seen as a burden.  It's your
sales pitch!  You can (and should!) tell what is changed and why, maybe
why not in some other way, etc., so that people in the future can figure
out why this change was made, and what it intended to do.  A good commit
message makes life much easier for reviewers, too, since not many
questions will remain.

Also.  If you find it hard to write a good commit message (or a good
changelog), chances are that your patch series is not so well structured;
simply chopping one patch in two or three often helps.

Anyway, you know all that.  I just don't want a year of GCC commits
without useful history, I hope I managed to explain my view.


Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Jeff Law]

On Wed, 2020-02-05 at 15:18 -0600, Segher Boessenkool wrote:
> On Mon, Feb 03, 2020 at 01:24:04PM -0700, Jeff Law wrote:
> > ANd yes, even though I have been a regular ChangeLog user, I rely more
> > and more on the git log these days.
> 
> As a reviewer, the changelog is priceless still.  We shouldn't drop the
> changelog before people write *good* commit messages (and we are still
> quite far from that goal).
I believe the current proposal is not to switch immediately, but to do
so after gcc-10 is released.  Feel free to suggest improvements to
ChangeLogs or summaries in the mean time to get folks to start
rethinking what they write in 'em.

And FWIW, we're talking about the ChangeLog *file* here.  If folks
continued writing the same log messages and put them into git, I
personally think that's sufficient to transition away from having a
ChangeLog file in the source tree.

I don't want to make perfect the enemy of the very very good here and
moving away from a ChangeLog file in the source tree is, IMHO, very
very good.

jeff

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Segher Boessenkool]

On Wed, Feb 05, 2020 at 06:43:54PM -0700, Jeff Law wrote:
> On Wed, 2020-02-05 at 15:18 -0600, Segher Boessenkool wrote:
> > As a reviewer, the changelog is priceless still.  We shouldn't drop the
> > changelog before people write *good* commit messages (and we are still
> > quite far from that goal).
> I believe the current proposal is not to switch immediately, but to do
> so after gcc-10 is released.  Feel free to suggest improvements to
> ChangeLogs or summaries in the mean time to get folks to start
> rethinking what they write in 'em.

I think I will be quite busy until the end of stage 4 already, but sure,
of course I will not shut up.

> And FWIW, we're talking about the ChangeLog *file* here.

I don't think we were.  I have absolutely nothing against only putting
the changelog entries at the end of the commit message and extracting
that later, of course (although it is almost trivial to copy it to the
changelog files manually, I have the same workflow for it as when we
still committed with SVN, just "git am" instead of more clunky stuff,
I suspect it will streamline things, esp. for new people).

> If folks
> continued writing the same log messages and put them into git, I
> personally think that's sufficient to transition away from having a
> ChangeLog file in the source tree.

Sure.  Just needs a flag day, always painful, but it will help.

> I don't want to make perfect the enemy of the very very good here and
> moving away from a ChangeLog file in the source tree is, IMHO, very
> very good.

If you rebase changelog files, then yes, it's a bloody pain ;-)


Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Richard Biener]

On Thu, Feb 6, 2020 at 2:51 PM Segher Boessenkool
<segher@kernel.crashing.org> wrote:
>
> On Wed, Feb 05, 2020 at 06:43:54PM -0700, Jeff Law wrote:
> > On Wed, 2020-02-05 at 15:18 -0600, Segher Boessenkool wrote:
> > > As a reviewer, the changelog is priceless still.  We shouldn't drop the
> > > changelog before people write *good* commit messages (and we are still
> > > quite far from that goal).
> > I believe the current proposal is not to switch immediately, but to do
> > so after gcc-10 is released.  Feel free to suggest improvements to
> > ChangeLogs or summaries in the mean time to get folks to start
> > rethinking what they write in 'em.
>
> I think I will be quite busy until the end of stage 4 already, but sure,
> of course I will not shut up.
>
> > And FWIW, we're talking about the ChangeLog *file* here.
>
> I don't think we were.  I have absolutely nothing against only putting
> the changelog entries at the end of the commit message and extracting
> that later, of course (although it is almost trivial to copy it to the
> changelog files manually, I have the same workflow for it as when we
> still committed with SVN, just "git am" instead of more clunky stuff,
> I suspect it will streamline things, esp. for new people).
>
> > If folks
> > continued writing the same log messages and put them into git, I
> > personally think that's sufficient to transition away from having a
> > ChangeLog file in the source tree.
>
> Sure.  Just needs a flag day, always painful, but it will help.
>
> > I don't want to make perfect the enemy of the very very good here and
> > moving away from a ChangeLog file in the source tree is, IMHO, very
> > very good.
>
> If you rebase changelog files, then yes, it's a bloody pain ;-)

So do you have a script that takes a commit with a ChangeLog at its end
and populates the appropriate ChangeLog files?  I'm trying to come up with
one to make the process less manual ... it's definitely a part that requires
more typing compared to svn.  ChangeLog file populating could be even
done on the server-side I guess (and not appropriately formatted logs
for the extraction/moving process rejected).

Richard.

>
> Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Jonathan Wakely]

On Thu, 6 Feb 2020 at 14:01, Richard Biener wrote:
> So do you have a script that takes a commit with a ChangeLog at its end
> and populates the appropriate ChangeLog files?  I'm trying to come up with
> one to make the process less manual ... it's definitely a part that requires
> more typing compared to svn.  ChangeLog file populating could be even
> done on the server-side I guess (and not appropriately formatted logs
> for the extraction/moving process rejected).

I have a script that does the opposite, which I've been using for
years. I edit the ChangeLog files as before, and a Git
prepare-commit-msg hook extracts the top changelog entry from each
file in the commit and pre-populates my commit msg with those entries.

To use it just put the two prepare-commit-msg* files from
https://gitlab.com/miscripts/miscripts/-/tree/master/gcc into your
$GCC_SRC/.git/hooks directory.

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Tom Tromey]

>>>>> "Jonathan" == Jonathan Wakely <jwakely.gcc@gmail.com> writes:

Jonathan> I have a script that does the opposite, which I've been using for
Jonathan> years. I edit the ChangeLog files as before, and a Git
Jonathan> prepare-commit-msg hook extracts the top changelog entry from each
Jonathan> file in the commit and pre-populates my commit msg with those entries.

Jonathan> To use it just put the two prepare-commit-msg* files from
Jonathan> https://gitlab.com/miscripts/miscripts/-/tree/master/gcc into your
Jonathan> $GCC_SRC/.git/hooks directory.

I do this too, combined with scripts to handle merge ChangeLogs
specially during rebase; scripts to update the dates on ChangeLog files
before pushing; and a wrapper for "git send-email" that strips the
ChangeLogs from the email (part of gdb patch submission rules).

You can find it all here

https://github.com/tromey/git-gnu-changelog

Tom

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Jason Merrill]

On Fri, Feb 7, 2020 at 1:44 PM Tom Tromey <tom@tromey.com> wrote:
>
> >>>>> "Jonathan" == Jonathan Wakely <jwakely.gcc@gmail.com> writes:
>
> Jonathan> I have a script that does the opposite, which I've been using for
> Jonathan> years. I edit the ChangeLog files as before, and a Git
> Jonathan> prepare-commit-msg hook extracts the top changelog entry from each
> Jonathan> file in the commit and pre-populates my commit msg with those entries.
>
> Jonathan> To use it just put the two prepare-commit-msg* files from
> Jonathan> https://gitlab.com/miscripts/miscripts/-/tree/master/gcc into your
> Jonathan> $GCC_SRC/.git/hooks directory.
>
> I do this too, combined with scripts to handle merge ChangeLogs
> specially during rebase; scripts to update the dates on ChangeLog files
> before pushing; and a wrapper for "git send-email" that strips the
> ChangeLogs from the email (part of gdb patch submission rules).

I omit ChangeLogs by adding ':!*/ChangeLog' to the end of the git
send-email command.  I don't remember where I found that incantation.

> You can find it all here
>
> https://github.com/tromey/git-gnu-changelog
>
> Tom
>

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Tom Tromey]

>>>>> "Jason" == Jason Merrill <jason@redhat.com> writes:

Jason> I omit ChangeLogs by adding ':!*/ChangeLog' to the end of the git
Jason> send-email command.  I don't remember where I found that incantation.

Cool, I did not know about this.

FWIW if you have the ChangeLog merger available, it's actually more
convenient if the patch includes the ChangeLog, because then applying it
with "git am" does the right thing.  Without this you have to edit the
ChangeLogs by hand instead.

Tom

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Segher Boessenkool]

On Fri, Feb 07, 2020 at 03:34:03PM -0700, Tom Tromey wrote:
> >>>>> "Jason" == Jason Merrill <jason@redhat.com> writes:
> 
> Jason> I omit ChangeLogs by adding ':!*/ChangeLog' to the end of the git
> Jason> send-email command.  I don't remember where I found that incantation.
> 
> Cool, I did not know about this.

Yeah, me neither.  And it isn't something new, it is ancient.

The manpage for this is "man gitrevisions".  It doesn't explain the !
though, and not wildcards even.

(dir.c in git.git, if you like spelunking).


Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Andrew Pinski]

On Sat, Feb 8, 2020 at 8:51 AM Segher Boessenkool
<segher@kernel.crashing.org> wrote:
>
> On Fri, Feb 07, 2020 at 03:34:03PM -0700, Tom Tromey wrote:
> > >>>>> "Jason" == Jason Merrill <jason@redhat.com> writes:
> >
> > Jason> I omit ChangeLogs by adding ':!*/ChangeLog' to the end of the git
> > Jason> send-email command.  I don't remember where I found that incantation.
> >
> > Cool, I did not know about this.
>
> Yeah, me neither.  And it isn't something new, it is ancient.
>
> The manpage for this is "man gitrevisions".  It doesn't explain the !
> though, and not wildcards even.
>
> (dir.c in git.git, if you like spelunking).

Note I was not able to get it to work with "git version 1.8.3.1"
(which is included with CentOS 7) but it worked with "git version
2.17.1".


Thanks,
Andrew Pinski

>
>
> Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Segher Boessenkool]

On Sat, Feb 08, 2020 at 03:55:33PM -0800, Andrew Pinski wrote:
> On Sat, Feb 8, 2020 at 8:51 AM Segher Boessenkool
> <segher@kernel.crashing.org> wrote:
> >
> > On Fri, Feb 07, 2020 at 03:34:03PM -0700, Tom Tromey wrote:
> > > >>>>> "Jason" == Jason Merrill <jason@redhat.com> writes:
> > >
> > > Jason> I omit ChangeLogs by adding ':!*/ChangeLog' to the end of the git
> > > Jason> send-email command.  I don't remember where I found that incantation.
> > >
> > > Cool, I did not know about this.
> >
> > Yeah, me neither.  And it isn't something new, it is ancient.
> >
> > The manpage for this is "man gitrevisions".  It doesn't explain the !
> > though, and not wildcards even.
> >
> > (dir.c in git.git, if you like spelunking).
> 
> Note I was not able to get it to work with "git version 1.8.3.1"
> (which is included with CentOS 7) but it worked with "git version
> 2.17.1".

Not 2.0.0 either, as another data point.

This works:

git log -p master -- . ':!/*/*ChangeLog*'

(But that is putting in more than just a refspec).


Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Matthew Malcomson]

On 08/02/2020 16:50, Segher Boessenkool wrote:
> On Fri, Feb 07, 2020 at 03:34:03PM -0700, Tom Tromey wrote:
>>>>>>> "Jason" == Jason Merrill <jason@redhat.com> writes:
>>
>> Jason> I omit ChangeLogs by adding ':!*/ChangeLog' to the end of the git
>> Jason> send-email command.  I don't remember where I found that incantation.
>>
>> Cool, I did not know about this.
> 
> Yeah, me neither.  And it isn't something new, it is ancient.
> 
> The manpage for this is "man gitrevisions".  It doesn't explain the !
> though, and not wildcards even.
> 
> (dir.c in git.git, if you like spelunking).
> 
> 
> Segher
> 


Just for anyone interested -- the manpage that describes the '!' is 
`gitglossary`.

It's under the description of `pathspec`, and has a long-form of 
`:(exclude)`.

http://man7.org/linux/man-pages/man7/gitglossary.7.html

MM

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Segher Boessenkool]

On Mon, Feb 10, 2020 at 05:51:10PM +0000, Matthew Malcomson wrote:
> Just for anyone interested -- the manpage that describes the '!' is 
> `gitglossary`.
> 
> It's under the description of `pathspec`, and has a long-form of 
> `:(exclude)`.

https://github.com/git/git/commit/93dbefb389a011c9107a3908fd38e743055be267
(2017, Git 2.14.3) made this more discoverable, and
https://github.com/git/git/commit/ef79b1f8704668a6cdf4278f9255e03ca785bfb4
is what implemented this originally (2014, Git 1.9.0).

Which makes me wonder why it did not work with 2.0.0?  Hrm.  Maybe I
messed up the syntax somewhere, for testing it.


Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Segher Boessenkool]

On Thu, Feb 06, 2020 at 03:01:20PM +0100, Richard Biener wrote:
> On Thu, Feb 6, 2020 at 2:51 PM Segher Boessenkool
> <segher@kernel.crashing.org> wrote:
> > If you rebase changelog files, then yes, it's a bloody pain ;-)
> 
> So do you have a script that takes a commit with a ChangeLog at its end
> and populates the appropriate ChangeLog files?

I develop my patches in Git, no changelog whatsoever.  Then I "git
format-patch" it.  I write the changelog in that file (also proper white
space, which "git commit" likes to destroy, I have a patch for that but
not installed everywhere).  "git send-email", wait for ack if necessary,
"git am" and copy-and-paste the changelog parts, "git add" those
changelog files, "git commit --amend", final sanity checks, and "git
commit fsf master".  I store the patch file to my archive, or I queue it
for backport directly.

> I'm trying to come up with
> one to make the process less manual ...

Yeah, don't look at me then :-)

I *like* having most of those steps, most of this should only be done by
people who are awake.

> it's definitely a part that requires
> more typing compared to svn.

Instead of "git am" I had "patch -p1 <", distributing the changelog parts
I just did in vi (as with git), then "svn ci", which pick up all modified
files directly (sometimes an "svn add" first).  It's pretty much the same
for me.

> ChangeLog file populating could be even
> done on the server-side I guess (and not appropriately formatted logs
> for the extraction/moving process rejected).

Yup.  As long as it's fool-proof, and we have some way to correct small
disasters (both of those!)


Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Richard Biener]

On Thu, Feb 6, 2020 at 11:25 PM Segher Boessenkool
<segher@kernel.crashing.org> wrote:
>
> On Thu, Feb 06, 2020 at 03:01:20PM +0100, Richard Biener wrote:
> > On Thu, Feb 6, 2020 at 2:51 PM Segher Boessenkool
> > <segher@kernel.crashing.org> wrote:
> > > If you rebase changelog files, then yes, it's a bloody pain ;-)
> >
> > So do you have a script that takes a commit with a ChangeLog at its end
> > and populates the appropriate ChangeLog files?
>
> I develop my patches in Git, no changelog whatsoever.  Then I "git
> format-patch" it.  I write the changelog in that file (also proper white
> space, which "git commit" likes to destroy, I have a patch for that but
> not installed everywhere).  "git send-email", wait for ack if necessary,
> "git am" and copy-and-paste the changelog parts, "git add" those
> changelog files, "git commit --amend", final sanity checks, and "git
> commit fsf master".  I store the patch file to my archive, or I queue it
> for backport directly.
>
> > I'm trying to come up with
> > one to make the process less manual ...
>
> Yeah, don't look at me then :-)
>
> I *like* having most of those steps, most of this should only be done by
> people who are awake.
>
> > it's definitely a part that requires
> > more typing compared to svn.
>
> Instead of "git am" I had "patch -p1 <", distributing the changelog parts
> I just did in vi (as with git), then "svn ci", which pick up all modified
> files directly (sometimes an "svn add" first).  It's pretty much the same
> for me.

I'm a believer on committing from the tree I actually tested, so it
was with SVN: patch, test, svn up, edit changelogs (cut&paste from the
patch header), svn diff | less (check wha'ts abotu to be committed), svn commit
with GIT its now: patch, test, git add/commit, git pull --rebase, edit
changelogs
(cut&paste from the patch header), git commit --amend, git push -n -v,
git show <what's pushed>, git push

So I wonder to "optimize" this somewhat.

> > ChangeLog file populating could be even
> > done on the server-side I guess (and not appropriately formatted logs
> > for the extraction/moving process rejected).
>
> Yup.  As long as it's fool-proof, and we have some way to correct small
> disasters (both of those!)

I'd definitely prefer server-side operation, and yes, that "script" looking
at the commit would need to have a "extract changelog parts" part
for a pre-accept hook and the actual part amending the commit with
the changelog file changes (not sure if that's even possible in a hook?).

Fixing mistakes in the ChangeLog files could be as easy as allowing
changes to ChangeLog files without that magic.

Anyway, I hope to put sth together with bash & [g]awk

Richard.

>
> Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Jonathan Wakely]

On Fri, 7 Feb 2020 at 09:20, Richard Biener <richard.guenther@gmail.com> wrote:
>
> On Thu, Feb 6, 2020 at 11:25 PM Segher Boessenkool
> <segher@kernel.crashing.org> wrote:
> > Instead of "git am" I had "patch -p1 <", distributing the changelog parts
> > I just did in vi (as with git), then "svn ci", which pick up all modified
> > files directly (sometimes an "svn add" first).  It's pretty much the same
> > for me.
>
> I'm a believer on committing from the tree I actually tested, so it

With Git you can't really have unwanted local commits present in a
tree if you use a sensible workflow, so if you tested in a tree that
was at commit 1234abcd and you push from another machine that is at
the same commit, you know there are no unintended differences.

> was with SVN: patch, test, svn up, edit changelogs (cut&paste from the
> patch header), svn diff | less (check wha'ts abotu to be committed), svn commit
> with GIT its now: patch, test, git add/commit,

There is no good reason to delay add+commit until after testing.

Committing locally is a completely different operation from committing
to svn, because nothing is "final" until you push to gcc.gnu.org. I
commit locally, and then either test locally or push to larger
machines where I test, or test in both places at once. Even if you
only work on one machine, keeping uncommitted changes around is error
prone (it's easier to lose the work, it's easier to not realise
exactly what you're testing, it's harder to verify that what you
tested is what you eventually pushed).

Applying somebody else's patch to test it can be done in a local
branch and committed locally. You can always delete that branch later
to clear up (a naming convention like "patch-20200207-foo-bar" helps
to keep those branches clearly labelled so you can easily decide which
to remove later).

Once you've committed something it has a commit hash to identify it,
so you can easily compare the state at that revision to some other
revision, at any later time.

tl;dr make more use of branches.

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Alan Modra]

On Fri, Feb 07, 2020 at 10:08:25AM +0000, Jonathan Wakely wrote:
> With Git you can't really have unwanted local commits present in a
> tree if you use a sensible workflow, so if you tested in a tree that
> was at commit 1234abcd and you push from another machine that is at
> the same commit, you know there are no unintended differences.

Maybe I don't have a sensible workflow, but often with lots of tiddly
little binutils patches I don't bother with branches for everything.
(I do use branches for larger work.)  I also like to test my patches.
I'll test individually on a few relevant targets but do a test over
a large number of targets (162 currently) for a bunch of patches.
Some of those patches tested might not be ready for commit upstream
(lacking comments, changelogs, even lacking that vital self review),
so I'll "git rebase -i" to put the ones that are ready first, then
"git push origin <commit id>:master"
just to push up to the relevant commit.  That works quite well for me.

-- 
Alan Modra
Australia Development Lab, IBM

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Segher Boessenkool]

On Sat, Feb 08, 2020 at 09:46:53AM +1030, Alan Modra wrote:
> On Fri, Feb 07, 2020 at 10:08:25AM +0000, Jonathan Wakely wrote:
> > With Git you can't really have unwanted local commits present in a
> > tree if you use a sensible workflow, so if you tested in a tree that
> > was at commit 1234abcd and you push from another machine that is at
> > the same commit, you know there are no unintended differences.
> 
> Maybe I don't have a sensible workflow, but often with lots of tiddly
> little binutils patches I don't bother with branches for everything.

Yup, same here.  And I sometimes "git revert" some of the patches I have
in my "work" tree to be able to test other patches, while making it easy
to get things back.  To prevent "death by a thousand branches" syndrome.
Apparently you can do similar with "git stash", but I never got the hang
of that.

Sometimes I revert a revert of a revert.  That's probably too much
though :-)

> Some of those patches tested might not be ready for commit upstream
> (lacking comments, changelogs, even lacking that vital self review),
> so I'll "git rebase -i" to put the ones that are ready first, then
> "git push origin <commit id>:master"
> just to push up to the relevant commit.  That works quite well for me.

I only ever push from master (or some other local tracking branch), and
never commit to those tracking branches other than immediately before
pushing.  This prevents mistakes.


Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Jonathan Wakely]

On Sat, 8 Feb 2020 at 19:58, Segher Boessenkool
<segher@kernel.crashing.org> wrote:
>
> On Sat, Feb 08, 2020 at 09:46:53AM +1030, Alan Modra wrote:
> > On Fri, Feb 07, 2020 at 10:08:25AM +0000, Jonathan Wakely wrote:
> > > With Git you can't really have unwanted local commits present in a
> > > tree if you use a sensible workflow, so if you tested in a tree that
> > > was at commit 1234abcd and you push from another machine that is at
> > > the same commit, you know there are no unintended differences.
> >
> > Maybe I don't have a sensible workflow, but often with lots of tiddly
> > little binutils patches I don't bother with branches for everything.
>
> Yup, same here.  And I sometimes "git revert" some of the patches I have
> in my "work" tree to be able to test other patches, while making it easy
> to get things back.  To prevent "death by a thousand branches" syndrome.
> Apparently you can do similar with "git stash", but I never got the hang
> of that.
>
> Sometimes I revert a revert of a revert.  That's probably too much
> though :-)

My main point was that Richi should be committing things, not working
with uncommitted patches hanging around making things dirty.

I like to use branches, but having a single branch with a series of
commits that you reorder and selectively push is still better than
uncommitted work.

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Segher Boessenkool]

On Sun, Feb 09, 2020 at 10:46:04AM +0000, Jonathan Wakely wrote:
> My main point was that Richi should be committing things, not working
> with uncommitted patches hanging around making things dirty.
> 
> I like to use branches, but having a single branch with a series of
> commits that you reorder and selectively push is still better than
> uncommitted work.

Yes, committing is cheap and you should do it after every few minutes
of typing.  It is much easier to untangle a mess with too many commits
than one with too few.


Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Segher Boessenkool]

Hi!

On Fri, Feb 07, 2020 at 10:19:56AM +0100, Richard Biener wrote:
> On Thu, Feb 6, 2020 at 11:25 PM Segher Boessenkool
> <segher@kernel.crashing.org> wrote:
> > Yeah, don't look at me then :-)
> >
> > I *like* having most of those steps, most of this should only be done by
> > people who are awake.
> >
> > > it's definitely a part that requires
> > > more typing compared to svn.
> >
> > Instead of "git am" I had "patch -p1 <", distributing the changelog parts
> > I just did in vi (as with git), then "svn ci", which pick up all modified
> > files directly (sometimes an "svn add" first).  It's pretty much the same
> > for me.
> 
> I'm a believer on committing from the tree I actually tested,

But you cannot.  Depending on which machine I use, a regstrap (all
languages) is between 2h and 40m, so when you push it upstream there
will usually be commits by other people as well.

I also test just master plus what I want to commit, which works fine
most of the time.  The master you test with is older than the one you
commit to (and yes, this sometimes matters :-/ ).

> so it
> was with SVN: patch, test, svn up, edit changelogs (cut&paste from the
> patch header), svn diff | less (check wha'ts abotu to be committed), svn commit
> with GIT its now: patch, test, git add/commit, git pull --rebase, edit
> changelogs
> (cut&paste from the patch header), git commit --amend, git push -n -v,
> git show <what's pushed>, git push
> 
> So I wonder to "optimize" this somewhat.

Having the patch as a file instead of just as a git commit makes the
cut-and-paste step a little easier (and more reliable).

Once you have done this many many times, you can skip some of the
verification steps because you just *know* the state of everything.  So
things will be a little faster over time, or at least it will feel that
way.

Handling the changelog as you want will make things more streamlined,
certainly.  It also makes it easier to mess things up...  We'll see what
will happen, I guess :-)

> > > ChangeLog file populating could be even
> > > done on the server-side I guess (and not appropriately formatted logs
> > > for the extraction/moving process rejected).
> >
> > Yup.  As long as it's fool-proof, and we have some way to correct small
> > disasters (both of those!)
> 
> I'd definitely prefer server-side operation, and yes, that "script" looking
> at the commit would need to have a "extract changelog parts" part
> for a pre-accept hook and the actual part amending the commit with
> the changelog file changes (not sure if that's even possible in a hook?).

If you can think of it, it is possible.  It's not even too hard I think.

You want the commit hook to actually populate the changelog files, which
are still normal checked-in files, aha!  That's a great idea.  That
completely gets rid of my "fix mistakes" worries, and it will probably
also get rid of the "yet another flag day" inconvenience.

> Fixing mistakes in the ChangeLog files could be as easy as allowing
> changes to ChangeLog files without that magic.

Yeah :-)

> Anyway, I hope to put sth together with bash & [g]awk

gl;hf

Should we require some simple markup in the commit message before the
changelogs?  Maybe

CL gcc/
	* blablalba etc.
CL gcc/testsuite/
	* gcc.target/...
etc.

(Dunno if just "^CL " is too short?)


Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Richard Earnshaw (lists)]

On 07/02/2020 13:48, Segher Boessenkool wrote:
> Hi!
> 
> On Fri, Feb 07, 2020 at 10:19:56AM +0100, Richard Biener wrote:
>> On Thu, Feb 6, 2020 at 11:25 PM Segher Boessenkool
>> <segher@kernel.crashing.org> wrote:
>>> Yeah, don't look at me then :-)
>>>
>>> I *like* having most of those steps, most of this should only be done by
>>> people who are awake.
>>>
>>>> it's definitely a part that requires
>>>> more typing compared to svn.
>>>
>>> Instead of "git am" I had "patch -p1 <", distributing the changelog parts
>>> I just did in vi (as with git), then "svn ci", which pick up all modified
>>> files directly (sometimes an "svn add" first).  It's pretty much the same
>>> for me.
>>
>> I'm a believer on committing from the tree I actually tested,
> 
> But you cannot.  Depending on which machine I use, a regstrap (all
> languages) is between 2h and 40m, so when you push it upstream there
> will usually be commits by other people as well.
> 
> I also test just master plus what I want to commit, which works fine
> most of the time.  The master you test with is older than the one you
> commit to (and yes, this sometimes matters :-/ ).
> 
>> so it
>> was with SVN: patch, test, svn up, edit changelogs (cut&paste from the
>> patch header), svn diff | less (check wha'ts abotu to be committed), svn commit
>> with GIT its now: patch, test, git add/commit, git pull --rebase, edit
>> changelogs
>> (cut&paste from the patch header), git commit --amend, git push -n -v,
>> git show <what's pushed>, git push
>>
>> So I wonder to "optimize" this somewhat.
> 
> Having the patch as a file instead of just as a git commit makes the
> cut-and-paste step a little easier (and more reliable).
> 
> Once you have done this many many times, you can skip some of the
> verification steps because you just *know* the state of everything.  So
> things will be a little faster over time, or at least it will feel that
> way.
> 
> Handling the changelog as you want will make things more streamlined,
> certainly.  It also makes it easier to mess things up...  We'll see what
> will happen, I guess :-)
> 
>>>> ChangeLog file populating could be even
>>>> done on the server-side I guess (and not appropriately formatted logs
>>>> for the extraction/moving process rejected).
>>>
>>> Yup.  As long as it's fool-proof, and we have some way to correct small
>>> disasters (both of those!)
>>
>> I'd definitely prefer server-side operation, and yes, that "script" looking
>> at the commit would need to have a "extract changelog parts" part
>> for a pre-accept hook and the actual part amending the commit with
>> the changelog file changes (not sure if that's even possible in a hook?).
> 
> If you can think of it, it is possible.  It's not even too hard I think.
> 
> You want the commit hook to actually populate the changelog files, which
> are still normal checked-in files, aha!  That's a great idea.  That
> completely gets rid of my "fix mistakes" worries, and it will probably
> also get rid of the "yet another flag day" inconvenience.
> 
>> Fixing mistakes in the ChangeLog files could be as easy as allowing
>> changes to ChangeLog files without that magic.
> 
> Yeah :-)
> 
>> Anyway, I hope to put sth together with bash & [g]awk
> 
> gl;hf
> 
> Should we require some simple markup in the commit message before the
> changelogs?  Maybe
> 
> CL gcc/
> 	* blablalba etc.
> CL gcc/testsuite/
> 	* gcc.target/...

> etc.
> 
> (Dunno if just "^CL " is too short?)

I was thinking "@CL <dir>", but ^CL would work just as well.

Any script should, in addition to extracting the author and email also 
grep for "Co-authored-by:" annotations and add those as additional 
contributors, of course.

R.

> 
> 
> Segher
> 

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Segher Boessenkool]

On Fri, Feb 07, 2020 at 01:56:08PM +0000, Richard Earnshaw (lists) wrote:
> On 07/02/2020 13:48, Segher Boessenkool wrote:
> >Should we require some simple markup in the commit message before the
> >changelogs?  Maybe
> >
> >CL gcc/
> >	* blablalba etc.
> >CL gcc/testsuite/
> >	* gcc.target/...
> 
> >etc.
> >
> >(Dunno if just "^CL " is too short?)
> 
> I was thinking "@CL <dir>", but ^CL would work just as well.

I meant ^ as in a regular expression, sorry for not being clear.

> Any script should, in addition to extracting the author and email also 
> grep for "Co-authored-by:" annotations and add those as additional 
> contributors, of course.

Is there no existing established practice for that?  (Or is this it and
so infrquent that I never have seen it :-) )


Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Richard Earnshaw (lists)]

On 07/02/2020 15:32, Segher Boessenkool wrote:
> On Fri, Feb 07, 2020 at 01:56:08PM +0000, Richard Earnshaw (lists) wrote:
>> On 07/02/2020 13:48, Segher Boessenkool wrote:
>>> Should we require some simple markup in the commit message before the
>>> changelogs?  Maybe
>>>
>>> CL gcc/
>>> 	* blablalba etc.
>>> CL gcc/testsuite/
>>> 	* gcc.target/...
>>
>>> etc.
>>>
>>> (Dunno if just "^CL " is too short?)
>>
>> I was thinking "@CL <dir>", but ^CL would work just as well.
> 
> I meant ^ as in a regular expression, sorry for not being clear.

Ah! I think something more syntactically explicit would help avoid the 
risk of mis-parsing.

> 
>> Any script should, in addition to extracting the author and email also
>> grep for "Co-authored-by:" annotations and add those as additional
>> contributors, of course.
> 
> Is there no existing established practice for that?  (Or is this it and
> so infrquent that I never have seen it :-) )
> 
> 

It's a convention (much like signed-off-by, etc).  But it's fairly 
widely accepted.  In the git conversion we added fields for it where we 
detected co-authorship in the ChangeLog entries.  Git has no specific 
field for recording additional authors, so this approach addresses this 
and some tools know how to add the appropriate accreditation based on it.

https://git.wiki.kernel.org/index.php/CommitMessageConventions

Also "git help interpret-trailers"

R.


> Segher
> 

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Segher Boessenkool]

On Fri, Feb 07, 2020 at 03:43:05PM +0000, Richard Earnshaw (lists) wrote:
> On 07/02/2020 15:32, Segher Boessenkool wrote:
> >On Fri, Feb 07, 2020 at 01:56:08PM +0000, Richard Earnshaw (lists) wrote:
> >>Any script should, in addition to extracting the author and email also
> >>grep for "Co-authored-by:" annotations and add those as additional
> >>contributors, of course.
> >
> >Is there no existing established practice for that?  (Or is this it and
> >so infrquent that I never have seen it :-) )
> 
> It's a convention (much like signed-off-by, etc).  But it's fairly 
> widely accepted.

Apparently it is mainly a github thing.

> In the git conversion we added fields for it where we 
> detected co-authorship in the ChangeLog entries.  Git has no specific 
> field for recording additional authors,

Yup.

I think in the future we will see a lot more Suggested-by: and
Reviewed-by: and Tested-by: and the like, and less of what we now mark
as co-authoring (also because there will be more smaller commits).


Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Richard Biener]

On Fri, Feb 7, 2020 at 4:43 PM Richard Earnshaw (lists)
<Richard.Earnshaw@arm.com> wrote:
>
> On 07/02/2020 15:32, Segher Boessenkool wrote:
> > On Fri, Feb 07, 2020 at 01:56:08PM +0000, Richard Earnshaw (lists) wrote:
> >> On 07/02/2020 13:48, Segher Boessenkool wrote:
> >>> Should we require some simple markup in the commit message before the
> >>> changelogs?  Maybe
> >>>
> >>> CL gcc/
> >>>     * blablalba etc.
> >>> CL gcc/testsuite/
> >>>     * gcc.target/...
> >>
> >>> etc.
> >>>
> >>> (Dunno if just "^CL " is too short?)
> >>
> >> I was thinking "@CL <dir>", but ^CL would work just as well.
> >
> > I meant ^ as in a regular expression, sorry for not being clear.
>
> Ah! I think something more syntactically explicit would help avoid the
> risk of mis-parsing.

One just has to err on the side of rejection ... sofar I've tried to start
matching at ^20 (the date line) until the last ^\tPR line (multiple
PR foo/12345 lines) then the actual ChangeLog file entries are
separated by blank likes where there's unwritten policy to
prepend those with ^\tDIR/ where DIR is for example cp (implicitely
inside gcc) or libgomp (toplevel).  There's easy enough heuristics
to detect the gcc/ and gcc/testsuite case which is most often
omitted (just see where the first mentioned file lives in).

That said, doing the parsing is reasonably easy but the question
is the execution environment of the git hook - does the script have
access to an actual checkout?  I guess not (so much for file
existance tests), but then how to amend the commit with new
file changes?

And yes, the idea was to keep the status quo (checked in and
updated ChangeLog files) but have the convenience of not manually
updating them.  And indeed that makes a transition to a post ChangeLog
file world easy.

Richard.

> >
> >> Any script should, in addition to extracting the author and email also
> >> grep for "Co-authored-by:" annotations and add those as additional
> >> contributors, of course.
> >
> > Is there no existing established practice for that?  (Or is this it and
> > so infrquent that I never have seen it :-) )
> >
> >
>
> It's a convention (much like signed-off-by, etc).  But it's fairly
> widely accepted.  In the git conversion we added fields for it where we
> detected co-authorship in the ChangeLog entries.  Git has no specific
> field for recording additional authors, so this approach addresses this
> and some tools know how to add the appropriate accreditation based on it.
>
> https://git.wiki.kernel.org/index.php/CommitMessageConventions
>
> Also "git help interpret-trailers"
>
> R.
>
>
> > Segher
> >
>

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Hans-Peter Nilsson]

On Thu, 6 Feb 2020, Segher Boessenkool wrote:
> Instead of "git am" I had "patch -p1 <",

May I suggest "git apply" instead of the good old patch program.

(The "-p1" is of course built-in and you never have to do a
manual roll-back or separate --dry-run pass.)

brgds, H-P

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Jakub Jelinek]

On Wed, Feb 05, 2020 at 06:43:54PM -0700, Jeff Law wrote:
> On Wed, 2020-02-05 at 15:18 -0600, Segher Boessenkool wrote:
> > On Mon, Feb 03, 2020 at 01:24:04PM -0700, Jeff Law wrote:
> > > ANd yes, even though I have been a regular ChangeLog user, I rely more
> > > and more on the git log these days.
> > 
> > As a reviewer, the changelog is priceless still.  We shouldn't drop the
> > changelog before people write *good* commit messages (and we are still
> > quite far from that goal).
> I believe the current proposal is not to switch immediately, but to do
> so after gcc-10 is released.  Feel free to suggest improvements to
> ChangeLogs or summaries in the mean time to get folks to start
> rethinking what they write in 'em.
> 
> And FWIW, we're talking about the ChangeLog *file* here.  If folks
> continued writing the same log messages and put them into git, I
> personally think that's sufficient to transition away from having a
> ChangeLog file in the source tree.

A precondition for such change would be a script that would verify
that each commit being pushed into the repository has the ChangeLog snippets
available.  And not just that.  With ChangeLog files, one can fix up the
entries later on if somebody makes a mistake; it happens from time to time
to everybody, some people make the same mistakes again and again.
If the ChangeLog entry is in the commit message, then in git (unlike svn) it
is not possible to fix it later once you push it into the public repository.

So, we'd need a script that checks for the common mistakes, furthermore
verifies that e.g. all files in the commit that need to have a ChangeLog
entry have one etc. and reports back any issues + rejects the push.
We would need to agree how do we express stuff going into different former
ChangeLog files, whether we require gcc/cp/ etc. prefixes before the lines,
or say require empty line for different former ChangeLog files and let the
extraction script figure it out from the changed files in the commit, etc.
We could have a rule that say New test. entries for newly added files
in */testsuite/* would be optional and let the generating script add those
(but say require entries if existing tests have been changed).
Plus the extraction script, and test properly both scripts for a while
before making it live.

	Jakub

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Segher Boessenkool]

On Thu, Feb 06, 2020 at 03:56:40PM +0100, Jakub Jelinek wrote:
> On Wed, Feb 05, 2020 at 06:43:54PM -0700, Jeff Law wrote:
> > And FWIW, we're talking about the ChangeLog *file* here.  If folks
> > continued writing the same log messages and put them into git, I
> > personally think that's sufficient to transition away from having a
> > ChangeLog file in the source tree.
> 
> A precondition for such change would be a script that would verify
> that each commit being pushed into the repository has the ChangeLog snippets
> available.  And not just that.  With ChangeLog files, one can fix up the
> entries later on if somebody makes a mistake; it happens from time to time
> to everybody, some people make the same mistakes again and again.

Yes.  Mistakes *will* happen, and for *everyone*.

> If the ChangeLog entry is in the commit message, then in git (unlike svn) it
> is not possible to fix it later once you push it into the public repository.
> 
> So, we'd need a script that checks for the common mistakes, furthermore
> verifies that e.g. all files in the commit that need to have a ChangeLog
> entry have one etc. and reports back any issues + rejects the push.

We also need a way to fix changelog entries for the errors that do seep
through (and that are bad enough that they do need fixing).  It doesn't
have to be easy or convenient, but we need *some* way to do it.

> We would need to agree how do we express stuff going into different former
> ChangeLog files, whether we require gcc/cp/ etc. prefixes before the lines,
> or say require empty line for different former ChangeLog files and let the
> extraction script figure it out from the changed files in the commit, etc.

The latter seems error-prone, but maybe I worry too much :-)

> We could have a rule that say New test. entries for newly added files
> in */testsuite/* would be optional and let the generating script add those
> (but say require entries if existing tests have been changed).

How much work does that safe?

> Plus the extraction script, and test properly both scripts for a while
> before making it live.

Yes, absolutely.  Maybe it could post it to gcc-cvs@ (as it will in the
future) (but not actually commit the (generated) changelog parts yet)?


Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Jakub Jelinek]

On Thu, Feb 06, 2020 at 10:17:54AM -0600, Segher Boessenkool wrote:
> > We would need to agree how do we express stuff going into different former
> > ChangeLog files, whether we require gcc/cp/ etc. prefixes before the lines,
> > or say require empty line for different former ChangeLog files and let the
> > extraction script figure it out from the changed files in the commit, etc.
> 
> The latter seems error-prone, but maybe I worry too much :-)

Depends on how good the scripts are.  The commit hook could figure out
cases which are non-ambiguous and the extraction script would handle those
and which cases are ambiguous and reject those.
> 
> > We could have a rule that say New test. entries for newly added files
> > in */testsuite/* would be optional and let the generating script add those
> > (but say require entries if existing tests have been changed).
> 
> How much work does that safe?

Some people don't like ChangeLog entries for testsuite additions (not me)
and in the official guidelines those are optional ATM I think.
Allowing them to be implicitly added for new testsuite files would hopefully
satisfy those, while allowing people who want to be more explicit (New
test. not being what they want to see for some reason e.g.) do so.

	Jakub

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Jason Merrill]

On Thu, Feb 6, 2020 at 11:25 AM Segher Boessenkool <
segher@kernel.crashing.org> wrote:

>
> We also need a way to fix changelog entries for the errors that do seep
> through (and that are bad enough that they do need fixing).  It doesn't
> have to be easy or convenient, but we need *some* way to do it.
>

git notes?

Jason

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Segher Boessenkool]

On Thu, Feb 06, 2020 at 01:57:49PM -0500, Jason Merrill wrote:
> On Thu, Feb 6, 2020 at 11:25 AM Segher Boessenkool <
> segher@kernel.crashing.org> wrote:
> 
> >
> > We also need a way to fix changelog entries for the errors that do seep
> > through (and that are bad enough that they do need fixing).  It doesn't
> > have to be easy or convenient, but we need *some* way to do it.
> >
> 
> git notes?

That could be done, sure, if that is easy to integrate with what eventually
generates the actual changelog files.  Good idea.


Segher

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Nathan Sidwell]

On 1/24/20 4:36 PM, Jeff Law wrote:
> On Fri, 2020-01-24 at 20:32 +0100, Eric Botcazou wrote:
>>> I strongly prefer to move towards relying on the git log.
>>
>> In my experience the output of git log is a total mess so cannot replace
>> ChangeLogs.  But we can well decide to drop ChangeLog for the testsuite.
> Well, glibc has moved to extracting them from git, building policies
> and scripts around that.  I'm pretty sure other significant projecs are
> also extracting their ChangeLogs from git.
> 
> We could do the same, selecting some magic date as the cutover point
> after which future ChangeLogs are extracted from GIT.  In fact, that's
> precisely what I'd like to see us do.

The GCC10 release date would seem a good point to do this.  That gives us around 
3 months to figure the details (and get stakeholder buy-in)

nathan

-- 
Nathan Sidwell

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Jeff Law]

On Sat, 2020-01-25 at 10:50 -0500, Nathan Sidwell wrote:
> On 1/24/20 4:36 PM, Jeff Law wrote:
> > On Fri, 2020-01-24 at 20:32 +0100, Eric Botcazou wrote:
> > > > I strongly prefer to move towards relying on the git log.
> > > 
> > > In my experience the output of git log is a total mess so cannot replace
> > > ChangeLogs.  But we can well decide to drop ChangeLog for the testsuite.
> > Well, glibc has moved to extracting them from git, building policies
> > and scripts around that.  I'm pretty sure other significant projecs are
> > also extracting their ChangeLogs from git.
> > 
> > We could do the same, selecting some magic date as the cutover point
> > after which future ChangeLogs are extracted from GIT.  In fact, that's
> > precisely what I'd like to see us do.
> 
> The GCC10 release date would seem a good point to do this.  That gives us around 
> 3 months to figure the details (and get stakeholder buy-in)
Yup.  That would be what I'd recommend we shoot for.  As you say it
gives time to work on the details and for folks to start changing their
habits.

jeff

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Richard Earnshaw (lists)]

On 25/01/2020 16:11, Jeff Law wrote:
> On Sat, 2020-01-25 at 10:50 -0500, Nathan Sidwell wrote:
>> On 1/24/20 4:36 PM, Jeff Law wrote:
>>> On Fri, 2020-01-24 at 20:32 +0100, Eric Botcazou wrote:
>>>>> I strongly prefer to move towards relying on the git log.
>>>>
>>>> In my experience the output of git log is a total mess so cannot replace
>>>> ChangeLogs.  But we can well decide to drop ChangeLog for the testsuite.
>>> Well, glibc has moved to extracting them from git, building policies
>>> and scripts around that.  I'm pretty sure other significant projecs are
>>> also extracting their ChangeLogs from git.
>>>
>>> We could do the same, selecting some magic date as the cutover point
>>> after which future ChangeLogs are extracted from GIT.  In fact, that's
>>> precisely what I'd like to see us do.
>>
>> The GCC10 release date would seem a good point to do this.  That gives us around
>> 3 months to figure the details (and get stakeholder buy-in)
> Yup.  That would be what I'd recommend we shoot for.  As you say it
> gives time to work on the details and for folks to start changing their
> habits.
> 
> jeff
> 


What about the maintained branches?  Do they change as well?  or would 
they keep the existing ChangeLog files?  If the latter, then backports 
will need more care, but if the former, then the cut-over date needs 
careful thought as they follow a different cadence.

R.

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Nathan Sidwell]

On 2/3/20 5:15 AM, Richard Earnshaw (lists) wrote:
> On 25/01/2020 16:11, Jeff Law wrote:
>> On Sat, 2020-01-25 at 10:50 -0500, Nathan Sidwell wrote:
>>> On 1/24/20 4:36 PM, Jeff Law wrote:
>>>> On Fri, 2020-01-24 at 20:32 +0100, Eric Botcazou wrote:
>>>>>> I strongly prefer to move towards relying on the git log.
>>>>>
>>>>> In my experience the output of git log is a total mess so cannot 
>>>>> replace
>>>>> ChangeLogs.  But we can well decide to drop ChangeLog for the 
>>>>> testsuite.
>>>> Well, glibc has moved to extracting them from git, building policies
>>>> and scripts around that.  I'm pretty sure other significant projecs are
>>>> also extracting their ChangeLogs from git.
>>>>
>>>> We could do the same, selecting some magic date as the cutover point
>>>> after which future ChangeLogs are extracted from GIT.  In fact, that's
>>>> precisely what I'd like to see us do.
>>>
>>> The GCC10 release date would seem a good point to do this.  That 
>>> gives us around
>>> 3 months to figure the details (and get stakeholder buy-in)
>> Yup.  That would be what I'd recommend we shoot for.  As you say it
>> gives time to work on the details and for folks to start changing their
>> habits.
>>
>> jeff
>>
> 
> 
> What about the maintained branches?  Do they change as well?  or would 
> they keep the existing ChangeLog files?  If the latter, then backports 
> will need more care, but if the former, then the cut-over date needs 
> careful thought as they follow a different cadence.

I have a mild preference for release branches to follow trunk's 
behaviour -- that's easier to remember.  But, I don't look after a 
release branch.

I'm not sure why you think the release cadence affects that?  Just add a 
top entry to all testsuite changelogs indicating it's no longer used. 
Perhaps I'm missing something.

Other branches at the branch owner's discretion.

nathan

-- 
Nathan Sidwell