Automatically generated ChangeLog files - script

Automatically generated ChangeLog files - script

[Martin Liška]

Hello.

Based on the initial discussion with Richi and Jakub, we would like to remove
need to update ChangeLog files with each git commit. Instead, a script will be
used to extract ChangeLog entries from a git commit messages.

I've started with extraction of ~2K commits since we switched to git (via git format-patch).
The script [1] parses majority of the git commits and provides error message when a format
is not supported. We currently support rich variety of formats and I tried to document
that here: [2]. Analysis of the existing commits can be seen here: [3] and the script
can now parse 1471/2032 of git commits.

The script can also generate a final version of ChangeLog entries, e.g.:

$ ./changelog.py patches/1957-c-generic-lambda-forwarding-function-PR94546.patch
OK
------ gcc/cp/ChangeLog ------
2020-04-22  Jason Merrill  <jason@redhat.com>

	PR c++/94546
	* pt.c (register_parameter_specializations): If the instantiation is
	still a parameter pack, don't wrap it in a NONTYPE_ARGUMENT_PACK.
	(tsubst_pack_expansion, tsubst_expr): Adjust.
------ gcc/testsuite/ChangeLog ------
2020-04-22  Jason Merrill  <jason@redhat.com>

	gcc/testsuite/g++.dg/cpp2a/lambda-generic-variadic20.C: New file.

Currently supported features can be seen here: [4]. Feel free to test the script
and suggest what can we improve. Our integration plan is to come up with
a new git hook that will first work only in a reporting mode (no commits refusal).
Later on, we can add the automatic generation of ChangeLog entries.

Martin

[1] https://github.com/marxin/gcc-changelog
[2] https://github.com/marxin/gcc-changelog#supported-changelog-format
[3] https://raw.githubusercontent.com/marxin/gcc-changelog/master/report.txt
[4] https://github.com/marxin/gcc-changelog/issues/1#issuecomment-620162080

Re: Automatically generated ChangeLog files - script

[Jakub Jelinek]

On Thu, Apr 30, 2020 at 03:29:10PM +0200, Martin Liška wrote:
> $ ./changelog.py patches/1957-c-generic-lambda-forwarding-function-PR94546.patch
> OK
> ------ gcc/cp/ChangeLog ------
> 2020-04-22  Jason Merrill  <jason@redhat.com>
> 
> 	PR c++/94546
> 	* pt.c (register_parameter_specializations): If the instantiation is
> 	still a parameter pack, don't wrap it in a NONTYPE_ARGUMENT_PACK.
> 	(tsubst_pack_expansion, tsubst_expr): Adjust.
> ------ gcc/testsuite/ChangeLog ------
> 2020-04-22  Jason Merrill  <jason@redhat.com>
> 
> 	gcc/testsuite/g++.dg/cpp2a/lambda-generic-variadic20.C: New file.

If this is what is really created, then for the new file, missing * space,
gcc/testsuite/ that shouldn't be there and missing PR c++/94546 line above
it.

> Currently supported features can be seen here: [4]. Feel free to test the script
> and suggest what can we improve. Our integration plan is to come up with
> a new git hook that will first work only in a reporting mode (no commits refusal).
> Later on, we can add the automatic generation of ChangeLog entries.

Once it is enforced we'd probably disallow touching ChangeLog files in a
commit with other changed files and during the maintainer-scripts/update_version_git
script ought to check in in addition to the DATESTAMP updates in the same
commit also updates to all the ChangeLog files since the last DATESTAMP
commit.  That way, if there is some ChangeLog entry screw-up that gets
through the script, one could still wait until DATESTAMP update and
afterwards in a separate commit adjust the ChangeLog files.

	Jakub

Re: Automatically generated ChangeLog files - script

[Martin Liška]

On 4/30/20 3:45 PM, Jakub Jelinek wrote:
> If this is what is really created, then for the new file, missing * space,
> gcc/testsuite/ that shouldn't be there and missing PR c++/94546 line above
> it.

I've just fixed all these, thanks!

About the 'PR c++/94546': it's mentioned in context of 'gcc/cp/ChangeLog',
so I would not add it.

Martin

Re: Automatically generated ChangeLog files - script

[Jakub Jelinek]

On Thu, Apr 30, 2020 at 05:14:34PM +0200, Martin Liška wrote:
> On 4/30/20 3:45 PM, Jakub Jelinek wrote:
> > If this is what is really created, then for the new file, missing * space,
> > gcc/testsuite/ that shouldn't be there and missing PR c++/94546 line above
> > it.
> 
> I've just fixed all these, thanks!
> 
> About the 'PR c++/94546': it's mentioned in context of 'gcc/cp/ChangeLog',
> so I would not add it.

IMHO the PR lines etc. should always go to all ChangeLog files, unless the
user clearly expresses he doesn't want that.  People shouldn't be committing
unrelated changes together (to trunk or release branches) in one commit.

So, if there is just one explicit record for one ChangeLog and new file
is needed for another one, IMHO that one should get it too.
One can prevent that by adding explicit entry.

gcc/cp/ChangeLog
2020-02-02  Me  <me@foobar>

	PR c++/12345
	* ...
should add PR c++/12345 even to gcc/testsuite/ChangeLog, similarly
2020-02-02  Me  <me@foobar>

	PR c++/12345
gcc/cp/
	* ...
(this one is clearly request that the first 3 lines go everywhere, just the
individual entries go into the subparts.
Now, if I want just PR c++/12345 in the gcc/cp/ChangeLog and not in
gcc/testsuite/ChangeLog, I can write:
2020-02-02  Me  <me@foobar>

gcc/cp/
	PR c++/12345
	* ...
gcc/testsuite/
	* ...: New file.
or e.g.
gcc/cp/ChangeLog:
2020-02-02  Me  <me@foobar>

	PR c++/12345
	* ...
gcc/testsuite/ChangeLog:
2020-02-02  Me  <me@foobar>

	* ...: New file.
or e.g.
2020-02-02  Me  <me@foobar>

gcc/cp/
	PR c++/12345
	* ...
gcc/c/
	PR c/12456
	* ...
In this last entry, as there were more than one ChangeLog snippets and
they had different PR lines, I wouldn't put anything into the automatically
added gcc/testsuite/ChangeLog entry.

Does this make sense?  Basically, try to do what is most likely the user
wanted.

	Jakub

Re: Automatically generated ChangeLog files - script

[Martin Liška]

On 4/30/20 5:29 PM, Jakub Jelinek wrote:
> In this last entry, as there were more than one ChangeLog snippets and
> they had different PR lines, I wouldn't put anything into the automatically
> added gcc/testsuite/ChangeLog entry.
> 
> Does this make sense?  Basically, try to do what is most likely the user
> wanted.

Works for me, it's implemented right now.

Martin

Re: Automatically generated ChangeLog files - script

[Martin Liška]

> Once it is enforced we'd probably disallow touching ChangeLog files in a
> commit with other changed files and during the maintainer-scripts/update_version_git
> script ought to check in in addition to the DATESTAMP updates in the same
> commit also updates to all the ChangeLog files since the last DATESTAMP
> commit.  That way, if there is some ChangeLog entry screw-up that gets
> through the script, one could still wait until DATESTAMP update and
> afterwards in a separate commit adjust the ChangeLog files.
> 
> 	Jakub
> 

Hello.

I've made a refactoring of the script I tend to divide it into 3 components:
- abstract GitCommit class that can check commit message and format and
   can generate final ChangeLog entries
- integration part into the git hooks ([1]): it should get git message and changed
   files and process _only_ format validation step; note that our hooks use Python2
   and so we should get message and modified files via a subprocess execution:
   $ git log -n1 ac6eaa55a5199196ea0a25763114ce05333a14d3  --format=%b
     When I implemented C++20 parenthesized initialization of aggregates
     I introduced this bogus cp_unevaluated_operand check, thus disabling
     ...
   $ git diff ac6eaa55a5199196ea0a25763114ce05333a14d3~..ac6eaa55a5199196ea0a25763114ce05333a14d3 --name-status
   M       gcc/cp/ChangeLog
   M       gcc/cp/call.c
   M       gcc/testsuite/ChangeLog
   A       gcc/testsuite/g++.dg/cpp2a/paren-init21.C

- a DATESTAMP update script - it can be a Python3 script that can utilize a nice git abstraction
   library: [2]; the script was added and can be tested with:
$ ./git_repo.py ~/Programming/gcc 6607bdd99994c834f92fce924abdaea3405f62dc
------ gcc/ChangeLog ------
2020-05-02  H.J. Lu  <hjl.tools@gmail.com>

	PR target/93492
	* cfgexpand.c (pass_expand::execute): Set crtl->patch_area_size
	and crtl->patch_area_entry.
	* emit-rtl.h (rtl_data): Add patch_area_size and patch_area_entry.
	* opts.c (common_handle_option): Limit
	function_entry_patch_area_size and function_entry_patch_area_start
	to USHRT_MAX.  Fix a typo in error message.
	* varasm.c (assemble_start_function): Use crtl->patch_area_size
	and crtl->patch_area_entry.
	* doc/invoke.texi: Document the maximum value for
	-fpatchable-function-entry.
------ gcc/c-family/ChangeLog ------
2020-05-02  H.J. Lu  <hjl.tools@gmail.com>

	PR target/93492
	* c-attribs.c (handle_patchable_function_entry_attribute): Limit
	value to USHRT_MAX (65535).
------ gcc/testsuite/ChangeLog ------
2020-05-02  H.J. Lu  <hjl.tools@gmail.com>

	PR target/93492
	* c-c++-common/patchable_function_entry-error-1.c: New test.
	* c-c++-common/patchable_function_entry-error-2.c: Likewise.
	* c-c++-common/patchable_function_entry-error-3.c: Likewise.

What's missing right now is how will we declare a Backport format.
Can we just use something like: 'Backport from 6607bdd99994c834f92fce924abdaea3405f62dc'?
It can rapidly simplify parsing of git messages and the git_repo.py can generate
corresponding ChangeLog entries.

Martin

[1] https://github.com/AdaCore/git-hooks
[2] https://gitpython.readthedocs.io/en/stable/tutorial.html

Re: Automatically generated ChangeLog files - script

[Jakub Jelinek]

On Mon, May 04, 2020 at 08:56:16PM +0200, Martin Liška wrote:
> What's missing right now is how will we declare a Backport format.
> Can we just use something like: 'Backport from 6607bdd99994c834f92fce924abdaea3405f62dc'?

No.  What we should allow is that people just git cherry-pick r11-1234-g123456
into a release branch and push it (of course after testing),
so we don't want the user to change the commit log in that case.
Can one just assume backport if the commit in question is already an ancestor of the master
or a newer release branch and is going to be added into the release branch?
The checking portion of the git-changelog script should behave the same in
that case, and the add ChangeLog snippets part should detect this (when it
is run as pre-commit hook I'm not sure if it is possible to check if it is
already ancestor of trunk, but when it would be run during DATESTAMP bump it
should easily check that) and compare the commit date between trunk and the
release branch commit, if they are the same, not do anything special,
otherwise add those Backported from ... lines.

	Jakub

Re: Automatically generated ChangeLog files - script

[Tobias Burnus]

On 5/4/20 9:05 PM, Jakub Jelinek via Gcc wrote:
> On Mon, May 04, 2020 at 08:56:16PM +0200, Martin Liška wrote:
>> What's missing right now is how will we declare a Backport format.
>> Can we just use something like: 'Backport from 6607bdd99994c834f92fce924abdaea3405f62dc'?
> No.  What we should allow is that people just git cherry-pick r11-1234-g123456
> into a release branch and push it (of course after testing),
> so we don't want the user to change the commit log in that case.

How does one handle partial backports? I mean those where
only half of the original patch applies easily to the old
branch and the other half is simply ignored? This happened
a couple of times to me, especially when applying a patch
to the last but one release branch.

Tobias

-----------------
Mentor Graphics (Deutschland) GmbH, Arnulfstraße 201, 80634 München / Germany
Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Alexander Walter

Re: Automatically generated ChangeLog files - script

[H.J. Lu]

On Mon, May 4, 2020 at 12:24 PM Tobias Burnus <tobias@codesourcery.com> wrote:
>
> On 5/4/20 9:05 PM, Jakub Jelinek via Gcc wrote:
> > On Mon, May 04, 2020 at 08:56:16PM +0200, Martin Liška wrote:
> >> What's missing right now is how will we declare a Backport format.
> >> Can we just use something like: 'Backport from 6607bdd99994c834f92fce924abdaea3405f62dc'?
> > No.  What we should allow is that people just git cherry-pick r11-1234-g123456
> > into a release branch and push it (of course after testing),
> > so we don't want the user to change the commit log in that case.
>
> How does one handle partial backports? I mean those where
> only half of the original patch applies easily to the old
> branch and the other half is simply ignored? This happened
> a couple of times to me, especially when applying a patch
> to the last but one release branch.

You need to resolve conflicts and update commit message with

$ git cherry-pick --continue

-- 
H.J.

Re: Automatically generated ChangeLog files - script

[Mark Eggleston]

On 04/05/2020 20:28, H.J. Lu via Gcc wrote:
> On Mon, May 4, 2020 at 12:24 PM Tobias Burnus <tobias@codesourcery.com> wrote:
>> On 5/4/20 9:05 PM, Jakub Jelinek via Gcc wrote:
>>> On Mon, May 04, 2020 at 08:56:16PM +0200, Martin Liška wrote:
>>>> What's missing right now is how will we declare a Backport format.
>>>> Can we just use something like: 'Backport from 6607bdd99994c834f92fce924abdaea3405f62dc'?
>>> No.  What we should allow is that people just git cherry-pick r11-1234-g123456
>>> into a release branch and push it (of course after testing),
>>> so we don't want the user to change the commit log in that case.
>> How does one handle partial backports? I mean those where
>> only half of the original patch applies easily to the old
>> branch and the other half is simply ignored? This happened
>> a couple of times to me, especially when applying a patch
>> to the last but one release branch.
> You need to resolve conflicts and update commit message with
>
> $ git cherry-pick --continue
>
The resolution of conflicts is likely to affect the changes intended for 
the change log files. The commit message can be updated accordingly using:

$ git commit --amend

-- 

https://www.codethink.co.uk/privacy.html

Automatically generated ChangeLog files - PHASE 1

[Martin Liška]

Hi.

Thanks to Jakub, we finally set up an experimental environment:
gcc.gnu.org/home/gccadmin/gcc-reposurgeon-8.git

The repository now contains a new pre-commit hook that validates
the git commit format ([1]) and provides a reasonable error message
when violated. The hook is based on [2] and the page also contains
a fuzzy definition of what is supported. Cloning [2], one can also
check what will be added to ChangeLog entries by:

$ ./git_changelog.py /home/marxin/Programming/gcc-reposurgeon-8 8a37df5e5cb2de8302f9412173103593ec53961e
------ gcc/ChangeLog ------
2020-01-13  Martin Jambor  <mjambor@suse.cz>

	PR ipa/93223
	* ipa-cp.c (devirtualization_time_bonus): Check whether isummary is
	NULL.
------ gcc/testsuite/ChangeLog ------
2020-01-13  Martin Jambor  <mjambor@suse.cz>

	PR ipa/93223
	testsuite/
	* g++.dg/ipa/pr93223.C: New test.

(one needs [3] Python package for that)

We encourage people to test both the hook and the script. We hope we'll cover
majority of the used formats. I also support _not_ using DATESTAMP and committer
name, these can be automatically deduced from a commit. That will simplify workflow
as people won't have to adjust a message before pushing.

Unresolved questions:
- format of reverted patches
- what to do with backports

Here I suggest to use native 'git revert XYZ' and 'git cherry-pick -x XYZ'.
Doing that the commit messages will provide link to original commit and the script
can later append corresponding 'Backported ..' or 'Reverted' line.

For the possible issues or questions, please open a Github issue at [4].

Thoughts?
Martin

[1] https://github.com/marxin/git-hooks/tree/gcc-changelog
[2] https://github.com/marxin/gcc-changelog
[3] https://gitpython.readthedocs.io/en/stable/intro.html
[4] https://github.com/marxin/gcc-changelog/issues

  

Re: Automatically generated ChangeLog files - PHASE 1

[Martin Liška]

I'm also CCing gcc-patches and fortran ML.

Martin

On 5/12/20 11:05 AM, Martin Liška wrote:
> Hi.
> 
> Thanks to Jakub, we finally set up an experimental environment:
> gcc.gnu.org/home/gccadmin/gcc-reposurgeon-8.git
> 
> The repository now contains a new pre-commit hook that validates
> the git commit format ([1]) and provides a reasonable error message
> when violated. The hook is based on [2] and the page also contains
> a fuzzy definition of what is supported. Cloning [2], one can also
> check what will be added to ChangeLog entries by:
> 
> $ ./git_changelog.py /home/marxin/Programming/gcc-reposurgeon-8 8a37df5e5cb2de8302f9412173103593ec53961e
> ------ gcc/ChangeLog ------
> 2020-01-13  Martin Jambor  <mjambor@suse.cz>
> 
>      PR ipa/93223
>      * ipa-cp.c (devirtualization_time_bonus): Check whether isummary is
>      NULL.
> ------ gcc/testsuite/ChangeLog ------
> 2020-01-13  Martin Jambor  <mjambor@suse.cz>
> 
>      PR ipa/93223
>      testsuite/
>      * g++.dg/ipa/pr93223.C: New test.
> 
> (one needs [3] Python package for that)
> 
> We encourage people to test both the hook and the script. We hope we'll cover
> majority of the used formats. I also support _not_ using DATESTAMP and committer
> name, these can be automatically deduced from a commit. That will simplify workflow
> as people won't have to adjust a message before pushing.
> 
> Unresolved questions:
> - format of reverted patches
> - what to do with backports
> 
> Here I suggest to use native 'git revert XYZ' and 'git cherry-pick -x XYZ'.
> Doing that the commit messages will provide link to original commit and the script
> can later append corresponding 'Backported ..' or 'Reverted' line.
> 
> For the possible issues or questions, please open a Github issue at [4].
> 
> Thoughts?
> Martin
> 
> [1] https://github.com/marxin/git-hooks/tree/gcc-changelog
> [2] https://github.com/marxin/gcc-changelog
> [3] https://gitpython.readthedocs.io/en/stable/intro.html
> [4] https://github.com/marxin/gcc-changelog/issues
> 
> 

Re: Automatically generated ChangeLog files - PHASE 1

[Jakub Jelinek]

On Tue, May 12, 2020 at 11:05:58AM +0200, Martin Liška wrote:
> Thanks to Jakub, we finally set up an experimental environment:
> gcc.gnu.org/home/gccadmin/gcc-reposurgeon-8.git
> 
> The repository now contains a new pre-commit hook that validates
> the git commit format ([1]) and provides a reasonable error message
> when violated. The hook is based on [2] and the page also contains
> a fuzzy definition of what is supported. Cloning [2], one can also
> check what will be added to ChangeLog entries by:

I'd like to add, when testing in gcc-reposurgeon-8.git, please avoid
referencing non-testing PRs so that they won't be spammed, you can e.g. use
PR93218 which has been created for the testing during svn -> git conversion.

	Jakub

Re: Automatically generated ChangeLog files - PHASE 1

[Richard Sandiford]

Martin Liška <mliska@suse.cz> writes:
> Hi.
>
> Thanks to Jakub, we finally set up an experimental environment:
> gcc.gnu.org/home/gccadmin/gcc-reposurgeon-8.git
>
> The repository now contains a new pre-commit hook that validates
> the git commit format ([1]) and provides a reasonable error message
> when violated. The hook is based on [2] and the page also contains
> a fuzzy definition of what is supported. Cloning [2], one can also
> check what will be added to ChangeLog entries by:
>
> $ ./git_changelog.py /home/marxin/Programming/gcc-reposurgeon-8 8a37df5e5cb2de8302f9412173103593ec53961e
> ------ gcc/ChangeLog ------
> 2020-01-13  Martin Jambor  <mjambor@suse.cz>
>
> 	PR ipa/93223
> 	* ipa-cp.c (devirtualization_time_bonus): Check whether isummary is
> 	NULL.
> ------ gcc/testsuite/ChangeLog ------
> 2020-01-13  Martin Jambor  <mjambor@suse.cz>
>
> 	PR ipa/93223
> 	testsuite/
> 	* g++.dg/ipa/pr93223.C: New test.
>
> (one needs [3] Python package for that)

As far as this particular example goes, shouldn't the "testsuite/" line
be dropped from the above?

Thanks,
Richard

Re: Automatically generated ChangeLog files - PHASE 1

[Martin Liška]

On 5/13/20 10:16 AM, Richard Sandiford wrote:
> As far as this particular example goes, shouldn't the "testsuite/" line
> be dropped from the above?

Good point. Fixes now with:

$ ./git_email.py patches/0020-IPA-Avoid-segfault-in-devirtualization_time_bonus-PR.patch
Errors:
first line should start with a tab, asterisk and space:"	testsuite/"

Martin

Re: Automatically generated ChangeLog files - PHASE 1

[Richard Biener]

On Wed, May 13, 2020 at 11:27 AM Martin Liška <mliska@suse.cz> wrote:
>
> On 5/13/20 10:16 AM, Richard Sandiford wrote:
> > As far as this particular example goes, shouldn't the "testsuite/" line
> > be dropped from the above?
>
> Good point. Fixes now with:
>
> $ ./git_email.py patches/0020-IPA-Avoid-segfault-in-devirtualization_time_bonus-PR.patch
> Errors:
> first line should start with a tab, asterisk and space:"        testsuite/"

Hmm, it's OK in the commit but it should be omitted in the
ChangeLog files.

Richard.

> Martin

Re: Automatically generated ChangeLog files - PHASE 1

[Martin Liška]

On 5/13/20 11:29 AM, Richard Biener wrote:
> Hmm, it's OK in the commit but it should be omitted in the
> ChangeLog files.

No, ChangeLog file identification should not be prepended
with a tab.
"testsuite/" will work.

Martin

Re: Automatically generated ChangeLog files - PHASE 1

[Thomas Koenig]

Hi Martin,

> Thanks to Jakub, we finally set up an experimental environment:
> gcc.gnu.org/home/gccadmin/gcc-reposurgeon-8.git

How does one go about testing this?  It would be helpful if you
could explain this assuming that the Fortran people have at best
an extremely shallow knowledge of git and no knowledge of Python :-)

Regards

	Thomas

Re: Automatically generated ChangeLog files - PHASE 1

[Jozef Lawrynowicz]

Hi,

On Tue, 12 May 2020 11:05:58 +0200
Martin Liška <mliska@suse.cz> wrote:

> Hi.
> 
> Thanks to Jakub, we finally set up an experimental environment:
> gcc.gnu.org/home/gccadmin/gcc-reposurgeon-8.git
> 
> The repository now contains a new pre-commit hook that validates
> the git commit format ([1]) and provides a reasonable error message
> when violated. The hook is based on [2] and the page also contains
> a fuzzy definition of what is supported. Cloning [2], one can also
> check what will be added to ChangeLog entries by:
> 
> $ ./git_changelog.py /home/marxin/Programming/gcc-reposurgeon-8 8a37df5e5cb2de8302f9412173103593ec53961e

"git_changelog.py" is nowhere to be found in the gcc-changelog or
gcc-reposurgeon-8 repos. Is it equivalent to any of the other scripts in
gcc-changelog?

Jozef

Re: Automatically generated ChangeLog files - PHASE 1

[Martin Liška]

On 5/13/20 11:50 AM, Jozef Lawrynowicz wrote:
> "git_changelog.py" is nowhere to be found in the gcc-changelog or
> gcc-reposurgeon-8 repos. Is it equivalent to any of the other scripts in
> gcc-changelog?

Yes, I made some refactoring and I'm going to send a proper gcc-patches submission.
The scripts will be located in contrib folder.

Martin

Re: Automatically generated ChangeLog files - PHASE 1

[Richard Earnshaw]

On 12/05/2020 10:05, Martin Liška wrote:
> Hi.
> 
> Thanks to Jakub, we finally set up an experimental environment:
> gcc.gnu.org/home/gccadmin/gcc-reposurgeon-8.git
> 
> The repository now contains a new pre-commit hook that validates
> the git commit format ([1]) and provides a reasonable error message
> when violated. The hook is based on [2] and the page also contains
> a fuzzy definition of what is supported. Cloning [2], one can also
> check what will be added to ChangeLog entries by:
> 
> $ ./git_changelog.py /home/marxin/Programming/gcc-reposurgeon-8
> 8a37df5e5cb2de8302f9412173103593ec53961e
> ------ gcc/ChangeLog ------

I realize you're trying to create a unique marker, but this is a bit of
a mouthful to type for each entry.  Couldn't we have a simpler unique
syntax which is very unlikely to occur in a normal part of a log file?
Perhaps something like

@@CL gcc

would be equivalent to the above and is much less than half the length
to type.

Also, a marker to indicate the end, so that if git annotations appear,
they can never get appended accidentally, eg, an empty "@@CL" marker
with no directory.


> 2020-01-13  Martin Jambor  <mjambor@suse.cz>
> 
>     PR ipa/93223
>     * ipa-cp.c (devirtualization_time_bonus): Check whether isummary is
>     NULL.
> ------ gcc/testsuite/ChangeLog ------
> 2020-01-13  Martin Jambor  <mjambor@suse.cz>
> 
>     PR ipa/93223
>     testsuite/
>     * g++.dg/ipa/pr93223.C: New test.
> 
> (one needs [3] Python package for that)
> 
> We encourage people to test both the hook and the script. We hope we'll
> cover
> majority of the used formats. I also support _not_ using DATESTAMP and
> committer
> name, these can be automatically deduced from a commit. That will
> simplify workflow
> as people won't have to adjust a message before pushing.
> 
> Unresolved questions:
> - format of reverted patches
> - what to do with backports
> 
> Here I suggest to use native 'git revert XYZ' and 'git cherry-pick -x XYZ'.
> Doing that the commit messages will provide link to original commit and
> the script
> can later append corresponding 'Backported ..' or 'Reverted' line.
> 
> For the possible issues or questions, please open a Github issue at [4].
> 
> Thoughts?
> Martin
> 
> [1] https://github.com/marxin/git-hooks/tree/gcc-changelog
> [2] https://github.com/marxin/gcc-changelog
> [3] https://gitpython.readthedocs.io/en/stable/intro.html
> [4] https://github.com/marxin/gcc-changelog/issues
> 
>  

ChangeLog files - server and client scripts

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 1876 bytes --]

Hi.

I'm sending the gcc-changelog relates scripts which should be added to contrib
folder. The patch contains:
- git_check_commit.py - checking script that verifies git message format
- git_update_version.py - a replacement of maintainer-scripts/update_version_git which
bumps DATESTAMP and generates ChangeLog entries (for now into ChangeLog.test files)
- git_commit.py, git_email.py and git_repository.py - helper classes

I also added a new git.config alias: 'gcc-verify' which can be used in the following
way:

$ git gcc-verify HEAD~2..HEAD -p -n
Checking 0e4009e9d523270e26856d2441c1be3d8119a477
OK
@@CL contrib
2020-05-13  Martin Liska  <mliska@suse.cz>

	* gcc-changelog/git_check_commit.py: New file.
	* gcc-changelog/git_commit.py: New file.
	* gcc-changelog/git_email.py: New file.
	* gcc-changelog/git_repository.py: New file.
	* gcc-changelog/git_update_version.py: New file.
	* gcc-git-customization.sh: Add gcc-verify alias.
@@CL
Checking 18edc195442291525e04f0fa4d5ef972155117da
OK
@@CL gcc
2020-05-13  Jakub Jelinek  <jakub@redhat.com>

	PR debug/95080
	* cfgrtl.c (purge_dead_edges): Skip over debug and note insns even
	if the last insn is a note.
@@CL gcc/testsuite
2020-05-13  Jakub Jelinek  <jakub@redhat.com>

	PR debug/95080
	* g++.dg/opt/pr95080.C: New test.
@@CL

Note the -n option which disables _strict mode_ (modification of both ChangeLog
and another files).

The second part is git hook that will reject all commits for release and master branches.
that violate ChangeLog format. Right now, strict mode is disabled in the hooks.

What's still missing to be done is format of Revert and Backport commits.
I suggest to use native 'git revert XYZ' and 'git cherry-pick -x XYZ'.
Doing that the commit messages will provide link to original commit and the script
can later append corresponding 'Backported ..' or 'Reverted' line.

Thoughts?
Martin

[-- Attachment #2: 0001-Add-gcc-changelog-related-scripts.patch --]
[-- Type: text/x-patch, Size: 33976 bytes --]

From 0e4009e9d523270e26856d2441c1be3d8119a477 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Wed, 13 May 2020 12:22:39 +0200
Subject: [PATCH] Add gcc-changelog related scripts.

contrib/ChangeLog:

2020-05-13  Martin Liska  <mliska@suse.cz>

	* gcc-changelog/git_check_commit.py: New file.
	* gcc-changelog/git_commit.py: New file.
	* gcc-changelog/git_email.py: New file.
	* gcc-changelog/git_repository.py: New file.
	* gcc-changelog/git_update_version.py: New file.
	* gcc-git-customization.sh: Add gcc-verify alias.
---
 contrib/gcc-changelog/git_check_commit.py   |  49 ++
 contrib/gcc-changelog/git_commit.py         | 536 ++++++++++++++++++++
 contrib/gcc-changelog/git_email.py          |  92 ++++
 contrib/gcc-changelog/git_repository.py     |  60 +++
 contrib/gcc-changelog/git_update_version.py | 105 ++++
 contrib/gcc-git-customization.sh            |   2 +
 6 files changed, 844 insertions(+)
 create mode 100755 contrib/gcc-changelog/git_check_commit.py
 create mode 100755 contrib/gcc-changelog/git_commit.py
 create mode 100755 contrib/gcc-changelog/git_email.py
 create mode 100755 contrib/gcc-changelog/git_repository.py
 create mode 100755 contrib/gcc-changelog/git_update_version.py

diff --git a/contrib/gcc-changelog/git_check_commit.py b/contrib/gcc-changelog/git_check_commit.py
new file mode 100755
index 00000000000..b2d1d08a242
--- /dev/null
+++ b/contrib/gcc-changelog/git_check_commit.py
@@ -0,0 +1,49 @@
+#!/usr/bin/env python3
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License as published by the Free
+# Software Foundation; either version 3, or (at your option) any later
+# version.
+#
+# GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+# for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.  */
+
+import argparse
+
+from git_repository import parse_git_revisions
+
+parser = argparse.ArgumentParser(description='Check git ChangeLog format '
+                                 'of a commit')
+parser.add_argument('revisions',
+                    help='Git revisions (e.g. hash~5..hash or just hash)')
+parser.add_argument('-g', '--git-path', default='.',
+                    help='Path to git repository')
+parser.add_argument('-p', '--print-changelog', action='store_true',
+                    help='Print final changelog entires')
+parser.add_argument('-n', '--allow-non-strict-mode', action='store_true',
+                    help='Allow non-strict mode (change in both ChangeLog and '
+                    'other files.')
+args = parser.parse_args()
+
+retval = 0
+for git_commit in parse_git_revisions(args.git_path, args.revisions,
+                                      not args.allow_non_strict_mode):
+    print('Checking %s' % git_commit.hexsha)
+    if git_commit.success:
+        print('OK')
+        if args.print_changelog:
+            git_commit.print_output()
+    else:
+        for error in git_commit.errors:
+            print('ERR: %s' % error)
+        retval = 1
+
+exit(retval)
diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
new file mode 100755
index 00000000000..a434f949ee6
--- /dev/null
+++ b/contrib/gcc-changelog/git_commit.py
@@ -0,0 +1,536 @@
+#!/usr/bin/env python3
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License as published by the Free
+# Software Foundation; either version 3, or (at your option) any later
+# version.
+#
+# GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+# for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.  */
+
+import os
+import re
+
+changelog_locations = set([
+    'config',
+    'contrib',
+    'contrib/header-tools',
+    'contrib/reghunt',
+    'contrib/regression',
+    'fixincludes',
+    'gcc/ada',
+    'gcc/analyzer',
+    'gcc/brig',
+    'gcc/c',
+    'gcc/c-family',
+    'gcc',
+    'gcc/cp',
+    'gcc/d',
+    'gcc/fortran',
+    'gcc/go',
+    'gcc/jit',
+    'gcc/lto',
+    'gcc/objc',
+    'gcc/objcp',
+    'gcc/po',
+    'gcc/testsuite',
+    'gnattools',
+    'gotools',
+    'include',
+    'intl',
+    'libada',
+    'libatomic',
+    'libbacktrace',
+    'libcc1',
+    'libcpp',
+    'libcpp/po',
+    'libdecnumber',
+    'libffi',
+    'libgcc',
+    'libgcc/config/avr/libf7',
+    'libgcc/config/libbid',
+    'libgfortran',
+    'libgomp',
+    'libhsail-rt',
+    'libiberty',
+    'libitm',
+    'libobjc',
+    'liboffloadmic',
+    'libphobos',
+    'libquadmath',
+    'libsanitizer',
+    'libssp',
+    'libstdc++-v3',
+    'libvtv',
+    'lto-plugin',
+    'maintainer-scripts',
+    'zlib'])
+
+bug_components = set([
+    'ada',
+    'analyzer',
+    'boehm-gc',
+    'bootstrap',
+    'c',
+    'c++',
+    'd',
+    'debug',
+    'demangler',
+    'driver',
+    'fastjar',
+    'fortran',
+    'gcov-profile',
+    'go',
+    'hsa',
+    'inline-asm',
+    'ipa',
+    'java',
+    'jit',
+    'libbacktrace',
+    'libf2c',
+    'libffi',
+    'libfortran',
+    'libgcc',
+    'libgcj',
+    'libgomp',
+    'libitm',
+    'libobjc',
+    'libquadmath',
+    'libstdc++',
+    'lto',
+    'middle-end',
+    'modula2',
+    'objc',
+    'objc++',
+    'other',
+    'pch',
+    'pending',
+    'plugins',
+    'preprocessor',
+    'regression',
+    'rtl-optimization',
+    'sanitizer',
+    'spam',
+    'target',
+    'testsuite',
+    'translation',
+    'tree-optimization',
+    'web'])
+
+ignored_prefixes = [
+    'gcc/d/dmd/',
+    'gcc/go/frontend/',
+    'libgo/',
+    'libphobos/libdruntime',
+    'libphobos/src/',
+    'libsanitizer/',
+    ]
+
+misc_files = [
+    'gcc/DATESTAMP',
+    'gcc/BASE-VER',
+    'gcc/DEV-PHASE'
+    ]
+
+author_line_regex = \
+        re.compile(r'^(?P<datetime>\d{4}-\d{2}-\d{2})\ {2}(?P<name>.*  <.*>)')
+additional_author_regex = re.compile(r'^\t(?P<spaces>\ *)?(?P<name>.*  <.*>)')
+changelog_regex = re.compile(r'^([a-z0-9+-/]*)/ChangeLog:?')
+pr_regex = re.compile(r'\tPR (?P<component>[a-z+-]+\/)?([0-9]+)$')
+star_prefix_regex = re.compile(r'\t\*(?P<spaces>\ *)(?P<content>.*)')
+
+LINE_LIMIT = 100
+TAB_WIDTH = 8
+CO_AUTHORED_BY_PREFIX = 'co-authored-by: '
+
+
+class Error:
+    def __init__(self, message, line=None):
+        self.message = message
+        self.line = line
+
+    def __repr__(self):
+        s = self.message
+        if self.line:
+            s += ':"%s"' % self.line
+        return s
+
+
+class ChangeLogEntry:
+    def __init__(self, folder, authors, prs):
+        self.folder = folder
+        # Python2 has not 'copy' function
+        self.author_lines = list(authors)
+        self.initial_prs = list(prs)
+        self.prs = list(prs)
+        self.lines = []
+
+    @property
+    def files(self):
+        files = []
+        for line in self.lines:
+            m = star_prefix_regex.match(line)
+            if m:
+                line = m.group('content')
+                if '(' in line:
+                    line = line[:line.index('(')]
+                if ':' in line:
+                    line = line[:line.index(':')]
+                for file in line.split(','):
+                    file = file.strip()
+                    if file:
+                        files.append(file)
+        return files
+
+    @property
+    def datetime(self):
+        for author in self.author_lines:
+            if author[1]:
+                return author[1]
+        return None
+
+    @property
+    def authors(self):
+        return [author_line[0] for author_line in self.author_lines]
+
+    @property
+    def is_empty(self):
+        return not self.lines and self.prs == self.initial_prs
+
+
+class GitCommit:
+    def __init__(self, hexsha, date, author, body, modified_files,
+                 strict=True):
+        self.hexsha = hexsha
+        self.lines = body
+        self.modified_files = modified_files
+        self.message = None
+        self.changes = None
+        self.changelog_entries = []
+        self.errors = []
+        self.date = date
+        self.author = author
+        self.top_level_authors = []
+        self.co_authors = []
+        self.top_level_prs = []
+
+        project_files = [f for f in self.modified_files
+                         if self.is_changelog_filename(f[0])
+                         or f[0] in misc_files]
+        if len(project_files) == len(self.modified_files):
+            # All modified files are only MISC files
+            return
+        elif project_files and strict:
+            self.errors.append(Error('ChangeLog, DATESTAMP, BASE-VER and '
+                                     'DEV-PHASE updates should be done '
+                                     'separately from normal commits'))
+            return
+
+        self.parse_lines()
+        if self.changes:
+            self.parse_changelog()
+            self.deduce_changelog_locations()
+            if not self.errors:
+                self.check_mentioned_files()
+                self.check_for_correct_changelog()
+
+    @property
+    def success(self):
+        return not self.errors
+
+    @property
+    def new_files(self):
+        return [x[0] for x in self.modified_files if x[1] == 'A']
+
+    @classmethod
+    def is_changelog_filename(cls, path):
+        return path.endswith('/ChangeLog') or path == 'ChangeLog'
+
+    @classmethod
+    def find_changelog_location(cls, name):
+        if name.startswith('\t'):
+            name = name[1:]
+        if name.endswith(':'):
+            name = name[:-1]
+        if name.endswith('/'):
+            name = name[:-1]
+        return name if name in changelog_locations else None
+
+    @classmethod
+    def format_git_author(cls, author):
+        assert '<' in author
+        return author.replace('<', ' <')
+
+    @classmethod
+    def parse_git_name_status(cls, string):
+        modified_files = []
+        for entry in string.split('\n'):
+            parts = entry.split('\t')
+            t = parts[0]
+            if t == 'A' or t == 'D' or t == 'M':
+                modified_files.append((parts[1], t))
+            elif t == 'R':
+                modified_files.append((parts[1], 'D'))
+                modified_files.append((parts[2], 'A'))
+        return modified_files
+
+    def parse_lines(self):
+        body = self.lines
+
+        for i, b in enumerate(body):
+            if not b:
+                continue
+            if (changelog_regex.match(b) or self.find_changelog_location(b)
+                    or star_prefix_regex.match(b) or pr_regex.match(b)
+                    or author_line_regex.match(b)):
+                self.changes = body[i:]
+                return
+        self.errors.append(Error('cannot find a ChangeLog location in '
+                                 'message'))
+
+    def parse_changelog(self):
+        last_entry = None
+        will_deduce = False
+        for line in self.changes:
+            if not line:
+                if last_entry and will_deduce:
+                    last_entry = None
+                continue
+            if line != line.rstrip():
+                self.errors.append(Error('trailing whitespace', line))
+            if len(line.replace('\t', ' ' * TAB_WIDTH)) > LINE_LIMIT:
+                self.errors.append(Error('line limit exceeds %d characters'
+                                         % LINE_LIMIT, line))
+            m = changelog_regex.match(line)
+            if m:
+                last_entry = ChangeLogEntry(m.group(1), self.top_level_authors,
+                                            self.top_level_prs)
+                self.changelog_entries.append(last_entry)
+            elif self.find_changelog_location(line):
+                last_entry = ChangeLogEntry(self.find_changelog_location(line),
+                                            self.top_level_authors,
+                                            self.top_level_prs)
+                self.changelog_entries.append(last_entry)
+            else:
+                author_tuple = None
+                pr_line = None
+                if author_line_regex.match(line):
+                    m = author_line_regex.match(line)
+                    author_tuple = (m.group('name'), m.group('datetime'))
+                elif additional_author_regex.match(line):
+                    m = additional_author_regex.match(line)
+                    if len(m.group('spaces')) != 4:
+                        msg = 'additional author must prepend with tab ' \
+                              'and 4 spaces'
+                        self.errors.append(Error(msg, line))
+                    else:
+                        author_tuple = (m.group('name'), None)
+                elif pr_regex.match(line):
+                    component = pr_regex.match(line).group('component')
+                    if not component:
+                        self.errors.append(Error('missing PR component', line))
+                        continue
+                    elif not component[:-1] in bug_components:
+                        self.errors.append(Error('invalid PR component', line))
+                        continue
+                    else:
+                        pr_line = line.lstrip()
+
+                if line.lower().startswith(CO_AUTHORED_BY_PREFIX):
+                    name = line[len(CO_AUTHORED_BY_PREFIX):]
+                    author = self.format_git_author(name)
+                    self.co_authors.append(author)
+                    continue
+
+                # ChangeLog name will be deduced later
+                if not last_entry:
+                    if author_tuple:
+                        self.top_level_authors.append(author_tuple)
+                        continue
+                    elif pr_line:
+                        # append to top_level_prs only when we haven't met
+                        # a ChangeLog entry
+                        if (pr_line not in self.top_level_prs
+                                and not self.changelog_entries):
+                            self.top_level_prs.append(pr_line)
+                        continue
+                    else:
+                        last_entry = ChangeLogEntry(None,
+                                                    self.top_level_authors,
+                                                    self.top_level_prs)
+                        self.changelog_entries.append(last_entry)
+                        will_deduce = True
+                elif author_tuple:
+                    last_entry.author_lines.append(author_tuple)
+                    continue
+
+                if not line.startswith('\t'):
+                    err = Error('line should start with a tab', line)
+                    self.errors.append(err)
+                elif pr_line:
+                    last_entry.prs.append(pr_line)
+                else:
+                    m = star_prefix_regex.match(line)
+                    if m:
+                        if len(m.group('spaces')) != 1:
+                            err = Error('one space should follow asterisk',
+                                        line)
+                            self.errors.append(err)
+                        else:
+                            last_entry.lines.append(line)
+                    else:
+                        if last_entry.is_empty:
+                            msg = 'first line should start with a tab, ' \
+                                  'asterisk and space'
+                            self.errors.append(Error(msg, line))
+                        else:
+                            last_entry.lines.append(line)
+
+    def get_file_changelog_location(self, changelog_file):
+        for file in self.modified_files:
+            if file[0] == changelog_file:
+                # root ChangeLog file
+                return ''
+            index = file[0].find('/' + changelog_file)
+            if index != -1:
+                return file[0][:index]
+        return None
+
+    def deduce_changelog_locations(self):
+        for entry in self.changelog_entries:
+            if not entry.folder:
+                changelog = None
+                for file in entry.files:
+                    location = self.get_file_changelog_location(file)
+                    if (location == ''
+                       or (location and location in changelog_locations)):
+                        if changelog and changelog != location:
+                            msg = 'could not deduce ChangeLog file, ' \
+                                  'not unique location'
+                            self.errors.append(Error(msg))
+                            return
+                        changelog = location
+                if changelog is not None:
+                    entry.folder = changelog
+                else:
+                    msg = 'could not deduce ChangeLog file'
+                    self.errors.append(Error(msg))
+
+    @classmethod
+    def in_ignored_location(cls, path):
+        for ignored in ignored_prefixes:
+            if path.startswith(ignored):
+                return True
+        return False
+
+    @classmethod
+    def get_changelog_by_path(cls, path):
+        components = path.split('/')
+        while components:
+            if '/'.join(components) in changelog_locations:
+                break
+            components = components[:-1]
+        return '/'.join(components)
+
+    def check_mentioned_files(self):
+        folder_count = len([x.folder for x in self.changelog_entries])
+        assert folder_count == len(self.changelog_entries)
+
+        mentioned_files = set()
+        for entry in self.changelog_entries:
+            if not entry.files:
+                msg = 'ChangeLog must contain a file entry'
+                self.errors.append(Error(msg, entry.folder))
+            assert not entry.folder.endswith('/')
+            for file in entry.files:
+                if not self.is_changelog_filename(file):
+                    mentioned_files.add(os.path.join(entry.folder, file))
+
+        cand = [x[0] for x in self.modified_files
+                if not self.is_changelog_filename(x[0])]
+        changed_files = set(cand)
+        for file in sorted(mentioned_files - changed_files):
+            self.errors.append(Error('file not changed in a patch', file))
+        for file in sorted(changed_files - mentioned_files):
+            if not self.in_ignored_location(file):
+                if file in self.new_files:
+                    changelog_location = self.get_changelog_by_path(file)
+                    # Python2: we cannot use next(filter(...))
+                    entries = filter(lambda x: x.folder == changelog_location,
+                                     self.changelog_entries)
+                    entries = list(entries)
+                    entry = entries[0] if entries else None
+                    if not entry:
+                        prs = self.top_level_prs
+                        if not prs:
+                            # if all ChangeLog entries have identical PRs
+                            # then use them
+                            prs = self.changelog_entries[0].prs
+                            for entry in self.changelog_entries:
+                                if entry.prs != prs:
+                                    prs = []
+                                    break
+                        entry = ChangeLogEntry(changelog_location,
+                                               self.top_level_authors,
+                                               prs)
+                        self.changelog_entries.append(entry)
+                    # strip prefix of the file
+                    assert file.startswith(entry.folder)
+                    file = file[len(entry.folder):].lstrip('/')
+                    entry.lines.append('\t* %s: New file.' % file)
+                else:
+                    msg = 'changed file not mentioned in a ChangeLog'
+                    self.errors.append(Error(msg, file))
+
+    def check_for_correct_changelog(self):
+        for entry in self.changelog_entries:
+            for file in entry.files:
+                full_path = os.path.join(entry.folder, file)
+                changelog_location = self.get_changelog_by_path(full_path)
+                if changelog_location != entry.folder:
+                    msg = 'wrong ChangeLog location "%s", should be "%s"'
+                    err = Error(msg % (entry.folder, changelog_location), file)
+                    self.errors.append(err)
+
+    def to_changelog_entries(self):
+        for entry in self.changelog_entries:
+            output = ''
+            timestamp = entry.datetime
+            if not timestamp:
+                timestamp = self.date.strftime('%Y-%m-%d')
+            authors = entry.authors if entry.authors else [self.author]
+            # add Co-Authored-By authors to all ChangeLog entries
+            for author in self.co_authors:
+                if author not in authors:
+                    authors.append(author)
+
+            for i, author in enumerate(authors):
+                if i == 0:
+                    output += '%s  %s\n' % (timestamp, author)
+                else:
+                    output += '\t    %s\n' % author
+            output += '\n'
+            for pr in entry.prs:
+                output += '\t%s\n' % pr
+            for line in entry.lines:
+                output += line + '\n'
+            yield (entry.folder, output.rstrip())
+
+    def print_output(self):
+        for entry, output in self.to_changelog_entries():
+            print('@@CL %s' % entry)
+            print(output)
+        print('@@CL')
+
+    def print_errors(self):
+        print('Errors:')
+        for error in self.errors:
+            print(error)
diff --git a/contrib/gcc-changelog/git_email.py b/contrib/gcc-changelog/git_email.py
new file mode 100755
index 00000000000..e1d6b70e80c
--- /dev/null
+++ b/contrib/gcc-changelog/git_email.py
@@ -0,0 +1,92 @@
+#!/usr/bin/env python3
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License as published by the Free
+# Software Foundation; either version 3, or (at your option) any later
+# version.
+#
+# GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+# for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.  */
+
+import os
+import sys
+from itertools import takewhile
+
+from dateutil.parser import parse
+
+from git_commit import GitCommit
+
+from unidiff import PatchSet
+
+DATE_PREFIX = 'Date: '
+FROM_PREFIX = 'From: '
+
+
+class GitEmail(GitCommit):
+    def __init__(self, filename, strict=False):
+        self.filename = filename
+        diff = PatchSet.from_filename(filename)
+        date = None
+        author = None
+
+        lines = open(self.filename).read().splitlines()
+        lines = list(takewhile(lambda line: line != '---', lines))
+        for line in lines:
+            if line.startswith(DATE_PREFIX):
+                date = parse(line[len(DATE_PREFIX):])
+            elif line.startswith(FROM_PREFIX):
+                author = GitCommit.format_git_author(line[len(FROM_PREFIX):])
+        header = list(takewhile(lambda line: line != '', lines))
+        body = lines[len(header) + 1:]
+
+        modified_files = []
+        for f in diff:
+            if f.is_added_file:
+                t = 'A'
+            elif f.is_removed_file:
+                t = 'D'
+            else:
+                t = 'M'
+            modified_files.append((f.path, t))
+        super().__init__(None, date, author, body, modified_files,
+                         strict=strict)
+
+
+if __name__ == '__main__':
+    if len(sys.argv) == 1:
+        allfiles = []
+        for root, _dirs, files in os.walk('patches'):
+            for f in files:
+                full = os.path.join(root, f)
+                allfiles.append(full)
+
+        success = 0
+        for full in sorted(allfiles):
+            email = GitEmail(full, False)
+            print(email.filename)
+            if email.success:
+                success += 1
+                print('  OK')
+            else:
+                for error in email.errors:
+                    print('  ERR: %s' % error)
+
+        print()
+        print('Successfully parsed: %d/%d' % (success, len(allfiles)))
+    else:
+        email = GitEmail(sys.argv[1], False)
+        if email.success:
+            print('OK')
+            email.print_output()
+        else:
+            if not email.lines:
+                print('Error: patch contains no parsed lines', file=sys.stderr)
+            email.print_errors()
diff --git a/contrib/gcc-changelog/git_repository.py b/contrib/gcc-changelog/git_repository.py
new file mode 100755
index 00000000000..0473fe73fba
--- /dev/null
+++ b/contrib/gcc-changelog/git_repository.py
@@ -0,0 +1,60 @@
+#!/usr/bin/env python3
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License as published by the Free
+# Software Foundation; either version 3, or (at your option) any later
+# version.
+#
+# GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+# for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.  */
+
+from datetime import datetime
+
+try:
+    from git import Repo
+except ImportError:
+    print('Cannot import GitPython package, please install the package:')
+    print('  Fedora, openSUSE: python3-GitPython')
+    print('  Debian, Ubuntu: python3-git')
+    exit(1)
+
+from git_commit import GitCommit
+
+
+def parse_git_revisions(repo_path, revisions, strict=False):
+    repo = Repo(repo_path)
+
+    parsed_commits = []
+    if '..' in revisions:
+        commits = list(repo.iter_commits(revisions))
+    else:
+        commits = [repo.commit(revisions)]
+
+    for commit in commits:
+        diff = repo.commit(commit.hexsha + '~').diff(commit.hexsha)
+
+        modified_files = []
+        for file in diff:
+            if file.new_file:
+                t = 'A'
+            elif file.deleted_file:
+                t = 'D'
+            else:
+                t = 'M'
+            modified_files.append((file.b_path, t))
+
+        date = datetime.utcfromtimestamp(commit.committed_date)
+        author = '%s  <%s>' % (commit.author.name, commit.author.email)
+        git_commit = GitCommit(commit.hexsha, date, author,
+                               commit.message.split('\n'), modified_files,
+                               strict=strict)
+        parsed_commits.append(git_commit)
+    return parsed_commits
diff --git a/contrib/gcc-changelog/git_update_version.py b/contrib/gcc-changelog/git_update_version.py
new file mode 100755
index 00000000000..c66b4d68e68
--- /dev/null
+++ b/contrib/gcc-changelog/git_update_version.py
@@ -0,0 +1,105 @@
+#!/usr/bin/env python3
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License as published by the Free
+# Software Foundation; either version 3, or (at your option) any later
+# version.
+#
+# GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+# for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.  */
+
+import argparse
+import datetime
+import os
+
+from git import Repo
+
+from git_repository import parse_git_revisions
+
+# TODO: remove sparta suffix
+current_timestamp = datetime.datetime.now().strftime('%Y%m%d sparta\n')
+
+
+def read_timestamp(path):
+    return open(path).read()
+
+
+def prepend_to_changelog_files(repo, folder, git_commit):
+    if not git_commit.success:
+        for error in git_commit.errors:
+            print(error)
+        # TODO: add error message
+        return
+    for entry, output in git_commit.to_changelog_entries():
+        # TODO
+        full_path = os.path.join(folder, entry, 'ChangeLog.test')
+        print('writting to %s' % full_path)
+        if os.path.exists(full_path):
+            content = open(full_path).read()
+        else:
+            content = ''
+        with open(full_path, 'w+') as f:
+            f.write(output)
+            if content:
+                f.write('\n\n')
+                f.write(content)
+        repo.git.add(full_path)
+
+
+active_refs = ['master', 'releases/gcc-8', 'releases/gcc-9', 'releases/gcc-10']
+
+parser = argparse.ArgumentParser(description='Update DATESTAMP and generate '
+                                 'ChangeLog entries')
+parser.add_argument('-g', '--git-path', default='.',
+                    help='Path to git repository')
+args = parser.parse_args()
+
+repo = Repo(args.git_path)
+origin = repo.remotes['origin']
+
+for ref in origin.refs:
+    assert ref.name.startswith('origin/')
+    name = ref.name[len('origin/'):]
+    if name in active_refs:
+        if name in repo.branches:
+            branch = repo.branches[name]
+        else:
+            branch = repo.create_head(name, ref).set_tracking_branch(ref)
+        origin.pull(rebase=True)
+        branch.checkout()
+        print('=== Working on: %s ===' % branch)
+        assert not repo.index.diff(None)
+        commit = branch.commit
+        commit_count = 1
+        while commit:
+            if (commit.author.email == 'gccadmin@gcc.gnu.org'
+                    and commit.message.strip() == 'Daily bump.'):
+                break
+            commit = commit.parents[0]
+            commit_count += 1
+
+        print('%d revisions since last Daily bump' % commit_count)
+        datestamp_path = os.path.join(args.git_path, 'gcc/DATESTAMP')
+        if read_timestamp(datestamp_path) != current_timestamp:
+            print('DATESTAMP will be changed:')
+            # TODO: set strict=True after testing period
+            commits = parse_git_revisions(args.git_path, '%s..HEAD'
+                                          % commit.hexsha, strict=False)
+            for git_commit in commits:
+                prepend_to_changelog_files(repo, args.git_path, git_commit)
+            # update timestamp
+            with open(datestamp_path, 'w+') as f:
+                f.write(current_timestamp)
+            repo.git.add(datestamp_path)
+            repo.index.commit('Daily bump.')
+            # TODO: push the repository
+        else:
+            print('DATESTAMP unchanged')
diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index a932bf8c06a..77c7d0ff4fb 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -25,6 +25,8 @@ git config alias.svn-rev '!f() { rev=$1; shift; git log --all --grep="^From-SVN:
 git config alias.gcc-descr \!"f() { if test \${1:-no} = --full; then c=\${2:-master}; r=\$(git describe --all --abbrev=40 --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-,r,p'); expr match \${r:-no} '^r[0-9]\\+\$' >/dev/null && r=\${r}-0-g\$(git rev-parse \${2:-master}); else c=\${1:-master}; r=\$(git describe --all --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)-\\([0-9]\\+\\)-g[0-9a-f]*\$,r\\2-\\3,p;s,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)\$,r\\2-0,p'); fi; if test -n \$r; then o=\$(git config --get gcc-config.upstream); rr=\$(echo \$r | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\\(-g[0-9a-f]\\+\\)\\?\$,\\1,p'); if git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$rr >/dev/null; then m=releases/gcc-\$rr; else m=master; fi; git merge-base --is-ancestor \$c \${o:-origin}/\$m && \echo \${r}; fi; }; f"
 git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream); r=\$(echo \$1 | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\$,\\1,p'); n=\$(echo \$1 | sed -n 's,^r[0-9]\\+-\\([0-9]\\+\\)\$,\\1,p'); test -z \$r && echo Invalid id \$1 && exit 1; h=\$(git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$r); test -z \$h && h=\$(git rev-parse --verify --quiet \${o:-origin}/master); p=\$(git describe --all --match 'basepoints/gcc-'\$r \$h | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+-\\([0-9]\\+\\)-g[0-9a-f]*\$,\\2,p;s,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+\$,0,p'); git rev-parse --verify \$h~\$(expr \$p - \$n); }; f"
 
+git config alias.gcc-verify '!f() { ./contrib/gcc-changelog/git_check_commit.py $@; } ; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
-- 
2.26.2


[-- Attachment #3: 0001-Add-gcc-changelog-hook-for-commit-message-format.patch --]
[-- Type: text/x-patch, Size: 22209 bytes --]

From 6367837722c62930c9e00e9aa82073d24813a4c1 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Mon, 11 May 2020 15:24:16 +0200
Subject: [PATCH] Add gcc-changelog hook for commit message format.

---
 hooks/git_commit.py        | 535 +++++++++++++++++++++++++++++++++++++
 hooks/pre_commit_checks.py |  29 +-
 hooks/updates/__init__.py  |   2 +-
 3 files changed, 563 insertions(+), 3 deletions(-)
 create mode 100755 hooks/git_commit.py

diff --git a/hooks/git_commit.py b/hooks/git_commit.py
new file mode 100755
index 0000000..5214cc3
--- /dev/null
+++ b/hooks/git_commit.py
@@ -0,0 +1,535 @@
+#!/usr/bin/env python3
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License as published by the Free
+# Software Foundation; either version 3, or (at your option) any later
+# version.
+#
+# GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+# for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.  */
+
+import os
+import re
+
+changelog_locations = set([
+    'config',
+    'contrib',
+    'contrib/header-tools',
+    'contrib/reghunt',
+    'contrib/regression',
+    'fixincludes',
+    'gcc/ada',
+    'gcc/analyzer',
+    'gcc/brig',
+    'gcc/c',
+    'gcc/c-family',
+    'gcc',
+    'gcc/cp',
+    'gcc/d',
+    'gcc/fortran',
+    'gcc/go',
+    'gcc/jit',
+    'gcc/lto',
+    'gcc/objc',
+    'gcc/objcp',
+    'gcc/po',
+    'gcc/testsuite',
+    'gnattools',
+    'gotools',
+    'include',
+    'intl',
+    'libada',
+    'libatomic',
+    'libbacktrace',
+    'libcc1',
+    'libcpp',
+    'libcpp/po',
+    'libdecnumber',
+    'libffi',
+    'libgcc',
+    'libgcc/config/avr/libf7',
+    'libgcc/config/libbid',
+    'libgfortran',
+    'libgomp',
+    'libhsail-rt',
+    'libiberty',
+    'libitm',
+    'libobjc',
+    'liboffloadmic',
+    'libphobos',
+    'libquadmath',
+    'libsanitizer',
+    'libssp',
+    'libstdc++-v3',
+    'libvtv',
+    'lto-plugin',
+    'maintainer-scripts',
+    'zlib'])
+
+bug_components = set([
+    'ada',
+    'analyzer',
+    'boehm-gc',
+    'bootstrap',
+    'c',
+    'c++',
+    'd',
+    'debug',
+    'demangler',
+    'driver',
+    'fastjar',
+    'fortran',
+    'gcov-profile',
+    'go',
+    'hsa',
+    'inline-asm',
+    'ipa',
+    'java',
+    'jit',
+    'libbacktrace',
+    'libf2c',
+    'libffi',
+    'libfortran',
+    'libgcc',
+    'libgcj',
+    'libgomp',
+    'libitm',
+    'libobjc',
+    'libquadmath',
+    'libstdc++',
+    'lto',
+    'middle-end',
+    'modula2',
+    'objc',
+    'objc++',
+    'other',
+    'pch',
+    'pending',
+    'plugins',
+    'preprocessor',
+    'regression',
+    'rtl-optimization',
+    'sanitizer',
+    'spam',
+    'target',
+    'testsuite',
+    'translation',
+    'tree-optimization',
+    'web'])
+
+ignored_prefixes = [
+    'gcc/d/dmd/',
+    'gcc/go/frontend/',
+    'libgo/',
+    'libphobos/libdruntime',
+    'libphobos/src/',
+    'libsanitizer/',
+    ]
+
+misc_files = [
+    'gcc/DATESTAMP',
+    'gcc/BASE-VER',
+    'gcc/DEV-PHASE'
+    ]
+
+author_line_regex = \
+        re.compile(r'^(?P<datetime>\d{4}-\d{2}-\d{2})\ {2}(?P<name>.*  <.*>)')
+additional_author_regex = re.compile(r'^\t(?P<spaces>\ *)?(?P<name>.*  <.*>)')
+changelog_regex = re.compile(r'^([a-z0-9+-/]*)/ChangeLog:?')
+pr_regex = re.compile(r'\tPR (?P<component>[a-z+-]+\/)?([0-9]+)$')
+star_prefix_regex = re.compile(r'\t\*(?P<spaces>\ *)(?P<content>.*)')
+
+LINE_LIMIT = 100
+TAB_WIDTH = 8
+CO_AUTHORED_BY_PREFIX = 'co-authored-by: '
+
+
+class Error:
+    def __init__(self, message, line=None):
+        self.message = message
+        self.line = line
+
+    def __repr__(self):
+        s = self.message
+        if self.line:
+            s += ':"%s"' % self.line
+        return s
+
+
+class ChangeLogEntry:
+    def __init__(self, folder, authors, prs):
+        self.folder = folder
+        # Python2 has not 'copy' function
+        self.author_lines = list(authors)
+        self.initial_prs = list(prs)
+        self.prs = list(prs)
+        self.lines = []
+
+    @property
+    def files(self):
+        files = []
+        for line in self.lines:
+            m = star_prefix_regex.match(line)
+            if m:
+                line = m.group('content')
+                if '(' in line:
+                    line = line[:line.index('(')]
+                if ':' in line:
+                    line = line[:line.index(':')]
+                for file in line.split(','):
+                    file = file.strip()
+                    if file:
+                        files.append(file)
+        return files
+
+    @property
+    def datetime(self):
+        for author in self.author_lines:
+            if author[1]:
+                return author[1]
+        return None
+
+    @property
+    def authors(self):
+        return [author_line[0] for author_line in self.author_lines]
+
+    @property
+    def is_empty(self):
+        return not self.lines and self.prs == self.initial_prs
+
+
+class GitCommit:
+    def __init__(self, hexsha, date, author, body, modified_files,
+                 strict=True):
+        self.hexsha = hexsha
+        self.lines = body
+        self.modified_files = modified_files
+        self.message = None
+        self.changes = None
+        self.changelog_entries = []
+        self.errors = []
+        self.date = date
+        self.author = author
+        self.top_level_authors = []
+        self.co_authors = []
+        self.top_level_prs = []
+
+        project_files = [f for f in self.modified_files
+                         if self.is_changelog_filename(f[0])
+                         or f[0] in misc_files]
+        if len(project_files) == len(self.modified_files):
+            # All modified files are only MISC files
+            return
+        elif project_files and strict:
+            self.errors.append(Error('ChangeLog, DATESTAMP, BASE-VER and '
+                                     'DEV-PHASE updates should be done '
+                                     'separately from normal commits'))
+            return
+
+        self.parse_lines()
+        if self.changes:
+            self.parse_changelog()
+            self.deduce_changelog_locations()
+            if not self.errors:
+                self.check_mentioned_files()
+                self.check_for_correct_changelog()
+
+    @property
+    def success(self):
+        return not self.errors
+
+    @property
+    def new_files(self):
+        return [x[0] for x in self.modified_files if x[1] == 'A']
+
+    @classmethod
+    def is_changelog_filename(cls, path):
+        return path.endswith('/ChangeLog') or path == 'ChangeLog'
+
+    @classmethod
+    def find_changelog_location(cls, name):
+        if name.startswith('\t'):
+            name = name[1:]
+        if name.endswith(':'):
+            name = name[:-1]
+        if name.endswith('/'):
+            name = name[:-1]
+        return name if name in changelog_locations else None
+
+    @classmethod
+    def format_git_author(cls, author):
+        assert '<' in author
+        return author.replace('<', ' <')
+
+    @classmethod
+    def parse_git_name_status(cls, string):
+        modified_files = []
+        for entry in string.split('\n'):
+            parts = entry.split('\t')
+            t = parts[0]
+            if t == 'A' or t == 'D' or t == 'M':
+                modified_files.append((parts[1], t))
+            elif t == 'R':
+                modified_files.append((parts[1], 'D'))
+                modified_files.append((parts[2], 'A'))
+        return modified_files
+
+    def parse_lines(self):
+        body = self.lines
+
+        for i, b in enumerate(body):
+            if not b:
+                continue
+            if (changelog_regex.match(b) or self.find_changelog_location(b)
+                    or star_prefix_regex.match(b) or pr_regex.match(b)
+                    or author_line_regex.match(b)):
+                self.changes = body[i:]
+                return
+        self.errors.append(Error('cannot find a ChangeLog location in '
+                                 'message'))
+
+    def parse_changelog(self):
+        last_entry = None
+        will_deduce = False
+        for line in self.changes:
+            if not line:
+                if last_entry and will_deduce:
+                    last_entry = None
+                continue
+            if line != line.rstrip():
+                self.errors.append(Error('trailing whitespace', line))
+            if len(line.replace('\t', ' ' * TAB_WIDTH)) > LINE_LIMIT:
+                self.errors.append(Error('line limit exceeds %d characters'
+                                         % LINE_LIMIT, line))
+            m = changelog_regex.match(line)
+            if m:
+                last_entry = ChangeLogEntry(m.group(1), self.top_level_authors,
+                                            self.top_level_prs)
+                self.changelog_entries.append(last_entry)
+            elif self.find_changelog_location(line):
+                last_entry = ChangeLogEntry(self.find_changelog_location(line),
+                                            self.top_level_authors,
+                                            self.top_level_prs)
+                self.changelog_entries.append(last_entry)
+            else:
+                author_tuple = None
+                pr_line = None
+                if author_line_regex.match(line):
+                    m = author_line_regex.match(line)
+                    author_tuple = (m.group('name'), m.group('datetime'))
+                elif additional_author_regex.match(line):
+                    m = additional_author_regex.match(line)
+                    if len(m.group('spaces')) != 4:
+                        msg = 'additional author must prepend with tab ' \
+                              'and 4 spaces'
+                        self.errors.append(Error(msg, line))
+                    else:
+                        author_tuple = (m.group('name'), None)
+                elif pr_regex.match(line):
+                    component = pr_regex.match(line).group('component')
+                    if not component:
+                        self.errors.append(Error('missing PR component', line))
+                        continue
+                    elif not component[:-1] in bug_components:
+                        self.errors.append(Error('invalid PR component', line))
+                        continue
+                    else:
+                        pr_line = line.lstrip()
+
+                if line.lower().startswith(CO_AUTHORED_BY_PREFIX):
+                    name = line[len(CO_AUTHORED_BY_PREFIX):]
+                    author = self.format_git_author(name)
+                    self.co_authors.append(author)
+                    continue
+
+                # ChangeLog name will be deduced later
+                if not last_entry:
+                    if author_tuple:
+                        self.top_level_authors.append(author_tuple)
+                        continue
+                    elif pr_line:
+                        # append to top_level_prs only when we haven't met
+                        # a ChangeLog entry
+                        if (pr_line not in self.top_level_prs
+                                and not self.changelog_entries):
+                            self.top_level_prs.append(pr_line)
+                        continue
+                    else:
+                        last_entry = ChangeLogEntry(None,
+                                                    self.top_level_authors,
+                                                    self.top_level_prs)
+                        self.changelog_entries.append(last_entry)
+                        will_deduce = True
+                elif author_tuple:
+                    last_entry.author_lines.append(author_tuple)
+                    continue
+
+                if not line.startswith('\t'):
+                    err = Error('line should start with a tab', line)
+                    self.errors.append(err)
+                elif pr_line:
+                    last_entry.prs.append(pr_line)
+                else:
+                    m = star_prefix_regex.match(line)
+                    if m:
+                        if len(m.group('spaces')) != 1:
+                            err = Error('one space should follow asterisk',
+                                        line)
+                            self.errors.append(err)
+                        else:
+                            last_entry.lines.append(line)
+                    else:
+                        if last_entry.is_empty:
+                            msg = 'first line should start with a tab, ' \
+                                  'asterisk and space'
+                            self.errors.append(Error(msg, line))
+                        else:
+                            last_entry.lines.append(line)
+
+    def get_file_changelog_location(self, changelog_file):
+        for file in self.modified_files:
+            if file[0] == changelog_file:
+                # root ChangeLog file
+                return ''
+            index = file[0].find('/' + changelog_file)
+            if index != -1:
+                return file[0][:index]
+        return None
+
+    def deduce_changelog_locations(self):
+        for entry in self.changelog_entries:
+            if not entry.folder:
+                changelog = None
+                for file in entry.files:
+                    location = self.get_file_changelog_location(file)
+                    if (location == ''
+                       or (location and location in changelog_locations)):
+                        if changelog and changelog != location:
+                            msg = 'could not deduce ChangeLog file, ' \
+                                  'not unique location'
+                            self.errors.append(Error(msg))
+                            return
+                        changelog = location
+                if changelog is not None:
+                    entry.folder = changelog
+                else:
+                    msg = 'could not deduce ChangeLog file'
+                    self.errors.append(Error(msg))
+
+    @classmethod
+    def in_ignored_location(cls, path):
+        for ignored in ignored_prefixes:
+            if path.startswith(ignored):
+                return True
+        return False
+
+    @classmethod
+    def get_changelog_by_path(cls, path):
+        components = path.split('/')
+        while components:
+            if '/'.join(components) in changelog_locations:
+                break
+            components = components[:-1]
+        return '/'.join(components)
+
+    def check_mentioned_files(self):
+        folder_count = len([x.folder for x in self.changelog_entries])
+        assert folder_count == len(self.changelog_entries)
+
+        mentioned_files = set()
+        for entry in self.changelog_entries:
+            if not entry.files:
+                msg = 'ChangeLog must contain a file entry'
+                self.errors.append(Error(msg, entry.folder))
+            assert not entry.folder.endswith('/')
+            for file in entry.files:
+                if not self.is_changelog_filename(file):
+                    mentioned_files.add(os.path.join(entry.folder, file))
+
+        cand = [x[0] for x in self.modified_files
+                if not self.is_changelog_filename(x[0])]
+        changed_files = set(cand)
+        for file in sorted(mentioned_files - changed_files):
+            self.errors.append(Error('file not changed in a patch', file))
+        for file in sorted(changed_files - mentioned_files):
+            if not self.in_ignored_location(file):
+                if file in self.new_files:
+                    changelog_location = self.get_changelog_by_path(file)
+                    # Python2: we cannot use next(filter(...))
+                    entries = filter(lambda x: x.folder == changelog_location,
+                                     self.changelog_entries)
+                    entries = list(entries)
+                    entry = entries[0] if entries else None
+                    if not entry:
+                        prs = self.top_level_prs
+                        if not prs:
+                            # if all ChangeLog entries have identical PRs
+                            # then use them
+                            prs = self.changelog_entries[0].prs
+                            for entry in self.changelog_entries:
+                                if entry.prs != prs:
+                                    prs = []
+                                    break
+                        entry = ChangeLogEntry(changelog_location,
+                                               self.top_level_authors,
+                                               prs)
+                        self.changelog_entries.append(entry)
+                    # strip prefix of the file
+                    assert file.startswith(entry.folder)
+                    file = file[len(entry.folder):].lstrip('/')
+                    entry.lines.append('\t* %s: New file.' % file)
+                else:
+                    msg = 'changed file not mentioned in a ChangeLog'
+                    self.errors.append(Error(msg, file))
+
+    def check_for_correct_changelog(self):
+        for entry in self.changelog_entries:
+            for file in entry.files:
+                full_path = os.path.join(entry.folder, file)
+                changelog_location = self.get_changelog_by_path(full_path)
+                if changelog_location != entry.folder:
+                    msg = 'wrong ChangeLog location "%s", should be "%s"'
+                    err = Error(msg % (entry.folder, changelog_location), file)
+                    self.errors.append(err)
+
+    def to_changelog_entries(self):
+        for entry in self.changelog_entries:
+            output = ''
+            timestamp = entry.datetime
+            if not timestamp:
+                timestamp = self.date.strftime('%Y-%m-%d')
+            authors = entry.authors if entry.authors else [self.author]
+            # add Co-Authored-By authors to all ChangeLog entries
+            for author in self.co_authors:
+                if author not in authors:
+                    authors.append(author)
+
+            for i, author in enumerate(authors):
+                if i == 0:
+                    output += '%s  %s\n' % (timestamp, author)
+                else:
+                    output += '\t    %s\n' % author
+            output += '\n'
+            for pr in entry.prs:
+                output += '\t%s\n' % pr
+            for line in entry.lines:
+                output += line + '\n'
+            yield (entry.folder, output.rstrip())
+
+    def print_output(self):
+        for entry, output in self.to_changelog_entries():
+            print('------ %s/ChangeLog ------ ' % entry)
+            print(output)
+
+    def print_errors(self):
+        print('Errors:')
+        for error in self.errors:
+            print(error)
diff --git a/hooks/pre_commit_checks.py b/hooks/pre_commit_checks.py
index b3546ba..dbdc269 100644
--- a/hooks/pre_commit_checks.py
+++ b/hooks/pre_commit_checks.py
@@ -3,6 +3,9 @@ from pipes import quote
 import re
 from subprocess import Popen, PIPE, STDOUT
 
+from git_commit import GitCommit
+from datetime import datetime
+
 from config import git_config
 from errors import InvalidUpdate
 from git import git, diff_tree, file_exists
@@ -326,8 +329,27 @@ def check_missing_ticket_number(rev, raw_rh):
         'Subject: %s' % raw_rh[0],
         ])
 
+def verify_changelog_format(rev, raw_body):
+    author = git.show(rev, format='%ae  <%aN>', no_patch=True)
+    committed_date = git.show(rev, format='%ct', no_patch=True)
+    changed_files = git.diff('%s~..%s' % (rev, rev), name_status=True)
+
+    date = datetime.utcfromtimestamp(int(committed_date))
+    # TODO: enable strict mode
+    git_commit = GitCommit(date, author, raw_body,
+                           GitCommit.parse_git_name_status(changed_files),
+                           strict=False)
 
-def check_revision_history(rev):
+    if git_commit.success:
+        # OK
+        pass
+    else:
+        message = 'ChangeLog format failed:\n'
+        for error in git_commit.errors:
+            message += 'ERR: %s\n' % error
+        raise InvalidUpdate(message)
+
+def check_revision_history(rev, project_name, short_ref_name):
     """Apply pre-commit checks to the commit's revision history.
 
     Raise InvalidUpdate if one or more style violation are detected.
@@ -370,7 +392,10 @@ def check_revision_history(rev):
                 'from SVN, not in new git commits.  If this is a cherry-pick '
                 'of a commit done in SVN, remove the From-SVN: line from '
                 'the commit message before pushing.')
-
+    if (len(raw_body) >= 1
+        and (short_ref_name == 'master'
+             or short_ref_name.startswith('releases/gcc-'))):
+        verify_changelog_format(rev, raw_body)
 
 def check_filename_collisions(rev):
     """raise InvalidUpdate if the name of two files only differ in casing.
diff --git a/hooks/updates/__init__.py b/hooks/updates/__init__.py
index 25bfb18..e47145f 100644
--- a/hooks/updates/__init__.py
+++ b/hooks/updates/__init__.py
@@ -263,7 +263,7 @@ class AbstractUpdate(object):
         if do_rh_style_checks:
             for commit in added:
                 if not commit.pre_existing_p:
-                    check_revision_history(commit.rev)
+                    check_revision_history(commit.rev, self.email_info.project_name, self.short_ref_name)
 
         reject_merge_commits = (
             self.search_config_option_list('hooks.reject-merge-commits')
-- 
2.26.2

ChangeLog files - server and client scripts (git cherry-pick)

[Martin Liška]

On 5/13/20 1:05 PM, Martin Liška wrote:
> I suggest to use native 'git revert XYZ' and 'git cherry-pick -x XYZ'.

I've prepared a working version of Revert format:
https://github.com/marxin/gcc-changelog/tree/cherry-pick

So using git cherry-pick -x HASH one gets something like:

$ cat patches-artificial/0001-Test-tree.h.patch
 From a71eeba28ffa2427d24d5b2654e93b261980b9e3 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Wed, 13 May 2020 13:19:22 +0200
Subject: [PATCH] Test tree.h.

gcc/ChangeLog:

2020-01-03  Martin Liska  <mliska@suse.cz>

	PR ipa/12345
	* tree.h: Just test it.

(cherry picked from commit a2bdf56b15b51c3a7bd988943bdbc42aa156f133)
---
  gcc/tree.h | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/gcc/tree.h b/gcc/tree.h
index 9ca9ab58ec0..99a9e1a73d9 100644
--- a/gcc/tree.h
+++ b/gcc/tree.h
@@ -1,6 +1,8 @@
  /* Definitions for the ubiquitous 'tree' type for GNU compilers.
     Copyright (C) 1989-2020 Free Software Foundation, Inc.
  
+
+
  This file is part of GCC.
  
  GCC is free software; you can redistribute it and/or modify it under
-- 
2.26.2

and the script generates:

$ ./git_email.py patches-artificial/0001-Test-tree.h.patch
OK
@@CL gcc
2020-05-13  Martin Liska  <mliska@suse.cz>

	Backport from master:
	2020-01-03  Martin Liska  <mliska@suse.cz>

	PR ipa/12345
	* tree.h: Just test it.
@@CL

So the datestamp and the author is taken from commit and original authors
are added after 'Backport from master' line. The script scans for the
'(cherry picked from commit' line in the message.

Benefit of the approach is that one can adjust the commit message (which influences
ChangeLog output).

Martin

Re: ChangeLog files - server and client scripts (git cherry-pick)

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 478 bytes --]

Hello.

I'm sending patch candidate that adds 2 new git aliases:
- gcc-backport - simple alias to 'git cherry-pick -x'
- gcc-revert - it similarly appends '(this reverts commit 365e3cde4978c6a7dbfa50865720226254c016be)'
to a reverted commit message

The script normally parses content of a git message and adds corresponding 'Revert:' or
'Backport from master:' lines. Right now, there's missing date of the original commit and
author. I hope it's acceptable.

Thoughts?
Martin

[-- Attachment #2: 0001-gcc-changelog-introduce-gcc-revert-and-gcc-backport.patch --]
[-- Type: text/x-patch, Size: 10606 bytes --]

From fd916394f66831ebe8f5cadb455d559aa3917fc3 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Thu, 14 May 2020 14:34:18 +0200
Subject: [PATCH] gcc-changelog: introduce gcc-revert and gcc-backport.

contrib/ChangeLog:

2020-05-14  Martin Liska  <mliska@suse.cz>

	* gcc-changelog/git_commit.py: Add support
	for CHERRY_PICK_PREFIX and REVERT_PREFIX.
	* gcc-changelog/test_email.py: Add 2 new tests.
	* gcc-changelog/test_patches.txt: Add 2 patches.
	* gcc-git-customization.sh: Add gcc-backport and
	gcc-revert aliases.
---
 contrib/gcc-changelog/git_commit.py    | 46 ++++++++++----
 contrib/gcc-changelog/test_email.py    | 14 ++++
 contrib/gcc-changelog/test_patches.txt | 88 ++++++++++++++++++++++++++
 contrib/gcc-git-customization.sh       |  3 +
 4 files changed, 140 insertions(+), 11 deletions(-)

diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
index 5214cc36538..bf82f6206b6 100755
--- a/contrib/gcc-changelog/git_commit.py
+++ b/contrib/gcc-changelog/git_commit.py
@@ -150,6 +150,8 @@ star_prefix_regex = re.compile(r'\t\*(?P<spaces>\ *)(?P<content>.*)')
 LINE_LIMIT = 100
 TAB_WIDTH = 8
 CO_AUTHORED_BY_PREFIX = 'co-authored-by: '
+CHERRY_PICK_PREFIX = '(cherry picked from commit '
+REVERT_PREFIX = '(this reverts commit '
 
 
 class Error:
@@ -221,6 +223,8 @@ class GitCommit:
         self.top_level_authors = []
         self.co_authors = []
         self.top_level_prs = []
+        self.cherry_pick = False
+        self.revert = False
 
         project_files = [f for f in self.modified_files
                          if self.is_changelog_filename(f[0])
@@ -372,7 +376,11 @@ class GitCommit:
                     last_entry.author_lines.append(author_tuple)
                     continue
 
-                if not line.startswith('\t'):
+                if line.startswith(CHERRY_PICK_PREFIX):
+                    self.cherry_pick = True
+                elif line.startswith(REVERT_PREFIX):
+                    self.revert = True
+                elif not line.startswith('\t'):
                     err = Error('line should start with a tab', line)
                     self.errors.append(err)
                 elif pr_line:
@@ -500,24 +508,40 @@ class GitCommit:
                     err = Error(msg % (entry.folder, changelog_location), file)
                     self.errors.append(err)
 
+    @classmethod
+    def format_authors_in_changelog(cls, authors, timestamp, prefix=''):
+        output = ''
+        for i, author in enumerate(authors):
+            if i == 0:
+                output += '%s%s  %s\n' % (prefix, timestamp, author)
+            else:
+                output += '%s\t    %s\n' % (prefix, author)
+        output += '\n'
+        return output
+
     def to_changelog_entries(self):
+        current_timestamp = self.date.strftime('%Y-%m-%d')
         for entry in self.changelog_entries:
             output = ''
             timestamp = entry.datetime
             if not timestamp:
                 timestamp = self.date.strftime('%Y-%m-%d')
             authors = entry.authors if entry.authors else [self.author]
-            # add Co-Authored-By authors to all ChangeLog entries
-            for author in self.co_authors:
-                if author not in authors:
-                    authors.append(author)
-
-            for i, author in enumerate(authors):
-                if i == 0:
-                    output += '%s  %s\n' % (timestamp, author)
+            if self.cherry_pick or self.revert:
+                output += self.format_authors_in_changelog(authors,
+                                                           current_timestamp)
+                if self.cherry_pick:
+                    header = 'Backport from master'
                 else:
-                    output += '\t    %s\n' % author
-            output += '\n'
+                    header = 'Revert'
+                output += '\t%s:\n' % header
+            else:
+                # add Co-Authored-By authors to all ChangeLog entries
+                for author in self.co_authors:
+                    if author not in authors:
+                        authors.append(author)
+
+                output += self.format_authors_in_changelog(authors, timestamp)
             for pr in entry.prs:
                 output += '\t%s\n' % pr
             for line in entry.lines:
diff --git a/contrib/gcc-changelog/test_email.py b/contrib/gcc-changelog/test_email.py
index 03abc763212..e1d955a9c21 100755
--- a/contrib/gcc-changelog/test_email.py
+++ b/contrib/gcc-changelog/test_email.py
@@ -258,3 +258,17 @@ class TestGccChangelog(unittest.TestCase):
         email = self.from_patch_glob('0020-IPA-Avoid')
         assert (email.errors[0].message
                 == 'first line should start with a tab, asterisk and space')
+
+    def test_backport(self):
+        email = self.get_git_email('0001-Test-tree.h.patch')
+        assert len(email.changelog_entries) == 1
+        entry, output = list(email.to_changelog_entries())[0]
+        assert entry == 'gcc'
+        assert '2020-05-13  Martin Liska  <mliska@suse.cz>' in output
+        assert '\tBackport from master:' in output
+
+    def test_revert(self):
+        email = self.get_git_email('0001-Revert-i386-Add-V2DFmode.patch')
+        assert len(email.changelog_entries) == 2
+        entry, output = list(email.to_changelog_entries())[0]
+        assert 'Revert:' in output
diff --git a/contrib/gcc-changelog/test_patches.txt b/contrib/gcc-changelog/test_patches.txt
index 39e4753c0ab..991dd5440a4 100644
--- a/contrib/gcc-changelog/test_patches.txt
+++ b/contrib/gcc-changelog/test_patches.txt
@@ -2381,4 +2381,92 @@ index 00000000000..66c87d48694
 +
 -- 
 2.26.1
+=== 0001-Test-tree.h.patch ===
+From a71eeba28ffa2427d24d5b2654e93b261980b9e3 Mon Sep 17 00:00:00 2001
+From: Martin Liska <mliska@suse.cz>
+Date: Wed, 13 May 2020 13:19:22 +0200
+Subject: [PATCH] Test tree.h.
+
+gcc/ChangeLog:
+
+2020-01-03  Martin Liska  <mliska@suse.cz>
+
+	PR ipa/12345
+	* tree.h: Just test it.
+
+(cherry picked from commit a2bdf56b15b51c3a7bd988943bdbc42aa156f133)
+---
+ gcc/tree.h | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/gcc/tree.h b/gcc/tree.h
+index 9ca9ab58ec0..99a9e1a73d9 100644
+--- a/gcc/tree.h
++++ b/gcc/tree.h
+@@ -1,6 +1,8 @@
+ /* Definitions for the ubiquitous 'tree' type for GNU compilers.
+    Copyright (C) 1989-2020 Free Software Foundation, Inc.
+ 
++
++
+ This file is part of GCC.
+ 
+ GCC is free software; you can redistribute it and/or modify it under
+-- 
+2.26.2
+
+=== 0001-Revert-i386-Add-V2DFmode.patch ===
+
+From 3bb5665870ddba0a4c6aaf264d4efb44dc6438fe Mon Sep 17 00:00:00 2001
+From: Martin Liska <mliska@suse.cz>
+Date: Thu, 14 May 2020 14:16:12 +0200
+Subject: [PATCH] Revert "i386: Add V2DFmode conversion functions [PR95046]"
+
+gcc/ChangeLog:
+
+	PR target/95046
+	* config/i386/sse.md (sse2_cvtpi2pd): Add memory to alternative 1.
+
+	(floatv2siv2df2): New expander.
+	(floatunsv2siv2df2): New insn pattern.
 
+	(fix_truncv2dfv2si2): New expander.
+	(fixuns_truncv2dfv2si2): New insn pattern.
+
+gcc/testsuite/ChangeLog:
+
+	PR target/95046
+	* gcc.target/i386/pr95046-6.c: New test.
+
+(this reverts commit 365e3cde4978c6a7dbfa50865720226254c016be)
+---
+ gcc/ChangeLog                             | 19 +++-------
+ gcc/config/i386/sse.md                    | 34 ++----------------
+ gcc/testsuite/ChangeLog                   |  5 ---
+ gcc/testsuite/gcc.target/i386/pr95046-6.c | 44 -----------------------
+ 4 files changed, 6 insertions(+), 96 deletions(-)
+ delete mode 100644 gcc/testsuite/gcc.target/i386/pr95046-6.c
+
+diff --git a/gcc/ChangeLog b/gcc/ChangeLog
+index 072ad082852..125f7c7c389 100644
+--- a/gcc/ChangeLog
++++ b/gcc/ChangeLog
+@@ -1 +1,2 @@
+
++
+diff --git a/gcc/config/i386/sse.md b/gcc/config/i386/sse.md
+index dc0ecbc182e..7a7ecd4be87 100644
+--- a/gcc/config/i386/sse.md
++++ b/gcc/config/i386/sse.md
+@@ -1 +1,2 @@
+
++
+diff --git a/gcc/testsuite/gcc.target/i386/pr95046-6.c b/gcc/testsuite/gcc.target/i386/pr95046-6.c
+deleted file mode 100644
+index dcc8999c446..00000000000
+--- a/gcc/testsuite/gcc.target/i386/pr95046-6.c
++++ /dev/null
+@@ -1 +0,0 @@
+-
+-- 
+2.26.2
diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index a932bf8c06a..7d5451a29df 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -25,6 +25,9 @@ git config alias.svn-rev '!f() { rev=$1; shift; git log --all --grep="^From-SVN:
 git config alias.gcc-descr \!"f() { if test \${1:-no} = --full; then c=\${2:-master}; r=\$(git describe --all --abbrev=40 --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-,r,p'); expr match \${r:-no} '^r[0-9]\\+\$' >/dev/null && r=\${r}-0-g\$(git rev-parse \${2:-master}); else c=\${1:-master}; r=\$(git describe --all --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)-\\([0-9]\\+\\)-g[0-9a-f]*\$,r\\2-\\3,p;s,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)\$,r\\2-0,p'); fi; if test -n \$r; then o=\$(git config --get gcc-config.upstream); rr=\$(echo \$r | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\\(-g[0-9a-f]\\+\\)\\?\$,\\1,p'); if git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$rr >/dev/null; then m=releases/gcc-\$rr; else m=master; fi; git merge-base --is-ancestor \$c \${o:-origin}/\$m && \echo \${r}; fi; }; f"
 git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream); r=\$(echo \$1 | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\$,\\1,p'); n=\$(echo \$1 | sed -n 's,^r[0-9]\\+-\\([0-9]\\+\\)\$,\\1,p'); test -z \$r && echo Invalid id \$1 && exit 1; h=\$(git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$r); test -z \$h && h=\$(git rev-parse --verify --quiet \${o:-origin}/master); p=\$(git describe --all --match 'basepoints/gcc-'\$r \$h | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+-\\([0-9]\\+\\)-g[0-9a-f]*\$,\\2,p;s,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+\$,0,p'); git rev-parse --verify \$h~\$(expr \$p - \$n); }; f"
 
+git config alias.gcc-backport '!f() { rev=$1; git cherry-pick -x $@; } ; f'
+git config alias.gcc-revert '!f() { rev=$1; git show $rev --format="Revert \"%s\"%n%n" --no-patch > commit.msg && git show $rev --format=%B --no-patch | tail -n +3 >> commit.msg && git show $rev --format="(this reverts commit %H)" --no-patch >> commit.msg && git revert -n $rev && git commit -F commit.msg; } ; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
-- 
2.26.2

Re: ChangeLog files - server and client scripts (git cherry-pick)

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 853 bytes --]

On 5/14/20 2:42 PM, Martin Liška wrote:
> Hello.
> 
> I'm sending patch candidate that adds 2 new git aliases:
> - gcc-backport - simple alias to 'git cherry-pick -x'
> - gcc-revert - it similarly appends '(this reverts commit 365e3cde4978c6a7dbfa50865720226254c016be)'
> to a reverted commit message
> 
> The script normally parses content of a git message and adds corresponding 'Revert:' or
> 'Backport from master:' lines. Right now, there's missing date of the original commit and
> author. I hope it's acceptable.
> 
> Thoughts?
> Martin

Hello.

I'm going to install the following patch that will allow '(cherry picked from commit hash)' line.
Generated ChangeLog entry will look the same as the original one (No Backported from leading lines).
Apart from that, I'm adding 'git gcc-backport' which is simple alias for 'cherry-pick -x'.

Martin


[-- Attachment #2: 0001-Add-gcc-backport-and-support-git-cherry-pick.patch --]
[-- Type: text/x-patch, Size: 4599 bytes --]

From 5394cd8d0ec4aa774228bff1687cdace5cdc7552 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Wed, 20 May 2020 09:49:48 +0200
Subject: [PATCH] Add gcc-backport and support git cherry pick.

Unknown ChangeLog:

2020-05-20  Martin Liska  <mliska@suse.cz>

	* contrib/gcc-changelog/git_commit.py: Support cherry pick
	prefix.
	* contrib/gcc-changelog/test_email.py: Test it.
	* contrib/gcc-changelog/test_patches.txt: Add new patch.
	* contrib/gcc-git-customization.sh: Add gcc-backport.
---
 contrib/gcc-changelog/git_commit.py    |  3 +++
 contrib/gcc-changelog/test_email.py    |  4 ++++
 contrib/gcc-changelog/test_patches.txt | 29 ++++++++++++++++++++++++++
 contrib/gcc-git-customization.sh       |  1 +
 4 files changed, 37 insertions(+)

diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
index f6b9c5b1586..5cc8c4f5935 100755
--- a/contrib/gcc-changelog/git_commit.py
+++ b/contrib/gcc-changelog/git_commit.py
@@ -150,6 +150,7 @@ star_prefix_regex = re.compile(r'\t\*(?P<spaces>\ *)(?P<content>.*)')
 LINE_LIMIT = 100
 TAB_WIDTH = 8
 CO_AUTHORED_BY_PREFIX = 'co-authored-by: '
+CHERRY_PICK_PREFIX = '(cherry picked from commit '
 
 
 class Error:
@@ -349,6 +350,8 @@ class GitCommit:
                     author = self.format_git_author(name)
                     self.co_authors.append(author)
                     continue
+                elif line.startswith(CHERRY_PICK_PREFIX):
+                    continue
 
                 # ChangeLog name will be deduced later
                 if not last_entry:
diff --git a/contrib/gcc-changelog/test_email.py b/contrib/gcc-changelog/test_email.py
index 03abc763212..5e99d3240e8 100755
--- a/contrib/gcc-changelog/test_email.py
+++ b/contrib/gcc-changelog/test_email.py
@@ -258,3 +258,7 @@ class TestGccChangelog(unittest.TestCase):
         email = self.from_patch_glob('0020-IPA-Avoid')
         assert (email.errors[0].message
                 == 'first line should start with a tab, asterisk and space')
+
+    def test_cherry_pick_format(self):
+        email = self.from_patch_glob('0001-c-Alias.patch')
+        assert not email.errors
diff --git a/contrib/gcc-changelog/test_patches.txt b/contrib/gcc-changelog/test_patches.txt
index 39e4753c0ab..ec667be9a92 100644
--- a/contrib/gcc-changelog/test_patches.txt
+++ b/contrib/gcc-changelog/test_patches.txt
@@ -2382,3 +2382,32 @@ index 00000000000..66c87d48694
 -- 
 2.26.1
 
+=== 0001-c-Alias.patch ===
+From 3f1a149fc35cdba988464562e2fb824b10652d6b Mon Sep 17 00:00:00 2001
+From: Nathan Sidwell <nathan@acm.org>
+Date: Tue, 19 May 2020 13:29:19 -0700
+Subject: [PATCH] c++: Alias template instantiation template info
+
+I discovered that the alias instantiation machinery would setup
+template_info, and then sometime later overwrite that with equivalent
+info.  This broke modules, because the template info, once set, is
+logically immutable.  Let's just not do that.
+
+	* pt.c (lookup_template_class_1): Do not reinit template_info of an
+	alias here.
+
+(cherry picked from commit 74744bb1f2847b5b9ce3e97e0fec9c23bb0e499f)
+---
+ gcc/cp/pt.c | 17 +++++++++++++++--
+ 1 file changed, 15 insertions(+), 2 deletions(-)
+
+diff --git a/gcc/cp/pt.c b/gcc/cp/pt.c
+index b8f03d18541..7230ac724ba 100644
+--- a/gcc/cp/pt.c
++++ b/gcc/cp/pt.c
+@@ -1 +1,2 @@
+
++
+-- 
+2.26.2
+
diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index 91d378ba32a..7a950ae5f38 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -26,6 +26,7 @@ git config alias.gcc-descr \!"f() { if test \${1:-no} = --full; then c=\${2:-mas
 git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream); r=\$(echo \$1 | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\$,\\1,p'); n=\$(echo \$1 | sed -n 's,^r[0-9]\\+-\\([0-9]\\+\\)\$,\\1,p'); test -z \$r && echo Invalid id \$1 && exit 1; h=\$(git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$r); test -z \$h && h=\$(git rev-parse --verify --quiet \${o:-origin}/master); p=\$(git describe --all --match 'basepoints/gcc-'\$r \$h | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+-\\([0-9]\\+\\)-g[0-9a-f]*\$,\\2,p;s,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+\$,0,p'); git rev-parse --verify \$h~\$(expr \$p - \$n); }; f"
 
 git config alias.gcc-verify '!f() { "`git rev-parse --show-toplevel`/contrib/gcc-changelog/git_check_commit.py" $@; } ; f'
+git config alias.gcc-backport '!f() { rev=$1; git cherry-pick -x $@; } ; f'
 
 git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog.py" $@; } ; f'
 
-- 
2.26.2

Re: ChangeLog files - server and client scripts (git cherry-pick)

[Thomas Koenig]

Hm, one question: I find the r11-1234 type commit to be much more
readable, in ChangeLog files and everywhere else.

Would it be possible to have that format instead of
"cherry picked from commit $HEX_SOUP" ?

Re: ChangeLog files - server and client scripts (git cherry-pick)

[Martin Liška]

On 5/20/20 11:19 AM, Thomas Koenig wrote:
> Hm, one question: I find the r11-1234 type commit to be much more
> readable, in ChangeLog files and everywhere else.
> 
> Would it be possible to have that format instead of
> "cherry picked from commit $HEX_SOUP" ?

I'm not aware of how to do it.
Please let's keep it simple as possible (cherry-pick -x). You can
always adjust the commit message and mention a r11-1234 revision
there.

Martin

Re: ChangeLog files - server and client scripts (git cherry-pick)

[Jakub Jelinek]

On Wed, May 20, 2020 at 11:19:49AM +0200, Thomas Koenig wrote:
> Hm, one question: I find the r11-1234 type commit to be much more
> readable, in ChangeLog files and everywhere else.
> 
> Would it be possible to have that format instead of
> "cherry picked from commit $HEX_SOUP" ?

I think if you git cherry-pick -x r11-1234-g123bcdef1234
then it will show up in the commit message rather than just
123bcdef1234.  That command will not accept just r11-1234,
as that is not a valid git commit-ish, so you'd need to do
git cherry-pick -x `git gcc-undescr r11-1234`
and then it would show just hash, or
git cherry-pick -x `git gcc-descr --full $(git gcc-undescr r11-1234)`
and then I think it should show up as that r11-1234-g............

	Jakub

Re: ChangeLog files - server and client scripts (git cherry-pick)

[Richard Earnshaw]

On 20/05/2020 10:27, Jakub Jelinek via Gcc wrote:
> On Wed, May 20, 2020 at 11:19:49AM +0200, Thomas Koenig wrote:
>> Hm, one question: I find the r11-1234 type commit to be much more
>> readable, in ChangeLog files and everywhere else.
>>
>> Would it be possible to have that format instead of
>> "cherry picked from commit $HEX_SOUP" ?
> 
> I think if you git cherry-pick -x r11-1234-g123bcdef1234
> then it will show up in the commit message rather than just
> 123bcdef1234.  That command will not accept just r11-1234,
> as that is not a valid git commit-ish, so you'd need to do
> git cherry-pick -x `git gcc-undescr r11-1234`
> and then it would show just hash, or
> git cherry-pick -x `git gcc-descr --full $(git gcc-undescr r11-1234)`
> and then I think it should show up as that r11-1234-g............
> 
> 	Jakub
> 

Do I smell a git gcc-pick alias?

R.

Re: ChangeLog files - server and client scripts

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 180 bytes --]

The scripts were just installed to master except the git alias.
I'm sending that in a separate patch. Now the alias can be used
from any subfolder in a gcc git repository.

Martin

[-- Attachment #2: 0001-Add-gcc-verify-alias.patch --]
[-- Type: text/x-patch, Size: 2553 bytes --]

From eb47191e8d8cbbda285c4df7eb2d1e98091edab9 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Wed, 13 May 2020 14:32:50 +0200
Subject: [PATCH] Add gcc-verify alias.

contrib/ChangeLog:

2020-05-13  Martin Liska  <mliska@suse.cz>

	* gcc-git-customization.sh: Add gcc-verify alias
	that uses contrib/gcc-changelog/git_check_commit.py.
---
 contrib/gcc-git-customization.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index a932bf8c06a..ce293d1fe42 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -25,6 +25,8 @@ git config alias.svn-rev '!f() { rev=$1; shift; git log --all --grep="^From-SVN:
 git config alias.gcc-descr \!"f() { if test \${1:-no} = --full; then c=\${2:-master}; r=\$(git describe --all --abbrev=40 --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-,r,p'); expr match \${r:-no} '^r[0-9]\\+\$' >/dev/null && r=\${r}-0-g\$(git rev-parse \${2:-master}); else c=\${1:-master}; r=\$(git describe --all --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)-\\([0-9]\\+\\)-g[0-9a-f]*\$,r\\2-\\3,p;s,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)\$,r\\2-0,p'); fi; if test -n \$r; then o=\$(git config --get gcc-config.upstream); rr=\$(echo \$r | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\\(-g[0-9a-f]\\+\\)\\?\$,\\1,p'); if git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$rr >/dev/null; then m=releases/gcc-\$rr; else m=master; fi; git merge-base --is-ancestor \$c \${o:-origin}/\$m && \echo \${r}; fi; }; f"
 git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream); r=\$(echo \$1 | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\$,\\1,p'); n=\$(echo \$1 | sed -n 's,^r[0-9]\\+-\\([0-9]\\+\\)\$,\\1,p'); test -z \$r && echo Invalid id \$1 && exit 1; h=\$(git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$r); test -z \$h && h=\$(git rev-parse --verify --quiet \${o:-origin}/master); p=\$(git describe --all --match 'basepoints/gcc-'\$r \$h | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+-\\([0-9]\\+\\)-g[0-9a-f]*\$,\\2,p;s,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+\$,0,p'); git rev-parse --verify \$h~\$(expr \$p - \$n); }; f"
 
+git config alias.gcc-verify '!f() { "`git rev-parse --show-toplevel`/contrib/gcc-changelog/git_check_commit.py" $@; } ; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
-- 
2.26.2

Re: ChangeLog files - server and client scripts

[Richard Earnshaw]

On 13/05/2020 12:05, Martin Liška wrote:
> Hi.
> 
> I'm sending the gcc-changelog relates scripts which should be added to
> contrib
> folder. The patch contains:
> - git_check_commit.py - checking script that verifies git message format
> - git_update_version.py - a replacement of
> maintainer-scripts/update_version_git which
> bumps DATESTAMP and generates ChangeLog entries (for now into
> ChangeLog.test files)
> - git_commit.py, git_email.py and git_repository.py - helper classes
> 
> I also added a new git.config alias: 'gcc-verify' which can be used in
> the following
> way:
> 
> $ git gcc-verify HEAD~2..HEAD -p -n
> Checking 0e4009e9d523270e26856d2441c1be3d8119a477
> OK
> @@CL contrib
> 2020-05-13  Martin Liska  <mliska@suse.cz>
> 
>     * gcc-changelog/git_check_commit.py: New file.
>     * gcc-changelog/git_commit.py: New file.
>     * gcc-changelog/git_email.py: New file.
>     * gcc-changelog/git_repository.py: New file.
>     * gcc-changelog/git_update_version.py: New file.
>     * gcc-git-customization.sh: Add gcc-verify alias.
> @@CL
> Checking 18edc195442291525e04f0fa4d5ef972155117da
> OK
> @@CL gcc
> 2020-05-13  Jakub Jelinek  <jakub@redhat.com>
> 
>     PR debug/95080
>     * cfgrtl.c (purge_dead_edges): Skip over debug and note insns even
>     if the last insn is a note.
> @@CL gcc/testsuite
> 2020-05-13  Jakub Jelinek  <jakub@redhat.com>
> 
>     PR debug/95080
>     * g++.dg/opt/pr95080.C: New test.
> @@CL
> 
> Note the -n option which disables _strict mode_ (modification of both
> ChangeLog
> and another files).
> 
> The second part is git hook that will reject all commits for release and
> master branches.
> that violate ChangeLog format. Right now, strict mode is disabled in the
> hooks.
> 
> What's still missing to be done is format of Revert and Backport commits.
> I suggest to use native 'git revert XYZ' and 'git cherry-pick -x XYZ'.
> Doing that the commit messages will provide link to original commit and
> the script
> can later append corresponding 'Backported ..' or 'Reverted' line.
> 
> Thoughts?
> Martin

I've just realized this doesn't give us an easy way to mark changes for
the root-level ChangeLog file, unless, perhaps "@@ CL ." works?

R.

Re: ChangeLog files - server and client scripts

[Martin Liška]

On 5/13/20 3:24 PM, Richard Earnshaw wrote:
> I've just realized this doesn't give us an easy way to mark changes for
> the root-level ChangeLog file, unless, perhaps "@@ CL ." works?

This works fine:
'ChangeLog:'

as seen for instance here:

commit 9ad3c1d81c129fc76594b9df5b798c380cbf03ee
Author: Stefan Schulze Frielinghaus <stefansf@linux.ibm.com>
Date:   Wed Apr 22 09:20:08 2020 +0200

     MAINTAINERS: add myself for write after approval
     
     ChangeLog:
     
     2020-04-22  Stefan Schulze Frielinghaus  <stefansf@linux.ibm.com>
     
             * MAINTAINERS (Write After Approval): add myself

Martin

Re: ChangeLog files - server and client scripts

[Joseph Myers]

On Wed, 13 May 2020, Martin Liška wrote:

> I'm sending the gcc-changelog relates scripts which should be added to contrib
> folder. The patch contains:
> - git_check_commit.py - checking script that verifies git message format

We need a documentation patch to contribute.html or gitwrite.html that 
describes the exact commit message format being used.

> - git_update_version.py - a replacement of
> maintainer-scripts/update_version_git which
> bumps DATESTAMP and generates ChangeLog entries (for now into ChangeLog.test
> files)

Where does this check things out?  (The existing ~gccadmin/gcc-checkout 
isn't suitable for that, it needs to stay on master to have the correct 
version of maintainer-scripts rather than being switched to other 
branches, though I suppose a second long-lived checkout that gets updated 
automatically could be used.  If you check things out somewhere else 
temporarily, it's important to be sure the checkout gets deleted 
afterwards rather than having multiple checkouts accumulating.  That's 
especially the case if you use a checkout in /tmp as a single GCC 
repository clone / checkout uses a significant proportion of the free 
space on the root filesystem; /sourceware/snapshot-tmp/gcc has more free 
space for large temporary directories.)

> The second part is git hook that will reject all commits for release and
> master branches.
> that violate ChangeLog format. Right now, strict mode is disabled in the
> hooks.

Note that the present state of having GCC-specific patches to the git 
hooks is supposed to be a temporary one; we want to move to all relevant 
GCC-specific configuration being in refs/meta/config rather than custom 
code, so GCC and sourceware can share a single instance of the hooks which 
in turn can use the same code as in the upstream AdaCore repository, so 
that future updates of the hooks from upstream are easier.  See the issues 
I filed at https://github.com/AdaCore/git-hooks/issues for the existing 
custom GCC changes and the pull request 
https://github.com/AdaCore/git-hooks/pull/12 to bring in implementations 
of many of those features (not sure if it covers everything or not).  So 
it's important to consider how these checks could be implemented without 
needing GCC-specific code directly in these hooks (for example, using the 
new hooks.update-hook mechanism added by one of the commits in that pull 
request, or getting extra features added to the upstream hooks in a 
generic form if necessary).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: ChangeLog files - server and client scripts

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 2971 bytes --]

On 5/13/20 7:53 PM, Joseph Myers wrote:
> On Wed, 13 May 2020, Martin Liška wrote:
> 
>> I'm sending the gcc-changelog relates scripts which should be added to contrib
>> folder. The patch contains:
>> - git_check_commit.py - checking script that verifies git message format
> 
> We need a documentation patch to contribute.html or gitwrite.html that
> describes the exact commit message format being used.

Sure, I'm sending patch for that.

> 
>> - git_update_version.py - a replacement of
>> maintainer-scripts/update_version_git which
>> bumps DATESTAMP and generates ChangeLog entries (for now into ChangeLog.test
>> files)
> 
> Where does this check things out?  (The existing ~gccadmin/gcc-checkout
> isn't suitable for that, it needs to stay on master to have the correct
> version of maintainer-scripts rather than being switched to other
> branches, though I suppose a second long-lived checkout that gets updated
> automatically could be used.  If you check things out somewhere else
> temporarily, it's important to be sure the checkout gets deleted
> afterwards rather than having multiple checkouts accumulating.  That's
> especially the case if you use a checkout in /tmp as a single GCC
> repository clone / checkout uses a significant proportion of the free
> space on the root filesystem; /sourceware/snapshot-tmp/gcc has more free
> space for large temporary directories.)

Well, we can make a proper git clone of the original repository that will
be used for the daily bumps. I bet we'll have a place for one more clone?

> 
>> The second part is git hook that will reject all commits for release and
>> master branches.
>> that violate ChangeLog format. Right now, strict mode is disabled in the
>> hooks.
> 
> Note that the present state of having GCC-specific patches to the git
> hooks is supposed to be a temporary one; we want to move to all relevant
> GCC-specific configuration being in refs/meta/config rather than custom
> code, so GCC and sourceware can share a single instance of the hooks which
> in turn can use the same code as in the upstream AdaCore repository, so
> that future updates of the hooks from upstream are easier.  See the issues
> I filed at https://github.com/AdaCore/git-hooks/issues for the existing
> custom GCC changes and the pull request
> https://github.com/AdaCore/git-hooks/pull/12 to bring in implementations
> of many of those features (not sure if it covers everything or not).  So
> it's important to consider how these checks could be implemented without
> needing GCC-specific code directly in these hooks (for example, using the
> new hooks.update-hook mechanism added by one of the commits in that pull
> request, or getting extra features added to the upstream hooks in a
> generic form if necessary).
> 

I welcome the attempt to unify the hooks with AdaCore upstream. I believe this
should not block gcc-changelog attempt now. Later on, we can add it to
hooks.update-hook mechanism.

Thanks,
Martin

[-- Attachment #2: gcc-changelog-documentation.patch --]
[-- Type: text/x-patch, Size: 11233 bytes --]

diff --git a/htdocs/codingconventions.html b/htdocs/codingconventions.html
index f4732ef6..ffa6db32 100644
--- a/htdocs/codingconventions.html
+++ b/htdocs/codingconventions.html
@@ -112,9 +112,14 @@ maintained and kept up to date.  In particular:</p>
 
 <h2 id="ChangeLogs">ChangeLogs</h2>
 
-<p>GCC requires ChangeLog entries for documentation changes; for the web
-pages (apart from <code>java/</code> and <code>libstdc++/</code>) the CVS
-commit logs are sufficient.</p>
+<p>
+ChangeLog entries are part of git commit messages and are automatically put
+into a corresponding ChangeLog file.  A ChangeLog template can be easily generated
+with <code>./contrib/mklog</code> script.  GCC offers a checking script that
+verifies a proper ChangeLog formatting (see <code>git gcc-verify</code> git alias).
+for a particular git commit.  The checking script covers most commonly used ChangeLog
+formats and the following paragraphs explain what it supports.
+</p>
 
 <p>See also what the <a
 href="http://www.gnu.org/prep/standards_toc.html">GNU Coding
@@ -124,19 +129,95 @@ in comments rather than the ChangeLog, though a single line overall
 description of the changes may be useful above the ChangeLog entry for
 a large batch of changes.</p>
 
-<p>For changes that are ported from another branch, we recommend to
-use a single entry whose body contains a verbatim copy of the original
-entries describing the changes on that branch, possibly preceded by a
-single-line overall description of the changes.</p>
+<h3>Components</h3>
+
+<ul>
+    <li><code class="other">git_description</code> - a leading text with git commit description</li>
+    <li><code class="other">author_timestamp</code> - line with timestamp and an author name and email (2 spaces before and after name) <br>
+        example: <code class="other">2020-04-23␣␣Martin Liska␣␣&lt;mliska@suse.cz&gt;</code></li>
+    <li><code class="other">additional_author</code> - line with additional commit author name and email (starting with a tabular and 4 spaces) <br>
+        example: <code class="other">\t␣␣␣␣Martin Liska␣␣&lt;mliska@suse.cz&gt;</code></li>
+    <li><code class="other">changelog_location</code> - a location to a ChangeLog file <br>
+        supported formats: <code class="other">a/b/c/ChangeLog</code>, <code class="other">a/b/c/ChangeLog:</code>, <code class="other">a/b/c/</code> (where ChangeLog file lives in the folder), <code class="other">\ta/b/c/</code> and <code class="other">a/b/c</code></li>
+    <li><code class="other">pr_entry</code> - bug report reference <br>
+        example: <code class="other">\tPR component/12345</code></li>
+    <li><code class="other">changelog_file</code> - a modified file mentined in a ChangeLog:
+        supported formats: <code class="other">\t* a/b/c/file.c:</code>, <code class="other">\t* a/b/c/file.c (function):</code>, <code class="other">\t* a/b/c/file1.c, a/b/c/file2.c:</code></li>
+    <li><code class="other">changelog_file_comment</code> - line that follows a <code class="other">changelog_file</code> with description of changes in the file;
+        must start with <code class="other">\t</code></li>
+    <li><code class="other">co_authored_by</code> - <a href="https://help.github.com/en/github/committing-changes-to-your-project/creating-a-commit-with-multiple-authors">GitHub format</a> for a Co-Authored-By</li>
+</ul>
+
+<h3>Format rules</h3>
+
+<ul>
+    <li><code class="other">git_description</code> - optional; ends right before one of the other compoments is found</li>
+    <li><code class="other">author_timestamp</code> - optional; when found before a <code class="other">changelog_file</code>, then it is added
+    to each changelog entry</li>
+    <li><code class="other">additional_author</code> - optional</li>
+    <li><code class="other">changelog_location</code> - optional; parser attempts to identify ChangeLog file based
+    on modified files; <code class="other">$changelog_location</code> belonging to a different ChangeLog must
+    be separated with an empty line</li>
+    <li><code class="other">pr_entry</code> - optional; can contain any number of PR entries</li>
+    <li><code class="other">changelog_file</code> - each <code>changelog_location</code> must contain at least one file</li>
+    <li><code class="other">changelog_file_comment</code> - optional</li>
+    <li><code class="other">co_authored_by</code> - optional, can contain more than one</li>
+</ul>
+
+<h3>Documented behaviour</h3>
+
+<ul>
+    <li>a missing <code class="other">changelog_location</code> file location can be deduced based on group of <code class="other">changelog_file</code>s</li>
+    <li>script automatically generates missing "New file." entries for files that are added in a commit</li>
+    <li>changed files that are not mentioned in a ChangeLog file generate an error</li>
+    <li>similarly for unchanged files that are mentioned in a ChangeLog file</li>
+    <li>a commit author and date stamp can be automatically deduced from a git commit - we recommend to use it</li>
+    <li><code class="other">co_authored_by</code> is added to each ChangeLog entry</li>
+    <li>a PR component is checked against list of valid components</li>
+    <li><code>ChangeLog</code> files, <code>DATESTAMP</code>, <code>BASE-VER</code> and <code>DEV-PHASE</code> can be modified only separately from other files</li>
+</ul>
 
-<p>There is no established convention on when ChangeLog entries are to
-be made for testsuite changes; see messages <a
-href="https://gcc.gnu.org/ml/gcc/2000-09/msg00287.html">1</a> and <a
-href="https://gcc.gnu.org/ml/gcc/2000-09/msg00290.html">2</a>.</p>
+<h3>Example patch</h3>
 
-<p>If your change fixes a PR, put <code>PR java/58</code> (where
-<code>java/58</code> is the actual number of the PR) at the top
-of the ChangeLog entry.</p>
+<pre><code>This patch adds a second movk pattern that models the instruction
+as a "normal" and/ior operation rather than an insertion.  It fixes
+the third insv_1.c failure in PR87763, which was a regression from
+GCC 8.
+
+2020-02-06  John Foo  &lt;john@example.com&gt;
+
+gcc/
+	PR target/87763
+	* config/aarch64/aarch64-protos.h (aarch64_movk_shift): Declare.
+	* config/aarch64/aarch64.c (aarch64_movk_shift): New function.
+	* config/aarch64/aarch64.md (aarch64_movk&lt;mode&gt;): New pattern.
+
+gcc/testsuite/
+	PR target/87763
+	* gcc.target/aarch64/movk_2.c: New test.
+
+Co-Authored-By: Jack Bar  &lt;jack@example.com&gt;
+</code></pre>
+
+<h3>Tokenized patch</h3>
+
+<pre>
+<code>$git_description
+
+$author_timestamp
+
+$changelog_location
+$pr_entry
+$changelog_file
+$changelog_file
+$changelog_file
+
+$changelog_location
+$pr_entry
+$changelog_file
+
+$co_authored_by</code>
+</pre>
 
 <h2 id="Portability">Portability</h2>
 
@@ -201,7 +282,6 @@ minimize the number of function prototypes, by defining them before
 their first use.  Function prototypes should only be used when
 necessary, to break mutually recursive cycles.</p>
 
-
 <h2 id="Makefiles">Makefiles</h2>
 
 <p><code>touch</code> should never be used in GCC Makefiles.  Instead
diff --git a/htdocs/contribute.html b/htdocs/contribute.html
index 3d03b9d1..80a4470e 100644
--- a/htdocs/contribute.html
+++ b/htdocs/contribute.html
@@ -177,22 +177,10 @@ testcases cannot be added.
 
 <dt>ChangeLog</dt>
 <dd>
-A ChangeLog entry as plaintext; see the various ChangeLog files for
-format and content, and the <a href="codingconventions.html">GCC
+A ChangeLog entry as plaintext; see the <a href="codingconventions.html#ChangeLogs">GCC
 coding conventions</a> and <a
 href="http://www.gnu.org/prep/standards_toc.html">GNU Coding
-Standards</a> for further information.  The ChangeLog entries should
-be plaintext rather than part of the patch since the top of the
-ChangeLog changes rapidly and a patch to the ChangeLog would probably
-no longer apply by the time your patch is reviewed.
-If your change fixes a PR, put text in the ChangeLog entry mentioning
-the PR.  Our infrastructure understands how to
-extract this information and automatically append the commit log to
-the PR.  In order to be recognized, the text must fit a particular
-form.  It must start with "PR", and then must include the category
-and PR number.  For instance, <code>PR java/2369</code> is
-valid.  Multiple PRs can be mentioned in a single message.
-</dd>
+Standards</a> for further information.</dd>
 
 <dt>Bootstrapping and testing</dt>
 <dd>
diff --git a/htdocs/gitwrite.html b/htdocs/gitwrite.html
index 97d60870..791213a5 100644
--- a/htdocs/gitwrite.html
+++ b/htdocs/gitwrite.html
@@ -25,7 +25,6 @@ maintainers and significant developers.</p>
   <li><a href="#checkin">Checking in a change</a></li>
   <li><a href="#example">Example check-in session</a></li>
   <li><a href="#branches">Creating and using branches</a></li>
-  <li><a href="#changelog">git-merge-changelog</a></li>
   <li><a href="#vendor">Personal and Vendor branches</a></li>
   <li><a href="#account">Tips&amp;Tricks around your account</a></li>
 </ol>
@@ -237,9 +236,10 @@ pull</code>" before attempting a checkin; this will save you a little
 time if someone else has modified the source tree since the last time
 you synced your sources.</li>
 
-<li>Apply the patch to your local tree and update the ChangeLog file.
-Use the current date/time for the ChangeLog entry, not the time that
-the patch was submitted.</li>
+<li>Apply the patch to your local tree.  ChangeLog entries will be
+automatically added to the corresponding ChangeLog files based
+on the git commit message.  See the documentation of
+<a href="codingconventions.html#ChangeLogs">ChangeLog format</a>.</li>
 
 <li>Make sure to rebuild any generated files that would be affected by
 the patch.  Make sure to check them in along with the files explicitly
@@ -347,30 +347,6 @@ accordingly.  It may be easier to cherry-pick some smaller changes onto master
 don't need to <code>merge --squash</code> squash, but still need to make sure
 the commits on the branch satisfy the above rules for commits.
 
-<hr />
-<h2 id="changelog">git-merge-changelog</h2>
-
-git's native handling of ChangeLog merges is pretty bad, but there's a separate
-git-merge-changelog tool that improves things dramatically.  Some operating
-system distributions have a git-merge-changelog package already, or to build it
-by hand you can do
-
-<blockquote><pre>
-git clone git://git.savannah.gnu.org/gnulib.git
-cd gnulib
-./gnulib-tool --create-testdir --dir=/tmp/testdir123 git-merge-changelog
-cd /tmp/testdir123
-./configure
-make
-make install
-</pre></blockquote>
-And then to enable it, do
-<blockquote><pre>
-git config --global merge.merge-changelog.name "GNU-style ChangeLog merge driver"
-git config --global merge.merge-changelog.driver "git-merge-changelog %O %A %B"
-echo "ChangeLog*   merge=merge-changelog" >> $GCCSRCDIR/.git/info/attributes
-</pre></blockquote>
-
 <hr />
 <h2 id="vendor">Personal and vendor branches</h2>
 
@@ -457,6 +433,7 @@ repository:</p>
   </li>
   <li><i>gcc-descr</i> - Undocumented</li>
   <li><i>gcc-undescr</i> - Undocumented</li>
+  <li><i>gcc-verify</i> - Undocumented</li>
 </ul>
 
 <p>The final customization that the script makes is to add a diff rule so

Re: ChangeLog files - server and client scripts

[Joseph Myers]

On Thu, 14 May 2020, Martin Liška wrote:

> On 5/13/20 7:53 PM, Joseph Myers wrote:
> > On Wed, 13 May 2020, Martin Liška wrote:
> > 
> > > I'm sending the gcc-changelog relates scripts which should be added to
> > > contrib
> > > folder. The patch contains:
> > > - git_check_commit.py - checking script that verifies git message format
> > 
> > We need a documentation patch to contribute.html or gitwrite.html that
> > describes the exact commit message format being used.
> 
> Sure, I'm sending patch for that.

Thanks.  There are references to author timestamps there.  The date in a 
ChangeLog entry should always be a commit timestamp, not an author one, so 
author timestamps present either in commit messages or in the git commit 
metadata should be ignored, with only the committer timestamps from the 
git commit metadata being used when generating ChangeLog files.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: ChangeLog files - server and client scripts

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 1069 bytes --]

On 5/14/20 6:47 PM, Joseph Myers wrote:
> On Thu, 14 May 2020, Martin Liška wrote:
> 
>> On 5/13/20 7:53 PM, Joseph Myers wrote:
>>> On Wed, 13 May 2020, Martin Liška wrote:
>>>
>>>> I'm sending the gcc-changelog relates scripts which should be added to
>>>> contrib
>>>> folder. The patch contains:
>>>> - git_check_commit.py - checking script that verifies git message format
>>>
>>> We need a documentation patch to contribute.html or gitwrite.html that
>>> describes the exact commit message format being used.
>>
>> Sure, I'm sending patch for that.
> 
> Thanks.  There are references to author timestamps there.  The date in a
> ChangeLog entry should always be a commit timestamp, not an author one, so
> author timestamps present either in commit messages or in the git commit
> metadata should be ignored, with only the committer timestamps from the
> git commit metadata being used when generating ChangeLog files.

You are fully right, a committer date is what should be used.
Fixed in the documentation, note that the scripts use committed date.

Martin


[-- Attachment #2: gcc-changelog-documentation-v2.patch --]
[-- Type: text/x-patch, Size: 11252 bytes --]

diff --git a/htdocs/codingconventions.html b/htdocs/codingconventions.html
index f4732ef6..d2e73962 100644
--- a/htdocs/codingconventions.html
+++ b/htdocs/codingconventions.html
@@ -112,9 +112,14 @@ maintained and kept up to date.  In particular:</p>
 
 <h2 id="ChangeLogs">ChangeLogs</h2>
 
-<p>GCC requires ChangeLog entries for documentation changes; for the web
-pages (apart from <code>java/</code> and <code>libstdc++/</code>) the CVS
-commit logs are sufficient.</p>
+<p>
+ChangeLog entries are part of git commit messages and are automatically put
+into a corresponding ChangeLog file.  A ChangeLog template can be easily generated
+with <code>./contrib/mklog</code> script.  GCC offers a checking script that
+verifies a proper ChangeLog formatting (see <code>git gcc-verify</code> git alias).
+for a particular git commit.  The checking script covers most commonly used ChangeLog
+formats and the following paragraphs explain what it supports.
+</p>
 
 <p>See also what the <a
 href="http://www.gnu.org/prep/standards_toc.html">GNU Coding
@@ -124,19 +129,95 @@ in comments rather than the ChangeLog, though a single line overall
 description of the changes may be useful above the ChangeLog entry for
 a large batch of changes.</p>
 
-<p>For changes that are ported from another branch, we recommend to
-use a single entry whose body contains a verbatim copy of the original
-entries describing the changes on that branch, possibly preceded by a
-single-line overall description of the changes.</p>
+<h3>Components</h3>
+
+<ul>
+    <li><code class="other">git_description</code> - a leading text with git commit description</li>
+    <li><code class="other">committer_timestamp</code> - line with timestamp and an author name and email (2 spaces before and after name) <br>
+        example: <code class="other">2020-04-23␣␣Martin Liska␣␣&lt;mliska@suse.cz&gt;</code></li>
+    <li><code class="other">additional_author</code> - line with additional commit author name and email (starting with a tabular and 4 spaces) <br>
+        example: <code class="other">\t␣␣␣␣Martin Liska␣␣&lt;mliska@suse.cz&gt;</code></li>
+    <li><code class="other">changelog_location</code> - a location to a ChangeLog file <br>
+        supported formats: <code class="other">a/b/c/ChangeLog</code>, <code class="other">a/b/c/ChangeLog:</code>, <code class="other">a/b/c/</code> (where ChangeLog file lives in the folder), <code class="other">\ta/b/c/</code> and <code class="other">a/b/c</code></li>
+    <li><code class="other">pr_entry</code> - bug report reference <br>
+        example: <code class="other">\tPR component/12345</code></li>
+    <li><code class="other">changelog_file</code> - a modified file mentined in a ChangeLog:
+        supported formats: <code class="other">\t* a/b/c/file.c:</code>, <code class="other">\t* a/b/c/file.c (function):</code>, <code class="other">\t* a/b/c/file1.c, a/b/c/file2.c:</code></li>
+    <li><code class="other">changelog_file_comment</code> - line that follows a <code class="other">changelog_file</code> with description of changes in the file;
+        must start with <code class="other">\t</code></li>
+    <li><code class="other">co_authored_by</code> - <a href="https://help.github.com/en/github/committing-changes-to-your-project/creating-a-commit-with-multiple-authors">GitHub format</a> for a Co-Authored-By</li>
+</ul>
+
+<h3>Format rules</h3>
+
+<ul>
+    <li><code class="other">git_description</code> - optional; ends right before one of the other compoments is found</li>
+    <li><code class="other">committer_timestamp</code> - optional; when found before a <code class="other">changelog_file</code>, then it is added
+    to each changelog entry</li>
+    <li><code class="other">additional_author</code> - optional</li>
+    <li><code class="other">changelog_location</code> - optional; parser attempts to identify ChangeLog file based
+    on modified files; <code class="other">$changelog_location</code> belonging to a different ChangeLog must
+    be separated with an empty line</li>
+    <li><code class="other">pr_entry</code> - optional; can contain any number of PR entries</li>
+    <li><code class="other">changelog_file</code> - each <code>changelog_location</code> must contain at least one file</li>
+    <li><code class="other">changelog_file_comment</code> - optional</li>
+    <li><code class="other">co_authored_by</code> - optional, can contain more than one</li>
+</ul>
+
+<h3>Documented behaviour</h3>
+
+<ul>
+    <li>a missing <code class="other">changelog_location</code> file location can be deduced based on group of <code class="other">changelog_file</code>s</li>
+    <li>script automatically generates missing "New file." entries for files that are added in a commit</li>
+    <li>changed files that are not mentioned in a ChangeLog file generate an error</li>
+    <li>similarly for unchanged files that are mentioned in a ChangeLog file</li>
+    <li>a commit author and committer date stamp can be automatically deduced from a git commit - we recommend to use it</li>
+    <li><code class="other">co_authored_by</code> is added to each ChangeLog entry</li>
+    <li>a PR component is checked against list of valid components</li>
+    <li><code>ChangeLog</code> files, <code>DATESTAMP</code>, <code>BASE-VER</code> and <code>DEV-PHASE</code> can be modified only separately from other files</li>
+</ul>
 
-<p>There is no established convention on when ChangeLog entries are to
-be made for testsuite changes; see messages <a
-href="https://gcc.gnu.org/ml/gcc/2000-09/msg00287.html">1</a> and <a
-href="https://gcc.gnu.org/ml/gcc/2000-09/msg00290.html">2</a>.</p>
+<h3>Example patch</h3>
 
-<p>If your change fixes a PR, put <code>PR java/58</code> (where
-<code>java/58</code> is the actual number of the PR) at the top
-of the ChangeLog entry.</p>
+<pre><code>This patch adds a second movk pattern that models the instruction
+as a "normal" and/ior operation rather than an insertion.  It fixes
+the third insv_1.c failure in PR87763, which was a regression from
+GCC 8.
+
+2020-02-06  John Foo  &lt;john@example.com&gt;
+
+gcc/
+	PR target/87763
+	* config/aarch64/aarch64-protos.h (aarch64_movk_shift): Declare.
+	* config/aarch64/aarch64.c (aarch64_movk_shift): New function.
+	* config/aarch64/aarch64.md (aarch64_movk&lt;mode&gt;): New pattern.
+
+gcc/testsuite/
+	PR target/87763
+	* gcc.target/aarch64/movk_2.c: New test.
+
+Co-Authored-By: Jack Bar  &lt;jack@example.com&gt;
+</code></pre>
+
+<h3>Tokenized patch</h3>
+
+<pre>
+<code>$git_description
+
+$committer_timestamp
+
+$changelog_location
+$pr_entry
+$changelog_file
+$changelog_file
+$changelog_file
+
+$changelog_location
+$pr_entry
+$changelog_file
+
+$co_authored_by</code>
+</pre>
 
 <h2 id="Portability">Portability</h2>
 
@@ -201,7 +282,6 @@ minimize the number of function prototypes, by defining them before
 their first use.  Function prototypes should only be used when
 necessary, to break mutually recursive cycles.</p>
 
-
 <h2 id="Makefiles">Makefiles</h2>
 
 <p><code>touch</code> should never be used in GCC Makefiles.  Instead
diff --git a/htdocs/contribute.html b/htdocs/contribute.html
index 3d03b9d1..80a4470e 100644
--- a/htdocs/contribute.html
+++ b/htdocs/contribute.html
@@ -177,22 +177,10 @@ testcases cannot be added.
 
 <dt>ChangeLog</dt>
 <dd>
-A ChangeLog entry as plaintext; see the various ChangeLog files for
-format and content, and the <a href="codingconventions.html">GCC
+A ChangeLog entry as plaintext; see the <a href="codingconventions.html#ChangeLogs">GCC
 coding conventions</a> and <a
 href="http://www.gnu.org/prep/standards_toc.html">GNU Coding
-Standards</a> for further information.  The ChangeLog entries should
-be plaintext rather than part of the patch since the top of the
-ChangeLog changes rapidly and a patch to the ChangeLog would probably
-no longer apply by the time your patch is reviewed.
-If your change fixes a PR, put text in the ChangeLog entry mentioning
-the PR.  Our infrastructure understands how to
-extract this information and automatically append the commit log to
-the PR.  In order to be recognized, the text must fit a particular
-form.  It must start with "PR", and then must include the category
-and PR number.  For instance, <code>PR java/2369</code> is
-valid.  Multiple PRs can be mentioned in a single message.
-</dd>
+Standards</a> for further information.</dd>
 
 <dt>Bootstrapping and testing</dt>
 <dd>
diff --git a/htdocs/gitwrite.html b/htdocs/gitwrite.html
index 97d60870..791213a5 100644
--- a/htdocs/gitwrite.html
+++ b/htdocs/gitwrite.html
@@ -25,7 +25,6 @@ maintainers and significant developers.</p>
   <li><a href="#checkin">Checking in a change</a></li>
   <li><a href="#example">Example check-in session</a></li>
   <li><a href="#branches">Creating and using branches</a></li>
-  <li><a href="#changelog">git-merge-changelog</a></li>
   <li><a href="#vendor">Personal and Vendor branches</a></li>
   <li><a href="#account">Tips&amp;Tricks around your account</a></li>
 </ol>
@@ -237,9 +236,10 @@ pull</code>" before attempting a checkin; this will save you a little
 time if someone else has modified the source tree since the last time
 you synced your sources.</li>
 
-<li>Apply the patch to your local tree and update the ChangeLog file.
-Use the current date/time for the ChangeLog entry, not the time that
-the patch was submitted.</li>
+<li>Apply the patch to your local tree.  ChangeLog entries will be
+automatically added to the corresponding ChangeLog files based
+on the git commit message.  See the documentation of
+<a href="codingconventions.html#ChangeLogs">ChangeLog format</a>.</li>
 
 <li>Make sure to rebuild any generated files that would be affected by
 the patch.  Make sure to check them in along with the files explicitly
@@ -347,30 +347,6 @@ accordingly.  It may be easier to cherry-pick some smaller changes onto master
 don't need to <code>merge --squash</code> squash, but still need to make sure
 the commits on the branch satisfy the above rules for commits.
 
-<hr />
-<h2 id="changelog">git-merge-changelog</h2>
-
-git's native handling of ChangeLog merges is pretty bad, but there's a separate
-git-merge-changelog tool that improves things dramatically.  Some operating
-system distributions have a git-merge-changelog package already, or to build it
-by hand you can do
-
-<blockquote><pre>
-git clone git://git.savannah.gnu.org/gnulib.git
-cd gnulib
-./gnulib-tool --create-testdir --dir=/tmp/testdir123 git-merge-changelog
-cd /tmp/testdir123
-./configure
-make
-make install
-</pre></blockquote>
-And then to enable it, do
-<blockquote><pre>
-git config --global merge.merge-changelog.name "GNU-style ChangeLog merge driver"
-git config --global merge.merge-changelog.driver "git-merge-changelog %O %A %B"
-echo "ChangeLog*   merge=merge-changelog" >> $GCCSRCDIR/.git/info/attributes
-</pre></blockquote>
-
 <hr />
 <h2 id="vendor">Personal and vendor branches</h2>
 
@@ -457,6 +433,7 @@ repository:</p>
   </li>
   <li><i>gcc-descr</i> - Undocumented</li>
   <li><i>gcc-undescr</i> - Undocumented</li>
+  <li><i>gcc-verify</i> - Undocumented</li>
 </ul>
 
 <p>The final customization that the script makes is to add a diff rule so

Re: ChangeLog files - server and client scripts

[Martin Liška]

Hello.

We've just installed server git hooks that verify git messages
for a correct ChangeLog format. For a limited time period, please
still apply ChangeLog changes to the corresponding ChangeLog files.
We'll use it for comparison of auto-generated CangeLog entries.

The format is documented here:
https://gcc.gnu.org/codingconventions.html#ChangeLogs

And I would recommend to install the new 'git gcc-verify' hook from:
contrib/gcc-git-customization.sh

Feel free to contact me about future troubles you'll see.

Thanks,
Martin

Re: ChangeLog files - server and client scripts

[Jonathan Wakely]

[-- Attachment #1: Type: text/plain, Size: 1175 bytes --]

On Tue, 19 May 2020 at 11:44, Martin Liška <mliska@suse.cz> wrote:
>
> Hello.
>
> We've just installed server git hooks that verify git messages
> for a correct ChangeLog format. For a limited time period, please
> still apply ChangeLog changes to the corresponding ChangeLog files.
> We'll use it for comparison of auto-generated CangeLog entries.
>
> The format is documented here:
> https://gcc.gnu.org/codingconventions.html#ChangeLogs
>
> And I would recommend to install the new 'git gcc-verify' hook from:
> contrib/gcc-git-customization.sh
>
> Feel free to contact me about future troubles you'll see.

The --allow-non-strict-mode option seems unnecessarily verbose. It's
not allowing non-strict mode, it's enabling it. Would --non-strict or
--relaxed be better?

And I don't understand the help text for it. 'Use non-strict mode
(change in both ChangeLog and other files.' It seems that non-strict
mode allows changes to certain "project files" that are not supposed
to be manually edited, but I can't correlate that to "change in both
ChangeLog and other files". It's also missing the closing parenthesis.

Would this patch make sense?

[-- Attachment #2: patch.txt --]
[-- Type: text/plain, Size: 928 bytes --]

diff --git a/contrib/gcc-changelog/git_check_commit.py b/contrib/gcc-changelog/git_check_commit.py
index 8553c90a96f..d504be9dbde 100755
--- a/contrib/gcc-changelog/git_check_commit.py
+++ b/contrib/gcc-changelog/git_check_commit.py
@@ -28,9 +28,9 @@ parser.add_argument('-g', '--git-path', default='.',
                     help='Path to git repository')
 parser.add_argument('-p', '--print-changelog', action='store_true',
                     help='Print final changelog entires')
-parser.add_argument('-n', '--allow-non-strict-mode', action='store_true',
-                    help='Allow non-strict mode (change in both ChangeLog and '
-                    'other files.')
+parser.add_argument('-n', '--non-strict-mode', action='store_true',
+                    help='Use non-strict mode (allow changes in ChangeLog and '
+                    'other automatically updated files.')
 args = parser.parse_args()
 
 retval = 0

Re: ChangeLog files - server and client scripts

[Jonathan Wakely]

[-- Attachment #1: Type: text/plain, Size: 1371 bytes --]

On Tue, 19 May 2020 at 23:19, Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
>
> On Tue, 19 May 2020 at 11:44, Martin Liška <mliska@suse.cz> wrote:
> >
> > Hello.
> >
> > We've just installed server git hooks that verify git messages
> > for a correct ChangeLog format. For a limited time period, please
> > still apply ChangeLog changes to the corresponding ChangeLog files.
> > We'll use it for comparison of auto-generated CangeLog entries.
> >
> > The format is documented here:
> > https://gcc.gnu.org/codingconventions.html#ChangeLogs
> >
> > And I would recommend to install the new 'git gcc-verify' hook from:
> > contrib/gcc-git-customization.sh
> >
> > Feel free to contact me about future troubles you'll see.
>
> The --allow-non-strict-mode option seems unnecessarily verbose. It's
> not allowing non-strict mode, it's enabling it. Would --non-strict or
> --relaxed be better?
>
> And I don't understand the help text for it. 'Use non-strict mode
> (change in both ChangeLog and other files.' It seems that non-strict
> mode allows changes to certain "project files" that are not supposed
> to be manually edited, but I can't correlate that to "change in both
> ChangeLog and other files". It's also missing the closing parenthesis.
>
> Would this patch make sense?

Or this one that actually adds the closing parenthesis :-)

[-- Attachment #2: patch.txt --]
[-- Type: text/plain, Size: 929 bytes --]

diff --git a/contrib/gcc-changelog/git_check_commit.py b/contrib/gcc-changelog/git_check_commit.py
index 8553c90a96f..4fa2bb0b4a2 100755
--- a/contrib/gcc-changelog/git_check_commit.py
+++ b/contrib/gcc-changelog/git_check_commit.py
@@ -28,9 +28,9 @@ parser.add_argument('-g', '--git-path', default='.',
                     help='Path to git repository')
 parser.add_argument('-p', '--print-changelog', action='store_true',
                     help='Print final changelog entires')
-parser.add_argument('-n', '--allow-non-strict-mode', action='store_true',
-                    help='Allow non-strict mode (change in both ChangeLog and '
-                    'other files.')
+parser.add_argument('-n', '--non-strict-mode', action='store_true',
+                    help='Use non-strict mode (allow changes in ChangeLog and '
+                    'other automatically updated files).')
 args = parser.parse_args()
 
 retval = 0

Re: ChangeLog files - server and client scripts

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 208 bytes --]

On 5/20/20 12:20 AM, Jonathan Wakely wrote:
> Or this one that actually adds the closing parenthesis

Thank you, it's definitely an improvement.

There's final version of the patch I've just applied.

Martin

[-- Attachment #2: 0001-git_check_commit-shorted-option-name.patch --]
[-- Type: text/x-patch, Size: 1762 bytes --]

From a55c1018c9d7c53b643203e7f71b06953fae86a1 Mon Sep 17 00:00:00 2001
From: Jonathan Wakely <jwakely@redhat.com>
Date: Wed, 20 May 2020 10:03:51 +0200
Subject: [PATCH] git_check_commit: shorted option name

contrib/ChangeLog:

2020-05-20  Martin Liska  <mliska@suse.cz>

	* gcc-changelog/git_check_commit.py: Change
	--allow-non-strict-mode to --non-strict-mode.
---
 contrib/gcc-changelog/git_check_commit.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/contrib/gcc-changelog/git_check_commit.py b/contrib/gcc-changelog/git_check_commit.py
index 8553c90a96f..2601ae4f613 100755
--- a/contrib/gcc-changelog/git_check_commit.py
+++ b/contrib/gcc-changelog/git_check_commit.py
@@ -28,14 +28,14 @@ parser.add_argument('-g', '--git-path', default='.',
                     help='Path to git repository')
 parser.add_argument('-p', '--print-changelog', action='store_true',
                     help='Print final changelog entires')
-parser.add_argument('-n', '--allow-non-strict-mode', action='store_true',
-                    help='Allow non-strict mode (change in both ChangeLog and '
-                    'other files.')
+parser.add_argument('-n', '--non-strict-mode', action='store_true',
+                    help='Use non-strict mode (allow changes in ChangeLog and '
+                    'other automatically updated files).')
 args = parser.parse_args()
 
 retval = 0
 for git_commit in parse_git_revisions(args.git_path, args.revisions,
-                                      not args.allow_non_strict_mode):
+                                      not args.non_strict_mode):
     res = 'OK' if git_commit.success else 'FAILED'
     print('Checking %s: %s' % (git_commit.hexsha, res))
     if git_commit.success:
-- 
2.26.2

Re: ChangeLog files - server and client scripts

[Rainer Orth]

Hi Martin,

> We've just installed server git hooks that verify git messages
> for a correct ChangeLog format. For a limited time period, please
> still apply ChangeLog changes to the corresponding ChangeLog files.
> We'll use it for comparison of auto-generated CangeLog entries.
>
> The format is documented here:
> https://gcc.gnu.org/codingconventions.html#ChangeLogs

two comments:

* Can you please avoid the use grey highlighting in that section?  Black
  script on a grey background is already hard to read for someone with
  reasonable vision.  I suspect it will be much harder for
  vision-impaired people.

* In changelog_location, you allow only (among others) "a/b/c/" and
  "\ta/b/c/".  Please also accept the "a/b/c:" and "\ta/b/c:" forms
  here: especially the second seems quite common.

Thanks.
        Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: ChangeLog files - server and client scripts

[Martin Liška]

On 5/21/20 5:14 PM, Rainer Orth wrote:
> Hi Martin,
> 
>> We've just installed server git hooks that verify git messages
>> for a correct ChangeLog format. For a limited time period, please
>> still apply ChangeLog changes to the corresponding ChangeLog files.
>> We'll use it for comparison of auto-generated CangeLog entries.
>>
>> The format is documented here:
>> https://gcc.gnu.org/codingconventions.html#ChangeLogs
> 
> two comments:
> 
> * Can you please avoid the use grey highlighting in that section?  Black
>    script on a grey background is already hard to read for someone with
>    reasonable vision.  I suspect it will be much harder for
>    vision-impaired people.

You are right, I fixed that.

> 
> * In changelog_location, you allow only (among others) "a/b/c/" and
>    "\ta/b/c/".  Please also accept the "a/b/c:" and "\ta/b/c:" forms
>    here: especially the second seems quite common.

Sure, can you please link some git revisions that use the format?

Thanks,
Martin

> 
> Thanks.
>          Rainer
> 

Re: ChangeLog files - server and client scripts

[Rainer Orth]

Hi Martin,

>> two comments:
>>
>> * Can you please avoid the use grey highlighting in that section?  Black
>>    script on a grey background is already hard to read for someone with
>>    reasonable vision.  I suspect it will be much harder for
>>    vision-impaired people.
>
> You are right, I fixed that.

thanks.

>> * In changelog_location, you allow only (among others) "a/b/c/" and
>>    "\ta/b/c/".  Please also accept the "a/b/c:" and "\ta/b/c:" forms
>>    here: especially the second seems quite common.
>
> Sure, can you please link some git revisions that use the format?

Apart from using it myself ;-), I've seen it several times, but no
examples off-hand.  However, it seems only consistent with the forms
ending in /ChangeLog where you allow the trailing colon, too.

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: ChangeLog files - server and client scripts

[Jason Merrill]

Was there a decision somewhere to require ChangeLog entries for all
testcase changes now, as the hook is enforcing?  They were optional before.

remote: *** ChangeLog format failed:
remote: ERR: changed file not mentioned in a
ChangeLog:"gcc/testsuite/g++.dg/parse/error33.C"

On Thu, May 21, 2020 at 11:38 AM Rainer Orth <ro@cebitec.uni-bielefeld.de>
wrote:

> Hi Martin,
>
> >> two comments:
> >>
> >> * Can you please avoid the use grey highlighting in that section?  Black
> >>    script on a grey background is already hard to read for someone with
> >>    reasonable vision.  I suspect it will be much harder for
> >>    vision-impaired people.
> >
> > You are right, I fixed that.
>
> thanks.
>
> >> * In changelog_location, you allow only (among others) "a/b/c/" and
> >>    "\ta/b/c/".  Please also accept the "a/b/c:" and "\ta/b/c:" forms
> >>    here: especially the second seems quite common.
> >
> > Sure, can you please link some git revisions that use the format?
>
> Apart from using it myself ;-), I've seen it several times, but no
> examples off-hand.  However, it seems only consistent with the forms
> ending in /ChangeLog where you allow the trailing colon, too.
>
>         Rainer
>
> --
>
> -----------------------------------------------------------------------------
> Rainer Orth, Center for Biotechnology, Bielefeld University
>
>

Re: ChangeLog files - server and client scripts

[Jakub Jelinek]

On Thu, May 21, 2020 at 02:52:37PM -0400, Jason Merrill wrote:
> Was there a decision somewhere to require ChangeLog entries for all
> testcase changes now, as the hook is enforcing?  They were optional before.
> 
> remote: *** ChangeLog format failed:
> remote: ERR: changed file not mentioned in a
> ChangeLog:"gcc/testsuite/g++.dg/parse/error33.C"

They are optional for newly added testcases (even other new files); for that
it will add : New test. or something similar, Martin knows the details.

But right now they are not optional if an existing file is modified, because
in that case no script can (easily) find out what to say about it,
: Some change.
would be too useless/fuzzy.

	Jakub

Re: ChangeLog files - server and client scripts

[Martin Liška]

On 5/21/20 8:52 PM, Jason Merrill wrote:
> Was there a decision somewhere to require ChangeLog entries for all testcase changes now, as the hook is enforcing?  They were optional before.

Right now we ignore newly added test-case, these don't have to be mentioned.
Can you please attach the patch (git format-patch)?

Are you talking about modified or delete test-cases? If so, we can definitely
relax the rules..

Martin

Re: ChangeLog files - server and client scripts

[Jason Merrill]

[-- Attachment #1: Type: text/plain, Size: 631 bytes --]

On Thu, May 21, 2020 at 2:58 PM Martin Liška <mliska@suse.cz> wrote:

> On 5/21/20 8:52 PM, Jason Merrill wrote:
> > Was there a decision somewhere to require ChangeLog entries for all
> testcase changes now, as the hook is enforcing?  They were optional before.
>
> Right now we ignore newly added test-case, these don't have to be
> mentioned.
> Can you please attach the patch (git format-patch)?
>
> Are you talking about modified or delete test-cases? If so, we can
> definitely
> relax the rules..
>

Modified.  Adjustments to expected errors in testcases don't seem to me
worth documenting in a ChangeLog.

[-- Attachment #2: 0001-c-Improve-error-recovery-for-parms.patch --]
[-- Type: text/x-patch, Size: 2562 bytes --]

From c5a15303c57171b5b284b7a8ec0eb97f991779d6 Mon Sep 17 00:00:00 2001
From: Jason Merrill <jason@redhat.com>
Date: Thu, 21 May 2020 10:27:11 -0400
Subject: [PATCH 1/4] c++: Improve error-recovery for parms.
To: gcc-patches@gcc.gnu.org

If a parameter is erroneous, we currently drop it, leading to "too many
arguments" errors later.  Treating the function as (...) avoids those
errors.

gcc/cp/ChangeLog
2020-05-21  Jason Merrill  <jason@redhat.com>

	* decl.c (grokparms): Return NULL_TREE if any parms were erroneous.
---
 gcc/cp/decl.c                        | 18 +++++++++++++-----
 gcc/testsuite/g++.dg/parse/error33.C |  4 ++--
 2 files changed, 15 insertions(+), 7 deletions(-)

diff --git a/gcc/cp/decl.c b/gcc/cp/decl.c
index 024ddc88a4c..a389579ee52 100644
--- a/gcc/cp/decl.c
+++ b/gcc/cp/decl.c
@@ -13961,7 +13961,10 @@ grokparms (tree parmlist, tree *parms)
 	break;
 
       if (! decl || TREE_TYPE (decl) == error_mark_node)
-	continue;
+	{
+	  any_error = 1;
+	  continue;
+	}
 
       type = TREE_TYPE (decl);
       if (VOID_TYPE_P (type))
@@ -14014,7 +14017,8 @@ grokparms (tree parmlist, tree *parms)
 	      TREE_TYPE (decl) = type;
 	    }
 	  else if (abstract_virtuals_error (decl, type))
-	    any_error = 1;  /* Seems like a good idea.  */
+	    /* Ignore any default argument.  */
+	    init = NULL_TREE;
 	  else if (cxx_dialect < cxx17 && INDIRECT_TYPE_P (type))
 	    {
 	      /* Before C++17 DR 393:
@@ -14043,9 +14047,7 @@ grokparms (tree parmlist, tree *parms)
 			 decl, t);
 	    }
 
-	  if (any_error)
-	    init = NULL_TREE;
-	  else if (init && !processing_template_decl)
+	  if (init && !processing_template_decl)
 	    init = check_default_argument (decl, init, tf_warning_or_error);
 	}
 
@@ -14058,6 +14060,12 @@ grokparms (tree parmlist, tree *parms)
   if (parm)
     result = chainon (result, void_list_node);
   *parms = decls;
+  if (any_error)
+    result = NULL_TREE;
+
+  if (any_error)
+    /* We had parm errors, recover by giving the function (...) type.  */
+    result = NULL_TREE;
 
   return result;
 }
diff --git a/gcc/testsuite/g++.dg/parse/error33.C b/gcc/testsuite/g++.dg/parse/error33.C
index 0d25386a879..61b0cc3f2dc 100644
--- a/gcc/testsuite/g++.dg/parse/error33.C
+++ b/gcc/testsuite/g++.dg/parse/error33.C
@@ -8,9 +8,9 @@ struct A
 
 typedef void (A::T)(); /* { dg-error "15:typedef name may not be a nested" } */
 
-void bar(T); /* { dg-message "note: declared here" } */
+void bar(T);
 
 void baz()
 {
-  bar(&A::foo); /* { dg-error "too many arguments" } */
+  bar(&A::foo);
 }
-- 
2.18.1

Re: ChangeLog files - server and client scripts

[Martin Liška]

On 5/21/20 9:51 PM, Jason Merrill wrote:
> Modified.  Adjustments to expected errors in testcases don't seem to me worth documenting in a ChangeLog.

I see. As Jakub mentioned, I would keep the hook stricter for now.

Martin

Re: ChangeLog files - server and client scripts

[Jason Merrill]

On Thu, May 21, 2020 at 4:27 PM Martin Liška <mliska@suse.cz> wrote:

> On 5/21/20 9:51 PM, Jason Merrill wrote:
> > Modified.  Adjustments to expected errors in testcases don't seem to me
> worth documenting in a ChangeLog.
>
> I see. As Jakub mentioned, I would keep the hook stricter for now.
>

Why?  What is the use of requiring ChangeLog entries at all for these
changes?

Jason

Re: ChangeLog files - server and client scripts

[Martin Liška]

On 5/21/20 11:01 PM, Jason Merrill wrote:
> Why?  What is the use of requiring ChangeLog entries at all for these changes?

I must confirm a common test-suite ChangeLog entry is something like:

$ grep ':' gcc/testsuite/ChangeLog | sed 's/.*://' | sort | uniq -c | sort -n | tac | head -n 15
    6309  Likewise.
    1306  New test.
     231  New.
      68  New testcase.
      55
      54  Ditto.
      47  New file.
      33  Same.
      27  Move into ...
      24  New tests.
      21  Add typedef for int32_t.
      19  This.  Clean up
      17  Rename to...
      16  This.
      12  ...this.

$ grep ':' gcc/testsuite/ChangeLog-2019 | sed 's/.*://' | sort | uniq -c | sort -n | tac | head -n 15
    2981  Likewise.
    2287  New test.
     399  New testcase.
     263  Same.
     263  New.
     217  Ditto.
     111  Adjust.
      88  New file.
      62
      32  Require exceptions.
      30  Remove.
      25  New
      20  Update test.
      17  New tests.
      15  Add

$ grep ':' gcc/testsuite/ChangeLog-2018 | sed 's/.*://' | sort | uniq -c | sort -n | tac | head -n 15
    5071  Likewise.
    2071  New test.
     443  New.
     413  New testcase.
     395  Remove.
     256  Ditto.
     224  Same.
     108  New file.
      66  Likwise.
      65  Adjust.
      53
      52  Dito.
      39  Add -flinker-output=nolto-rel.
      32  New test case.
      26  Delete testcase.

So I'm open for relaxation of the rule.
What about the others?

Martin

Re: ChangeLog files - server and client scripts

[Martin Liška]

On 5/21/20 5:14 PM, Rainer Orth wrote:
> * In changelog_location, you allow only (among others) "a/b/c/" and
>    "\ta/b/c/".  Please also accept the "a/b/c:" and "\ta/b/c:" forms
>    here: especially the second seems quite common.

Ok, I believe these formats are supported as well. Feel free to mention
some git revisions that are not recognized.

Thanks,
Martin

Re: ChangeLog files - server and client scripts

[Alexandre Oliva]

On May 25, 2020, Martin Liška <mliska@suse.cz> wrote:

> On 5/21/20 5:14 PM, Rainer Orth wrote:
>> * In changelog_location, you allow only (among others) "a/b/c/" and
>> "\ta/b/c/".  Please also accept the "a/b/c:" and "\ta/b/c:" forms
>> here: especially the second seems quite common.

> Ok, I believe these formats are supported as well. Feel free to mention
> some git revisions that are not recognized.

I've long used the following syntax to start ChangeLog entries:

for  <some/dir>/ChangeLog

It was introduced over 20 years ago, with the (so far never formally
released) GNU CVS-Utilities.  Among other goodies, there were scripts to
turn diffs for ChangeLog files into the above format, and vice-versa,
that I've used to this day.  It went through cvs, svn and git.  It would
be quite nice if I could keep on using it with GCC.

The patch below seems to be enough to pass gcc-verify, and to recognize
and print the expected ChangeLog files.  I suppose I'll have to adjust
the formatting to be able to push it, but, aside from that, is it ok to
install?

Do any hooks need to be adjusted to match?


I'm also a little concerned about '*/ChangeLog.*' files.  Are we no
longer supposed to introduce them, or new ChangeLog entries to them?  Or
should the scripts be extended to cover them?


for  contrib/ChangeLog

	* gcc-changelog/git_commit.py (changelog_regex): Accept optional
	'for' prefix.

diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
index 2cfdbc8..b8362c1 100755
--- a/contrib/gcc-changelog/git_commit.py
+++ b/contrib/gcc-changelog/git_commit.py
@@ -144,7 +144,7 @@ misc_files = [
 author_line_regex = \
         re.compile(r'^(?P<datetime>\d{4}-\d{2}-\d{2})\ {2}(?P<name>.*  <.*>)')
 additional_author_regex = re.compile(r'^\t(?P<spaces>\ *)?(?P<name>.*  <.*>)')
-changelog_regex = re.compile(r'^([a-z0-9+-/]*)/ChangeLog:?')
+changelog_regex = re.compile(r'^(?:[fF]or +)([a-z0-9+-/]*)/ChangeLog:?')
 pr_regex = re.compile(r'\tPR (?P<component>[a-z+-]+\/)?([0-9]+)$')
 dr_regex = re.compile(r'\tDR ([0-9]+)$')
 star_prefix_regex = re.compile(r'\t\*(?P<spaces>\ *)(?P<content>.*)')



-- 
Alexandre Oliva, freedom fighter    he/him    https://FSFLA.org/blogs/lxo/
Free Software Evangelist              Stallman was right, but he's left :(
GNU Toolchain Engineer           Live long and free, and prosper ethically

Re: ChangeLog files - server and client scripts

[Martin Liška]

On 5/26/20 7:31 AM, Alexandre Oliva wrote:
> On May 25, 2020, Martin Liška <mliska@suse.cz> wrote:
> 
>> On 5/21/20 5:14 PM, Rainer Orth wrote:
>>> * In changelog_location, you allow only (among others) "a/b/c/" and
>>> "\ta/b/c/".  Please also accept the "a/b/c:" and "\ta/b/c:" forms
>>> here: especially the second seems quite common.
> 
>> Ok, I believe these formats are supported as well. Feel free to mention
>> some git revisions that are not recognized.

Hello.

> 
> I've long used the following syntax to start ChangeLog entries:
> 
> for  <some/dir>/ChangeLog

Ah, it's new for me.

> 
> It was introduced over 20 years ago, with the (so far never formally
> released) GNU CVS-Utilities.  Among other goodies, there were scripts to
> turn diffs for ChangeLog files into the above format, and vice-versa,
> that I've used to this day.  It went through cvs, svn and git.  It would
> be quite nice if I could keep on using it with GCC.

Sure. Starting from now, you don't need to put ChangeLog entries to their
corresponding files, it will be done automatically.

> 
> The patch below seems to be enough to pass gcc-verify, and to recognize
> and print the expected ChangeLog files.  I suppose I'll have to adjust
> the formatting to be able to push it, but, aside from that, is it ok to
> install?

I'm fine with the patch. Alternative approach is to start using
./contrib/mklog.py (a.k.a. git mklog).

> 
> Do any hooks need to be adjusted to match?

Yes, we sync the script from the GCC repository.

> 
> 
> I'm also a little concerned about '*/ChangeLog.*' files.  Are we no
> longer supposed to introduce them, or new ChangeLog entries to them?  Or
> should the scripts be extended to cover them?

Right now we cover only ChangeLog files, so e.g. ChangeLog.dataflow is not affected
(and checked). For newly added ChangeLog files, we can add them, but they must first
appear in the git_commit.py script where we list all allowed locations.

Martin

> 
> 
> for  contrib/ChangeLog
> 
> 	* gcc-changelog/git_commit.py (changelog_regex): Accept optional
> 	'for' prefix.
> 
> diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
> index 2cfdbc8..b8362c1 100755
> --- a/contrib/gcc-changelog/git_commit.py
> +++ b/contrib/gcc-changelog/git_commit.py
> @@ -144,7 +144,7 @@ misc_files = [
>   author_line_regex = \
>           re.compile(r'^(?P<datetime>\d{4}-\d{2}-\d{2})\ {2}(?P<name>.*  <.*>)')
>   additional_author_regex = re.compile(r'^\t(?P<spaces>\ *)?(?P<name>.*  <.*>)')
> -changelog_regex = re.compile(r'^([a-z0-9+-/]*)/ChangeLog:?')
> +changelog_regex = re.compile(r'^(?:[fF]or +)([a-z0-9+-/]*)/ChangeLog:?')
>   pr_regex = re.compile(r'\tPR (?P<component>[a-z+-]+\/)?([0-9]+)$')
>   dr_regex = re.compile(r'\tDR ([0-9]+)$')
>   star_prefix_regex = re.compile(r'\t\*(?P<spaces>\ *)(?P<content>.*)')
> 
> 
> 

Re: ChangeLog files - server and client scripts

[Alexandre Oliva]

Hi, Martin,

On May 26, 2020, Martin Liška <mliska@suse.cz> wrote:

>> I've long used the following syntax to start ChangeLog entries:
>> 
>> for  <some/dir>/ChangeLog

> Ah, it's new for me.

>> 
>> It was introduced over 20 years ago, with the (so far never formally
>> released) GNU CVS-Utilities.  Among other goodies, there were scripts to
>> turn diffs for ChangeLog files into the above format, and vice-versa,
>> that I've used to this day.  It went through cvs, svn and git.  It would
>> be quite nice if I could keep on using it with GCC.             ^^

For clarity, I meant the syntax in the last sentence above.  The
ChangeLog-related functionality in the scripts now becomes mostly
obsolete.

>> The patch below seems to be enough to pass gcc-verify, and to recognize
>> and print the expected ChangeLog files.

'cept it broke cases without 'for' because I missed a '?' in the
regexp.  Good thing I had to adjust for the old format to be able to
push it ;-)  2x0 ;-)

>> Do any hooks need to be adjusted to match?

> Yes, we sync the script from the GCC repository.

Here's what I'm about to push


accept for dir/ChangeLog entries

From: Alexandre Oliva <oliva@gnu.org>

I've long introduced ChangeLog entries as "for  dir/ChangeLog", a
format adopted by GNU CVS-Utilities some 20 years ago.  My commits
have been formatted like this forever.

This patch makes it acceptable for git gcc-verify.


contrib/ChangeLog:

	* gcc-changelog/git_commit.py (changelog_regex): Accept optional
	'for' prefix.
---
 contrib/gcc-changelog/git_commit.py |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
index 2cfdbc8..732a9bd8 100755
--- a/contrib/gcc-changelog/git_commit.py
+++ b/contrib/gcc-changelog/git_commit.py
@@ -144,7 +144,7 @@ misc_files = [
 author_line_regex = \
         re.compile(r'^(?P<datetime>\d{4}-\d{2}-\d{2})\ {2}(?P<name>.*  <.*>)')
 additional_author_regex = re.compile(r'^\t(?P<spaces>\ *)?(?P<name>.*  <.*>)')
-changelog_regex = re.compile(r'^([a-z0-9+-/]*)/ChangeLog:?')
+changelog_regex = re.compile(r'^(?:[fF]or +)?([a-z0-9+-/]*)/ChangeLog:?')
 pr_regex = re.compile(r'\tPR (?P<component>[a-z+-]+\/)?([0-9]+)$')
 dr_regex = re.compile(r'\tDR ([0-9]+)$')
 star_prefix_regex = re.compile(r'\t\*(?P<spaces>\ *)(?P<content>.*)')


-- 
Alexandre Oliva, freedom fighter    he/him    https://FSFLA.org/blogs/lxo/
Free Software Evangelist              Stallman was right, but he's left :(
GNU Toolchain Engineer           Live long and free, and prosper ethically

Re: ChangeLog files - server and client scripts

[Ian Lance Taylor]

On Tue, May 19, 2020 at 2:26 AM Martin Liška <mliska@suse.cz> wrote:
>
> We've just installed server git hooks that verify git messages
> for a correct ChangeLog format. For a limited time period, please
> still apply ChangeLog changes to the corresponding ChangeLog files.
> We'll use it for comparison of auto-generated CangeLog entries.
>
> The format is documented here:
> https://gcc.gnu.org/codingconventions.html#ChangeLogs
>
> And I would recommend to install the new 'git gcc-verify' hook from:
> contrib/gcc-git-customization.sh
>
> Feel free to contact me about future troubles you'll see.

Hi, this unfortunately breaks gccgo development.  Significant parts of
the gccgo sources are simply copied from other repositories.  Those
other repositories do not use ChangeLog files.  The git commit hook
should not require ChangeLog files for those changes.  And, when the
time comes, no ChangeLog files should be created for changes in those
directories.

The directories in question are

gcc/go/gofrontend
libgo
gcc/testsuite/go.test/test

This is as documented in the README.gcc files in those directories.

How can I exempt those directories from the server hook?  Right now,
as far as I can tell, I cannot commit Go changes.  I get

remote: *** ChangeLog format failed:
remote: ERR: cannot find a ChangeLog location in message
remote:
remote: Please see: https://gcc.gnu.org/codingconventions.html#ChangeLogs
remote:
remote: error: hook declined to update refs/heads/master
To git+ssh://gcc.gnu.org/git/gcc
 ! [remote rejected]         master -> master (hook declined)
error: failed to push some refs to 'git+ssh://gcc.gnu.org/git/gcc'

Thanks.

Ian

Re: ChangeLog files - server and client scripts

[Jakub Jelinek]

On Thu, May 21, 2020 at 03:12:21PM -0700, Ian Lance Taylor via Gcc wrote:
> Hi, this unfortunately breaks gccgo development.  Significant parts of
> the gccgo sources are simply copied from other repositories.  Those
> other repositories do not use ChangeLog files.  The git commit hook
> should not require ChangeLog files for those changes.  And, when the
> time comes, no ChangeLog files should be created for changes in those
> directories.
> 
> The directories in question are
> 
> gcc/go/gofrontend
> libgo
> gcc/testsuite/go.test/test

The script has:
ignored_prefixes = [
    'gcc/d/dmd/',
    'gcc/go/frontend/',
    'libgo/',
    'libphobos/libdruntime',
    'libphobos/src/',
    'libsanitizer/',
    ]
so perhaps it just misses gcc/testsuite/go.test/test ?
Or what exact files you've changed in your script?

	Jakub

Re: ChangeLog files - server and client scripts

[Martin Liška]

On 5/22/20 6:57 AM, Jakub Jelinek wrote:
> so perhaps it just misses gcc/testsuite/go.test/test ?

Hello.

I've just added the location to ignored locations.

> Or what exact files you've changed in your script?

@Ian: Please send us patch with git format-patch.
@Jakub: Can you please sync up the script to the server hooks? I'll
be AFK till Monday.

Martin

Re: ChangeLog files - server and client scripts

[Richard Earnshaw]

On 22/05/2020 05:57, Jakub Jelinek wrote:
> On Thu, May 21, 2020 at 03:12:21PM -0700, Ian Lance Taylor via Gcc wrote:
>> Hi, this unfortunately breaks gccgo development.  Significant parts of
>> the gccgo sources are simply copied from other repositories.  Those
>> other repositories do not use ChangeLog files.  The git commit hook
>> should not require ChangeLog files for those changes.  And, when the
>> time comes, no ChangeLog files should be created for changes in those
>> directories.
>>
>> The directories in question are
>>
>> gcc/go/gofrontend
>> libgo
>> gcc/testsuite/go.test/test
> 
> The script has:
> ignored_prefixes = [
>     'gcc/d/dmd/',
>     'gcc/go/frontend/',

The directory is gcc/go/gofrontend

so it's missing 'go' from frontend.

>     'libgo/',
>     'libphobos/libdruntime',
>     'libphobos/src/',
>     'libsanitizer/',
>     ]
> so perhaps it just misses gcc/testsuite/go.test/test ?
> Or what exact files you've changed in your script?
> 
> 	Jakub
> 

R.

Re: ChangeLog files - server and client scripts

[Jakub Jelinek]

On Fri, May 22, 2020 at 12:04:10PM +0100, Richard Earnshaw wrote:
> >> The directories in question are
> >>
> >> gcc/go/gofrontend
> >> libgo
> >> gcc/testsuite/go.test/test
> > 
> > The script has:
> > ignored_prefixes = [
> >     'gcc/d/dmd/',
> >     'gcc/go/frontend/',
> 
> The directory is gcc/go/gofrontend
> 
> so it's missing 'go' from frontend.

Thanks for spotting.  I believe Martin said he will be afk
today, so I've fixed it for him and committed as obvious and
am going to install into git-hooks now too.

diff --git a/contrib/ChangeLog b/contrib/ChangeLog
index 7b61bb8915b..64a0db18e58 100644
--- a/contrib/ChangeLog
+++ b/contrib/ChangeLog
@@ -1,3 +1,9 @@
+2020-05-22  Jakub Jelinek  <jakub@redhat.com>
+
+	* gcc-changelog/git_commit.py: Add trailing / to
+	gcc/testsuite/go.test/test and replace gcc/go/frontend/
+	with gcc/go/gofrontend/ in ignored locations.
+
 2020-05-22  Martin Liska  <mliska@suse.cz>
 
 	* gcc-changelog/git_commit.py: Add gcc/testsuite/go.test/test
diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
index ba9f5ce9650..8c5fa2c0fc9 100755
--- a/contrib/gcc-changelog/git_commit.py
+++ b/contrib/gcc-changelog/git_commit.py
@@ -127,8 +127,8 @@ bug_components = set([
 
 ignored_prefixes = [
     'gcc/d/dmd/',
-    'gcc/go/frontend/',
-    'gcc/testsuite/go.test/test',
+    'gcc/go/gofrontend/',
+    'gcc/testsuite/go.test/test/',
     'libgo/',
     'libphobos/libdruntime',
     'libphobos/src/',

	Jakub

Re: ChangeLog files - server and client scripts

[Ian Lance Taylor]

[-- Attachment #1: Type: text/plain, Size: 2250 bytes --]

On Fri, May 22, 2020 at 4:11 AM Jakub Jelinek <jakub@redhat.com> wrote:
>
> On Fri, May 22, 2020 at 12:04:10PM +0100, Richard Earnshaw wrote:
> > >> The directories in question are
> > >>
> > >> gcc/go/gofrontend
> > >> libgo
> > >> gcc/testsuite/go.test/test
> > >
> > > The script has:
> > > ignored_prefixes = [
> > >     'gcc/d/dmd/',
> > >     'gcc/go/frontend/',
> >
> > The directory is gcc/go/gofrontend
> >
> > so it's missing 'go' from frontend.
>
> Thanks for spotting.  I believe Martin said he will be afk
> today, so I've fixed it for him and committed as obvious and
> am going to install into git-hooks now too.
>
> diff --git a/contrib/ChangeLog b/contrib/ChangeLog
> index 7b61bb8915b..64a0db18e58 100644
> --- a/contrib/ChangeLog
> +++ b/contrib/ChangeLog
> @@ -1,3 +1,9 @@
> +2020-05-22  Jakub Jelinek  <jakub@redhat.com>
> +
> +       * gcc-changelog/git_commit.py: Add trailing / to
> +       gcc/testsuite/go.test/test and replace gcc/go/frontend/
> +       with gcc/go/gofrontend/ in ignored locations.
> +
>  2020-05-22  Martin Liska  <mliska@suse.cz>
>
>         * gcc-changelog/git_commit.py: Add gcc/testsuite/go.test/test
> diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
> index ba9f5ce9650..8c5fa2c0fc9 100755
> --- a/contrib/gcc-changelog/git_commit.py
> +++ b/contrib/gcc-changelog/git_commit.py
> @@ -127,8 +127,8 @@ bug_components = set([
>
>  ignored_prefixes = [
>      'gcc/d/dmd/',
> -    'gcc/go/frontend/',
> -    'gcc/testsuite/go.test/test',
> +    'gcc/go/gofrontend/',
> +    'gcc/testsuite/go.test/test/',
>      'libgo/',
>      'libphobos/libdruntime',
>      'libphobos/src/',

Thanks for looking into this.

Unfortunately, my push is still failing.  I'm not sure why.

remote: *** ChangeLog format failed:
remote: ERR: cannot find a ChangeLog location in message
remote:
remote: Please see: https://gcc.gnu.org/codingconventions.html#ChangeLogs
remote:
remote: error: hook declined to update refs/heads/master
To git+ssh://gcc.gnu.org/git/gcc
 ! [remote rejected]         master -> master (hook declined)
error: failed to push some refs to 'git+ssh://gcc.gnu.org/git/gcc'


I've attached the output of "git format-patch -k 1 --stdout", in case
that helps.

Ian

[-- Attachment #2: x.txt --]
[-- Type: text/plain, Size: 2664 bytes --]

From 81994eab700da7fea6644541c163aa0f0f3b8cf1 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Cl=C3=A9ment=20Chigot?= <chigot.c@gmail.com>
Date: Tue, 19 May 2020 16:03:54 +0200
Subject: libgo: update x/sys/cpu after gccgo support added

Reviewed-on: https://go-review.googlesource.com/c/gofrontend/+/234597
---
 gcc/go/gofrontend/MERGE                       |  2 +-
 .../sys/cpu/{cpu_aix_ppc64.go => cpu_aix.go}  |  2 +-
 .../golang.org/x/sys/cpu/syscall_aix_gccgo.go | 27 +++++++++++++++++++
 3 files changed, 29 insertions(+), 2 deletions(-)
 rename libgo/go/golang.org/x/sys/cpu/{cpu_aix_ppc64.go => cpu_aix.go} (96%)
 create mode 100644 libgo/go/golang.org/x/sys/cpu/syscall_aix_gccgo.go

diff --git a/gcc/go/gofrontend/MERGE b/gcc/go/gofrontend/MERGE
index bc9c1f07eda..284374820b0 100644
--- a/gcc/go/gofrontend/MERGE
+++ b/gcc/go/gofrontend/MERGE
@@ -1,4 +1,4 @@
-bc27341f245a5cc54ac7530d037a609db72b677c
+ea58b8491064fbed18a220571a3043c38dccf7c7
 
 The first line of this file holds the git revision number of the last
 merge done from the gofrontend repository.
diff --git a/libgo/go/golang.org/x/sys/cpu/cpu_aix_ppc64.go b/libgo/go/golang.org/x/sys/cpu/cpu_aix.go
similarity index 96%
rename from libgo/go/golang.org/x/sys/cpu/cpu_aix_ppc64.go
rename to libgo/go/golang.org/x/sys/cpu/cpu_aix.go
index b0ede112d4e..02d03129e50 100644
--- a/libgo/go/golang.org/x/sys/cpu/cpu_aix_ppc64.go
+++ b/libgo/go/golang.org/x/sys/cpu/cpu_aix.go
@@ -2,7 +2,7 @@
 // Use of this source code is governed by a BSD-style
 // license that can be found in the LICENSE file.
 
-// +build aix,ppc64
+// +build aix
 
 package cpu
 
diff --git a/libgo/go/golang.org/x/sys/cpu/syscall_aix_gccgo.go b/libgo/go/golang.org/x/sys/cpu/syscall_aix_gccgo.go
new file mode 100644
index 00000000000..2609cc49ae7
--- /dev/null
+++ b/libgo/go/golang.org/x/sys/cpu/syscall_aix_gccgo.go
@@ -0,0 +1,27 @@
+// Copyright 2020 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+// Recreate a getsystemcfg syscall handler instead of
+// using the one provided by x/sys/unix to avoid having
+// the dependency between them. (See golang.org/issue/32102)
+// Morover, this file will be used during the building of
+// gccgo's libgo and thus must not use a CGo method.
+
+// +build aix
+// +build gccgo
+
+package cpu
+
+import (
+	"syscall"
+)
+
+//extern getsystemcfg
+func gccgoGetsystemcfg(label uint32) (r uint64)
+
+func callgetsystemcfg(label int) (r1 uintptr, e1 syscall.Errno) {
+	r1 = uintptr(gccgoGetsystemcfg(uint32(label)))
+	e1 = syscall.GetErrno()
+	return
+}
-- 
2.27.0.rc0.183.gde8f92d652-goog

Re: ChangeLog files - server and client scripts

[Jakub Jelinek]

On Fri, May 22, 2020 at 12:37:29PM -0700, Ian Lance Taylor wrote:
> Thanks for looking into this.
> 
> Unfortunately, my push is still failing.  I'm not sure why.
> 
> remote: *** ChangeLog format failed:
> remote: ERR: cannot find a ChangeLog location in message
> remote:
> remote: Please see: https://gcc.gnu.org/codingconventions.html#ChangeLogs
> remote:
> remote: error: hook declined to update refs/heads/master
> To git+ssh://gcc.gnu.org/git/gcc
>  ! [remote rejected]         master -> master (hook declined)
> error: failed to push some refs to 'git+ssh://gcc.gnu.org/git/gcc'
> 
> 
> I've attached the output of "git format-patch -k 1 --stdout", in case
> that helps.

Bet the script first looks for the ChangeLog entry and only considers the
ignored prefixes if it finds files in the patch that are not mentioned in
the ChangeLog entry.  So, if you say modified gcc/go/whatever.cc and had
ChangeLog entry for that and not for the files you've changed, it would be
ok.

So, I think before emitting the above message, it should look through the
patch and if it finds all files in ignored prefixes, it should just not to
do anything.

We'll need it e.g. for the DATESTAMP bump job too which only modifies those
files too and doesn't write ChangeLog entry for that.

I'm sorry but I don't know the script well enough to fix it quickly, will
defer to Martin as the author.

Can you wait with the commit until Monday?  If not, I could just temporarily
disable this for your commit.

	Jakub

Re: ChangeLog files - server and client scripts

[Ian Lance Taylor]

On Fri, May 22, 2020 at 12:48 PM Jakub Jelinek <jakub@redhat.com> wrote:
>
> On Fri, May 22, 2020 at 12:37:29PM -0700, Ian Lance Taylor wrote:
> > Thanks for looking into this.
> >
> > Unfortunately, my push is still failing.  I'm not sure why.
> >
> > remote: *** ChangeLog format failed:
> > remote: ERR: cannot find a ChangeLog location in message
> > remote:
> > remote: Please see: https://gcc.gnu.org/codingconventions.html#ChangeLogs
> > remote:
> > remote: error: hook declined to update refs/heads/master
> > To git+ssh://gcc.gnu.org/git/gcc
> >  ! [remote rejected]         master -> master (hook declined)
> > error: failed to push some refs to 'git+ssh://gcc.gnu.org/git/gcc'
> >
> >
> > I've attached the output of "git format-patch -k 1 --stdout", in case
> > that helps.
>
> Bet the script first looks for the ChangeLog entry and only considers the
> ignored prefixes if it finds files in the patch that are not mentioned in
> the ChangeLog entry.  So, if you say modified gcc/go/whatever.cc and had
> ChangeLog entry for that and not for the files you've changed, it would be
> ok.
>
> So, I think before emitting the above message, it should look through the
> patch and if it finds all files in ignored prefixes, it should just not to
> do anything.
>
> We'll need it e.g. for the DATESTAMP bump job too which only modifies those
> files too and doesn't write ChangeLog entry for that.
>
> I'm sorry but I don't know the script well enough to fix it quickly, will
> defer to Martin as the author.
>
> Can you wait with the commit until Monday?  If not, I could just temporarily
> disable this for your commit.

Sure, I can wait.  Thanks for looking at it.

Ian

Re: ChangeLog files - server and client scripts

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 191 bytes --]

On 5/23/20 12:14 AM, Ian Lance Taylor wrote:
> Sure, I can wait.  Thanks for looking at it.

Hello.

Thank you for patience. There's a patch that fixes that,
I'm going to install it.

Martin

[-- Attachment #2: 0001-Allow-only-ignored-files-in-ChangeLog-entries.patch --]
[-- Type: text/x-patch, Size: 4168 bytes --]

From 76e18b91250f265a37d85063860fb38aa8f6aac3 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Mon, 25 May 2020 09:40:50 +0200
Subject: [PATCH] Allow only ignored files in ChangeLog entries.

contrib/ChangeLog:

2020-05-25  Martin Liska  <mliska@suse.cz>

	* gcc-changelog/git_commit.py: Add trailing '/'
	for libdruntime.  Allow empty changelog for
	only ignored files.
	* gcc-changelog/test_email.py: New test for go
	patch in ignored location.
	* gcc-changelog/test_patches.txt: Add test.
---
 contrib/gcc-changelog/git_commit.py    |  5 +--
 contrib/gcc-changelog/test_email.py    |  4 +++
 contrib/gcc-changelog/test_patches.txt | 43 ++++++++++++++++++++++++++
 3 files changed, 50 insertions(+), 2 deletions(-)

diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
index 8c5fa2c0fc9..2cfdbc83d09 100755
--- a/contrib/gcc-changelog/git_commit.py
+++ b/contrib/gcc-changelog/git_commit.py
@@ -130,7 +130,7 @@ ignored_prefixes = [
     'gcc/go/gofrontend/',
     'gcc/testsuite/go.test/test/',
     'libgo/',
-    'libphobos/libdruntime',
+    'libphobos/libdruntime/',
     'libphobos/src/',
     'libsanitizer/',
     ]
@@ -233,7 +233,8 @@ class GitCommit:
 
         project_files = [f for f in self.modified_files
                          if self.is_changelog_filename(f[0])
-                         or f[0] in misc_files]
+                         or f[0] in misc_files
+                         or self.in_ignored_location(f[0])]
         if len(project_files) == len(self.modified_files):
             # All modified files are only MISC files
             return
diff --git a/contrib/gcc-changelog/test_email.py b/contrib/gcc-changelog/test_email.py
index d522e6ef7e3..aa516c6e6d1 100755
--- a/contrib/gcc-changelog/test_email.py
+++ b/contrib/gcc-changelog/test_email.py
@@ -276,3 +276,7 @@ class TestGccChangelog(unittest.TestCase):
     def test_dr_entry(self):
         email = self.from_patch_glob('0001-c-C-20-DR-2237.patch')
         assert email.changelog_entries[0].prs == ['DR 2237']
+
+    def test_changes_only_in_ignored_location(self):
+        email = self.from_patch_glob('0001-go-in-ignored-location.patch')
+        assert not email.errors
diff --git a/contrib/gcc-changelog/test_patches.txt b/contrib/gcc-changelog/test_patches.txt
index 3445c3d9f11..58fd81c85c9 100644
--- a/contrib/gcc-changelog/test_patches.txt
+++ b/contrib/gcc-changelog/test_patches.txt
@@ -2568,3 +2568,46 @@ index a6a5d975af3..a8082d39aca 100644
 @@ -1 +1,2 @@
 
 +
+
+=== 0001-go-in-ignored-location.patch ===
+From 81994eab700da7fea6644541c163aa0f0f3b8cf1 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Cl=C3=A9ment=20Chigot?= <chigot.c@gmail.com>
+Date: Tue, 19 May 2020 16:03:54 +0200
+Subject: libgo: update x/sys/cpu after gccgo support added
+
+Reviewed-on: https://go-review.googlesource.com/c/gofrontend/+/234597
+---
+ gcc/go/gofrontend/MERGE                       |  2 +-
+ .../sys/cpu/{cpu_aix_ppc64.go => cpu_aix.go}  |  2 +-
+ .../golang.org/x/sys/cpu/syscall_aix_gccgo.go | 27 +++++++++++++++++++
+ 3 files changed, 29 insertions(+), 2 deletions(-)
+ rename libgo/go/golang.org/x/sys/cpu/{cpu_aix_ppc64.go => cpu_aix.go} (96%)
+ create mode 100644 libgo/go/golang.org/x/sys/cpu/syscall_aix_gccgo.go
+
+diff --git a/gcc/go/gofrontend/MERGE b/gcc/go/gofrontend/MERGE
+index bc9c1f07eda..284374820b0 100644
+--- a/gcc/go/gofrontend/MERGE
++++ b/gcc/go/gofrontend/MERGE
+@@ -1 +1,2 @@
+
++
+diff --git a/libgo/go/golang.org/x/sys/cpu/cpu_aix_ppc64.go b/libgo/go/golang.org/x/sys/cpu/cpu_aix.go
+similarity index 96%
+rename from libgo/go/golang.org/x/sys/cpu/cpu_aix_ppc64.go
+rename to libgo/go/golang.org/x/sys/cpu/cpu_aix.go
+index b0ede112d4e..02d03129e50 100644
+--- a/libgo/go/golang.org/x/sys/cpu/cpu_aix_ppc64.go
++++ b/libgo/go/golang.org/x/sys/cpu/cpu_aix.go
+@@ -1 +1,2 @@
+
++
+diff --git a/libgo/go/golang.org/x/sys/cpu/syscall_aix_gccgo.go b/libgo/go/golang.org/x/sys/cpu/syscall_aix_gccgo.go
+new file mode 100644
+index 00000000000..2609cc49ae7
+--- /dev/null
++++ b/libgo/go/golang.org/x/sys/cpu/syscall_aix_gccgo.go
+@@ -0,0 +1 @@
++
+
+-- 
+2.27.0.rc0.183.gde8f92d652-goog
-- 
2.26.2

Re: ChangeLog files - server and client scripts

[Ian Lance Taylor]

On Mon, May 25, 2020 at 12:48 AM Martin Liška <mliska@suse.cz> wrote:
>
> On 5/23/20 12:14 AM, Ian Lance Taylor wrote:
> > Sure, I can wait.  Thanks for looking at it.
>
> Hello.
>
> Thank you for patience. There's a patch that fixes that,
> I'm going to install it.

Thanks.  I was able to push my change to master.

Ian

git-hooks: integrate ChangeLog format check

[Martin Liška]

On 5/13/20 7:53 PM, Joseph Myers wrote:
>   See the issues
> I filed athttps://github.com/AdaCore/git-hooks/issues  for the existing
> custom GCC changes and the pull request
> https://github.com/AdaCore/git-hooks/pull/12  to bring in implementations
> of many of those features (not sure if it covers everything or not).

Hey Joseph.

If I see correctly, the pull request was merged.
Are we now in a position where we can separate the GCC-specific
ChangeLog check in gcc-hooks? If so, can you please guide me
a bit?

Thanks,
Martin

New mklog script

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 822 bytes --]

Hi.

Since we moved to git world and we're in the preparation for ChangeLog messages
being in git commit messages, I think it's the right time to also simplify mklog
script.

I'm sending a new version (which should eventually replace contrib/mklog and contrib/mklog.pl).
Changes made in the version:

- the script uses unifdiff - it rapidly simplifies parsing of the '+-!' lines that is done
   in contrib/mklog
- no author nor date stamp is used - that all can be get from git
- --inline option is not supported - I don't see a use-case for it now
- the new script has a unit tests (just few of them for now)

I compares results in between the old Python script for last 80 commits and it's very close,
in some cases it does even better.

I'm planning to maintain and improve the script for the future.

Thoughts?
Martin

[-- Attachment #2: 0001-Add-mklog-ng.py-and-gcc-mklog-git-alias.patch --]
[-- Type: text/x-patch, Size: 15530 bytes --]

From 9fa5d13856f0f5ba153801baf57d4a732829f609 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Fri, 15 May 2020 00:44:07 +0200
Subject: [PATCH] Add mklog-ng.py and gcc-mklog git alias.

contrib/ChangeLog:

	* gcc-git-customization.sh: Add gcc-mklog alias.
	* mklog_ng.py: New file.
	* test_mklog_ng.py: New file.
---
 contrib/gcc-git-customization.sh |   2 +
 contrib/mklog_ng.py              | 192 +++++++++++++++++++++++++++++++
 contrib/test_mklog_ng.py         | 158 +++++++++++++++++++++++++
 3 files changed, 352 insertions(+)
 create mode 100755 contrib/mklog_ng.py
 create mode 100755 contrib/test_mklog_ng.py

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index a932bf8c06a..b7b97327be3 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -25,6 +25,8 @@ git config alias.svn-rev '!f() { rev=$1; shift; git log --all --grep="^From-SVN:
 git config alias.gcc-descr \!"f() { if test \${1:-no} = --full; then c=\${2:-master}; r=\$(git describe --all --abbrev=40 --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-,r,p'); expr match \${r:-no} '^r[0-9]\\+\$' >/dev/null && r=\${r}-0-g\$(git rev-parse \${2:-master}); else c=\${1:-master}; r=\$(git describe --all --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)-\\([0-9]\\+\\)-g[0-9a-f]*\$,r\\2-\\3,p;s,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)\$,r\\2-0,p'); fi; if test -n \$r; then o=\$(git config --get gcc-config.upstream); rr=\$(echo \$r | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\\(-g[0-9a-f]\\+\\)\\?\$,\\1,p'); if git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$rr >/dev/null; then m=releases/gcc-\$rr; else m=master; fi; git merge-base --is-ancestor \$c \${o:-origin}/\$m && \echo \${r}; fi; }; f"
 git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream); r=\$(echo \$1 | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\$,\\1,p'); n=\$(echo \$1 | sed -n 's,^r[0-9]\\+-\\([0-9]\\+\\)\$,\\1,p'); test -z \$r && echo Invalid id \$1 && exit 1; h=\$(git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$r); test -z \$h && h=\$(git rev-parse --verify --quiet \${o:-origin}/master); p=\$(git describe --all --match 'basepoints/gcc-'\$r \$h | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+-\\([0-9]\\+\\)-g[0-9a-f]*\$,\\2,p;s,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+\$,0,p'); git rev-parse --verify \$h~\$(expr \$p - \$n); }; f"
 
+git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog_ng.py" $@; } ; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
diff --git a/contrib/mklog_ng.py b/contrib/mklog_ng.py
new file mode 100755
index 00000000000..a67fc007759
--- /dev/null
+++ b/contrib/mklog_ng.py
@@ -0,0 +1,192 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import argparse
+import os
+import re
+import sys
+
+from unidiff import PatchSet
+
+pr_regex = re.compile(r'(\/(\/|\*)|[Cc*!])\s+(?P<pr>PR [a-z+-]+\/[0-9]+)')
+identifier_regex = re.compile(r'^([a-zA-Z0-9_#].*)')
+comment_regex = re.compile(r'^\/\*')
+struct_regex = re.compile(r'^((class|struct|union|enum)\s+[a-zA-Z0-9_]+)')
+macro_regex = re.compile(r'#\s*(define|undef)\s+([a-zA-Z0-9_]+)')
+super_macro_regex = re.compile(r'^DEF[A-Z0-9_]+\s*\(([a-zA-Z0-9_]+)')
+fn_regex = re.compile(r'([a-zA-Z_][^()\s]*)\s*\([^*]')
+template_and_param_regex = re.compile(r'<[^<>]*>')
+
+function_extensions = set(['.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def'])
+
+help_message = """\
+Generate ChangeLog template for PATCH.
+PATCH must be generated using diff(1)'s -up or -cp options
+(or their equivalent in git).
+"""
+
+script_folder = os.path.realpath(__file__)
+gcc_root = os.path.dirname(os.path.dirname(script_folder))
+
+
+def find_changelog(path):
+    folder = os.path.split(path)[0]
+    while True:
+        if os.path.exists(os.path.join(gcc_root, folder, 'ChangeLog')):
+            return folder
+        folder = os.path.dirname(folder)
+        if folder == '':
+            return folder
+    raise AssertionError()
+
+
+def extract_function_name(line):
+    if comment_regex.match(line):
+        return None
+    m = struct_regex.search(line)
+    if m:
+        # Struct declaration
+        return m.group(1)
+    m = macro_regex.search(line)
+    if m:
+        # Macro definition
+        return m.group(2)
+    m = super_macro_regex.search(line)
+    if m:
+        # Supermacro
+        return m.group(1)
+    m = fn_regex.search(line)
+    if m:
+        # Discard template and function parameters.
+        fn = m.group(1)
+        fn = re.sub(template_and_param_regex, '', fn)
+        return fn.rstrip()
+    return None
+
+
+def try_add_function(functions, line):
+    fn = extract_function_name(line)
+    if fn and fn not in functions:
+        functions.append(fn)
+    return bool(fn)
+
+
+def generate_changelog(data, no_functions=False):
+    changelogs = {}
+    sorted_changelogs = []
+    prs = []
+    out = ''
+    diff = PatchSet(data)
+
+    for file in diff:
+        changelog = find_changelog(file.path)
+        if changelog not in changelogs:
+            changelogs[changelog] = []
+            sorted_changelogs.append(changelog)
+        changelogs[changelog].append(file)
+
+        if 'testsuite' in file.path and file.is_added_file:
+            for line in list(file)[0]:
+                m = pr_regex.search(line.value)
+                if m:
+                    pr = m.group('pr')
+                    if pr not in prs:
+                        prs.append(pr)
+                else:
+                    break
+
+    for changelog in sorted_changelogs:
+        files = changelogs[changelog]
+        out += '%s:\n' % os.path.join(changelog, 'ChangeLog')
+        out += '\n'
+        for pr in prs:
+            out += '\t%s\n' % pr
+        for file in files:
+            assert file.path.startswith(changelog)
+            in_tests = 'testsuite' in changelog or 'testsuite' in file.path
+            relative_path = file.path[len(changelog):].lstrip('/')
+            functions = []
+            if file.is_added_file:
+                msg = 'New test' if in_tests else 'New file'
+                out += '\t* %s: %s.\n' % (relative_path, msg)
+            elif file.is_removed_file:
+                out += '\t* %s: Removed.\n' % (relative_path)
+            else:
+                if not no_functions:
+                    for hunk in file:
+                        # Do not add function names for testsuite files
+                        extension = os.path.splitext(relative_path)[1]
+                        if not in_tests and extension in function_extensions:
+                            last_fn = None
+                            modified_visited = False
+                            success = False
+                            for line in hunk:
+                                m = identifier_regex.match(line.value)
+                                if line.is_added or line.is_removed:
+                                    if not line.value.strip():
+                                        continue
+                                    modified_visited = True
+                                    if m and try_add_function(functions,
+                                                              m.group(1)):
+                                        last_fn = None
+                                        success = True
+                                elif line.is_context:
+                                    if last_fn and modified_visited:
+                                        try_add_function(functions, last_fn)
+                                        last_fn = None
+                                        modified_visited = False
+                                        success = True
+                                    elif m:
+                                        last_fn = m.group(1)
+                                        modified_visited = False
+                            if not success:
+                                try_add_function(functions,
+                                                 hunk.section_header)
+                if functions:
+                    out += '\t* %s (%s):\n' % (relative_path, functions[0])
+                    for fn in functions[1:]:
+                        out += '\t(%s):\n' % fn
+                else:
+                    out += '\t* %s:\n' % relative_path
+        out += '\n'
+    return out
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description=help_message)
+    parser.add_argument('input', nargs='?',
+                        help='Patch file (or missing, read standard input)')
+    parser.add_argument('-s', '--no-functions', action='store_true',
+                        help='Do not generate function names in ChangeLogs')
+    args = parser.parse_args()
+    if args.input == '-':
+        args.input = None
+
+    input = open(args.input) if args.input else sys.stdin
+    output = generate_changelog(input.read(), args.no_functions)
+    print(output, end='')
diff --git a/contrib/test_mklog_ng.py b/contrib/test_mklog_ng.py
new file mode 100755
index 00000000000..6e68277b1bb
--- /dev/null
+++ b/contrib/test_mklog_ng.py
@@ -0,0 +1,158 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import unittest
+
+from mklog_ng import generate_changelog
+
+PATCH1 = '''\
+diff --git a/gcc/config/riscv/riscv.h b/gcc/config/riscv/riscv.h
+index 567c23380fe..e6209ede9d6 100644
+--- a/gcc/config/riscv/riscv.h
++++ b/gcc/config/riscv/riscv.h
+@@ -920,6 +920,7 @@ extern unsigned riscv_stack_boundary;
+ #define SHIFT_RS1 15
+ #define SHIFT_IMM 20
+ #define IMM_BITS 12
++#define C_S_BITS 5
+ #define C_SxSP_BITS 6
+ 
+ #define IMM_REACH (1LL << IMM_BITS)
+@@ -929,6 +930,10 @@ extern unsigned riscv_stack_boundary;
+ #define SWSP_REACH (4LL << C_SxSP_BITS)
+ #define SDSP_REACH (8LL << C_SxSP_BITS)
+ 
++/* This is the maximum value that can be represented in a compressed load/store
++   offset (an unsigned 5-bit value scaled by 4).  */
++#define CSW_MAX_OFFSET ((4LL << C_S_BITS) - 1) & ~3
++
+ /* Called from RISCV_REORG, this is defined in riscv-sr.c.  */
+ 
+ extern void riscv_remove_unneeded_save_restore_calls (void);
+
+'''
+
+EXPECTED1 = '''\
+gcc/ChangeLog:
+
+	* config/riscv/riscv.h (C_S_BITS):
+	(CSW_MAX_OFFSET):
+
+'''
+
+PATCH2 = '''\
+diff --git a/gcc/targhooks.h b/gcc/targhooks.h
+index 9704d23f1db..b572a36e8cf 100644
+--- a/gcc/targhooks.h
++++ b/gcc/targhooks.h
+@@ -120,7 +120,7 @@ extern bool default_empty_mask_is_expensive (unsigned);
+ extern void *default_init_cost (class loop *);
+ extern unsigned default_add_stmt_cost (class vec_info *, void *, int,
+ 				       enum vect_cost_for_stmt,
+-				       class _stmt_vec_info *, int,
++				       class _stmt_vec_info *, tree, int,
+ 				       enum vect_cost_model_location);
+ extern void default_finish_cost (void *, unsigned *, unsigned *, unsigned *);
+ extern void default_destroy_cost_data (void *);
+@@ -186,6 +186,7 @@ extern tree default_emutls_var_init (tree, tree, tree);
+ extern unsigned int default_hard_regno_nregs (unsigned int, machine_mode);
+ extern bool default_hard_regno_scratch_ok (unsigned int);
+ extern bool default_mode_dependent_address_p (const_rtx, addr_space_t);
++extern bool default_new_address_profitable_p (rtx, rtx_insn *, rtx);
+ extern bool default_target_option_valid_attribute_p (tree, tree, tree, int);
+ extern bool default_target_option_pragma_parse (tree, tree);
+ extern bool default_target_can_inline_p (tree, tree);
+
+'''
+
+EXPECTED2 = '''\
+gcc/ChangeLog:
+
+	* targhooks.h (default_add_stmt_cost):
+	(default_new_address_profitable_p):
+
+'''
+
+PATCH3 = '''\
+diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
+index 2b1e33f94ae..7f47402f9b9 100644
+--- a/libcpp/include/cpplib.h
++++ b/libcpp/include/cpplib.h
+@@ -173,7 +173,7 @@ enum c_lang {CLK_GNUC89 = 0, CLK_GNUC99, CLK_GNUC11, CLK_GNUC17, CLK_GNUC2X,
+ 	     CLK_STDC2X,
+ 	     CLK_GNUCXX, CLK_CXX98, CLK_GNUCXX11, CLK_CXX11,
+ 	     CLK_GNUCXX14, CLK_CXX14, CLK_GNUCXX17, CLK_CXX17,
+-	     CLK_GNUCXX2A, CLK_CXX2A, CLK_ASM};
++	     CLK_GNUCXX20, CLK_CXX20, CLK_ASM};
+ 
+ /* Payload of a NUMBER, STRING, CHAR or COMMENT token.  */
+ struct GTY(()) cpp_string {
+@@ -484,7 +484,7 @@ struct cpp_options
+   /* Nonzero for C2X decimal floating-point constants.  */
+   unsigned char dfp_constants;
+ 
+-  /* Nonzero for C++2a __VA_OPT__ feature.  */
++  /* Nonzero for C++20 __VA_OPT__ feature.  */
+   unsigned char va_opt;
+ 
+   /* Nonzero for the '::' token.  */
+
+'''
+
+EXPECTED3 = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h (enum c_lang):
+	(struct cpp_options):
+
+'''
+
+EXPECTED3B = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h:
+
+'''
+
+
+class TestMklog(unittest.TestCase):
+    def test_macro_definition(self):
+        changelog = generate_changelog(PATCH1)
+        assert changelog == EXPECTED1
+
+    def test_changed_argument(self):
+        changelog = generate_changelog(PATCH2)
+        assert changelog == EXPECTED2
+
+    def test_enum_and_struct(self):
+        changelog = generate_changelog(PATCH3)
+        assert changelog == EXPECTED3
+
+    def test_no_function(self):
+        changelog = generate_changelog(PATCH3, True)
+        assert changelog == EXPECTED3B
-- 
2.26.2

Re: New mklog script

[David Malcolm]

On Fri, 2020-05-15 at 10:59 +0200, Martin Liška wrote:
> Hi.
> 
> Since we moved to git world and we're in the preparation for
> ChangeLog messages
> being in git commit messages, I think it's the right time to also
> simplify mklog
> script.
> 
> I'm sending a new version (which should eventually replace
> contrib/mklog and contrib/mklog.pl).
> Changes made in the version:
> 
> - the script uses unifdiff - it rapidly simplifies parsing of the '+-
> !' lines that is done
>    in contrib/mklog
> - no author nor date stamp is used - that all can be get from git
> - --inline option is not supported - I don't see a use-case for it
> now
> - the new script has a unit tests (just few of them for now)
> 
> I compares results in between the old Python script for last 80
> commits and it's very close,
> in some cases it does even better.
> 
> I'm planning to maintain and improve the script for the future.
> 
> Thoughts?
> Martin

> +class TestMklog(unittest.TestCase):
> +    def test_macro_definition(self):
> +        changelog = generate_changelog(PATCH1)
> +        assert changelog == EXPECTED1
> +
> +    def test_changed_argument(self):
> +        changelog = generate_changelog(PATCH2)
> +        assert changelog == EXPECTED2
> +
> +    def test_enum_and_struct(self):
> +        changelog = generate_changelog(PATCH3)
> +        assert changelog == EXPECTED3
> +
> +    def test_no_function(self):
> +        changelog = generate_changelog(PATCH3, True)
> +        assert changelog == EXPECTED3B

Use self.assertEqual(a, b) rather than assert a == b, so that if it
fails you get a multiline diff:

e.g.:

import unittest

class TestMklog(unittest.TestCase):
    def test_macro_definition(self):
        self.assertEqual('''
first
second
third''', '''
first
SECOND
third''')

unittest.main()


has this output:

F
======================================================================
FAIL: test_macro_definition (__main__.TestMklog)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/foo.py", line 11, in test_macro_definition
    third''')
AssertionError: '\nfirst\nsecond\nthird' != '\nfirst\nSECOND\nthird'
  
  first
- second
+ SECOND
  third

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (failures=1)

which is much easier to debug than the output from assert a == b, which
is just:

F
======================================================================
FAIL: test_macro_definition (__main__.TestMklog)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/foo.py", line 11, in test_macro_definition
    third''')
AssertionError

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (failures=1)

Re: New mklog script

[Martin Liška]

On 5/15/20 12:58 PM, David Malcolm wrote:
> On Fri, 2020-05-15 at 10:59 +0200, Martin Liška wrote:
>> Hi.
>>
>> Since we moved to git world and we're in the preparation for
>> ChangeLog messages
>> being in git commit messages, I think it's the right time to also
>> simplify mklog
>> script.
>>
>> I'm sending a new version (which should eventually replace
>> contrib/mklog and contrib/mklog.pl).
>> Changes made in the version:
>>
>> - the script uses unifdiff - it rapidly simplifies parsing of the '+-
>> !' lines that is done
>>     in contrib/mklog
>> - no author nor date stamp is used - that all can be get from git
>> - --inline option is not supported - I don't see a use-case for it
>> now
>> - the new script has a unit tests (just few of them for now)
>>
>> I compares results in between the old Python script for last 80
>> commits and it's very close,
>> in some cases it does even better.
>>
>> I'm planning to maintain and improve the script for the future.
>>
>> Thoughts?
>> Martin
> 
>> +class TestMklog(unittest.TestCase):
>> +    def test_macro_definition(self):
>> +        changelog = generate_changelog(PATCH1)
>> +        assert changelog == EXPECTED1
>> +
>> +    def test_changed_argument(self):
>> +        changelog = generate_changelog(PATCH2)
>> +        assert changelog == EXPECTED2
>> +
>> +    def test_enum_and_struct(self):
>> +        changelog = generate_changelog(PATCH3)
>> +        assert changelog == EXPECTED3
>> +
>> +    def test_no_function(self):
>> +        changelog = generate_changelog(PATCH3, True)
>> +        assert changelog == EXPECTED3B

Thank you David for review.

However I see the same output for both operator== and assertEqual. Probably
because of usage of pytest version 4?

assertEqual:

$ pytest contrib/test_mklog_ng.py
Test session starts (platform: linux, Python 3.8.2, pytest 4.6.9, pytest-sugar 0.9.3)
benchmark: 3.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/marxin/Programming/gcc
plugins: xdist-1.32.0, sugar-0.9.3, forked-1.1.3, benchmark-3.2.3, aspectlib-1.5.0, cov-2.8.1, flake8-1.0.5
collecting ...
  contrib/test_mklog_ng.py ✓                                                                                                                                                                                                                                       25% ██▌

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestMklog.test_enum_and_struct ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_mklog_ng.TestMklog testMethod=test_enum_and_struct>

     def test_enum_and_struct(self):
         changelog = generate_changelog(PATCH3)
>       self.assertEqual(changelog, EXPECTED3)
E       AssertionError: 'libc[23 chars]clude/cpplib.h (enum c_lang):\n\t(struct cpp_options):\n\n' != 'libc[23 chars]clude/cppli22b.h (enum c_lang):\n\t(struct cpp_optio44ns):\n\n'
E         libcpp/ChangeLog:
E
E       - 	* include/cpplib.h (enum c_lang):
E       + 	* include/cppli22b.h (enum c_lang):
E       ? 	               ++
E       - 	(struct cpp_options):
E       + 	(struct cpp_optio44ns):
E       ? 	                 ++

contrib/test_mklog_ng.py:154: AssertionError

operator==:

pytest contrib/test_mklog_ng.py
Test session starts (platform: linux, Python 3.8.2, pytest 4.6.9, pytest-sugar 0.9.3)
benchmark: 3.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/marxin/Programming/gcc
plugins: xdist-1.32.0, sugar-0.9.3, forked-1.1.3, benchmark-3.2.3, aspectlib-1.5.0, cov-2.8.1, flake8-1.0.5
collecting ...
  contrib/test_mklog_ng.py ✓                                                                                                                                                                                                                                       25% ██▌

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestMklog.test_enum_and_struct ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_mklog_ng.TestMklog testMethod=test_enum_and_struct>

     def test_enum_and_struct(self):
         changelog = generate_changelog(PATCH3)
>       assert changelog == EXPECTED3
E       AssertionError: assert 'libcpp/Chang...options):\n\n' == 'libcpp/Change...tio44ns):\n\n'
E           libcpp/ChangeLog:
E
E         - 	* include/cpplib.h (enum c_lang):
E         + 	* include/cppli22b.h (enum c_lang):
E         ? 	               ++
E         - 	(struct cpp_options):
E         + 	(struct cpp_optio44ns):...
E
E         ...Full output truncated (3 lines hidden), use '-vv' to show

Martin

> 
> Use self.assertEqual(a, b) rather than assert a == b, so that if it
> fails you get a multiline diff:
> 
> e.g.:
> 
> import unittest
> 
> class TestMklog(unittest.TestCase):
>      def test_macro_definition(self):
>          self.assertEqual('''
> first
> second
> third''', '''
> first
> SECOND
> third''')
> 
> unittest.main()
> 
> 
> has this output:
> 
> F
> ======================================================================
> FAIL: test_macro_definition (__main__.TestMklog)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>    File "/tmp/foo.py", line 11, in test_macro_definition
>      third''')
> AssertionError: '\nfirst\nsecond\nthird' != '\nfirst\nSECOND\nthird'
>    
>    first
> - second
> + SECOND
>    third
> 
> ----------------------------------------------------------------------
> Ran 1 test in 0.000s
> 
> FAILED (failures=1)
> 
> which is much easier to debug than the output from assert a == b, which
> is just:
> 
> F
> ======================================================================
> FAIL: test_macro_definition (__main__.TestMklog)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>    File "/tmp/foo.py", line 11, in test_macro_definition
>      third''')
> AssertionError
> 
> ----------------------------------------------------------------------
> Ran 1 test in 0.000s
> 
> FAILED (failures=1)
> 

Re: New mklog script

[David Malcolm]

On Fri, 2020-05-15 at 13:20 +0200, Martin Liška wrote:
> On 5/15/20 12:58 PM, David Malcolm wrote:
> > On Fri, 2020-05-15 at 10:59 +0200, Martin Liška wrote:
> > > Hi.
> > > 
> > > Since we moved to git world and we're in the preparation for
> > > ChangeLog messages
> > > being in git commit messages, I think it's the right time to also
> > > simplify mklog
> > > script.
> > > 
> > > I'm sending a new version (which should eventually replace
> > > contrib/mklog and contrib/mklog.pl).
> > > Changes made in the version:
> > > 
> > > - the script uses unifdiff - it rapidly simplifies parsing of the
> > > '+-
> > > !' lines that is done
> > >     in contrib/mklog
> > > - no author nor date stamp is used - that all can be get from git
> > > - --inline option is not supported - I don't see a use-case for
> > > it
> > > now
> > > - the new script has a unit tests (just few of them for now)
> > > 
> > > I compares results in between the old Python script for last 80
> > > commits and it's very close,
> > > in some cases it does even better.
> > > 
> > > I'm planning to maintain and improve the script for the future.
> > > 
> > > Thoughts?
> > > Martin
> > > +class TestMklog(unittest.TestCase):
> > > +    def test_macro_definition(self):
> > > +        changelog = generate_changelog(PATCH1)
> > > +        assert changelog == EXPECTED1
> > > +
> > > +    def test_changed_argument(self):
> > > +        changelog = generate_changelog(PATCH2)
> > > +        assert changelog == EXPECTED2
> > > +
> > > +    def test_enum_and_struct(self):
> > > +        changelog = generate_changelog(PATCH3)
> > > +        assert changelog == EXPECTED3
> > > +
> > > +    def test_no_function(self):
> > > +        changelog = generate_changelog(PATCH3, True)
> > > +        assert changelog == EXPECTED3B
> 
> Thank you David for review.
> 
> However I see the same output for both operator== and assertEqual.
> Probably
> because of usage of pytest version 4?

Ah, yes.  pytest does "magical" things with frame inspection IIRC to
scrape the locals out of the failing python stack frame.

Dave

Re: New mklog script

[Marek Polacek]

On Fri, May 15, 2020 at 10:59:56AM +0200, Martin Liška wrote:
> Hi.
> 
> Since we moved to git world and we're in the preparation for ChangeLog messages
> being in git commit messages, I think it's the right time to also simplify mklog
> script.
> 
> I'm sending a new version (which should eventually replace contrib/mklog and contrib/mklog.pl).
> Changes made in the version:
> 
> - the script uses unifdiff - it rapidly simplifies parsing of the '+-!' lines that is done
>   in contrib/mklog

Nice!

> - no author nor date stamp is used - that all can be get from git

This is good.

> - --inline option is not supported - I don't see a use-case for it now

I actually use mklog -i all the time.  But I can work around it if it
disappears.

--
Marek Polacek • Red Hat, Inc. • 300 A St, Boston, MA

Re: New mklog script

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 410 bytes --]

On 5/15/20 2:42 PM, Marek Polacek wrote:
> I actually use mklog -i all the time.  But I can work around it if it
> disappears.

Ah, I can see a consumer.
There's an updated version that supports that.

For the future, will you still use the option? Wouldn't be better
to put the ChangeLog content directly to commit message? Note
that you won't have to copy the entries to a particular ChangeLog file.

Martin

[-- Attachment #2: 0001-Add-mklog-ng.py-and-gcc-mklog-git-alias.patch --]
[-- Type: text/x-patch, Size: 16031 bytes --]

From d7d5e3aa7450449a8b0cb30d6bf485538990ea3f Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Fri, 15 May 2020 00:44:07 +0200
Subject: [PATCH] Add mklog-ng.py and gcc-mklog git alias.

contrib/ChangeLog:

	* gcc-git-customization.sh: Add gcc-mklog alias.
	* mklog_ng.py: New file.
	* test_mklog_ng.py: New file.
---
 contrib/gcc-git-customization.sh |   2 +
 contrib/mklog_ng.py              | 209 +++++++++++++++++++++++++++++++
 contrib/test_mklog_ng.py         | 158 +++++++++++++++++++++++
 3 files changed, 369 insertions(+)
 create mode 100755 contrib/mklog_ng.py
 create mode 100755 contrib/test_mklog_ng.py

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index a932bf8c06a..b7b97327be3 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -25,6 +25,8 @@ git config alias.svn-rev '!f() { rev=$1; shift; git log --all --grep="^From-SVN:
 git config alias.gcc-descr \!"f() { if test \${1:-no} = --full; then c=\${2:-master}; r=\$(git describe --all --abbrev=40 --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-,r,p'); expr match \${r:-no} '^r[0-9]\\+\$' >/dev/null && r=\${r}-0-g\$(git rev-parse \${2:-master}); else c=\${1:-master}; r=\$(git describe --all --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)-\\([0-9]\\+\\)-g[0-9a-f]*\$,r\\2-\\3,p;s,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)\$,r\\2-0,p'); fi; if test -n \$r; then o=\$(git config --get gcc-config.upstream); rr=\$(echo \$r | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\\(-g[0-9a-f]\\+\\)\\?\$,\\1,p'); if git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$rr >/dev/null; then m=releases/gcc-\$rr; else m=master; fi; git merge-base --is-ancestor \$c \${o:-origin}/\$m && \echo \${r}; fi; }; f"
 git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream); r=\$(echo \$1 | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\$,\\1,p'); n=\$(echo \$1 | sed -n 's,^r[0-9]\\+-\\([0-9]\\+\\)\$,\\1,p'); test -z \$r && echo Invalid id \$1 && exit 1; h=\$(git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$r); test -z \$h && h=\$(git rev-parse --verify --quiet \${o:-origin}/master); p=\$(git describe --all --match 'basepoints/gcc-'\$r \$h | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+-\\([0-9]\\+\\)-g[0-9a-f]*\$,\\2,p;s,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+\$,0,p'); git rev-parse --verify \$h~\$(expr \$p - \$n); }; f"
 
+git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog_ng.py" $@; } ; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
diff --git a/contrib/mklog_ng.py b/contrib/mklog_ng.py
new file mode 100755
index 00000000000..8dca6dbeef0
--- /dev/null
+++ b/contrib/mklog_ng.py
@@ -0,0 +1,209 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import argparse
+import os
+import re
+import sys
+import tempfile
+
+from unidiff import PatchSet
+
+pr_regex = re.compile(r'(\/(\/|\*)|[Cc*!])\s+(?P<pr>PR [a-z+-]+\/[0-9]+)')
+identifier_regex = re.compile(r'^([a-zA-Z0-9_#].*)')
+comment_regex = re.compile(r'^\/\*')
+struct_regex = re.compile(r'^((class|struct|union|enum)\s+[a-zA-Z0-9_]+)')
+macro_regex = re.compile(r'#\s*(define|undef)\s+([a-zA-Z0-9_]+)')
+super_macro_regex = re.compile(r'^DEF[A-Z0-9_]+\s*\(([a-zA-Z0-9_]+)')
+fn_regex = re.compile(r'([a-zA-Z_][^()\s]*)\s*\([^*]')
+template_and_param_regex = re.compile(r'<[^<>]*>')
+
+function_extensions = set(['.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def'])
+
+help_message = """\
+Generate ChangeLog template for PATCH.
+PATCH must be generated using diff(1)'s -up or -cp options
+(or their equivalent in git).
+"""
+
+inline_message = """\
+Prepends ChangeLog to PATCH.
+If PATCH is not stdin, modifies PATCH in-place,
+otherwise writes to stdout.'
+"""
+
+script_folder = os.path.realpath(__file__)
+gcc_root = os.path.dirname(os.path.dirname(script_folder))
+
+
+def find_changelog(path):
+    folder = os.path.split(path)[0]
+    while True:
+        if os.path.exists(os.path.join(gcc_root, folder, 'ChangeLog')):
+            return folder
+        folder = os.path.dirname(folder)
+        if folder == '':
+            return folder
+    raise AssertionError()
+
+
+def extract_function_name(line):
+    if comment_regex.match(line):
+        return None
+    m = struct_regex.search(line)
+    if m:
+        # Struct declaration
+        return m.group(1)
+    m = macro_regex.search(line)
+    if m:
+        # Macro definition
+        return m.group(2)
+    m = super_macro_regex.search(line)
+    if m:
+        # Supermacro
+        return m.group(1)
+    m = fn_regex.search(line)
+    if m:
+        # Discard template and function parameters.
+        fn = m.group(1)
+        fn = re.sub(template_and_param_regex, '', fn)
+        return fn.rstrip()
+    return None
+
+
+def try_add_function(functions, line):
+    fn = extract_function_name(line)
+    if fn and fn not in functions:
+        functions.append(fn)
+    return bool(fn)
+
+
+def generate_changelog(data, no_functions=False):
+    changelogs = {}
+    sorted_changelogs = []
+    prs = []
+    out = ''
+    diff = PatchSet(data)
+
+    for file in diff:
+        changelog = find_changelog(file.path)
+        if changelog not in changelogs:
+            changelogs[changelog] = []
+            sorted_changelogs.append(changelog)
+        changelogs[changelog].append(file)
+
+        if 'testsuite' in file.path and file.is_added_file:
+            for line in list(file)[0]:
+                m = pr_regex.search(line.value)
+                if m:
+                    pr = m.group('pr')
+                    if pr not in prs:
+                        prs.append(pr)
+                else:
+                    break
+
+    for changelog in sorted_changelogs:
+        files = changelogs[changelog]
+        out += '%s:\n' % os.path.join(changelog, 'ChangeLog')
+        out += '\n'
+        for pr in prs:
+            out += '\t%s\n' % pr
+        for file in files:
+            assert file.path.startswith(changelog)
+            in_tests = 'testsuite' in changelog or 'testsuite' in file.path
+            relative_path = file.path[len(changelog):].lstrip('/')
+            functions = []
+            if file.is_added_file:
+                msg = 'New test' if in_tests else 'New file'
+                out += '\t* %s: %s.\n' % (relative_path, msg)
+            elif file.is_removed_file:
+                out += '\t* %s: Removed.\n' % (relative_path)
+            else:
+                if not no_functions:
+                    for hunk in file:
+                        # Do not add function names for testsuite files
+                        extension = os.path.splitext(relative_path)[1]
+                        if not in_tests and extension in function_extensions:
+                            last_fn = None
+                            modified_visited = False
+                            success = False
+                            for line in hunk:
+                                m = identifier_regex.match(line.value)
+                                if line.is_added or line.is_removed:
+                                    if not line.value.strip():
+                                        continue
+                                    modified_visited = True
+                                    if m and try_add_function(functions,
+                                                              m.group(1)):
+                                        last_fn = None
+                                        success = True
+                                elif line.is_context:
+                                    if last_fn and modified_visited:
+                                        try_add_function(functions, last_fn)
+                                        last_fn = None
+                                        modified_visited = False
+                                        success = True
+                                    elif m:
+                                        last_fn = m.group(1)
+                                        modified_visited = False
+                            if not success:
+                                try_add_function(functions,
+                                                 hunk.section_header)
+                if functions:
+                    out += '\t* %s (%s):\n' % (relative_path, functions[0])
+                    for fn in functions[1:]:
+                        out += '\t(%s):\n' % fn
+                else:
+                    out += '\t* %s:\n' % relative_path
+        out += '\n'
+    return out
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description=help_message)
+    parser.add_argument('input', nargs='?',
+                        help='Patch file (or missing, read standard input)')
+    parser.add_argument('-s', '--no-functions', action='store_true',
+                        help='Do not generate function names in ChangeLogs')
+    parser.add_argument('-i', '--inline', action='store_true',
+                        help=inline_message)
+    args = parser.parse_args()
+    if args.input == '-':
+        args.input = None
+
+    input = open(args.input) if args.input else sys.stdin
+    data = input.read()
+    output = generate_changelog(data, args.no_functions)
+
+    if args.inline and args.input:
+        with tempfile.NamedTemporaryFile(mode='w', delete=False) as f:
+            f.write(output)
+            f.write(data)
+        os.rename(f.name, args.input)
+    else:
+        print(output, end='')
diff --git a/contrib/test_mklog_ng.py b/contrib/test_mklog_ng.py
new file mode 100755
index 00000000000..6e68277b1bb
--- /dev/null
+++ b/contrib/test_mklog_ng.py
@@ -0,0 +1,158 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import unittest
+
+from mklog_ng import generate_changelog
+
+PATCH1 = '''\
+diff --git a/gcc/config/riscv/riscv.h b/gcc/config/riscv/riscv.h
+index 567c23380fe..e6209ede9d6 100644
+--- a/gcc/config/riscv/riscv.h
++++ b/gcc/config/riscv/riscv.h
+@@ -920,6 +920,7 @@ extern unsigned riscv_stack_boundary;
+ #define SHIFT_RS1 15
+ #define SHIFT_IMM 20
+ #define IMM_BITS 12
++#define C_S_BITS 5
+ #define C_SxSP_BITS 6
+ 
+ #define IMM_REACH (1LL << IMM_BITS)
+@@ -929,6 +930,10 @@ extern unsigned riscv_stack_boundary;
+ #define SWSP_REACH (4LL << C_SxSP_BITS)
+ #define SDSP_REACH (8LL << C_SxSP_BITS)
+ 
++/* This is the maximum value that can be represented in a compressed load/store
++   offset (an unsigned 5-bit value scaled by 4).  */
++#define CSW_MAX_OFFSET ((4LL << C_S_BITS) - 1) & ~3
++
+ /* Called from RISCV_REORG, this is defined in riscv-sr.c.  */
+ 
+ extern void riscv_remove_unneeded_save_restore_calls (void);
+
+'''
+
+EXPECTED1 = '''\
+gcc/ChangeLog:
+
+	* config/riscv/riscv.h (C_S_BITS):
+	(CSW_MAX_OFFSET):
+
+'''
+
+PATCH2 = '''\
+diff --git a/gcc/targhooks.h b/gcc/targhooks.h
+index 9704d23f1db..b572a36e8cf 100644
+--- a/gcc/targhooks.h
++++ b/gcc/targhooks.h
+@@ -120,7 +120,7 @@ extern bool default_empty_mask_is_expensive (unsigned);
+ extern void *default_init_cost (class loop *);
+ extern unsigned default_add_stmt_cost (class vec_info *, void *, int,
+ 				       enum vect_cost_for_stmt,
+-				       class _stmt_vec_info *, int,
++				       class _stmt_vec_info *, tree, int,
+ 				       enum vect_cost_model_location);
+ extern void default_finish_cost (void *, unsigned *, unsigned *, unsigned *);
+ extern void default_destroy_cost_data (void *);
+@@ -186,6 +186,7 @@ extern tree default_emutls_var_init (tree, tree, tree);
+ extern unsigned int default_hard_regno_nregs (unsigned int, machine_mode);
+ extern bool default_hard_regno_scratch_ok (unsigned int);
+ extern bool default_mode_dependent_address_p (const_rtx, addr_space_t);
++extern bool default_new_address_profitable_p (rtx, rtx_insn *, rtx);
+ extern bool default_target_option_valid_attribute_p (tree, tree, tree, int);
+ extern bool default_target_option_pragma_parse (tree, tree);
+ extern bool default_target_can_inline_p (tree, tree);
+
+'''
+
+EXPECTED2 = '''\
+gcc/ChangeLog:
+
+	* targhooks.h (default_add_stmt_cost):
+	(default_new_address_profitable_p):
+
+'''
+
+PATCH3 = '''\
+diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
+index 2b1e33f94ae..7f47402f9b9 100644
+--- a/libcpp/include/cpplib.h
++++ b/libcpp/include/cpplib.h
+@@ -173,7 +173,7 @@ enum c_lang {CLK_GNUC89 = 0, CLK_GNUC99, CLK_GNUC11, CLK_GNUC17, CLK_GNUC2X,
+ 	     CLK_STDC2X,
+ 	     CLK_GNUCXX, CLK_CXX98, CLK_GNUCXX11, CLK_CXX11,
+ 	     CLK_GNUCXX14, CLK_CXX14, CLK_GNUCXX17, CLK_CXX17,
+-	     CLK_GNUCXX2A, CLK_CXX2A, CLK_ASM};
++	     CLK_GNUCXX20, CLK_CXX20, CLK_ASM};
+ 
+ /* Payload of a NUMBER, STRING, CHAR or COMMENT token.  */
+ struct GTY(()) cpp_string {
+@@ -484,7 +484,7 @@ struct cpp_options
+   /* Nonzero for C2X decimal floating-point constants.  */
+   unsigned char dfp_constants;
+ 
+-  /* Nonzero for C++2a __VA_OPT__ feature.  */
++  /* Nonzero for C++20 __VA_OPT__ feature.  */
+   unsigned char va_opt;
+ 
+   /* Nonzero for the '::' token.  */
+
+'''
+
+EXPECTED3 = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h (enum c_lang):
+	(struct cpp_options):
+
+'''
+
+EXPECTED3B = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h:
+
+'''
+
+
+class TestMklog(unittest.TestCase):
+    def test_macro_definition(self):
+        changelog = generate_changelog(PATCH1)
+        assert changelog == EXPECTED1
+
+    def test_changed_argument(self):
+        changelog = generate_changelog(PATCH2)
+        assert changelog == EXPECTED2
+
+    def test_enum_and_struct(self):
+        changelog = generate_changelog(PATCH3)
+        assert changelog == EXPECTED3
+
+    def test_no_function(self):
+        changelog = generate_changelog(PATCH3, True)
+        assert changelog == EXPECTED3B
-- 
2.26.2

Re: New mklog script

[Marek Polacek]

On Fri, May 15, 2020 at 03:12:27PM +0200, Martin Liška wrote:
> On 5/15/20 2:42 PM, Marek Polacek wrote:
> > I actually use mklog -i all the time.  But I can work around it if it
> > disappears.
> 
> Ah, I can see a consumer.
> There's an updated version that supports that.
> 
> For the future, will you still use the option? Wouldn't be better
> to put the ChangeLog content directly to commit message? Note
> that you won't have to copy the entries to a particular ChangeLog file.

The way I do it is to generate a patch using format-patch, use mklog -i
on it, then add the ChangeLog entry to the commit message via commit --amend.

Anything that has to do with ChangeLogs is pointless make-work, so the less
I have to do, the better.  ;-)

Marek

Re: New mklog script

[Martin Liška]

On 5/15/20 3:22 PM, Marek Polacek wrote:
> On Fri, May 15, 2020 at 03:12:27PM +0200, Martin Liška wrote:
>> On 5/15/20 2:42 PM, Marek Polacek wrote:
>>> I actually use mklog -i all the time.  But I can work around it if it
>>> disappears.
>>
>> Ah, I can see a consumer.
>> There's an updated version that supports that.
>>
>> For the future, will you still use the option? Wouldn't be better
>> to put the ChangeLog content directly to commit message? Note
>> that you won't have to copy the entries to a particular ChangeLog file.
> 
> The way I do it is to generate a patch using format-patch, use mklog -i
> on it, then add the ChangeLog entry to the commit message via commit --amend.

Hmm, you can do much better with:

$ git diff | ./contrib/mklog > changelog && git commit -a -t changelog

Or for an already created commit you can do:

$ git diff HEAD~ | ./contrib/mklog > changelog && git commit -a --amend -e -F changelog

That said, I believe usage of -i is legacy.

Martin

> 
> Anything that has to do with ChangeLogs is pointless make-work, so the less
> I have to do, the better.  ;-)
> 
> Marek
> 

Re: New mklog script

[Jason Merrill]

On Fri, May 15, 2020 at 11:39 AM Martin Liška <mliska@suse.cz> wrote:
>
> On 5/15/20 3:22 PM, Marek Polacek wrote:
> > On Fri, May 15, 2020 at 03:12:27PM +0200, Martin Liška wrote:
> >> On 5/15/20 2:42 PM, Marek Polacek wrote:
> >>> I actually use mklog -i all the time.  But I can work around it if it
> >>> disappears.
> >>
> >> Ah, I can see a consumer.
> >> There's an updated version that supports that.
> >>
> >> For the future, will you still use the option? Wouldn't be better
> >> to put the ChangeLog content directly to commit message? Note
> >> that you won't have to copy the entries to a particular ChangeLog file.
> >
> > The way I do it is to generate a patch using format-patch, use mklog -i
> > on it, then add the ChangeLog entry to the commit message via commit --amend.
>
> Hmm, you can do much better with:
>
> $ git diff | ./contrib/mklog > changelog && git commit -a -t changelog
>
> Or for an already created commit you can do:
>
> $ git diff HEAD~ | ./contrib/mklog > changelog && git commit -a --amend -e -F changelog

With these git aliases:

        mklog-editor = "!f() { git show | git gcc-mklog >> $1; }; f"
        addlog = "!f() { GIT_EDITOR='git mklog-editor' git commit --amend; }; f"

I can 'git addlog' to append the output of mklog to the current
commit.  Probably better would be to do something with
prepare-commit-msg.

Jason

Re: New mklog script

[Jason Merrill]

On Thu, May 21, 2020 at 6:03 PM Jason Merrill <jason@redhat.com> wrote:
>
> On Fri, May 15, 2020 at 11:39 AM Martin Liška <mliska@suse.cz> wrote:
> >
> > On 5/15/20 3:22 PM, Marek Polacek wrote:
> > > On Fri, May 15, 2020 at 03:12:27PM +0200, Martin Liška wrote:
> > >> On 5/15/20 2:42 PM, Marek Polacek wrote:
> > >>> I actually use mklog -i all the time.  But I can work around it if it
> > >>> disappears.
> > >>
> > >> Ah, I can see a consumer.
> > >> There's an updated version that supports that.
> > >>
> > >> For the future, will you still use the option? Wouldn't be better
> > >> to put the ChangeLog content directly to commit message? Note
> > >> that you won't have to copy the entries to a particular ChangeLog file.
> > >
> > > The way I do it is to generate a patch using format-patch, use mklog -i
> > > on it, then add the ChangeLog entry to the commit message via commit --amend.
> >
> > Hmm, you can do much better with:
> >
> > $ git diff | ./contrib/mklog > changelog && git commit -a -t changelog
> >
> > Or for an already created commit you can do:
> >
> > $ git diff HEAD~ | ./contrib/mklog > changelog && git commit -a --amend -e -F changelog
>
> With these git aliases:
>
>         mklog-editor = "!f() { git show | git gcc-mklog >> $1; }; f"
>         addlog = "!f() { GIT_EDITOR='git mklog-editor' git commit --amend; }; f"
>
> I can 'git addlog' to append the output of mklog to the current
> commit.  Probably better would be to do something with
> prepare-commit-msg.

This is pretty rudimentary, but good enough as a start:

#!/bin/sh

#COMMIT_MSG_FILE=$1
#COMMIT_SOURCE=$2
#SHA1=$3

if ! [ -f "$1" ]; then exit 0; fi

#echo "# $0 $1 $2 $3" >> $1

if fgrep 'ChangeLog:' $1 > /dev/null 2>&1; then exit 0; fi

if [ -z "$2" ]; then
    cmd="diff --cached"
elif [ $2 == commit ]; then
    cmd="show $3"
else
    exit 0
fi

git $cmd | git gcc-mklog >> $1

Re: New mklog script

[Martin Liška]

On 5/22/20 11:01 PM, Jason Merrill wrote:
> On Thu, May 21, 2020 at 6:03 PM Jason Merrill <jason@redhat.com> wrote:
>>
>> On Fri, May 15, 2020 at 11:39 AM Martin Liška <mliska@suse.cz> wrote:
>>>
>>> On 5/15/20 3:22 PM, Marek Polacek wrote:
>>>> On Fri, May 15, 2020 at 03:12:27PM +0200, Martin Liška wrote:
>>>>> On 5/15/20 2:42 PM, Marek Polacek wrote:
>>>>>> I actually use mklog -i all the time.  But I can work around it if it
>>>>>> disappears.
>>>>>
>>>>> Ah, I can see a consumer.
>>>>> There's an updated version that supports that.
>>>>>
>>>>> For the future, will you still use the option? Wouldn't be better
>>>>> to put the ChangeLog content directly to commit message? Note
>>>>> that you won't have to copy the entries to a particular ChangeLog file.
>>>>
>>>> The way I do it is to generate a patch using format-patch, use mklog -i
>>>> on it, then add the ChangeLog entry to the commit message via commit --amend.
>>>
>>> Hmm, you can do much better with:
>>>
>>> $ git diff | ./contrib/mklog > changelog && git commit -a -t changelog
>>>
>>> Or for an already created commit you can do:
>>>
>>> $ git diff HEAD~ | ./contrib/mklog > changelog && git commit -a --amend -e -F changelog
>>
>> With these git aliases:
>>
>>          mklog-editor = "!f() { git show | git gcc-mklog >> $1; }; f"
>>          addlog = "!f() { GIT_EDITOR='git mklog-editor' git commit --amend; }; f"
>>
>> I can 'git addlog' to append the output of mklog to the current
>> commit.  Probably better would be to do something with
>> prepare-commit-msg.
> 
> This is pretty rudimentary, but good enough as a start:

I like the idea of usage of the prepare commit hook.

> 
> #!/bin/sh
> 
> #COMMIT_MSG_FILE=$1
> #COMMIT_SOURCE=$2
> #SHA1=$3

It's better to use the named arguments.

> 
> if ! [ -f "$1" ]; then exit 0; fi
> 
> #echo "# $0 $1 $2 $3" >> $1
> 
> if fgrep 'ChangeLog:' $1 > /dev/null 2>&1; then exit 0; fi
> 
> if [ -z "$2" ]; then
>      cmd="diff --cached"
> elif [ $2 == commit ]; then
>      cmd="show $3"
> else
>      exit 0
> fi
> 
> git $cmd | git gcc-mklog >> $1
> 

Well, that will generate changelog entry for each commit.
For a user branch development, it's not desirable.

What about more explicit approach:

1) making an alias for: git diff | git gcc-mklog > commit.msg
2) hook:

if test -f commit.msg; then
   cat commit.msg >> "$COMMIT_MSG_FILE"
   rm commit.msg
fi

So the changelog is created explicitly and included implicitly.

Martin

Re: New mklog script

[Jason Merrill]

[-- Attachment #1: Type: text/plain, Size: 2915 bytes --]

On Mon, May 25, 2020 at 5:23 AM Martin Liška <mliska@suse.cz> wrote:
>
> On 5/22/20 11:01 PM, Jason Merrill wrote:
> > On Thu, May 21, 2020 at 6:03 PM Jason Merrill <jason@redhat.com> wrote:
> >>
> >> On Fri, May 15, 2020 at 11:39 AM Martin Liška <mliska@suse.cz> wrote:
> >>>
> >>> On 5/15/20 3:22 PM, Marek Polacek wrote:
> >>>> On Fri, May 15, 2020 at 03:12:27PM +0200, Martin Liška wrote:
> >>>>> On 5/15/20 2:42 PM, Marek Polacek wrote:
> >>>>>> I actually use mklog -i all the time.  But I can work around it if it
> >>>>>> disappears.
> >>>>>
> >>>>> Ah, I can see a consumer.
> >>>>> There's an updated version that supports that.
> >>>>>
> >>>>> For the future, will you still use the option? Wouldn't be better
> >>>>> to put the ChangeLog content directly to commit message? Note
> >>>>> that you won't have to copy the entries to a particular ChangeLog file.
> >>>>
> >>>> The way I do it is to generate a patch using format-patch, use mklog -i
> >>>> on it, then add the ChangeLog entry to the commit message via commit --amend.
> >>>
> >>> Hmm, you can do much better with:
> >>>
> >>> $ git diff | ./contrib/mklog > changelog && git commit -a -t changelog
> >>>
> >>> Or for an already created commit you can do:
> >>>
> >>> $ git diff HEAD~ | ./contrib/mklog > changelog && git commit -a --amend -e -F changelog
> >>
> >> With these git aliases:
> >>
> >>          mklog-editor = "!f() { git show | git gcc-mklog >> $1; }; f"
> >>          addlog = "!f() { GIT_EDITOR='git mklog-editor' git commit --amend; }; f"
> >>
> >> I can 'git addlog' to append the output of mklog to the current
> >> commit.  Probably better would be to do something with
> >> prepare-commit-msg.
> >
> > This is pretty rudimentary, but good enough as a start:
>
> I like the idea of usage of the prepare commit hook.
>
> >
> > #!/bin/sh
> >
> > #COMMIT_MSG_FILE=$1
> > #COMMIT_SOURCE=$2
> > #SHA1=$3
>
> It's better to use the named arguments.
>
> >
> > if ! [ -f "$1" ]; then exit 0; fi
> >
> > #echo "# $0 $1 $2 $3" >> $1
> >
> > if fgrep 'ChangeLog:' $1 > /dev/null 2>&1; then exit 0; fi
> >
> > if [ -z "$2" ]; then
> >      cmd="diff --cached"
> > elif [ $2 == commit ]; then
> >      cmd="show $3"
> > else
> >      exit 0
> > fi
> >
> > git $cmd | git gcc-mklog >> $1
> >
>
> Well, that will generate changelog entry for each commit.
> For a user branch development, it's not desirable.

It isn't that useful for intermediate commits, but I was thinking it
wasn't harmful either.  But I can remove the on-by-default aspects.

> What about more explicit approach:
>
> 1) making an alias for: git diff | git gcc-mklog > commit.msg
> 2) hook:
>
> if test -f commit.msg; then
>    cat commit.msg >> "$COMMIT_MSG_FILE"
>    rm commit.msg
> fi
>
> So the changelog is created explicitly and included implicitly.

How about this?

[-- Attachment #2: mklog-msg.diff --]
[-- Type: text/x-patch, Size: 2649 bytes --]

commit 787893dc41fb8288994a2350943c22e2388476e7
Author: Jason Merrill <jason@redhat.com>
Date:   Fri May 22 18:40:35 2020 -0400

    gcc-git: Add prepare-commit-msg.
    
    This patch introduces a prepare-commit-msg hook that appends a ChangeLog
    skeleton to a commit message when the GCC_FORCE_MKLOG environment variable
    is set, and a 'git commit-mklog' command set that variable while running
    'git commit'.
    
    contrib/ChangeLog:
    
            * prepare-commit-msg: New file.
            * gcc-git-customization.sh: Install it.  Add commit-mklog alias.

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index 7a950ae5f38..a3f7da8d20b 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -30,6 +30,11 @@ git config alias.gcc-backport '!f() { rev=$1; git cherry-pick -x $@; } ; f'
 
 git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog.py" $@; } ; f'
 
+hookdir=`git rev-parse --git-path hooks`
+install "`git rev-parse --show-toplevel`/contrib/prepare-commit-msg" "$hookdir"
+
+git config alias.commit-mklog '!f() { GCC_FORCE_MKLOG=1 git commit "$@"; }; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
diff --git a/contrib/prepare-commit-msg b/contrib/prepare-commit-msg
new file mode 100644
index 00000000000..b06080b927c
--- /dev/null
+++ b/contrib/prepare-commit-msg
@@ -0,0 +1,43 @@
+#!/bin/sh
+
+COMMIT_MSG_FILE=$1
+COMMIT_SOURCE=$2
+SHA1=$3
+
+#echo "# $*" > $HOME/prepare-commit-msg-args
+
+# Can't do anything if $COMMIT_MSG_FILE isn't a file.
+if ! [ -f "$COMMIT_MSG_FILE" ]; then exit 0; fi
+
+# Don't do anything unless requested to.
+if [ -z "$GCC_FORCE_MKLOG" ]; then exit 0; fi
+
+if [ -z "$COMMIT_SOURCE" ] || [ $COMMIT_SOURCE = template ]; then
+    # No source or "template" means new commit.
+    cmd="diff --cached"
+
+elif [ $COMMIT_SOURCE = message ]; then
+    # "message" means -m, assume a new commit if there are any changes staged.
+    if ! git diff --cached --quiet; then
+	cmd="diff --cached"
+    else
+	cmd="diff --cached HEAD^"
+    fi
+
+    # Add a blank line before the ChangeLog entries.
+    echo >> "$COMMIT_MSG_FILE"
+
+elif [ $COMMIT_SOURCE = commit ]; then
+    # The message of an existing commit.  If it's HEAD, assume --amend;
+    # otherwise, assume a new commit with -C.
+    if [ $SHA1 = HEAD ]; then
+	cmd="diff --cached HEAD^"
+    else
+	cmd="diff --cached"
+    fi
+else
+    # Do nothing for merge or squash.
+    exit 0
+fi
+
+git $cmd | git gcc-mklog >> "$COMMIT_MSG_FILE"

Re: New mklog script

[Richard Earnshaw]

On 25/05/2020 20:41, Jason Merrill via Gcc-patches wrote:
> On Mon, May 25, 2020 at 5:23 AM Martin Liška <mliska@suse.cz> wrote:
>>
>> On 5/22/20 11:01 PM, Jason Merrill wrote:
>>> On Thu, May 21, 2020 at 6:03 PM Jason Merrill <jason@redhat.com> wrote:
>>>>
>>>> On Fri, May 15, 2020 at 11:39 AM Martin Liška <mliska@suse.cz> wrote:
>>>>>
>>>>> On 5/15/20 3:22 PM, Marek Polacek wrote:
>>>>>> On Fri, May 15, 2020 at 03:12:27PM +0200, Martin Liška wrote:
>>>>>>> On 5/15/20 2:42 PM, Marek Polacek wrote:
>>>>>>>> I actually use mklog -i all the time.  But I can work around it if it
>>>>>>>> disappears.
>>>>>>>
>>>>>>> Ah, I can see a consumer.
>>>>>>> There's an updated version that supports that.
>>>>>>>
>>>>>>> For the future, will you still use the option? Wouldn't be better
>>>>>>> to put the ChangeLog content directly to commit message? Note
>>>>>>> that you won't have to copy the entries to a particular ChangeLog file.
>>>>>>
>>>>>> The way I do it is to generate a patch using format-patch, use mklog -i
>>>>>> on it, then add the ChangeLog entry to the commit message via commit --amend.
>>>>>
>>>>> Hmm, you can do much better with:
>>>>>
>>>>> $ git diff | ./contrib/mklog > changelog && git commit -a -t changelog
>>>>>
>>>>> Or for an already created commit you can do:
>>>>>
>>>>> $ git diff HEAD~ | ./contrib/mklog > changelog && git commit -a --amend -e -F changelog
>>>>
>>>> With these git aliases:
>>>>
>>>>          mklog-editor = "!f() { git show | git gcc-mklog >> $1; }; f"
>>>>          addlog = "!f() { GIT_EDITOR='git mklog-editor' git commit --amend; }; f"
>>>>
>>>> I can 'git addlog' to append the output of mklog to the current
>>>> commit.  Probably better would be to do something with
>>>> prepare-commit-msg.
>>>
>>> This is pretty rudimentary, but good enough as a start:
>>
>> I like the idea of usage of the prepare commit hook.
>>
>>>
>>> #!/bin/sh
>>>
>>> #COMMIT_MSG_FILE=$1
>>> #COMMIT_SOURCE=$2
>>> #SHA1=$3
>>
>> It's better to use the named arguments.
>>
>>>
>>> if ! [ -f "$1" ]; then exit 0; fi
>>>
>>> #echo "# $0 $1 $2 $3" >> $1
>>>
>>> if fgrep 'ChangeLog:' $1 > /dev/null 2>&1; then exit 0; fi
>>>
>>> if [ -z "$2" ]; then
>>>      cmd="diff --cached"
>>> elif [ $2 == commit ]; then
>>>      cmd="show $3"
>>> else
>>>      exit 0
>>> fi
>>>
>>> git $cmd | git gcc-mklog >> $1
>>>
>>
>> Well, that will generate changelog entry for each commit.
>> For a user branch development, it's not desirable.
> 
> It isn't that useful for intermediate commits, but I was thinking it
> wasn't harmful either.  But I can remove the on-by-default aspects.
> 
>> What about more explicit approach:
>>
>> 1) making an alias for: git diff | git gcc-mklog > commit.msg
>> 2) hook:
>>
>> if test -f commit.msg; then
>>    cat commit.msg >> "$COMMIT_MSG_FILE"
>>    rm commit.msg
>> fi
>>
>> So the changelog is created explicitly and included implicitly.
> 
> How about this?
> 

> +git config alias.commit-mklog '!f() { GCC_FORCE_MKLOG=1 git commit
"$@"; }; f'
> +

I thought we had a convention that aliases we added were prefixed with
'gcc-'?  This seems to go against that.

R.

Re: New mklog script

[Martin Liška]

On 5/26/20 12:23 PM, Richard Earnshaw wrote:
> I thought we had a convention that aliases we added were prefixed with
> 'gcc-'?  This seems to go against that.

You are right, but this one is so handy ;)
What name do you suggest?

Martin

Re: New mklog script

[Richard Earnshaw]

On 26/05/2020 12:14, Martin Liška wrote:
> On 5/26/20 12:23 PM, Richard Earnshaw wrote:
>> I thought we had a convention that aliases we added were prefixed with
>> 'gcc-'?  This seems to go against that.
> 
> You are right, but this one is so handy ;)
> What name do you suggest?
> 
> Martin

gcc-ci?

R.

Re: New mklog script

[Martin Liška]

On 5/26/20 1:18 PM, Richard Earnshaw wrote:
> On 26/05/2020 12:14, Martin Liška wrote:
>> On 5/26/20 12:23 PM, Richard Earnshaw wrote:
>>> I thought we had a convention that aliases we added were prefixed with
>>> 'gcc-'?  This seems to go against that.
>>
>> You are right, but this one is so handy ;)
>> What name do you suggest?
>>
>> Martin
> 
> gcc-ci?

What the abbreviation stands for?

Martin

> 
> R.
> 

Re: New mklog script

[Richard Earnshaw]

On 26/05/2020 14:09, Martin Liška wrote:
> On 5/26/20 1:18 PM, Richard Earnshaw wrote:
>> On 26/05/2020 12:14, Martin Liška wrote:
>>> On 5/26/20 12:23 PM, Richard Earnshaw wrote:
>>>> I thought we had a convention that aliases we added were prefixed with
>>>> 'gcc-'?  This seems to go against that.
>>>
>>> You are right, but this one is so handy ;)
>>> What name do you suggest?
>>>
>>> Martin
>>
>> gcc-ci?
> 
> What the abbreviation stands for?
> 
> Martin
> 
>>
>> R.
>>
> 

CheckIn

For those who come from the SVN days where ci was the standard
abbreviation for committing :-)

R.

Re: New mklog script

[Martin Liška]

On 5/26/20 3:11 PM, Richard Earnshaw wrote:
> On 26/05/2020 14:09, Martin Liška wrote:
>> On 5/26/20 1:18 PM, Richard Earnshaw wrote:
>>> On 26/05/2020 12:14, Martin Liška wrote:
>>>> On 5/26/20 12:23 PM, Richard Earnshaw wrote:
>>>>> I thought we had a convention that aliases we added were prefixed with
>>>>> 'gcc-'?  This seems to go against that.
>>>>
>>>> You are right, but this one is so handy ;)
>>>> What name do you suggest?
>>>>
>>>> Martin
>>>
>>> gcc-ci?
>>
>> What the abbreviation stands for?
>>
>> Martin
>>
>>>
>>> R.
>>>
>>
> 
> CheckIn
> 
> For those who come from the SVN days where ci was the standard
> abbreviation for committing :-)

Ah, I see. Anyway, I prefer the original name even though it violates
the naming policy.

Let other express their preferences (Jason?).

Martin

> 
> R.
> 

Re: New mklog script

[Jakub Jelinek]

On Tue, May 26, 2020 at 03:14:54PM +0200, Martin Liška wrote:
> > > > gcc-ci?
> > > 
> > > What the abbreviation stands for?
> > > 
> > > Martin
> > > 
> > > > 
> > > > R.
> > > > 
> > > 
> > 
> > CheckIn
> > 
> > For those who come from the SVN days where ci was the standard
> > abbreviation for committing :-)
> 
> Ah, I see. Anyway, I prefer the original name even though it violates
> the naming policy.
> 
> Let other express their preferences (Jason?).

gcc-ci looks nice to me.

	Jakub

Re: New mklog script

[Martin Sebor]

On 5/26/20 7:14 AM, Martin Liška wrote:
> On 5/26/20 3:11 PM, Richard Earnshaw wrote:
>> On 26/05/2020 14:09, Martin Liška wrote:
>>> On 5/26/20 1:18 PM, Richard Earnshaw wrote:
>>>> On 26/05/2020 12:14, Martin Liška wrote:
>>>>> On 5/26/20 12:23 PM, Richard Earnshaw wrote:
>>>>>> I thought we had a convention that aliases we added were prefixed 
>>>>>> with
>>>>>> 'gcc-'?  This seems to go against that.
>>>>>
>>>>> You are right, but this one is so handy ;)
>>>>> What name do you suggest?
>>>>>
>>>>> Martin
>>>>
>>>> gcc-ci?
>>>
>>> What the abbreviation stands for?
>>>
>>> Martin
>>>
>>>>
>>>> R.
>>>>
>>>
>>
>> CheckIn
>>
>> For those who come from the SVN days where ci was the standard
>> abbreviation for committing :-)
> 
> Ah, I see. Anyway, I prefer the original name even though it violates
> the naming policy.
> 
> Let other express their preferences (Jason?).

Personally I struggle to keep track of all the different command
and function names (e.g., in GCC) and what they do.  I find
consistent names and behaviors helpful (and conversely,
inconsistencies trip me up time and time again).  So if/since there
is a convention to prefix gcc Git commands with gcc- I would prefer
to use it consistently, even if shorter names might seem nicer or
more convenient.  We don't check things in so often that typing
a few extra characters should be a burden.

By the way, it's nice that the existing gcc- aliases are documented
on https://gcc.gnu.org/gitwrite.html.  I would suggest to add this
one there as well.

Martin

Re: New mklog script

[Jason Merrill]

On Tue, May 26, 2020 at 11:38 AM Martin Sebor <msebor@gmail.com> wrote:

> On 5/26/20 7:14 AM, Martin Liška wrote:
> > On 5/26/20 3:11 PM, Richard Earnshaw wrote:
> >> On 26/05/2020 14:09, Martin Liška wrote:
> >>> On 5/26/20 1:18 PM, Richard Earnshaw wrote:
> >>>> On 26/05/2020 12:14, Martin Liška wrote:
> >>>>> On 5/26/20 12:23 PM, Richard Earnshaw wrote:
> >>>>>> I thought we had a convention that aliases we added were prefixed
> >>>>>> with 'gcc-'?  This seems to go against that.
> >>>>>
> >>>>> You are right, but this one is so handy ;)
> >>>>> What name do you suggest?
> >>>>
> >>>> gcc-ci?
> >>>
> >>> What the abbreviation stands for?
> >>
> >> CheckIn
> >>
> >> For those who come from the SVN days where ci was the standard
> >> abbreviation for committing :-)
> >
> > Ah, I see. Anyway, I prefer the original name even though it violates
> > the naming policy.
> >
> > Let other express their preferences (Jason?).
>

gcc-ci suggests that it should always be used for commits to gcc, which is
not my intent; the new alias is only used when you want to ask for a new
ChangeLog skeleton to be added.

gcc-ci-log would be better if you want something short; I'd prefer
gcc-commit-mklog, and let people define their own shorter aliases as
desired.


> Personally I struggle to keep track of all the different command
> and function names (e.g., in GCC) and what they do.  I find
> consistent names and behaviors helpful (and conversely,
> inconsistencies trip me up time and time again).  So if/since there
> is a convention to prefix gcc Git commands with gcc- I would prefer
> to use it consistently, even if shorter names might seem nicer or
> more convenient.  We don't check things in so often that typing
> a few extra characters should be a burden.
>

And people can always create their own aliases.


> By the way, it's nice that the existing gcc- aliases are documented
> on https://gcc.gnu.org/gitwrite.html.  I would suggest to add this
> one there as well.
>

Definitely.

Jason

Re: New mklog script

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 245 bytes --]

On 5/26/20 8:06 PM, Jason Merrill wrote:
> gcc-ci-log would be better if you want something short; I'd prefer gcc-commit-mklog, and let people define their own shorter aliases as desired.

Hello.

There's a rename I'm going to install.

Martin

[-- Attachment #2: 0001-Rename-commit-mklog-alias-to-gcc-commit-mklog.patch --]
[-- Type: text/x-patch, Size: 1303 bytes --]

From b423f910dcc2a58a86b61cc5b966a81066abbf12 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Wed, 27 May 2020 10:16:21 +0200
Subject: [PATCH] Rename commit-mklog alias to gcc-commit-mklog.

contrib/ChangeLog:

	* gcc-git-customization.sh: Rename
	commit-mklog to gcc-commit-mklog.
---
 contrib/gcc-git-customization.sh | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index dcc42683fa6..0e56dcf9873 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -27,10 +27,8 @@ git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream)
 
 git config alias.gcc-verify '!f() { "`git rev-parse --show-toplevel`/contrib/gcc-changelog/git_check_commit.py" $@; } ; f'
 git config alias.gcc-backport '!f() { rev=$1; git cherry-pick -x $@; } ; f'
-
 git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog.py" $@; } ; f'
-
-git config alias.commit-mklog '!f() { GCC_FORCE_MKLOG=1 git commit "$@"; }; f'
+git config alias.gcc-commit-mklog '!f() { GCC_FORCE_MKLOG=1 git commit "$@"; }; f'
 
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
-- 
2.26.2

Re: New mklog script

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 287 bytes --]

On 5/26/20 5:38 PM, Martin Sebor wrote:
> By the way, it's nice that the existing gcc- aliases are documented
> on https://gcc.gnu.org/gitwrite.html.  I would suggest to add this
> one there as well.

Yes. I added the documentation bit and pushed ho master.

Martin

[-- Attachment #2: 0001-Document-gcc-commit-mklog-hook.patch --]
[-- Type: text/x-patch, Size: 870 bytes --]

From 035bdc56110914329c860870e338463793fb5597 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Wed, 27 May 2020 10:22:01 +0200
Subject: [PATCH] Document gcc-commit-mklog hook.

---
 htdocs/gitwrite.html | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/htdocs/gitwrite.html b/htdocs/gitwrite.html
index e3a1305b..b9bcb768 100644
--- a/htdocs/gitwrite.html
+++ b/htdocs/gitwrite.html
@@ -436,6 +436,8 @@ repository:</p>
   <li><i>gcc-verify</i> - verify ChangeLog format for a particular commit</li>
   <li><i>gcc-backport</i> - alias for <code>git cherry-pick -x</code></li>
   <li><i>gcc-mklog</i> - generate a ChangeLog template for a patch</li>
+  <li><i>gcc-commit-mklog</i> - commit a git revision with a pre-filled
+      ChangeLog template</li>
 </ul>
 
 <p>The final customization that the script makes is to add a diff rule so
-- 
2.26.2

git gcc-backport.py : No such file or directory

[Thomas Koenig]

Hi Martin,

currently, doing git gcc-backport from a gcc9 branch results in

$ git gcc-backport r11-588-g8df7ee67f6fdc780e9453f2baa8d1bf62c000761
environment: /home/ig25/Gcc/gcc9/contrib/git-backport.py: No such file 
or directory

Would it make sense to push the git-backport.py file to the release
branches, as well?

Regards

	Thomas

Re: git gcc-backport.py : No such file or directory

[Martin Liška]

On 5/27/20 4:11 PM, Thomas Koenig wrote:
> Hi Martin,
> 
> currently, doing git gcc-backport from a gcc9 branch results in
> 
> $ git gcc-backport r11-588-g8df7ee67f6fdc780e9453f2baa8d1bf62c000761
> environment: /home/ig25/Gcc/gcc9/contrib/git-backport.py: No such file or directory
> 
> Would it make sense to push the git-backport.py file to the release
> branches, as well?

Yes, I'm aware of the limitation and let me do it now.

Thanks for reminder,
Martin

> 
> Regards
> 
>      Thomas

Re: git gcc-backport.py : No such file or directory

[Martin Liška]

On 5/27/20 4:13 PM, Martin Liška wrote:
> Yes, I'm aware of the limitation and let me do it now.

I've just backported the script to all active branches.

Please try it.
Martin

Re: git gcc-backport.py : No such file or directory

[Thomas Koenig]

Hi Martin,

> On 5/27/20 4:13 PM, Martin Liška wrote:
>> Yes, I'm aware of the limitation and let me do it now.
> 
> I've just backported the script to all active branches.
> 
> Please try it.

Works without a hitch, thanks!  I have just successfully pushed r9-8628
to the gcc-9 branch.

Regards

	Thomas

Re: New mklog script

[Martin Sebor]

On 5/15/20 2:59 AM, Martin Liška wrote:
> Hi.
> 
> Since we moved to git world and we're in the preparation for ChangeLog 
> messages
> being in git commit messages, I think it's the right time to also 
> simplify mklog
> script.
> 
> I'm sending a new version (which should eventually replace contrib/mklog 
> and contrib/mklog.pl).
> Changes made in the version:
> 
> - the script uses unifdiff - it rapidly simplifies parsing of the '+-!' 
> lines that is done
>    in contrib/mklog
> - no author nor date stamp is used - that all can be get from git
> - --inline option is not supported - I don't see a use-case for it now
> - the new script has a unit tests (just few of them for now)
> 
> I compares results in between the old Python script for last 80 commits 
> and it's very close,
> in some cases it does even better.
> 
> I'm planning to maintain and improve the script for the future.
> 
> Thoughts?

It's pretty nice.  I have a script of my own that does the same thing
in a slightly different way.  Here's an example of its output:
https://gcc.gnu.org/pipermail/gcc-patches/attachments/20200323/5437db5a/attachment-0001.bin

I find this format more helpful for the reasons below so unless your
script can be tweaked to do something similar I'd like to be able to
continue to use mine going forward with the new infrastructure.

As for my comments on mklog_ng.py: In the one test I did the script
produced a single long ChangeLog entry with all the files in the diff
I gave it, tests and all, in alphabetical order.  The script fills in
"New test." for new tests.  The rest has to be edited as one would
expect.

I would find the output easier to work with if it a) grouped files by
"subsystem" corresponding to each ChangeLog directory (and if it also
identified each subsystem), b) put the testsuite section last, and
(as a bonus) c) grouped all new files in each section together.

First, I find this logical grouping helpful in thinking about how
the changes are structured (e.g., would it make sense to restructure
them or break things up to reduce coupling and make review easier),
and whom they need to be reviewed by.

Second, this is the grouping I'm already used to from my own script
(so YMMV here of course).

Finally, my script also looks up bugs in Bugzilla and adds a line with
each bug number and its Summary at the top of the patch.  This helps me
double-check the spelling of the bug id(s) in case I transpose digits
etc.

Martin

PS My script modifies the patch file in place: it adds the ChangeLog
section if it doesn't exist yet, but it doesn't do anything it does.
I'd love for it to check the existing ChangeLog if it exists and
update it when it finds differences between it and the latest patch
that aren't reflected there.

Without this, each time a patch changes I have to review the entry
and update it as necessary.  That makes it too easy to miss things.

Re: New mklog script

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 4351 bytes --]

On 5/15/20 5:06 PM, Martin Sebor wrote:
> On 5/15/20 2:59 AM, Martin Liška wrote:
>> Hi.
>>
>> Since we moved to git world and we're in the preparation for ChangeLog messages
>> being in git commit messages, I think it's the right time to also simplify mklog
>> script.
>>
>> I'm sending a new version (which should eventually replace contrib/mklog and contrib/mklog.pl).
>> Changes made in the version:
>>
>> - the script uses unifdiff - it rapidly simplifies parsing of the '+-!' lines that is done
>>    in contrib/mklog
>> - no author nor date stamp is used - that all can be get from git
>> - --inline option is not supported - I don't see a use-case for it now
>> - the new script has a unit tests (just few of them for now)
>>
>> I compares results in between the old Python script for last 80 commits and it's very close,
>> in some cases it does even better.
>>
>> I'm planning to maintain and improve the script for the future.
>>
>> Thoughts?

Hello Martin.

I welcome the feedback. Leitmotif of the gcc-changelog changes is to simplify
scripts used by individual contributors. I must confess I have a special script
that takes content of changelog entries, applies them to individual files, or
appends Backported header to them. All that should be gone and only git messages
will be used.

> 
> It's pretty nice.  I have a script of my own that does the same thing
> in a slightly different way.  Here's an example of its output:
> https://gcc.gnu.org/pipermail/gcc-patches/attachments/20200323/5437db5a/attachment-0001.bin
> 
> I find this format more helpful for the reasons below so unless your
> script can be tweaked to do something similar I'd like to be able to
> continue to use mine going forward with the new infrastructure.

Let's extend the contrib script.

> 
> As for my comments on mklog_ng.py: In the one test I did the script
> produced a single long ChangeLog entry with all the files in the diff
> I gave it, tests and all, in alphabetical order.  The script fills in
> "New test." for new tests.  The rest has to be edited as one would
> expect.
> 
> I would find the output easier to work with if it a) grouped files by
> "subsystem" corresponding to each ChangeLog directory (and if it also

The script does that, let's consider a patch 'p':

./contrib/mklog_ng.py p
gcc/ChangeLog:

	* ipa-icf.c:
	* varasm.c:

gcc/testsuite/ChangeLog:

	* gcc.dg/pr40209.c:

The idea is to put changes in the order in which it appears in a patch.
Because normal work flow is to follow a patch and fulfill ChangeLog entries.
As seen in the patch, changes in varasm.c follow the change in
gcc/testsuite/, which can't be handled as varasm.c needs to be documented
in gcc/ChangeLog.

> identified each subsystem), b) put the testsuite section last, and
> (as a bonus)

Good idea, I'll do it!

> c) grouped all new files in each section together.

Likewise here. And what about deleted files at the very end of a section?

> 
> First, I find this logical grouping helpful in thinking about how
> the changes are structured (e.g., would it make sense to restructure
> them or break things up to reduce coupling and make review easier),
> and whom they need to be reviewed by.

Fully yes, please see the updated patch and tell me what's missing.

> 
> Second, this is the grouping I'm already used to from my own script
> (so YMMV here of course).
> 
> Finally, my script also looks up bugs in Bugzilla and adds a line with
> each bug number and its Summary at the top of the patch.  This helps me
> double-check the spelling of the bug id(s) in case I transpose digits
> etc.

Can you please share how do you do it? It would be easy to add it.

> 
> Martin
> 
> PS My script modifies the patch file in place: it adds the ChangeLog
> section if it doesn't exist yet, but it doesn't do anything it does.
> I'd love for it to check the existing ChangeLog if it exists and
> update it when it finds differences between it and the latest patch
> that aren't reflected there.

Well, what about using more git approach? I mean putting all you need
into a commit message.

> 
> Without this, each time a patch changes I have to review the entry
> and update it as necessary.  That makes it too easy to miss things.

What you can use for a git commit is 'git gcc-verify' which will
inform you about missing (undocumented) Changes.

Martin

[-- Attachment #2: p --]
[-- Type: text/plain, Size: 892 bytes --]

diff --git a/gcc/ipa-icf.c b/gcc/ipa-icf.c
index aab79492357..f0df1002488 100644
--- a/gcc/ipa-icf.c
+++ b/gcc/ipa-icf.c
@@ -1,5 +1,7 @@
 
 
+
+
 /* Interprocedural Identical Code Folding pass
    Copyright (C) 2014-2020 Free Software Foundation, Inc.
 
diff --git a/gcc/testsuite/gcc.dg/pr40209.c b/gcc/testsuite/gcc.dg/pr40209.c
index 4e77df5c2e6..c23d69d1f1b 100644
--- a/gcc/testsuite/gcc.dg/pr40209.c
+++ b/gcc/testsuite/gcc.dg/pr40209.c
@@ -1,6 +1,8 @@
 /* { dg-do compile } */
 /* { dg-options "-O2 -fprofile-use -fopt-info -Wno-missing-profile" } */
 
+
+
 void process(const char *s);
 
 struct BaseHolder {
diff --git a/gcc/varasm.c b/gcc/varasm.c
index f062e48071f..fd3c7ca8cf3 100644
--- a/gcc/varasm.c
+++ b/gcc/varasm.c
@@ -1,3 +1,5 @@
+
+
 /* Output variables, constants and external declarations, for GNU compiler.
    Copyright (C) 1987-2020 Free Software Foundation, Inc.
 

[-- Attachment #3: 0001-New-mklog-script.patch --]
[-- Type: text/x-patch, Size: 18999 bytes --]

From 58719696e8b1acc06d788c0db069c4194b61e56f Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Fri, 15 May 2020 00:44:07 +0200
Subject: [PATCH] New mklog script.

contrib/ChangeLog:

2020-05-15  Martin Liska  <mliska@suse.cz>

	* gcc-git-customization.sh: Add
	alias.gcc-mklog new hook.
	* mklog_ng.py: New file.
	* test_mklog_ng.py: New file.
---
 contrib/gcc-git-customization.sh |   2 +
 contrib/mklog_ng.py              | 200 ++++++++++++++++++++++
 contrib/test_mklog_ng.py         | 275 +++++++++++++++++++++++++++++++
 gcc/ipa-icf.c                    |   2 +
 4 files changed, 479 insertions(+)
 create mode 100755 contrib/mklog_ng.py
 create mode 100755 contrib/test_mklog_ng.py

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index a932bf8c06a..b7b97327be3 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -25,6 +25,8 @@ git config alias.svn-rev '!f() { rev=$1; shift; git log --all --grep="^From-SVN:
 git config alias.gcc-descr \!"f() { if test \${1:-no} = --full; then c=\${2:-master}; r=\$(git describe --all --abbrev=40 --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-,r,p'); expr match \${r:-no} '^r[0-9]\\+\$' >/dev/null && r=\${r}-0-g\$(git rev-parse \${2:-master}); else c=\${1:-master}; r=\$(git describe --all --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)-\\([0-9]\\+\\)-g[0-9a-f]*\$,r\\2-\\3,p;s,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)\$,r\\2-0,p'); fi; if test -n \$r; then o=\$(git config --get gcc-config.upstream); rr=\$(echo \$r | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\\(-g[0-9a-f]\\+\\)\\?\$,\\1,p'); if git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$rr >/dev/null; then m=releases/gcc-\$rr; else m=master; fi; git merge-base --is-ancestor \$c \${o:-origin}/\$m && \echo \${r}; fi; }; f"
 git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream); r=\$(echo \$1 | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\$,\\1,p'); n=\$(echo \$1 | sed -n 's,^r[0-9]\\+-\\([0-9]\\+\\)\$,\\1,p'); test -z \$r && echo Invalid id \$1 && exit 1; h=\$(git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$r); test -z \$h && h=\$(git rev-parse --verify --quiet \${o:-origin}/master); p=\$(git describe --all --match 'basepoints/gcc-'\$r \$h | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+-\\([0-9]\\+\\)-g[0-9a-f]*\$,\\2,p;s,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+\$,0,p'); git rev-parse --verify \$h~\$(expr \$p - \$n); }; f"
 
+git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog_ng.py" $@; } ; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
diff --git a/contrib/mklog_ng.py b/contrib/mklog_ng.py
new file mode 100755
index 00000000000..228f87f0156
--- /dev/null
+++ b/contrib/mklog_ng.py
@@ -0,0 +1,200 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import argparse
+import os
+import re
+import sys
+
+from unidiff import PatchSet
+
+pr_regex = re.compile(r'(\/(\/|\*)|[Cc*!])\s+(?P<pr>PR [a-z+-]+\/[0-9]+)')
+identifier_regex = re.compile(r'^([a-zA-Z0-9_#].*)')
+comment_regex = re.compile(r'^\/\*')
+struct_regex = re.compile(r'^((class|struct|union|enum)\s+[a-zA-Z0-9_]+)')
+macro_regex = re.compile(r'#\s*(define|undef)\s+([a-zA-Z0-9_]+)')
+super_macro_regex = re.compile(r'^DEF[A-Z0-9_]+\s*\(([a-zA-Z0-9_]+)')
+fn_regex = re.compile(r'([a-zA-Z_][^()\s]*)\s*\([^*]')
+template_and_param_regex = re.compile(r'<[^<>]*>')
+
+function_extensions = set(['.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def'])
+
+help_message = """\
+Generate ChangeLog template for PATCH.
+PATCH must be generated using diff(1)'s -up or -cp options
+(or their equivalent in git).
+"""
+
+script_folder = os.path.realpath(__file__)
+gcc_root = os.path.dirname(os.path.dirname(script_folder))
+
+
+def find_changelog(path):
+    folder = os.path.split(path)[0]
+    while True:
+        if os.path.exists(os.path.join(gcc_root, folder, 'ChangeLog')):
+            return folder
+        folder = os.path.dirname(folder)
+        if folder == '':
+            return folder
+    raise AssertionError()
+
+
+def extract_function_name(line):
+    if comment_regex.match(line):
+        return None
+    m = struct_regex.search(line)
+    if m:
+        # Struct declaration
+        return m.group(1)
+    m = macro_regex.search(line)
+    if m:
+        # Macro definition
+        return m.group(2)
+    m = super_macro_regex.search(line)
+    if m:
+        # Supermacro
+        return m.group(1)
+    m = fn_regex.search(line)
+    if m:
+        # Discard template and function parameters.
+        fn = m.group(1)
+        fn = re.sub(template_and_param_regex, '', fn)
+        return fn.rstrip()
+    return None
+
+
+def try_add_function(functions, line):
+    fn = extract_function_name(line)
+    if fn and fn not in functions:
+        functions.append(fn)
+    return bool(fn)
+
+
+def sort_changelog_files(changed_file):
+    return (changed_file.is_added_file, changed_file.is_removed_file)
+
+
+def generate_changelog(data, no_functions=False):
+    changelogs = {}
+    changelog_list = []
+    prs = []
+    out = ''
+    diff = PatchSet(data)
+
+    for file in diff:
+        changelog = find_changelog(file.path)
+        if changelog not in changelogs:
+            changelogs[changelog] = []
+            changelog_list.append(changelog)
+        changelogs[changelog].append(file)
+
+        # Extract PR entries from newly added tests
+        if 'testsuite' in file.path and file.is_added_file:
+            for line in list(file)[0]:
+                m = pr_regex.search(line.value)
+                if m:
+                    pr = m.group('pr')
+                    if pr not in prs:
+                        prs.append(pr)
+                else:
+                    break
+
+    # sort ChangeLog so that 'testsuite' is at the end
+    for changelog in sorted(changelog_list, key=lambda x: 'testsuite' in x):
+        files = changelogs[changelog]
+        out += '%s:\n' % os.path.join(changelog, 'ChangeLog')
+        out += '\n'
+        for pr in prs:
+            out += '\t%s\n' % pr
+        # new and deleted files should be at the end
+        for file in sorted(files, key=sort_changelog_files):
+            assert file.path.startswith(changelog)
+            in_tests = 'testsuite' in changelog or 'testsuite' in file.path
+            relative_path = file.path[len(changelog):].lstrip('/')
+            functions = []
+            if file.is_added_file:
+                msg = 'New test' if in_tests else 'New file'
+                out += '\t* %s: %s.\n' % (relative_path, msg)
+            elif file.is_removed_file:
+                out += '\t* %s: Removed.\n' % (relative_path)
+            else:
+                if not no_functions:
+                    for hunk in file:
+                        # Do not add function names for testsuite files
+                        extension = os.path.splitext(relative_path)[1]
+                        if not in_tests and extension in function_extensions:
+                            last_fn = None
+                            modified_visited = False
+                            success = False
+                            for line in hunk:
+                                m = identifier_regex.match(line.value)
+                                if line.is_added or line.is_removed:
+                                    if not line.value.strip():
+                                        continue
+                                    modified_visited = True
+                                    if m and try_add_function(functions,
+                                                              m.group(1)):
+                                        last_fn = None
+                                        success = True
+                                elif line.is_context:
+                                    if last_fn and modified_visited:
+                                        try_add_function(functions, last_fn)
+                                        last_fn = None
+                                        modified_visited = False
+                                        success = True
+                                    elif m:
+                                        last_fn = m.group(1)
+                                        modified_visited = False
+                            if not success:
+                                try_add_function(functions,
+                                                 hunk.section_header)
+                if functions:
+                    out += '\t* %s (%s):\n' % (relative_path, functions[0])
+                    for fn in functions[1:]:
+                        out += '\t(%s):\n' % fn
+                else:
+                    out += '\t* %s:\n' % relative_path
+        out += '\n'
+    return out
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description=help_message)
+    parser.add_argument('input', nargs='?',
+                        help='Patch file (or missing, read standard input)')
+    parser.add_argument('-s', '--no-functions', action='store_true',
+                        help='Do not generate function names in ChangeLogs')
+    args = parser.parse_args()
+    if args.input == '-':
+        args.input = None
+
+    input = open(args.input) if args.input else sys.stdin
+    data = input.read()
+    output = generate_changelog(data, args.no_functions)
+    print(output, end='')
diff --git a/contrib/test_mklog_ng.py b/contrib/test_mklog_ng.py
new file mode 100755
index 00000000000..6f626eb49e1
--- /dev/null
+++ b/contrib/test_mklog_ng.py
@@ -0,0 +1,275 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import unittest
+
+from mklog_ng import generate_changelog
+
+PATCH1 = '''\
+diff --git a/gcc/config/riscv/riscv.h b/gcc/config/riscv/riscv.h
+index 567c23380fe..e6209ede9d6 100644
+--- a/gcc/config/riscv/riscv.h
++++ b/gcc/config/riscv/riscv.h
+@@ -920,6 +920,7 @@ extern unsigned riscv_stack_boundary;
+ #define SHIFT_RS1 15
+ #define SHIFT_IMM 20
+ #define IMM_BITS 12
++#define C_S_BITS 5
+ #define C_SxSP_BITS 6
+ 
+ #define IMM_REACH (1LL << IMM_BITS)
+@@ -929,6 +930,10 @@ extern unsigned riscv_stack_boundary;
+ #define SWSP_REACH (4LL << C_SxSP_BITS)
+ #define SDSP_REACH (8LL << C_SxSP_BITS)
+ 
++/* This is the maximum value that can be represented in a compressed load/store
++   offset (an unsigned 5-bit value scaled by 4).  */
++#define CSW_MAX_OFFSET ((4LL << C_S_BITS) - 1) & ~3
++
+ /* Called from RISCV_REORG, this is defined in riscv-sr.c.  */
+ 
+ extern void riscv_remove_unneeded_save_restore_calls (void);
+
+'''
+
+EXPECTED1 = '''\
+gcc/ChangeLog:
+
+	* config/riscv/riscv.h (C_S_BITS):
+	(CSW_MAX_OFFSET):
+
+'''
+
+PATCH2 = '''\
+diff --git a/gcc/targhooks.h b/gcc/targhooks.h
+index 9704d23f1db..b572a36e8cf 100644
+--- a/gcc/targhooks.h
++++ b/gcc/targhooks.h
+@@ -120,7 +120,7 @@ extern bool default_empty_mask_is_expensive (unsigned);
+ extern void *default_init_cost (class loop *);
+ extern unsigned default_add_stmt_cost (class vec_info *, void *, int,
+ 				       enum vect_cost_for_stmt,
+-				       class _stmt_vec_info *, int,
++				       class _stmt_vec_info *, tree, int,
+ 				       enum vect_cost_model_location);
+ extern void default_finish_cost (void *, unsigned *, unsigned *, unsigned *);
+ extern void default_destroy_cost_data (void *);
+@@ -186,6 +186,7 @@ extern tree default_emutls_var_init (tree, tree, tree);
+ extern unsigned int default_hard_regno_nregs (unsigned int, machine_mode);
+ extern bool default_hard_regno_scratch_ok (unsigned int);
+ extern bool default_mode_dependent_address_p (const_rtx, addr_space_t);
++extern bool default_new_address_profitable_p (rtx, rtx_insn *, rtx);
+ extern bool default_target_option_valid_attribute_p (tree, tree, tree, int);
+ extern bool default_target_option_pragma_parse (tree, tree);
+ extern bool default_target_can_inline_p (tree, tree);
+
+'''
+
+EXPECTED2 = '''\
+gcc/ChangeLog:
+
+	* targhooks.h (default_add_stmt_cost):
+	(default_new_address_profitable_p):
+
+'''
+
+PATCH3 = '''\
+diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
+index 2b1e33f94ae..7f47402f9b9 100644
+--- a/libcpp/include/cpplib.h
++++ b/libcpp/include/cpplib.h
+@@ -173,7 +173,7 @@ enum c_lang {CLK_GNUC89 = 0, CLK_GNUC99, CLK_GNUC11, CLK_GNUC17, CLK_GNUC2X,
+ 	     CLK_STDC2X,
+ 	     CLK_GNUCXX, CLK_CXX98, CLK_GNUCXX11, CLK_CXX11,
+ 	     CLK_GNUCXX14, CLK_CXX14, CLK_GNUCXX17, CLK_CXX17,
+-	     CLK_GNUCXX2A, CLK_CXX2A, CLK_ASM};
++	     CLK_GNUCXX20, CLK_CXX20, CLK_ASM};
+ 
+ /* Payload of a NUMBER, STRING, CHAR or COMMENT token.  */
+ struct GTY(()) cpp_string {
+@@ -484,7 +484,7 @@ struct cpp_options
+   /* Nonzero for C2X decimal floating-point constants.  */
+   unsigned char dfp_constants;
+ 
+-  /* Nonzero for C++2a __VA_OPT__ feature.  */
++  /* Nonzero for C++20 __VA_OPT__ feature.  */
+   unsigned char va_opt;
+ 
+   /* Nonzero for the '::' token.  */
+
+'''
+
+EXPECTED3 = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h (enum c_lang):
+	(struct cpp_options):
+
+'''
+
+EXPECTED3B = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h:
+
+'''
+
+PATCH4 = '''\
+diff --git a/gcc/ipa-icf.c b/gcc/ipa-icf.c
+index aab79492357..f0df1002488 100644
+--- a/gcc/ipa-icf.c
++++ b/gcc/ipa-icf.c
+@@ -1,5 +1,7 @@
+ 
+ 
++
++
+ /* Interprocedural Identical Code Folding pass
+    Copyright (C) 2014-2020 Free Software Foundation, Inc.
+ 
+diff --git a/gcc/testsuite/gcc.dg/pr32374.c b/gcc/testsuite/gcc.dg/pr32374.c
+deleted file mode 100644
+index de15d559f5b..00000000000
+--- a/gcc/testsuite/gcc.dg/pr32374.c
++++ /dev/null
+@@ -1,20 +0,0 @@
+-/* { dg-do compile } */
+-/* { dg-options "-O2" } */
+-
+-extern int *stderr;
+-
+-void f (int *, const char *, ...);
+-
+-void g (const char *conf_name)
+-{
+-  typedef struct
+-  {
+-    const char *label;
+-    const int value;
+-  } Section;
+-
+-  const Section sections[2] = { {"", 0}, {"", 1} };
+-
+-  f (stderr, "", "", conf_name, 0, sections[0]);
+-  f (stderr, "", "", conf_name, 0, sections[0]);
+-}
+diff --git a/gcc/testsuite/gcc.dg/pr40209.c b/gcc/testsuite/gcc.dg/pr40209.c
+index 4e77df5c2e6..c23d69d1f1b 100644
+--- a/gcc/testsuite/gcc.dg/pr40209.c
++++ b/gcc/testsuite/gcc.dg/pr40209.c
+@@ -1,6 +1,8 @@
+ /* { dg-do compile } */
+ /* { dg-options "-O2 -fprofile-use -fopt-info -Wno-missing-profile" } */
+ 
++
++
+ void process(const char *s);
+ 
+ struct BaseHolder {
+diff --git a/gcc/testsuite/gcc.dg/pr50209.c b/gcc/testsuite/gcc.dg/pr50209.c
+new file mode 100644
+index 00000000000..b28b04f6431
+--- /dev/null
++++ b/gcc/testsuite/gcc.dg/pr50209.c
+@@ -0,0 +1,3 @@
++
++
++
+diff --git a/gcc/testsuite/gcc.dg/pr63567-1.c b/gcc/testsuite/gcc.dg/pr63567-1.c
+index 97da171563e..00c5ecc11fa 100644
+--- a/gcc/testsuite/gcc.dg/pr63567-1.c
++++ b/gcc/testsuite/gcc.dg/pr63567-1.c
+@@ -1,3 +1,4 @@
++
+ /* PR c/63567 */
+ /* { dg-do compile } */
+ /* { dg-options "" } */
+diff --git a/gcc/varasm.c b/gcc/varasm.c
+index f062e48071f..fd3c7ca8cf3 100644
+--- a/gcc/varasm.c
++++ b/gcc/varasm.c
+@@ -1,3 +1,5 @@
++
++
+ /* Output variables, constants and external declarations, for GNU compiler.
+    Copyright (C) 1987-2020 Free Software Foundation, Inc.
+ 
+diff --git a/libssp/gets-chk.c b/libssp/gets-chk.c
+index 4ad78c1f77b..6687b368038 100644
+--- a/libssp/gets-chk.c
++++ b/libssp/gets-chk.c
+@@ -32,6 +32,8 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
+ <http://www.gnu.org/licenses/>.  */
+ 
+ 
++
++
+ #include "config.h"
+ #include <ssp/ssp.h>
+ #include <stdarg.h>
+'''
+
+EXPECTED4 = '''\
+gcc/ChangeLog:
+
+	* ipa-icf.c:
+	* varasm.c:
+
+libssp/ChangeLog:
+
+	* gets-chk.c:
+
+gcc/testsuite/ChangeLog:
+
+	* gcc.dg/pr40209.c:
+	* gcc.dg/pr63567-1.c:
+	* gcc.dg/pr32374.c: Removed.
+	* gcc.dg/pr50209.c: New test.
+
+'''
+
+class TestMklog(unittest.TestCase):
+    def test_macro_definition(self):
+        changelog = generate_changelog(PATCH1)
+        assert changelog == EXPECTED1
+
+    def test_changed_argument(self):
+        changelog = generate_changelog(PATCH2)
+        assert changelog == EXPECTED2
+
+    def test_enum_and_struct(self):
+        changelog = generate_changelog(PATCH3)
+        assert changelog == EXPECTED3
+
+    def test_no_function(self):
+        changelog = generate_changelog(PATCH3, True)
+        assert changelog == EXPECTED3B
+
+    def test_sorting(self):
+        changelog = generate_changelog(PATCH4)
+        assert changelog == EXPECTED4
diff --git a/gcc/ipa-icf.c b/gcc/ipa-icf.c
index 069de9d82fb..aab79492357 100644
--- a/gcc/ipa-icf.c
+++ b/gcc/ipa-icf.c
@@ -1,3 +1,5 @@
+
+
 /* Interprocedural Identical Code Folding pass
    Copyright (C) 2014-2020 Free Software Foundation, Inc.
 
-- 
2.26.2

Re: New mklog script

[Jakub Jelinek]

On Tue, May 19, 2020 at 10:11:28AM +0200, Martin Liška wrote:
> > I find this format more helpful for the reasons below so unless your
> > script can be tweaked to do something similar I'd like to be able to
> > continue to use mine going forward with the new infrastructure.
> 
> Let's extend the contrib script.

BTW, concerning mklog, the very common problem is that it doesn't do the
right thing because the patch doesn't contain enough context to figure out
what exactly has changed.  If the script would be used together with git
rather than just on a patch file, perhaps it could handle more, like
ask git for a patch with unlimited context (like -U100000000 on patch does).
The common problems I remember is that e.g. when changing a function comment
above some function, it is attributed to the previous function rather than
following, labels in function confusing it:
 void
 foo ()
 {
   ...
 label:
   ...
-  ...
+  ...
 }
will result in (label), GTY markers confusing it
 struct GTY foobar {
   ...
-  ...
+  ...
 };
resulting in (struct GTY) or so, another common problem is too large
function names (or more often *.md define_* names); here I'm afraid
diff doesn't have an argument to not truncate it, or sometimes e.g. changes
to #define being attributed to something else.
I know some of the issues can be pretty hard to deal with.

	Jakub

Re: New mklog script

[Martin Liška]

On 5/19/20 10:23 AM, Jakub Jelinek wrote:
> On Tue, May 19, 2020 at 10:11:28AM +0200, Martin Liška wrote:
>>> I find this format more helpful for the reasons below so unless your
>>> script can be tweaked to do something similar I'd like to be able to
>>> continue to use mine going forward with the new infrastructure.
>>
>> Let's extend the contrib script.
> 
> BTW, concerning mklog, the very common problem is that it doesn't do the
> right thing because the patch doesn't contain enough context to figure out
> what exactly has changed.  If the script would be used together with git
> rather than just on a patch file, perhaps it could handle more, like
> ask git for a patch with unlimited context (like -U100000000 on patch does).

Good idea but the regex parsing takes some time.

> The common problems I remember is that e.g. when changing a function comment
> above some function, it is attributed to the previous function rather than
> following, labels in function confusing it:
>   void
>   foo ()
>   {
>     ...
>   label:
>     ...
> -  ...
> +  ...
>   }

I've just tested that and it will take function for patch context (sem_variable::equals):
@@ -1875,6 +1875,7 @@ sem_variable::equals (tree t1, tree t2)
      default:
        return return_false_with_msg ("Unknown TREE code reached");
      }
+
  }

> will result in (label), GTY markers confusing it
>   struct GTY foobar {
>     ...
> -  ...
> +  ...
>   };
> resulting in (struct GTY)

Yes, I know about these and I'll improve stripping of GTY markers.

> or so, another common problem is too large
> function names (or more often *.md define_* names); here I'm afraid
> diff doesn't have an argument to not truncate it, or sometimes e.g. changes
> to #define being attributed to something else.
> I know some of the issues can be pretty hard to deal with.

;)

Martin

> 
> 	Jakub
> 

Re: New mklog script

[Michael Matz]

Hello,

On Tue, 19 May 2020, Martin Liška wrote:

> > The common problems I remember is that e.g. when changing a function comment
> > above some function, it is attributed to the previous function rather than
> > following, labels in function confusing it:
> >   void
> >   foo ()
> >   {
> >     ...
> >   label:
> >     ...
> > -  ...
> > +  ...
> >   }
> 
> I've just tested that and it will take function for patch context
> (sem_variable::equals):
> @@ -1875,6 +1875,7 @@ sem_variable::equals (tree t1, tree t2)
>      default:
>        return return_false_with_msg ("Unknown TREE code reached");
>      }
> +
>  }

No, the problem happens when the label is at column 0, like function names 
are.  Basically diff -p uses a regexp morally equivalent to 
'^[[:alpha:]$_]' to detect function headers, and git diff -p and friends 
followed suit.  But it should use something like
'^[[:alpha:]$_].*[^:]$' to rule out things ending with ':'.  See also diff 
-F for GNU diff.


Ciao,
Michael.

Re: New mklog script

[Richard Earnshaw]

On 19/05/2020 15:51, Michael Matz wrote:
> Hello,
> 
> On Tue, 19 May 2020, Martin Liška wrote:
> 
>>> The common problems I remember is that e.g. when changing a function comment
>>> above some function, it is attributed to the previous function rather than
>>> following, labels in function confusing it:
>>>   void
>>>   foo ()
>>>   {
>>>     ...
>>>   label:
>>>     ...
>>> -  ...
>>> +  ...
>>>   }
>>
>> I've just tested that and it will take function for patch context
>> (sem_variable::equals):
>> @@ -1875,6 +1875,7 @@ sem_variable::equals (tree t1, tree t2)
>>      default:
>>        return return_false_with_msg ("Unknown TREE code reached");
>>      }
>> +
>>  }
> 
> No, the problem happens when the label is at column 0, like function names 
> are.  Basically diff -p uses a regexp morally equivalent to 
> '^[[:alpha:]$_]' to detect function headers, and git diff -p and friends 
> followed suit.  But it should use something like
> '^[[:alpha:]$_].*[^:]$' to rule out things ending with ':'.  See also diff 
> -F for GNU diff.
> 
> 
> Ciao,
> Michael.
> 

This is really a wart in the GNU coding style.  And one reason why I
tend to indent such labels by a single space.  It particularly affects
things like class definitions where public, private, etc statements
often appear in column 0.

IMO, it would be nice to get an official change in the coding style for
this, it's really irritating.

R.

Re: New mklog script

[Jakub Jelinek]

On Tue, May 19, 2020 at 05:21:16PM +0100, Richard Earnshaw wrote:
> This is really a wart in the GNU coding style.  And one reason why I
> tend to indent such labels by a single space.  It particularly affects
> things like class definitions where public, private, etc statements
> often appear in column 0.
> 
> IMO, it would be nice to get an official change in the coding style for
> this, it's really irritating.

It doesn't have to be just label,
void
foo ()
{
  ...
#define X ...
  ...
#undef X
  ...
}
does the similar thing for mklog.

	Jakub

Re: New mklog script

[Michael Matz]

Hello,

On Tue, 19 May 2020, Jakub Jelinek wrote:

> On Tue, May 19, 2020 at 05:21:16PM +0100, Richard Earnshaw wrote:
> > This is really a wart in the GNU coding style.  And one reason why I
> > tend to indent such labels by a single space.  It particularly affects
> > things like class definitions where public, private, etc statements
> > often appear in column 0.
> > 
> > IMO, it would be nice to get an official change in the coding style for
> > this, it's really irritating.
> 
> It doesn't have to be just label,
> void
> foo ()
> {
>   ...
> #define X ...
>   ...
> #undef X
>   ...
> }
> does the similar thing for mklog.

That particular one would be a mere bug in mklog then.  diff -p regards 
only members of [[:alpha:]$_] as acceptable start characters of function 
names (i.e. indeed things that can start a C identifier (ignoring details 
like non-base characters) with the '$' extension), of which '#' is none.


Ciao,
Michael.

Re: New mklog script

[Jonathan Wakely]

On Tue, 19 May 2020 at 09:26, Jakub Jelinek via Gcc <gcc@gcc.gnu.org> wrote:
>
> On Tue, May 19, 2020 at 10:11:28AM +0200, Martin Liška wrote:
> > > I find this format more helpful for the reasons below so unless your
> > > script can be tweaked to do something similar I'd like to be able to
> > > continue to use mine going forward with the new infrastructure.
> >
> > Let's extend the contrib script.
>
> BTW, concerning mklog, the very common problem is that it doesn't do the
> right thing because the patch doesn't contain enough context to figure out
> what exactly has changed.  If the script would be used together with git
> rather than just on a patch file, perhaps it could handle more, like
> ask git for a patch with unlimited context (like -U100000000 on patch does).
> The common problems I remember is that e.g. when changing a function comment
> above some function, it is attributed to the previous function rather than
> following, labels in function confusing it:
>  void
>  foo ()
>  {
>    ...
>  label:
>    ...
> -  ...
> +  ...
>  }
> will result in (label), GTY markers confusing it
>  struct GTY foobar {
>    ...
> -  ...
> +  ...
>  };
> resulting in (struct GTY) or so, another common problem is too large
> function names (or more often *.md define_* names); here I'm afraid
> diff doesn't have an argument to not truncate it, or sometimes e.g. changes
> to #define being attributed to something else.
> I know some of the issues can be pretty hard to deal with.

This isn't a complaint, because I don't expect them to work, but
mklog.py (and diff --show-c-function) are useless for libstdc++ code,
they can't handle namespaces and result in nonsense like:

--- a/libstdc++-v3/src/c++11/random.cc
+++ b/libstdc++-v3/src/c++11/random.cc
@@ -97,7 +97,7 @@ namespace std _GLIBCXX_VISIBILITY(default)

and

libstdc++-v3/ChangeLog:

        * src/c++11/random.cc (_GLIBCXX_VISIBILITY):
        * testsuite/26_numerics/random/random_device/94087.cc: New test.

Re: New mklog script

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 238 bytes --]

On 5/19/20 10:11 AM, Martin Liška wrote:
> Can you please share how do you do it? It would be easy to add it.

I added the feature via --fill-up-bug-titles option. It uses common
request and beatifulsoup packages.

Martin

[-- Attachment #2: 0001-New-mklog-script.patch --]
[-- Type: text/x-patch, Size: 21078 bytes --]

From 5450c99b54131d1942ece3ffb6bbe415b1c85151 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Fri, 15 May 2020 00:44:07 +0200
Subject: [PATCH] New mklog script.

contrib/ChangeLog:

2020-05-15  Martin Liska  <mliska@suse.cz>

	* gcc-git-customization.sh: Add
	alias.gcc-mklog new hook.
	* mklog_ng.py: New file.
	* test_mklog_ng.py: New file.
---
 contrib/gcc-git-customization.sh |   2 +
 contrib/mklog_ng.py              | 223 ++++++++++++++++++++
 contrib/test_mklog_ng.py         | 345 +++++++++++++++++++++++++++++++
 gcc/ipa-icf.c                    |   2 +
 4 files changed, 572 insertions(+)
 create mode 100755 contrib/mklog_ng.py
 create mode 100755 contrib/test_mklog_ng.py

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index a932bf8c06a..b7b97327be3 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -25,6 +25,8 @@ git config alias.svn-rev '!f() { rev=$1; shift; git log --all --grep="^From-SVN:
 git config alias.gcc-descr \!"f() { if test \${1:-no} = --full; then c=\${2:-master}; r=\$(git describe --all --abbrev=40 --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-,r,p'); expr match \${r:-no} '^r[0-9]\\+\$' >/dev/null && r=\${r}-0-g\$(git rev-parse \${2:-master}); else c=\${1:-master}; r=\$(git describe --all --match 'basepoints/gcc-[0-9]*' \$c | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)-\\([0-9]\\+\\)-g[0-9a-f]*\$,r\\2-\\3,p;s,^\\(tags/\\)\\?basepoints/gcc-\\([0-9]\\+\\)\$,r\\2-0,p'); fi; if test -n \$r; then o=\$(git config --get gcc-config.upstream); rr=\$(echo \$r | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\\(-g[0-9a-f]\\+\\)\\?\$,\\1,p'); if git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$rr >/dev/null; then m=releases/gcc-\$rr; else m=master; fi; git merge-base --is-ancestor \$c \${o:-origin}/\$m && \echo \${r}; fi; }; f"
 git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream); r=\$(echo \$1 | sed -n 's,^r\\([0-9]\\+\\)-[0-9]\\+\$,\\1,p'); n=\$(echo \$1 | sed -n 's,^r[0-9]\\+-\\([0-9]\\+\\)\$,\\1,p'); test -z \$r && echo Invalid id \$1 && exit 1; h=\$(git rev-parse --verify --quiet \${o:-origin}/releases/gcc-\$r); test -z \$h && h=\$(git rev-parse --verify --quiet \${o:-origin}/master); p=\$(git describe --all --match 'basepoints/gcc-'\$r \$h | sed -n 's,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+-\\([0-9]\\+\\)-g[0-9a-f]*\$,\\2,p;s,^\\(tags/\\)\\?basepoints/gcc-[0-9]\\+\$,0,p'); git rev-parse --verify \$h~\$(expr \$p - \$n); }; f"
 
+git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog_ng.py" $@; } ; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
diff --git a/contrib/mklog_ng.py b/contrib/mklog_ng.py
new file mode 100755
index 00000000000..cc3f937c253
--- /dev/null
+++ b/contrib/mklog_ng.py
@@ -0,0 +1,223 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import argparse
+import bs4
+import os
+import re
+import requests
+import sys
+
+from unidiff import PatchSet
+
+pr_regex = re.compile(r'(\/(\/|\*)|[Cc*!])\s+(?P<pr>PR [a-z+-]+\/[0-9]+)')
+identifier_regex = re.compile(r'^([a-zA-Z0-9_#].*)')
+comment_regex = re.compile(r'^\/\*')
+struct_regex = re.compile(r'^((class|struct|union|enum)\s+[a-zA-Z0-9_]+)')
+macro_regex = re.compile(r'#\s*(define|undef)\s+([a-zA-Z0-9_]+)')
+super_macro_regex = re.compile(r'^DEF[A-Z0-9_]+\s*\(([a-zA-Z0-9_]+)')
+fn_regex = re.compile(r'([a-zA-Z_][^()\s]*)\s*\([^*]')
+template_and_param_regex = re.compile(r'<[^<>]*>')
+
+function_extensions = set(['.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def'])
+
+help_message = """\
+Generate ChangeLog template for PATCH.
+PATCH must be generated using diff(1)'s -up or -cp options
+(or their equivalent in git).
+"""
+
+script_folder = os.path.realpath(__file__)
+gcc_root = os.path.dirname(os.path.dirname(script_folder))
+
+
+def find_changelog(path):
+    folder = os.path.split(path)[0]
+    while True:
+        if os.path.exists(os.path.join(gcc_root, folder, 'ChangeLog')):
+            return folder
+        folder = os.path.dirname(folder)
+        if folder == '':
+            return folder
+    raise AssertionError()
+
+
+def extract_function_name(line):
+    if comment_regex.match(line):
+        return None
+    m = struct_regex.search(line)
+    if m:
+        # Struct declaration
+        return m.group(1)
+    m = macro_regex.search(line)
+    if m:
+        # Macro definition
+        return m.group(2)
+    m = super_macro_regex.search(line)
+    if m:
+        # Supermacro
+        return m.group(1)
+    m = fn_regex.search(line)
+    if m:
+        # Discard template and function parameters.
+        fn = m.group(1)
+        fn = re.sub(template_and_param_regex, '', fn)
+        return fn.rstrip()
+    return None
+
+
+def try_add_function(functions, line):
+    fn = extract_function_name(line)
+    if fn and fn not in functions:
+        functions.append(fn)
+    return bool(fn)
+
+
+def sort_changelog_files(changed_file):
+    return (changed_file.is_added_file, changed_file.is_removed_file)
+
+
+def get_pr_titles(prs):
+    if not prs:
+        return ''
+
+    output = ''
+    for pr in prs:
+        id = pr.split('/')[-1]
+        r = requests.get('https://gcc.gnu.org/PR%s' % id)
+        html = bs4.BeautifulSoup(r.text, features='lxml')
+        title = html.title.text
+        title = title[title.find('–') + 1:].strip()
+        output += '%s - %s\n' % (pr, title)
+    output += '\n'
+    return output
+
+def generate_changelog(data, no_functions=False, fill_pr_titles=False):
+    changelogs = {}
+    changelog_list = []
+    prs = []
+    out = ''
+    diff = PatchSet(data)
+
+    for file in diff:
+        changelog = find_changelog(file.path)
+        if changelog not in changelogs:
+            changelogs[changelog] = []
+            changelog_list.append(changelog)
+        changelogs[changelog].append(file)
+
+        # Extract PR entries from newly added tests
+        if 'testsuite' in file.path and file.is_added_file:
+            for line in list(file)[0]:
+                m = pr_regex.search(line.value)
+                if m:
+                    pr = m.group('pr')
+                    if pr not in prs:
+                        prs.append(pr)
+                else:
+                    break
+
+    if fill_pr_titles:
+        out += get_pr_titles(prs)
+
+    # sort ChangeLog so that 'testsuite' is at the end
+    for changelog in sorted(changelog_list, key=lambda x: 'testsuite' in x):
+        files = changelogs[changelog]
+        out += '%s:\n' % os.path.join(changelog, 'ChangeLog')
+        out += '\n'
+        for pr in prs:
+            out += '\t%s\n' % pr
+        # new and deleted files should be at the end
+        for file in sorted(files, key=sort_changelog_files):
+            assert file.path.startswith(changelog)
+            in_tests = 'testsuite' in changelog or 'testsuite' in file.path
+            relative_path = file.path[len(changelog):].lstrip('/')
+            functions = []
+            if file.is_added_file:
+                msg = 'New test' if in_tests else 'New file'
+                out += '\t* %s: %s.\n' % (relative_path, msg)
+            elif file.is_removed_file:
+                out += '\t* %s: Removed.\n' % (relative_path)
+            else:
+                if not no_functions:
+                    for hunk in file:
+                        # Do not add function names for testsuite files
+                        extension = os.path.splitext(relative_path)[1]
+                        if not in_tests and extension in function_extensions:
+                            last_fn = None
+                            modified_visited = False
+                            success = False
+                            for line in hunk:
+                                m = identifier_regex.match(line.value)
+                                if line.is_added or line.is_removed:
+                                    if not line.value.strip():
+                                        continue
+                                    modified_visited = True
+                                    if m and try_add_function(functions,
+                                                              m.group(1)):
+                                        last_fn = None
+                                        success = True
+                                elif line.is_context:
+                                    if last_fn and modified_visited:
+                                        try_add_function(functions, last_fn)
+                                        last_fn = None
+                                        modified_visited = False
+                                        success = True
+                                    elif m:
+                                        last_fn = m.group(1)
+                                        modified_visited = False
+                            if not success:
+                                try_add_function(functions,
+                                                 hunk.section_header)
+                if functions:
+                    out += '\t* %s (%s):\n' % (relative_path, functions[0])
+                    for fn in functions[1:]:
+                        out += '\t(%s):\n' % fn
+                else:
+                    out += '\t* %s:\n' % relative_path
+        out += '\n'
+    return out
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description=help_message)
+    parser.add_argument('input', nargs='?',
+                        help='Patch file (or missing, read standard input)')
+    parser.add_argument('-s', '--no-functions', action='store_true',
+                        help='Do not generate function names in ChangeLogs')
+    parser.add_argument('-p', '--fill-up-bug-titles', action='store_true',
+                        help='Download title of mentioned PRs')
+    args = parser.parse_args()
+    if args.input == '-':
+        args.input = None
+
+    input = open(args.input) if args.input else sys.stdin
+    data = input.read()
+    output = generate_changelog(data, args.no_functions,
+                                args.fill_up_bug_titles)
+    print(output, end='')
diff --git a/contrib/test_mklog_ng.py b/contrib/test_mklog_ng.py
new file mode 100755
index 00000000000..7e948a17c6e
--- /dev/null
+++ b/contrib/test_mklog_ng.py
@@ -0,0 +1,345 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import unittest
+
+from mklog_ng import generate_changelog
+
+PATCH1 = '''\
+diff --git a/gcc/config/riscv/riscv.h b/gcc/config/riscv/riscv.h
+index 567c23380fe..e6209ede9d6 100644
+--- a/gcc/config/riscv/riscv.h
++++ b/gcc/config/riscv/riscv.h
+@@ -920,6 +920,7 @@ extern unsigned riscv_stack_boundary;
+ #define SHIFT_RS1 15
+ #define SHIFT_IMM 20
+ #define IMM_BITS 12
++#define C_S_BITS 5
+ #define C_SxSP_BITS 6
+ 
+ #define IMM_REACH (1LL << IMM_BITS)
+@@ -929,6 +930,10 @@ extern unsigned riscv_stack_boundary;
+ #define SWSP_REACH (4LL << C_SxSP_BITS)
+ #define SDSP_REACH (8LL << C_SxSP_BITS)
+ 
++/* This is the maximum value that can be represented in a compressed load/store
++   offset (an unsigned 5-bit value scaled by 4).  */
++#define CSW_MAX_OFFSET ((4LL << C_S_BITS) - 1) & ~3
++
+ /* Called from RISCV_REORG, this is defined in riscv-sr.c.  */
+ 
+ extern void riscv_remove_unneeded_save_restore_calls (void);
+
+'''
+
+EXPECTED1 = '''\
+gcc/ChangeLog:
+
+	* config/riscv/riscv.h (C_S_BITS):
+	(CSW_MAX_OFFSET):
+
+'''
+
+PATCH2 = '''\
+diff --git a/gcc/targhooks.h b/gcc/targhooks.h
+index 9704d23f1db..b572a36e8cf 100644
+--- a/gcc/targhooks.h
++++ b/gcc/targhooks.h
+@@ -120,7 +120,7 @@ extern bool default_empty_mask_is_expensive (unsigned);
+ extern void *default_init_cost (class loop *);
+ extern unsigned default_add_stmt_cost (class vec_info *, void *, int,
+ 				       enum vect_cost_for_stmt,
+-				       class _stmt_vec_info *, int,
++				       class _stmt_vec_info *, tree, int,
+ 				       enum vect_cost_model_location);
+ extern void default_finish_cost (void *, unsigned *, unsigned *, unsigned *);
+ extern void default_destroy_cost_data (void *);
+@@ -186,6 +186,7 @@ extern tree default_emutls_var_init (tree, tree, tree);
+ extern unsigned int default_hard_regno_nregs (unsigned int, machine_mode);
+ extern bool default_hard_regno_scratch_ok (unsigned int);
+ extern bool default_mode_dependent_address_p (const_rtx, addr_space_t);
++extern bool default_new_address_profitable_p (rtx, rtx_insn *, rtx);
+ extern bool default_target_option_valid_attribute_p (tree, tree, tree, int);
+ extern bool default_target_option_pragma_parse (tree, tree);
+ extern bool default_target_can_inline_p (tree, tree);
+
+'''
+
+EXPECTED2 = '''\
+gcc/ChangeLog:
+
+	* targhooks.h (default_add_stmt_cost):
+	(default_new_address_profitable_p):
+
+'''
+
+PATCH3 = '''\
+diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
+index 2b1e33f94ae..7f47402f9b9 100644
+--- a/libcpp/include/cpplib.h
++++ b/libcpp/include/cpplib.h
+@@ -173,7 +173,7 @@ enum c_lang {CLK_GNUC89 = 0, CLK_GNUC99, CLK_GNUC11, CLK_GNUC17, CLK_GNUC2X,
+ 	     CLK_STDC2X,
+ 	     CLK_GNUCXX, CLK_CXX98, CLK_GNUCXX11, CLK_CXX11,
+ 	     CLK_GNUCXX14, CLK_CXX14, CLK_GNUCXX17, CLK_CXX17,
+-	     CLK_GNUCXX2A, CLK_CXX2A, CLK_ASM};
++	     CLK_GNUCXX20, CLK_CXX20, CLK_ASM};
+ 
+ /* Payload of a NUMBER, STRING, CHAR or COMMENT token.  */
+ struct GTY(()) cpp_string {
+@@ -484,7 +484,7 @@ struct cpp_options
+   /* Nonzero for C2X decimal floating-point constants.  */
+   unsigned char dfp_constants;
+ 
+-  /* Nonzero for C++2a __VA_OPT__ feature.  */
++  /* Nonzero for C++20 __VA_OPT__ feature.  */
+   unsigned char va_opt;
+ 
+   /* Nonzero for the '::' token.  */
+
+'''
+
+EXPECTED3 = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h (enum c_lang):
+	(struct cpp_options):
+
+'''
+
+EXPECTED3B = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h:
+
+'''
+
+PATCH4 = '''\
+diff --git a/gcc/ipa-icf.c b/gcc/ipa-icf.c
+index aab79492357..f0df1002488 100644
+--- a/gcc/ipa-icf.c
++++ b/gcc/ipa-icf.c
+@@ -1,5 +1,7 @@
+ 
+ 
++
++
+ /* Interprocedural Identical Code Folding pass
+    Copyright (C) 2014-2020 Free Software Foundation, Inc.
+ 
+diff --git a/gcc/testsuite/gcc.dg/pr32374.c b/gcc/testsuite/gcc.dg/pr32374.c
+deleted file mode 100644
+index de15d559f5b..00000000000
+--- a/gcc/testsuite/gcc.dg/pr32374.c
++++ /dev/null
+@@ -1,20 +0,0 @@
+-/* { dg-do compile } */
+-/* { dg-options "-O2" } */
+-
+-extern int *stderr;
+-
+-void f (int *, const char *, ...);
+-
+-void g (const char *conf_name)
+-{
+-  typedef struct
+-  {
+-    const char *label;
+-    const int value;
+-  } Section;
+-
+-  const Section sections[2] = { {"", 0}, {"", 1} };
+-
+-  f (stderr, "", "", conf_name, 0, sections[0]);
+-  f (stderr, "", "", conf_name, 0, sections[0]);
+-}
+diff --git a/gcc/testsuite/gcc.dg/pr40209.c b/gcc/testsuite/gcc.dg/pr40209.c
+index 4e77df5c2e6..c23d69d1f1b 100644
+--- a/gcc/testsuite/gcc.dg/pr40209.c
++++ b/gcc/testsuite/gcc.dg/pr40209.c
+@@ -1,6 +1,8 @@
+ /* { dg-do compile } */
+ /* { dg-options "-O2 -fprofile-use -fopt-info -Wno-missing-profile" } */
+ 
++
++
+ void process(const char *s);
+ 
+ struct BaseHolder {
+diff --git a/gcc/testsuite/gcc.dg/pr50209.c b/gcc/testsuite/gcc.dg/pr50209.c
+new file mode 100644
+index 00000000000..b28b04f6431
+--- /dev/null
++++ b/gcc/testsuite/gcc.dg/pr50209.c
+@@ -0,0 +1,3 @@
++
++
++
+diff --git a/gcc/testsuite/gcc.dg/pr63567-1.c b/gcc/testsuite/gcc.dg/pr63567-1.c
+index 97da171563e..00c5ecc11fa 100644
+--- a/gcc/testsuite/gcc.dg/pr63567-1.c
++++ b/gcc/testsuite/gcc.dg/pr63567-1.c
+@@ -1,3 +1,4 @@
++
+ /* PR c/63567 */
+ /* { dg-do compile } */
+ /* { dg-options "" } */
+diff --git a/gcc/varasm.c b/gcc/varasm.c
+index f062e48071f..fd3c7ca8cf3 100644
+--- a/gcc/varasm.c
++++ b/gcc/varasm.c
+@@ -1,3 +1,5 @@
++
++
+ /* Output variables, constants and external declarations, for GNU compiler.
+    Copyright (C) 1987-2020 Free Software Foundation, Inc.
+ 
+diff --git a/libssp/gets-chk.c b/libssp/gets-chk.c
+index 4ad78c1f77b..6687b368038 100644
+--- a/libssp/gets-chk.c
++++ b/libssp/gets-chk.c
+@@ -32,6 +32,8 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
+ <http://www.gnu.org/licenses/>.  */
+ 
+ 
++
++
+ #include "config.h"
+ #include <ssp/ssp.h>
+ #include <stdarg.h>
+'''
+
+EXPECTED4 = '''\
+gcc/ChangeLog:
+
+	* ipa-icf.c:
+	* varasm.c:
+
+libssp/ChangeLog:
+
+	* gets-chk.c:
+
+gcc/testsuite/ChangeLog:
+
+	* gcc.dg/pr40209.c:
+	* gcc.dg/pr63567-1.c:
+	* gcc.dg/pr32374.c: Removed.
+	* gcc.dg/pr50209.c: New test.
+
+'''
+
+PATCH5 = '''\
+diff --git a/gcc/testsuite/gcc.target/i386/pr95046-6.c b/gcc/testsuite/gcc.target/i386/pr95046-6.c
+new file mode 100644
+index 00000000000..dcc8999c446
+--- /dev/null
++++ b/gcc/testsuite/gcc.target/i386/pr95046-6.c
+@@ -0,0 +1,44 @@
++/* PR target/95046 */
++/* { dg-do compile { target { ! ia32 } } } */
++/* { dg-options "-O3 -mavx512vl" } */
++
++
++double r[2];
++int s[2];
++unsigned int u[2];
++
++void
++test_float (void)
++{
++  for (int i = 0; i < 2; i++)
++    r[i] = s[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvtdq2pd" } } */
++
++void
++test_ufloat (void)
++{
++  for (int i = 0; i < 2; i++)
++    r[i] = u[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvtudq2pd" } } */
++
++void
++test_fix (void)
++{
++  for (int i = 0; i < 2; i++)
++    s[i] = r[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvttpd2dqx" } } */
++
++void
++test_ufix (void)
++{
++  for (int i = 0; i < 2; i++)
++    u[i] = r[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvttpd2udqx" } } */
+-- 
+2.26.2
+
+'''
+
+EXPECTED5 = '''\
+PR target/95046 - Vectorize V2SFmode operations
+
+gcc/testsuite/ChangeLog:
+
+	PR target/95046
+	* gcc.target/i386/pr95046-6.c: New test.
+
+'''
+
+class TestMklog(unittest.TestCase):
+    def test_macro_definition(self):
+        changelog = generate_changelog(PATCH1)
+        assert changelog == EXPECTED1
+
+    def test_changed_argument(self):
+        changelog = generate_changelog(PATCH2)
+        assert changelog == EXPECTED2
+
+    def test_enum_and_struct(self):
+        changelog = generate_changelog(PATCH3)
+        assert changelog == EXPECTED3
+
+    def test_no_function(self):
+        changelog = generate_changelog(PATCH3, True)
+        assert changelog == EXPECTED3B
+
+    def test_sorting(self):
+        changelog = generate_changelog(PATCH4)
+        assert changelog == EXPECTED4
+
+    def test_pr_bugzilla_download(self):
+        changelog = generate_changelog(PATCH5, fill_pr_titles=True)
+        assert changelog == EXPECTED5
diff --git a/gcc/ipa-icf.c b/gcc/ipa-icf.c
index 069de9d82fb..aab79492357 100644
--- a/gcc/ipa-icf.c
+++ b/gcc/ipa-icf.c
@@ -1,3 +1,5 @@
+
+
 /* Interprocedural Identical Code Folding pass
    Copyright (C) 2014-2020 Free Software Foundation, Inc.
 
-- 
2.26.2

Re: New mklog script

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 568 bytes --]

On 5/19/20 10:53 AM, Martin Liška wrote:
> On 5/19/20 10:11 AM, Martin Liška wrote:
>> Can you please share how do you do it? It would be easy to add it.
> 
> I added the feature via --fill-up-bug-titles option. It uses common
> request and beatifulsoup packages.
> 
> Martin

Ok, I'm going to install the following 2 patches that put 2 legacy scripts
into contrib/legacy folder. And a new 'git gcc-mklog' alias is added for the
new one.

Hope other are fine with the change. I'm planning to work on the new script
and fix future limitations.

Martin

[-- Attachment #2: 0002-New-mklog-script.patch --]
[-- Type: text/x-patch, Size: 19340 bytes --]

From 577083b84f6851eee0b2d36e26aef42f69676942 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Fri, 15 May 2020 00:44:07 +0200
Subject: [PATCH 2/2] New mklog script.

contrib/ChangeLog:

2020-05-15  Martin Liska  <mliska@suse.cz>

	* gcc-git-customization.sh: Add
	alias.gcc-mklog new hook.
	* mklog.py: New file.
	* test_mklog.py: New file.
---
 contrib/gcc-git-customization.sh |   2 +
 contrib/mklog.py                 | 223 ++++++++++++++++++++
 contrib/test_mklog.py            | 345 +++++++++++++++++++++++++++++++
 3 files changed, 570 insertions(+)
 create mode 100755 contrib/mklog.py
 create mode 100755 contrib/test_mklog.py

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index ce293d1fe42..91d378ba32a 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -27,6 +27,8 @@ git config alias.gcc-undescr \!"f() { o=\$(git config --get gcc-config.upstream)
 
 git config alias.gcc-verify '!f() { "`git rev-parse --show-toplevel`/contrib/gcc-changelog/git_check_commit.py" $@; } ; f'
 
+git config alias.gcc-mklog '!f() { "`git rev-parse --show-toplevel`/contrib/mklog.py" $@; } ; f'
+
 # Make diff on MD files use "(define" as a function marker.
 # Use this in conjunction with a .gitattributes file containing
 # *.md    diff=md
diff --git a/contrib/mklog.py b/contrib/mklog.py
new file mode 100755
index 00000000000..cc3f937c253
--- /dev/null
+++ b/contrib/mklog.py
@@ -0,0 +1,223 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import argparse
+import bs4
+import os
+import re
+import requests
+import sys
+
+from unidiff import PatchSet
+
+pr_regex = re.compile(r'(\/(\/|\*)|[Cc*!])\s+(?P<pr>PR [a-z+-]+\/[0-9]+)')
+identifier_regex = re.compile(r'^([a-zA-Z0-9_#].*)')
+comment_regex = re.compile(r'^\/\*')
+struct_regex = re.compile(r'^((class|struct|union|enum)\s+[a-zA-Z0-9_]+)')
+macro_regex = re.compile(r'#\s*(define|undef)\s+([a-zA-Z0-9_]+)')
+super_macro_regex = re.compile(r'^DEF[A-Z0-9_]+\s*\(([a-zA-Z0-9_]+)')
+fn_regex = re.compile(r'([a-zA-Z_][^()\s]*)\s*\([^*]')
+template_and_param_regex = re.compile(r'<[^<>]*>')
+
+function_extensions = set(['.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def'])
+
+help_message = """\
+Generate ChangeLog template for PATCH.
+PATCH must be generated using diff(1)'s -up or -cp options
+(or their equivalent in git).
+"""
+
+script_folder = os.path.realpath(__file__)
+gcc_root = os.path.dirname(os.path.dirname(script_folder))
+
+
+def find_changelog(path):
+    folder = os.path.split(path)[0]
+    while True:
+        if os.path.exists(os.path.join(gcc_root, folder, 'ChangeLog')):
+            return folder
+        folder = os.path.dirname(folder)
+        if folder == '':
+            return folder
+    raise AssertionError()
+
+
+def extract_function_name(line):
+    if comment_regex.match(line):
+        return None
+    m = struct_regex.search(line)
+    if m:
+        # Struct declaration
+        return m.group(1)
+    m = macro_regex.search(line)
+    if m:
+        # Macro definition
+        return m.group(2)
+    m = super_macro_regex.search(line)
+    if m:
+        # Supermacro
+        return m.group(1)
+    m = fn_regex.search(line)
+    if m:
+        # Discard template and function parameters.
+        fn = m.group(1)
+        fn = re.sub(template_and_param_regex, '', fn)
+        return fn.rstrip()
+    return None
+
+
+def try_add_function(functions, line):
+    fn = extract_function_name(line)
+    if fn and fn not in functions:
+        functions.append(fn)
+    return bool(fn)
+
+
+def sort_changelog_files(changed_file):
+    return (changed_file.is_added_file, changed_file.is_removed_file)
+
+
+def get_pr_titles(prs):
+    if not prs:
+        return ''
+
+    output = ''
+    for pr in prs:
+        id = pr.split('/')[-1]
+        r = requests.get('https://gcc.gnu.org/PR%s' % id)
+        html = bs4.BeautifulSoup(r.text, features='lxml')
+        title = html.title.text
+        title = title[title.find('–') + 1:].strip()
+        output += '%s - %s\n' % (pr, title)
+    output += '\n'
+    return output
+
+def generate_changelog(data, no_functions=False, fill_pr_titles=False):
+    changelogs = {}
+    changelog_list = []
+    prs = []
+    out = ''
+    diff = PatchSet(data)
+
+    for file in diff:
+        changelog = find_changelog(file.path)
+        if changelog not in changelogs:
+            changelogs[changelog] = []
+            changelog_list.append(changelog)
+        changelogs[changelog].append(file)
+
+        # Extract PR entries from newly added tests
+        if 'testsuite' in file.path and file.is_added_file:
+            for line in list(file)[0]:
+                m = pr_regex.search(line.value)
+                if m:
+                    pr = m.group('pr')
+                    if pr not in prs:
+                        prs.append(pr)
+                else:
+                    break
+
+    if fill_pr_titles:
+        out += get_pr_titles(prs)
+
+    # sort ChangeLog so that 'testsuite' is at the end
+    for changelog in sorted(changelog_list, key=lambda x: 'testsuite' in x):
+        files = changelogs[changelog]
+        out += '%s:\n' % os.path.join(changelog, 'ChangeLog')
+        out += '\n'
+        for pr in prs:
+            out += '\t%s\n' % pr
+        # new and deleted files should be at the end
+        for file in sorted(files, key=sort_changelog_files):
+            assert file.path.startswith(changelog)
+            in_tests = 'testsuite' in changelog or 'testsuite' in file.path
+            relative_path = file.path[len(changelog):].lstrip('/')
+            functions = []
+            if file.is_added_file:
+                msg = 'New test' if in_tests else 'New file'
+                out += '\t* %s: %s.\n' % (relative_path, msg)
+            elif file.is_removed_file:
+                out += '\t* %s: Removed.\n' % (relative_path)
+            else:
+                if not no_functions:
+                    for hunk in file:
+                        # Do not add function names for testsuite files
+                        extension = os.path.splitext(relative_path)[1]
+                        if not in_tests and extension in function_extensions:
+                            last_fn = None
+                            modified_visited = False
+                            success = False
+                            for line in hunk:
+                                m = identifier_regex.match(line.value)
+                                if line.is_added or line.is_removed:
+                                    if not line.value.strip():
+                                        continue
+                                    modified_visited = True
+                                    if m and try_add_function(functions,
+                                                              m.group(1)):
+                                        last_fn = None
+                                        success = True
+                                elif line.is_context:
+                                    if last_fn and modified_visited:
+                                        try_add_function(functions, last_fn)
+                                        last_fn = None
+                                        modified_visited = False
+                                        success = True
+                                    elif m:
+                                        last_fn = m.group(1)
+                                        modified_visited = False
+                            if not success:
+                                try_add_function(functions,
+                                                 hunk.section_header)
+                if functions:
+                    out += '\t* %s (%s):\n' % (relative_path, functions[0])
+                    for fn in functions[1:]:
+                        out += '\t(%s):\n' % fn
+                else:
+                    out += '\t* %s:\n' % relative_path
+        out += '\n'
+    return out
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description=help_message)
+    parser.add_argument('input', nargs='?',
+                        help='Patch file (or missing, read standard input)')
+    parser.add_argument('-s', '--no-functions', action='store_true',
+                        help='Do not generate function names in ChangeLogs')
+    parser.add_argument('-p', '--fill-up-bug-titles', action='store_true',
+                        help='Download title of mentioned PRs')
+    args = parser.parse_args()
+    if args.input == '-':
+        args.input = None
+
+    input = open(args.input) if args.input else sys.stdin
+    data = input.read()
+    output = generate_changelog(data, args.no_functions,
+                                args.fill_up_bug_titles)
+    print(output, end='')
diff --git a/contrib/test_mklog.py b/contrib/test_mklog.py
new file mode 100755
index 00000000000..ca7b9e79d95
--- /dev/null
+++ b/contrib/test_mklog.py
@@ -0,0 +1,345 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING.  If not, write to
+# the Free Software Foundation, 51 Franklin Street, Fifth Floor,
+# Boston, MA 02110-1301, USA.
+
+# This script parses a .diff file generated with 'diff -up' or 'diff -cp'
+# and adds a skeleton ChangeLog file to the file. It does not try to be
+# too smart when parsing function names, but it produces a reasonable
+# approximation.
+#
+# Author: Martin Liska <mliska@suse.cz>
+
+import unittest
+
+from mklog import generate_changelog
+
+PATCH1 = '''\
+diff --git a/gcc/config/riscv/riscv.h b/gcc/config/riscv/riscv.h
+index 567c23380fe..e6209ede9d6 100644
+--- a/gcc/config/riscv/riscv.h
++++ b/gcc/config/riscv/riscv.h
+@@ -920,6 +920,7 @@ extern unsigned riscv_stack_boundary;
+ #define SHIFT_RS1 15
+ #define SHIFT_IMM 20
+ #define IMM_BITS 12
++#define C_S_BITS 5
+ #define C_SxSP_BITS 6
+ 
+ #define IMM_REACH (1LL << IMM_BITS)
+@@ -929,6 +930,10 @@ extern unsigned riscv_stack_boundary;
+ #define SWSP_REACH (4LL << C_SxSP_BITS)
+ #define SDSP_REACH (8LL << C_SxSP_BITS)
+ 
++/* This is the maximum value that can be represented in a compressed load/store
++   offset (an unsigned 5-bit value scaled by 4).  */
++#define CSW_MAX_OFFSET ((4LL << C_S_BITS) - 1) & ~3
++
+ /* Called from RISCV_REORG, this is defined in riscv-sr.c.  */
+ 
+ extern void riscv_remove_unneeded_save_restore_calls (void);
+
+'''
+
+EXPECTED1 = '''\
+gcc/ChangeLog:
+
+	* config/riscv/riscv.h (C_S_BITS):
+	(CSW_MAX_OFFSET):
+
+'''
+
+PATCH2 = '''\
+diff --git a/gcc/targhooks.h b/gcc/targhooks.h
+index 9704d23f1db..b572a36e8cf 100644
+--- a/gcc/targhooks.h
++++ b/gcc/targhooks.h
+@@ -120,7 +120,7 @@ extern bool default_empty_mask_is_expensive (unsigned);
+ extern void *default_init_cost (class loop *);
+ extern unsigned default_add_stmt_cost (class vec_info *, void *, int,
+ 				       enum vect_cost_for_stmt,
+-				       class _stmt_vec_info *, int,
++				       class _stmt_vec_info *, tree, int,
+ 				       enum vect_cost_model_location);
+ extern void default_finish_cost (void *, unsigned *, unsigned *, unsigned *);
+ extern void default_destroy_cost_data (void *);
+@@ -186,6 +186,7 @@ extern tree default_emutls_var_init (tree, tree, tree);
+ extern unsigned int default_hard_regno_nregs (unsigned int, machine_mode);
+ extern bool default_hard_regno_scratch_ok (unsigned int);
+ extern bool default_mode_dependent_address_p (const_rtx, addr_space_t);
++extern bool default_new_address_profitable_p (rtx, rtx_insn *, rtx);
+ extern bool default_target_option_valid_attribute_p (tree, tree, tree, int);
+ extern bool default_target_option_pragma_parse (tree, tree);
+ extern bool default_target_can_inline_p (tree, tree);
+
+'''
+
+EXPECTED2 = '''\
+gcc/ChangeLog:
+
+	* targhooks.h (default_add_stmt_cost):
+	(default_new_address_profitable_p):
+
+'''
+
+PATCH3 = '''\
+diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
+index 2b1e33f94ae..7f47402f9b9 100644
+--- a/libcpp/include/cpplib.h
++++ b/libcpp/include/cpplib.h
+@@ -173,7 +173,7 @@ enum c_lang {CLK_GNUC89 = 0, CLK_GNUC99, CLK_GNUC11, CLK_GNUC17, CLK_GNUC2X,
+ 	     CLK_STDC2X,
+ 	     CLK_GNUCXX, CLK_CXX98, CLK_GNUCXX11, CLK_CXX11,
+ 	     CLK_GNUCXX14, CLK_CXX14, CLK_GNUCXX17, CLK_CXX17,
+-	     CLK_GNUCXX2A, CLK_CXX2A, CLK_ASM};
++	     CLK_GNUCXX20, CLK_CXX20, CLK_ASM};
+ 
+ /* Payload of a NUMBER, STRING, CHAR or COMMENT token.  */
+ struct GTY(()) cpp_string {
+@@ -484,7 +484,7 @@ struct cpp_options
+   /* Nonzero for C2X decimal floating-point constants.  */
+   unsigned char dfp_constants;
+ 
+-  /* Nonzero for C++2a __VA_OPT__ feature.  */
++  /* Nonzero for C++20 __VA_OPT__ feature.  */
+   unsigned char va_opt;
+ 
+   /* Nonzero for the '::' token.  */
+
+'''
+
+EXPECTED3 = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h (enum c_lang):
+	(struct cpp_options):
+
+'''
+
+EXPECTED3B = '''\
+libcpp/ChangeLog:
+
+	* include/cpplib.h:
+
+'''
+
+PATCH4 = '''\
+diff --git a/gcc/ipa-icf.c b/gcc/ipa-icf.c
+index aab79492357..f0df1002488 100644
+--- a/gcc/ipa-icf.c
++++ b/gcc/ipa-icf.c
+@@ -1,5 +1,7 @@
+ 
+ 
++
++
+ /* Interprocedural Identical Code Folding pass
+    Copyright (C) 2014-2020 Free Software Foundation, Inc.
+ 
+diff --git a/gcc/testsuite/gcc.dg/pr32374.c b/gcc/testsuite/gcc.dg/pr32374.c
+deleted file mode 100644
+index de15d559f5b..00000000000
+--- a/gcc/testsuite/gcc.dg/pr32374.c
++++ /dev/null
+@@ -1,20 +0,0 @@
+-/* { dg-do compile } */
+-/* { dg-options "-O2" } */
+-
+-extern int *stderr;
+-
+-void f (int *, const char *, ...);
+-
+-void g (const char *conf_name)
+-{
+-  typedef struct
+-  {
+-    const char *label;
+-    const int value;
+-  } Section;
+-
+-  const Section sections[2] = { {"", 0}, {"", 1} };
+-
+-  f (stderr, "", "", conf_name, 0, sections[0]);
+-  f (stderr, "", "", conf_name, 0, sections[0]);
+-}
+diff --git a/gcc/testsuite/gcc.dg/pr40209.c b/gcc/testsuite/gcc.dg/pr40209.c
+index 4e77df5c2e6..c23d69d1f1b 100644
+--- a/gcc/testsuite/gcc.dg/pr40209.c
++++ b/gcc/testsuite/gcc.dg/pr40209.c
+@@ -1,6 +1,8 @@
+ /* { dg-do compile } */
+ /* { dg-options "-O2 -fprofile-use -fopt-info -Wno-missing-profile" } */
+ 
++
++
+ void process(const char *s);
+ 
+ struct BaseHolder {
+diff --git a/gcc/testsuite/gcc.dg/pr50209.c b/gcc/testsuite/gcc.dg/pr50209.c
+new file mode 100644
+index 00000000000..b28b04f6431
+--- /dev/null
++++ b/gcc/testsuite/gcc.dg/pr50209.c
+@@ -0,0 +1,3 @@
++
++
++
+diff --git a/gcc/testsuite/gcc.dg/pr63567-1.c b/gcc/testsuite/gcc.dg/pr63567-1.c
+index 97da171563e..00c5ecc11fa 100644
+--- a/gcc/testsuite/gcc.dg/pr63567-1.c
++++ b/gcc/testsuite/gcc.dg/pr63567-1.c
+@@ -1,3 +1,4 @@
++
+ /* PR c/63567 */
+ /* { dg-do compile } */
+ /* { dg-options "" } */
+diff --git a/gcc/varasm.c b/gcc/varasm.c
+index f062e48071f..fd3c7ca8cf3 100644
+--- a/gcc/varasm.c
++++ b/gcc/varasm.c
+@@ -1,3 +1,5 @@
++
++
+ /* Output variables, constants and external declarations, for GNU compiler.
+    Copyright (C) 1987-2020 Free Software Foundation, Inc.
+ 
+diff --git a/libssp/gets-chk.c b/libssp/gets-chk.c
+index 4ad78c1f77b..6687b368038 100644
+--- a/libssp/gets-chk.c
++++ b/libssp/gets-chk.c
+@@ -32,6 +32,8 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
+ <http://www.gnu.org/licenses/>.  */
+ 
+ 
++
++
+ #include "config.h"
+ #include <ssp/ssp.h>
+ #include <stdarg.h>
+'''
+
+EXPECTED4 = '''\
+gcc/ChangeLog:
+
+	* ipa-icf.c:
+	* varasm.c:
+
+libssp/ChangeLog:
+
+	* gets-chk.c:
+
+gcc/testsuite/ChangeLog:
+
+	* gcc.dg/pr40209.c:
+	* gcc.dg/pr63567-1.c:
+	* gcc.dg/pr32374.c: Removed.
+	* gcc.dg/pr50209.c: New test.
+
+'''
+
+PATCH5 = '''\
+diff --git a/gcc/testsuite/gcc.target/i386/pr95046-6.c b/gcc/testsuite/gcc.target/i386/pr95046-6.c
+new file mode 100644
+index 00000000000..dcc8999c446
+--- /dev/null
++++ b/gcc/testsuite/gcc.target/i386/pr95046-6.c
+@@ -0,0 +1,44 @@
++/* PR target/95046 */
++/* { dg-do compile { target { ! ia32 } } } */
++/* { dg-options "-O3 -mavx512vl" } */
++
++
++double r[2];
++int s[2];
++unsigned int u[2];
++
++void
++test_float (void)
++{
++  for (int i = 0; i < 2; i++)
++    r[i] = s[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvtdq2pd" } } */
++
++void
++test_ufloat (void)
++{
++  for (int i = 0; i < 2; i++)
++    r[i] = u[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvtudq2pd" } } */
++
++void
++test_fix (void)
++{
++  for (int i = 0; i < 2; i++)
++    s[i] = r[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvttpd2dqx" } } */
++
++void
++test_ufix (void)
++{
++  for (int i = 0; i < 2; i++)
++    u[i] = r[i];
++}
++
++/* { dg-final { scan-assembler "\tvcvttpd2udqx" } } */
+-- 
+2.26.2
+
+'''
+
+EXPECTED5 = '''\
+PR target/95046 - Vectorize V2SFmode operations
+
+gcc/testsuite/ChangeLog:
+
+	PR target/95046
+	* gcc.target/i386/pr95046-6.c: New test.
+
+'''
+
+class TestMklog(unittest.TestCase):
+    def test_macro_definition(self):
+        changelog = generate_changelog(PATCH1)
+        assert changelog == EXPECTED1
+
+    def test_changed_argument(self):
+        changelog = generate_changelog(PATCH2)
+        assert changelog == EXPECTED2
+
+    def test_enum_and_struct(self):
+        changelog = generate_changelog(PATCH3)
+        assert changelog == EXPECTED3
+
+    def test_no_function(self):
+        changelog = generate_changelog(PATCH3, True)
+        assert changelog == EXPECTED3B
+
+    def test_sorting(self):
+        changelog = generate_changelog(PATCH4)
+        assert changelog == EXPECTED4
+
+    def test_pr_bugzilla_download(self):
+        changelog = generate_changelog(PATCH5, fill_pr_titles=True)
+        assert changelog == EXPECTED5
-- 
2.26.2


[-- Attachment #3: 0001-Move-2-mklog-scripts-to-legacy-subfolder.patch --]
[-- Type: text/x-patch, Size: 799 bytes --]

From 28bbebabc1d4849555ef62d79de034850a4f069d Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Tue, 19 May 2020 11:02:03 +0200
Subject: [PATCH 1/2] Move 2 mklog scripts to legacy subfolder.

contrib/ChangeLog:

	* legacy/mklog: Moved from mklog.
	* legacy/mklog.pl: Moved from mklog.pl.
---
 contrib/{ => legacy}/mklog    | 0
 contrib/{ => legacy}/mklog.pl | 0
 2 files changed, 0 insertions(+), 0 deletions(-)
 rename contrib/{ => legacy}/mklog (100%)
 rename contrib/{ => legacy}/mklog.pl (100%)

diff --git a/contrib/mklog b/contrib/legacy/mklog
similarity index 100%
rename from contrib/mklog
rename to contrib/legacy/mklog
diff --git a/contrib/mklog.pl b/contrib/legacy/mklog.pl
similarity index 100%
rename from contrib/mklog.pl
rename to contrib/legacy/mklog.pl
-- 
2.26.2

Re: New mklog script

[Joseph Myers]

On Tue, 19 May 2020, Martin Liška wrote:

> On 5/19/20 10:11 AM, Martin Liška wrote:
> > Can you please share how do you do it? It would be easy to add it.
> 
> I added the feature via --fill-up-bug-titles option. It uses common
> request and beatifulsoup packages.

The REST interface is much better to use for extracting bug data than 
screen scraping of HTML output.  Fetch e.g. 
https://gcc.gnu.org/bugzilla/rest.cgi/bug?id=12345&include_fields=summary 
to get JSON bug data (change or omit include_fields if you want more than 
just the summary).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: New mklog script

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 690 bytes --]

On 5/19/20 5:53 PM, Joseph Myers wrote:
> On Tue, 19 May 2020, Martin Liška wrote:
> 
>> On 5/19/20 10:11 AM, Martin Liška wrote:
>>> Can you please share how do you do it? It would be easy to add it.
>>
>> I added the feature via --fill-up-bug-titles option. It uses common
>> request and beatifulsoup packages.
> 
> The REST interface is much better to use for extracting bug data than
> screen scraping of HTML output.  Fetch e.g.
> https://gcc.gnu.org/bugzilla/rest.cgi/bug?id=12345&include_fields=summary
> to get JSON bug data (change or omit include_fields if you want more than
> just the summary).
> 

You are right, there's a patch I'm going to install.

Martin


[-- Attachment #2: 0001-Use-REST-API-for-bug-titles-in-mklog.patch --]
[-- Type: text/x-patch, Size: 1860 bytes --]

From b5a89069a074aff0ae94176c676eda069ff0a1c3 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Tue, 19 May 2020 21:14:36 +0200
Subject: [PATCH] Use REST API for bug titles in mklog.

contrib/ChangeLog:

2020-05-19  Martin Liska  <mliska@suse.cz>

	* mklog.py: Use REST API for bug title downloading.
---
 contrib/mklog.py | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/contrib/mklog.py b/contrib/mklog.py
index 45559afbe6b..b27fad0ca2e 100755
--- a/contrib/mklog.py
+++ b/contrib/mklog.py
@@ -31,8 +31,6 @@ import os
 import re
 import sys
 
-import bs4
-
 import requests
 
 from unidiff import PatchSet
@@ -46,6 +44,8 @@ macro_regex = re.compile(r'#\s*(define|undef)\s+([a-zA-Z0-9_]+)')
 super_macro_regex = re.compile(r'^DEF[A-Z0-9_]+\s*\(([a-zA-Z0-9_]+)')
 fn_regex = re.compile(r'([a-zA-Z_][^()\s]*)\s*\([^*]')
 template_and_param_regex = re.compile(r'<[^<>]*>')
+bugzilla_url = 'https://gcc.gnu.org/bugzilla/rest.cgi/bug?id=%s&' \
+               'include_fields=summary'
 
 function_extensions = set(['.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def'])
 
@@ -106,18 +106,16 @@ def sort_changelog_files(changed_file):
 
 
 def get_pr_titles(prs):
-    if not prs:
-        return ''
-
     output = ''
     for pr in prs:
         id = pr.split('/')[-1]
-        r = requests.get('https://gcc.gnu.org/PR%s' % id)
-        html = bs4.BeautifulSoup(r.text, features='lxml')
-        title = html.title.text
-        title = title[title.find('–') + 1:].strip()
-        output += '%s - %s\n' % (pr, title)
-    output += '\n'
+        r = requests.get(bugzilla_url % id)
+        bugs = r.json()['bugs']
+        if len(bugs) == 1:
+            output += '%s - %s\n' % (pr, bugs[0]['summary'])
+            print(output)
+    if output:
+        output += '\n'
     return output
 
 
-- 
2.26.2

Re: New mklog script

[Jonathan Wakely]

On Tue, 19 May 2020 at 17:13, Joseph Myers wrote:
>
> On Tue, 19 May 2020, Martin Liška wrote:
>
> > On 5/19/20 10:11 AM, Martin Liška wrote:
> > > Can you please share how do you do it? It would be easy to add it.
> >
> > I added the feature via --fill-up-bug-titles option. It uses common
> > request and beatifulsoup packages.
>
> The REST interface is much better to use for extracting bug data than
> screen scraping of HTML output.  Fetch e.g.
> https://gcc.gnu.org/bugzilla/rest.cgi/bug?id=12345&include_fields=summary
> to get JSON bug data (change or omit include_fields if you want more than
> just the summary).

REST+JSON is probably better for the mklog.py script, but in case this
is useful to anybody else, I get bugzilla info by fetching the XML for
a bug using the URL
https://gcc.gnu.org/bugzilla/show_bug.cgi?ctype=xml&id=NNNN and
applying this stylesheet:

    <?xml version="1.0" encoding="utf-8"?>
    <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
version="1.0">
    <xsl:output method="text"/>
    <xsl:template match="/">
      <xsl:apply-templates select="/bugzilla/bug/short_desc"/>
      <xsl:text> </xsl:text>
      <xsl:apply-templates select="/bugzilla/bug/bug_status"/>
      <xsl:text> </xsl:text>
      <xsl:apply-templates select="/bugzilla/bug/resolution"/>
      <xsl:text>
    </xsl:text>
    </xsl:template>
    </xsl:stylesheet>

Re: New mklog script

[Martin Liška]

Hello Martin.

Can you please compare the current mklog.py. Is there anything
you miss compared to your current script?

Thanks,
Martin

Re: New mklog script

[Martin Sebor]

On 5/21/20 2:16 AM, Martin Liška wrote:
> Hello Martin.
> 
> Can you please compare the current mklog.py. Is there anything
> you miss compared to your current script?

Nope, it matches the format I get with my script and even works
better and runs faster.  Very nice!  I'll be happy to switch to
using it instead.

Thanks!
Martin

PS A couple of ideas for future enhancements are to have the script
print "New function." or "New type." for newly added functions and
types, and to print "Adjust comments." for changes to comments alone.

Re: New mklog script

[Thomas Koenig]

Hi,

what's currently in trunk (as of a few hours ago) fails for me with

   File "contrib/mklog.py", line 36, in <module>
     from unidiff import PatchSet
ModuleNotFoundError: No module named 'unidiff'

I think this is an error which would have to be taken into account
one way or another - maybe include it in download_prerequisites ?

Regards

	Thomas

Re: New mklog script

[Jonathan Wakely]

On Fri, 22 May 2020 at 19:15, Thomas Koenig via Gcc <gcc@gcc.gnu.org> wrote:
>
> Hi,
>
> what's currently in trunk (as of a few hours ago) fails for me with
>
>    File "contrib/mklog.py", line 36, in <module>
>      from unidiff import PatchSet
> ModuleNotFoundError: No module named 'unidiff'
>
> I think this is an error which would have to be taken into account
> one way or another - maybe include it in download_prerequisites ?

No, most people do not want download_prerequisites to start installing
python packages, they should use their system's package manager.


>
> Regards
>
>         Thomas

Re: New mklog script

[Martin Liška]

On 5/22/20 6:43 PM, Martin Sebor wrote:
> On 5/21/20 2:16 AM, Martin Liška wrote:
>> Hello Martin.
>>
>> Can you please compare the current mklog.py. Is there anything
>> you miss compared to your current script?
> 
> Nope, it matches the format I get with my script and even works
> better and runs faster.  Very nice!  I'll be happy to switch to
> using it instead.

Great, good to hear!

> 
> Thanks!
> Martin
> 
> PS A couple of ideas for future enhancements are to have the script
> print "New function." or "New type." for newly added functions and
> types, and to print "Adjust comments." for changes to comments alone.

Feel free to send patches for the new script ;)

Martin

Re: Automatically generated ChangeLog files - script

[Pierre-Marie de Rodat]

[-- Attachment #1: Type: text/plain, Size: 1342 bytes --]

Hello Martin,

First, thank you for your work on this new ChangeLog workflow. :-)

I’d like to report a “regression”: I can’t push the attached patch:
> remote: *** ChangeLog format failed:
> remote: ERR: changed file not mentioned in a ChangeLog:"gcc/ada/sem_ch4.adb"
> remote: ERR: changed file not mentioned in a ChangeLog:"gcc/ada/sem_ch7.adb"
> remote: ERR: changed file not mentioned in a ChangeLog:"gcc/ada/sem_ch8.adb"
> remote: ERR: changed file not mentioned in a ChangeLog:"gcc/ada/sem_elab.adb"
> remote: ERR: changed file not mentioned in a ChangeLog:"gcc/ada/sem_type.adb"
> remote: ERR: changed file not mentioned in a ChangeLog:"gcc/ada/sem_util.adb"
> remote: 
> remote: Please see: https://gcc.gnu.org/codingconventions.html#ChangeLogs
> remote: 
> remote: error: hook declined to update refs/heads/master

It looks like the hook does not accept multi-line ChangeLog entries 
affecting multiple files:
>         * contracts.adb, einfo.adb, exp_ch9.adb, sem_ch12.adb,
>         sem_ch4.adb, sem_ch7.adb, sem_ch8.adb, sem_elab.adb,
>         sem_type.adb, sem_util.adb: Reuse Is_Package_Or_Generic_Package
>         where possible (similarly, reuse Is_Concurrent_Type if it was
>         possible in the same expressions).

Would it be possible to enhance the hook to support that?

Thanks!

-- 
Pierre-Marie de Rodat

[-- Attachment #2: 0001-Ada-Reuse-Is_Package_Or_Generic_Package-where-possib.patch --]
[-- Type: text/x-patch, Size: 11230 bytes --]

From 557b268fffffdeb0980a17411f458eee333f55c6 Mon Sep 17 00:00:00 2001
From: Piotr Trojanek <trojanek@adacore.com>
Date: Thu, 12 Dec 2019 11:45:24 +0100
Subject: [PATCH] [Ada] Reuse Is_Package_Or_Generic_Package where possible

2020-05-26  Piotr Trojanek  <trojanek@adacore.com>

gcc/ada/

	* contracts.adb, einfo.adb, exp_ch9.adb, sem_ch12.adb,
	sem_ch4.adb, sem_ch7.adb, sem_ch8.adb, sem_elab.adb,
	sem_type.adb, sem_util.adb: Reuse Is_Package_Or_Generic_Package
	where possible (similarly, reuse Is_Concurrent_Type if it was
	possible in the same expressions).
---
 gcc/ada/contracts.adb |  2 +-
 gcc/ada/einfo.adb     | 22 +++++++++++-----------
 gcc/ada/exp_ch9.adb   |  2 +-
 gcc/ada/sem_ch12.adb  |  2 +-
 gcc/ada/sem_ch4.adb   |  2 +-
 gcc/ada/sem_ch7.adb   |  6 +++---
 gcc/ada/sem_ch8.adb   |  6 +++---
 gcc/ada/sem_elab.adb  |  2 +-
 gcc/ada/sem_type.adb  |  2 +-
 gcc/ada/sem_util.adb  |  6 +++---
 10 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/gcc/ada/contracts.adb b/gcc/ada/contracts.adb
index 981bb91..d58f136 100644
--- a/gcc/ada/contracts.adb
+++ b/gcc/ada/contracts.adb
@@ -213,7 +213,7 @@ package body Contracts is
       --    Initializes
       --    Part_Of (instantiation only)
 
-      elsif Ekind_In (Id, E_Generic_Package, E_Package) then
+      elsif Is_Package_Or_Generic_Package (Id) then
          if Nam_In (Prag_Nam, Name_Abstract_State,
                               Name_Initial_Condition,
                               Name_Initializes)
diff --git a/gcc/ada/einfo.adb b/gcc/ada/einfo.adb
index 98b508f..1df8ed0 100644
--- a/gcc/ada/einfo.adb
+++ b/gcc/ada/einfo.adb
@@ -713,7 +713,7 @@ package body Einfo is
 
    function Abstract_States (Id : E) return L is
    begin
-      pragma Assert (Ekind_In (Id, E_Generic_Package, E_Package));
+      pragma Assert (Is_Package_Or_Generic_Package (Id));
       return Elist25 (Id);
    end Abstract_States;
 
@@ -837,7 +837,7 @@ package body Einfo is
 
    function Body_Entity (Id : E) return E is
    begin
-      pragma Assert (Ekind_In (Id, E_Package, E_Generic_Package));
+      pragma Assert (Is_Package_Or_Generic_Package (Id));
       return Node19 (Id);
    end Body_Entity;
 
@@ -1424,8 +1424,8 @@ package body Einfo is
 
    function First_Private_Entity (Id : E) return E is
    begin
-      pragma Assert (Ekind_In (Id, E_Package, E_Generic_Package)
-                       or else Ekind (Id) in Concurrent_Kind);
+      pragma Assert (Is_Package_Or_Generic_Package (Id)
+                       or else Is_Concurrent_Type (Id));
       return Node16 (Id);
    end First_Private_Entity;
 
@@ -3044,7 +3044,7 @@ package body Einfo is
 
    function Package_Instantiation (Id : E) return N is
    begin
-      pragma Assert (Ekind_In (Id, E_Package, E_Generic_Package));
+      pragma Assert (Is_Package_Or_Generic_Package (Id));
       return Node26 (Id);
    end Package_Instantiation;
 
@@ -3883,7 +3883,7 @@ package body Einfo is
 
    procedure Set_Abstract_States (Id : E; V : L) is
    begin
-      pragma Assert (Ekind_In (Id, E_Generic_Package, E_Package));
+      pragma Assert (Is_Package_Or_Generic_Package (Id));
       Set_Elist25 (Id, V);
    end Set_Abstract_States;
 
@@ -4006,7 +4006,7 @@ package body Einfo is
 
    procedure Set_Body_Entity (Id : E; V : E) is
    begin
-      pragma Assert (Ekind_In (Id, E_Package, E_Generic_Package));
+      pragma Assert (Is_Package_Or_Generic_Package (Id));
       Set_Node19 (Id, V);
    end Set_Body_Entity;
 
@@ -4593,8 +4593,8 @@ package body Einfo is
 
    procedure Set_First_Private_Entity (Id : E; V : E) is
    begin
-      pragma Assert (Ekind_In (Id, E_Package, E_Generic_Package)
-                       or else Ekind (Id) in Concurrent_Kind);
+      pragma Assert (Is_Package_Or_Generic_Package (Id)
+                       or else Is_Concurrent_Type (Id));
       Set_Node16 (Id, V);
    end Set_First_Private_Entity;
 
@@ -7827,7 +7827,7 @@ package body Einfo is
 
    function Has_Non_Null_Abstract_State (Id : E) return B is
    begin
-      pragma Assert (Ekind_In (Id, E_Generic_Package, E_Package));
+      pragma Assert (Is_Package_Or_Generic_Package (Id));
 
       return
         Present (Abstract_States (Id))
@@ -7863,7 +7863,7 @@ package body Einfo is
    -----------------------------
 
    function Has_Null_Abstract_State (Id : E) return B is
-      pragma Assert (Ekind_In (Id, E_Generic_Package, E_Package));
+      pragma Assert (Is_Package_Or_Generic_Package (Id));
 
       States : constant Elist_Id := Abstract_States (Id);
 
diff --git a/gcc/ada/exp_ch9.adb b/gcc/ada/exp_ch9.adb
index 64ac353..392a221 100644
--- a/gcc/ada/exp_ch9.adb
+++ b/gcc/ada/exp_ch9.adb
@@ -6167,7 +6167,7 @@ package body Exp_Ch9 is
                --  this safe. This is a common (if dubious) idiom.
 
                elsif S = Scope (Prot)
-                 and then Ekind_In (S, E_Package, E_Generic_Package)
+                 and then Is_Package_Or_Generic_Package (S)
                  and then Nkind (Parent (E)) = N_Object_Declaration
                  and then Nkind (Parent (Parent (E))) = N_Package_Body
                then
diff --git a/gcc/ada/sem_ch12.adb b/gcc/ada/sem_ch12.adb
index dc3a3c2..209e060 100644
--- a/gcc/ada/sem_ch12.adb
+++ b/gcc/ada/sem_ch12.adb
@@ -10364,7 +10364,7 @@ package body Sem_Ch12 is
       --  such as a parent generic within the body of a generic child.
 
       if not Is_Entity_Name (Actual)
-        or else not Ekind_In (Entity (Actual), E_Generic_Package, E_Package)
+        or else not Is_Package_Or_Generic_Package (Entity (Actual))
       then
          Error_Msg_N
            ("expect package instance to instantiate formal", Actual);
diff --git a/gcc/ada/sem_ch4.adb b/gcc/ada/sem_ch4.adb
index 5910112..702f265 100644
--- a/gcc/ada/sem_ch4.adb
+++ b/gcc/ada/sem_ch4.adb
@@ -9480,7 +9480,7 @@ package body Sem_Ch4 is
             Type_Scope : constant Entity_Id := Scope (T);
             Op_List    : Elist_Id := Primitive_Operations (T);
          begin
-            if Ekind_In (Type_Scope, E_Package, E_Generic_Package)
+            if Is_Package_Or_Generic_Package (Type_Scope)
               and then ((In_Package_Body (Type_Scope)
               and then In_Open_Scopes (Type_Scope)) or else In_Instance_Body)
             then
diff --git a/gcc/ada/sem_ch7.adb b/gcc/ada/sem_ch7.adb
index 6d9a1db..f217dfd 100644
--- a/gcc/ada/sem_ch7.adb
+++ b/gcc/ada/sem_ch7.adb
@@ -2428,7 +2428,7 @@ package body Sem_Ch7 is
       --  defined in the associated package, subject to at least one Part_Of
       --  constituent.
 
-      if Ekind_In (P, E_Generic_Package, E_Package) then
+      if Is_Package_Or_Generic_Package (P) then
          declare
             States     : constant Elist_Id := Abstract_States (P);
             State_Elmt : Elmt_Id;
@@ -3322,7 +3322,7 @@ package body Sem_Ch7 is
       --  performed if the caller requests this behavior.
 
       if Do_Abstract_States
-        and then Ekind_In (Pack_Id, E_Generic_Package, E_Package)
+        and then Is_Package_Or_Generic_Package (Pack_Id)
         and then Has_Non_Null_Abstract_State (Pack_Id)
         and then Requires_Body
       then
@@ -3380,7 +3380,7 @@ package body Sem_Ch7 is
       --  provided). If Ignore_Abstract_State is True, we don't do this check
       --  (so we can use Unit_Requires_Body to check for some other reason).
 
-      elsif Ekind_In (Pack_Id, E_Generic_Package, E_Package)
+      elsif Is_Package_Or_Generic_Package (Pack_Id)
         and then Present (Abstract_States (Pack_Id))
         and then not Is_Null_State
                        (Node (First_Elmt (Abstract_States (Pack_Id))))
diff --git a/gcc/ada/sem_ch8.adb b/gcc/ada/sem_ch8.adb
index f083f7c..7f50b40 100644
--- a/gcc/ada/sem_ch8.adb
+++ b/gcc/ada/sem_ch8.adb
@@ -5950,7 +5950,7 @@ package body Sem_Ch8 is
 
                   --  Package or generic package is always a simple reference
 
-                  if Ekind_In (E, E_Package, E_Generic_Package) then
+                  if Is_Package_Or_Generic_Package (E) then
                      Generate_Reference (E, N, 'r');
 
                   --  Else see if we have a left hand side
@@ -8779,7 +8779,7 @@ package body Sem_Ch8 is
 
       --  Set Default_Storage_Pool field of the library unit if necessary
 
-      if Ekind_In (S, E_Package, E_Generic_Package)
+      if Is_Package_Or_Generic_Package (S)
         and then
           Nkind (Parent (Unit_Declaration_Node (S))) = N_Compilation_Unit
       then
@@ -8949,7 +8949,7 @@ package body Sem_Ch8 is
 
       if Is_Child_Unit (S)
         and then Present (E)
-        and then Ekind_In (E, E_Package, E_Generic_Package)
+        and then Is_Package_Or_Generic_Package (E)
         and then
           Nkind (Parent (Unit_Declaration_Node (E))) = N_Compilation_Unit
       then
diff --git a/gcc/ada/sem_elab.adb b/gcc/ada/sem_elab.adb
index f3cac46..dbf3fac 100644
--- a/gcc/ada/sem_elab.adb
+++ b/gcc/ada/sem_elab.adb
@@ -13826,7 +13826,7 @@ package body Sem_Elab is
          --  be on another machine.
 
          if Ekind (Body_Id) = E_Package_Body
-           and then Ekind_In (Spec_Id, E_Generic_Package, E_Package)
+           and then Is_Package_Or_Generic_Package (Spec_Id)
            and then (Is_Remote_Call_Interface (Spec_Id)
                       or else Is_Remote_Types (Spec_Id))
          then
diff --git a/gcc/ada/sem_type.adb b/gcc/ada/sem_type.adb
index e5d01dd..1868568 100644
--- a/gcc/ada/sem_type.adb
+++ b/gcc/ada/sem_type.adb
@@ -1383,7 +1383,7 @@ package body Sem_Type is
       begin
          return In_Same_List (Parent (Typ), Op_Decl)
            or else
-             (Ekind_In (Scop, E_Package, E_Generic_Package)
+             (Is_Package_Or_Generic_Package (Scop)
                and then List_Containing (Op_Decl) =
                               Visible_Declarations (Parent (Scop))
                and then List_Containing (Parent (Typ)) =
diff --git a/gcc/ada/sem_util.adb b/gcc/ada/sem_util.adb
index b980b4c..c1b1d9e 100644
--- a/gcc/ada/sem_util.adb
+++ b/gcc/ada/sem_util.adb
@@ -3407,7 +3407,7 @@ package body Sem_Util is
          --  Stop the traversal when a package subject to a null abstract state
          --  has been found.
 
-         if Ekind_In (Context, E_Generic_Package, E_Package)
+         if Is_Package_Or_Generic_Package (Context)
            and then Has_Null_Abstract_State (Context)
          then
             exit;
@@ -12978,7 +12978,7 @@ package body Sem_Util is
 
       begin
          if Present (Pkg)
-           and then Ekind_In (Pkg, E_Generic_Package, E_Package)
+           and then Is_Package_Or_Generic_Package (Pkg)
          then
             while Nkind (Pkg_Decl) /= N_Package_Specification loop
                Pkg_Decl := Parent (Pkg_Decl);
@@ -25283,7 +25283,7 @@ package body Sem_Util is
               or else
                 (Present (Scope (Val))
                   and then Is_Implementation_Defined (Scope (Val))))
-           and then not (Ekind_In (Val, E_Package, E_Generic_Package)
+           and then not (Is_Package_Or_Generic_Package (Val)
                           and then Is_Library_Level_Entity (Val))
          then
             Check_Restriction (No_Implementation_Identifiers, Post_Node);
-- 
2.1.4

Re: Automatically generated ChangeLog files - script

[Martin Liška]

On 5/26/20 12:15 PM, Pierre-Marie de Rodat wrote:
> Hello Martin,
> 
> First, thank you for your work on this new ChangeLog workflow. :-)

Hello.

Thank you.

> 
> I’d like to report a “regression”: I can’t push the attached patch:
>> remote: *** ChangeLog format failed:
>> remote: ERR: changed file not mentioned in a ChangeLog:"gcc/ada/sem_ch4.adb"
>> remote: ERR: changed file not mentioned in a ChangeLog:"gcc/ada/sem_ch7.adb"
>> remote: ERR: changed file not mentioned in a ChangeLog:"gcc/ada/sem_ch8.adb"
>> remote: ERR: changed file not mentioned in a ChangeLog:"gcc/ada/sem_elab.adb"
>> remote: ERR: changed file not mentioned in a ChangeLog:"gcc/ada/sem_type.adb"
>> remote: ERR: changed file not mentioned in a ChangeLog:"gcc/ada/sem_util.adb"
>> remote: remote: Please see: https://gcc.gnu.org/codingconventions.html#ChangeLogs
>> remote: remote: error: hook declined to update refs/heads/master
> 
> It looks like the hook does not accept multi-line ChangeLog entries affecting multiple files:
>>         * contracts.adb, einfo.adb, exp_ch9.adb, sem_ch12.adb,
>>         sem_ch4.adb, sem_ch7.adb, sem_ch8.adb, sem_elab.adb,
>>         sem_type.adb, sem_util.adb: Reuse Is_Package_Or_Generic_Package
>>         where possible (similarly, reuse Is_Concurrent_Type if it was
>>         possible in the same expressions).
> 
> Would it be possible to enhance the hook to support that?

It's not supported right now and it will make the filename parsing much more complicated.
I would recommend using ./contrib/mklog and using:

* a.adb: Foo bar.
* b.adb: Likewise.
...

Thanks,
Martin

> 
> Thanks!
> 

Re: Automatically generated ChangeLog files - script

[Rainer Orth]

Hi Martin,

>> It looks like the hook does not accept multi-line ChangeLog entries
>> affecting multiple files:
>>>         * contracts.adb, einfo.adb, exp_ch9.adb, sem_ch12.adb,
>>>         sem_ch4.adb, sem_ch7.adb, sem_ch8.adb, sem_elab.adb,
>>>         sem_type.adb, sem_util.adb: Reuse Is_Package_Or_Generic_Package
>>>         where possible (similarly, reuse Is_Concurrent_Type if it was
>>>         possible in the same expressions).
>>
>> Would it be possible to enhance the hook to support that?
>
> It's not supported right now and it will make the filename parsing much
> more complicated.

however, that's a format Emacs' ChangeLog mode uses and supports, and
it's way less chatty than the one-file-per-line one, both for writers
and readers.

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: Automatically generated ChangeLog files - script

[Martin Liška]

On 5/26/20 2:35 PM, Rainer Orth wrote:
> Hi Martin,
> 
>>> It looks like the hook does not accept multi-line ChangeLog entries
>>> affecting multiple files:
>>>>          * contracts.adb, einfo.adb, exp_ch9.adb, sem_ch12.adb,
>>>>          sem_ch4.adb, sem_ch7.adb, sem_ch8.adb, sem_elab.adb,
>>>>          sem_type.adb, sem_util.adb: Reuse Is_Package_Or_Generic_Package
>>>>          where possible (similarly, reuse Is_Concurrent_Type if it was
>>>>          possible in the same expressions).
>>>
>>> Would it be possible to enhance the hook to support that?
>>
>> It's not supported right now and it will make the filename parsing much
>> more complicated.
> 
> however, that's a format Emacs' ChangeLog mode uses and supports, and
> it's way less chatty than the one-file-per-line one, both for writers
> and readers.

I see, but as mentioned it makes the parsing of the list files much more
complicated. Feel free to provide a patch that will support multi-line
entries.

Martin

> 
> 	Rainer
> 

Re: Automatically generated ChangeLog files - script

[Pierre-Marie de Rodat]

On 26/05/2020 15:09, Martin Liška wrote:
> I see, but as mentioned it makes the parsing of the list files much more
> complicated. Feel free to provide a patch that will support multi-line
> entries.

Thanks! As long as there is no objection to support this, I have no 
problem giving it a try. :-) I just sent two patches on gcc-patches@. By 
the way, is there a delay between the time the gcc-changelog scripts are 
updated and the time they are used in pre-commit hooks?

-- 
Pierre-Marie de Rodat

Re: Automatically generated ChangeLog files - script

[Martin Liška]

On 5/26/20 4:50 PM, Pierre-Marie de Rodat wrote:
> On 26/05/2020 15:09, Martin Liška wrote:
>> I see, but as mentioned it makes the parsing of the list files much more
>> complicated. Feel free to provide a patch that will support multi-line
>> entries.
> 
> Thanks! As long as there is no objection to support this, I have no problem giving it a try. :-) I just sent two patches on gcc-patches@. By the way, is there a delay between the time the gcc-changelog scripts are updated and the time they are used in pre-commit hooks?

The script needs to be synced by somebody to the server.
I asked Jakub to do it for me.

Martin

> 

Re: Automatically generated ChangeLog files - script

[Alexandre Oliva]

On May 26, 2020, Martin Liška <mliska@suse.cz> wrote:

> On 5/26/20 12:15 PM, Pierre-Marie de Rodat wrote:
>>>         * contracts.adb, einfo.adb, exp_ch9.adb, sem_ch12.adb,

> It's not supported right now and it will make the filename parsing
> much more complicated.

Another colleague recently run into a problem with either:

	* $filename <$case>:

or

	* $filename [$condition]:

I can't recall which one it was, but the following patch is supposed to
implement both.  Alas, I couldn't figure out how to test it:
git_check_commit.py is failing with:

Traceback (most recent call last):
  File "contrib/gcc-changelog/git_check_commit.py", line 38, in <module>
    not args.non_strict_mode):
  File "/l/tmp/build/gcc/contrib/gcc-changelog/git_repository.py", line 57, in parse_git_revisions
    elif file.renamed_file:
AttributeError: 'Diff' object has no attribute 'renamed_file'


accept <case> and [cond] in ChangeLog

From: Alexandre Oliva <oliva@adacore.com>

Only '(' and ':' currently terminate file lists in ChangeLog entries
in the ChangeLog parser.  This rules out such legitimate entries as:

	* filename <CASE>:
	* filename [COND]:

This patch extends the ChangeLog parser to recognize these forms.


for  contrib/ChangeLog

	* gcc-changelog/git_commit.py: Support CASE and COND.
---
 contrib/gcc-changelog/git_commit.py |   16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
index 4a78793..537c667 100755
--- a/contrib/gcc-changelog/git_commit.py
+++ b/contrib/gcc-changelog/git_commit.py
@@ -154,6 +154,7 @@ changelog_regex = re.compile(r'^(?:[fF]or +)?([a-z0-9+-/]*)ChangeLog:?')
 pr_regex = re.compile(r'\tPR (?P<component>[a-z+-]+\/)?([0-9]+)$')
 dr_regex = re.compile(r'\tDR ([0-9]+)$')
 star_prefix_regex = re.compile(r'\t\*(?P<spaces>\ *)(?P<content>.*)')
+end_of_location_regex = re.compile(r'[[<(:]')
 
 LINE_LIMIT = 100
 TAB_WIDTH = 8
@@ -203,14 +204,13 @@ class ChangeLogEntry:
                 line = m.group('content')
 
             if in_location:
-                # Strip everything that is not a filename in "line": entities
-                # "(NAME)", entry text (the colon, if present, and anything
-                # that follows it).
-                if '(' in line:
-                    line = line[:line.index('(')]
-                    in_location = False
-                if ':' in line:
-                    line = line[:line.index(':')]
+                # Strip everything that is not a filename in "line":
+                # entities "(NAME)", cases "<PATTERN>", conditions
+                # "[COND]", entry text (the colon, if present, and
+                # anything that follows it).
+                m = end_of_location_regex.search(line)
+                if m:
+                    line = line[:m.start()]
                     in_location = False
 
                 # At this point, all that's left is a list of filenames


-- 
Alexandre Oliva, freedom fighter    he/him    https://FSFLA.org/blogs/lxo/
Free Software Evangelist              Stallman was right, but he's left :(
GNU Toolchain Engineer           Live long and free, and prosper ethically

Re: Automatically generated ChangeLog files - script

[Martin Liška]

On 6/22/20 3:15 PM, Alexandre Oliva wrote:
> On May 26, 2020, Martin Liška <mliska@suse.cz> wrote:
> 
>> On 5/26/20 12:15 PM, Pierre-Marie de Rodat wrote:
>>>>          * contracts.adb, einfo.adb, exp_ch9.adb, sem_ch12.adb,
> 
>> It's not supported right now and it will make the filename parsing
>> much more complicated.

Hello.

I support the patch:

> 
> Another colleague recently run into a problem with either:
> 
> 	* $filename <$case>:
> 
> or
> 
> 	* $filename [$condition]:
> 
> I can't recall which one it was, but the following patch is supposed to
> implement both.  Alas, I couldn't figure out how to test it:
> git_check_commit.py is failing with:
> 
> Traceback (most recent call last):
>    File "contrib/gcc-changelog/git_check_commit.py", line 38, in <module>
>      not args.non_strict_mode):
>    File "/l/tmp/build/gcc/contrib/gcc-changelog/git_repository.py", line 57, in parse_git_revisions
>      elif file.renamed_file:
> AttributeError: 'Diff' object has no attribute 'renamed_file'
> 
> 
> accept <case> and [cond] in ChangeLog
> 
> From: Alexandre Oliva <oliva@adacore.com>
> 
> Only '(' and ':' currently terminate file lists in ChangeLog entries
> in the ChangeLog parser.  This rules out such legitimate entries as:
> 
> 	* filename <CASE>:
> 	* filename [COND]:
> 
> This patch extends the ChangeLog parser to recognize these forms.
> 
> 
> for  contrib/ChangeLog
> 
> 	* gcc-changelog/git_commit.py: Support CASE and COND.
> ---
>   contrib/gcc-changelog/git_commit.py |   16 ++++++++--------
>   1 file changed, 8 insertions(+), 8 deletions(-)
> 
> diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
> index 4a78793..537c667 100755
> --- a/contrib/gcc-changelog/git_commit.py
> +++ b/contrib/gcc-changelog/git_commit.py
> @@ -154,6 +154,7 @@ changelog_regex = re.compile(r'^(?:[fF]or +)?([a-z0-9+-/]*)ChangeLog:?')
>   pr_regex = re.compile(r'\tPR (?P<component>[a-z+-]+\/)?([0-9]+)$')
>   dr_regex = re.compile(r'\tDR ([0-9]+)$')
>   star_prefix_regex = re.compile(r'\t\*(?P<spaces>\ *)(?P<content>.*)')
> +end_of_location_regex = re.compile(r'[[<(:]')

Please escape the '[':
+end_of_location_regex = re.compile(r'[\[<(:]')

and please a test-case for it.

Thanks,
Martin

>   
>   LINE_LIMIT = 100
>   TAB_WIDTH = 8
> @@ -203,14 +204,13 @@ class ChangeLogEntry:
>                   line = m.group('content')
>   
>               if in_location:
> -                # Strip everything that is not a filename in "line": entities
> -                # "(NAME)", entry text (the colon, if present, and anything
> -                # that follows it).
> -                if '(' in line:
> -                    line = line[:line.index('(')]
> -                    in_location = False
> -                if ':' in line:
> -                    line = line[:line.index(':')]
> +                # Strip everything that is not a filename in "line":
> +                # entities "(NAME)", cases "<PATTERN>", conditions
> +                # "[COND]", entry text (the colon, if present, and
> +                # anything that follows it).
> +                m = end_of_location_regex.search(line)
> +                if m:
> +                    line = line[:m.start()]
>                       in_location = False
>   
>                   # At this point, all that's left is a list of filenames
> 
> 

Re: Automatically generated ChangeLog files - script

[Martin Liška]

@Alexander: May I please remind you this?

Martin

On 6/24/20 10:28 AM, Martin Liška wrote:
> On 6/22/20 3:15 PM, Alexandre Oliva wrote:
>> On May 26, 2020, Martin Liška <mliska@suse.cz> wrote:
>>
>>> On 5/26/20 12:15 PM, Pierre-Marie de Rodat wrote:
>>>>>          * contracts.adb, einfo.adb, exp_ch9.adb, sem_ch12.adb,
>>
>>> It's not supported right now and it will make the filename parsing
>>> much more complicated.
> 
> Hello.
> 
> I support the patch:
> 
>>
>> Another colleague recently run into a problem with either:
>>
>>     * $filename <$case>:
>>
>> or
>>
>>     * $filename [$condition]:
>>
>> I can't recall which one it was, but the following patch is supposed to
>> implement both.  Alas, I couldn't figure out how to test it:
>> git_check_commit.py is failing with:
>>
>> Traceback (most recent call last):
>>    File "contrib/gcc-changelog/git_check_commit.py", line 38, in <module>
>>      not args.non_strict_mode):
>>    File "/l/tmp/build/gcc/contrib/gcc-changelog/git_repository.py", line 57, in parse_git_revisions
>>      elif file.renamed_file:
>> AttributeError: 'Diff' object has no attribute 'renamed_file'
>>
>>
>> accept <case> and [cond] in ChangeLog
>>
>> From: Alexandre Oliva <oliva@adacore.com>
>>
>> Only '(' and ':' currently terminate file lists in ChangeLog entries
>> in the ChangeLog parser.  This rules out such legitimate entries as:
>>
>>     * filename <CASE>:
>>     * filename [COND]:
>>
>> This patch extends the ChangeLog parser to recognize these forms.
>>
>>
>> for  contrib/ChangeLog
>>
>>     * gcc-changelog/git_commit.py: Support CASE and COND.
>> ---
>>   contrib/gcc-changelog/git_commit.py |   16 ++++++++--------
>>   1 file changed, 8 insertions(+), 8 deletions(-)
>>
>> diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
>> index 4a78793..537c667 100755
>> --- a/contrib/gcc-changelog/git_commit.py
>> +++ b/contrib/gcc-changelog/git_commit.py
>> @@ -154,6 +154,7 @@ changelog_regex = re.compile(r'^(?:[fF]or +)?([a-z0-9+-/]*)ChangeLog:?')
>>   pr_regex = re.compile(r'\tPR (?P<component>[a-z+-]+\/)?([0-9]+)$')
>>   dr_regex = re.compile(r'\tDR ([0-9]+)$')
>>   star_prefix_regex = re.compile(r'\t\*(?P<spaces>\ *)(?P<content>.*)')
>> +end_of_location_regex = re.compile(r'[[<(:]')
> 
> Please escape the '[':
> +end_of_location_regex = re.compile(r'[\[<(:]')
> 
> and please a test-case for it.
> 
> Thanks,
> Martin
> 
>>   LINE_LIMIT = 100
>>   TAB_WIDTH = 8
>> @@ -203,14 +204,13 @@ class ChangeLogEntry:
>>                   line = m.group('content')
>>               if in_location:
>> -                # Strip everything that is not a filename in "line": entities
>> -                # "(NAME)", entry text (the colon, if present, and anything
>> -                # that follows it).
>> -                if '(' in line:
>> -                    line = line[:line.index('(')]
>> -                    in_location = False
>> -                if ':' in line:
>> -                    line = line[:line.index(':')]
>> +                # Strip everything that is not a filename in "line":
>> +                # entities "(NAME)", cases "<PATTERN>", conditions
>> +                # "[COND]", entry text (the colon, if present, and
>> +                # anything that follows it).
>> +                m = end_of_location_regex.search(line)
>> +                if m:
>> +                    line = line[:m.start()]
>>                       in_location = False
>>                   # At this point, all that's left is a list of filenames
>>
>>
> 

Re: Automatically generated ChangeLog files - script

[Alexandre Oliva]

Sorry it took me so long to get back to this.

On Jun 24, 2020, Martin Liška <mliska@suse.cz> wrote:

> Please escape the '[':
> +end_of_location_regex = re.compile(r'[\[<(:]')

check

> and please a test-case for it.

check

Thanks, I've made the changes; sorry it took me so long.

I couldn't figure out how to run the internal gcc-changelog test.


accept <case> and [cond] in ChangeLog

From: Alexandre Oliva <oliva@adacore.com>

Only '(' and ':' currently terminate file lists in ChangeLog entries
in the ChangeLog parser.  This rules out such legitimate entries as:

	* filename <CASE>:
	* filename [COND]:

This patch extends the ChangeLog parser to recognize these forms.


for  contrib/ChangeLog

	* gcc-changelog/git_commit.py: Support CASE and COND.
	* gcc-changelog/test_patches.txt: Add test.
---
 contrib/gcc-changelog/git_commit.py    |   16 +++++++--------
 contrib/gcc-changelog/test_patches.txt |   35 ++++++++++++++++++++++++++++++++
 2 files changed, 43 insertions(+), 8 deletions(-)

diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
index 4a78793..900a294 100755
--- a/contrib/gcc-changelog/git_commit.py
+++ b/contrib/gcc-changelog/git_commit.py
@@ -154,6 +154,7 @@ changelog_regex = re.compile(r'^(?:[fF]or +)?([a-z0-9+-/]*)ChangeLog:?')
 pr_regex = re.compile(r'\tPR (?P<component>[a-z+-]+\/)?([0-9]+)$')
 dr_regex = re.compile(r'\tDR ([0-9]+)$')
 star_prefix_regex = re.compile(r'\t\*(?P<spaces>\ *)(?P<content>.*)')
+end_of_location_regex = re.compile(r'[\[<(:]')
 
 LINE_LIMIT = 100
 TAB_WIDTH = 8
@@ -203,14 +204,13 @@ class ChangeLogEntry:
                 line = m.group('content')
 
             if in_location:
-                # Strip everything that is not a filename in "line": entities
-                # "(NAME)", entry text (the colon, if present, and anything
-                # that follows it).
-                if '(' in line:
-                    line = line[:line.index('(')]
-                    in_location = False
-                if ':' in line:
-                    line = line[:line.index(':')]
+                # Strip everything that is not a filename in "line":
+                # entities "(NAME)", cases "<PATTERN>", conditions
+                # "[COND]", entry text (the colon, if present, and
+                # anything that follows it).
+                m = end_of_location_regex.search(line)
+                if m:
+                    line = line[:m.start()]
                     in_location = False
 
                 # At this point, all that's left is a list of filenames
diff --git a/contrib/gcc-changelog/test_patches.txt b/contrib/gcc-changelog/test_patches.txt
index 1463fb9..2bf5d1a 100644
--- a/contrib/gcc-changelog/test_patches.txt
+++ b/contrib/gcc-changelog/test_patches.txt
@@ -3160,3 +3160,38 @@ index 823eb539993..4ec22162c12 100644
 -- 
 2.27.0
 
+=== 0001-Check-for-more-missing-math-decls-on-vxworks.patch ===
+From 0edfc1fd22405ee8e946101e44cd8edc0ee12047 Mon Sep 17 00:00:00 2001
+From: Douglas B Rupp <douglas.b.rupp@gmail.com>
+Date: Sun, 31 May 2020 13:25:28 -0700
+Subject: [PATCH] Check for more missing math decls on vxworks.
+
+Use the GLIBCXX_CHECK_MATH_DECL macro to check for the full list of
+vxworks math decls.
+
+for libstdc++-v3/ChangeLog:
+
+	* crossconfig.m4 <*-vxworks>: Check for more math decls.
+	* configure [FAKEPATCH]: Rebuild.
+---
+ libstdc++-v3/configure      | 255 ++++++++++++++++++++++++++++++++++++++++++++
+ libstdc++-v3/crossconfig.m4 |   3 +-
+ 2 files changed, 257 insertions(+), 1 deletion(-)
+
+diff --git a/libstdc++-v3/configure b/libstdc++-v3/configure
+index b5beb45..4ef678e 100755
+--- a/libstdc++-v3/configure
++++ b/libstdc++-v3/configure
+@@ -1 +1,2 @@
+ 
++
+diff --git a/libstdc++-v3/crossconfig.m4 b/libstdc++-v3/crossconfig.m4
+index fe18288..313f84d 100644
+--- a/libstdc++-v3/crossconfig.m4
++++ b/libstdc++-v3/crossconfig.m4
+@@ -1 +1,2 @@
+ 
++
+-- 
+2.7.4
+


-- 
Alexandre Oliva, freedom fighter    he/him    https://FSFLA.org/blogs/lxo/
Free Software Evangelist              Stallman was right, but he's left :(
GNU Toolchain Engineer           Live long and free, and prosper ethically

Re: Automatically generated ChangeLog files - script

[Martin Liška]

On 7/7/20 4:14 AM, Alexandre Oliva wrote:
> Sorry it took me so long to get back to this.

Don't worry!

> 
> On Jun 24, 2020, Martin Liška <mliska@suse.cz> wrote:
> 
>> Please escape the '[':
>> +end_of_location_regex = re.compile(r'[\[<(:]')
> 
> check
> 
>> and please a test-case for it.
> 
> check
> 
> Thanks, I've made the changes; sorry it took me so long.
> 
> I couldn't figure out how to run the internal gcc-changelog test.

I've just installed your patch and added the test, please see the change
in test_email.py in:
https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=a759bfc7cf238b9fc5bf97884297fc69d8cdf2b5

Thank you for the patch,
Martin

> 
> 
> accept <case> and [cond] in ChangeLog
> 
> From: Alexandre Oliva <oliva@adacore.com>
> 
> Only '(' and ':' currently terminate file lists in ChangeLog entries
> in the ChangeLog parser.  This rules out such legitimate entries as:
> 
> 	* filename <CASE>:
> 	* filename [COND]:
> 
> This patch extends the ChangeLog parser to recognize these forms.
> 
> 
> for  contrib/ChangeLog
> 
> 	* gcc-changelog/git_commit.py: Support CASE and COND.
> 	* gcc-changelog/test_patches.txt: Add test.
> ---
>   contrib/gcc-changelog/git_commit.py    |   16 +++++++--------
>   contrib/gcc-changelog/test_patches.txt |   35 ++++++++++++++++++++++++++++++++
>   2 files changed, 43 insertions(+), 8 deletions(-)
> 
> diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
> index 4a78793..900a294 100755
> --- a/contrib/gcc-changelog/git_commit.py
> +++ b/contrib/gcc-changelog/git_commit.py
> @@ -154,6 +154,7 @@ changelog_regex = re.compile(r'^(?:[fF]or +)?([a-z0-9+-/]*)ChangeLog:?')
>   pr_regex = re.compile(r'\tPR (?P<component>[a-z+-]+\/)?([0-9]+)$')
>   dr_regex = re.compile(r'\tDR ([0-9]+)$')
>   star_prefix_regex = re.compile(r'\t\*(?P<spaces>\ *)(?P<content>.*)')
> +end_of_location_regex = re.compile(r'[\[<(:]')
>   
>   LINE_LIMIT = 100
>   TAB_WIDTH = 8
> @@ -203,14 +204,13 @@ class ChangeLogEntry:
>                   line = m.group('content')
>   
>               if in_location:
> -                # Strip everything that is not a filename in "line": entities
> -                # "(NAME)", entry text (the colon, if present, and anything
> -                # that follows it).
> -                if '(' in line:
> -                    line = line[:line.index('(')]
> -                    in_location = False
> -                if ':' in line:
> -                    line = line[:line.index(':')]
> +                # Strip everything that is not a filename in "line":
> +                # entities "(NAME)", cases "<PATTERN>", conditions
> +                # "[COND]", entry text (the colon, if present, and
> +                # anything that follows it).
> +                m = end_of_location_regex.search(line)
> +                if m:
> +                    line = line[:m.start()]
>                       in_location = False
>   
>                   # At this point, all that's left is a list of filenames
> diff --git a/contrib/gcc-changelog/test_patches.txt b/contrib/gcc-changelog/test_patches.txt
> index 1463fb9..2bf5d1a 100644
> --- a/contrib/gcc-changelog/test_patches.txt
> +++ b/contrib/gcc-changelog/test_patches.txt
> @@ -3160,3 +3160,38 @@ index 823eb539993..4ec22162c12 100644
>   --
>   2.27.0
>   
> +=== 0001-Check-for-more-missing-math-decls-on-vxworks.patch ===
> +From 0edfc1fd22405ee8e946101e44cd8edc0ee12047 Mon Sep 17 00:00:00 2001
> +From: Douglas B Rupp <douglas.b.rupp@gmail.com>
> +Date: Sun, 31 May 2020 13:25:28 -0700
> +Subject: [PATCH] Check for more missing math decls on vxworks.
> +
> +Use the GLIBCXX_CHECK_MATH_DECL macro to check for the full list of
> +vxworks math decls.
> +
> +for libstdc++-v3/ChangeLog:
> +
> +	* crossconfig.m4 <*-vxworks>: Check for more math decls.
> +	* configure [FAKEPATCH]: Rebuild.
> +---
> + libstdc++-v3/configure      | 255 ++++++++++++++++++++++++++++++++++++++++++++
> + libstdc++-v3/crossconfig.m4 |   3 +-
> + 2 files changed, 257 insertions(+), 1 deletion(-)
> +
> +diff --git a/libstdc++-v3/configure b/libstdc++-v3/configure
> +index b5beb45..4ef678e 100755
> +--- a/libstdc++-v3/configure
> ++++ b/libstdc++-v3/configure
> +@@ -1 +1,2 @@
> +
> ++
> +diff --git a/libstdc++-v3/crossconfig.m4 b/libstdc++-v3/crossconfig.m4
> +index fe18288..313f84d 100644
> +--- a/libstdc++-v3/crossconfig.m4
> ++++ b/libstdc++-v3/crossconfig.m4
> +@@ -1 +1,2 @@
> +
> ++
> +--
> +2.7.4
> +
> 
>