[PATCH] gcc-changelog: Add libstdc++-v3/testsuite to wildcard prefixes

[PATCH] gcc-changelog: Add libstdc++-v3/testsuite to wildcard prefixes

[Jonathan Wakely]

[-- Attachment #1: Type: text/plain, Size: 1232 bytes --]

This allows using "testsuite/*" in libstdc++-v3/ChangeLog entries, which
was one of the original motivations for adding wildcard support in the
first place:
https://gcc.gnu.org/pipermail/gcc/2020-June/232719.html

contrib/ChangeLog:

	* gcc-changelog/git_commit.py (wildcard_prefixes): Add libstdc++
	testsuite directory.

OK to push?

This still doesn't actually meet my needs, because I'd like to be able
to do:

	* testsuite/foo/*: Something for foo tests.
	* testsuite/bar/baz/*: Something for baz tests.

So it would be nice if wildcards worked for any directory below a
wildcard prefix (as long as the directory actually exists).

It also seems that I can't have a commit where all changes are
described by a wildcard entry. It seems to require a non-wildcard one
too. But that means I can't make changes to hundreds of tests without
also making some other change, or doing something like:

	* testsuite/foo/foo-1.cc: A specific change.
        * testsuite/*: Likewise.

I suppose I can live with the latter, but it would still be nice if
this worked:

	* testsuite/foo/foo-1.cc: A specific change.
        * testsuite/foo/*: Likewise.
        * testsuite/bar/baz/*: Likewise.

i.e. sub-directories below a wildcard prefix.



[-- Attachment #2: patch.txt --]
[-- Type: text/plain, Size: 989 bytes --]

commit 5cdb6541a49f1ac716a3f1be281c9487d9b198c5
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Fri Nov 27 16:00:58 2020

    gcc-changelog: Add libstdc++-v3/testsuite to wildcard prefixes
    
    This allows using "testsuite/*" in libstdc++-v3/ChangeLog entries, which
    was one of the original motivations for adding wildcard support in the
    first place:
    https://gcc.gnu.org/pipermail/gcc/2020-June/232719.html
    
    contrib/ChangeLog:
    
            * gcc-changelog/git_commit.py (wildcard_prefixes): Add libstdc++
            testsuite directory.

diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
index d0ac23c22aa4..3ebc6fd305e0 100755
--- a/contrib/gcc-changelog/git_commit.py
+++ b/contrib/gcc-changelog/git_commit.py
@@ -138,7 +138,8 @@ ignored_prefixes = {
 
 wildcard_prefixes = {
     'gcc/testsuite/',
-    'libstdc++-v3/doc/html/'
+    'libstdc++-v3/doc/html/',
+    'libstdc++-v3/testsuite/'
     }
 
 misc_files = {

Re: [PATCH] gcc-changelog: Add libstdc++-v3/testsuite to wildcard prefixes

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 2170 bytes --]

On 11/27/20 5:23 PM, Jonathan Wakely wrote:
> This allows using "testsuite/*" in libstdc++-v3/ChangeLog entries, which
> was one of the original motivations for adding wildcard support in the
> first place:
> https://gcc.gnu.org/pipermail/gcc/2020-June/232719.html
> 
> contrib/ChangeLog:
> 
> 	* gcc-changelog/git_commit.py (wildcard_prefixes): Add libstdc++
> 	testsuite directory.
> 
> OK to push?

Hello.

It's fine, please push it.

> 
> This still doesn't actually meet my needs, because I'd like to be able
> to do:
> 
> 	* testsuite/foo/*: Something for foo tests.
> 	* testsuite/bar/baz/*: Something for baz tests.

Fixed by the patch that I've just pushed.

> 
> So it would be nice if wildcards worked for any directory below a
> wildcard prefix (as long as the directory actually exists).

It works for me now:

$ git show c41c8c19dcc17f319c1306424f8e7de4868ce037 --stat
commit c41c8c19dcc17f319c1306424f8e7de4868ce037
Author: Martin Liska <mliska@suse.cz>
Date:   Mon Nov 30 10:35:05 2020 +0100

     Test me.
     
     libstdc++-v3/ChangeLog:
     
             * doc/html/*: All you need is love.

  libstdc++-v3/doc/html/manual/debug.html | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)

$ git gcc-verify -p c41c8c19dcc17f319c1306424f8e7de4868ce037
Checking c41c8c19dcc17f319c1306424f8e7de4868ce037: OK
------ libstdc++-v3/ChangeLog ------
2020-11-30  Martin Liska  <mliska@suse.cz>

	* doc/html/*: All you need is love.

Does it also work for you now? Or does it still have a limitation?

Thanks,
Martin

> 
> It also seems that I can't have a commit where all changes are
> described by a wildcard entry. It seems to require a non-wildcard one
> too. But that means I can't make changes to hundreds of tests without
> also making some other change, or doing something like:
> 
> 	* testsuite/foo/foo-1.cc: A specific change.
>          * testsuite/*: Likewise.
> 
> I suppose I can live with the latter, but it would still be nice if
> this worked:
> 
> 	* testsuite/foo/foo-1.cc: A specific change.
>          * testsuite/foo/*: Likewise.
>          * testsuite/bar/baz/*: Likewise.
> 
> i.e. sub-directories below a wildcard prefix.
> 
> 


[-- Attachment #2: 0001-changelog-Allow-wildcard-pattern-only.patch --]
[-- Type: text/x-patch, Size: 1026 bytes --]

From fbbce1c6e98dec378955b1a591d2dff31caa01f5 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Mon, 30 Nov 2020 10:37:57 +0100
Subject: [PATCH] changelog: Allow wildcard pattern only.

contrib/ChangeLog:

	* gcc-changelog/git_commit.py: Allow wildcard pattern only.
---
 contrib/gcc-changelog/git_commit.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
index d0ac23c22aa..5366d95469c 100755
--- a/contrib/gcc-changelog/git_commit.py
+++ b/contrib/gcc-changelog/git_commit.py
@@ -560,7 +560,7 @@ class GitCommit:
         mentioned_patterns = []
         used_patterns = set()
         for entry in self.changelog_entries:
-            if not entry.files:
+            if not entry.files and not entry.file_patterns:
                 msg = 'no files mentioned for ChangeLog in directory'
                 self.errors.append(Error(msg, entry.folder))
             assert not entry.folder.endswith('/')
-- 
2.29.2

Re: [PATCH] gcc-changelog: Add libstdc++-v3/testsuite to wildcard prefixes

[Jonathan Wakely]

On Mon, 30 Nov 2020 at 09:44, Martin Liška <mliska@suse.cz> wrote:
>
> On 11/27/20 5:23 PM, Jonathan Wakely wrote:
> > This allows using "testsuite/*" in libstdc++-v3/ChangeLog entries, which
> > was one of the original motivations for adding wildcard support in the
> > first place:
> > https://gcc.gnu.org/pipermail/gcc/2020-June/232719.html
> >
> > contrib/ChangeLog:
> >
> >       * gcc-changelog/git_commit.py (wildcard_prefixes): Add libstdc++
> >       testsuite directory.
> >
> > OK to push?
>
> Hello.
>
> It's fine, please push it.

Done.

>
> >
> > This still doesn't actually meet my needs, because I'd like to be able
> > to do:
> >
> >       * testsuite/foo/*: Something for foo tests.
> >       * testsuite/bar/baz/*: Something for baz tests.
>
> Fixed by the patch that I've just pushed.
>
> >
> > So it would be nice if wildcards worked for any directory below a
> > wildcard prefix (as long as the directory actually exists).
>
> It works for me now:
>
> $ git show c41c8c19dcc17f319c1306424f8e7de4868ce037 --stat
> commit c41c8c19dcc17f319c1306424f8e7de4868ce037
> Author: Martin Liska <mliska@suse.cz>
> Date:   Mon Nov 30 10:35:05 2020 +0100
>
>      Test me.
>
>      libstdc++-v3/ChangeLog:
>
>              * doc/html/*: All you need is love.
>
>   libstdc++-v3/doc/html/manual/debug.html | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
>
> $ git gcc-verify -p c41c8c19dcc17f319c1306424f8e7de4868ce037
> Checking c41c8c19dcc17f319c1306424f8e7de4868ce037: OK
> ------ libstdc++-v3/ChangeLog ------
> 2020-11-30  Martin Liska  <mliska@suse.cz>
>
>         * doc/html/*: All you need is love.
>
> Does it also work for you now? Or does it still have a limitation?

I can have a commit which only uses wildcards now, thanks.

I still can't use sub-directories with wildcards, but I can live with that.



> Thanks,
> Martin
>
> >
> > It also seems that I can't have a commit where all changes are
> > described by a wildcard entry. It seems to require a non-wildcard one
> > too. But that means I can't make changes to hundreds of tests without
> > also making some other change, or doing something like:
> >
> >       * testsuite/foo/foo-1.cc: A specific change.
> >          * testsuite/*: Likewise.
> >
> > I suppose I can live with the latter, but it would still be nice if
> > this worked:
> >
> >       * testsuite/foo/foo-1.cc: A specific change.
> >          * testsuite/foo/*: Likewise.
> >          * testsuite/bar/baz/*: Likewise.
> >
> > i.e. sub-directories below a wildcard prefix.
> >
> >
>

Re: [PATCH] gcc-changelog: Add libstdc++-v3/testsuite to wildcard prefixes

[Martin Liška]

On 11/30/20 12:58 PM, Jonathan Wakely wrote:
> I can have a commit which only uses wildcards now, thanks.

Good!

> 
> I still can't use sub-directories with wildcards, but I can live with that.

Why? The example I presented did so:

libstdc++-v3/ChangeLog:
	* doc/html/*: All you need is love.

and I modified the following file:
libstdc++-v3/doc/html/manual/debug.html | 3 ++-

Isn't that a sub-directory?

Martin

Re: [PATCH] gcc-changelog: Add libstdc++-v3/testsuite to wildcard prefixes

[Jonathan Wakely]

On Mon, 30 Nov 2020 at 12:14, Martin Liška <mliska@suse.cz> wrote:
>
> On 11/30/20 12:58 PM, Jonathan Wakely wrote:
> > I can have a commit which only uses wildcards now, thanks.
>
> Good!
>
> >
> > I still can't use sub-directories with wildcards, but I can live with that.
>
> Why? The example I presented did so:
>
> libstdc++-v3/ChangeLog:
>         * doc/html/*: All you need is love.
>
> and I modified the following file:
> libstdc++-v3/doc/html/manual/debug.html | 3 ++-
>
> Isn't that a sub-directory?

But libstdc++/doc/html is the entry in the wildcard_prefixes list.
What I mean is using a sub-directory in the changelog:

  * doc/html/manual/*: Love is all you need.

The use case is for subsets of the testsuite. I've added
libstdc++-v3/testsuite to the wildcard_prefixes, which means I can now
do this:

 * testsuite/*: Update all 28_regex tests.

But I would like to do this:

 * testsuite/28_regex/*: Update all tests.

The concrete use case is:

* testsuite/20_util/specialized_algorithms/pstl/*: Add dg-timeout-factor.
* testsuite/25_algorithms/pstl/*: Likewise.
* testsuite/26_numerics/pstl/*: Likewise.
* testsuite/28_regex/*: Likewise.

I'm modifying four specific sub-directories, so I'd like to say that
in the changelog. Currently to allow that I'd have to add every
directory under libstdc++-v3/testsuite to the wildcard_prefixes (which
doesn't scale) or just be vague instead of saying which
sub-directories are affected:

* testsuite/*: Add dg-timeout-factor to most pstl and regex tests.

Re: [PATCH] gcc-changelog: Add libstdc++-v3/testsuite to wildcard prefixes

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 1736 bytes --]

On 11/30/20 1:23 PM, Jonathan Wakely wrote:
> On Mon, 30 Nov 2020 at 12:14, Martin Liška <mliska@suse.cz> wrote:
>>
>> On 11/30/20 12:58 PM, Jonathan Wakely wrote:
>>> I can have a commit which only uses wildcards now, thanks.
>>
>> Good!
>>
>>>
>>> I still can't use sub-directories with wildcards, but I can live with that.
>>
>> Why? The example I presented did so:
>>
>> libstdc++-v3/ChangeLog:
>>          * doc/html/*: All you need is love.
>>
>> and I modified the following file:
>> libstdc++-v3/doc/html/manual/debug.html | 3 ++-
>>
>> Isn't that a sub-directory?
> 
> But libstdc++/doc/html is the entry in the wildcard_prefixes list.
> What I mean is using a sub-directory in the changelog:
> 
>    * doc/html/manual/*: Love is all you need.
> 
> The use case is for subsets of the testsuite. I've added
> libstdc++-v3/testsuite to the wildcard_prefixes, which means I can now
> do this:
> 
>   * testsuite/*: Update all 28_regex tests.
> 
> But I would like to do this:
> 
>   * testsuite/28_regex/*: Update all tests.

Ahh, got it. Fix with the attached patch I've just pushed.

Hope it's fine now?
Martin

> 
> The concrete use case is:
> 
> * testsuite/20_util/specialized_algorithms/pstl/*: Add dg-timeout-factor.
> * testsuite/25_algorithms/pstl/*: Likewise.
> * testsuite/26_numerics/pstl/*: Likewise.
> * testsuite/28_regex/*: Likewise.
> 
> I'm modifying four specific sub-directories, so I'd like to say that
> in the changelog. Currently to allow that I'd have to add every
> directory under libstdc++-v3/testsuite to the wildcard_prefixes (which
> doesn't scale) or just be vague instead of saying which
> sub-directories are affected:
> 
> * testsuite/*: Add dg-timeout-factor to most pstl and regex tests.
> 


[-- Attachment #2: 0001-changelog-allow-subdir-wildcard-entry.patch --]
[-- Type: text/x-patch, Size: 4026 bytes --]

From 1e5c644fb62e3e1757cf74fe8a14873d5e8ba53e Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Mon, 30 Nov 2020 13:41:26 +0100
Subject: [PATCH] changelog: allow subdir wildcard entry

contrib/ChangeLog:

	* gcc-changelog/git_commit.py: Allow sub-directory wildcard
	changelog entry. Fix a typo caused by apostrophe escaping.
	* gcc-changelog/test_email.py: Test it.
	* gcc-changelog/test_patches.txt: Likewise.
---
 contrib/gcc-changelog/git_commit.py    |  4 ++--
 contrib/gcc-changelog/test_email.py    |  9 ++++++++-
 contrib/gcc-changelog/test_patches.txt | 24 ++++++++++++++++++++++++
 3 files changed, 34 insertions(+), 3 deletions(-)

diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py
index 6487de3271d..57fba756d32 100755
--- a/contrib/gcc-changelog/git_commit.py
+++ b/contrib/gcc-changelog/git_commit.py
@@ -493,7 +493,7 @@ class GitCommit:
         for entry in self.changelog_entries:
             for pattern in entry.file_patterns:
                 name = os.path.join(entry.folder, pattern)
-                if name not in wildcard_prefixes:
+                if not [name.startswith(pr) for pr in wildcard_prefixes]:
                     msg = 'unsupported wildcard prefix'
                     self.errors.append(Error(msg, name))
 
@@ -617,7 +617,7 @@ class GitCommit:
 
         for pattern in mentioned_patterns:
             if pattern not in used_patterns:
-                error = 'pattern doesn''t match any changed files'
+                error = "pattern doesn't match any changed files"
                 self.errors.append(Error(error, pattern))
 
     def check_for_correct_changelog(self):
diff --git a/contrib/gcc-changelog/test_email.py b/contrib/gcc-changelog/test_email.py
index 579194cdc24..48ecc1ee6a6 100755
--- a/contrib/gcc-changelog/test_email.py
+++ b/contrib/gcc-changelog/test_email.py
@@ -333,7 +333,7 @@ class TestGccChangelog(unittest.TestCase):
         assert not email.errors
         email = self.from_patch_glob('0002-libstdc-Fake-test-change-1.patch')
         assert len(email.errors) == 1
-        msg = 'pattern doesn''t match any changed files'
+        msg = "pattern doesn't match any changed files"
         assert email.errors[0].message == msg
         assert email.errors[0].line == 'libstdc++-v3/doc/html/'
         email = self.from_patch_glob('0003-libstdc-Fake-test-change-2.patch')
@@ -386,3 +386,10 @@ class TestGccChangelog(unittest.TestCase):
         email = self.from_patch_glob('0001-lto-fix-LTO-debug')
         assert not email.errors
         assert len(email.changelog_entries) == 1
+
+    def test_wildcard_in_subdir(self):
+        email = self.from_patch_glob('0001-Wildcard-subdirs.patch')
+        assert len(email.changelog_entries) == 1
+        err = email.errors[0]
+        assert err.message == "pattern doesn't match any changed files"
+        assert err.line == 'libstdc++-v3/testsuite/28_regex_not-existing/'
diff --git a/contrib/gcc-changelog/test_patches.txt b/contrib/gcc-changelog/test_patches.txt
index 9e9c5cd893f..02ce28d20b8 100644
--- a/contrib/gcc-changelog/test_patches.txt
+++ b/contrib/gcc-changelog/test_patches.txt
@@ -3320,3 +3320,27 @@ index 7c9d492f6a4..37e73348cb7 100644
 -- 
 2.25.1
 
+=== 0001-Wildcard-subdirs.patch ===
+From b798205595426c53eb362065f6ed6c320dcc161d Mon Sep 17 00:00:00 2001
+From: Martin Liska <mliska@suse.cz>
+Date: Mon, 30 Nov 2020 13:27:51 +0100
+Subject: [PATCH] Fix it.
+
+libstdc++-v3/ChangeLog:
+
+	* testsuite/28_regex/*: Fix them all.
+	* testsuite/28_regex_not-existing/*: Fix them all.
+---
+ contrib/gcc-changelog/git_commit.py          | 1 +
+ libstdc++-v3/testsuite/28_regex/init-list.cc | 1 +
+ 2 files changed, 2 insertions(+)
+
+diff --git a/libstdc++-v3/testsuite/28_regex/init-list.cc b/libstdc++-v3/testsuite/28_regex/init-list.cc
+index f51453f019a..d10ecf483f4 100644
+--- a/libstdc++-v3/testsuite/28_regex/init-list.cc
++++ b/libstdc++-v3/testsuite/28_regex/init-list.cc
+@@ -1 +1,2 @@
+ 
++
+--
+2.29.2
-- 
2.29.2

Re: [PATCH] gcc-changelog: Add libstdc++-v3/testsuite to wildcard prefixes

[Jonathan Wakely]

On Mon, 30 Nov 2020 at 12:43, Martin Liška <mliska@suse.cz> wrote:
>
> On 11/30/20 1:23 PM, Jonathan Wakely wrote:
> > On Mon, 30 Nov 2020 at 12:14, Martin Liška <mliska@suse.cz> wrote:
> >>
> >> On 11/30/20 12:58 PM, Jonathan Wakely wrote:
> >>> I can have a commit which only uses wildcards now, thanks.
> >>
> >> Good!
> >>
> >>>
> >>> I still can't use sub-directories with wildcards, but I can live with that.
> >>
> >> Why? The example I presented did so:
> >>
> >> libstdc++-v3/ChangeLog:
> >>          * doc/html/*: All you need is love.
> >>
> >> and I modified the following file:
> >> libstdc++-v3/doc/html/manual/debug.html | 3 ++-
> >>
> >> Isn't that a sub-directory?
> >
> > But libstdc++/doc/html is the entry in the wildcard_prefixes list.
> > What I mean is using a sub-directory in the changelog:
> >
> >    * doc/html/manual/*: Love is all you need.
> >
> > The use case is for subsets of the testsuite. I've added
> > libstdc++-v3/testsuite to the wildcard_prefixes, which means I can now
> > do this:
> >
> >   * testsuite/*: Update all 28_regex tests.
> >
> > But I would like to do this:
> >
> >   * testsuite/28_regex/*: Update all tests.
>
> Ahh, got it. Fix with the attached patch I've just pushed.
>
> Hope it's fine now?

Yes, this is perfect, thanks!