From: Tobias Burnus <tobias@codesourcery.com>
To: "Martin Sebor" <msebor@gmail.com>,
"Jason Merrill" <jason@redhat.com>,
"Martin Liška" <mliska@suse.cz>
Cc: Jakub Jelinek <jakub@redhat.com>,
gcc Mailing List <gcc@gcc.gnu.org>,
Jonathan Wakely <jwakely@redhat.com>,
gcc-patches <gcc-patches@gcc.gnu.org>
Subject: [Patch, v2] contrib/mklog.py: Improve PR handling (was: Re: git gcc-commit-mklog doesn't extract PR number to ChangeLog)
Date: Mon, 21 Jun 2021 09:54:41 +0200 [thread overview]
Message-ID: <3c66efd4-eb5e-f2bb-6138-4126b5909c9c@codesourcery.com> (raw)
In-Reply-To: <71b4a023-efb2-6c6a-9ced-93cce7c96540@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2417 bytes --]
On 17.06.21 02:17, Martin Sebor via Gcc wrote:
> @@ -147,6 +152,12 @@ def generate_changelog(data, no_functions=False, fill_pr_titles=False):
>
> # Extract PR entries from newly added tests
> if 'testsuite' in file.path and file.is_added_file:
> + name = os.path.basename(file.path)
> + name = os.path.splitext(name)[0]
> + if name.startswith("pr"):
> + name = name[2:]
> + name = "PR " + name
> + prs.append(name)
I think you need a regular expression to extract the PR – as it will both match
too much and to little. We have file names such as:
* libstdc++-pr91488.C (other prefix)
* PR37039.f90 (capitalized PR)
* pr98218-1.C (suffix with '-')
* pr40724_1.f (suffix with '_')
* pr101023a.C (suffix with a letter)
But otherwise, I like that idea.
* * *
Changes in my patch compared to v1:
- (From Martin's patch:) Extract the PR from new-files file
name (using pattern matching), but only take the PR if the
PR wasn't found in the file as PR comment.
(The latter happens, e.g., with b376b1ef389.)
- Avoid printing the same PR multiple times as summary line
(duplicates occur due to 'PR 134' vs. 'PR comp/123' vs.
'PR othercomp/123') — This does not avoid all issues but at least
some. If this becomes a real world issue, we can try harder.
OK to commit this one? — Comments?
* * *
I did leave out other changes as they seem to be less clear cut,
and which can be still be handled as follow up. Like:
- Adding 'Resolves:' (as in some cases it only resolves part of
the PR)
- ... other changes/patches I missed. (This thread has too many
emails.) In particular, if
^PR <comp>/<pr> - ....
is accepted by gcc-commit/, then there is no need to list the
PRs individually later on. But currently, it is still required.
* * *
Cross ref:
* v1 of my patch was at
https://gcc.gnu.org/pipermail/gcc/2021-June/236498.html
* Discussion of the -b option is at
https://gcc.gnu.org/pipermail/gcc/2021-June/236519.html
* Martin S's patch (partially quoted above) is at
https://gcc.gnu.org/pipermail/gcc/2021-June/236460.html
Tobias
-----------------
Mentor Graphics (Deutschland) GmbH, Arnulfstrasse 201, 80634 München Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Frank Thürauf
[-- Attachment #2: mklog-p-v2.diff --]
[-- Type: text/x-patch, Size: 5558 bytes --]
contrib/mklog.py: Improve PR handling
Co-authored-by: Martin Sebor <msebor@redhat.com>
contrib/ChangeLog:
* mklog.py (bugzilla_url): Fetch also component.
(pr_filename_regex): New.
(get_pr_titles): Update PR string with correct format and component.
(generate_changelog): Take additional PRs; extract PR from the
filename.
(__main__): Add -b/--pr-numbers argument.
contrib/mklog.py | 41 ++++++++++++++++++++++++++++++++---------
1 file changed, 32 insertions(+), 9 deletions(-)
diff --git a/contrib/mklog.py b/contrib/mklog.py
index 1f59055e723..bba6c1a0e1a 100755
--- a/contrib/mklog.py
+++ b/contrib/mklog.py
@@ -42,6 +42,7 @@ pr_regex = re.compile(r'(\/(\/|\*)|[Cc*!])\s+(?P<pr>PR [a-z+-]+\/[0-9]+)')
prnum_regex = re.compile(r'PR (?P<comp>[a-z+-]+)/(?P<num>[0-9]+)')
dr_regex = re.compile(r'(\/(\/|\*)|[Cc*!])\s+(?P<dr>DR [0-9]+)')
dg_regex = re.compile(r'{\s+dg-(error|warning)')
+pr_filename_regex = re.compile(r'(^|[\W_])[Pp][Rr](?P<pr>\d{4,})')
identifier_regex = re.compile(r'^([a-zA-Z0-9_#].*)')
comment_regex = re.compile(r'^\/\*')
struct_regex = re.compile(r'^(class|struct|union|enum)\s+'
@@ -52,7 +53,7 @@ fn_regex = re.compile(r'([a-zA-Z_][^()\s]*)\s*\([^*]')
template_and_param_regex = re.compile(r'<[^<>]*>')
md_def_regex = re.compile(r'\(define.*\s+"(.*)"')
bugzilla_url = 'https://gcc.gnu.org/bugzilla/rest.cgi/bug?id=%s&' \
- 'include_fields=summary'
+ 'include_fields=summary,component'
function_extensions = {'.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def', '.md'}
@@ -118,20 +119,23 @@ def sort_changelog_files(changed_file):
def get_pr_titles(prs):
- output = ''
- for pr in prs:
+ output = []
+ for idx, pr in enumerate(prs):
pr_id = pr.split('/')[-1]
r = requests.get(bugzilla_url % pr_id)
bugs = r.json()['bugs']
if len(bugs) == 1:
- output += '%s - %s\n' % (pr, bugs[0]['summary'])
- print(output)
+ prs[idx] = 'PR %s/%s' % (bugs[0]['component'], pr_id)
+ out = '%s - %s\n' % (prs[idx], bugs[0]['summary'])
+ if out not in output:
+ output.append(out)
if output:
- output += '\n'
- return output
+ output.append('')
+ return '\n'.join(output)
-def generate_changelog(data, no_functions=False, fill_pr_titles=False):
+def generate_changelog(data, no_functions=False, fill_pr_titles=False,
+ additional_prs=None):
changelogs = {}
changelog_list = []
prs = []
@@ -139,6 +143,8 @@ def generate_changelog(data, no_functions=False, fill_pr_titles=False):
diff = PatchSet(data)
global firstpr
+ if additional_prs:
+ prs = [pr for pr in additional_prs if pr not in prs]
for file in diff:
# skip files that can't be parsed
if file.path == '/dev/null':
@@ -154,21 +160,33 @@ def generate_changelog(data, no_functions=False, fill_pr_titles=False):
# Only search first ten lines as later lines may
# contains commented code which a note that it
# has not been tested due to a certain PR or DR.
+ this_file_prs = []
for line in list(file)[0][0:10]:
m = pr_regex.search(line.value)
if m:
pr = m.group('pr')
if pr not in prs:
prs.append(pr)
+ this_file_prs.append(pr.split('/')[-1])
else:
m = dr_regex.search(line.value)
if m:
dr = m.group('dr')
if dr not in prs:
prs.append(dr)
+ this_file_prs.append(dr.split('/')[-1])
elif dg_regex.search(line.value):
# Found dg-warning/dg-error line
break
+ # PR number in the file name
+ fname = os.path.basename(file.path)
+ fname = os.path.splitext(fname)[0]
+ m = pr_filename_regex.search(fname)
+ if m:
+ pr = m.group('pr')
+ pr2 = "PR " + pr
+ if pr not in this_file_prs and pr2 not in prs:
+ prs.append(pr2)
if prs:
firstpr = prs[0]
@@ -286,6 +304,8 @@ if __name__ == '__main__':
parser = argparse.ArgumentParser(description=help_message)
parser.add_argument('input', nargs='?',
help='Patch file (or missing, read standard input)')
+ parser.add_argument('-b', '--pr-numbers', action='append',
+ help='Add the specified PRs (comma separated)')
parser.add_argument('-s', '--no-functions', action='store_true',
help='Do not generate function names in ChangeLogs')
parser.add_argument('-p', '--fill-up-bug-titles', action='store_true',
@@ -308,8 +328,11 @@ if __name__ == '__main__':
if args.update_copyright:
update_copyright(data)
else:
+ pr_numbers = args.pr_numbers
+ if pr_numbers:
+ pr_numbers = [b for i in args.pr_numbers for b in i.split(',')]
output = generate_changelog(data, args.no_functions,
- args.fill_up_bug_titles)
+ args.fill_up_bug_titles, pr_numbers)
if args.changelog:
lines = open(args.changelog).read().split('\n')
start = list(takewhile(lambda l: not l.startswith('#'), lines))
next prev parent reply other threads:[~2021-06-21 7:54 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <4c1114a7-2377-99e4-d451-1a086857e991@linux.ibm.com>
2021-06-10 5:22 ` git gcc-commit-mklog doesn't extract PR number to ChangeLog Xionghu Luo
2021-06-10 6:17 ` Martin Liška
2021-06-10 6:25 ` Xionghu Luo
2021-06-10 8:07 ` Martin Liška
2021-06-10 6:35 ` Tobias Burnus
2021-06-10 8:07 ` Martin Liška
2021-06-10 9:44 ` Jonathan Wakely
2021-06-10 10:01 ` Jonathan Wakely
2021-06-10 10:08 ` Jakub Jelinek
2021-06-10 10:40 ` Jonathan Wakely
2021-06-10 14:55 ` Martin Sebor
2021-06-10 15:54 ` Tobias Burnus
2021-06-10 16:05 ` Jonathan Wakely
2021-06-10 15:56 ` Jonathan Wakely
2021-06-10 17:06 ` Martin Sebor
2021-06-10 17:20 ` Martin Sebor
2021-06-10 17:30 ` Jakub Jelinek
2021-06-10 18:55 ` Martin Sebor
2021-06-10 19:09 ` Jakub Jelinek
2021-06-10 21:16 ` Martin Sebor
2021-06-10 21:28 ` Jakub Jelinek
2021-06-10 21:56 ` Martin Sebor
2021-06-11 9:13 ` Jonathan Wakely
2021-06-11 17:02 ` Martin Sebor
2021-06-11 17:05 ` Jakub Jelinek
2021-06-11 17:32 ` Jonathan Wakely
2021-06-11 18:01 ` Martin Sebor
2021-06-11 18:14 ` Jonathan Wakely
2021-06-16 0:56 ` Hans-Peter Nilsson
2021-06-16 2:03 ` Martin Sebor
2021-06-16 3:42 ` Jason Merrill
2021-06-16 14:31 ` Martin Sebor
2021-06-16 20:49 ` Jason Merrill
2021-06-16 21:45 ` Martin Sebor
2021-06-16 23:45 ` Jason Merrill
2021-06-17 0:17 ` Martin Sebor
2021-06-17 0:40 ` Jason Merrill
2021-06-17 1:01 ` Martin Sebor
2021-06-17 1:46 ` Jason Merrill
2021-06-17 10:18 ` Jonathan Wakely
2021-06-17 14:55 ` Martin Sebor
2021-06-17 15:11 ` Michael Matz
2021-06-17 15:33 ` Martin Sebor
2021-06-17 16:31 ` Jakub Jelinek
2021-06-17 16:32 ` Jonathan Wakely
2021-06-17 18:00 ` Martin Sebor
2021-06-17 10:08 ` Richard Earnshaw
2021-06-17 17:12 ` Joseph Myers
2021-06-17 17:21 ` Jason Merrill
2021-06-17 17:21 ` Jakub Jelinek
2021-06-18 9:32 ` Richard Earnshaw
2021-06-18 11:05 ` [Patch] contrib/mklog.py: Improve PR handling (was: git gcc-commit-mklog doesn't extract PR number to ChangeLog) Tobias Burnus
2021-06-18 11:10 ` Jonathan Wakely
2021-06-18 11:24 ` Jakub Jelinek
2021-06-18 11:25 ` Tobias Burnus
2021-06-18 11:40 ` Jonathan Wakely
2021-06-21 7:28 ` Martin Liška
2021-06-18 16:40 ` [Patch] contrib/mklog.py: Improve PR handling Martin Sebor
2021-06-18 14:41 ` [Patch] contrib/mklog.py: Improve PR handling (was: git gcc-commit-mklog doesn't extract PR number to ChangeLog) Jason Merrill
2021-06-18 16:47 ` [Patch] contrib/mklog.py: Improve PR handling Martin Sebor
2021-06-18 16:59 ` Iain Sandoe
2021-06-21 6:42 ` [Patch] contrib/mklog.py: Improve PR handling (was: git gcc-commit-mklog doesn't extract PR number to ChangeLog) Tobias Burnus
2021-06-21 7:26 ` Martin Liška
2021-06-21 8:02 ` Iain Sandoe
2021-06-21 7:54 ` Tobias Burnus [this message]
2021-06-21 8:09 ` [Patch, v2] contrib/mklog.py: Improve PR handling (was: " Martin Liška
2021-06-21 8:37 ` Tobias Burnus
2021-06-21 12:53 ` Martin Liška
2021-06-21 13:26 ` Tobias Burnus
2021-06-22 7:30 ` [RFC][PATCH] contrib: add git-commit-mklog wrapper Martin Liška
2021-06-22 8:23 ` Tobias Burnus
2021-06-22 8:31 ` Martin Liška
2021-06-22 18:40 ` Jason Merrill
2021-06-23 7:40 ` Martin Liška
2021-06-16 13:46 ` git gcc-commit-mklog doesn't extract PR number to ChangeLog Jonathan Wakely
2021-06-16 17:44 ` Hans-Peter Nilsson
2021-06-11 9:08 ` Jonathan Wakely
2021-06-11 9:35 ` Jonathan Wakely
2021-06-11 15:43 ` Joseph Myers
2021-06-11 17:02 ` Jonathan Wakely
2021-06-10 11:51 ` [Patch] contrig/gcc-changelog: Check that PR in subject in in changelog (was:: git gcc-commit-mklog doesn't extract PR number to ChangeLog) Tobias Burnus
2021-06-10 11:54 ` [Patch] contrig/gcc-changelog: Check that PR in subject in in changelog Florian Weimer
2021-06-10 12:45 ` Jonathan Wakely
2021-06-10 9:41 ` git gcc-commit-mklog doesn't extract PR number to ChangeLog Jonathan Wakely
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3c66efd4-eb5e-f2bb-6138-4126b5909c9c@codesourcery.com \
--to=tobias@codesourcery.com \
--cc=gcc-patches@gcc.gnu.org \
--cc=gcc@gcc.gnu.org \
--cc=jakub@redhat.com \
--cc=jason@redhat.com \
--cc=jwakely@redhat.com \
--cc=mliska@suse.cz \
--cc=msebor@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).