[Patch] contrib/mklog.py: Improve PR handling (was: git gcc-commit-mklog doesn't extract PR number to ChangeLog)

[Tobias Burnus <tobias@codesourcery.com>]

[-- Attachment #1: Type: text/plain, Size: 2357 bytes --]

On 18.06.21 11:32, Richard Earnshaw via Gcc wrote:
> On 17/06/2021 18:21, Jakub Jelinek wrote:
>> mklog as is doesn't fill in the details (descriptions of the changes
>> to each function etc.), nor is realiable in many cases, and with Jason's
>> recent change just fills in the first and last part of the first line
>> but not the important middle part.
>> So, the developer has to hand edit it anyway and that I'd consider also
>> be the right time when the verification whether the PR being mentioned
>> is the right one etc.  So no need to add a question asked by the script
>> at another point.
> That misses my point.  If we use a tool to help doing this we can make
> the tool also scrape the entry out of bugzilla and print the summary
> line as a stronger visual check that the number has been typed
> correctly.  We get many bug attributions wrong simply because two digits
> have been transposed: visually checking the summary line is a far
> stronger check that the correct number has been entered.

I want to point out that 'mklog -p' already outputs the bug-summary lines
at the top of the generated changelog, by fetching them from Bugzilla

This patch extends this by:
* Being able to specify the PR numbers on the command line in addition
   (currently, they are only extracted from the testsuite patches)
* For -p, updating the format and correcting/filling-in the component
* Avoiding to print the fetched PR lines summary lines multiple times.

The new '-b' option takes PR numbers, but treats them in the same way
as the PRs extracted from the examples: They are simply output, i.e.
'5' is printed as '\t5\n', which is not that useful.

But with -p added, it becomes rather nice. For instance:

   git diff |./contrib/mklog.py -b foo/12394 -b 100123 -p

nows prints:

PR c++/12394 - internal compiler error: in write_type, at cp/mangle.c:1517
PR fortran/100123 - -ftree-fre gives incorrect result in subroutine with array declared as length 1

         PR c++/12394
         PR fortran/100123

gcc/ChangeLog:


I think that works rather nice :-)

Martin (and all): What do you think? Patch attached.

Tobias

-----------------
Mentor Graphics (Deutschland) GmbH, Arnulfstrasse 201, 80634 München Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Frank Thürauf

[-- Attachment #2: mklog-p.diff --]
[-- Type: text/x-patch, Size: 3216 bytes --]

contrib/mklog.py: Improve PR handling

contrib/ChangeLog:

	* mklog.py (bugzilla_url): Fetch also component.
	(get_pr_titles): Update PR string with correct format and component.
	(generate_changelog): Take additional PRs.
	(__main__): Add -b/--pr-numbers argument.

diff --git a/contrib/mklog.py b/contrib/mklog.py
index 1f59055e723..d95f056c9cc 100755
--- a/contrib/mklog.py
+++ b/contrib/mklog.py
@@ -52,7 +52,7 @@ fn_regex = re.compile(r'([a-zA-Z_][^()\s]*)\s*\([^*]')
 template_and_param_regex = re.compile(r'<[^<>]*>')
 md_def_regex = re.compile(r'\(define.*\s+"(.*)"')
 bugzilla_url = 'https://gcc.gnu.org/bugzilla/rest.cgi/bug?id=%s&' \
-               'include_fields=summary'
+               'include_fields=summary,component'
 
 function_extensions = {'.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def', '.md'}
 
@@ -119,19 +119,20 @@ def sort_changelog_files(changed_file):
 
 def get_pr_titles(prs):
     output = ''
-    for pr in prs:
+    for idx, pr in enumerate(prs):
         pr_id = pr.split('/')[-1]
         r = requests.get(bugzilla_url % pr_id)
         bugs = r.json()['bugs']
         if len(bugs) == 1:
-            output += '%s - %s\n' % (pr, bugs[0]['summary'])
-            print(output)
+            prs[idx] = 'PR %s/%s' % (bugs[0]['component'], pr_id)
+            output += '%s - %s\n' % (prs[idx], bugs[0]['summary'])
     if output:
         output += '\n'
     return output
 
 
-def generate_changelog(data, no_functions=False, fill_pr_titles=False):
+def generate_changelog(data, no_functions=False, fill_pr_titles=False,
+                       additional_prs=None):
     changelogs = {}
     changelog_list = []
     prs = []
@@ -139,6 +140,8 @@ def generate_changelog(data, no_functions=False, fill_pr_titles=False):
     diff = PatchSet(data)
     global firstpr
 
+    if additional_prs:
+        prs = [pr for pr in additional_prs if pr not in prs]
     for file in diff:
         # skip files that can't be parsed
         if file.path == '/dev/null':
@@ -286,6 +289,8 @@ if __name__ == '__main__':
     parser = argparse.ArgumentParser(description=help_message)
     parser.add_argument('input', nargs='?',
                         help='Patch file (or missing, read standard input)')
+    parser.add_argument('-b', '--pr-numbers', action='append',
+                        help='Add the specified PRs (comma separated)')
     parser.add_argument('-s', '--no-functions', action='store_true',
                         help='Do not generate function names in ChangeLogs')
     parser.add_argument('-p', '--fill-up-bug-titles', action='store_true',
@@ -308,8 +313,11 @@ if __name__ == '__main__':
     if args.update_copyright:
         update_copyright(data)
     else:
+        pr_numbers = args.pr_numbers
+        if pr_numbers:
+            pr_numbers = [b for i in args.pr_numbers for b in i.split(',')]
         output = generate_changelog(data, args.no_functions,
-                                    args.fill_up_bug_titles)
+                                    args.fill_up_bug_titles, pr_numbers)
         if args.changelog:
             lines = open(args.changelog).read().split('\n')
             start = list(takewhile(lambda l: not l.startswith('#'), lines))