[PATCH 0/3] mklog improvements

[PATCH 0/3] mklog improvements

[Martin Liska]

Hi.

I'm sending format independent changes to mklog that should
improve the script. It addresses couple of improvement
mentioned here:
https://gcc.gnu.org/ml/gcc-patches/2019-08/msg00031.html

Martin

Martin Liska (3):
  Use argparse.ArgumentParser for mklog.
  mklog: parse PR references from new test files
  mklog: Do not print changed functions in testsuite

 contrib/mklog | 98 ++++++++++++++++++++++++---------------------------
 1 file changed, 47 insertions(+), 51 deletions(-)

-- 
2.22.0

[PATCH 3/3] mklog: Do not print changed functions in testsuite

[Martin Liska]

[-- Attachment #1: Type: text/plain, Size: 204 bytes --]


contrib/ChangeLog:

2019-08-13  Martin Liska  <mliska@suse.cz>

	* mklog: Do not print changed functions for
	testsuite files.
---
 contrib/mklog | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0003-mklog-Do-not-print-changed-functions-in-testsuite.patch --]
[-- Type: text/x-patch; name="0003-mklog-Do-not-print-changed-functions-in-testsuite.patch", Size: 554 bytes --]

diff --git a/contrib/mklog b/contrib/mklog
index 85242002357..1a0e82d1ddd 100755
--- a/contrib/mklog
+++ b/contrib/mklog
@@ -407,7 +407,8 @@ otherwise writes to stdout.'
         change_msg = ": Remove.\n"
 
     _, ext = os.path.splitext(d.filename)
-    if not change_msg and ext in ['.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def']:
+    if (not change_msg and ext in ['.c', '.cpp', '.C', '.cc', '.h', '.inc', '.def']
+        and not 'testsuite' in d.filename):
       fns = []
       for hunk in d.hunks:
         for fn in find_changed_funs(hunk):

[PATCH 2/3] mklog: parse PR references from new test files

[Martin Liska]

[-- Attachment #1: Type: text/plain, Size: 223 bytes --]


contrib/ChangeLog:

2019-08-13  Martin Liska  <mliska@suse.cz>

	* mklog: Parse PR references from newly added
	test files.
---
 contrib/mklog | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0002-mklog-parse-PR-references-from-new-test-files.patch --]
[-- Type: text/x-patch; name="0002-mklog-parse-PR-references-from-new-test-files.patch", Size: 1839 bytes --]

diff --git a/contrib/mklog b/contrib/mklog
index e7a513fad5c..85242002357 100755
--- a/contrib/mklog
+++ b/contrib/mklog
@@ -40,6 +40,8 @@ from subprocess import Popen, PIPE
 
 me = os.path.basename(sys.argv[0])
 
+pr_regex = re.compile('\+(\/(\/|\*)|[Cc*!])\s+(PR [a-z+-]+\/[0-9]+)')
+
 def error(msg):
   sys.stderr.write("%s: error: %s\n" % (me, msg))
   sys.exit(1)
@@ -299,7 +301,7 @@ def parse_patch(contents):
         if l != r:
           break
         comps.append(l)
-    
+
       if not comps:
         error("failed to extract common name for %s and %s" % (left, right))
 
@@ -338,6 +340,14 @@ def parse_patch(contents):
 
   return diffs
 
+
+def get_pr_from_testcase(line):
+    r = pr_regex.search(line)
+    if r != None:
+        return r.group(3)
+    else:
+        return None
+
 def main():
   name, email = read_user_info()
 
@@ -372,6 +382,7 @@ otherwise writes to stdout.'
   # Generate template ChangeLog.
 
   logs = {}
+  prs = []
   for d in diffs:
     log_name = d.clname
 
@@ -387,6 +398,9 @@ otherwise writes to stdout.'
       if hunk0.is_file_addition():
         if re.search(r'testsuite.*(?<!\.exp)$', d.filename):
           change_msg = ': New test.\n'
+          pr = get_pr_from_testcase(hunk0.lines[0])
+          if pr and pr not in prs:
+              prs.append(pr)
         else:
           change_msg = ": New file.\n"
       elif hunk0.is_file_removal():
@@ -426,13 +440,17 @@ otherwise writes to stdout.'
 
   # Print log
   date = time.strftime('%Y-%m-%d')
+  bugmsg = ''
+  if len(prs):
+    bugmsg = '\n'.join(['\t' + pr for pr in prs]) + '\n'
+
   for log_name, msg in sorted(logs.items()):
     out.write("""\
 %s:
 
 %s  %s  <%s>
 
-%s\n""" % (log_name, date, name, email, msg))
+%s%s\n""" % (log_name, date, name, email, bugmsg, msg))
 
   if args.inline:
     # Append patch body

[PATCH 1/3] Use argparse.ArgumentParser for mklog.

[Martin Liska]

[-- Attachment #1: Type: text/plain, Size: 235 bytes --]


contrib/ChangeLog:

2019-08-02  Martin Liska  <mliska@suse.cz>

	* mklog: Use argparse instead of getopt.
---
 contrib/mklog | 73 ++++++++++++++++++---------------------------------
 1 file changed, 25 insertions(+), 48 deletions(-)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Use-argparse.ArgumentParser-for-mklog.patch --]
[-- Type: text/x-patch; name="0001-Use-argparse.ArgumentParser-for-mklog.patch", Size: 3627 bytes --]

diff --git a/contrib/mklog b/contrib/mklog
index 15558cfbfe3..e7a513fad5c 100755
--- a/contrib/mklog
+++ b/contrib/mklog
@@ -28,11 +28,11 @@
 #
 # Author: Yury Gribov <tetra2005@gmail.com>
 
+import argparse
 import sys
 import re
 import os.path
 import os
-import getopt
 import tempfile
 import time
 import shutil
@@ -66,21 +66,6 @@ class RegexCache(object):
 
 cache = RegexCache()
 
-def print_help_and_exit():
-    print("""\
-Usage: %s [-i | --inline] [PATCH]
-Generate ChangeLog template for PATCH.
-PATCH must be generated using diff(1)'s -up or -cp options
-(or their equivalent in Subversion/git).
-
-When PATCH is - or missing, read standard input.
-
-When -i is used, prepends ChangeLog to PATCH.
-If PATCH is not stdin, modifies PATCH in-place, otherwise writes
-to stdout.
-""" % me)
-    sys.exit(1)
-
 def run(cmd, die_on_error):
   """Simple wrapper for Popen."""
   proc = Popen(cmd.split(' '), stderr = PIPE, stdout = PIPE)
@@ -356,38 +341,30 @@ def parse_patch(contents):
 def main():
   name, email = read_user_info()
 
-  try:
-    opts, args = getopt.getopt(sys.argv[1:], 'hiv', ['help', 'verbose', 'inline'])
-  except getopt.GetoptError as err:
-    error(str(err))
-
-  inline = False
-  verbose = 0
-
-  for o, a in opts:
-    if o in ('-h', '--help'):
-      print_help_and_exit()
-    elif o in ('-i', '--inline'):
-      inline = True
-    elif o in ('-v', '--verbose'):
-      verbose += 1
-    else:
-      assert False, "unhandled option"
-
-  if len(args) == 0:
-    args = ['-']
+  help_message =  """\
+Generate ChangeLog template for PATCH.
+PATCH must be generated using diff(1)'s -up or -cp options
+(or their equivalent in Subversion/git).
+"""
 
-  if len(args) == 1 and args[0] == '-':
-    input = sys.stdin
-  elif len(args) == 1:
-    input = open(args[0])
-  else:
-    error("too many arguments; for more details run with -h")
+  inline_message = """\
+Prepends ChangeLog to PATCH.
+If PATCH is not stdin, modifies PATCH in-place,
+otherwise writes to stdout.'
+"""
 
+  parser = argparse.ArgumentParser(description = help_message)
+  parser.add_argument('-v', '--verbose', action = 'store_true', help = 'Verbose messages')
+  parser.add_argument('-i', '--inline', action = 'store_true', help = inline_message)
+  parser.add_argument('input', nargs = '?', help = 'Patch file (or missing, read standard input)')
+  args = parser.parse_args()
+  if args.input == '-':
+      args.input = None
+  input = open(args.input) if args.input else sys.stdin
   contents = input.read()
   diffs = parse_patch(contents)
 
-  if verbose:
+  if args.verbose:
     print("Parse results:")
     for d in diffs:
       d.dump()
@@ -431,7 +408,7 @@ def main():
 
     logs[log_name] += change_msg if change_msg else ":\n"
 
-  if inline and args[0] != '-':
+  if args.inline and args.input:
     # Get a temp filename, rather than an open filehandle, because we use
     # the open to truncate.
     fd, tmp = tempfile.mkstemp("tmp.XXXXXXXX")
@@ -439,7 +416,7 @@ def main():
 
     # Copy permissions to temp file
     # (old Pythons do not support shutil.copymode)
-    shutil.copymode(args[0], tmp)
+    shutil.copymode(args.input, tmp)
 
     # Open the temp file, clearing contents.
     out = open(tmp, 'w')
@@ -457,14 +434,14 @@ def main():
 
 %s\n""" % (log_name, date, name, email, msg))
 
-  if inline:
+  if args.inline:
     # Append patch body
     out.write(contents)
 
-    if args[0] != '-':
+    if args.input:
       # Write new contents atomically
       out.close()
-      shutil.move(tmp, args[0])
+      shutil.move(tmp, args.input)
 
 if __name__ == '__main__':
     main()

Re: [PATCH 0/3] mklog improvements

[Martin Liška]

PING^1

On 8/13/19 9:49 AM, Martin Liska wrote:
> Hi.
> 
> I'm sending format independent changes to mklog that should
> improve the script. It addresses couple of improvement
> mentioned here:
> https://gcc.gnu.org/ml/gcc-patches/2019-08/msg00031.html
> 
> Martin
> 
> Martin Liska (3):
>   Use argparse.ArgumentParser for mklog.
>   mklog: parse PR references from new test files
>   mklog: Do not print changed functions in testsuite
> 
>  contrib/mklog | 98 ++++++++++++++++++++++++---------------------------
>  1 file changed, 47 insertions(+), 51 deletions(-)
> 

Re: [PATCH 0/3] mklog improvements

[Jeff Law]

On 8/30/19 2:55 AM, Martin Liška wrote:
> PING^1
> 
> On 8/13/19 9:49 AM, Martin Liska wrote:
>> Hi.
>>
>> I'm sending format independent changes to mklog that should
>> improve the script. It addresses couple of improvement
>> mentioned here:
>> https://gcc.gnu.org/ml/gcc-patches/2019-08/msg00031.html
>>
>> Martin
>>
>> Martin Liska (3):
>>   Use argparse.ArgumentParser for mklog.
>>   mklog: parse PR references from new test files
>>   mklog: Do not print changed functions in testsuite
So there's a ton of follow-ups (primarily between Jakub and yourself).

What's the state of the changes Jakub has requested?  Can we go forward
with the patch as-is and iterate on further improvements?

Jeff

Re: [PATCH 0/3] mklog improvements

[Martin Liška]

On 9/3/19 6:37 PM, Jeff Law wrote:
> On 8/30/19 2:55 AM, Martin Liška wrote:
>> PING^1
>>
>> On 8/13/19 9:49 AM, Martin Liska wrote:
>>> Hi.
>>>
>>> I'm sending format independent changes to mklog that should
>>> improve the script. It addresses couple of improvement
>>> mentioned here:
>>> https://gcc.gnu.org/ml/gcc-patches/2019-08/msg00031.html
>>>
>>> Martin
>>>
>>> Martin Liska (3):
>>>    Use argparse.ArgumentParser for mklog.
>>>    mklog: parse PR references from new test files
>>>    mklog: Do not print changed functions in testsuite
> So there's a ton of follow-ups (primarily between Jakub and yourself).

Yes, the follow up is about possible change to ChangeLog format. I first
provided a patch, but later I realized that I would rather stick with
the current format.

> 
> What's the state of the changes Jakub has requested?  Can we go forward
> with the patch as-is and iterate on further improvements?

Yes, I'm going to install these 3 patches which do not change the format.

Martin

> 
> Jeff
> 

Re: [PATCH 0/3] mklog improvements

[Jeff Law]

On 9/4/19 2:05 AM, Martin Liška wrote:
> On 9/3/19 6:37 PM, Jeff Law wrote:
>> On 8/30/19 2:55 AM, Martin Liška wrote:
>>> PING^1
>>>
>>> On 8/13/19 9:49 AM, Martin Liska wrote:
>>>> Hi.
>>>>
>>>> I'm sending format independent changes to mklog that should
>>>> improve the script. It addresses couple of improvement
>>>> mentioned here:
>>>> https://gcc.gnu.org/ml/gcc-patches/2019-08/msg00031.html
>>>>
>>>> Martin
>>>>
>>>> Martin Liska (3):
>>>>    Use argparse.ArgumentParser for mklog.
>>>>    mklog: parse PR references from new test files
>>>>    mklog: Do not print changed functions in testsuite
>> So there's a ton of follow-ups (primarily between Jakub and yourself).
> 
> Yes, the follow up is about possible change to ChangeLog format. I first
> provided a patch, but later I realized that I would rather stick with
> the current format.
I'd rather drop ChangeLogs and generate them from the VCS, but that's
another discussion entirely :-)

> 
>>
>> What's the state of the changes Jakub has requested?  Can we go forward
>> with the patch as-is and iterate on further improvements?
> 
> Yes, I'm going to install these 3 patches which do not change the format.
THat works for me.

jeff