[PATCH] mklog: Add --append option to auto add generate ChangeLog to patch file

[PATCH] mklog: Add --append option to auto add generate ChangeLog to patch file

[Lehua Ding]

Hi,

This tiny patch add --append option to mklog.py that support add generated
ChangeLog to the corresponding patch file. With this option there is no need
to manually copy the generated ChangeLog to the patch file. e.g.:

Run `mklog.py -a /path/to/this/patch` will add the generated ChangeLog

```
contrib/ChangeLog:

	* mklog.py:
```

to the right place of the /path/to/this/patch file.

Best,
Lehua

contrib/ChangeLog:

	* mklog.py: Add --append option.

---
 contrib/mklog.py | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/contrib/mklog.py b/contrib/mklog.py
index 777212c98d7..26230b9b4f2 100755
--- a/contrib/mklog.py
+++ b/contrib/mklog.py
@@ -358,6 +358,8 @@ if __name__ == '__main__':
                              'file')
     parser.add_argument('--update-copyright', action='store_true',
                         help='Update copyright in ChangeLog files')
+    parser.add_argument('-a', '--append', action='store_true',
+                        help='Append the generate ChangeLog to the patch file')
     args = parser.parse_args()
     if args.input == '-':
         args.input = None
@@ -370,7 +372,30 @@ if __name__ == '__main__':
     else:
         output = generate_changelog(data, args.no_functions,
                                     args.fill_up_bug_titles, args.pr_numbers)
-        if args.changelog:
+        if args.append:
+            if (not args.input):
+                raise Exception("`-a or --append` option not support standard input")
+            lines = []
+            with open(args.input, 'r', newline='\n') as f:
+                # 1 -> not find the possible start of diff log
+                # 2 -> find the possible start of diff log
+                # 3 -> finish add ChangeLog to the patch file
+                maybe_diff_log = 1
+                for line in f:
+                    if maybe_diff_log == 1 and line == "---\n":
+                        maybe_diff_log = 2
+                    elif maybe_diff_log == 2 and \
+                         re.match("\s[^\s]+\s+\|\s\d+\s[+\-]+\n", line):
+                        lines += [output, "---\n", line]
+                        maybe_diff_log = 3
+                    else:
+                        # the possible start is not the true start.
+                        if maybe_diff_log == 2:
+                            maybe_diff_log = 1
+                        lines.append(line)
+            with open(args.input, "w") as f:
+                f.writelines(lines)
+        elif args.changelog:
             lines = open(args.changelog).read().split('\n')
             start = list(takewhile(skip_line_in_changelog, lines))
             end = lines[len(start):]
-- 
2.36.1

Re: [PATCH] mklog: Add --append option to auto add generate ChangeLog to patch file

[Jeff Law]

On 7/11/23 22:01, Lehua Ding wrote:
> Hi,
> 
> This tiny patch add --append option to mklog.py that support add generated
> ChangeLog to the corresponding patch file. With this option there is no need
> to manually copy the generated ChangeLog to the patch file. e.g.:
> 
> Run `mklog.py -a /path/to/this/patch` will add the generated ChangeLog
> 
> ```
> contrib/ChangeLog:
> 
> 	* mklog.py:
> ```
> 
> to the right place of the /path/to/this/patch file.
> 
> Best,
> Lehua
> 
> contrib/ChangeLog:
> 
> 	* mklog.py: Add --append option.
OK for the trunk.
jeff

Re: [PATCH] mklog: Add --append option to auto add generate ChangeLog to patch file

[Lehua Ding]

[-- Attachment #1: Type: text/plain, Size: 1147 bytes --]

Commited to the trunk, thanks Jeff.
 
 
------------------ Original ------------------
From: &nbsp;"Jeff&nbsp;Law"<jeffreyalaw@gmail.com&gt;;
Date: &nbsp;Thu, Jul 13, 2023 00:04 AM
To: &nbsp;"Lehua Ding"<lehua.ding@rivai.ai&gt;; "gcc-patches"<gcc-patches@gcc.gnu.org&gt;; 
Cc: &nbsp;"juzhe.zhong"<juzhe.zhong@rivai.ai&gt;; "mliska"<mliska@suse.cz&gt;; 
Subject: &nbsp;Re: [PATCH] mklog: Add --append option to auto add generate ChangeLog to patch file

&nbsp;



On 7/11/23 22:01, Lehua Ding wrote:
&gt; Hi,
&gt; 
&gt; This tiny patch add --append option to mklog.py that support add generated
&gt; ChangeLog to the corresponding patch file. With this option there is no need
&gt; to manually copy the generated ChangeLog to the patch file. e.g.:
&gt; 
&gt; Run `mklog.py -a /path/to/this/patch` will add the generated ChangeLog
&gt; 
&gt; ```
&gt; contrib/ChangeLog:
&gt; 
&gt; 	* mklog.py:
&gt; ```
&gt; 
&gt; to the right place of the /path/to/this/patch file.
&gt; 
&gt; Best,
&gt; Lehua
&gt; 
&gt; contrib/ChangeLog:
&gt; 
&gt; 	* mklog.py: Add --append option.
OK for the trunk.
jeff

Re: [PATCH] mklog: Add --append option to auto add generate ChangeLog to patch file

[Martin Jambor]

Hello Lehua,

On Wed, Jul 12 2023, Lehua Ding wrote:
> Hi,
>
> This tiny patch add --append option to mklog.py that support add generated
> ChangeLog to the corresponding patch file. With this option there is no need
> to manually copy the generated ChangeLog to the patch file. e.g.:
>
> Run `mklog.py -a /path/to/this/patch` will add the generated ChangeLog
>
> ```
> contrib/ChangeLog:
>
> 	* mklog.py:
> ```

this patch caused flake8 to complain about contrib/mklog.py:

$ flake8 contrib/mklog.py
contrib/mklog.py:377:80: E501 line too long (85 > 79 characters)
contrib/mklog.py:388:26: E127 continuation line over-indented for visual indent
contrib/mklog.py:388:36: W605 invalid escape sequence '\s'
contrib/mklog.py:388:40: W605 invalid escape sequence '\s'
contrib/mklog.py:388:44: W605 invalid escape sequence '\s'
contrib/mklog.py:388:47: W605 invalid escape sequence '\|'
contrib/mklog.py:388:49: W605 invalid escape sequence '\s'
contrib/mklog.py:388:51: W605 invalid escape sequence '\d'
contrib/mklog.py:388:54: W605 invalid escape sequence '\s'
contrib/mklog.py:388:58: W605 invalid escape sequence '\-'

Can you please have a look and ideally fix the issues?

Thanks,

Martin


>
> to the right place of the /path/to/this/patch file.
>
> Best,
> Lehua
>
> contrib/ChangeLog:
>
> 	* mklog.py: Add --append option.
>
> ---
>  contrib/mklog.py | 27 ++++++++++++++++++++++++++-
>  1 file changed, 26 insertions(+), 1 deletion(-)
>
> diff --git a/contrib/mklog.py b/contrib/mklog.py
> index 777212c98d7..26230b9b4f2 100755
> --- a/contrib/mklog.py
> +++ b/contrib/mklog.py
> @@ -358,6 +358,8 @@ if __name__ == '__main__':
>                               'file')
>      parser.add_argument('--update-copyright', action='store_true',
>                          help='Update copyright in ChangeLog files')
> +    parser.add_argument('-a', '--append', action='store_true',
> +                        help='Append the generate ChangeLog to the patch file')
>      args = parser.parse_args()
>      if args.input == '-':
>          args.input = None
> @@ -370,7 +372,30 @@ if __name__ == '__main__':
>      else:
>          output = generate_changelog(data, args.no_functions,
>                                      args.fill_up_bug_titles, args.pr_numbers)
> -        if args.changelog:
> +        if args.append:
> +            if (not args.input):
> +                raise Exception("`-a or --append` option not support standard input")
> +            lines = []
> +            with open(args.input, 'r', newline='\n') as f:
> +                # 1 -> not find the possible start of diff log
> +                # 2 -> find the possible start of diff log
> +                # 3 -> finish add ChangeLog to the patch file
> +                maybe_diff_log = 1
> +                for line in f:
> +                    if maybe_diff_log == 1 and line == "---\n":
> +                        maybe_diff_log = 2
> +                    elif maybe_diff_log == 2 and \
> +                         re.match("\s[^\s]+\s+\|\s\d+\s[+\-]+\n", line):
> +                        lines += [output, "---\n", line]
> +                        maybe_diff_log = 3
> +                    else:
> +                        # the possible start is not the true start.
> +                        if maybe_diff_log == 2:
> +                            maybe_diff_log = 1
> +                        lines.append(line)
> +            with open(args.input, "w") as f:
> +                f.writelines(lines)
> +        elif args.changelog:
>              lines = open(args.changelog).read().split('\n')
>              start = list(takewhile(skip_line_in_changelog, lines))
>              end = lines[len(start):]
> -- 
> 2.36.1

Re: [PATCH] mklog: Add --append option to auto add generate ChangeLog to patch file

[=?gb18030?B?TGVodWEgRGluZw==?=]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="gb18030", Size: 1325 bytes --]

Hi Martin,


> this patch caused flake8 to complain about contrib/mklog.py:
> 
> $ flake8 contrib/mklog.py
> contrib/mklog.py:377:80: E501 line too long (85 > 79 characters)
> contrib/mklog.py:388:26: E127 continuation line over-indented for visual indent
> contrib/mklog.py:388:36: W605 invalid escape sequence '\s'
> contrib/mklog.py:388:40: W605 invalid escape sequence '\s'
> contrib/mklog.py:388:44: W605 invalid escape sequence '\s'
> contrib/mklog.py:388:47: W605 invalid escape sequence '\|'
> contrib/mklog.py:388:49: W605 invalid escape sequence '\s'
> contrib/mklog.py:388:51: W605 invalid escape sequence '\d'
> contrib/mklog.py:388:54: W605 invalid escape sequence '\s'
> contrib/mklog.py:388:58: W605 invalid escape sequence '\-'
> 
> Can you please have a look and ideally fix the issues?


Thank you for pointing out this.
I will fix these format errors in another fix patch[1].
I tried to fix the following format error but couldn't
find a way, do you know how to fix this error?



contrib/mklog.py:388:26: E127 continuation line over-indented for visual indent


Best,
Lehua


[1] https://gcc.gnu.org/pipermail/gcc-patches/2023-July/624880.html

Re: [PATCH] mklog: Add --append option to auto add generate ChangeLog to patch file

[Martin Jambor]

Hello Lehua,

On Fri, Jul 21 2023, Lehua Ding wrote:
> Hi Martin,
>
>
> > this patch caused flake8 to complain about contrib/mklog.py:
> > 
> > $ flake8 contrib/mklog.py
> > contrib/mklog.py:377:80: E501 line too long (85 > 79 characters)
> > contrib/mklog.py:388:26: E127 continuation line over-indented for visual indent
> > contrib/mklog.py:388:36: W605 invalid escape sequence '\s'
> > contrib/mklog.py:388:40: W605 invalid escape sequence '\s'
> > contrib/mklog.py:388:44: W605 invalid escape sequence '\s'
> > contrib/mklog.py:388:47: W605 invalid escape sequence '\|'
> > contrib/mklog.py:388:49: W605 invalid escape sequence '\s'
> > contrib/mklog.py:388:51: W605 invalid escape sequence '\d'
> > contrib/mklog.py:388:54: W605 invalid escape sequence '\s'
> > contrib/mklog.py:388:58: W605 invalid escape sequence '\-'
> > 
> > Can you please have a look and ideally fix the issues?
>
>
> Thank you for pointing out this.
> I will fix these format errors in another fix patch[1].

Thanks!

> I tried to fix the following format error but couldn't
> find a way, do you know how to fix this error?
>
>
> contrib/mklog.py:388:26: E127 continuation line over-indented for visual indent

I am no python expert but the following seems to work:

diff --git a/contrib/mklog.py b/contrib/mklog.py
index 26230b9b4f2..2563d19bc99 100755
--- a/contrib/mklog.py
+++ b/contrib/mklog.py
@@ -384,8 +384,8 @@ if __name__ == '__main__':
                 for line in f:
                     if maybe_diff_log == 1 and line == "---\n":
                         maybe_diff_log = 2
-                    elif maybe_diff_log == 2 and \
-                         re.match("\s[^\s]+\s+\|\s\d+\s[+\-]+\n", line):
+                    elif (maybe_diff_log == 2 and
+                          re.match("\s[^\s]+\s+\|\s\d+\s[+\-]+\n", line)):
                         lines += [output, "---\n", line]
                         maybe_diff_log = 3
                     else:

Martin

Re: [PATCH] mklog: Add --append option to auto add generate ChangeLog to patch file

[=?gb18030?B?TGVodWEgRGluZw==?=]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="gb18030", Size: 111 bytes --]

> I am no python expert but the following seems to work:


Thank you so much, it works for me.


Lehua

Re: [PATCH] mklog: Add --append option to auto add generate ChangeLog to patch file

[Lehua Ding]

[-- Attachment #1: Type: text/plain, Size: 278 bytes --]

Hi Martin,


By the way, is there a standard format required for these Python files?
I see that other Python files have similar format error when checked
using flake8. If so, it feels necessary to configure a git hook on git server
to do this check.


Best,
Lehua

Re: [PATCH] mklog: Add --append option to auto add generate ChangeLog to patch file

[Martin Jambor]

Hello Lehua,

On Fri, Jul 21 2023, Lehua Ding wrote:
> Hi Martin,
>
>
> By the way, is there a standard format required for these Python files?

Generally, our Python coding conventions are at
https://gcc.gnu.org/codingconventions.html#python

> I see that other Python files have similar format error when checked
> using flake8.

For historic reasons (i.e. Martin Liška set it up that way), we
currently use flake8 to check python formatting of
contrib/gcc-changelog, contrib/mklog.py and
maintainer-scripts/branch_changer.py and use pytest to check
contrib/gcc-changelog and contrib/test_mklog.py.  That is how I found
out.

I guess many of the files predate the coding conventions and so don't
adhere to them.  Patches to fix them are welcome (I guess) but at least
we should not regress (I guess).

> If so, it feels necessary to configure a git hook on git server to do
> this check.

Performing more thorough checks on pushed commits is a much larger topic
than this thread.  FWIW, I would not oppose to checking python scripts
that are known to be OK.

Martin

Re: [PATCH] mklog: Add --append option to auto add generate ChangeLog to patch file

[Lehua Ding]

[-- Attachment #1: Type: text/plain, Size: 222 bytes --]

Hi Martin,


Thank you for telling me about the Python code format specification.
I'm no idea how to add checks for pushed commits.
Anyway, first make sure I don't introduce new format errors myself.


Best,
Lehua