[PATCH] Port update-copyright.py to Python3

[PATCH] Port update-copyright.py to Python3

[Martin Liška]

The patch ports the script to Python3. There's a small issue that
we have some ISO 8859 files, like:

libstdc++-v3/testsuite/22_locale/messages/members/char/2.cc

In that case we first try utf8 encoding and then iso8859 in Python's
open function.

I'm going to install the script if there are no objections.
Martin

contrib/ChangeLog:

	* update-copyright.py: Port to python3 by guessing encoding
	(first utf8, then iso8859). Add 2 more ignores: .png and .pyc.
---
  contrib/update-copyright.py | 25 +++++++++++++++++++------
  1 file changed, 19 insertions(+), 6 deletions(-)

diff --git a/contrib/update-copyright.py b/contrib/update-copyright.py
index bc65208d9cb..5603b8eac16 100755
--- a/contrib/update-copyright.py
+++ b/contrib/update-copyright.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
  #
  # Copyright (C) 2013-2020 Free Software Foundation, Inc.
  #
@@ -64,7 +64,10 @@ class GenericFilter:
      def __init__ (self):
          self.skip_files = set()
          self.skip_dirs = set()
-        self.skip_extensions = set()
+        self.skip_extensions = set([
+                '.png',
+                '.pyc',
+                ])
          self.fossilised_files = set()
          self.own_files = set()
  
@@ -307,7 +310,7 @@ class Copyright:
              # If it looks like the copyright is incomplete, add the next line.
              while not self.is_complete (match):
                  try:
-                    next_line = file.next()
+                    next_line = file.readline()
                  except StopIteration:
                      break
  
@@ -381,6 +384,15 @@ class Copyright:
  
          return (line != orig_line, line, next_line)
  
+    def guess_encoding (self, pathname):
+        for encoding in ('utf8', 'iso8859'):
+            try:
+                open(pathname, 'r', encoding=encoding).read()
+                return encoding
+            except UnicodeDecodeError:
+                pass
+        return None
+
      def process_file (self, dir, filename, filter):
          pathname = os.path.join (dir, filename)
          if filename.endswith ('.tmp'):
@@ -395,7 +407,8 @@ class Copyright:
          changed = False
          line_filter = filter.get_line_filter (dir, filename)
          mode = None
-        with open (pathname, 'r') as file:
+        encoding = self.guess_encoding(pathname)
+        with open (pathname, 'r', encoding=encoding) as file:
              prev = None
              mode = os.fstat (file.fileno()).st_mode
              for line in file:
@@ -421,7 +434,7 @@ class Copyright:
          # If something changed, write the new file out.
          if changed and self.errors.ok():
              tmp_pathname = pathname + '.tmp'
-            with open (tmp_pathname, 'w') as file:
+            with open (tmp_pathname, 'w', encoding=encoding) as file:
                  for line in lines:
                      file.write (line)
                  os.fchmod (file.fileno(), mode)
@@ -432,7 +445,7 @@ class Copyright:
      def process_tree (self, tree, filter):
          for (dir, subdirs, filenames) in os.walk (tree):
              # Don't recurse through directories that should be skipped.
-            for i in xrange (len (subdirs) - 1, -1, -1):
+            for i in range (len (subdirs) - 1, -1, -1):
                  if filter.skip_dir (dir, subdirs[i]):
                      del subdirs[i]
  
-- 
2.29.2

Re: [PATCH] Port update-copyright.py to Python3

[Thomas Schwinge]

[-- Attachment #1: Type: text/plain, Size: 753 bytes --]

Hi!

On 2021-01-04T11:15:22+0100, Martin Liška <mliska@suse.cz> wrote:
> The patch ports the script to Python3.

Turns out, there is another issue, observed in combination with a
few "BadYear" occurrences due to "improper" copyright lines (Bill,
for your information).  OK to push "Fix 'contrib/update-copyright.py':
'TypeError: exceptions must derive from BaseException'" as well as
"Fix 'Copyright (C) 2020-21' into '2020-2021'", see attached?


Grüße
 Thomas


-----------------
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-contrib-update-copyright.py-TypeError-exceptions.patch --]
[-- Type: text/x-diff, Size: 2784 bytes --]

From 3cffeead3b7f900999ec7885ae044e63e44deff3 Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <thomas@codesourcery.com>
Date: Fri, 22 Oct 2021 15:54:42 +0200
Subject: [PATCH] Fix 'contrib/update-copyright.py': 'TypeError: exceptions
 must derive from BaseException'

Running 'contrib/update-copyright.py' currently fails:

    [...]
    Traceback (most recent call last):
      File "contrib/update-copyright.py", line 365, in update_copyright
        canon_form = self.canonicalise_years (dir, filename, filter, years)
      File "contrib/update-copyright.py", line 270, in canonicalise_years
        (min_year, max_year) = self.year_range (years)
      File "contrib/update-copyright.py", line 253, in year_range
        year_list = [self.parse_year (year)
      File "contrib/update-copyright.py", line 253, in <listcomp>
        year_list = [self.parse_year (year)
      File "contrib/update-copyright.py", line 250, in parse_year
        raise self.BadYear (string)
    TypeError: exceptions must derive from BaseException

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "contrib/update-copyright.py", line 796, in <module>
        GCCCmdLine().main()
      File "contrib/update-copyright.py", line 527, in main
        self.copyright.process_tree (dir, filter)
      File "contrib/update-copyright.py", line 458, in process_tree
        self.process_file (dir, filename, filter)
      File "contrib/update-copyright.py", line 421, in process_file
        res = self.update_copyright (dir, filename, filter,
      File "contrib/update-copyright.py", line 366, in update_copyright
        except self.BadYear as e:
    TypeError: catching classes that do not inherit from BaseException is not allowed

Fix up for commit 3b25e83536bcd1b2977659a2c6d9f0f9bf2a3152
"Port update-copyright.py to Python3".

	contrib/
	* update-copyright.py (class BadYear): Derive from 'Exception'.
---
 contrib/update-copyright.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/contrib/update-copyright.py b/contrib/update-copyright.py
index 2b2bb11d2e6..d13b963a147 100755
--- a/contrib/update-copyright.py
+++ b/contrib/update-copyright.py
@@ -1,6 +1,6 @@
 #!/usr/bin/env python3
 #
-# Copyright (C) 2013-2020 Free Software Foundation, Inc.
+# Copyright (C) 2013-2021 Free Software Foundation, Inc.
 #
 # This script is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -233,7 +233,7 @@ class Copyright:
     def add_external_author (self, holder):
         self.holders[holder] = None
 
-    class BadYear():
+    class BadYear (Exception):
         def __init__ (self, year):
             self.year = year
 
-- 
2.33.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0001-Fix-Copyright-C-2020-21-into-2020-2021.patch --]
[-- Type: text/x-diff, Size: 3900 bytes --]

From 881f3e7701ab7ae5269db72cb33a7879b7e94e09 Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <thomas@codesourcery.com>
Date: Fri, 22 Oct 2021 16:01:54 +0200
Subject: [PATCH] Fix 'Copyright (C) 2020-21' into '2020-2021'

'contrib/update-copyright.py' currently complains:

    gcc/config/rs6000/rs6000-gen-builtins.c: unrecognised year: 21
    gcc/config/rs6000/rs6000-overload.def: unrecognised year: 21
    gcc/config/rs6000/rbtree.h: unrecognised year: 21
    gcc/config/rs6000/rbtree.c: unrecognised year: 21
    gcc/config/rs6000/rs6000-builtin-new.def: unrecognised year: 21

Fix up files added in commit fa5f8b49e55caf5bb341f5eb6b5ab828b9286425
"rs6000: Red-black tree implementation for balanced tree search",
commit 4a720a9547320699aceda7d2e0b08de5ab40132f
"rs6000: Add initial input files",
commit bd5b625228d545d5ecb35df24f9f094edc95e3fa
"rs6000: Initial create of rs6000-gen-builtins.c".

	gcc/
	* config/rs6000/rbtree.c: Fix 'Copyright (C) 2020-21' into '2020-2021'
	* config/rs6000/rbtree.h: Likewise.
	* config/rs6000/rs6000-builtin-new.def: Likewise.
	* config/rs6000/rs6000-gen-builtins.c: Likewise.
	* config/rs6000/rs6000-overload.def: Likewise.
---
 gcc/config/rs6000/rbtree.c               | 2 +-
 gcc/config/rs6000/rbtree.h               | 2 +-
 gcc/config/rs6000/rs6000-builtin-new.def | 2 +-
 gcc/config/rs6000/rs6000-gen-builtins.c  | 2 +-
 gcc/config/rs6000/rs6000-overload.def    | 2 +-
 5 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/gcc/config/rs6000/rbtree.c b/gcc/config/rs6000/rbtree.c
index 37a559c1fbc..d3d03a62433 100644
--- a/gcc/config/rs6000/rbtree.c
+++ b/gcc/config/rs6000/rbtree.c
@@ -1,5 +1,5 @@
 /* Partial red-black tree implementation for rs6000-gen-builtins.c.
-   Copyright (C) 2020-21 Free Software Foundation, Inc.
+   Copyright (C) 2020-2021 Free Software Foundation, Inc.
    Contributed by Bill Schmidt, IBM <wschmidt@linux.ibm.com>
 
 This file is part of GCC.
diff --git a/gcc/config/rs6000/rbtree.h b/gcc/config/rs6000/rbtree.h
index fab0001ffde..bd462f15788 100644
--- a/gcc/config/rs6000/rbtree.h
+++ b/gcc/config/rs6000/rbtree.h
@@ -1,5 +1,5 @@
 /* Partial red-black tree implementation for rs6000-gen-builtins.c.
-   Copyright (C) 2020-21 Free Software Foundation, Inc.
+   Copyright (C) 2020-2021 Free Software Foundation, Inc.
    Contributed by Bill Schmidt, IBM <wschmidt@linux.ibm.com>
 
 This file is part of GCC.
diff --git a/gcc/config/rs6000/rs6000-builtin-new.def b/gcc/config/rs6000/rs6000-builtin-new.def
index 1966516551e..cae5c6b8556 100644
--- a/gcc/config/rs6000/rs6000-builtin-new.def
+++ b/gcc/config/rs6000/rs6000-builtin-new.def
@@ -1,5 +1,5 @@
 ; Built-in functions for PowerPC.
-; Copyright (C) 2020-21 Free Software Foundation, Inc.
+; Copyright (C) 2020-2021 Free Software Foundation, Inc.
 ; Contributed by Bill Schmidt, IBM <wschmidt@linux.ibm.com>
 ;
 ; This file is part of GCC.
diff --git a/gcc/config/rs6000/rs6000-gen-builtins.c b/gcc/config/rs6000/rs6000-gen-builtins.c
index 7f711210aff..2af2302cd11 100644
--- a/gcc/config/rs6000/rs6000-gen-builtins.c
+++ b/gcc/config/rs6000/rs6000-gen-builtins.c
@@ -1,5 +1,5 @@
 /* Generate built-in function initialization and recognition for Power.
-   Copyright (C) 2020-21 Free Software Foundation, Inc.
+   Copyright (C) 2020-2021 Free Software Foundation, Inc.
    Contributed by Bill Schmidt, IBM <wschmidt@linux.ibm.com>
 
 This file is part of GCC.
diff --git a/gcc/config/rs6000/rs6000-overload.def b/gcc/config/rs6000/rs6000-overload.def
index 4f583312f36..531a4fcd1af 100644
--- a/gcc/config/rs6000/rs6000-overload.def
+++ b/gcc/config/rs6000/rs6000-overload.def
@@ -1,5 +1,5 @@
 ; Overloaded built-in functions for PowerPC.
-; Copyright (C) 2020-21 Free Software Foundation, Inc.
+; Copyright (C) 2020-2021 Free Software Foundation, Inc.
 ; Contributed by Bill Schmidt, IBM <wschmidt@linux.ibm.com>
 ;
 ; This file is part of GCC.
-- 
2.33.0

Re: [PATCH] Port update-copyright.py to Python3

[Martin Liška]

On 10/22/21 23:00, Thomas Schwinge wrote:
> |Turns out, there is another issue, observed in combination with a few "BadYear" occurrences due to "improper" copyright lines (Bill, for your information). OK to push "Fix 'contrib/update-copyright.py':|

Thank you for the fix, it seems to me obvious the change!

Martin