[committed] libstdc++: Reformat Python code

[committed] libstdc++: Reformat Python code

[Jonathan Wakely]

Tested x86_64-linux (GDB 13.2, Python 3.11). Pushed to trunk.

-- >8 --

Some of these changes were suggested by autopep8's --aggressive
option, others are for readability.

Break long lines by splitting strings across multiple lines, or
introducing local variables to hold results.

Use raw strings for regular expressions, so that backslashes don't need
to be escaped.

libstdc++-v3/ChangeLog:

	* python/libstdcxx/v6/printers.py: Break long lines. Use raw
	strings for regular expressions. Add whitespace around
	operators.
	(is_member_of_namespace): Use isinstance to check type.
	(is_specialization_of): Likewise. Adjust template_name
	for versioned namespace instead of duplicating the re.match
	call.
	(StdExpAnyPrinter._string_types): New static method.
	(StdExpAnyPrinter.to_string): Use _string_types.
---
 libstdc++-v3/python/libstdcxx/v6/printers.py | 122 ++++++++++++-------
 1 file changed, 75 insertions(+), 47 deletions(-)

diff --git a/libstdc++-v3/python/libstdcxx/v6/printers.py b/libstdc++-v3/python/libstdcxx/v6/printers.py
index 7889235ce1c..3f22ba23452 100644
--- a/libstdc++-v3/python/libstdcxx/v6/printers.py
+++ b/libstdc++-v3/python/libstdcxx/v6/printers.py
@@ -145,7 +145,6 @@ def lookup_templ_spec(templ, *args):
 
 # Use this to find container node types instead of find_type,
 # see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91997 for details.
-
 def lookup_node_type(nodename, containertype):
     """
     Lookup specialization of template nodename corresponding to containertype.
@@ -188,7 +187,7 @@ def is_member_of_namespace(typ, *namespaces):
     Test whether a type is a member of one of the specified namespaces.
     The type can be specified as a string or a gdb.Type object.
     """
-    if type(typ) is gdb.Type:
+    if isinstance(typ, gdb.Type):
         typ = str(typ)
     typ = strip_versioned_namespace(typ)
     for namespace in namespaces:
@@ -205,10 +204,10 @@ def is_specialization_of(x, template_name):
     without any 'std' qualification.
     """
     global _versioned_namespace
-    if type(x) is gdb.Type:
+    if isinstance(x, gdb.Type):
         x = x.tag
     if _versioned_namespace:
-        return re.match('^std::(%s)?%s<.*>$' % (_versioned_namespace, template_name), x) is not None
+        template_name = '(%s)?%s' % (_versioned_namespace, template_name)
     return re.match('^std::%s<.*>$' % template_name, x) is not None
 
 
@@ -225,9 +224,9 @@ def strip_inline_namespaces(type_str):
     type_str = type_str.replace('std::__cxx11::', 'std::')
     expt_ns = 'std::experimental::'
     for lfts_ns in ('fundamentals_v1', 'fundamentals_v2'):
-        type_str = type_str.replace(expt_ns+lfts_ns+'::', expt_ns)
+        type_str = type_str.replace(expt_ns + lfts_ns + '::', expt_ns)
     fs_ns = expt_ns + 'filesystem::'
-    type_str = type_str.replace(fs_ns+'v1::', fs_ns)
+    type_str = type_str.replace(fs_ns + 'v1::', fs_ns)
     return type_str
 
 
@@ -365,7 +364,8 @@ class UniquePointerPrinter:
         return SmartPtrIterator(unique_ptr_get(self.val))
 
     def to_string(self):
-        return ('std::unique_ptr<%s>' % (str(self.val.type.template_argument(0))))
+        t = self.val.type.template_argument(0)
+        return 'std::unique_ptr<{}>'.format(str(t))
 
 
 def get_value_from_aligned_membuf(buf, valtype):
@@ -597,7 +597,8 @@ class StdBitIteratorPrinter:
     def to_string(self):
         if not self.val['_M_p']:
             return 'non-dereferenceable iterator for std::vector<bool>'
-        return bool(self.val['_M_p'].dereference() & (1 << self.val['_M_offset']))
+        return bool(self.val['_M_p'].dereference()
+                    & (1 << self.val['_M_offset']))
 
 
 class StdBitReferencePrinter:
@@ -1087,9 +1088,10 @@ class StdStringStreamPrinter:
         self.val = val
         self.typename = typename
 
-        # Check if the stream was redirected:
-        # This is essentially: val['_M_streambuf'] == val['_M_stringbuf'].address
-        # However, GDB can't resolve the virtual inheritance, so we do that manually
+        # Check if the stream was redirected. This is essentially:
+        # val['_M_streambuf'] != val['_M_stringbuf'].address
+        # However, GDB can't resolve the virtual inheritance, so we do that
+        # manually.
         basetype = [f.type for f in val.type.fields() if f.is_base_class][0]
         gdb.set_convenience_variable('__stream', val.cast(basetype).address)
         self.streambuf = gdb.parse_and_eval('$__stream->rdbuf()')
@@ -1097,7 +1099,8 @@ class StdStringStreamPrinter:
 
     def to_string(self):
         if self.was_redirected:
-            return "%s redirected to %s" % (self.typename, self.streambuf.dereference())
+            return "%s redirected to %s" % (
+                self.typename, self.streambuf.dereference())
         return self.val['_M_stringbuf']
 
     def display_hint(self):
@@ -1309,8 +1312,9 @@ class SingleObjContainerPrinter(object):
         return self._contained(self.contained_value)
 
     def display_hint(self):
-        # if contained value is a map we want to display in the same way
-        if hasattr(self.visualizer, 'children') and hasattr(self.visualizer, 'display_hint'):
+        if (hasattr(self.visualizer, 'children')
+                and hasattr(self.visualizer, 'display_hint')):
+            # If contained value is a map we want to display in the same way.
             return self.visualizer.display_hint()
         return self.hint
 
@@ -1344,8 +1348,8 @@ class StdExpAnyPrinter(SingleObjContainerPrinter):
 
     def __init__(self, typename, val):
         self.typename = strip_versioned_namespace(typename)
-        self.typename = re.sub(
-            '^std::experimental::fundamentals_v\d::', 'std::experimental::', self.typename, 1)
+        self.typename = re.sub(r'^std::experimental::fundamentals_v\d::',
+                               'std::experimental::', self.typename, 1)
         self.val = val
         self.contained_type = None
         contained_value = None
@@ -1356,7 +1360,16 @@ class StdExpAnyPrinter(SingleObjContainerPrinter):
             if not func:
                 raise ValueError(
                     "Invalid function pointer in %s" % (self.typename))
-            rx = r"""({0}::_Manager_\w+<.*>)::_S_manage\((enum )?{0}::_Op, (const {0}|{0} const) ?\*, (union )?{0}::_Arg ?\*\)""".format(typename)
+            # We want to use this regular expression:
+            # T::_Manager_xxx<.*>::_S_manage\(T::_Op, const T\*, T::_Arg\*\)
+            # where T is std::any or std::experimental::any.
+            # But we need to account for variances in demangled names
+            # between GDB versions, e.g. 'enum T::_Op' instead of 'T::_Op'.
+            rx = (
+                r"({0}::_Manager_\w+<.*>)::_S_manage\("
+                r"(enum )?{0}::_Op, (const {0}|{0} const) ?\*, "
+                r"(union )?{0}::_Arg ?\*\)"
+            ).format(typename)
             m = re.match(rx, func)
             if not m:
                 raise ValueError(
@@ -1365,23 +1378,13 @@ class StdExpAnyPrinter(SingleObjContainerPrinter):
             mgrname = m.group(1)
             # FIXME need to expand 'std::string' so that gdb.lookup_type works
             if 'std::string' in mgrname:
-                # This lookup for std::string might return the __cxx11 version,
-                # but that's not necessarily the one used by the std::any
-                # manager function we're trying to find.
-                strings = {
-                    str(gdb.lookup_type('std::string').strip_typedefs())}
-                # So also consider all the other possible std::string types!
-                s = 'basic_string<char, std::char_traits<char>, std::allocator<char> >'
-                quals = ['std::', 'std::__cxx11::',
-                         'std::' + _versioned_namespace]
-                strings |= {q+s for q in quals}  # set of unique strings
                 mgrtypes = []
-                for s in strings:
+                for s in _string_types():
                     try:
-                        x = re.sub("std::string(?!\w)", s, m.group(1))
+                        x = re.sub(r"std::string(?!\w)", s, m.group(1))
                         # The following lookup might raise gdb.error if the
-                        # manager function was never instantiated for 's' in the
-                        # program, because there will be no such type.
+                        # manager function was never instantiated for 's' in
+                        # the program, because there will be no such type.
                         mgrtypes.append(gdb.lookup_type(x))
                     except gdb.error:
                         pass
@@ -1392,7 +1395,9 @@ class StdExpAnyPrinter(SingleObjContainerPrinter):
                     # Maybe find the address of each type's _S_manage and
                     # compare to the address stored in _M_manager?
                     raise ValueError(
-                        'Cannot uniquely determine std::string type used in std::any')
+                        'Cannot uniquely determine std::string type '
+                        'used in std::any'
+                    )
                 mgrtype = mgrtypes[0]
             else:
                 mgrtype = gdb.lookup_type(mgrname)
@@ -1419,6 +1424,19 @@ class StdExpAnyPrinter(SingleObjContainerPrinter):
         valtype = self._recognize(self.contained_type)
         return desc + strip_versioned_namespace(str(valtype))
 
+    @staticmethod
+    def _string_types():
+        # This lookup for std::string might return the __cxx11 version,
+        # but that's not necessarily the one used by the std::any
+        # manager function we're trying to find.
+        strings = {str(gdb.lookup_type('std::string').strip_typedefs())}
+        # So also consider all the other possible std::string types!
+        s = 'basic_string<char, std::char_traits<char>, std::allocator<char> >'
+        quals = ['std::', 'std::__cxx11::',
+                 'std::' + _versioned_namespace]
+        strings |= {q + s for q in quals}  # set of unique strings
+        return strings
+
 
 class StdExpOptionalPrinter(SingleObjContainerPrinter):
     """Print a std::optional or std::experimental::optional."""
@@ -1427,7 +1445,8 @@ class StdExpOptionalPrinter(SingleObjContainerPrinter):
         valtype = self._recognize(val.type.template_argument(0))
         typename = strip_versioned_namespace(typename)
         self.typename = re.sub(
-            '^std::(experimental::|)(fundamentals_v\d::|)(.*)', r'std::\1\3', typename, 1)
+            r'^std::(experimental::|)(fundamentals_v\d::|)(.*)',
+            r'std::\1\3', typename, 1)
         payload = val['_M_payload']
         if self.typename.startswith('std::experimental'):
             engaged = val['_M_engaged']
@@ -1636,7 +1655,7 @@ class StdPathPrinter:
                 # We can't access _Impl::_M_size because _Impl is incomplete
                 # so cast to int* to access the _M_size member at offset zero,
                 int_type = gdb.lookup_type('int')
-                cmpt_type = gdb.lookup_type(pathtype+'::_Cmpt')
+                cmpt_type = gdb.lookup_type(pathtype + '::_Cmpt')
                 char_type = gdb.lookup_type('char')
                 impl = impl.cast(int_type.pointer())
                 size = impl.dereference()
@@ -1710,7 +1729,7 @@ class StdCmpCatPrinter:
     """Print a comparison category object."""
 
     def __init__(self, typename, val):
-        self.typename = typename[typename.rfind(':')+1:]
+        self.typename = typename[typename.rfind(':') + 1:]
         self.val = val['_M_value']
 
     def to_string(self):
@@ -1771,14 +1790,15 @@ class StdErrorCodePrinter:
 
     @classmethod
     def _category_info(cls, cat):
-        "Return details of a std::error_category"
+        """Return details of a std::error_category."""
 
         name = None
         enum = None
         is_errno = False
 
         # Try these first, or we get "warning: RTTI symbol not found" when
-        # using cat.dynamic_type on the local class types for Net TS categories.
+        # using cat.dynamic_type on the local class types for Net TS
+        # categories.
         func, enum = cls._match_net_ts_category(cat)
         if func is not None:
             return (None, func, enum, is_errno)
@@ -1896,7 +1916,8 @@ class StdSpanPrinter:
     def __init__(self, typename, val):
         self.typename = strip_versioned_namespace(typename)
         self.val = val
-        if val.type.template_argument(1) == gdb.parse_and_eval('static_cast<std::size_t>(-1)'):
+        size_max = gdb.parse_and_eval('static_cast<std::size_t>(-1)')
+        if val.type.template_argument(1) == size_max:
             self.size = val['_M_extent']['_M_extent_value']
         else:
             self.size = val.type.template_argument(1)
@@ -1939,7 +1960,8 @@ class StdAtomicPrinter:
         self.value_type = self.val.type.template_argument(0)
         if self.value_type.tag is not None:
             typ = strip_versioned_namespace(self.value_type.tag)
-            if typ.startswith('std::shared_ptr<') or typ.startswith('std::weak_ptr<'):
+            if (typ.startswith('std::shared_ptr<')
+                    or typ.startswith('std::weak_ptr<')):
                 impl = val['_M_impl']
                 self.shptr_printer = SharedPointerPrinter(typename, impl)
                 self.children = self._shptr_children
@@ -2500,11 +2522,11 @@ def add_one_template_type_printer(obj, name, defargs):
       3: 'std::equal_to<{0}>',
       4: 'std::allocator<std::pair<const {0}, {1}> >' }
     """
-    printer = TemplateTypePrinter('std::'+name, defargs)
+    printer = TemplateTypePrinter('std::' + name, defargs)
     gdb.types.register_type_printer(obj, printer)
 
     # Add type printer for same type in debug namespace:
-    printer = TemplateTypePrinter('std::__debug::'+name, defargs)
+    printer = TemplateTypePrinter('std::__debug::' + name, defargs)
     gdb.types.register_type_printer(obj, printer)
 
     if _versioned_namespace and not '__cxx11' in name:
@@ -2513,11 +2535,11 @@ def add_one_template_type_printer(obj, name, defargs):
         # PR 86112 Cannot use dict comprehension here:
         defargs = dict((n, d.replace('std::', ns))
                        for (n, d) in defargs.items())
-        printer = TemplateTypePrinter(ns+name, defargs)
+        printer = TemplateTypePrinter(ns + name, defargs)
         gdb.types.register_type_printer(obj, printer)
 
         # Add type printer for same type in debug namespace:
-        printer = TemplateTypePrinter('std::__debug::'+name, defargs)
+        printer = TemplateTypePrinter('std::__debug::' + name, defargs)
         gdb.types.register_type_printer(obj, printer)
 
 
@@ -2568,7 +2590,8 @@ class FilteringTypePrinter(object):
 
             if self.type_obj is None:
                 if self.targ1 is not None:
-                    if not type_obj.tag.startswith('{}<{}'.format(self.template, self.targ1)):
+                    s = '{}<{}'.format(self.template, self.targ1)
+                    if not type_obj.tag.startswith(s):
                         # Filter didn't match.
                         return None
                 elif not type_obj.tag.startswith(self.template):
@@ -2583,12 +2606,17 @@ class FilteringTypePrinter(object):
             if self.type_obj is None:
                 return None
 
-            if gdb.types.get_basic_type(self.type_obj) == gdb.types.get_basic_type(type_obj):
+            t1 = gdb.types.get_basic_type(self.type_obj)
+            t2 = gdb.types.get_basic_type(type_obj)
+            if t1 == t2:
                 return strip_inline_namespaces(self.name)
 
-            # Workaround ambiguous typedefs matching both std:: and std::__cxx11:: symbols.
+            # Workaround ambiguous typedefs matching both std:: and
+            # std::__cxx11:: symbols.
             if self.template.split('::')[-1] == 'basic_string':
-                if self.type_obj.tag.replace('__cxx11::', '') == type_obj.tag.replace('__cxx11::', ''):
+                s1 = self.type_obj.tag.replace('__cxx11::', '')
+                s2 = type_obj.tag.replace('__cxx11::', '')
+                if s1 == s2:
                     return strip_inline_namespaces(self.name)
 
             return None
-- 
2.41.0

Re: [committed] libstdc++: Reformat Python code

[Romain GEISSLER]

> Le 13 nov. 2023 à 16:24, Jonathan Wakely <jwakely@redhat.com> a écrit :
> 
> Yes, I'll do that backport (and most of the other Python improvements
> too, at least for gcc-13).
> 
> Thanks for raising it.
> 

Cool thanks ! ;)

In the meantime, in my own toolchains I have silenced (without fixing it) the warnings
with this simple patch (in case anyone else wants to quickly get rid of it, but it’s not
really a good long term workaround).

--- libstdc++-v3/python/libstdcxx/v6/__init__.py
+++ libstdc++-v3/python/libstdcxx/v6/__init__.py
@@ -1 +1,2 @@
-
+import warnings
+warnings.filterwarnings("ignore", category=SyntaxWarning)

Re: [committed] libstdc++: Reformat Python code

[Jonathan Wakely]

On Mon, 13 Nov 2023 at 14:41, Romain GEISSLER
<romain.geissler@amadeus.com> wrote:
>
> > Le 28 sept. 2023 à 22:21, Jonathan Wakely <jwakely redhat ! com> a écrit :
> >
> > Tested x86_64-linux (GDB 13.2, Python 3.11). Pushed to trunk.
> >
> > -- >8 --
> >
> > Some of these changes were suggested by autopep8's --aggressive
> > option, others are for readability.
> >
> > Break long lines by splitting strings across multiple lines, or
> > introducing local variables to hold results.
> >
> > Use raw strings for regular expressions, so that backslashes don't need
> > to be escaped.
>
> Hi Jonathan,
>
> FYI, it seems that with python 3.12, the bits "Use raw strings for regular expressions"
> seems to fix the following new Python warnings:
>
> /opt/1A/toolchain/x86_64-v23.0.19/lib/../share/gcc-13.2.1/python/libstdcxx/v6/printers.py:1273: SyntaxWarning: invalid escape sequence '\d'
>   self.typename = re.sub('^std::experimental::fundamentals_v\d::', 'std::experimental::', self.typename, 1)
> /opt/1A/toolchain/x86_64-v23.0.19/lib/../share/gcc-13.2.1/python/libstdcxx/v6/printers.py:1302: SyntaxWarning: invalid escape sequence '\w'
>   x = re.sub("std::string(?!\w)", s, m.group(1))
>  … (snapped, there are a bit more than that in total).
>
> How ok would it be to backport to the branches still maintained the "raw string" fix,
> in order to avoid deprecation warnings as soon as people use gdb with python >= 3.12 ?

Yes, I'll do that backport (and most of the other Python improvements
too, at least for gcc-13).

Thanks for raising it.

Re: [committed] libstdc++: Reformat Python code

[Romain GEISSLER]

> Le 28 sept. 2023 à 22:21, Jonathan Wakely <jwakely redhat ! com> a écrit :
> 
> Tested x86_64-linux (GDB 13.2, Python 3.11). Pushed to trunk.
> 
> -- >8 --
> 
> Some of these changes were suggested by autopep8's --aggressive
> option, others are for readability.
> 
> Break long lines by splitting strings across multiple lines, or
> introducing local variables to hold results.
> 
> Use raw strings for regular expressions, so that backslashes don't need
> to be escaped.

Hi Jonathan,

FYI, it seems that with python 3.12, the bits "Use raw strings for regular expressions"
seems to fix the following new Python warnings:

/opt/1A/toolchain/x86_64-v23.0.19/lib/../share/gcc-13.2.1/python/libstdcxx/v6/printers.py:1273: SyntaxWarning: invalid escape sequence '\d'
  self.typename = re.sub('^std::experimental::fundamentals_v\d::', 'std::experimental::', self.typename, 1)
/opt/1A/toolchain/x86_64-v23.0.19/lib/../share/gcc-13.2.1/python/libstdcxx/v6/printers.py:1302: SyntaxWarning: invalid escape sequence '\w'
  x = re.sub("std::string(?!\w)", s, m.group(1))
 … (snapped, there are a bit more than that in total).

How ok would it be to backport to the branches still maintained the "raw string" fix,
in order to avoid deprecation warnings as soon as people use gdb with python >= 3.12 ?

Thanks,
Romain