Re: [PATCH] [gdb/build] Fix static cast of virtual base

[Andrew Burgess <aburgess@redhat.com>]

Tom de Vries <tdevries@suse.de> writes:

> With this change in bfd/development.sh:
> ...
> -development=true
> +development=false
> ...
> we run into:
> ...
> In file included from tui-data.h:28:0,
>                  from tui-command.c:24:
> gdb-checked-static-cast.h: In instantiation of \
>   ‘T gdb::checked_static_cast(V*) [with T = tui_cmd_window*; V = tui_win_info]’:
> tui-command.c:65:15:   required from here
> gdb-checked-static-cast.h:63:14: error: cannot convert from pointer to base \
>   class ‘tui_win_info’ to pointer to derived class ‘tui_cmd_window’ because \
>   the base is virtual
>    T result = static_cast<T> (v);
>               ^~~~~~~~~~~~~~~~~~

Issues like there were supposed to be spotted by the static_asserts in
gdb::checked_static_cast.  Clearly the existing asserts are not good
enough.

I'd like to propose the patch below which would have prevented this
issue sneaking in unseen.  Thoughts?

Thanks,
Andrew

---

commit 9d7cb56af0361a2276621203e0e508b6fda780f9
Author: Andrew Burgess <aburgess@redhat.com>
Date:   Wed Mar 6 17:28:48 2024 +0000

    gdb: use static_cast in gdb::checked_static_cast
    
    This commit:
    
      commit 6fe4779ac4b1874c995345e3eabd89cb1a05fbdf
      Date:   Sat Feb 24 11:00:20 2024 +0100
    
          [gdb/build] Fix static cast of virtual base
    
    addressed an issue where GDB would not compile in production mode due
    to a use of gdb::checked_static_cast.  The problem was that we were
    asking GDB to cast from a virtual base class to a sub-class, this
    works fine when using dynamic_cast, but does not work with
    static_cast.
    
    The gdb::checked_static_cast actually uses dynamic_cast under the hood
    in development mode in order to ensure that the cast is valid, while
    in a production build we use static_cast as this is more efficient.
    
    What this meant however, was that when gdb::checked_static_cast was
    used to cast from a virtual base class, the dynamic_cast of a
    non-production build worked fine, while the production build's
    static_cast caused a build failure.
    
    However, the gdb::checked_static_cast function already contains some
    static_assert calls that are intended to catch any issues with invalid
    type casting, the goal of these asserts was to prevent issues like
    this: the build only failing in production mode.  Clearly the current
    asserts are not enough.
    
    I don't think there is a std::is_virtual_base type trait check, so
    what I propose instead is that in non-production mode we also make use
    of static_cast.  This will ensure that any errors that crop up in
    production mode should also be revealed in non-production mode, and
    should catch issues like this in the future.
    
    There should be no user visible changes after this commit.
    
    Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31399

diff --git a/gdbsupport/gdb-checked-static-cast.h b/gdbsupport/gdb-checked-static-cast.h
index 227010e46ea..33cab48cd9b 100644
--- a/gdbsupport/gdb-checked-static-cast.h
+++ b/gdbsupport/gdb-checked-static-cast.h
@@ -57,8 +57,14 @@ checked_static_cast (V *v)
   if (v == nullptr)
     return nullptr;
 
-  T result = dynamic_cast<T> (v);
-  gdb_assert (result != nullptr);
+  gdb_assert (dynamic_cast<T> (v) != nullptr);
+
+  /* If a base class of V is virtual then the dynamic_cast will succeed,
+     but the production mode static_cast will fail.  So having checked with
+     the dynamic_cast that we didn't get nullptr, now use static_cast to
+     catch the virtual base case.  This has the side effect of guaranteeing
+     that GDB will compile in production mode.  */
+  T result = static_cast<T> (v);
 #else
   T result = static_cast<T> (v);
 #endif