Re: [PATCH] [gdb/build] Fix static cast of virtual base

[Andrew Burgess <aburgess@redhat.com>]

Simon Marchi <simon.marchi@polymtl.ca> writes:

> On 3/6/24 12:39, Andrew Burgess wrote:
>> Tom de Vries <tdevries@suse.de> writes:
>> 
>>> With this change in bfd/development.sh:
>>> ...
>>> -development=true
>>> +development=false
>>> ...
>>> we run into:
>>> ...
>>> In file included from tui-data.h:28:0,
>>>                  from tui-command.c:24:
>>> gdb-checked-static-cast.h: In instantiation of \
>>>   ‘T gdb::checked_static_cast(V*) [with T = tui_cmd_window*; V = tui_win_info]’:
>>> tui-command.c:65:15:   required from here
>>> gdb-checked-static-cast.h:63:14: error: cannot convert from pointer to base \
>>>   class ‘tui_win_info’ to pointer to derived class ‘tui_cmd_window’ because \
>>>   the base is virtual
>>>    T result = static_cast<T> (v);
>>>               ^~~~~~~~~~~~~~~~~~
>> 
>> Issues like there were supposed to be spotted by the static_asserts in
>> gdb::checked_static_cast.  Clearly the existing asserts are not good
>> enough.
>> 
>> I'd like to propose the patch below which would have prevented this
>> issue sneaking in unseen.  Thoughts?
>> 
>> Thanks,
>> Andrew
>> 
>> ---
>> 
>> commit 9d7cb56af0361a2276621203e0e508b6fda780f9
>> Author: Andrew Burgess <aburgess@redhat.com>
>> Date:   Wed Mar 6 17:28:48 2024 +0000
>> 
>>     gdb: use static_cast in gdb::checked_static_cast
>>     
>>     This commit:
>>     
>>       commit 6fe4779ac4b1874c995345e3eabd89cb1a05fbdf
>>       Date:   Sat Feb 24 11:00:20 2024 +0100
>>     
>>           [gdb/build] Fix static cast of virtual base
>>     
>>     addressed an issue where GDB would not compile in production mode due
>>     to a use of gdb::checked_static_cast.  The problem was that we were
>>     asking GDB to cast from a virtual base class to a sub-class, this
>>     works fine when using dynamic_cast, but does not work with
>>     static_cast.
>>     
>>     The gdb::checked_static_cast actually uses dynamic_cast under the hood
>>     in development mode in order to ensure that the cast is valid, while
>>     in a production build we use static_cast as this is more efficient.
>>     
>>     What this meant however, was that when gdb::checked_static_cast was
>>     used to cast from a virtual base class, the dynamic_cast of a
>>     non-production build worked fine, while the production build's
>>     static_cast caused a build failure.
>>     
>>     However, the gdb::checked_static_cast function already contains some
>>     static_assert calls that are intended to catch any issues with invalid
>>     type casting, the goal of these asserts was to prevent issues like
>>     this: the build only failing in production mode.  Clearly the current
>>     asserts are not enough.
>>     
>>     I don't think there is a std::is_virtual_base type trait check, so
>>     what I propose instead is that in non-production mode we also make use
>>     of static_cast.  This will ensure that any errors that crop up in
>>     production mode should also be revealed in non-production mode, and
>>     should catch issues like this in the future.
>>     
>>     There should be no user visible changes after this commit.
>>     
>>     Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31399
>> 
>> diff --git a/gdbsupport/gdb-checked-static-cast.h b/gdbsupport/gdb-checked-static-cast.h
>> index 227010e46ea..33cab48cd9b 100644
>> --- a/gdbsupport/gdb-checked-static-cast.h
>> +++ b/gdbsupport/gdb-checked-static-cast.h
>> @@ -57,8 +57,14 @@ checked_static_cast (V *v)
>>    if (v == nullptr)
>>      return nullptr;
>>  
>> -  T result = dynamic_cast<T> (v);
>> -  gdb_assert (result != nullptr);
>> +  gdb_assert (dynamic_cast<T> (v) != nullptr);
>> +
>> +  /* If a base class of V is virtual then the dynamic_cast will succeed,
>> +     but the production mode static_cast will fail.  So having checked with
>> +     the dynamic_cast that we didn't get nullptr, now use static_cast to
>> +     catch the virtual base case.  This has the side effect of guaranteeing
>> +     that GDB will compile in production mode.  */
>> +  T result = static_cast<T> (v);
>>  #else
>>    T result = static_cast<T> (v);
>>  #endif
>> 
>
> This is similar to what I proposed here:
>
> https://inbox.sourceware.org/gdb-patches/24af4ea8-5426-4ce4-b1c5-12858b38a952@simark.ca/
>
> The idea is the same, to have a static_cast in the DEVELOPMENT branch.
> I kinda like my version better, as it factors out the static cast
> (notice that both branches have identical static_cast lines after your
> patch) and the  ifdef is just around a single assert.  Also, I'm pretty
> sure the nullptr check is not necessary, as both dynamic_cast and
> static_cast can handle it.

I agree yours is better.  Do you have an official patch somewhere that I
can add a +1 too?  If not then below is an updated version inline with
you proposal.  Feel free to push your version if you have it somewhere.

Thanks,
Andrew

--

commit 2a82aa7d140387fd728641846d9c4ec7d770b067
Author: Andrew Burgess <aburgess@redhat.com>
Date:   Wed Mar 6 17:28:48 2024 +0000

    gdb: use static_cast in gdb::checked_static_cast
    
    This commit:
    
      commit 6fe4779ac4b1874c995345e3eabd89cb1a05fbdf
      Date:   Sat Feb 24 11:00:20 2024 +0100
    
          [gdb/build] Fix static cast of virtual base
    
    addressed an issue where GDB would not compile in production mode due
    to a use of gdb::checked_static_cast.  The problem was that we were
    asking GDB to cast from a virtual base class to a sub-class, this
    works fine when using dynamic_cast, but does not work with
    static_cast.
    
    The gdb::checked_static_cast actually uses dynamic_cast under the hood
    in development mode in order to ensure that the cast is valid, while
    in a production build we use static_cast as this is more efficient.
    
    What this meant however, was that when gdb::checked_static_cast was
    used to cast from a virtual base class, the dynamic_cast of a
    non-production build worked fine, while the production build's
    static_cast caused a build failure.
    
    However, the gdb::checked_static_cast function already contains some
    static_assert calls that are intended to catch any issues with invalid
    type casting, the goal of these asserts was to prevent issues like
    this: the build only failing in production mode.  Clearly the current
    asserts are not enough.
    
    I don't think there is a std::is_virtual_base type trait check, so
    what I propose instead is that in non-production mode we also make use
    of static_cast.  This will ensure that any errors that crop up in
    production mode should also be revealed in non-production mode, and
    should catch issues like this in the future.
    
    There should be no user visible changes after this commit.
    
    Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31399
    
    Co-Authored-By: Simon Marchi <simon.marchi@polymtl.ca>

diff --git a/gdbsupport/gdb-checked-static-cast.h b/gdbsupport/gdb-checked-static-cast.h
index 227010e46ea..24fa7a4ba04 100644
--- a/gdbsupport/gdb-checked-static-cast.h
+++ b/gdbsupport/gdb-checked-static-cast.h
@@ -54,16 +54,12 @@ checked_static_cast (V *v)
 		 "types must be related");
 
 #ifdef DEVELOPMENT
-  if (v == nullptr)
-    return nullptr;
-
-  T result = dynamic_cast<T> (v);
-  gdb_assert (result != nullptr);
-#else
-  T result = static_cast<T> (v);
+  gdb_assert (v == nullptr || dynamic_cast<T> (v) != nullptr);
 #endif
 
-  return result;
+  /* If a base class of V is virtual then the dynamic_cast above will
+     succeed, but this static_cast will fail.  */
+  return static_cast<T> (v);
 }
 
 /* Same as the above, but to cast from a reference type to another.  */