Re: [PATCH] [gdb/build] Fix static cast of virtual base

[Simon Marchi <simon.marchi@polymtl.ca>]

On 3/6/24 12:39, Andrew Burgess wrote:
> Tom de Vries <tdevries@suse.de> writes:
> 
>> With this change in bfd/development.sh:
>> ...
>> -development=true
>> +development=false
>> ...
>> we run into:
>> ...
>> In file included from tui-data.h:28:0,
>>                  from tui-command.c:24:
>> gdb-checked-static-cast.h: In instantiation of \
>>   ‘T gdb::checked_static_cast(V*) [with T = tui_cmd_window*; V = tui_win_info]’:
>> tui-command.c:65:15:   required from here
>> gdb-checked-static-cast.h:63:14: error: cannot convert from pointer to base \
>>   class ‘tui_win_info’ to pointer to derived class ‘tui_cmd_window’ because \
>>   the base is virtual
>>    T result = static_cast<T> (v);
>>               ^~~~~~~~~~~~~~~~~~
> 
> Issues like there were supposed to be spotted by the static_asserts in
> gdb::checked_static_cast.  Clearly the existing asserts are not good
> enough.
> 
> I'd like to propose the patch below which would have prevented this
> issue sneaking in unseen.  Thoughts?
> 
> Thanks,
> Andrew
> 
> ---
> 
> commit 9d7cb56af0361a2276621203e0e508b6fda780f9
> Author: Andrew Burgess <aburgess@redhat.com>
> Date:   Wed Mar 6 17:28:48 2024 +0000
> 
>     gdb: use static_cast in gdb::checked_static_cast
>     
>     This commit:
>     
>       commit 6fe4779ac4b1874c995345e3eabd89cb1a05fbdf
>       Date:   Sat Feb 24 11:00:20 2024 +0100
>     
>           [gdb/build] Fix static cast of virtual base
>     
>     addressed an issue where GDB would not compile in production mode due
>     to a use of gdb::checked_static_cast.  The problem was that we were
>     asking GDB to cast from a virtual base class to a sub-class, this
>     works fine when using dynamic_cast, but does not work with
>     static_cast.
>     
>     The gdb::checked_static_cast actually uses dynamic_cast under the hood
>     in development mode in order to ensure that the cast is valid, while
>     in a production build we use static_cast as this is more efficient.
>     
>     What this meant however, was that when gdb::checked_static_cast was
>     used to cast from a virtual base class, the dynamic_cast of a
>     non-production build worked fine, while the production build's
>     static_cast caused a build failure.
>     
>     However, the gdb::checked_static_cast function already contains some
>     static_assert calls that are intended to catch any issues with invalid
>     type casting, the goal of these asserts was to prevent issues like
>     this: the build only failing in production mode.  Clearly the current
>     asserts are not enough.
>     
>     I don't think there is a std::is_virtual_base type trait check, so
>     what I propose instead is that in non-production mode we also make use
>     of static_cast.  This will ensure that any errors that crop up in
>     production mode should also be revealed in non-production mode, and
>     should catch issues like this in the future.
>     
>     There should be no user visible changes after this commit.
>     
>     Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31399
> 
> diff --git a/gdbsupport/gdb-checked-static-cast.h b/gdbsupport/gdb-checked-static-cast.h
> index 227010e46ea..33cab48cd9b 100644
> --- a/gdbsupport/gdb-checked-static-cast.h
> +++ b/gdbsupport/gdb-checked-static-cast.h
> @@ -57,8 +57,14 @@ checked_static_cast (V *v)
>    if (v == nullptr)
>      return nullptr;
>  
> -  T result = dynamic_cast<T> (v);
> -  gdb_assert (result != nullptr);
> +  gdb_assert (dynamic_cast<T> (v) != nullptr);
> +
> +  /* If a base class of V is virtual then the dynamic_cast will succeed,
> +     but the production mode static_cast will fail.  So having checked with
> +     the dynamic_cast that we didn't get nullptr, now use static_cast to
> +     catch the virtual base case.  This has the side effect of guaranteeing
> +     that GDB will compile in production mode.  */
> +  T result = static_cast<T> (v);
>  #else
>    T result = static_cast<T> (v);
>  #endif
> 

This is similar to what I proposed here:

https://inbox.sourceware.org/gdb-patches/24af4ea8-5426-4ce4-b1c5-12858b38a952@simark.ca/

The idea is the same, to have a static_cast in the DEVELOPMENT branch.
I kinda like my version better, as it factors out the static cast
(notice that both branches have identical static_cast lines after your
patch) and the  ifdef is just around a single assert.  Also, I'm pretty
sure the nullptr check is not necessary, as both dynamic_cast and
static_cast can handle it.

Simon