[PATCH] [gdb/build] Fix static cast of virtual base

[PATCH] [gdb/build] Fix static cast of virtual base

[Tom de Vries]

With this change in bfd/development.sh:
...
-development=true
+development=false
...
we run into:
...
In file included from tui-data.h:28:0,
                 from tui-command.c:24:
gdb-checked-static-cast.h: In instantiation of \
  ‘T gdb::checked_static_cast(V*) [with T = tui_cmd_window*; V = tui_win_info]’:
tui-command.c:65:15:   required from here
gdb-checked-static-cast.h:63:14: error: cannot convert from pointer to base \
  class ‘tui_win_info’ to pointer to derived class ‘tui_cmd_window’ because \
  the base is virtual
   T result = static_cast<T> (v);
              ^~~~~~~~~~~~~~~~~~
...

Fix this by using dynamic_cast instead of gdb::checked_static_cast in
TUI_CMD_WIN and TUI_STATUS_WIN.

Tested on x86_64-linux, with development set to false.

Reported-By: Robert Xiao <spam_hole@shaw.ca>
Reported-By: Simon Marchi <simark@simark.ca>

PR build/31399
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31399
---
 gdb/tui/tui-data.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gdb/tui/tui-data.h b/gdb/tui/tui-data.h
index 1714f5ae455..90ab01f79af 100644
--- a/gdb/tui/tui-data.h
+++ b/gdb/tui/tui-data.h
@@ -296,9 +296,9 @@ extern struct tui_win_info *tui_win_list[MAX_MAJOR_WINDOWS];
 #define TUI_DATA_WIN \
   (gdb::checked_static_cast<tui_data_window *> (tui_win_list[DATA_WIN]))
 #define TUI_CMD_WIN \
-  (gdb::checked_static_cast<tui_cmd_window *> (tui_win_list[CMD_WIN]))
+  (dynamic_cast<tui_cmd_window *> (tui_win_list[CMD_WIN]))
 #define TUI_STATUS_WIN \
-  (gdb::checked_static_cast<tui_status_window *> (tui_win_list[STATUS_WIN]))
+  (dynamic_cast<tui_status_window *> (tui_win_list[STATUS_WIN]))
 
 /* All the windows that are currently instantiated, in layout
    order.  */

base-commit: 94aadbf48f608a2d65d4a0ba259f43dada16fda4
-- 
2.35.3

Re: [PATCH] [gdb/build] Fix static cast of virtual base

[Tom Tromey]

>>>>> "Tom" == Tom de Vries <tdevries@suse.de> writes:

Tom> Fix this by using dynamic_cast instead of gdb::checked_static_cast in
Tom> TUI_CMD_WIN and TUI_STATUS_WIN.

Tom> Tested on x86_64-linux, with development set to false.

Tom> Reported-By: Robert Xiao <spam_hole@shaw.ca>
Tom> Reported-By: Simon Marchi <simark@simark.ca>

Tom> PR build/31399
Tom> Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31399

Ok.  Approved-By: Tom Tromey <tom@tromey.com>

Tom

Re: [PATCH] [gdb/build] Fix static cast of virtual base

[Andrew Burgess]

Tom de Vries <tdevries@suse.de> writes:

> With this change in bfd/development.sh:
> ...
> -development=true
> +development=false
> ...
> we run into:
> ...
> In file included from tui-data.h:28:0,
>                  from tui-command.c:24:
> gdb-checked-static-cast.h: In instantiation of \
>   ‘T gdb::checked_static_cast(V*) [with T = tui_cmd_window*; V = tui_win_info]’:
> tui-command.c:65:15:   required from here
> gdb-checked-static-cast.h:63:14: error: cannot convert from pointer to base \
>   class ‘tui_win_info’ to pointer to derived class ‘tui_cmd_window’ because \
>   the base is virtual
>    T result = static_cast<T> (v);
>               ^~~~~~~~~~~~~~~~~~

Issues like there were supposed to be spotted by the static_asserts in
gdb::checked_static_cast.  Clearly the existing asserts are not good
enough.

I'd like to propose the patch below which would have prevented this
issue sneaking in unseen.  Thoughts?

Thanks,
Andrew

---

commit 9d7cb56af0361a2276621203e0e508b6fda780f9
Author: Andrew Burgess <aburgess@redhat.com>
Date:   Wed Mar 6 17:28:48 2024 +0000

    gdb: use static_cast in gdb::checked_static_cast
    
    This commit:
    
      commit 6fe4779ac4b1874c995345e3eabd89cb1a05fbdf
      Date:   Sat Feb 24 11:00:20 2024 +0100
    
          [gdb/build] Fix static cast of virtual base
    
    addressed an issue where GDB would not compile in production mode due
    to a use of gdb::checked_static_cast.  The problem was that we were
    asking GDB to cast from a virtual base class to a sub-class, this
    works fine when using dynamic_cast, but does not work with
    static_cast.
    
    The gdb::checked_static_cast actually uses dynamic_cast under the hood
    in development mode in order to ensure that the cast is valid, while
    in a production build we use static_cast as this is more efficient.
    
    What this meant however, was that when gdb::checked_static_cast was
    used to cast from a virtual base class, the dynamic_cast of a
    non-production build worked fine, while the production build's
    static_cast caused a build failure.
    
    However, the gdb::checked_static_cast function already contains some
    static_assert calls that are intended to catch any issues with invalid
    type casting, the goal of these asserts was to prevent issues like
    this: the build only failing in production mode.  Clearly the current
    asserts are not enough.
    
    I don't think there is a std::is_virtual_base type trait check, so
    what I propose instead is that in non-production mode we also make use
    of static_cast.  This will ensure that any errors that crop up in
    production mode should also be revealed in non-production mode, and
    should catch issues like this in the future.
    
    There should be no user visible changes after this commit.
    
    Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31399

diff --git a/gdbsupport/gdb-checked-static-cast.h b/gdbsupport/gdb-checked-static-cast.h
index 227010e46ea..33cab48cd9b 100644
--- a/gdbsupport/gdb-checked-static-cast.h
+++ b/gdbsupport/gdb-checked-static-cast.h
@@ -57,8 +57,14 @@ checked_static_cast (V *v)
   if (v == nullptr)
     return nullptr;
 
-  T result = dynamic_cast<T> (v);
-  gdb_assert (result != nullptr);
+  gdb_assert (dynamic_cast<T> (v) != nullptr);
+
+  /* If a base class of V is virtual then the dynamic_cast will succeed,
+     but the production mode static_cast will fail.  So having checked with
+     the dynamic_cast that we didn't get nullptr, now use static_cast to
+     catch the virtual base case.  This has the side effect of guaranteeing
+     that GDB will compile in production mode.  */
+  T result = static_cast<T> (v);
 #else
   T result = static_cast<T> (v);
 #endif

Re: [PATCH] [gdb/build] Fix static cast of virtual base

[Simon Marchi]

On 3/6/24 12:39, Andrew Burgess wrote:
> Tom de Vries <tdevries@suse.de> writes:
> 
>> With this change in bfd/development.sh:
>> ...
>> -development=true
>> +development=false
>> ...
>> we run into:
>> ...
>> In file included from tui-data.h:28:0,
>>                  from tui-command.c:24:
>> gdb-checked-static-cast.h: In instantiation of \
>>   ‘T gdb::checked_static_cast(V*) [with T = tui_cmd_window*; V = tui_win_info]’:
>> tui-command.c:65:15:   required from here
>> gdb-checked-static-cast.h:63:14: error: cannot convert from pointer to base \
>>   class ‘tui_win_info’ to pointer to derived class ‘tui_cmd_window’ because \
>>   the base is virtual
>>    T result = static_cast<T> (v);
>>               ^~~~~~~~~~~~~~~~~~
> 
> Issues like there were supposed to be spotted by the static_asserts in
> gdb::checked_static_cast.  Clearly the existing asserts are not good
> enough.
> 
> I'd like to propose the patch below which would have prevented this
> issue sneaking in unseen.  Thoughts?
> 
> Thanks,
> Andrew
> 
> ---
> 
> commit 9d7cb56af0361a2276621203e0e508b6fda780f9
> Author: Andrew Burgess <aburgess@redhat.com>
> Date:   Wed Mar 6 17:28:48 2024 +0000
> 
>     gdb: use static_cast in gdb::checked_static_cast
>     
>     This commit:
>     
>       commit 6fe4779ac4b1874c995345e3eabd89cb1a05fbdf
>       Date:   Sat Feb 24 11:00:20 2024 +0100
>     
>           [gdb/build] Fix static cast of virtual base
>     
>     addressed an issue where GDB would not compile in production mode due
>     to a use of gdb::checked_static_cast.  The problem was that we were
>     asking GDB to cast from a virtual base class to a sub-class, this
>     works fine when using dynamic_cast, but does not work with
>     static_cast.
>     
>     The gdb::checked_static_cast actually uses dynamic_cast under the hood
>     in development mode in order to ensure that the cast is valid, while
>     in a production build we use static_cast as this is more efficient.
>     
>     What this meant however, was that when gdb::checked_static_cast was
>     used to cast from a virtual base class, the dynamic_cast of a
>     non-production build worked fine, while the production build's
>     static_cast caused a build failure.
>     
>     However, the gdb::checked_static_cast function already contains some
>     static_assert calls that are intended to catch any issues with invalid
>     type casting, the goal of these asserts was to prevent issues like
>     this: the build only failing in production mode.  Clearly the current
>     asserts are not enough.
>     
>     I don't think there is a std::is_virtual_base type trait check, so
>     what I propose instead is that in non-production mode we also make use
>     of static_cast.  This will ensure that any errors that crop up in
>     production mode should also be revealed in non-production mode, and
>     should catch issues like this in the future.
>     
>     There should be no user visible changes after this commit.
>     
>     Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31399
> 
> diff --git a/gdbsupport/gdb-checked-static-cast.h b/gdbsupport/gdb-checked-static-cast.h
> index 227010e46ea..33cab48cd9b 100644
> --- a/gdbsupport/gdb-checked-static-cast.h
> +++ b/gdbsupport/gdb-checked-static-cast.h
> @@ -57,8 +57,14 @@ checked_static_cast (V *v)
>    if (v == nullptr)
>      return nullptr;
>  
> -  T result = dynamic_cast<T> (v);
> -  gdb_assert (result != nullptr);
> +  gdb_assert (dynamic_cast<T> (v) != nullptr);
> +
> +  /* If a base class of V is virtual then the dynamic_cast will succeed,
> +     but the production mode static_cast will fail.  So having checked with
> +     the dynamic_cast that we didn't get nullptr, now use static_cast to
> +     catch the virtual base case.  This has the side effect of guaranteeing
> +     that GDB will compile in production mode.  */
> +  T result = static_cast<T> (v);
>  #else
>    T result = static_cast<T> (v);
>  #endif
> 

This is similar to what I proposed here:

https://inbox.sourceware.org/gdb-patches/24af4ea8-5426-4ce4-b1c5-12858b38a952@simark.ca/

The idea is the same, to have a static_cast in the DEVELOPMENT branch.
I kinda like my version better, as it factors out the static cast
(notice that both branches have identical static_cast lines after your
patch) and the  ifdef is just around a single assert.  Also, I'm pretty
sure the nullptr check is not necessary, as both dynamic_cast and
static_cast can handle it.

Simon

Re: [PATCH] [gdb/build] Fix static cast of virtual base

[Tom Tromey]

>>>>> "Simon" == Simon Marchi <simon.marchi@polymtl.ca> writes:

>> -  T result = dynamic_cast<T> (v);
>> -  gdb_assert (result != nullptr);
>> +  gdb_assert (dynamic_cast<T> (v) != nullptr);
>> +
>> +  /* If a base class of V is virtual then the dynamic_cast will succeed,
>> +     but the production mode static_cast will fail.  So having checked with
>> +     the dynamic_cast that we didn't get nullptr, now use static_cast to
>> +     catch the virtual base case.  This has the side effect of guaranteeing
>> +     that GDB will compile in production mode.  */
>> +  T result = static_cast<T> (v);
>> #else
>> T result = static_cast<T> (v);
>> #endif
>> 

Simon> This is similar to what I proposed here:

Simon> https://inbox.sourceware.org/gdb-patches/24af4ea8-5426-4ce4-b1c5-12858b38a952@simark.ca/

Simon> The idea is the same, to have a static_cast in the DEVELOPMENT branch.
Simon> I kinda like my version better, as it factors out the static cast
Simon> (notice that both branches have identical static_cast lines after your
Simon> patch) and the  ifdef is just around a single assert.  Also, I'm pretty
Simon> sure the nullptr check is not necessary, as both dynamic_cast and
Simon> static_cast can handle it.

I think either of these would be fine.

Tom

Re: [PATCH] [gdb/build] Fix static cast of virtual base

[Andrew Burgess]

Simon Marchi <simon.marchi@polymtl.ca> writes:

> On 3/6/24 12:39, Andrew Burgess wrote:
>> Tom de Vries <tdevries@suse.de> writes:
>> 
>>> With this change in bfd/development.sh:
>>> ...
>>> -development=true
>>> +development=false
>>> ...
>>> we run into:
>>> ...
>>> In file included from tui-data.h:28:0,
>>>                  from tui-command.c:24:
>>> gdb-checked-static-cast.h: In instantiation of \
>>>   ‘T gdb::checked_static_cast(V*) [with T = tui_cmd_window*; V = tui_win_info]’:
>>> tui-command.c:65:15:   required from here
>>> gdb-checked-static-cast.h:63:14: error: cannot convert from pointer to base \
>>>   class ‘tui_win_info’ to pointer to derived class ‘tui_cmd_window’ because \
>>>   the base is virtual
>>>    T result = static_cast<T> (v);
>>>               ^~~~~~~~~~~~~~~~~~
>> 
>> Issues like there were supposed to be spotted by the static_asserts in
>> gdb::checked_static_cast.  Clearly the existing asserts are not good
>> enough.
>> 
>> I'd like to propose the patch below which would have prevented this
>> issue sneaking in unseen.  Thoughts?
>> 
>> Thanks,
>> Andrew
>> 
>> ---
>> 
>> commit 9d7cb56af0361a2276621203e0e508b6fda780f9
>> Author: Andrew Burgess <aburgess@redhat.com>
>> Date:   Wed Mar 6 17:28:48 2024 +0000
>> 
>>     gdb: use static_cast in gdb::checked_static_cast
>>     
>>     This commit:
>>     
>>       commit 6fe4779ac4b1874c995345e3eabd89cb1a05fbdf
>>       Date:   Sat Feb 24 11:00:20 2024 +0100
>>     
>>           [gdb/build] Fix static cast of virtual base
>>     
>>     addressed an issue where GDB would not compile in production mode due
>>     to a use of gdb::checked_static_cast.  The problem was that we were
>>     asking GDB to cast from a virtual base class to a sub-class, this
>>     works fine when using dynamic_cast, but does not work with
>>     static_cast.
>>     
>>     The gdb::checked_static_cast actually uses dynamic_cast under the hood
>>     in development mode in order to ensure that the cast is valid, while
>>     in a production build we use static_cast as this is more efficient.
>>     
>>     What this meant however, was that when gdb::checked_static_cast was
>>     used to cast from a virtual base class, the dynamic_cast of a
>>     non-production build worked fine, while the production build's
>>     static_cast caused a build failure.
>>     
>>     However, the gdb::checked_static_cast function already contains some
>>     static_assert calls that are intended to catch any issues with invalid
>>     type casting, the goal of these asserts was to prevent issues like
>>     this: the build only failing in production mode.  Clearly the current
>>     asserts are not enough.
>>     
>>     I don't think there is a std::is_virtual_base type trait check, so
>>     what I propose instead is that in non-production mode we also make use
>>     of static_cast.  This will ensure that any errors that crop up in
>>     production mode should also be revealed in non-production mode, and
>>     should catch issues like this in the future.
>>     
>>     There should be no user visible changes after this commit.
>>     
>>     Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31399
>> 
>> diff --git a/gdbsupport/gdb-checked-static-cast.h b/gdbsupport/gdb-checked-static-cast.h
>> index 227010e46ea..33cab48cd9b 100644
>> --- a/gdbsupport/gdb-checked-static-cast.h
>> +++ b/gdbsupport/gdb-checked-static-cast.h
>> @@ -57,8 +57,14 @@ checked_static_cast (V *v)
>>    if (v == nullptr)
>>      return nullptr;
>>  
>> -  T result = dynamic_cast<T> (v);
>> -  gdb_assert (result != nullptr);
>> +  gdb_assert (dynamic_cast<T> (v) != nullptr);
>> +
>> +  /* If a base class of V is virtual then the dynamic_cast will succeed,
>> +     but the production mode static_cast will fail.  So having checked with
>> +     the dynamic_cast that we didn't get nullptr, now use static_cast to
>> +     catch the virtual base case.  This has the side effect of guaranteeing
>> +     that GDB will compile in production mode.  */
>> +  T result = static_cast<T> (v);
>>  #else
>>    T result = static_cast<T> (v);
>>  #endif
>> 
>
> This is similar to what I proposed here:
>
> https://inbox.sourceware.org/gdb-patches/24af4ea8-5426-4ce4-b1c5-12858b38a952@simark.ca/
>
> The idea is the same, to have a static_cast in the DEVELOPMENT branch.
> I kinda like my version better, as it factors out the static cast
> (notice that both branches have identical static_cast lines after your
> patch) and the  ifdef is just around a single assert.  Also, I'm pretty
> sure the nullptr check is not necessary, as both dynamic_cast and
> static_cast can handle it.

I agree yours is better.  Do you have an official patch somewhere that I
can add a +1 too?  If not then below is an updated version inline with
you proposal.  Feel free to push your version if you have it somewhere.

Thanks,
Andrew

--

commit 2a82aa7d140387fd728641846d9c4ec7d770b067
Author: Andrew Burgess <aburgess@redhat.com>
Date:   Wed Mar 6 17:28:48 2024 +0000

    gdb: use static_cast in gdb::checked_static_cast
    
    This commit:
    
      commit 6fe4779ac4b1874c995345e3eabd89cb1a05fbdf
      Date:   Sat Feb 24 11:00:20 2024 +0100
    
          [gdb/build] Fix static cast of virtual base
    
    addressed an issue where GDB would not compile in production mode due
    to a use of gdb::checked_static_cast.  The problem was that we were
    asking GDB to cast from a virtual base class to a sub-class, this
    works fine when using dynamic_cast, but does not work with
    static_cast.
    
    The gdb::checked_static_cast actually uses dynamic_cast under the hood
    in development mode in order to ensure that the cast is valid, while
    in a production build we use static_cast as this is more efficient.
    
    What this meant however, was that when gdb::checked_static_cast was
    used to cast from a virtual base class, the dynamic_cast of a
    non-production build worked fine, while the production build's
    static_cast caused a build failure.
    
    However, the gdb::checked_static_cast function already contains some
    static_assert calls that are intended to catch any issues with invalid
    type casting, the goal of these asserts was to prevent issues like
    this: the build only failing in production mode.  Clearly the current
    asserts are not enough.
    
    I don't think there is a std::is_virtual_base type trait check, so
    what I propose instead is that in non-production mode we also make use
    of static_cast.  This will ensure that any errors that crop up in
    production mode should also be revealed in non-production mode, and
    should catch issues like this in the future.
    
    There should be no user visible changes after this commit.
    
    Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31399
    
    Co-Authored-By: Simon Marchi <simon.marchi@polymtl.ca>

diff --git a/gdbsupport/gdb-checked-static-cast.h b/gdbsupport/gdb-checked-static-cast.h
index 227010e46ea..24fa7a4ba04 100644
--- a/gdbsupport/gdb-checked-static-cast.h
+++ b/gdbsupport/gdb-checked-static-cast.h
@@ -54,16 +54,12 @@ checked_static_cast (V *v)
 		 "types must be related");
 
 #ifdef DEVELOPMENT
-  if (v == nullptr)
-    return nullptr;
-
-  T result = dynamic_cast<T> (v);
-  gdb_assert (result != nullptr);
-#else
-  T result = static_cast<T> (v);
+  gdb_assert (v == nullptr || dynamic_cast<T> (v) != nullptr);
 #endif
 
-  return result;
+  /* If a base class of V is virtual then the dynamic_cast above will
+     succeed, but this static_cast will fail.  */
+  return static_cast<T> (v);
 }
 
 /* Same as the above, but to cast from a reference type to another.  */

Re: [PATCH] [gdb/build] Fix static cast of virtual base

[Simon Marchi]

On 3/11/24 06:25, Andrew Burgess wrote:
>> This is similar to what I proposed here:
>>
>> https://inbox.sourceware.org/gdb-patches/24af4ea8-5426-4ce4-b1c5-12858b38a952@simark.ca/
>>
>> The idea is the same, to have a static_cast in the DEVELOPMENT branch.
>> I kinda like my version better, as it factors out the static cast
>> (notice that both branches have identical static_cast lines after your
>> patch) and the  ifdef is just around a single assert.  Also, I'm pretty
>> sure the nullptr check is not necessary, as both dynamic_cast and
>> static_cast can handle it.
> 
> I agree yours is better.  Do you have an official patch somewhere that I
> can add a +1 too?  If not then below is an updated version inline with
> you proposal.  Feel free to push your version if you have it somewhere.

I didn't have a proper patch ready, what you sent LGTM.  Thanks a lot
for taking care of it.

Simon

Re: [PATCH] [gdb/build] Fix static cast of virtual base

[Andrew Burgess]

Simon Marchi <simon.marchi@polymtl.ca> writes:

> On 3/11/24 06:25, Andrew Burgess wrote:
>>> This is similar to what I proposed here:
>>>
>>> https://inbox.sourceware.org/gdb-patches/24af4ea8-5426-4ce4-b1c5-12858b38a952@simark.ca/
>>>
>>> The idea is the same, to have a static_cast in the DEVELOPMENT branch.
>>> I kinda like my version better, as it factors out the static cast
>>> (notice that both branches have identical static_cast lines after your
>>> patch) and the  ifdef is just around a single assert.  Also, I'm pretty
>>> sure the nullptr check is not necessary, as both dynamic_cast and
>>> static_cast can handle it.
>> 
>> I agree yours is better.  Do you have an official patch somewhere that I
>> can add a +1 too?  If not then below is an updated version inline with
>> you proposal.  Feel free to push your version if you have it somewhere.
>
> I didn't have a proper patch ready, what you sent LGTM.  Thanks a lot
> for taking care of it.

Thanks.  I pushed this patch.

I also pushed the patch below which I spotted in the same area.

Thanks,
Andrew

---

commit 8695c3a693849bf6ebd3c6d0620f03bcc6429604
Author: Andrew Burgess <aburgess@redhat.com>
Date:   Wed Mar 6 17:48:55 2024 +0000

    gdbsupport: rename include guard in gdb-checked-static-cast.h
    
    I noticed in passing that the include guard in the file
    gdbsupport/gdb-checked-static-cast.h was wrong, it includes the word
    DYNAMIC when STATIC would be better, fixed in this commit.
    
    There should be no user visible changes after this commit.

diff --git a/gdbsupport/gdb-checked-static-cast.h b/gdbsupport/gdb-checked-static-cast.h
index 24fa7a4ba04..97843fab225 100644
--- a/gdbsupport/gdb-checked-static-cast.h
+++ b/gdbsupport/gdb-checked-static-cast.h
@@ -15,8 +15,8 @@
    You should have received a copy of the GNU General Public License
    along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
 
-#ifndef COMMON_GDB_CHECKED_DYNAMIC_CAST_H
-#define COMMON_GDB_CHECKED_DYNAMIC_CAST_H
+#ifndef COMMON_GDB_CHECKED_STATIC_CAST_H
+#define COMMON_GDB_CHECKED_STATIC_CAST_H
 
 #include "gdbsupport/traits.h"
 
@@ -80,4 +80,4 @@ checked_static_cast (V &v)
 
 }
 
-#endif /* COMMON_GDB_CHECKED_DYNAMIC_CAST_H */
+#endif /* COMMON_GDB_CHECKED_STATIC_CAST_H */