[Bug tree-optimization/99074] New: gcc 8 and above is crashing with dynamic_cast<>() on null pointer with optimization level -O1 and above

[Bug tree-optimization/99074] New: gcc 8 and above is crashing with dynamic_cast<>() on null pointer with optimization level -O1 and above

[keith.halligan at microfocus dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99074

            Bug ID: 99074
           Summary: gcc 8 and above is crashing with dynamic_cast<>() on
                    null pointer with optimization level -O1 and above
           Product: gcc
           Version: 8.3.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: keith.halligan at microfocus dot com
  Target Milestone: ---

Created attachment 50169
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=50169&action=edit
comparison of the disassembly at -O0, O1, and -O2 levels

When compiling the small snippet below with g++/gcc v8.3.1 the dynamic_cast<>()
operator with a null pointer argument will result in a segmentation fault at
optimizations levels from -O1 and above.

I am running this on both RHEL 8 and SLES 15, in addition I installed gcc-9 and
gcc-10 toolchain packages, and the crashes were noticed on these newer
compilers as well.

I've also tested the code on RHEL 5 and 7 with gcc 4.1 and 4.8, and there's no
crash at any optimization levels

I've attached a file showing the difference in disassembly between -O0 and -O1,
it looks at lower levels the call to dynamic_cast is bypassed under certain
conditions, that doesn't happen at -O1 and above.

The code was compiled with: g++ -Wall -m32 -O2 -o test_dyn_cast
test_dyn_cast.cpp -v -save-temps

=====
// file: test_dyn_cast.cpp
//

#include <cstdio>

class Base {
  public:
    virtual ~Base() {}
    virtual void op() = 0;
};

class Object: public virtual Base {
};

class AbstractBase : public virtual Base {
  public:
    Object* _to_object() { return dynamic_cast<Object*>(this); }
};

class MyAbstractClass : public virtual AbstractBase {
  public:
    static MyAbstractClass* _nil() { return 0; }
};

int main(void) {
    MyAbstractClass *my_abs_type = MyAbstractClass::_nil();
    AbstractBase *abs_base = my_abs_type;
    Object *obj = abs_base->_to_object();

    printf("object is: %p\n", obj);

    return 0;
}

======

$ g++ -Wall -m32 -O2 -o test_dyn_cast test_dyn_cast.cpp -v -save-temps
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap
--enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man
--infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla
--enable-shared --enable-threads=posix --enable-checking=release
--enable-multilib --with-system-zlib --enable-__cxa_atexit
--disable-libunwind-exceptions --enable-gnu-unique-object
--enable-linker-build-id --with-gcc-major-version-only
--with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl
--disable-libmpx --enable-offload-targets=nvptx-none --without-cuda-driver
--enable-gnu-indirect-function --enable-cet --with-tune=generic
--with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)
COLLECT_GCC_OPTIONS='-Wall' '-m32' '-O2' '-o' 'test_dyn_cast' '-v'
'-save-temps' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
 /usr/libexec/gcc/x86_64-redhat-linux/8/cc1plus -E -quiet -v -imultilib 32
-D_GNU_SOURCE test_dyn_cast.cpp -m32 -mtune=generic -march=x86-64 -Wall -O2
-fpch-preprocess -o test_dyn_cast.ii
ignoring nonexistent directory
"/usr/lib/gcc/x86_64-redhat-linux/8/include-fixed"
ignoring nonexistent directory
"/usr/lib/gcc/x86_64-redhat-linux/8/../../../../x86_64-redhat-linux/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8

/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/x86_64-redhat-linux/32
 /usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/backward
 /usr/lib/gcc/x86_64-redhat-linux/8/include
 /usr/local/include
 /usr/include
End of search list.
COLLECT_GCC_OPTIONS='-Wall' '-m32' '-O2' '-o' 'test_dyn_cast' '-v'
'-save-temps' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
 /usr/libexec/gcc/x86_64-redhat-linux/8/cc1plus -fpreprocessed test_dyn_cast.ii
-quiet -dumpbase test_dyn_cast.cpp -m32 -mtune=generic -march=x86-64 -auxbase
test_dyn_cast -O2 -Wall -version -o test_dyn_cast.s
GNU C++14 (GCC) version 8.3.1 20191121 (Red Hat 8.3.1-5) (x86_64-redhat-linux)
        compiled by GNU C version 8.3.1 20191121 (Red Hat 8.3.1-5), GMP version
6.1.2, MPFR version 3.1.6-p2, MPC version 1.0.2, isl version isl-0.16.1-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
GNU C++14 (GCC) version 8.3.1 20191121 (Red Hat 8.3.1-5) (x86_64-redhat-linux)
        compiled by GNU C version 8.3.1 20191121 (Red Hat 8.3.1-5), GMP version
6.1.2, MPFR version 3.1.6-p2, MPC version 1.0.2, isl version isl-0.16.1-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: d41110238f51283cc9cdc51fe0924921
COLLECT_GCC_OPTIONS='-Wall' '-m32' '-O2' '-o' 'test_dyn_cast' '-v'
'-save-temps' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
 as -v --32 -o test_dyn_cast.o test_dyn_cast.s
GNU assembler version 2.30 (x86_64-redhat-linux) using BFD version version
2.30-73.el8
COMPILER_PATH=/usr/libexec/gcc/x86_64-redhat-linux/8/:/usr/libexec/gcc/x86_64-redhat-linux/8/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/8/:/usr/lib/gcc/x86_64-redhat-linux/
LIBRARY_PATH=/usr/lib/gcc/x86_64-redhat-linux/8/32/:/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib/:/lib/../lib/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-redhat-linux/8/:/usr/lib/gcc/x86_64-redhat-linux/8/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-Wall' '-m32' '-O2' '-o' 'test_dyn_cast' '-v'
'-save-temps' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
 /usr/libexec/gcc/x86_64-redhat-linux/8/collect2 -plugin
/usr/libexec/gcc/x86_64-redhat-linux/8/liblto_plugin.so
-plugin-opt=/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper
-plugin-opt=-fresolution=test_dyn_cast.res -plugin-opt=-pass-through=-lgcc_s
-plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lc
-plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc --build-id
--no-add-needed --eh-frame-hdr --hash-style=gnu -m elf_i386 -dynamic-linker
/lib/ld-linux.so.2 -o test_dyn_cast
/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib/crt1.o
/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib/crti.o
/usr/lib/gcc/x86_64-redhat-linux/8/32/crtbegin.o
-L/usr/lib/gcc/x86_64-redhat-linux/8/32
-L/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib -L/lib/../lib
-L/usr/lib/../lib -L/usr/lib/gcc/x86_64-redhat-linux/8
-L/usr/lib/gcc/x86_64-redhat-linux/8/../../.. test_dyn_cast.o -lstdc++ -lm
-lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/lib/gcc/x86_64-redhat-linux/8/32/crtend.o
/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib/crtn.o
COLLECT_GCC_OPTIONS='-Wall' '-m32' '-O2' '-o' 'test_dyn_cast' '-v'
'-save-temps' '-shared-libgcc' '-mtune=generic' '-march=x86-64'

=======

$ ./test_dyn_cast
Segmentation fault (core dumped)

=======

$ gdb ./test_dyn_cast
GNU gdb (GDB) Red Hat Enterprise Linux 8.2-11.el8
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./test_dyn_cast...(no debugging symbols found)...done.
(gdb) run
Starting program: /x1/users/keithh/test_dyn_cast

Program received signal SIGSEGV, Segmentation fault.
0xf7e9a023 in __cxxabiv1::__dynamic_cast (src_ptr=0x0, src_type=0x80486ec
<typeinfo for AbstractBase>, dst_type=0x80486c4 <typeinfo for Object>,
src2dst=-2) at ../../../../libstdc++-v3/libsupc++/dyncast.cc:50
50        const void *vtable = *static_cast <const void *const *> (src_ptr);
(gdb) bt
#0  0xf7e9a023 in __cxxabiv1::__dynamic_cast (src_ptr=0x0, src_type=0x80486ec
<typeinfo for AbstractBase>, dst_type=0x80486c4 <typeinfo for Object>,
src2dst=-2)
    at ../../../../libstdc++-v3/libsupc++/dyncast.cc:50
#1  0x080484b4 in main ()
(gdb) quit

[Bug tree-optimization/99074] gcc 8 and above is crashing with dynamic_cast<>() on null pointer with optimization level -O1 and above

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99074

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |INVALID

--- Comment #1 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Your testcase is invalid, contains undefined behavior as can be seen e.g. if
you built it with -fsanitize=undefined.
Calling a non-static member function on a nullptr object is UB.
And anything can happen afterwards.

[Bug tree-optimization/99074] gcc 8 and above is crashing with dynamic_cast<>() on null pointer with optimization level -O1 and above

[msebor at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99074

Martin Sebor <msebor at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |diagnostic
   Last reconfirmed|                            |2021-02-12
                 CC|                            |msebor at gcc dot gnu.org
         Resolution|INVALID                     |---
   Target Milestone|---                         |12.0
             Status|RESOLVED                    |ASSIGNED
     Ever confirmed|0                           |1
           Assignee|unassigned at gcc dot gnu.org      |msebor at gcc dot gnu.org

--- Comment #2 from Martin Sebor <msebor at gcc dot gnu.org> ---
True, it's undefined, and regrettably not caught by -Wnonnull.  The warning
runs both too early (in the FE) and too late (in CCP).  But can we do better
than simply crashing?  Clang can, as can ICC (no crash there).  It seems to me
the call to __dynamic_cast with the null pointer could be intercepted.  At
compile-time, GCC could warn, and at runtime, the call could return null.  The
following is a proof of concept that does that:

$ /build/gcc-master/gcc/xg++ -B /build/gcc-master/gcc -nostdinc++ -L
/build/gcc-master/x86_64-pc-linux-gnu/libstdc++-v3/src/.libs -O2 -Wall
pr99074.C &&
LD_LIBRARY_PATH=/build/gcc-master/x86_64-pc-linux-gnu/libstdc++-v3/src/.libs
./a.out
In member function ‘Object* AbstractBase::_to_object()’,
    inlined from ‘int main()’ at pr99074.C:24:39:
pr99074.C:12:35: warning: argument 1 null where non-null expected [-Wnonnull]
   12 |     Object* _to_object() { return dynamic_cast<Object*>(this); }
      |                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~
pr99074.C: In function ‘int main()’:
pr99074.C:12:61: note: in a call to function ‘void* __dynamic_cast(const void*,
const __cxxabiv1::__class_type_info*, const __cxxabiv1::__class_type_info*,
long int)’ declared ‘nonnull’
   12 |     Object* _to_object() { return dynamic_cast<Object*>(this); }
      |                                                             ^
object is: (nil)

Let me confirm this bug and propose the change below.

diff --git a/gcc/cp/rtti.c b/gcc/cp/rtti.c
index b41d95469c6..c54e431de31 100644
--- a/gcc/cp/rtti.c
+++ b/gcc/cp/rtti.c
@@ -30,6 +30,7 @@ along with GCC; see the file COPYING3.  If not see
 #include "stor-layout.h"
 #include "c-family/c-pragma.h"
 #include "gcc-rich-location.h"
+#include "attribs.h"

 /* C++ returns type information to the user in struct type_info
    objects. We also use type information to implement dynamic_cast and
@@ -767,6 +768,10 @@ build_dynamic_cast_1 (location_t loc, tree type, tree
expr,
              dcast_fn = (build_library_fn_ptr
                          (fn_name, fn_type, ECF_LEAF | ECF_PURE |
ECF_NOTHROW));
              pop_abi_namespace (flags);
+
+             tree attr = tree_cons (get_identifier ("nonnull"), 0, 0);
+             decl_attributes (&dcast_fn, attr, 0);
+
              dynamic_cast_node = dcast_fn;
            }
          result = build_cxx_call (dcast_fn, 4, elems, complain);
diff --git a/libstdc++-v3/libsupc++/dyncast.cc
b/libstdc++-v3/libsupc++/dyncast.cc
index b7d98495ad3..6e9d81b312d 100644
--- a/libstdc++-v3/libsupc++/dyncast.cc
+++ b/libstdc++-v3/libsupc++/dyncast.cc
@@ -47,6 +47,8 @@ __dynamic_cast (const void *src_ptr,    // object started
from
                 const __class_type_info *dst_type, // desired target type
                 ptrdiff_t src2dst) // how src and dst are related
   {
+  if (!src_ptr)
+    return NULL;
   const void *vtable = *static_cast <const void *const *> (src_ptr);
   const vtable_prefix *prefix =
     (adjust_pointer <vtable_prefix>

[Bug tree-optimization/99074] gcc 8 and above is crashing with dynamic_cast<>() on null pointer with optimization level -O1 and above

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99074

--- Comment #3 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Martin Sebor <msebor@gcc.gnu.org>:

https://gcc.gnu.org/g:6d134ca4b963706f31251f061fc180e517b32546

commit r11-7350-g6d134ca4b963706f31251f061fc180e517b32546
Author: Martin Sebor <msebor@redhat.com>
Date:   Tue Feb 23 14:09:00 2021 -0700

    PR c++/99074 - crash in dynamic_cast<>() on null pointer

    libstdc++-v3/ChangeLog:

            PR c++/99074
            * libsupc++/dyncast.cc (__dynamic_cast): Return null when
            first argument is null.

    gcc/testsuite/ChangeLog:

            PR c++/99074
            * g++.dg/warn/Wnonnull11.C: New test.

[Bug tree-optimization/99074] gcc 8 and above is crashing with dynamic_cast<>() on null pointer with optimization level -O1 and above

[msebor at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99074

Martin Sebor <msebor at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
      Known to fail|                            |10.2.0, 8.3.0, 9.3.0
           See Also|                            |https://gcc.gnu.org/bugzill
                   |                            |a/show_bug.cgi?id=99229
      Known to work|                            |11.0
         Resolution|---                         |FIXED

--- Comment #4 from Martin Sebor <msebor at gcc dot gnu.org> ---
r11-7350 changes __dynamic_cast to handle null pointers gracefully.  But the
part to diagnose the user code bug was not approved.  I raised pr99229 to track
it.

[Bug tree-optimization/99074] gcc 8 and above is crashing with dynamic_cast<>() on null pointer with optimization level -O1 and above

[keith.halligan at microfocus dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99074

--- Comment #5 from Keith Halligan <keith.halligan at microfocus dot com> ---
Thanks Martin, I appreciate your work on this.

 - Keith

[Bug tree-optimization/99074] gcc 8 and above is crashing with dynamic_cast<>() on null pointer with optimization level -O1 and above

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99074

--- Comment #6 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Martin Sebor from comment #2)
> --- a/libstdc++-v3/libsupc++/dyncast.cc
> +++ b/libstdc++-v3/libsupc++/dyncast.cc
> @@ -47,6 +47,8 @@ __dynamic_cast (const void *src_ptr,    // object started
> from
>                  const __class_type_info *dst_type, // desired target type
>                  ptrdiff_t src2dst) // how src and dst are related
>    {
> +  if (!src_ptr)
> +    return NULL;

Shouldn't this use __builtin_expect(!src_ptr, 0)? The FE is only supposed to
call this function when we have a non-null pointer, which is true for programs
with no UB, so this check potentially pessimizes correct programs.

[Bug tree-optimization/99074] gcc 8 and above is crashing with dynamic_cast<>() on null pointer with optimization level -O1 and above

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99074

--- Comment #7 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
It certainly shouldn't hurt.  I think the middle-end predicts ptr == NULL
comparisons as unlikely, but __builtin_expect is stronger probability than that
and makes it clearer that it is extremely unlikely.

[Bug tree-optimization/99074] gcc 8 and above is crashing with dynamic_cast<>() on null pointer with optimization level -O1 and above

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99074

--- Comment #8 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Jonathan Wakely from comment #6)
> Shouldn't this use __builtin_expect(!src_ptr, 0)? The FE is only supposed to
> call this function when we have a non-null pointer, which is true for
> programs with no UB, so this check potentially pessimizes correct programs.

It doesn't affect codegen, so GCC seems to guess correctly already. But it
might be worth adding it anyway, to be certain.