[Bug breakpoints/29277] New: jit_event_handler: Assertion `jiter->jiter_data != nullptr' failed with separate debug files

[Bug breakpoints/29277] New: jit_event_handler: Assertion `jiter->jiter_data != nullptr' failed with separate debug files

[guillaume at morinfr dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29277

            Bug ID: 29277
           Summary: jit_event_handler: Assertion `jiter->jiter_data !=
                    nullptr' failed with separate debug files
           Product: gdb
           Version: 12.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: breakpoints
          Assignee: unassigned at sourceware dot org
          Reporter: guillaume at morinfr dot org
  Target Milestone: ---

Similar to https://sourceware.org/bugzilla/show_bug.cgi?id=27889, we are
hitting the same assertion handler with separate debug files.

This is just by running the program. No bp set but it does crash in the
breakpoint code:

----- Backtrace -----                                                           
0x55c94740570e gdb_internal_backtrace_1                                         
        /usr/scratch/tmp/gdb-12.1/gdb/bt-utils.c:122                            
0x55c9474057b6 _Z22gdb_internal_backtracev                                      
        /usr/scratch/tmp/gdb-12.1/gdb/bt-utils.c:168                            
0x55c9479240d6 internal_vproblem                                                
        /usr/scratch/tmp/gdb-12.1/gdb/utils.c:394                               
0x55c947924409 _Z15internal_verrorPKciS0_P13__va_list_tag                       
        /usr/scratch/tmp/gdb-12.1/gdb/utils.c:471                               
0x55c947a87bcc _Z14internal_errorPKciS0_z                                       
        /usr/scratch/tmp/gdb-12.1/gdbsupport/errors.cc:55                       
0x55c94766927a _Z17jit_event_handlerP7gdbarchP7objfile                          
        /usr/scratch/tmp/gdb-12.1/gdb/jit.c:1252                                
0x55c9473d2a40 handle_jit_event                                                 
        /usr/scratch/tmp/gdb-12.1/gdb/breakpoint.c:5552                         
0x55c9473d2dbd _Z20bpstat_run_callbacksP6bpstat                                 
        /usr/scratch/tmp/gdb-12.1/gdb/breakpoint.c:5753                         
0x55c947650b18 process_event_stop_test                                          
        /usr/scratch/tmp/gdb-12.1/gdb/infrun.c:6452                             
0x55c9476509ed handle_signal_stop                                               
        /usr/scratch/tmp/gdb-12.1/gdb/infrun.c:6420
0x55c94764ed12 handle_inferior_event
        /usr/scratch/tmp/gdb-12.1/gdb/infrun.c:5681
0x55c94764ac59 _Z20fetch_inferior_eventv
        /usr/scratch/tmp/gdb-12.1/gdb/infrun.c:4085
0x55c94762b0bb _Z22inferior_event_handler19inferior_event_type
        /usr/scratch/tmp/gdb-12.1/gdb/inf-loop.c:41
0x55c947690fbf handle_target_event
        /usr/scratch/tmp/gdb-12.1/gdb/linux-nat.c:4154
0x55c947a889e1 handle_file_event
        /usr/scratch/tmp/gdb-12.1/gdbsupport/event-loop.cc:574
0x55c947a88f80 gdb_wait_for_event
        /usr/scratch/tmp/gdb-12.1/gdbsupport/event-loop.cc:700
0x55c947a87d00 _Z16gdb_do_one_eventv
        /usr/scratch/tmp/gdb-12.1/gdbsupport/event-loop.cc:212
0x55c9476be153 start_event_loop
        /usr/scratch/tmp/gdb-12.1/gdb/main.c:421
0x55c9476be287 captured_command_loop
        /usr/scratch/tmp/gdb-12.1/gdb/main.c:481
0x55c9476bfd5f captured_main
        /usr/scratch/tmp/gdb-12.1/gdb/main.c:1351
0x55c9476bfdc5 _Z8gdb_mainP18captured_main_args
        /usr/scratch/tmp/gdb-12.1/gdb/main.c:1366
0x55c9473071fe main
        /usr/scratch/tmp/gdb-12.1/gdb/gdb.c:32

The objfile is:
p  *jit_bp_sym.objfile
$2 = {original_name = 0x55c94a129950
"build-id/cb/38660bc052d7379d50faef77e2dec1.debug", addr_low = 0, flags =
{m_enum_value = (OBJF_REORDERED | OBJF_PSYMTABS_READ | OBJF_MAINLINE)},
  pspace = 0x55c949e5a8b0, compunit_symtabs = 0x55c94aea7cc0, obfd =
0x55c94a076e50, per_bfd = 0x55c94a0e7760, mtime = 1655843985, objfile_obstack =
{chunk_size = 4064, chunk = 0x55c94f504a10,
    object_base = 0x55c94f508f90 "", next_free = 0x55c94f508f90 "", chunk_limit
= 0x55c94f508fdb "", temp = {i = 0, p = 0x0}, alignment_mask = 15, chunkfun =
{plain = 0x55c9473716d4 <xmalloc(size_t)>,
    extra = 0x55c9473716d4 <xmalloc(size_t)>}, freefun = {plain =
0x55c94772b7d2 <xfree<void>(void*)>, extra = 0x55c94772b7d2
<xfree<void>(void*)>}, extra_arg = 0x0, use_extra_arg = 0, maybe_empty_object =
0,
    alloc_failed = 0}, sf = 0x55c947f5f5e0 <elf_sym_fns>, qf =
std::forward_list = {[0] = std::unique_ptr<quick_symbol_functions> = {get() =
0x55c94a109de0}}, registry_data = {data = 0x55c94a12ac00,
    num_data = 21}, section_offsets = std::vector of length 46, capacity 46 =
{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
    0, 0}, sect_index_text = 12, sect_index_data = 28, sect_index_bss = 31,
sect_index_rodata = 15, sections = 0x55c94a1299a0, sections_end =
0x55c94a129df0, separate_debug_objfile = 0x0,
  separate_debug_objfile_backlink = 0x55c94a024d80, separate_debug_objfile_link
= 0x0, stats = {n_syms = 34783, n_stabs = 0, n_types = 31535, sz_strtab = 0},
template_symbols = 0x55c94f402c90,
  static_links = std::unique_ptr<htab> = {get() = 0x0}, jiter_data =
std::unique_ptr<jiter_objfile_data> = {get() = 0x0}, jited_data =
std::unique_ptr<jited_objfile_data> = {get() = 0x0},
  skip_jit_symbol_lookup = false}

A member of our team suggested here that the code should be calling
jit_event_handler should be called with the
jit_bp_sym.objfile->separate_debug_objfile_backlink if it is set.  With this
change, gdb does not crash anymore but we're unfamiliar with the gdb code so
this might be incorrect.

Thanks in advance.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug breakpoints/29277] jit_event_handler: Assertion `jiter->jiter_data != nullptr' failed with separate debug files

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29277

Tom de Vries <vries at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |vries at gcc dot gnu.org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug breakpoints/29277] jit_event_handler: Assertion `jiter->jiter_data != nullptr' failed with separate debug files

[hector.oron at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29277

HectorOron <hector.oron at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hector.oron at gmail dot com

--- Comment #1 from HectorOron <hector.oron at gmail dot com> ---
Proposed patch:

--- gdb-12.1.orig/gdb/breakpoint.c
+++ gdb-12.1/gdb/breakpoint.c
@@ -5549,7 +5549,10 @@ handle_jit_event (CORE_ADDR address)
      function needs to be updated too.  */
   bound_minimal_symbol jit_bp_sym = lookup_minimal_symbol_by_pc (address);
   gdb_assert (jit_bp_sym.objfile != nullptr);
-  jit_event_handler (gdbarch, jit_bp_sym.objfile);
+  if (jit_bp_sym.objfile->separate_debug_objfile_backlink)
+    jit_event_handler (gdbarch,
jit_bp_sym.objfile->separate_debug_objfile_backlink);
+  else
+    jit_event_handler (gdbarch, jit_bp_sym.objfile);

   target_terminal::inferior ();
 }

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug breakpoints/29277] jit_event_handler: Assertion `jiter->jiter_data != nullptr' failed with separate debug files

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29277

--- Comment #2 from Tom de Vries <vries at gcc dot gnu.org> ---
It would be nice if you could post a reproducer.

AFAIU, a reproducer would trigger this assert:
...
diff --git a/gdb/breakpoint.c b/gdb/breakpoint.c
index 74f53368464..fb4d58fa66d 100644
--- a/gdb/breakpoint.c
+++ b/gdb/breakpoint.c
@@ -5668,6 +5668,7 @@ handle_jit_event (CORE_ADDR address)
      function needs to be updated too.  */
   bound_minimal_symbol jit_bp_sym = lookup_minimal_symbol_by_pc (address);
   gdb_assert (jit_bp_sym.objfile != nullptr);
+  gdb_assert (jit_bp_sym.objfile->separate_debug_objfile_backlink == nullptr);
   jit_event_handler (gdbarch, jit_bp_sym.objfile);

   target_terminal::inferior ();
...
but I haven't been able to, when runnning the gdb.base/jit-*.exp test-cases
using target board cc-with-gnu-debuglink, which was my best bet.

Alternatively, can you explain why we end up here with
jit_bp_sym.objfile->separate_debug_objfile_backlink != nullptr?  AFAIU, the fix
from PR27889 should prevent that from occurring.

Is there something peculiar about the executable / separate debug file
combination?  Can you post a readelf -S for both (or even better, if possible,
attach or post them in some dropbox format)?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug breakpoints/29277] jit_event_handler: Assertion `jiter->jiter_data != nullptr' failed with separate debug files

[schwab@linux-m68k.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29277

--- Comment #3 from Andreas Schwab <schwab@linux-m68k.org> ---
I'm seeing this on riscv64:

$ gdb/gdb --args qemu-riscv64 /usr/bin/cat
GNU gdb (GDB) 13.0.50.20220801-git
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "riscv64-suse-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from qemu-riscv64...
Reading symbols from
/usr/lib/debug/usr/bin/qemu-riscv64-7.0.0-82.1.riscv64.debug...
(gdb) r
Starting program: /usr/bin/qemu-riscv64 /usr/bin/cat
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/lp64d/libthread_db.so.1".
[New Thread 0x3ff7ffb3c0 (LWP 1363)]
../../gdb/gdb/jit.c:1247: internal-error: jit_event_handler: Assertion
`jiter->jiter_data != nullptr' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
----- Backtrace -----
0x8c5cd gdb_internal_backtrace_1
        ../../gdb/gdb/bt-utils.c:122
0x8c5cd _Z22gdb_internal_backtracev
        ../../gdb/gdb/bt-utils.c:168
0x330621 internal_vproblem
        ../../gdb/gdb/utils.c:396
0x3307f5 _Z15internal_verrorPKciS0_Pv
        ../../gdb/gdb/utils.c:476
0x3b50cd _Z14internal_errorPKciS0_z
        ../../gdb/gdbsupport/errors.cc:55
0x1a9cfd _Z17jit_event_handlerP7gdbarchP7objfile
        ../../gdb/gdb/jit.c:1247
0x7b1b5 handle_jit_event
        ../../gdb/gdb/breakpoint.c:5672
0x7b1b5 _Z20bpstat_run_callbacksP6bpstat
        ../../gdb/gdb/breakpoint.c:5875
0x19ef05 process_event_stop_test
        ../../gdb/gdb/infrun.c:6651
0x1a22eb handle_stop_requested
        ../../gdb/gdb/infrun.c:4627
0x1a22eb handle_stop_requested
        ../../gdb/gdb/infrun.c:4622
0x1a22eb handle_inferior_event
        ../../gdb/gdb/infrun.c:5883
0x1a3c0d _Z20fetch_inferior_eventv
        ../../gdb/gdb/infrun.c:4233
0x3b53c9 gdb_wait_for_event
        ../../gdb/gdbsupport/event-loop.cc:692
0x3b585d _Z16gdb_do_one_eventv
        ../../gdb/gdbsupport/event-loop.cc:210
0x1d5a29 start_event_loop
        ../../gdb/gdb/main.c:411
0x1d5a29 captured_command_loop
        ../../gdb/gdb/main.c:471
0x1d70e9 captured_main
        ../../gdb/gdb/main.c:1329
0x1d70e9 _Z8gdb_mainP18captured_main_args
        ../../gdb/gdb/main.c:1344
0x21137 main
        ../../gdb/gdb/gdb.c:32
---------------------
../../gdb/gdb/jit.c:1247: internal-error: jit_event_handler: Assertion
`jiter->jiter_data != nullptr' failed.


(top-gdb) p jit_bp_sym.objfile.jiter_data
$2 = {
  _M_t = {<std::__uniq_ptr_impl<jiter_objfile_data,
std::default_delete<jiter_objfile_data> >> = {
      _M_t = {<std::_Tuple_impl<0, jiter_objfile_data*,
std::default_delete<jiter_objfile_data> >> = {<std::_Tuple_impl<1,
std::default_delete<jiter_objfile_data> >> = {<std::_Head_base<1,
std::default_delete<jiter_objfile_data>, true>> = {_M_head_impl = {<No data
fields>}}, <No data fields>}, <std::_Head_base<0, jiter_objfile_data*, false>>
= {
            _M_head_impl = 0x0}, <No data fields>}, <No data fields>}}, <No
data fields>}}
(top-gdb) p jit_bp_sym.objfile.separate_debug_objfile_backlink.jiter_data
$5 = {
  _M_t = {<std::__uniq_ptr_impl<jiter_objfile_data,
std::default_delete<jiter_objfile_data> >> = {
      _M_t = {<std::_Tuple_impl<0, jiter_objfile_data*,
std::default_delete<jiter_objfile_data> >> = {<std::_Tuple_impl<1,
std::default_delete<jiter_objfile_data> >> = {<std::_Head_base<1,
std::default_delete<jiter_objfile_data>, true>> = {_M_head_impl = {<No data
fields>}}, <No data fields>}, <std::_Head_base<0, jiter_objfile_data*, false>>
= {
            _M_head_impl = 0xa9f8e0}, <No data fields>}, <No data fields>}},
<No data fields>}}

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug breakpoints/29277] jit_event_handler: Assertion `jiter->jiter_data != nullptr' failed with separate debug files

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29277

--- Comment #4 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Andreas Schwab from comment #3)
> I'm seeing this on riscv64:
> 
> $ gdb/gdb --args qemu-riscv64 /usr/bin/cat
> GNU gdb (GDB) 13.0.50.20220801-git
> Copyright (C) 2022 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> Type "show copying" and "show warranty" for details.
> This GDB was configured as "riscv64-suse-linux".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> <https://www.gnu.org/software/gdb/bugs/>.
> Find the GDB manual and other documentation resources online at:
>     <http://www.gnu.org/software/gdb/documentation/>.
> 
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> Reading symbols from qemu-riscv64...
> Reading symbols from
> /usr/lib/debug/usr/bin/qemu-riscv64-7.0.0-82.1.riscv64.debug...
> (gdb) r
> Starting program: /usr/bin/qemu-riscv64 /usr/bin/cat
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib64/lp64d/libthread_db.so.1".
> [New Thread 0x3ff7ffb3c0 (LWP 1363)]
> ../../gdb/gdb/jit.c:1247: internal-error: jit_event_handler: Assertion
> `jiter->jiter_data != nullptr' failed.
> A problem internal to GDB has been detected,
> further debugging may prove unreliable.

Reproduced, by downloading
https://download.opensuse.org/ports/riscv/tumbleweed/images/openSUSE-Tumbleweed-RISC-V-JeOS-devel.riscv64-rootfs.riscv64.tar.xz
and unpacking it in ~/riscv64, and then doing:
...
$ gdb -q -batch -ex "set environment QEMU_LD_PREFIX ~/riscv64" -ex run --args
qemu-riscv64 ./usr/bin/cat
...

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug breakpoints/29277] jit_event_handler: Assertion `jiter->jiter_data != nullptr' failed with separate debug files

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29277

--- Comment #5 from Tom de Vries <vries at gcc dot gnu.org> ---
Fixed by:
...
diff --git a/gdb/breakpoint.c b/gdb/breakpoint.c
index dae96d205be..4ea22f4bd9c 100644
--- a/gdb/breakpoint.c
+++ b/gdb/breakpoint.c
@@ -5669,7 +5669,10 @@ handle_jit_event (CORE_ADDR address)
      function needs to be updated too.  */
   bound_minimal_symbol jit_bp_sym = lookup_minimal_symbol_by_pc (address);
   gdb_assert (jit_bp_sym.objfile != nullptr);
-  jit_event_handler (gdbarch, jit_bp_sym.objfile);
+  objfile *objfile = jit_bp_sym.objfile;
+  if (objfile->separate_debug_objfile_backlink)
+    objfile = objfile->separate_debug_objfile_backlink;
+  jit_event_handler (gdbarch, objfile);

   target_terminal::inferior ();
 }
...
which is essentially a refactoring of the patch from comment 1.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug breakpoints/29277] jit_event_handler: Assertion `jiter->jiter_data != nullptr' failed with separate debug files

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29277

--- Comment #6 from Tom de Vries <vries at gcc dot gnu.org> ---
Reproduced with target board cc-with-gnu-debuglink.exp and test-case
gdb.base/jit-reader.exp, and this trigger patch:
...
diff --git a/gdb/contrib/cc-with-tweaks.sh b/gdb/contrib/cc-with-tweaks.sh
index 7919d2da85a..ea064e6a564 100755
--- a/gdb/contrib/cc-with-tweaks.sh
+++ b/gdb/contrib/cc-with-tweaks.sh
@@ -267,7 +267,7 @@ if [ "$want_gnu_debuglink" = true ]; then
     debug_file="$tmpdir"/$(basename "$output_file").debug

     # Create stripped and debug versions of output_file.
-    strip --strip-debug "${output_file}" \
+    strip --strip-all "${output_file}" \
          -o "${stripped_file}"
     rc=$?
     [ $rc != 0 ] && exit $rc
...

Note btw that with the fix we no longer run into the assert, but there are
still plenty of fails in test-cases gdb.base/jit-*.exp.  Maybe there are
hardcoded assumptions in the test-cases about minimal symbols being present?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug breakpoints/29277] jit_event_handler: Assertion `jiter->jiter_data != nullptr' failed with separate debug files

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29277

Tom de Vries <vries at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |schwab@linux-m68k.org

--- Comment #7 from Tom de Vries <vries at gcc dot gnu.org> ---
*** Bug 29680 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug breakpoints/29277] jit_event_handler: Assertion `jiter->jiter_data != nullptr' failed with separate debug files

[ssbssa at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29277

Hannes Domani <ssbssa at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ssbssa at sourceware dot org

--- Comment #8 from Hannes Domani <ssbssa at sourceware dot org> ---
There was a commit mentioning this PR committed some time ago, so can this be
closed?:
https://sourceware.org/pipermail/gdb-patches/2022-October/192843.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug breakpoints/29277] jit_event_handler: Assertion `jiter->jiter_data != nullptr' failed with separate debug files

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29277

Tom de Vries <vries at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
   Target Milestone|---                         |13.1
         Resolution|---                         |FIXED

--- Comment #9 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Hannes Domani from comment #8)
> There was a commit mentioning this PR committed some time ago, so can this
> be closed?:
> https://sourceware.org/pipermail/gdb-patches/2022-October/192843.html

Indeed fixed by commit 508ccf9b3e1 ("[gdb] Fix assert in handle_jit_event"),
thanks for the reminder.

-- 
You are receiving this mail because:
You are on the CC list for the bug.