[Bug lto/106334] New: [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[Bug lto/106334] New: [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[slyfox at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106334

            Bug ID: 106334
           Summary: [13 Regression] lto -g ICE in
                    dwarf2out_register_external_die at dwarf2out.cc:6072
           Product: gcc
           Version: 13.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: lto
          Assignee: unassigned at gcc dot gnu.org
          Reporter: slyfox at gcc dot gnu.org
                CC: marxin at gcc dot gnu.org
  Target Milestone: ---

Noticed ICE on nix-2.10.3 project. It started using LTO recently.

I extracted something that looks like a reproducer, needs 3 files:

// $ cat eval.hh
void listElems();

namespace nix {
struct Value {

  auto listItems() {
    struct ListIterable {
      typedef int* iterator;
      iterator begin() __attribute__((noipa)) { return 0; }
      iterator end() __attribute__((noipa)) { return 0; }
    };
    listElems();
    return ListIterable{};
  }
};
}

// $ cat eval-cache.cc
#include "eval.hh"

void getListOfStrings(nix::Value & v) {
  for (auto elem : v.listItems())
    ;
}

// $ cat eval.cc
#include "eval.hh"

struct iterator {
  bool operator!=(iterator);
  void operator++();
  int operator*() { return 0; }
};
template <typename T> auto enumerate(T) {
  struct iterable_wrapper {
    auto begin() { return iterator{}; }
    auto end() { return iterator{}; }
  };
  return iterable_wrapper{};
}
void EvalStatecoerceToString(nix::Value & v) {
  for (auto v : enumerate(v.listItems()))
    ;
}

Triggering the crash:

$ g++ -Werror=return-type -O1 -flto -fPIC -g -I. -o eval-cache.o -c
eval-cache.cc
$ g++ -Werror=return-type -O1 -flto -fPIC -g -I. -o eval.o -c eval.cc
$ g++ -Werror=return-type -O1 -flto -fPIC -g -I. -o libnixexpr.so -shared
eval-cache.o eval.o
lto1: internal compiler error: in dwarf2out_register_external_die, at
dwarf2out.cc:6072
0xa059db dwarf2out_register_external_die
        ../../gcc-13-20220710/gcc/dwarf2out.cc:6072
0x903d0d lto_read_decls
        ../../gcc-13-20220710/gcc/lto/lto-common.cc:1952
0x904a7a lto_file_finalize
        ../../gcc-13-20220710/gcc/lto/lto-common.cc:2271
0x904a7a lto_create_files_from_ids
        ../../gcc-13-20220710/gcc/lto/lto-common.cc:2281
0x904a7a lto_file_read
        ../../gcc-13-20220710/gcc/lto/lto-common.cc:2336
0x904a7a read_cgraph_and_symbols(unsigned int, char const**)
        ../../gcc-13-20220710/gcc/lto/lto-common.cc:2784
0x8eca42 lto_main()
        ../../gcc-13-20220710/gcc/lto/lto.cc:626

Using '-g0' makes the file compile.

Compiler is this week's gcc-13 snapshot:

$ g++ -v
Using built-in specs.
COLLECT_GCC=/<<NIX>>/gcc-debug-13.0.0/bin/g++
COLLECT_LTO_WRAPPER=/<<NIX>>/gcc-debug-13.0.0/libexec/gcc/x86_64-unknown-linux-gnu/13.0.0/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with:
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 13.0.0 20220710 (experimental) (GCC)

[Bug lto/106334] [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106334

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |13.0
           Keywords|                            |ice-on-valid-code

[Bug lto/106334] [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106334

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
           Assignee|unassigned at gcc dot gnu.org      |rguenth at gcc dot gnu.org
             Status|UNCONFIRMED                 |ASSIGNED
   Last reconfirmed|                            |2022-07-18

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> ---
Confirmed.  Do we eventually ggc_free during stream-in?!

Breakpoint 5, dwarf2out_register_external_die (decl=<function_decl
0x7ffff66f0f00 listItems>, sym=0x42c6e98 "eval.cc.eec30469", off=478) at
/space/rguenther/src/gcc/gcc/dwarf2out.cc:6072
6072      gcc_checking_assert (!external_die_map->get (decl));
(gdb) p sym
$4 = 0x42c6e98 "eval.cc.eec30469"
(gdb) p off
$5 = 478
(gdb) c
Continuing.
Breakpoint 5, dwarf2out_register_external_die (decl=<function_decl
0x7ffff66f0f00 operator*>, sym=0x42c6e98 "eval.cc.eec30469", off=455) at
/space/rguenther/src/gcc/gcc/dwarf2out.cc:6072
6072      gcc_checking_assert (!external_die_map->get (decl));
(gdb) p sym
$6 = 0x42c6e98 "eval.cc.eec30469"
(gdb) p off
$7 = 455

so we have the "same" decl (address) but with different decls.  Hmm.

#0  free_node (node=<ggc_freed 0x7ffff66f0f00>)
    at /space/rguenther/src/gcc/gcc/tree.cc:1342
#1  0x0000000000b78e43 in unify_scc (data_in=0x42b3840, from=240, len=4, 
    scc_entry_len=1, scc_hash=852538354)
    at /space/rguenther/src/gcc/gcc/lto/lto-common.cc:1769
#2  0x0000000000b79590 in lto_read_decls (decl_data=0x7ffff68cdc60, 
    data=0x42c66b0, resolutions=...)
    at /space/rguenther/src/gcc/gcc/lto/lto-common.cc:1930

and

#0  dwarf2out_register_external_die (
    decl=<function_decl 0x7ffff66f0f00 listItems>, 
    sym=0x42c6e98 "eval.cc.eec30469", off=478)
    at /space/rguenther/src/gcc/gcc/dwarf2out.cc:6072
#1  0x00000000010ec851 in lto_input_tree (ib=0x7fffffffd770, data_in=0x42b3840)
    at /space/rguenther/src/gcc/gcc/lto-streamer-in.cc:1911
#2  0x00000000010ec141 in lto_read_tree_1 (ib=0x7fffffffd770, 
    data_in=0x42b3840, expr=<type_decl 0x7ffff66ef7b8 ListIterable>)
    at /space/rguenther/src/gcc/gcc/lto-streamer-in.cc:1706
#3  0x00000000010ec3ef in lto_input_scc (ib=0x7fffffffd770, data_in=0x42b3840, 
    len=0x7fffffffd72c, entry_len=0x7fffffffd728, shared_scc=true)
    at /space/rguenther/src/gcc/gcc/lto-streamer-in.cc:1798
#4  0x0000000000b7945f in lto_read_decls (decl_data=0x7ffff68cdc60, 
    data=0x42c66b0, resolutions=...)
    at /space/rguenther/src/gcc/gcc/lto/lto-common.cc:1904

note the dwarf2out registering during WPA is just using its hashtable as
temporary storage, overwriting an existing entry should be fine there.
An alternative would be to delay registering until we know the node is
needed or providing a hook to unregister entries when we do free_node.

The issue is probably latent everywhere - interesting that it didn't pop
up yet.

The "cheapest solution" is to short-cut the assert when flag_wpa (it's
off with release checking anyway).

[Bug lto/106334] [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106334

--- Comment #2 from Martin Liška <marxin at gcc dot gnu.org> ---
Btw. started with r12-6678-g254ada46ae0f21bd.

[Bug lto/106334] [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106334

--- Comment #3 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Richard Biener <rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:0f129766fdb687394f0eea04f69268b5cc034cda

commit r13-1742-g0f129766fdb687394f0eea04f69268b5cc034cda
Author: Richard Biener <rguenther@suse.de>
Date:   Tue Jul 19 10:02:40 2022 +0200

    lto/106334 - relax assert during WPA tree merging

    The dwarf2out map of tree to symbol + offset is populated too early
    when streaming in trees so that when WPA tree merging decides to
    recycle them the mapping prevails and if we are unlucky the same
    address is used for another tree with a symbol + offset DIE to
    record.  The following mitigates the resulting ICE by relaxing the
    assert, allowing re-use of a slot during WPA.  Delaying the register
    would be better but it's already somewhat hairy and uglifying this
    further doesn't look too important right now.

            PR lto/106334
            * dwarf2out.cc (dwarf2out_register_external_die): Allow
            map entry re-use during WPA.

[Bug lto/106334] [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106334

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |ice-checking
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #4 from Richard Biener <rguenth at gcc dot gnu.org> ---
Fixed on trunk - not sure if worth backporting to release branches.

[Bug lto/106334] [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[slyfox at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106334

Sergei Trofimovich <slyfox at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |NEW
         Resolution|FIXED                       |---

--- Comment #5 from Sergei Trofimovich <slyfox at gcc dot gnu.org> ---
I'll reopen this bug. I still see the crash in original example. I can make
#comment1 example crash by using `-flto-partition=none` in the link phase:

$ ./result/bin/g++ -O1 -flto -fPIC -g -I. -o eval-cache.o -c eval-cache.cc &&
./result/bin/g++ -O1 -flto -fPIC -g -I. -o eval.o -c eval.cc &&
./result/bin/g++ -O1 -flto -fPIC -g -I. -o libnixexpr.so -shared eval-cache.o
eval.o -flto-partition=none
lto1: internal compiler error: in dwarf2out_register_external_die, at
dwarf2out.cc:6076
0x1dc6714 diagnostic_impl(rich_location*, diagnostic_metadata const*, int, char
const*, __va_list_tag (*) [1], diagnostic_t)
        ???:0
0x1dc7366 internal_error(char const*, ...)
        ???:0
0x8b7012 fancy_abort(char const*, int, char const*)
        ???:0
0xa07036 dwarf2out_register_external_die(tree_node*, char const*, unsigned
long)
        ???:0
0x904efd lto_read_decls(lto_file_decl_data*, void const*,
vec<ld_plugin_symbol_resolution, va_heap, vl_ptr>)
        ???:0
0x905c6a read_cgraph_and_symbols(unsigned int, char const**)
        ???:0
0x8edc52 lto_main()
        ???:0

The compiler should already cpntain the fix:

$ ./result/bin/g++ -v | unnix
Using built-in specs.
COLLECT_GCC=/nix/store/w9kh67lghm1mq5gbcgg06d00d982cyj2-gcc-13.0.0/bin/g++
COLLECT_LTO_WRAPPER=/nix/store/w9kh67lghm1mq5gbcgg06d00d982cyj2-gcc-13.0.0/libexec/gcc/x86_64-unknown-linux-gnu/13.0.0/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with:
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 13.0.0 20220724 (experimental) (GCC)

[Bug lto/106334] [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106334

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED

--- Comment #6 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Sergei Trofimovich from comment #5)
> I'll reopen this bug. I still see the crash in original example. I can make
> #comment1 example crash by using `-flto-partition=none` in the link phase:

Meh.  OK, so change the guard to !(in_lto_p && !flag_wpa) || ...

Stupid -flto-partition=none ...

[Bug lto/106334] [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106334

--- Comment #7 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Richard Biener <rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:bc7526f6fca0e6ac3bd462ae54170fa464539148

commit r13-1925-gbc7526f6fca0e6ac3bd462ae54170fa464539148
Author: Richard Biener <rguenther@suse.de>
Date:   Mon Aug 1 10:06:49 2022 +0200

    lto/106334 - fix previous fix wrt -flto-partition=none

    This adjusts the assert guard to include -flto-partition=none which
    behaves as WPA.

            PR lto/106334
            * dwarf2out.cc (dwarf2out_register_external_die): Adjust
            assert.

[Bug lto/106334] [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106334

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #8 from Richard Biener <rguenth at gcc dot gnu.org> ---
Fixed again.

[Bug lto/106334] [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106334

--- Comment #9 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Richard Biener <rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:2a1448f2763a72c83e2ec496f78243a975b0d44e

commit r13-1987-g2a1448f2763a72c83e2ec496f78243a975b0d44e
Author: Richard Biener <rguenther@suse.de>
Date:   Mon Aug 8 09:07:23 2022 +0200

    lto/106540 - fix LTO tree input wrt dwarf2out_register_external_die

    I've revisited the earlier two workarounds for
dwarf2out_register_external_die
    getting duplicate entries.  It turns out that r11-525-g03d90a20a1afcb
    added dref_queue pruning to lto_input_tree but decl reading uses that
    to stream in DECL_INITIAL even when in the middle of SCC streaming.
    When that SCC then gets thrown away we can end up with debug nodes
    registered which isn't supposed to happen.  The following adjusts
    the DECL_INITIAL streaming to go the in-SCC way, using lto_input_tree_1,
    since no SCCs are expected at this point, just refs.

            PR lto/106540
            PR lto/106334
            * dwarf2out.cc (dwarf2out_register_external_die): Restore
            original assert.
            * lto-streamer-in.cc (lto_read_tree_1): Use lto_input_tree_1
            to input DECL_INITIAL, avoiding to commit drefs.

[Bug lto/106334] [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106334

--- Comment #10 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-12 branch has been updated by Richard Biener
<rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:4769ac6c5dfde2810a0266fe388211edc644e623

commit r12-8676-g4769ac6c5dfde2810a0266fe388211edc644e623
Author: Richard Biener <rguenther@suse.de>
Date:   Mon Aug 8 09:07:23 2022 +0200

    lto/106540 - fix LTO tree input wrt dwarf2out_register_external_die

    I've revisited the earlier two workarounds for
dwarf2out_register_external_die
    getting duplicate entries.  It turns out that r11-525-g03d90a20a1afcb
    added dref_queue pruning to lto_input_tree but decl reading uses that
    to stream in DECL_INITIAL even when in the middle of SCC streaming.
    When that SCC then gets thrown away we can end up with debug nodes
    registered which isn't supposed to happen.  The following adjusts
    the DECL_INITIAL streaming to go the in-SCC way, using lto_input_tree_1,
    since no SCCs are expected at this point, just refs.

            PR lto/106540
            PR lto/106334
            * lto-streamer-in.cc (lto_read_tree_1): Use lto_input_tree_1
            to input DECL_INITIAL, avoiding to commit drefs.

    (cherry picked from commit 2a1448f2763a72c83e2ec496f78243a975b0d44e)

[Bug lto/106334] [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106334

--- Comment #11 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Richard Biener
<rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:752d7664325037789ee4b634672e8e737ad9469c

commit r11-10307-g752d7664325037789ee4b634672e8e737ad9469c
Author: Richard Biener <rguenther@suse.de>
Date:   Mon Aug 8 09:07:23 2022 +0200

    lto/106540 - fix LTO tree input wrt dwarf2out_register_external_die

    I've revisited the earlier two workarounds for
dwarf2out_register_external_die
    getting duplicate entries.  It turns out that r11-525-g03d90a20a1afcb
    added dref_queue pruning to lto_input_tree but decl reading uses that
    to stream in DECL_INITIAL even when in the middle of SCC streaming.
    When that SCC then gets thrown away we can end up with debug nodes
    registered which isn't supposed to happen.  The following adjusts
    the DECL_INITIAL streaming to go the in-SCC way, using lto_input_tree_1,
    since no SCCs are expected at this point, just refs.

            PR lto/106540
            PR lto/106334
            * lto-streamer-in.c (lto_read_tree_1): Use lto_input_tree_1
            to input DECL_INITIAL, avoiding to commit drefs.

    (cherry picked from commit 4769ac6c5dfde2810a0266fe388211edc644e623)

[Bug lto/106334] [13 Regression] lto -g ICE in dwarf2out_register_external_die at dwarf2out.cc:6072

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106334

--- Comment #12 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-10 branch has been updated by Richard Biener
<rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:d2fa5521282cc7b5510e1e2c431a56d12693d74f

commit r10-11039-gd2fa5521282cc7b5510e1e2c431a56d12693d74f
Author: Richard Biener <rguenther@suse.de>
Date:   Mon Aug 8 09:07:23 2022 +0200

    lto/106540 - fix LTO tree input wrt dwarf2out_register_external_die

    I've revisited the earlier two workarounds for
dwarf2out_register_external_die
    getting duplicate entries.  It turns out that r11-525-g03d90a20a1afcb
    added dref_queue pruning to lto_input_tree but decl reading uses that
    to stream in DECL_INITIAL even when in the middle of SCC streaming.
    When that SCC then gets thrown away we can end up with debug nodes
    registered which isn't supposed to happen.  The following adjusts
    the DECL_INITIAL streaming to go the in-SCC way, using lto_input_tree_1,
    since no SCCs are expected at this point, just refs.

            PR lto/106540
            PR lto/106334
            * lto-streamer-in.c (lto_read_tree_1): Use lto_input_tree_1
            to input DECL_INITIAL, avoiding to commit drefs.

    (cherry picked from commit 4769ac6c5dfde2810a0266fe388211edc644e623)