[Bug c++/106943] New: GCC building clang/llvm with LTO flags causes ICE in clang

[Bug c++/106943] New: GCC building clang/llvm with LTO flags causes ICE in clang

[immoloism at googlemail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

            Bug ID: 106943
           Summary: GCC building clang/llvm with LTO flags causes ICE in
                    clang
           Product: gcc
           Version: 12.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: immoloism at googlemail dot com
  Target Milestone: ---

Created attachment 53574
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=53574&action=edit
ICE error

I've found an interesting ICE which only occurs when LTO flags are turned on in
GCC while building clang and llvm while compiling compiler-rt-sanitizers. If I
rebuild clang/llvm without the flto cflags then the bug does not appear.

CFLAGS used to build clang/llvm "-march=native -O2 -pipe -fgraphite-identity
-floop-nest-optimize -flto=8 -fuse-linker-plugin -falign-functions=32"

This is the first time I've run into anything like this so please ask if any
other files or tests will help you identify the cause and I'd be happy to
oblige.

I have reported this to the LLVM team and they believe this could be an issue
your end so hopefully you have some ideas to add.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[immoloism at googlemail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #1 from Immolo <immoloism at googlemail dot com> ---
https://github.com/llvm/llvm-project/issues/57740

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Does it happen it you don't build with  -fgraphite-identity
-floop-nest-optimize ?

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[immoloism at googlemail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #3 from Immolo <immoloism at googlemail dot com> ---
I just rebuilt to test and yes it does still fail with the graphite flags
disabled with the same warnings.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2022-09-15
     Ever confirmed|0                           |1
           Keywords|                            |needs-reduction, wrong-code
             Status|UNCONFIRMED                 |WAITING

--- Comment #4 from Richard Biener <rguenth at gcc dot gnu.org> ---
OK, so this is clang/llvm ICEing when clang/llvm is built with GCC and LTO.

Obviously needs a more reduced testcase.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[immoloism at googlemail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #5 from Immolo <immoloism at googlemail dot com> ---
How would I go about creating a reduce for this as I'd assume it's to with
running something llvm-reduce with
`/var/tmp/portage/sys-libs/compiler-rt-sanitizers-15.0.0/work/compiler-rt/lib/sanitizer_common/..
 -DNDEBUG -march=native -O2 -pipe -falign-functions=32 -m32 -Wthread-safety
-Wthread-safety-reference -Wthread-safety-beta -O3 -MD -MT
lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.i386.dir/sanitizer_deadlock_detector2.cpp.o
-MF
lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.i386.dir/sanitizer_deadlock_detector2.cpp.o.d
-o
lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.i386.dir/sanitizer_deadlock_detector2.cpp.o
-c
/var/tmp/portage/sys-libs/compiler-rt-sanitizers-15.0.0/work/compiler-rt/lib/sanitizer_common/sanitizer_deadlock_detector2.cpp`
however I can't seem to find a guide to make this understandable.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[amonakov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

Alexander Monakov <amonakov at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |amonakov at gcc dot gnu.org

--- Comment #6 from Alexander Monakov <amonakov at gcc dot gnu.org> ---
I'm giving this a shot, and with -flto-partition=1to1 there's a ranger ICE
(below).

What's the current best practice for LTO debugging? I don't imagine there's an
easy way to identify which of 2000 lto1 invocation crashes, or attach gdb to
it? Or at least generate a corefile?

0x10308fd internal_error(char const*, ...)
        ???:0
0x13dcf8c operator_bitwise_not::fold_range(irange&, tree_node*, irange const&,
irange const&, tree_code) const
        ???:0
0x136882a fold_using_range::range_of_range_op(irange&, gimple*, fur_source&)
        ???:0
0x1323eb5 fold_using_range::fold_stmt(irange&, gimple*, fur_source&,
tree_node*)
        ???:0
0x134a417 path_range_query::range_of_stmt(irange&, gimple*, tree_node*)
        ???:0
0x13499b4 path_range_query::range_defined_in_block(irange&, tree_node*,
basic_block_def*)
        ???:0
0x135503a path_range_query::internal_range_of_expr(irange&, tree_node*,
gimple*)
        ???:0
0x1368424 fold_using_range::range_of_range_op(irange&, gimple*, fur_source&)
        ???:0
0x1323eb5 fold_using_range::fold_stmt(irange&, gimple*, fur_source&,
tree_node*)
        ???:0
0x134a417 path_range_query::range_of_stmt(irange&, gimple*, tree_node*)
        ???:0
0x13499b4 path_range_query::range_defined_in_block(irange&, tree_node*,
basic_block_def*)
        ???:0
0x135503a path_range_query::internal_range_of_expr(irange&, tree_node*,
gimple*)
        ???:0
0x1368424 fold_using_range::range_of_range_op(irange&, gimple*, fur_source&)
        ???:0
0x1323eb5 fold_using_range::fold_stmt(irange&, gimple*, fur_source&,
tree_node*)
        ???:0
0x134a417 path_range_query::range_of_stmt(irange&, gimple*, tree_node*)
        ???:0
0x13499b4 path_range_query::range_defined_in_block(irange&, tree_node*,
basic_block_def*)
        ???:0
0x135503a path_range_query::internal_range_of_expr(irange&, tree_node*,
gimple*)
        ???:0
0x1368424 fold_using_range::range_of_range_op(irange&, gimple*, fur_source&)
        ???:0
0x1323eb5 fold_using_range::fold_stmt(irange&, gimple*, fur_source&,
tree_node*)
        ???:0
0x134a417 path_range_query::range_of_stmt(irange&, gimple*, tree_node*)
        ???:0

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[amonakov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #7 from Alexander Monakov <amonakov at gcc dot gnu.org> ---
This problem seems to go way back. I'm told even gcc-9 broke LLVM like that.

For my investigation, I took latest gcc-11 snapshot and llvm-13.0.1.

My conclusion that it is a lifetime-dse violation in LLVM (they already pass
-fno-strict-aliasing, but apparently lifetime-dse issue is latent without LTO).

As evidence, I submit the following:

It breaks with -flto -flto-partition=1to1 with the following backtrace:

#4  0x00007ffff4557216 in __assert_fail () from /lib64/libc.so.6
#5  0x00007ffff6d233a9 in llvm::User::allocHungoffUses () at
/tmp/llvm-project-llvmorg-13.0.1/llvm/lib/IR/User.cpp:45
#6  0x00007ffff676945f in llvm::PHINode::allocHungoffUses () at
/tmp/llvm-project-llvmorg-13.0.1/llvm/include/llvm/IR/Instructions.h:2633
#7  llvm::PHINode::PHINode () at
/tmp/llvm-project-llvmorg-13.0.1/llvm/include/llvm/IR/Instructions.h:2611
#8  llvm::PHINode::Create () at
/tmp/llvm-project-llvmorg-13.0.1/llvm/include/llvm/IR/Instructions.h:2642
#9  0x00007ffff5c633f4 in llvm::InstCombinerImpl::foldPHIArgLoadIntoPHI () at
/tmp/llvm-project-llvmorg-13.0.1/llvm/lib/Transforms/InstCombine/InstCombinePHI.cpp:691
#10 llvm::InstCombinerImpl::foldPHIArgOpIntoPHI () at
/tmp/llvm-project-llvmorg-13.0.1/llvm/lib/Transforms/InstCombine/InstCombinePHI.cpp:835
#11 0x00007ffff5c64656 in llvm::InstCombinerImpl::visitPHINode () at
/tmp/llvm-project-llvmorg-13.0.1/llvm/lib/Transforms/InstCombine/InstCombinePHI.cpp:1316
#12 0x00007ffff5c0c1b7 in llvm::InstCombinerImpl::run () at
/tmp/llvm-project-llvmorg-13.0.1/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp:3902
#13 combineInstructionsOverFunction () at
/tmp/llvm-project-llvmorg-13.0.1/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp:4192
#14 0x00007ffff522efa4 in llvm::InstCombinePass::run () at
/tmp/llvm-project-llvmorg-13.0.1/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp:4223
#15 0x00007ffff66463de in llvm::detail::PassModel<llvm::Function,
llvm::InstCombinePass, llvm::PreservedAnalyses,
llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&,
llvm::AnalysisManager<llvm::Function>&) ()
    at
/tmp/llvm-project-llvmorg-13.0.1/llvm/include/llvm/IR/PassManagerInternal.h:85
#16 0x00007ffff51250da in llvm::PassManager<llvm::Function,
llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&,
llvm::AnalysisManager<llvm::Function>&) () at
/tmp/llvm-project-llvmorg-13.0.1/llvm/include/llvm/IR/PassManager.h:509
#17 0x00007ffff65ffb6e in llvm::detail::PassModel<llvm::Function,
llvm::PassManager<llvm::Function, llvm::AnalysisManager<llvm::Function>>,
llvm::PreservedAnalyses,
llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&,
llvm::AnalysisManager<llvm::Function>&) () at
/tmp/llvm-project-llvmorg-13.0.1/llvm/include/llvm/IR/PassManagerInternal.h:85
#18 0x00007ffff5158512 in llvm::ModuleToFunctionPassAdaptor::run () at
/tmp/llvm-project-llvmorg-13.0.1/llvm/lib/IR/PassManager.cpp:117
#19 0x00007ffff664513e in llvm::detail::PassModel<llvm::Module,
llvm::ModuleToFunctionPassAdaptor, llvm::PreservedAnalyses,
llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&,
llvm::AnalysisManager<llvm::Module>&) ()
    at
/tmp/llvm-project-llvmorg-13.0.1/llvm/include/llvm/IR/PassManagerInternal.h:85
#20 0x00007ffff51243f9 in llvm::PassManager<llvm::Module,
llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&,
llvm::AnalysisManager<llvm::Module>&) () at
/tmp/llvm-project-llvmorg-13.0.1/llvm/include/llvm/IR/PassManager.h:509
#21 0x00000000007816d5 in EmitAssemblyWithNewPassManager () at
/tmp/llvm-project-llvmorg-13.0.1/clang/lib/CodeGen/BackendUtil.cpp:1494
#22 0x00000000007842b8 in clang::EmitBackendOutput () at
/tmp/llvm-project-llvmorg-13.0.1/clang/lib/CodeGen/BackendUtil.cpp:1660
#23 0x0000000000d1a7d2 in clang::BackendConsumer::HandleTranslationUnit () at
/tmp/llvm-project-llvmorg-13.0.1/clang/lib/CodeGen/CodeGenAction.cpp:334
#24 0x0000000001e3e499 in clang::ParseAST () at
/tmp/llvm-project-llvmorg-13.0.1/clang/lib/Parse/ParseAST.cpp:171
#25 0x0000000000bdd349 in clang::FrontendAction::Execute () at
/tmp/llvm-project-llvmorg-13.0.1/clang/lib/Frontend/FrontendAction.cpp:951
#26 0x0000000000b91358 in clang::CompilerInstance::ExecuteAction () at
/tmp/llvm-project-llvmorg-13.0.1/clang/lib/Frontend/CompilerInstance.cpp:974
#27 0x0000000002003b6b in clang::ExecuteCompilerInvocation () at
/tmp/llvm-project-llvmorg-13.0.1/clang/lib/FrontendTool/ExecuteCompilerInvocation.cpp:278
#28 0x00000000019bb7ca in cc1_main () at
/tmp/llvm-project-llvmorg-13.0.1/clang/tools/driver/cc1_main.cpp:246
#29 0x000000000073290d in ExecuteCC1Tool () at
/tmp/llvm-project-llvmorg-13.0.1/clang/tools/driver/driver.cpp:338
#30 0x0000000000697eb0 in main () at
/tmp/llvm-project-llvmorg-13.0.1/clang/tools/driver/driver.cpp:409

With the input to clang being just

int f(int *p, int c)
{
        return c ? p[0] : c+p[1];
}

In frame 5 in the backtrace, User.cpp has this highly suspicious snippet:

void *User::operator new(size_t Size) {
  // Allocate space for a single Use*
  void *Storage = ::operator new(Size + sizeof(Use *));
  Use **HungOffOperandList = static_cast<Use **>(Storage);
  User *Obj = reinterpret_cast<User *>(HungOffOperandList + 1);
  Obj->NumUserOperands = 0;
  Obj->HasHungOffUses = true;
  Obj->HasDescriptor = false;
  *HungOffOperandList = nullptr;
  return Obj;
}

and finally adding -flifetime-dse=1 cures the problem.

As for ranger ICE in gcc-12... sigh.

How shall we proceed here?

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[amonakov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #8 from Alexander Monakov <amonakov at gcc dot gnu.org> ---
Ah, forgot to mention that compiler the offending User.cpp without -flto also
avoids the problem.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

Xi Ruoyao <xry111 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |xry111 at gcc dot gnu.org

--- Comment #9 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
To me it looks like https://github.com/llvm/llvm-project/issues/24952.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[amonakov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #10 from Alexander Monakov <amonakov at gcc dot gnu.org> ---
Indeed, that makes things easier, thanks.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #11 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
Is it supported to use different -flifetime-dse settings for TUs when LTO is
enabled?  If yes we can submit a LLVM change to add -flifetime-dse=1 for
User.cpp.o and mark this MOVED.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[amonakov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #12 from Alexander Monakov <amonakov at gcc dot gnu.org> ---
That would not fix the problem, lifetime-dse affects code that creates 'class
User' objects, not the implementation of its 'operator new' override.

(also the linked bug says "MDNode has the same pattern and the same issues")

Is there any need to over-engineer this like that? I would hope enabling
-fno-lifetime-dse globally would not be controversial for LLVM, especially
given the existing bug report and long-term use of -fno-strict-aliasing.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

Jan Hubicka <hubicka at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hubicka at gcc dot gnu.org

--- Comment #13 from Jan Hubicka <hubicka at gcc dot gnu.org> ---
> What's the current best practice for LTO debugging? I don't imagine there's an > easy way to identify which of 2000 lto1 invocation crashes, or attach gdb to it? Or at least generate a corefile

What I do is to compile with --save-temps --verbose and then one can see which
ltrans ICEd in error message from internal make.  The dump also contains all of
ltrans command lines (sometimes garbled by parallel output unoless you are
willing to wait for -flto=1). Invoke the corresponding ltrnans command line
from gdb and wait for crash.

Indeed it is quite long time problem with clang not building with lifetime DSE
and strict aliasing.  I wonder why this is not fixed on clang side?

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[amonakov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #14 from Alexander Monakov <amonakov at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #13)
> Indeed it is quite long time problem with clang not building with lifetime
> DSE and strict aliasing.  I wonder why this is not fixed on clang side?

Because the problems were not communicated? I knew that Firefox needed
-flifetime-dse=1, but it's the first time I hear that any such problems in
Clang/LLVM were identified.

I could not find any workaround for lifetime-dse in SUSE spec file for llvm16.
Are you saying it was known and worked around somehow? Or it is not manifesting
because LLVM is built without LTO?

Re: [Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[Jan Hubicka]

> > Indeed it is quite long time problem with clang not building with lifetime
> > DSE and strict aliasing.  I wonder why this is not fixed on clang side?
> 
> Because the problems were not communicated? I knew that Firefox needed
> -flifetime-dse=1, but it's the first time I hear that any such problems in
> Clang/LLVM were identified.
> 
> I could not find any workaround for lifetime-dse in SUSE spec file for llvm16.
> Are you saying it was known and worked around somehow? Or it is not manifesting
> because LLVM is built without LTO?

I think opensuse package outs-out LTO probably for this reason.  I am
sometimes using LLVM as benchmark of LTO and PGO, so it would be great
to have this enabled in the packages, but I had no time to do that so
far.  LLVM built with LTO and PGO builds quite a lot faster.  I was
filling bugreport for that some time ago and it seems that the bugreport
linked above has quite good analysis about what breaks.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[hubicka at ucw dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #15 from Jan Hubicka <hubicka at ucw dot cz> ---
> > Indeed it is quite long time problem with clang not building with lifetime
> > DSE and strict aliasing.  I wonder why this is not fixed on clang side?
> 
> Because the problems were not communicated? I knew that Firefox needed
> -flifetime-dse=1, but it's the first time I hear that any such problems in
> Clang/LLVM were identified.
> 
> I could not find any workaround for lifetime-dse in SUSE spec file for llvm16.
> Are you saying it was known and worked around somehow? Or it is not manifesting
> because LLVM is built without LTO?

I think opensuse package outs-out LTO probably for this reason.  I am
sometimes using LLVM as benchmark of LTO and PGO, so it would be great
to have this enabled in the packages, but I had no time to do that so
far.  LLVM built with LTO and PGO builds quite a lot faster.  I was
filling bugreport for that some time ago and it seems that the bugreport
linked above has quite good analysis about what breaks.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #16 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
(In reply to Alexander Monakov from comment #14)
> (In reply to Jan Hubicka from comment #13)
> > Indeed it is quite long time problem with clang not building with lifetime
> > DSE and strict aliasing.  I wonder why this is not fixed on clang side?
> 
> Because the problems were not communicated? I knew that Firefox needed
> -flifetime-dse=1, but it's the first time I hear that any such problems in
> Clang/LLVM were identified.
> 
> I could not find any workaround for lifetime-dse in SUSE spec file for
> llvm16. Are you saying it was known and worked around somehow? Or it is not
> manifesting because LLVM is built without LTO?

I guess it's because the "official" way to build LLVM with LTO is performing a
2-stage bootstrap and only enable LTO for stage 2 [1], and LLVM/Clang performs
lifetime DSE less aggressively so it does not break itself...

[1]: https://releases.llvm.org/16.0.0/docs/BuildingADistribution.html

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[amonakov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #17 from Alexander Monakov <amonakov at gcc dot gnu.org> ---
Right, thanks, I think SUSE build log confirms that (careful, large file):
https://build.opensuse.org/public/build/openSUSE:Factory/standard/x86_64/llvm16/_log

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #18 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
(In reply to Alexander Monakov from comment #12)

> Is there any need to over-engineer this like that? I would hope enabling
> -fno-lifetime-dse globally would not be controversial for LLVM

Maybe.  Should we send a patch?

> especially given the existing bug report and long-term use of -fno-strict-aliasing.

If I read the LLVM code correctly, -fno-strict-aliasing is enabled for Clang,
but not other LLVM parts.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[hubicka at ucw dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #19 from Jan Hubicka <hubicka at ucw dot cz> ---
> > Is there any need to over-engineer this like that? I would hope enabling
> > -fno-lifetime-dse globally would not be controversial for LLVM

It would be really nice to have the ranger bug fixed.  Since lifetime
DSE is all handled in C++ FE there is no good reason why it should not
work to LTO togehter objects compiled with the flag and without...

llvm is really nice benchmark for LTO and PGO, so it would be nice if it
was not fragile and built with -O3 -flto.  In my testing GCC produced
binary with LTO+PGO is a lot smaller. It seems to also generate code
faster but parsing is bit slower, which I think may be related to
-fstrict-aliasing.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #20 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #19)
> > > Is there any need to over-engineer this like that? I would hope enabling
> > > -fno-lifetime-dse globally would not be controversial for LLVM
> 
> It would be really nice to have the ranger bug fixed.  Since lifetime
> DSE is all handled in C++ FE there is no good reason why it should not
> work to LTO togehter objects compiled with the flag and without...

I guess it's just some pure "lucky and unlucky" thing which may happen with any
optimization change as the LLVM developers intends to invoke undefined
behavior...
Maybe one day LLVM will break itself as well.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[amonakov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #21 from Alexander Monakov <amonakov at gcc dot gnu.org> ---
(In reply to Xi Ruoyao from comment #18)
> Maybe.  Should we send a patch?

Yes, if we have a volunteer.

> If I read the LLVM code correctly, -fno-strict-aliasing is enabled for
> Clang, but not other LLVM parts.

Right, thanks for the correction.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[amonakov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #22 from Alexander Monakov <amonakov at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #19)
> It would be really nice to have the ranger bug fixed.  Since lifetime
> DSE is all handled in C++ FE there is no good reason why it should not
> work to LTO togehter objects compiled with the flag and without...

Wait. There's absolutely no connection between lifetime-dse and the ranger
issue uncovered by -flto-partition=1to1. Selective -fno-lifetime-dse surely
works, as you said it's fully handled by the front-end. I was just saying that
applying -fno-lifetime-dse to User.cpp is insufficient because problematic
codegen happens in other translation units (where objects of class User are
constructed).

Fully agreed that fixing the ranger bug would be nice regardless.

> llvm is really nice benchmark for LTO and PGO, so it would be nice if it
> was not fragile and built with -O3 -flto.  In my testing GCC produced
> binary with LTO+PGO is a lot smaller. It seems to also generate code
> faster but parsing is bit slower, which I think may be related to
> -fstrict-aliasing.

Makes sense I guess.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|wrong-code                  |ice-on-valid-code

--- Comment #23 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Alexander Monakov from comment #6)
> I'm giving this a shot, and with -flto-partition=1to1 there's a ranger ICE
> (below).
> 
> What's the current best practice for LTO debugging? I don't imagine there's
> an easy way to identify which of 2000 lto1 invocation crashes, or attach gdb
> to it? Or at least generate a corefile?

The easiest way to debug this kind of ICE is first get a reduced testcase.
1) Generate all of the preprocessed sources
2) Write a simple script which does the following:
** Remove the old object files
** compile the (already generated) preprocessed sources with -flto
** compile (link) the object files with the -r option and what ever options
** check if the ICE was there
3) Find the min number of preprocessed sources required to reproduce the issue
4) then reduce the remaining sources (either using delta or cvise or what ever
tool you want to use)

Hope this helps.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[amonakov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #24 from Alexander Monakov <amonakov at gcc dot gnu.org> ---
Appreciate the advice. So far I've managed to reduce the number of LTO inputs
down to two files, RegisterBankInfo.cpp.o plus APInt.cpp.o. I also built
gcc-12.3 with lineinfo and have a better backtrace:

during GIMPLE pass: thread
/tmp/llvm-project-llvmorg-13.0.1/llvm/lib/CodeGen/GlobalISel/RegisterBankInfo.cpp:
In member function 'verify':
/tmp/llvm-project-llvmorg-13.0.1/llvm/lib/CodeGen/GlobalISel/RegisterBankInfo.cpp:548:6:
internal compiler error: Segmentation fault
  548 | bool RegisterBankInfo::ValueMapping::verify(unsigned
MeaningfulBitWidth) const {
      |      ^
0xb369ef crash_signal
        ../../gcc-12.3.0/gcc/toplev.cc:322
0x17f8977 operator_bitwise_not::fold_range(irange&, tree_node*, irange const&,
irange const&, tree_code) const
        ../../gcc-12.3.0/gcc/range-op.cc:3479
0x17f8977 operator_bitwise_not::fold_range(irange&, tree_node*, irange const&,
irange const&, tree_code) const
        ../../gcc-12.3.0/gcc/range-op.cc:3465
0x171cd0a fold_using_range::range_of_range_op(irange&, gimple*, fur_source&)
        ../../gcc-12.3.0/gcc/gimple-range-fold.cc:608
0x171e7e0 fold_using_range::fold_stmt(irange&, gimple*, fur_source&,
tree_node*)
        ../../gcc-12.3.0/gcc/gimple-range-fold.cc:554
0x171eb4c fold_range(irange&, gimple*, range_query*)
        ../../gcc-12.3.0/gcc/gimple-range-fold.cc:315
0xc985fd path_range_query::range_of_stmt(irange&, gimple*, tree_node*)
        ../../gcc-12.3.0/gcc/gimple-range-path.cc:771
0xc99bc4 path_range_query::range_defined_in_block(irange&, tree_node*,
basic_block_def*)
        ../../gcc-12.3.0/gcc/gimple-range-path.cc:356
0xc99dde path_range_query::internal_range_of_expr(irange&, tree_node*, gimple*)
        ../../gcc-12.3.0/gcc/gimple-range-path.cc:209
0xc99dde path_range_query::internal_range_of_expr(irange&, tree_node*, gimple*)
        ../../gcc-12.3.0/gcc/gimple-range-path.cc:193
0xc99ef0 path_range_query::range_of_expr(irange&, tree_node*, gimple*)
        ../../gcc-12.3.0/gcc/gimple-range-path.cc:225
0x171cabf fold_using_range::range_of_range_op(irange&, gimple*, fur_source&)
        ../../gcc-12.3.0/gcc/gimple-range-fold.cc:602
0x171e7e0 fold_using_range::fold_stmt(irange&, gimple*, fur_source&,
tree_node*)
        ../../gcc-12.3.0/gcc/gimple-range-fold.cc:554
0x171eb4c fold_range(irange&, gimple*, range_query*)
        ../../gcc-12.3.0/gcc/gimple-range-fold.cc:315
0xc985fd path_range_query::range_of_stmt(irange&, gimple*, tree_node*)
        ../../gcc-12.3.0/gcc/gimple-range-path.cc:771
0xc99bc4 path_range_query::range_defined_in_block(irange&, tree_node*,
basic_block_def*)
        ../../gcc-12.3.0/gcc/gimple-range-path.cc:356
0xc99dde path_range_query::internal_range_of_expr(irange&, tree_node*, gimple*)
        ../../gcc-12.3.0/gcc/gimple-range-path.cc:209
0xc99dde path_range_query::internal_range_of_expr(irange&, tree_node*, gimple*)
        ../../gcc-12.3.0/gcc/gimple-range-path.cc:193
0xc99ef0 path_range_query::range_of_expr(irange&, tree_node*, gimple*)
        ../../gcc-12.3.0/gcc/gimple-range-path.cc:225
0x171ccab fold_using_range::range_of_range_op(irange&, gimple*, fur_source&)
        ../../gcc-12.3.0/gcc/gimple-range-fold.cc:602
Please submit a full bug report, with preprocessed source (by using
-freport-bug).

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #25 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Alexander Monakov from comment #24)
> Appreciate the advice. So far I've managed to reduce the number of LTO
> inputs down to two files, RegisterBankInfo.cpp.o plus APInt.cpp.o. I also
> built gcc-12.3 with lineinfo and have a better backtrace:
> 
> during GIMPLE pass: thread
> /tmp/llvm-project-llvmorg-13.0.1/llvm/lib/CodeGen/GlobalISel/
> RegisterBankInfo.cpp: In member function 'verify':
> /tmp/llvm-project-llvmorg-13.0.1/llvm/lib/CodeGen/GlobalISel/
> RegisterBankInfo.cpp:548:6: internal compiler error: Segmentation fault
>   548 | bool RegisterBankInfo::ValueMapping::verify(unsigned
> MeaningfulBitWidth) const {
>       |      ^
> 0xb369ef crash_signal
>         ../../gcc-12.3.0/gcc/toplev.cc:322
> 0x17f8977 operator_bitwise_not::fold_range(irange&, tree_node*, irange
> const&, irange const&, tree_code) const
>         ../../gcc-12.3.0/gcc/range-op.cc:3479
> 0x17f8977 operator_bitwise_not::fold_range(irange&, tree_node*, irange
> const&, irange const&, tree_code) const
>         ../../gcc-12.3.0/gcc/range-op.cc:3465
> 0x171cd0a fold_using_range::range_of_range_op(irange&, gimple*, fur_source&)
>         ../../gcc-12.3.0/gcc/gimple-range-fold.cc:608
> 0x171e7e0 fold_using_range::fold_stmt(irange&, gimple*, fur_source&,
> tree_node*)
>         ../../gcc-12.3.0/gcc/gimple-range-fold.cc:554
> 0x171eb4c fold_range(irange&, gimple*, range_query*)
>         ../../gcc-12.3.0/gcc/gimple-range-fold.cc:315
> 0xc985fd path_range_query::range_of_stmt(irange&, gimple*, tree_node*)
>         ../../gcc-12.3.0/gcc/gimple-range-path.cc:771
> 0xc99bc4 path_range_query::range_defined_in_block(irange&, tree_node*,
> basic_block_def*)
>         ../../gcc-12.3.0/gcc/gimple-range-path.cc:356
> 0xc99dde path_range_query::internal_range_of_expr(irange&, tree_node*,
> gimple*)
>         ../../gcc-12.3.0/gcc/gimple-range-path.cc:209
> 0xc99dde path_range_query::internal_range_of_expr(irange&, tree_node*,
> gimple*)
>         ../../gcc-12.3.0/gcc/gimple-range-path.cc:193
> 0xc99ef0 path_range_query::range_of_expr(irange&, tree_node*, gimple*)
>         ../../gcc-12.3.0/gcc/gimple-range-path.cc:225
> 0x171cabf fold_using_range::range_of_range_op(irange&, gimple*, fur_source&)
>         ../../gcc-12.3.0/gcc/gimple-range-fold.cc:602
> 0x171e7e0 fold_using_range::fold_stmt(irange&, gimple*, fur_source&,
> tree_node*)
>         ../../gcc-12.3.0/gcc/gimple-range-fold.cc:554
> 0x171eb4c fold_range(irange&, gimple*, range_query*)
>         ../../gcc-12.3.0/gcc/gimple-range-fold.cc:315
> 0xc985fd path_range_query::range_of_stmt(irange&, gimple*, tree_node*)
>         ../../gcc-12.3.0/gcc/gimple-range-path.cc:771
> 0xc99bc4 path_range_query::range_defined_in_block(irange&, tree_node*,
> basic_block_def*)
>         ../../gcc-12.3.0/gcc/gimple-range-path.cc:356
> 0xc99dde path_range_query::internal_range_of_expr(irange&, tree_node*,
> gimple*)
>         ../../gcc-12.3.0/gcc/gimple-range-path.cc:209
> 0xc99dde path_range_query::internal_range_of_expr(irange&, tree_node*,
> gimple*)
>         ../../gcc-12.3.0/gcc/gimple-range-path.cc:193
> 0xc99ef0 path_range_query::range_of_expr(irange&, tree_node*, gimple*)
>         ../../gcc-12.3.0/gcc/gimple-range-path.cc:225
> 0x171ccab fold_using_range::range_of_range_op(irange&, gimple*, fur_source&)
>         ../../gcc-12.3.0/gcc/gimple-range-fold.cc:602
> Please submit a full bug report, with preprocessed source (by using
> -freport-bug).

This looks like a stack overflow happening. To speed up reducing, reducing the
stack size from 8MB (default) down to 4MB might help speed up things.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[amonakov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #26 from Alexander Monakov <amonakov at gcc dot gnu.org> ---
Would that help? GCC raises its own stack limit to 64MB:

gcc.cc:  stack_limit_increase (64 * 1024 * 1024);
toplev.cc:  stack_limit_increase (64 * 1024 * 1024);

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #27 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Alexander Monakov from comment #26)
> Would that help? GCC raises its own stack limit to 64MB:
> 
> gcc.cc:  stack_limit_increase (64 * 1024 * 1024);
> toplev.cc:  stack_limit_increase (64 * 1024 * 1024);

Oh I forgot about that.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #28 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
(In reply to Alexander Monakov from comment #21)
> (In reply to Xi Ruoyao from comment #18)
> > Maybe.  Should we send a patch?
> 
> Yes, if we have a volunteer.

I'm creating it, but from the description of the LLVM issue 24952:

"To put it simply, operator delete for class User inspects memory of the object
after the end of its lifetime. This shows as a use-after-dtor error when
running under MemorySanitizer."

So it seems technically we'll need -fno-lifetime-dse here?  Our docs say
-flifetime-dse only "preserve stores before the constructor starts" but "still
treat the object as dead after the destructor".

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #29 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
(In reply to Xi Ruoyao from comment #28)
> (In reply to Alexander Monakov from comment #21)
> > (In reply to Xi Ruoyao from comment #18)
> > > Maybe.  Should we send a patch?
> > 
> > Yes, if we have a volunteer.
> 
> I'm creating it, but from the description of the LLVM issue 24952:
> 
> "To put it simply, operator delete for class User inspects memory of the
> object after the end of its lifetime. This shows as a use-after-dtor error
> when running under MemorySanitizer."
> 
> So it seems technically we'll need -fno-lifetime-dse here?  Our docs say
> -flifetime-dse only "preserve stores before the constructor starts" but
  ^^^^^^^^^^^^^^  I mean, -flifetime-dse=1
> "still treat the object as dead after the destructor".

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #30 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
https://reviews.llvm.org/D150505.

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[amonakov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #31 from Alexander Monakov <amonakov at gcc dot gnu.org> ---
(In reply to Xi Ruoyao from comment #28)
> "To put it simply, operator delete for class User inspects memory of the
> object after the end of its lifetime. This shows as a use-after-dtor error
> when running under MemorySanitizer."
> 
> So it seems technically we'll need -fno-lifetime-dse here?  Our docs say
> -flifetime-dse only "preserve stores before the constructor starts" but
> "still treat the object as dead after the destructor".

Agreed, -fno-lifetime-dse seems appropriate here. Thank you for proposing a
patch on the LLVM side!

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[amonakov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

--- Comment #32 from Alexander Monakov <amonakov at gcc dot gnu.org> ---
Ranger ICE is PR 109841 (reduced so it doesn't need LTO).

[Bug c++/106943] GCC building clang/llvm with LTO flags causes ICE in clang

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106943

Xi Ruoyao <xry111 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |MOVED
             Status|WAITING                     |RESOLVED
                URL|                            |https://github.com/llvm/llv
                   |                            |m-project/commit/94f7c961c7
                   |                            |8d8fdbc05898cfbbf88094de45c
                   |                            |1ad

--- Comment #33 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
-fno-lifetime-dse has been added into LLVM building system as a workaround.