[Bug gcov-profile/97461] [11 Regression] allocate_gcov_kvp() deadlocks in firefox LTO+PGO build (overridden malloc() recursion)

["marxin at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97461

--- Comment #8 from Martin Liška <marxin at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #7)
> > No. The only thing we support is a recursive malloc as seen in:
> > ./gcc/testsuite/gcc.dg/tree-prof/indir-call-prof-malloc.c
> > 
> > It was added in g:bc2b1a232b1825b421a1aaa21a0865b2d1e4e08c as we use a
> > statically allocated buffer when we recursively entry allocate_gcov_kvp.
> > 
> > However this is different as we can't call malloc/calloc from the function as
> > we're in code that initializes a memory allocator.
> > 
> > We can mitigate the issue with a pair of new functions __gcov_supress_malloc
> > and __gcov_alloc_malloc that will be called by a custom memory allocator.
> > 
> > What do you think about it?
> 
> How this works with the llvm implementation (that is very similar here,
> right?)
> 
> Honza

They have the very same problem when I disable a statically pre-allocated
buffers with -mllvm -vp-static-alloc=0:

Program received signal SIGILL, Illegal instruction.
0x00000000004014e6 in calloc (nmemb=1, size=8) at pr97461.c:103
103         if (malloc_depth != 0) __builtin_trap();
(gdb) bt
#0  0x00000000004014e6 in calloc (nmemb=1, size=8) at pr97461.c:103
#1  0x0000000000401ae1 in allocateValueProfileCounters (Data=0x40a2c8) at
/home/marxin/Programming/llvm-project/compiler-rt/lib/profile/InstrProfilingValue.c:101
#2  0x0000000000401c45 in instrumentTargetValueImpl (CountValue=1,
CounterIndex=0, Data=0x40a2c8, TargetValue=4199264) at
/home/marxin/Programming/llvm-project/compiler-rt/lib/profile/InstrProfilingValue.c:146
#3  __llvm_profile_instrument_target (TargetValue=4199264, Data=0x40a2c8,
CounterIndex=0) at
/home/marxin/Programming/llvm-project/compiler-rt/lib/profile/InstrProfilingValue.c:232
#4  0x000000000040148f in malloc_impl (size=56) at pr97461.c:85
#5  0x00000000004013fe in malloc (size=56) at pr97461.c:95
#6  0x00007ffff7e048a3 in __add_to_environ (name=0x406138
"__LLVM_PROFILE_RT_INIT_ONCE", value=<optimized out>, combined=<optimized out>,
replace=<optimized out>) at setenv.c:215
#7  0x0000000000402ce4 in truncateCurrentFile ()
#8  0x00000000004039bc in parseAndSetFilename ()
#9  0x0000000000404134 in __llvm_profile_initialize ()
#10 0x0000000000405e95 in __libc_csu_init (argc=argc@entry=1,
argv=argv@entry=0x7fffffffdfa8, envp=0x7fffffffdfb8) at elf-init.c:89
#11 0x00007ffff7decd9a in __libc_start_main (main=0x401580 <main>, argc=1,
argv=0x7fffffffdfa8, init=0x405e50 <__libc_csu_init>, fini=<optimized out>,
rtld_fini=<optimized out>, stack_end=0x7fffffffdf98) at ../csu/libc-start.c:270
#12 0x00000000004012aa in _start () at ../sysdeps/x86_64/start.S:120