Re: [PATCH] powerpc: Define USE_PPC64_NOTOC iff compiler and linker also supports it

[Adhemerval Zanella <adhemerval.zanella@linaro.org>]

On 25/11/2021 14:02, Tulio Magno Quites Machado Filho wrote:
> Adhemerval Zanella via Libc-alpha <libc-alpha@sourceware.org> writes:
> 
>> On 19/11/2021 12:52, Tulio Magno Quites Machado Filho wrote:
>>> 2. LLD itself crashes while linking the tests.
>>>    I need to collect more details.
>>
>> Do you have a log of what is happening? Could it be something related to the
>> debian provided LLD? Could you check with the official package?
> 
> Using the official package and --with-cpu=power9, I'm able to build and run
> the tests. I get:
> 
> FAIL: elf/ifuncmain1pic
> FAIL: elf/ifuncmain1pie
> FAIL: elf/ifuncmain1vis
> FAIL: elf/ifuncmain1vispic
> FAIL: elf/ifuncmain1vispie
> FAIL: elf/ifuncmain3
> FAIL: elf/ifuncmain6pie
> XPASS: elf/tst-protected1a
> XPASS: elf/tst-protected1b
> FAIL: elf/tst-tlsopt-powerpc
> UNSUPPORTED: math/test-fesetexcept-traps
> UNSUPPORTED: math/test-fexcept-traps
> UNSUPPORTED: misc/tst-adjtimex
> UNSUPPORTED: misc/tst-clock_adjtime
> UNSUPPORTED: misc/tst-ntp_adjtime
> UNSUPPORTED: misc/tst-pkey
> FAIL: nptl/tst-execstack
> UNSUPPORTED: time/tst-clock_settime
> UNSUPPORTED: time/tst-settimeofday
> Summary of test results:
>       9 FAIL
>    4663 PASS
>       8 UNSUPPORTED
>      16 XFAIL
>       2 XPASS
> 
> All the 9 failures are unexpected.

That's what I saw on a power9 machine as well.

> 
> I had to kill elf/ifuncmain1pie by hand because it was stuck in an infinite
> loop in foo_protected, constantly loading its own address and jumping to it.

Yeah, I will fix it by using libsupport for elf/ifuncmain1.c. 

> 
> The contents of the function are not what I expected.  It's almost identical to
> its __plt_ counterpart except that it doesn't save r2:
> 
> 0000000000010d90 <foo_protected>:
>    10d90:       01 00 82 3d     addis   r12,r2,1
>    10d94:       a0 80 8c e9     ld      r12,-32608(r12)
>    10d98:       a6 03 89 7d     mtctr   r12
>    10d9c:       20 04 80 4e     bctr
> 
> =======

This does seem a lld issue and I also see some issues with ifunc and protected
functions on armhf.  I spent some time to debug, and it could get them to work
by forcing the creation of canonical entries on ELF/Relocations.cpp:1501). I 
have not dig into to understand what is happening exactly on lld side.

> 
> Going back to the discussion around the crashes, I can only reproduce them with
> the Debian build and using a high number of jobs (e.g -j150).  That generates an
> unusable log that I can only distinguish that a few lld processes received
> SIGABRT.
> If I use a lower -j value, e.g. -j10, the crashes do not happen.
> 
> This is what I managed to parse:
> 
> terminate called after throwing an instance of 'std::system_error'
> what():  Resource temporarily unavailable
> PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
> /usr/lib/powerpc64le-linux-gnu/libLLVM-13.so.1(_ZN4llvm3sys15PrintStackTraceERNS_11raw_ostreamEi+0x4
> c)[0x7fffb262c50c]
> ... <= Hard to understand what is going on here
> collect2: fatal error: ld terminated with signal 6 [Aborted]
> compilation terminated.
> make[2]: *** [../Rules:277: /build/glibc-build.lld/math/test-signgam-ullong-static] Error 1
> make[2]: *** [../Rules:277: /build/glibc-build.lld/math/atest-exp] Error 1
> make[2]: *** [../Rules:277: /build/glibc-build.lld/math/atest-sincos] Error 1
> make[2]: *** [../Rules:230: /build/glibc-build.lld/math/test-matherr] Error 1
> make[2]: *** [../Rules:230: /build/glibc-build.lld/math/test-matherr-2] Error 1
> make[2]: *** [../Rules:230: /build/glibc-build.lld/math/test-math-isinff] Error 1
> make[2]: *** [../Rules:277: /build/glibc-build.lld/math/test-signgam-ullong-init-static] Error 1
> 

I think this is an unrelated issue and due the fact lld by default uses parallel
algorithms on some operations and they scale based on the system core count.
So running multiple lld will create a *lot* of threads (for instance, -j150 on 
a 128 core machine can potentially create 19200 threads) which might reach some 
system enforced limit (for instance through systemd).

I usually build llvm with -DLLVM_ENABLE_THREADS=false to avoid it, since I also
did not see much speed up by using threads with lld and glibc.