glibc tunables to disable axv not updating memcpy

glibc tunables to disable axv not updating memcpy

[Vineet Gupta]

Hi,

I've been playing with x86_64 tunables to see how they are plumbed into 
ifunc machinery, but running into a something which I don't understand 
(this is glibc 2.35 so pretty recent).

I'm disabling AVX and friends as:
export 
GLIBC_TUNABLES=glibc.cpu.hwcaps=--SSSE3,-SSE4_2,AVX,-AVX2,-AVX512F,-AVX512VL


A simple program which calls memcpy, seems to be calling 
__memmove_evex_unaligned_erms() despite my tunable disabling AVX512VL.

|
| main (argc=2, argv=0x7fffffffdce8) at test-mcpu.c:10
| 10		memcpy(dest, src, atoi(argv[1]));
| (gdb) s
| __memmove_evex_unaligned_erms () at 
../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:307
| 307	../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S


glibc dependency on AVX512VL for the said routine/

| 	      IFUNC_IMPL_ADD (array, i, memmove,
|			      CPU_FEATURE_USABLE (AVX512VL),
|			      __memmove_evex_unaligned_erms)
|

There used to be AVX*_Usable for tunables but it seems that was changed 
in 107e6a3c2212ba7a3a (" x86: Support usable check for all CPU features")

I also don't understand how the memcpy is being redirected to memmove, 
since assembler in test has a PLT call to memcpy.

LD_DEBUG=all doesn't print any ifunc related info.

I've tried hacking x86 elf_machine_lazy_rel() add dl_debug_printf() 
around R_X86_64_IRELATIVE - which seems to work but there's too less 
info to help answer the exact ifunc plumbing.

Any ideas where to look ?

Thx,
-Vineet

Re: glibc tunables to disable axv not updating memcpy

[Florian Weimer]

* Vineet Gupta via Libc-help:

> Hi,
>
> I've been playing with x86_64 tunables to see how they are plumbed
> into ifunc machinery, but running into a something which I don't
> understand (this is glibc 2.35 so pretty recent).
>
> I'm disabling AVX and friends as:
> export
> GLIBC_TUNABLES=glibc.cpu.hwcaps=--SSSE3,-SSE4_2,AVX,-AVX2,-AVX512F,-AVX512VL
>
>
> A simple program which calls memcpy, seems to be calling
> __memmove_evex_unaligned_erms() despite my tunable disabling AVX512VL.
>
> |
> | main (argc=2, argv=0x7fffffffdce8) at test-mcpu.c:10
> | 10		memcpy(dest, src, atoi(argv[1]));
> | (gdb) s
> | __memmove_evex_unaligned_erms () at
>   ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:307
> | 307	../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S
>
>
> glibc dependency on AVX512VL for the said routine/
>
> | 	      IFUNC_IMPL_ADD (array, i, memmove,
> |			      CPU_FEATURE_USABLE (AVX512VL),
> |			      __memmove_evex_unaligned_erms)
> |

This is for the test suite and the benchmarks.  The production selector
is in sysdeps/x86_64/multiarch/ifunc-memmove.h.  You should be able to
set a line-based breakpoint and single-step through it.

> LD_DEBUG=all doesn't print any ifunc related info.

I want to add logging, which is why I asked on the binutils list:

Subject: Named local symbols in the ELF dynamic symbol table
To: binutils@sourceware.org
Date: Tue, 06 Dec 2022 18:11:08 +0100 (1 day, 15 hours, 17 minutes ago)
Message-ID: <87y1rke9ub.fsf@oldenburg.str.redhat.com>

If we have the symbol names in the dynamic symbol tables, we can use
dladdr or a custom implementation of it to get symbol names.

If I don't get a reply, I'll settle for printing relative addresses.
People can then use an external tool to resolve them to symbols.

Thanks,
Florian

glibc ifunc debug (was Re: glibc tunables to disable axv not updating memcpy)

[Vineet Gupta]

On 12/8/22 00:31, Florian Weimer wrote:
> * Vineet Gupta via Libc-help:
>
>> Hi,
>>
>> I've been playing with x86_64 tunables to see how they are plumbed
>> into ifunc machinery, but running into a something which I don't
>> understand (this is glibc 2.35 so pretty recent).
>>
>> I'm disabling AVX and friends as:
>> export
>> GLIBC_TUNABLES=glibc.cpu.hwcaps=--SSSE3,-SSE4_2,AVX,-AVX2,-AVX512F,-AVX512VL
>>
>>
>> A simple program which calls memcpy, seems to be calling
>> __memmove_evex_unaligned_erms() despite my tunable disabling AVX512VL.
>>
>> |
>> | main (argc=2, argv=0x7fffffffdce8) at test-mcpu.c:10
>> | 10		memcpy(dest, src, atoi(argv[1]));
>> | (gdb) s
>> | __memmove_evex_unaligned_erms () at
>>    ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:307
>> | 307	../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S
>>
>>
>> glibc dependency on AVX512VL for the said routine/
>>
>> | 	      IFUNC_IMPL_ADD (array, i, memmove,
>> |			      CPU_FEATURE_USABLE (AVX512VL),
>> |			      __memmove_evex_unaligned_erms)
>> |
> This is for the test suite and the benchmarks.  The production selector
> is in sysdeps/x86_64/multiarch/ifunc-memmove.h.  You should be able to
> set a line-based breakpoint and single-step through it.

Unfortunately gdb doesn't want to put a breakpoint in the ifunc header.
I tried a -g3 -O2 build and excellent tips at [1] - Ubuntu 22.04 - gdb 12.1

| gdb elf/ld-linux-x86-64.so.2
| set exec-wrapper env LD_LIBRARY_PATH=....
| b _start
| ..
| b sysdeps/x86_64/multiarch/ifunc-memmove.h:48
| No source file named sysdeps/x86_64/multiarch/ifunc-memmove.h
| b ../sysdeps/x86_64/multiarch/ifunc-memmove.h:48

It does seem to recognize other headers in x86_64.
Do I need a newer gdb ? Also can't build glibc with -O0 anyways.

Thx,
-Vineet

[1] https://sourceware.org/glibc/wiki/Debugging/Development_Debugging