* undefined reference to `__aarch64_cas4_sync' error on arm64 native build @ 2024-01-02 3:29 richard clark 2024-01-02 8:53 ` richard clark 0 siblings, 1 reply; 18+ messages in thread From: richard clark @ 2024-01-02 3:29 UTC (permalink / raw) To: gcc-help; +Cc: linux-kernel, linux-arm-kernel Hi, I have a strong power arm64 box, and the linux distro is ubuntu 22.04, the native gcc version is: $ gcc --version gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Copyright (C) 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. It will abort the kernel build with the complaint by 'make Image': ld: Unexpected GOT/PLT entries detected! ld: Unexpected run-time procedure linkages detected! ld: ID map text too big or misaligned ld: drivers/net/nvidia_eth.o: in function `osi_lock_irq_enabled': osi_hal.c:(.text+0x3cc): undefined reference to `__aarch64_cas4_sync' ... But the cross-compile with aarch64-linux-gnu-gcc on the x86 box doesn't show the above error message. Any comments/suggestions? Thanks very much! Richard ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-02 3:29 undefined reference to `__aarch64_cas4_sync' error on arm64 native build richard clark @ 2024-01-02 8:53 ` richard clark 2024-01-04 18:18 ` Mark Rutland 0 siblings, 1 reply; 18+ messages in thread From: richard clark @ 2024-01-02 8:53 UTC (permalink / raw) To: gcc-help; +Cc: linux-kernel, linux-arm-kernel Ah, the driver is trying to use the native gcc built atomic ops like: __sync_val_compare_and_swap, but it seems the native aarch64 doesn't provide these builtin atomic primitives while they are in the cross compile toolchain. The issue can be resolved by replacing the **__sync_val_compare_and_swap** with **atomic_cmpxchg**. But don't know why the native aarch64 toolchain doesn't have those builtin atomic functions... On Tue, Jan 2, 2024 at 11:29 AM richard clark <richard.xnu.clark@gmail.com> wrote: > > Hi, > > I have a strong power arm64 box, and the linux distro is ubuntu 22.04, > the native gcc version is: > > $ gcc --version > gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 > Copyright (C) 2021 Free Software Foundation, Inc. > This is free software; see the source for copying conditions. There is NO > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > > It will abort the kernel build with the complaint by 'make Image': > ld: Unexpected GOT/PLT entries detected! > ld: Unexpected run-time procedure linkages detected! > ld: ID map text too big or misaligned > ld: drivers/net/nvidia_eth.o: in function `osi_lock_irq_enabled': > osi_hal.c:(.text+0x3cc): undefined reference to `__aarch64_cas4_sync' > ... > > But the cross-compile with aarch64-linux-gnu-gcc on the x86 box > doesn't show the above error message. > Any comments/suggestions? Thanks very much! > > Richard ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-02 8:53 ` richard clark @ 2024-01-04 18:18 ` Mark Rutland 2024-01-08 1:28 ` richard clark 0 siblings, 1 reply; 18+ messages in thread From: Mark Rutland @ 2024-01-04 18:18 UTC (permalink / raw) To: richard clark; +Cc: gcc-help, linux-kernel, linux-arm-kernel On Tue, Jan 02, 2024 at 04:53:53PM +0800, richard clark wrote: > Ah, the driver is trying to use the native gcc built atomic ops like: > __sync_val_compare_and_swap, but it seems the native aarch64 doesn't > provide these builtin atomic primitives while they are in the cross > compile toolchain. > The issue can be resolved by replacing the > **__sync_val_compare_and_swap** with **atomic_cmpxchg**. Yup, that's the right thing to do; drivers *shouldn't* use the builtins directly, and *should* use the kernel's native atomic*() API. > But don't know why the native aarch64 toolchain doesn't have those > builtin atomic functions... I suspect this is down to your toolchain enabling -moutline-atomics by default; that expands the builtins into calls to out-of-line functions. I suspect your cross-compile toolchain doesn't enable that by default. As above, since nothing should be using the builtins, we don't implement out-of-line versions nor do we override the option. Mark. > On Tue, Jan 2, 2024 at 11:29 AM richard clark > <richard.xnu.clark@gmail.com> wrote: > > > > Hi, > > > > I have a strong power arm64 box, and the linux distro is ubuntu 22.04, > > the native gcc version is: > > > > $ gcc --version > > gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 > > Copyright (C) 2021 Free Software Foundation, Inc. > > This is free software; see the source for copying conditions. There is NO > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > > > > It will abort the kernel build with the complaint by 'make Image': > > ld: Unexpected GOT/PLT entries detected! > > ld: Unexpected run-time procedure linkages detected! > > ld: ID map text too big or misaligned > > ld: drivers/net/nvidia_eth.o: in function `osi_lock_irq_enabled': > > osi_hal.c:(.text+0x3cc): undefined reference to `__aarch64_cas4_sync' > > ... > > > > But the cross-compile with aarch64-linux-gnu-gcc on the x86 box > > doesn't show the above error message. > > Any comments/suggestions? Thanks very much! > > > > Richard > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-04 18:18 ` Mark Rutland @ 2024-01-08 1:28 ` richard clark 2024-01-08 10:51 ` Mark Rutland 0 siblings, 1 reply; 18+ messages in thread From: richard clark @ 2024-01-08 1:28 UTC (permalink / raw) To: Mark Rutland; +Cc: gcc-help, linux-kernel, linux-arm-kernel Hi Mark, On Fri, Jan 5, 2024 at 2:18 AM Mark Rutland <mark.rutland@arm.com> wrote: > > On Tue, Jan 02, 2024 at 04:53:53PM +0800, richard clark wrote: > > Ah, the driver is trying to use the native gcc built atomic ops like: > > __sync_val_compare_and_swap, but it seems the native aarch64 doesn't > > provide these builtin atomic primitives while they are in the cross > > compile toolchain. > > The issue can be resolved by replacing the > > **__sync_val_compare_and_swap** with **atomic_cmpxchg**. > > Yup, that's the right thing to do; drivers *shouldn't* use the builtins > directly, and *should* use the kernel's native atomic*() API. > Right, that's the way I did to fix that. > > > But don't know why the native aarch64 toolchain doesn't have those > > builtin atomic functions... > > I suspect this is down to your toolchain enabling -moutline-atomics by default; > that expands the builtins into calls to out-of-line functions. I suspect your > cross-compile toolchain doesn't enable that by default. > > As above, since nothing should be using the builtins, we don't implement > out-of-line versions nor do we override the option. > AFAIK, the native build for the kernel will not link to the libc.so but the userland application does, the builtin atomic primitives are implemented in the glibc: target-host $ objdump -t /lib/aarch64-linux-gnu/libc.so.6 | grep __aarch64_cas4 0000000000130950 l F .text 0000000000000034 __aarch64_cas4_relax 0000000000130a10 l F .text 0000000000000034 __aarch64_cas4_rel 0000000000130990 l F .text 0000000000000034 __aarch64_cas4_acq seems the '__sync_val_compare_and_swap' used in the application will be renamed to _aarch64_cas4_{relax, rel, acq}. so the kernel will complain it will link to an 'undefined reference'. But interesting, why the cross-compile kernel will not generate the 'undefined reference', the cross-compile/build kernel will link to the glibc? > > Mark. > > > On Tue, Jan 2, 2024 at 11:29 AM richard clark > > <richard.xnu.clark@gmail.com> wrote: > > > > > > Hi, > > > > > > I have a strong power arm64 box, and the linux distro is ubuntu 22.04, > > > the native gcc version is: > > > > > > $ gcc --version > > > gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 > > > Copyright (C) 2021 Free Software Foundation, Inc. > > > This is free software; see the source for copying conditions. There is NO > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > > > > > > It will abort the kernel build with the complaint by 'make Image': > > > ld: Unexpected GOT/PLT entries detected! > > > ld: Unexpected run-time procedure linkages detected! > > > ld: ID map text too big or misaligned > > > ld: drivers/net/nvidia_eth.o: in function `osi_lock_irq_enabled': > > > osi_hal.c:(.text+0x3cc): undefined reference to `__aarch64_cas4_sync' > > > ... > > > > > > But the cross-compile with aarch64-linux-gnu-gcc on the x86 box > > > doesn't show the above error message. > > > Any comments/suggestions? Thanks very much! > > > > > > Richard > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-08 1:28 ` richard clark @ 2024-01-08 10:51 ` Mark Rutland 2024-01-08 10:56 ` Xi Ruoyao 2024-01-09 2:52 ` richard clark 0 siblings, 2 replies; 18+ messages in thread From: Mark Rutland @ 2024-01-08 10:51 UTC (permalink / raw) To: richard clark; +Cc: gcc-help, linux-kernel, linux-arm-kernel On Mon, Jan 08, 2024 at 09:28:56AM +0800, richard clark wrote: > On Fri, Jan 5, 2024 at 2:18 AM Mark Rutland <mark.rutland@arm.com> wrote: > > On Tue, Jan 02, 2024 at 04:53:53PM +0800, richard clark wrote: > > > But don't know why the native aarch64 toolchain doesn't have those > > > builtin atomic functions... > > > > I suspect this is down to your toolchain enabling -moutline-atomics by default; > > that expands the builtins into calls to out-of-line functions. I suspect your > > cross-compile toolchain doesn't enable that by default. > > > > As above, since nothing should be using the builtins, we don't implement > > out-of-line versions nor do we override the option. > > > AFAIK, the native build for the kernel will not link to the libc.so > but the userland application does, the builtin atomic primitives are > implemented in the glibc: > target-host $ objdump -t /lib/aarch64-linux-gnu/libc.so.6 | grep __aarch64_cas4 > 0000000000130950 l F .text 0000000000000034 __aarch64_cas4_relax > 0000000000130a10 l F .text 0000000000000034 __aarch64_cas4_rel > 0000000000130990 l F .text 0000000000000034 __aarch64_cas4_acq > seems the '__sync_val_compare_and_swap' used in the application will > be renamed to _aarch64_cas4_{relax, rel, acq}. so the kernel will > complain it will > link to an 'undefined reference'. But interesting, why the > cross-compile kernel will not generate the 'undefined reference', the > cross-compile/build kernel will link to the glibc? This is due to a difference in default options between the two compilers; the kernel isn't linked against libc in either case. Your native compiler evidently has -moutline-atomics enabled in its default options. With that enabled, the builtin atomics generate calls to out-of-line functions which the kernel itself does not provide, and hence those result in a link-time error. Your cross-compiler evidently does not have -moutline-atomics enabled in its default options. Without that enabled, the builtin atomics generate inline atomic instructions rather than function calls. Since these don't depend on external functions there's no link-time error. If you pass 'mno-outline-atomics' to your native compiler, the problem should disappear. Mark. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-08 10:51 ` Mark Rutland @ 2024-01-08 10:56 ` Xi Ruoyao 2024-01-09 2:55 ` richard clark 2024-01-09 2:52 ` richard clark 1 sibling, 1 reply; 18+ messages in thread From: Xi Ruoyao @ 2024-01-08 10:56 UTC (permalink / raw) To: Mark Rutland, richard clark; +Cc: gcc-help, linux-kernel, linux-arm-kernel On Mon, 2024-01-08 at 10:51 +0000, Mark Rutland via Gcc-help wrote: > > AFAIK, the native build for the kernel will not link to the libc.so > > but the userland application does, the builtin atomic primitives are > > implemented in the glibc: > > target-host $ objdump -t /lib/aarch64-linux-gnu/libc.so.6 | grep __aarch64_cas4 > > 0000000000130950 l F .text 0000000000000034 __aarch64_cas4_relax > > 0000000000130a10 l F .text 0000000000000034 __aarch64_cas4_rel > > 0000000000130990 l F .text 0000000000000034 __aarch64_cas4_acq > > seems the '__sync_val_compare_and_swap' used in the application will > > be renamed to _aarch64_cas4_{relax, rel, acq}. so the kernel will > > complain it will > > link to an 'undefined reference'. But interesting, why the > > cross-compile kernel will not generate the 'undefined reference', the > > cross-compile/build kernel will link to the glibc? > > This is due to a difference in default options between the two compilers; the > kernel isn't linked against libc in either case. And even if it's not the kernel but a normal application, it still cannot use these functions from Glibc as the objdump output contains "l", meaning these symbols are local symbols and they cannot referred somewhere out of the libc.so.6 itself. -- Xi Ruoyao <xry111@xry111.site> School of Aerospace Science and Technology, Xidian University ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-08 10:56 ` Xi Ruoyao @ 2024-01-09 2:55 ` richard clark 2024-01-09 3:05 ` Xi Ruoyao 0 siblings, 1 reply; 18+ messages in thread From: richard clark @ 2024-01-09 2:55 UTC (permalink / raw) To: Xi Ruoyao; +Cc: Mark Rutland, gcc-help, linux-kernel, linux-arm-kernel On Mon, Jan 8, 2024 at 6:56 PM Xi Ruoyao <xry111@xry111.site> wrote: > > On Mon, 2024-01-08 at 10:51 +0000, Mark Rutland via Gcc-help wrote: > > > AFAIK, the native build for the kernel will not link to the libc.so > > > but the userland application does, the builtin atomic primitives are > > > implemented in the glibc: > > > target-host $ objdump -t /lib/aarch64-linux-gnu/libc.so.6 | grep __aarch64_cas4 > > > 0000000000130950 l F .text 0000000000000034 __aarch64_cas4_relax > > > 0000000000130a10 l F .text 0000000000000034 __aarch64_cas4_rel > > > 0000000000130990 l F .text 0000000000000034 __aarch64_cas4_acq > > > seems the '__sync_val_compare_and_swap' used in the application will > > > be renamed to _aarch64_cas4_{relax, rel, acq}. so the kernel will > > > complain it will > > > link to an 'undefined reference'. But interesting, why the > > > cross-compile kernel will not generate the 'undefined reference', the > > > cross-compile/build kernel will link to the glibc? > > > > This is due to a difference in default options between the two compilers; the > > kernel isn't linked against libc in either case. > > And even if it's not the kernel but a normal application, it still > cannot use these functions from Glibc as the objdump output contains > "l", meaning these symbols are local symbols and they cannot referred > somewhere out of the libc.so.6 itself. Actually you can call those builtin atomic functions in you normal application without link time error, even execute the output binary in the target machine in case of cross-compile, only if the linked .so is in your target environment. > > -- > Xi Ruoyao <xry111@xry111.site> > School of Aerospace Science and Technology, Xidian University ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-09 2:55 ` richard clark @ 2024-01-09 3:05 ` Xi Ruoyao 2024-01-09 6:04 ` richard clark 2024-01-09 7:48 ` Segher Boessenkool 0 siblings, 2 replies; 18+ messages in thread From: Xi Ruoyao @ 2024-01-09 3:05 UTC (permalink / raw) To: richard clark; +Cc: Mark Rutland, gcc-help, linux-kernel, linux-arm-kernel On Tue, 2024-01-09 at 10:55 +0800, richard clark wrote: > On Mon, Jan 8, 2024 at 6:56 PM Xi Ruoyao <xry111@xry111.site> wrote: > > > > On Mon, 2024-01-08 at 10:51 +0000, Mark Rutland via Gcc-help wrote: > > > > AFAIK, the native build for the kernel will not link to the libc.so > > > > but the userland application does, the builtin atomic primitives are > > > > implemented in the glibc: > > > > target-host $ objdump -t /lib/aarch64-linux-gnu/libc.so.6 | grep __aarch64_cas4 > > > > 0000000000130950 l F .text 0000000000000034 __aarch64_cas4_relax > > > > 0000000000130a10 l F .text 0000000000000034 __aarch64_cas4_rel > > > > 0000000000130990 l F .text 0000000000000034 __aarch64_cas4_acq > > > > seems the '__sync_val_compare_and_swap' used in the application will > > > > be renamed to _aarch64_cas4_{relax, rel, acq}. so the kernel will > > > > complain it will > > > > link to an 'undefined reference'. But interesting, why the > > > > cross-compile kernel will not generate the 'undefined reference', the > > > > cross-compile/build kernel will link to the glibc? > > > > > > This is due to a difference in default options between the two compilers; the > > > kernel isn't linked against libc in either case. > > > > And even if it's not the kernel but a normal application, it still > > cannot use these functions from Glibc as the objdump output contains > > "l", meaning these symbols are local symbols and they cannot referred > > somewhere out of the libc.so.6 itself. > Actually you can call those builtin atomic functions in you normal > application without link time error, even execute the output binary in > the target machine in case of cross-compile, only if the linked .so is > in your target environment. Because these functions are provided by libgcc.a: xry111@defiant:~$ objdump -t /usr/lib/gcc/aarch64-unknown-linux-gnu/13.2.0/libgcc.a | grep cas cas_1_1.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas1_relax cas_2_1.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas2_relax cas_4_1.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas4_relax cas_8_1.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas8_relax cas_16_1.o: file format elf64-littleaarch64 0000000000000000 g F .text 000000000000003c .hidden __aarch64_cas16_relax cas_1_2.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas1_acq cas_2_2.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas2_acq cas_4_2.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas4_acq cas_8_2.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas8_acq cas_16_2.o: file format elf64-littleaarch64 0000000000000000 g F .text 000000000000003c .hidden __aarch64_cas16_acq cas_1_3.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas1_rel cas_2_3.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas2_rel cas_4_3.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas4_rel cas_8_3.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas8_rel cas_16_3.o: file format elf64-littleaarch64 0000000000000000 g F .text 000000000000003c .hidden __aarch64_cas16_rel cas_1_4.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas1_acq_rel cas_2_4.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas2_acq_rel cas_4_4.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas4_acq_rel cas_8_4.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000034 .hidden __aarch64_cas8_acq_rel cas_16_4.o: file format elf64-littleaarch64 0000000000000000 g F .text 000000000000003c .hidden __aarch64_cas16_acq_rel cas_1_5.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000038 .hidden __aarch64_cas1_sync cas_2_5.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000038 .hidden __aarch64_cas2_sync cas_4_5.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000038 .hidden __aarch64_cas4_sync cas_8_5.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000038 .hidden __aarch64_cas8_sync cas_16_5.o: file format elf64-littleaarch64 0000000000000000 g F .text 0000000000000040 .hidden __aarch64_cas16_sync It seems libc.so.6 just get these functions from libgcc.a (a hidden global symbol becomes local when you link it into a shared library). But the Linux kernel cannot use neither libc.so nor libgcc.a. (I know some non-Linux kernel developers are overusing libgcc.a for kernels, but IMO this is just wrong and Linux developers also do not do this. If the Linux kernel needs a symbol from libgcc the developers just provide their own implementation.) -- Xi Ruoyao <xry111@xry111.site> School of Aerospace Science and Technology, Xidian University ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-09 3:05 ` Xi Ruoyao @ 2024-01-09 6:04 ` richard clark 2024-01-09 7:48 ` Segher Boessenkool 1 sibling, 0 replies; 18+ messages in thread From: richard clark @ 2024-01-09 6:04 UTC (permalink / raw) To: Xi Ruoyao; +Cc: Mark Rutland, gcc-help, linux-kernel, linux-arm-kernel On Tue, Jan 9, 2024 at 11:06 AM Xi Ruoyao <xry111@xry111.site> wrote: > > On Tue, 2024-01-09 at 10:55 +0800, richard clark wrote: > > On Mon, Jan 8, 2024 at 6:56 PM Xi Ruoyao <xry111@xry111.site> wrote: > > > > > > On Mon, 2024-01-08 at 10:51 +0000, Mark Rutland via Gcc-help wrote: > > > > > AFAIK, the native build for the kernel will not link to the libc.so > > > > > but the userland application does, the builtin atomic primitives are > > > > > implemented in the glibc: > > > > > target-host $ objdump -t /lib/aarch64-linux-gnu/libc.so.6 | grep __aarch64_cas4 > > > > > 0000000000130950 l F .text 0000000000000034 __aarch64_cas4_relax > > > > > 0000000000130a10 l F .text 0000000000000034 __aarch64_cas4_rel > > > > > 0000000000130990 l F .text 0000000000000034 __aarch64_cas4_acq > > > > > seems the '__sync_val_compare_and_swap' used in the application will > > > > > be renamed to _aarch64_cas4_{relax, rel, acq}. so the kernel will > > > > > complain it will > > > > > link to an 'undefined reference'. But interesting, why the > > > > > cross-compile kernel will not generate the 'undefined reference', the > > > > > cross-compile/build kernel will link to the glibc? > > > > > > > > This is due to a difference in default options between the two compilers; the > > > > kernel isn't linked against libc in either case. > > > > > > And even if it's not the kernel but a normal application, it still > > > cannot use these functions from Glibc as the objdump output contains > > > "l", meaning these symbols are local symbols and they cannot referred > > > somewhere out of the libc.so.6 itself. > > Actually you can call those builtin atomic functions in you normal > > application without link time error, even execute the output binary in > > the target machine in case of cross-compile, only if the linked .so is > > in your target environment. > > Because these functions are provided by libgcc.a: > > xry111@defiant:~$ objdump -t /usr/lib/gcc/aarch64-unknown-linux-gnu/13.2.0/libgcc.a | grep cas > cas_1_1.o: file format elf64-littleaarch64 [...] > 0000000000000000 g F .text 0000000000000038 .hidden __aarch64_cas1_sync > cas_2_5.o: file format elf64-littleaarch64 > 0000000000000000 g F .text 0000000000000038 .hidden __aarch64_cas2_sync > cas_4_5.o: file format elf64-littleaarch64 [...] > > It seems libc.so.6 just get these functions from libgcc.a (a hidden > global symbol becomes local when you link it into a shared library). I think libgcc.a should be built with '-fvisibility=hidden' option and hidden symbols, then as you said it will become a local sym when linked with the libc.so.6. > But the Linux kernel cannot use neither libc.so nor libgcc.a. (I know > some non-Linux kernel developers are overusing libgcc.a for kernels, but > IMO this is just wrong and Linux developers also do not do this. If the > Linux kernel needs a symbol from libgcc the developers just provide > their own implementation.) Right, kernel should use its own implementation while not the one from libc.so... > -- > Xi Ruoyao <xry111@xry111.site> > School of Aerospace Science and Technology, Xidian University ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-09 3:05 ` Xi Ruoyao 2024-01-09 6:04 ` richard clark @ 2024-01-09 7:48 ` Segher Boessenkool 2024-01-09 8:09 ` Xi Ruoyao 1 sibling, 1 reply; 18+ messages in thread From: Segher Boessenkool @ 2024-01-09 7:48 UTC (permalink / raw) To: Xi Ruoyao Cc: richard clark, Mark Rutland, gcc-help, linux-kernel, linux-arm-kernel On Tue, Jan 09, 2024 at 11:05:57AM +0800, Xi Ruoyao via Gcc-help wrote: > But the Linux kernel cannot use neither libc.so nor libgcc.a. I have built Linux using libgcc for many years. It is as easy as +LIBGCC := $(shell $(CC) $(KBUILD_CFLAGS) -print-libgcc-file-name) +libs-y += $(LIBGCC) > (I know > some non-Linux kernel developers are overusing libgcc.a for kernels, but > IMO this is just wrong and Linux developers also do not do this. If the > Linux kernel needs a symbol from libgcc the developers just provide > their own implementation.) Yes, and often they have fallen behind. When they eventually catch up they usually just copy the GCC code anyway. Originally the only reasonable argument for not linking against libgcc was so kernel code would not accidentally use double-length divisions. There are other simple ways to have all uses of __divti3 and similar create link errors, so that is not really a good argument. libgcc is an essential part of the compiler. For most targets, for most code, GCC will not generate function calls, there usually are faster (or smaller) things it can do, but it still is necessary to have libgcc for more uncommon things. Using a partial copy of it, behind the times, and maybe even incompatible, is not a great idea. Segher ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-09 7:48 ` Segher Boessenkool @ 2024-01-09 8:09 ` Xi Ruoyao 2024-01-09 8:26 ` Segher Boessenkool 0 siblings, 1 reply; 18+ messages in thread From: Xi Ruoyao @ 2024-01-09 8:09 UTC (permalink / raw) To: Segher Boessenkool Cc: richard clark, Mark Rutland, gcc-help, linux-kernel, linux-arm-kernel On Tue, 2024-01-09 at 01:48 -0600, Segher Boessenkool wrote: > On Tue, Jan 09, 2024 at 11:05:57AM +0800, Xi Ruoyao via Gcc-help wrote: > > But the Linux kernel cannot use neither libc.so nor libgcc.a. > > I have built Linux using libgcc for many years. It is as easy as > > +LIBGCC := $(shell $(CC) $(KBUILD_CFLAGS) -print-libgcc-file-name) > +libs-y += $(LIBGCC) > > > (I know > > some non-Linux kernel developers are overusing libgcc.a for kernels, but > > IMO this is just wrong and Linux developers also do not do this. If the > > Linux kernel needs a symbol from libgcc the developers just provide > > their own implementation.) > > Yes, and often they have fallen behind. When they eventually catch up > they usually just copy the GCC code anyway. > > Originally the only reasonable argument for not linking against libgcc > was so kernel code would not accidentally use double-length divisions. > There are other simple ways to have all uses of __divti3 and similar > create link errors, so that is not really a good argument. > > libgcc is an essential part of the compiler. For most targets, for most > code, GCC will not generate function calls, there usually are faster (or > smaller) things it can do, but it still is necessary to have libgcc for > more uncommon things. Using a partial copy of it, behind the times, > and maybe even incompatible, is not a great idea. But for many targets the kernel uses the soft-float ABI while the user space uses the hard-float ABI and the linker generally refuses to link them altogether. You may argue that "hey, build all multilibs" but I'd say it's stupid and it does not solve all issues: If GCC is configured with things like --with-arch=something, libgcc can contain (FP/vector) instructions unsafe to use in kernel w/o special handling. Currently this issue is making the process very nasty to build systemd-boot on LoongArch desktop distros where the toolchain configured with vector extensions enabled by default. If libgcc is so vital GCC needs to provide a way to make it work for a set of compiler switches incompatible with any pre-built multilib. For example, installing the source of libgcc into /usr/lib/gcc and provide a tool to build a libgcc.a with the specified options. -- Xi Ruoyao <xry111@xry111.site> School of Aerospace Science and Technology, Xidian University ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-09 8:09 ` Xi Ruoyao @ 2024-01-09 8:26 ` Segher Boessenkool 2024-01-10 5:59 ` richard clark 0 siblings, 1 reply; 18+ messages in thread From: Segher Boessenkool @ 2024-01-09 8:26 UTC (permalink / raw) To: Xi Ruoyao Cc: richard clark, Mark Rutland, gcc-help, linux-kernel, linux-arm-kernel On Tue, Jan 09, 2024 at 04:09:20PM +0800, Xi Ruoyao wrote: > If GCC is configured with things like --with-arch=something, libgcc can > contain (FP/vector) instructions unsafe to use in kernel w/o special > handling. No, it can not. Please file a bug (<https://gcc.gnu.org/bugs.html>) if this happens to you. Thanks! --with-arch= only sets the *default* options, the user can override this always (with an -march= option, say). > If libgcc is so vital GCC needs to provide a way to make it work for a > set of compiler switches incompatible with any pre-built multilib. For > example, installing the source of libgcc into /usr/lib/gcc and provide a > tool to build a libgcc.a with the specified options. That is how things are supposed to work, already. Either libgcc uses only instructions that work everywhere, or libgcc is multilibbed, things like that. In all normal cases this is completely transparent to the compiler user. Segher ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-09 8:26 ` Segher Boessenkool @ 2024-01-10 5:59 ` richard clark 2024-01-10 14:10 ` Segher Boessenkool 0 siblings, 1 reply; 18+ messages in thread From: richard clark @ 2024-01-10 5:59 UTC (permalink / raw) To: Segher Boessenkool Cc: Xi Ruoyao, Mark Rutland, gcc-help, linux-kernel, linux-arm-kernel Hi Segher, A ported driver in linux kernel calls '__sync_val_compare_and_swap', the cross-compiler 'aarch64-linux-gnu-gcc' doesn't complain '__aarch64_cas1_sync' undefined reference, but the native compiler will complain. As Mark mentioned, I double check that both cross and native compiler should have ''-moutline-atomics' option enabled, do you know the reason for that? ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-10 5:59 ` richard clark @ 2024-01-10 14:10 ` Segher Boessenkool 2024-01-11 1:42 ` richard clark 0 siblings, 1 reply; 18+ messages in thread From: Segher Boessenkool @ 2024-01-10 14:10 UTC (permalink / raw) To: richard clark Cc: Xi Ruoyao, Mark Rutland, gcc-help, linux-kernel, linux-arm-kernel On Wed, Jan 10, 2024 at 01:59:53PM +0800, richard clark wrote: > A ported driver in linux kernel calls '__sync_val_compare_and_swap', That is a builtin function. It does not necessarily expand to an actual function call. aarch64 will typically expand it to inline code. > the cross-compiler 'aarch64-linux-gnu-gcc' doesn't complain > '__aarch64_cas1_sync' undefined reference, but the native compiler > will complain. As Mark mentioned, I double check that both cross and > native compiler should have ''-moutline-atomics' option enabled, do > you know the reason for that? Ah, so you are requesting external functions. I am no aarch64 expert, but apparently there were some problems in GCC 11, maybe that is what you saw? Or with some distros, anyway. It also matters if you have ARMv8.1-A enabled, the LSE instructions. Open a GCC bug report if you have more details? Thanks, Segher ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-10 14:10 ` Segher Boessenkool @ 2024-01-11 1:42 ` richard clark 2024-01-11 11:26 ` Mark Rutland 0 siblings, 1 reply; 18+ messages in thread From: richard clark @ 2024-01-11 1:42 UTC (permalink / raw) To: Segher Boessenkool Cc: Xi Ruoyao, Mark Rutland, gcc-help, linux-kernel, linux-arm-kernel On Wed, Jan 10, 2024 at 10:12 PM Segher Boessenkool <segher@kernel.crashing.org> wrote: > > On Wed, Jan 10, 2024 at 01:59:53PM +0800, richard clark wrote: > > A ported driver in linux kernel calls '__sync_val_compare_and_swap', > > That is a builtin function. It does not necessarily expand to an actual > function call. aarch64 will typically expand it to inline code. > native gcc version: gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 cross-compiler gcc version: aarch64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Interesting, the same '__sync_val_compare_and_swap' in the .c file will be 'U __aarch64_cas4_sync' in the .o file compiled by native, will be 't __cmpxchg_case_mb_32' in the .o file compiled by aarch64-linux-gnu-gcc... don't know what the reason is > > > the cross-compiler 'aarch64-linux-gnu-gcc' doesn't complain > > '__aarch64_cas1_sync' undefined reference, but the native compiler > > will complain. As Mark mentioned, I double check that both cross and > > native compiler should have ''-moutline-atomics' option enabled, do > > you know the reason for that? > > Ah, so you are requesting external functions. > > I am no aarch64 expert, but apparently there were some problems in > GCC 11, maybe that is what you saw? Or with some distros, anyway. > > It also matters if you have ARMv8.1-A enabled, the LSE instructions. > > Open a GCC bug report if you have more details? Thanks, > > > Segher ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-11 1:42 ` richard clark @ 2024-01-11 11:26 ` Mark Rutland 2024-01-15 9:23 ` richard clark 0 siblings, 1 reply; 18+ messages in thread From: Mark Rutland @ 2024-01-11 11:26 UTC (permalink / raw) To: richard clark Cc: Segher Boessenkool, Xi Ruoyao, gcc-help, linux-kernel, linux-arm-kernel On Thu, Jan 11, 2024 at 09:42:40AM +0800, richard clark wrote: > On Wed, Jan 10, 2024 at 10:12 PM Segher Boessenkool > <segher@kernel.crashing.org> wrote: > > > > On Wed, Jan 10, 2024 at 01:59:53PM +0800, richard clark wrote: > > > A ported driver in linux kernel calls '__sync_val_compare_and_swap', > > > > That is a builtin function. It does not necessarily expand to an actual > > function call. aarch64 will typically expand it to inline code. > > > native gcc version: > gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 > cross-compiler gcc version: > aarch64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 > > Interesting, the same '__sync_val_compare_and_swap' in the .c file > will be 'U __aarch64_cas4_sync' in the .o file compiled by native, > will be 't __cmpxchg_case_mb_32' in the .o file compiled by > aarch64-linux-gnu-gcc... don't know what the reason is The __cmpxchg_case_mb_32() function is kernel code from arch/arm64/include/asm/cmpxchg.h, so I do not believe that's being generated by the compiler from __sync_val_compare_and_swap(). Are you certain that's being built from the exact same C file? Are you able to share the code in question? Where has it come from in the first place? Thanks, Mark. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-11 11:26 ` Mark Rutland @ 2024-01-15 9:23 ` richard clark 0 siblings, 0 replies; 18+ messages in thread From: richard clark @ 2024-01-15 9:23 UTC (permalink / raw) To: Mark Rutland Cc: Segher Boessenkool, Xi Ruoyao, gcc-help, linux-kernel, linux-arm-kernel Hi Mark, On Thu, Jan 11, 2024 at 7:26 PM Mark Rutland <mark.rutland@arm.com> wrote: > > On Thu, Jan 11, 2024 at 09:42:40AM +0800, richard clark wrote: > > On Wed, Jan 10, 2024 at 10:12 PM Segher Boessenkool > > <segher@kernel.crashing.org> wrote: > > > > > > On Wed, Jan 10, 2024 at 01:59:53PM +0800, richard clark wrote: > > > > A ported driver in linux kernel calls '__sync_val_compare_and_swap', > > > > > > That is a builtin function. It does not necessarily expand to an actual > > > function call. aarch64 will typically expand it to inline code. > > > > > native gcc version: > > gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 > > cross-compiler gcc version: > > aarch64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 > > > > Interesting, the same '__sync_val_compare_and_swap' in the .c file > > will be 'U __aarch64_cas4_sync' in the .o file compiled by native, > > will be 't __cmpxchg_case_mb_32' in the .o file compiled by > > aarch64-linux-gnu-gcc... don't know what the reason is > > The __cmpxchg_case_mb_32() function is kernel code from > arch/arm64/include/asm/cmpxchg.h, so I do not believe that's being generated by > the compiler from __sync_val_compare_and_swap(). > > Are you certain that's being built from the exact same C file? > > Are you able to share the code in question? Where has it come from in the first > place? > Ah, double check and the misleading info about the native and cross compiler, the native gcc(Ubuntu 11.4.0-1ubuntu1~22.04) and the cross-compiler - aarch64-linux-gnu-gcc(Ubuntu 11.4.0-1ubuntu1~22.04) will generate the same link error, but the cross-compiler - aarch64-buildroot-linux-gnu-gcc. (aarch64-buildroot-linux-gnu-gcc.br_real (Buildroot 2020.08) 9.3.0) will not generate the link error, I believe it should be as you said, the aarch64-buildroot-linux-gnu-gcc doesn't enable '-moutline-atomics' by default. > > Thanks, > Mark. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: undefined reference to `__aarch64_cas4_sync' error on arm64 native build 2024-01-08 10:51 ` Mark Rutland 2024-01-08 10:56 ` Xi Ruoyao @ 2024-01-09 2:52 ` richard clark 1 sibling, 0 replies; 18+ messages in thread From: richard clark @ 2024-01-09 2:52 UTC (permalink / raw) To: Mark Rutland; +Cc: gcc-help, linux-kernel, linux-arm-kernel On Mon, Jan 8, 2024 at 6:51 PM Mark Rutland <mark.rutland@arm.com> wrote: > > This is due to a difference in default options between the two compilers; the > kernel isn't linked against libc in either case. > > Your native compiler evidently has -moutline-atomics enabled in its default > options. With that enabled, the builtin atomics generate calls to out-of-line > functions which the kernel itself does not provide, and hence those result in a > link-time error. > > Your cross-compiler evidently does not have -moutline-atomics enabled in its > default options. Without that enabled, the builtin atomics generate inline > atomic instructions rather than function calls. Since these don't depend on > external functions there's no link-time error. > > If you pass 'mno-outline-atomics' to your native compiler, the problem should > disappear. > Right, this should be the root cause, just pass 'mno-outline-atomics' to my native compiler, the kernel link time issue doesn't show. But, I test a userland application with '__sync_val_compare_and_swap' used, for the 'aarch64-linux-gnu-gcc test.c', the 'nm a.out' will output: ... 0000000000000960 t __aarch64_cas4_sync 0000000000011011 B __aarch64_have_lse_atomics ... From the above output, seems the cross compiler has '-moutline-atomics' enabled by default, also with 'aarch64-linux-gnu-gcc -mno-outline-atomics test.c', the 'nm a.out' doesn't show the '_aarch64_cas4_sync'. The same result for the native build for the same test.c, this result can't explain why the cross-compile linux kernel will not generate the link time error,... > > Mark. ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2024-01-15 9:23 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-01-02 3:29 undefined reference to `__aarch64_cas4_sync' error on arm64 native build richard clark 2024-01-02 8:53 ` richard clark 2024-01-04 18:18 ` Mark Rutland 2024-01-08 1:28 ` richard clark 2024-01-08 10:51 ` Mark Rutland 2024-01-08 10:56 ` Xi Ruoyao 2024-01-09 2:55 ` richard clark 2024-01-09 3:05 ` Xi Ruoyao 2024-01-09 6:04 ` richard clark 2024-01-09 7:48 ` Segher Boessenkool 2024-01-09 8:09 ` Xi Ruoyao 2024-01-09 8:26 ` Segher Boessenkool 2024-01-10 5:59 ` richard clark 2024-01-10 14:10 ` Segher Boessenkool 2024-01-11 1:42 ` richard clark 2024-01-11 11:26 ` Mark Rutland 2024-01-15 9:23 ` richard clark 2024-01-09 2:52 ` richard clark
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).