[PATCH] ld: Add lib32 directories for 32-bit emulation on FreeBSD/amd64

[PATCH] ld: Add lib32 directories for 32-bit emulation on FreeBSD/amd64

[Rainer Orth]

[-- Attachment #1: Type: text/plain, Size: 989 bytes --]

GNU ld currently fails to link 32-bit executables on FreeBSD/amd64 when
the linked libraries have dependencies on shared objects themselves:

$ gcc -m32 -o ei ei.c -lexecinfo
/var/gcc/binutils/amd64/lib/gcc/amd64-pc-freebsd14.0/13.2.0/../../../../amd64-pc-freebsd14.0/bin/ld: warning: libelf.so.2, needed by /usr/lib/../lib32/libexecinfo.so, not found (try using -rpath or -rpath-link)
/var/gcc/binutils/amd64/lib/gcc/amd64-pc-freebsd14.0/13.2.0/../../../../amd64-pc-freebsd14.0/bin/ld: /usr/lib/../lib32/libexecinfo.so: undefined reference to `elf_begin@R1.0'
[...]

Fixed by handling FreeBSD/amd64 like Linux/x86.

Tested on amd64-pc-freebsd14.0.

Ok for trunk?

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University


2023-12-13  Rainer Orth  <ro@CeBiTec.Uni-Bielefeld.DE>

	ld:
	* emulparams/elf_i386.sh (LIBPATH_SUFFIX, LIBPATH_SUFFIX_SKIP)
	[x86_64*-freebsd*]: Set for i386 emulation.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: ld-freebsd-lib32.patch --]
[-- Type: text/x-patch, Size: 332 bytes --]

diff --git a/ld/emulparams/elf_i386.sh b/ld/emulparams/elf_i386.sh
--- a/ld/emulparams/elf_i386.sh
+++ b/ld/emulparams/elf_i386.sh
@@ -39,4 +39,12 @@ case "$target" in
 	;;
     esac
     ;;
+  x86_64*-freebsd*)
+    case "$EMULATION_NAME" in
+      *i386*)
+	LIBPATH_SUFFIX=32
+	LIBPATH_SUFFIX_SKIP=64
+	;;
+    esac
+    ;;
 esac

Re: [PATCH] ld: Add lib32 directories for 32-bit emulation on FreeBSD/amd64

[Rainer Orth]

Ping?  It's been a week:

	https://sourceware.org/pipermail/binutils/2023-December/131178.html

	Rainer


> GNU ld currently fails to link 32-bit executables on FreeBSD/amd64 when
> the linked libraries have dependencies on shared objects themselves:
>
> $ gcc -m32 -o ei ei.c -lexecinfo
> /var/gcc/binutils/amd64/lib/gcc/amd64-pc-freebsd14.0/13.2.0/../../../../amd64-pc-freebsd14.0/bin/ld:
> warning: libelf.so.2, needed by /usr/lib/../lib32/libexecinfo.so, not found
> (try using -rpath or -rpath-link)
> /var/gcc/binutils/amd64/lib/gcc/amd64-pc-freebsd14.0/13.2.0/../../../../amd64-pc-freebsd14.0/bin/ld:
> /usr/lib/../lib32/libexecinfo.so: undefined reference to `elf_begin@R1.0'
> [...]
>
> Fixed by handling FreeBSD/amd64 like Linux/x86.
>
> Tested on amd64-pc-freebsd14.0.
>
> Ok for trunk?
>
> 	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: [PATCH] ld: Add lib32 directories for 32-bit emulation on FreeBSD/amd64

[Jan Beulich]

On 19.12.2023 23:23, Rainer Orth wrote:
> Ping?  It's been a week:
> 
> 	https://sourceware.org/pipermail/binutils/2023-December/131178.html
> 
> 	Rainer

Not knowing FreeBSD it's hard to approve a change like this. Specifically, ...

>> GNU ld currently fails to link 32-bit executables on FreeBSD/amd64 when
>> the linked libraries have dependencies on shared objects themselves:
>>
>> $ gcc -m32 -o ei ei.c -lexecinfo
>> /var/gcc/binutils/amd64/lib/gcc/amd64-pc-freebsd14.0/13.2.0/../../../../amd64-pc-freebsd14.0/bin/ld:
>> warning: libelf.so.2, needed by /usr/lib/../lib32/libexecinfo.so, not found
>> (try using -rpath or -rpath-link)
>> /var/gcc/binutils/amd64/lib/gcc/amd64-pc-freebsd14.0/13.2.0/../../../../amd64-pc-freebsd14.0/bin/ld:
>> /usr/lib/../lib32/libexecinfo.so: undefined reference to `elf_begin@R1.0'
>> [...]
>>
>> Fixed by handling FreeBSD/amd64 like Linux/x86.
>>
>> Tested on amd64-pc-freebsd14.0.

... it doesn't look implausible that things may have worked on earlier
versions (or else perhaps someone would have noticed long ago), and that
hence your change might break things there.

Jan

Re: [PATCH] ld: Add lib32 directories for 32-bit emulation on FreeBSD/amd64

[Rainer Orth]

Jan Beulich <jbeulich@suse.com> writes:

> On 19.12.2023 23:23, Rainer Orth wrote:
>> Ping?  It's been a week:
>> 
>> 	https://sourceware.org/pipermail/binutils/2023-December/131178.html
>> 
>> 	Rainer
>
> Not knowing FreeBSD it's hard to approve a change like this. Specifically, ...

Unfortunately, there's no listed FreeBSD binutils maintainer.  I'm
Cc'ing John Baldwin who actively maintains gdb on FreeBSD.  Maybe he can
shed some insight or knows someone else who could.

>>> GNU ld currently fails to link 32-bit executables on FreeBSD/amd64 when
>>> the linked libraries have dependencies on shared objects themselves:
>>>
>>> $ gcc -m32 -o ei ei.c -lexecinfo
>>> /var/gcc/binutils/amd64/lib/gcc/amd64-pc-freebsd14.0/13.2.0/../../../../amd64-pc-freebsd14.0/bin/ld:
>>> warning: libelf.so.2, needed by /usr/lib/../lib32/libexecinfo.so, not found
>>> (try using -rpath or -rpath-link)
>>> /var/gcc/binutils/amd64/lib/gcc/amd64-pc-freebsd14.0/13.2.0/../../../../amd64-pc-freebsd14.0/bin/ld:
>>> /usr/lib/../lib32/libexecinfo.so: undefined reference to `elf_begin@R1.0'
>>> [...]
>>>
>>> Fixed by handling FreeBSD/amd64 like Linux/x86.
>>>
>>> Tested on amd64-pc-freebsd14.0.
>
> ... it doesn't look implausible that things may have worked on earlier
> versions (or else perhaps someone would have noticed long ago), and that
> hence your change might break things there.

I'm certain they didn't: I originally developed this patch 4 years ago,
but either forgot to submit it back then or hoped an active member of
the FreeBSD community would.  This must have been in the FreeBSD 11 or
12 timeframe, and obviously nothing has happened/been improved since.

My recent forays into FreeBSD have been less than pleasant,
unfortunately: see GCC PR target/112959 (install.tex needs updates on
FreeBSD) for an overview of the issues on the GCC side.  It seems the
FreeBSD community either cares little about GCC and binutils these days
(having moved to lld as /usr/bin/ld and clang/clang++), or doesn't
believe in upstream bug reports, let alone submitting patches ;-(  This
is not just about GCC/binutils; the same seems to happen on the LLVM
side where they completely ripped out the cmake-based build system and
replaced it with one of their own (based on BSD make).  Trying to build
upstream LLVM on FreeBSD is just as unpleasant as GCC.  E.g. GCC won't
work with lld (cf. GCC PR target/112745), so you need GNU ld here...

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: [PATCH] ld: Add lib32 directories for 32-bit emulation on FreeBSD/amd64

[John Baldwin]

On 12/20/23 1:12 AM, Rainer Orth wrote:
> Jan Beulich <jbeulich@suse.com> writes:
> 
>> On 19.12.2023 23:23, Rainer Orth wrote:
>>> Ping?  It's been a week:
>>>
>>> 	https://sourceware.org/pipermail/binutils/2023-December/131178.html
>>>
>>> 	Rainer
>>
>> Not knowing FreeBSD it's hard to approve a change like this. Specifically, ...
> 
> Unfortunately, there's no listed FreeBSD binutils maintainer.  I'm
> Cc'ing John Baldwin who actively maintains gdb on FreeBSD.  Maybe he can
> shed some insight or knows someone else who could.
> 
>>>> GNU ld currently fails to link 32-bit executables on FreeBSD/amd64 when
>>>> the linked libraries have dependencies on shared objects themselves:
>>>>
>>>> $ gcc -m32 -o ei ei.c -lexecinfo
>>>> /var/gcc/binutils/amd64/lib/gcc/amd64-pc-freebsd14.0/13.2.0/../../../../amd64-pc-freebsd14.0/bin/ld:
>>>> warning: libelf.so.2, needed by /usr/lib/../lib32/libexecinfo.so, not found
>>>> (try using -rpath or -rpath-link)
>>>> /var/gcc/binutils/amd64/lib/gcc/amd64-pc-freebsd14.0/13.2.0/../../../../amd64-pc-freebsd14.0/bin/ld:
>>>> /usr/lib/../lib32/libexecinfo.so: undefined reference to `elf_begin@R1.0'
>>>> [...]
>>>>
>>>> Fixed by handling FreeBSD/amd64 like Linux/x86.
>>>>
>>>> Tested on amd64-pc-freebsd14.0.
>>
>> ... it doesn't look implausible that things may have worked on earlier
>> versions (or else perhaps someone would have noticed long ago), and that
>> hence your change might break things there.
> 
> I'm certain they didn't: I originally developed this patch 4 years ago,
> but either forgot to submit it back then or hoped an active member of
> the FreeBSD community would.  This must have been in the FreeBSD 11 or
> 12 timeframe, and obviously nothing has happened/been improved since.

FreeBSD has always used /usr/lib and for the "native" ABI and /usr/lib32 for
32-bit ABIs on 64-bit platforms.  This includes both i386 on x86-64 as well
as 32-bit powerpc on powerpc64 and more recently 32-bit arm (armv7) on
aarch64.  Given that, I believe the patch to be correct (and it likely applies
for powerpc64* using "powerpc" emulation and aarch64* using "armv7" emulation).

> My recent forays into FreeBSD have been less than pleasant,
> unfortunately: see GCC PR target/112959 (install.tex needs updates on
> FreeBSD) for an overview of the issues on the GCC side.  It seems the
> FreeBSD community either cares little about GCC and binutils these days
> (having moved to lld as /usr/bin/ld and clang/clang++), or doesn't
> believe in upstream bug reports, let alone submitting patches ;-(  This
> is not just about GCC/binutils; the same seems to happen on the LLVM
> side where they completely ripped out the cmake-based build system and
> replaced it with one of their own (based on BSD make).  Trying to build
> upstream LLVM on FreeBSD is just as unpleasant as GCC.  E.g. GCC won't
> work with lld (cf. GCC PR target/112745), so you need GNU ld here...

While LLVM is the primary toolchain for FreeBSD, GCC is not completely
ignored.  I maintain a set of ports for GCC packages customized to build
FreeBSD's base system.  I recently added a new port for GCC 13 for this
purpose and can currently build FreeBSD's development trunk (userland +
kernel) on x86-64 with both GCC 12 and GCC 13.  (Right now GCC 12 is
used in a CI job in FreeBSD's CI infrastructure, and the GCC 13 job is
being added this week or next.)

I do have a couple of patches to GCC I should post upstream (switching
i386 to default to -march=i686 in newer versions, and adding a
__freebsd_kprintf__ format for the in-kernel printf() function), just
haven't done paperwork yet on the GCC side.

In regards to the build system, FreeBSD's entire base system builds with
a single build infrastructure using make (and always has).  For any
external tool used in the base, build glue written in make is used.  This
was true of GCC and binutils when they were part of base as well.

-- 
John Baldwin

Re: [PATCH] ld: Add lib32 directories for 32-bit emulation on FreeBSD/amd64

[Jan Beulich]

On 20.12.2023 18:47, John Baldwin wrote:
> On 12/20/23 1:12 AM, Rainer Orth wrote:
>> Jan Beulich <jbeulich@suse.com> writes:
>>
>>> On 19.12.2023 23:23, Rainer Orth wrote:
>>>> Ping?  It's been a week:
>>>>
>>>> 	https://sourceware.org/pipermail/binutils/2023-December/131178.html
>>>>
>>>> 	Rainer
>>>
>>> Not knowing FreeBSD it's hard to approve a change like this. Specifically, ...
>>
>> Unfortunately, there's no listed FreeBSD binutils maintainer.  I'm
>> Cc'ing John Baldwin who actively maintains gdb on FreeBSD.  Maybe he can
>> shed some insight or knows someone else who could.
>>
>>>>> GNU ld currently fails to link 32-bit executables on FreeBSD/amd64 when
>>>>> the linked libraries have dependencies on shared objects themselves:
>>>>>
>>>>> $ gcc -m32 -o ei ei.c -lexecinfo
>>>>> /var/gcc/binutils/amd64/lib/gcc/amd64-pc-freebsd14.0/13.2.0/../../../../amd64-pc-freebsd14.0/bin/ld:
>>>>> warning: libelf.so.2, needed by /usr/lib/../lib32/libexecinfo.so, not found
>>>>> (try using -rpath or -rpath-link)
>>>>> /var/gcc/binutils/amd64/lib/gcc/amd64-pc-freebsd14.0/13.2.0/../../../../amd64-pc-freebsd14.0/bin/ld:
>>>>> /usr/lib/../lib32/libexecinfo.so: undefined reference to `elf_begin@R1.0'
>>>>> [...]
>>>>>
>>>>> Fixed by handling FreeBSD/amd64 like Linux/x86.
>>>>>
>>>>> Tested on amd64-pc-freebsd14.0.
>>>
>>> ... it doesn't look implausible that things may have worked on earlier
>>> versions (or else perhaps someone would have noticed long ago), and that
>>> hence your change might break things there.
>>
>> I'm certain they didn't: I originally developed this patch 4 years ago,
>> but either forgot to submit it back then or hoped an active member of
>> the FreeBSD community would.  This must have been in the FreeBSD 11 or
>> 12 timeframe, and obviously nothing has happened/been improved since.
> 
> FreeBSD has always used /usr/lib and for the "native" ABI and /usr/lib32 for
> 32-bit ABIs on 64-bit platforms.  This includes both i386 on x86-64 as well
> as 32-bit powerpc on powerpc64 and more recently 32-bit arm (armv7) on
> aarch64.  Given that, I believe the patch to be correct (and it likely applies
> for powerpc64* using "powerpc" emulation and aarch64* using "armv7" emulation).

John - thanks for the feedback.

Rainer - based on the above, your change is approved.

Jan

Re: [PATCH] ld: Add lib32 directories for 32-bit emulation on FreeBSD/amd64

[Rainer Orth]

Hi John,

>>> ... it doesn't look implausible that things may have worked on earlier
>>> versions (or else perhaps someone would have noticed long ago), and that
>>> hence your change might break things there.
>> I'm certain they didn't: I originally developed this patch 4 years ago,
>> but either forgot to submit it back then or hoped an active member of
>> the FreeBSD community would.  This must have been in the FreeBSD 11 or
>> 12 timeframe, and obviously nothing has happened/been improved since.
>
> FreeBSD has always used /usr/lib and for the "native" ABI and /usr/lib32 for
> 32-bit ABIs on 64-bit platforms.  This includes both i386 on x86-64 as well
> as 32-bit powerpc on powerpc64 and more recently 32-bit arm (armv7) on
> aarch64.  Given that, I believe the patch to be correct (and it likely applies
> for powerpc64* using "powerpc" emulation and aarch64* using "armv7" emulation).

good to see one platform that is consitent across architectures here ;-)
Given that I don't have access to any of the other FreeBSD platforms,
I'll leave such a patch to those that do.

>> My recent forays into FreeBSD have been less than pleasant,
>> unfortunately: see GCC PR target/112959 (install.tex needs updates on
>> FreeBSD) for an overview of the issues on the GCC side.  It seems the
>> FreeBSD community either cares little about GCC and binutils these days
>> (having moved to lld as /usr/bin/ld and clang/clang++), or doesn't
>> believe in upstream bug reports, let alone submitting patches ;-(  This
>> is not just about GCC/binutils; the same seems to happen on the LLVM
>> side where they completely ripped out the cmake-based build system and
>> replaced it with one of their own (based on BSD make).  Trying to build
>> upstream LLVM on FreeBSD is just as unpleasant as GCC.  E.g. GCC won't
>> work with lld (cf. GCC PR target/112745), so you need GNU ld here...
>
> While LLVM is the primary toolchain for FreeBSD, GCC is not completely
> ignored.  I maintain a set of ports for GCC packages customized to build
> FreeBSD's base system.  I recently added a new port for GCC 13 for this
> purpose and can currently build FreeBSD's development trunk (userland +
> kernel) on x86-64 with both GCC 12 and GCC 13.  (Right now GCC 12 is
> used in a CI job in FreeBSD's CI infrastructure, and the GCC 13 job is
> being added this week or next.)

There are regular posts of FreeBSD testresults to the gcc-testresults
list, so you are not alone here.  This matches what the Oracle Solaris
guys do for both their solaris-userland repo and use of GCC as a shadow
compiler building OS/Net (the core OS including kernel and Solaris
userland).

> I do have a couple of patches to GCC I should post upstream (switching
> i386 to default to -march=i686 in newer versions, and adding a
> __freebsd_kprintf__ format for the in-kernel printf() function), just
> haven't done paperwork yet on the GCC side.

Good: maybe the other issues I've found building GCC trunk on FreeBSD
can also be resolved and the experience for non-FreeBSD developers
become a bit smoother in the future.  I believe both those GCC
developers (testing their patches on more platforms) and FreeBSD (having
less patches that break their builds) would benefit from that.

> In regards to the build system, FreeBSD's entire base system builds with
> a single build infrastructure using make (and always has).  For any
> external tool used in the base, build glue written in make is used.  This
> was true of GCC and binutils when they were part of base as well.

I won't argue with that.  What becomes problematic IMO is when you
cannot build upstream GCC or LLVM without additional patches: having to
investigate build problems before you even get to testing your own
patches takes a patience that few developers will have.

Thanks for your explanations.

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University