x86_64-gnu port -- help needed

x86_64-gnu port -- help needed

[Sergey Bugaev]

[-- Attachment #1: Type: text/plain, Size: 4042 bytes --]

Hello!

So continuing with the x86_64-gnu port, I wrote and tweaked some things and
got it to compile further, and now I'm facing linking issues. I've been
scratching my head (figuratively...) about this one for > 24 hours now, so
it's only appropriate that I ask for some help.

First, the error message:

x86_64-gnu-gcc -nostdlib -nostartfiles -r -o elf/librtld.map.o
elf/librtld.mapT.o '-Wl,-(' elf/dl-allobjs.os libc_pic.a
mach/libmachuser_pic.a hurd/libhurduser_pic.a -lgcc '-Wl,-)'
-Wl,-Map,elf/librtld.mapT
x86_64-gnu/bin/ld: libc_pic.a(_exit.os): in function `__GI__exit':
posix/../sysdeps/mach/hurd/_exit.c:50: multiple definition of `__GI__exit';
elf/dl-allobjs.os:elf/../sysdeps/mach/hurd/dl-sysdep.c:728: first defined
here
x86_64-gnu/bin/ld: libc_pic.a(strtoul.os): in function
`__GI___strtoul_internal':
stdlib/../sysdeps/wordsize-64/strtoul.c:108: multiple definition of
`__GI___strtoul_internal';
elf/dl-allobjs.os:elf/../sysdeps/mach/hurd/dl-sysdep.c:713: first defined
here

(I've trimmed down the paths to be less unwieldy; in reality they're all
absolute, e.g.
/home/sergey/dev/crosshurd64/src/glibc/build/elf/librtld.map.o)

My understanding is that __GI__ symbols are created by the
hidden_proto/hidden_def macros from include/libc-symbols.h, and used for
avoiding PLT when using glibc symbols from inside glibc itself (how it's
different to the __-prefixed symbols, I have not yet figured out).

There is indeed the __GI__exit strong .text symbol in both
libc_pic.a(_exit.os) and elf/dl-allobjs.os:

$ nm -A libc_pic.a 2>/dev/null | rg __GI__exit
libc_pic.a:version.os:                 U __GI__exit
libc_pic.a:abort.os:                 U __GI__exit
libc_pic.a:exit.os:                 U __GI__exit
libc_pic.a:_exit.os:0000000000000460 T __GI__exit
libc_pic.a:daemon.os:                 U __GI__exit
libc_pic.a:openchild.os:                 U __GI__exit
libc_pic.a:forkpty.os:                 U __GI__exit

$ nm -A elf/dl-allobjs.os | rg _exit
elf/dl-allobjs.os:00000000000192b0 T __check__exit_no_hidden
elf/dl-allobjs.os:00000000000192b0 W _exit
elf/dl-allobjs.os:00000000000192b0 T __GI__exit
elf/dl-allobjs.os:                 U __proc_mark_exit

but this same thing is true of my i686-gnu build, which somehow works!

$ nm -A libc_pic.a 2>/dev/null | rg __GI__exit
libc_pic.a:version.os:         U __GI__exit
libc_pic.a:abort.os:         U __GI__exit
libc_pic.a:exit.os:         U __GI__exit
libc_pic.a:_exit.os:000004a0 T __GI__exit
libc_pic.a:daemon.os:         U __GI__exit
libc_pic.a:openchild.os:         U __GI__exit
libc_pic.a:forkpty.os:         U __GI__exit

$ nm -A elf/dl-allobjs.os | rg _exit
elf/dl-allobjs.os:000190a0 T __check__exit_no_hidden
elf/dl-allobjs.os:000190a0 W _exit
elf/dl-allobjs.os:000190a0 T __GI__exit
elf/dl-allobjs.os:         U __proc_mark_exit

Here, it just says
i686-gnu-gcc -nostdlib -nostartfiles -r -o elf/librtld.map.o
elf/librtld.mapT.o '-Wl,-(' elf/dl-allobjs.os libc_pic.a
mach/libmachuser_pic.a hurd/libhurduser_pic.a -lgcc '-Wl,-)'
-Wl,-Map,elf/librtld.mapT
i686-gnu/bin/ld: warning: elf/dl-allobjs.os: requires executable stack
(because the .note.GNU-stack section is executable)

(Again, I've trimmed the paths.)

Questions:
1. What is this about, what's it even trying to do? If this is linking the
dynamic linker (ld.so / rtld — these are the same thing, right?), then why
does it need the dl-sysdep version if the real version is available? If
this is something else, such as maybe testing that the dynamic linker and
glibc proper don't have symbol conflicts, then clearly they do!
2. Why does dl-sysdep define a strong __GI__exit and not a weak one? Isn't
the point to upgrade to the full version once it's available? (But would
that even work, considering __GI__ function calls do not go through PLT?)
3. How come this works on i686-gnu, the duplicated symbols are clearly
present there too!
4. Whatever the answer to #2 is, why doesn't it work on x86_64-gnu?

Sergey

Re: x86_64-gnu port -- help needed

[Andreas Schwab]

On Mär 09 2023, Sergey Bugaev wrote:

> x86_64-gnu-gcc -nostdlib -nostartfiles -r -o elf/librtld.map.o
> elf/librtld.mapT.o '-Wl,-(' elf/dl-allobjs.os libc_pic.a
> mach/libmachuser_pic.a hurd/libhurduser_pic.a -lgcc '-Wl,-)'
> -Wl,-Map,elf/librtld.mapT
> x86_64-gnu/bin/ld: libc_pic.a(_exit.os): in function `__GI__exit':
> posix/../sysdeps/mach/hurd/_exit.c:50: multiple definition of `__GI__exit';
> elf/dl-allobjs.os:elf/../sysdeps/mach/hurd/dl-sysdep.c:728: first defined
> here

The only symbol provided by _exit.os that is not defined by
../sysdeps/mach/hurd/dl-sysdep.c is _Exit.  Does anything call this
function?

> x86_64-gnu/bin/ld: libc_pic.a(strtoul.os): in function
> `__GI___strtoul_internal':
> stdlib/../sysdeps/wordsize-64/strtoul.c:108: multiple definition of
> `__GI___strtoul_internal';
> elf/dl-allobjs.os:elf/../sysdeps/mach/hurd/dl-sysdep.c:713: first defined
> here

Similarily, something is pulling in strtoul.os because it references a
symbol from there not defined by ../sysdeps/mach/hurd/dl-sysdep.c.

elf/librtld.mapT should tell you where the references come from.

> 1. What is this about, what's it even trying to do? If this is linking the
> dynamic linker (ld.so / rtld — these are the same thing, right?), then why
> does it need the dl-sysdep version if the real version is available?

The comments in ../sysdeps/mach/hurd/dl-sysdep.c indicate that the
dynamic linker wants to use some stripped down implementation of these
functions.

> 3. How come this works on i686-gnu, the duplicated symbols are clearly
> present there too!

Pure luck, perhaps.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: x86_64-gnu port -- help needed

[Sergey Bugaev]

On Thu, Mar 9, 2023 at 8:26 PM Andreas Schwab <schwab@linux-m68k.org> wrote:
> Similarily, something is pulling in strtoul.os because it references a
> symbol from there not defined by ../sysdeps/mach/hurd/dl-sysdep.c.
>
> elf/librtld.mapT should tell you where the references come from.

Thank you!!!

Apparently I'm misunderstanding how static linking works. I have
always assumed that linking a static library works just like linking
in all of its object files -- i.e. conceptually *all* object files are
linked in, but the linker could optimize some away if they are not
referenced by anything else. But what you're saying seems to indicate
that object files are only pulled in by something referencing a symbol
that they define, i.e. this is done lazily, not eagerly. In my mental
model, it is fully expected that you do get all the errors like
duplicate symbols and undefined symbol references from the static
library, even if you don't actually use much of it, because
conceptually you're linking all of it in. That's why I was so
surprised to see this work for i686-gnu.

That also explains why it's seemingly linking all of glibc into ld.so:
it's only supposed to pull in some (hopefully very few) objects that
it needs, and not all of it.

And by inspecting elf/librtld.mapT, I see that the reason it's pulling
_exit.os in is that ld.so needs __libc_tls_initialized in a bunch of
places, and that starts a chain of dependencies that ultimately leads
to _exit.os. And that also explains that difference between i686-gnu
and x86_64-gnu: i686-gnu just kind of looks at %gs to figure out if
TLS has been initialized, but on x86_64 I had to add
__libc_tls_initialized as an explicit flag.

I recall that ldsodefs.h/GL is the way to share some piece of data
between ld.so and glibc proper? Should I put __libc_tls_initialized in
there? How do I access GL() values from assembler, do I need to add it
to some xxx-offsets.sym file? In fact, I see that there is already a
sysdeps/x86_64/rtld-offsets.sym, that contains offsetof (struct
rtld_global, _dl_tls_generation), so likely yes.

Alternatively, what if I made a separate weak declaration of
__libc_tls_initialized in dl-sysdep.c, would that work & get
automatically upgraded to the "real" one once glibc is loaded? Do I
understand it correctly that the downside of this approach is that
this will generate another GLIBC_PRIVATE symbol reference (or will
it?), and that's not nice, whereas rtld_global is one symbol
multiplexing many useful pieces of data?

Sergey

Re: x86_64-gnu port -- help needed

[Jessica Clarke]

On 9 Mar 2023, at 19:44, Sergey Bugaev <bugaevc@gmail.com> wrote:
> 
> On Thu, Mar 9, 2023 at 8:26 PM Andreas Schwab <schwab@linux-m68k.org> wrote:
>> Similarily, something is pulling in strtoul.os because it references a
>> symbol from there not defined by ../sysdeps/mach/hurd/dl-sysdep.c.
>> 
>> elf/librtld.mapT should tell you where the references come from.
> 
> Thank you!!!
> 
> Apparently I'm misunderstanding how static linking works. I have
> always assumed that linking a static library works just like linking
> in all of its object files -- i.e. conceptually *all* object files are
> linked in, but the linker could optimize some away if they are not
> referenced by anything else. But what you're saying seems to indicate
> that object files are only pulled in by something referencing a symbol
> that they define, i.e. this is done lazily, not eagerly.

Object files listed explicitly on the command line are eagerly added.
Object files from libraries are lazily added. With --gc-sections some
parts of the resulting binary may also later be deleted, but that is
opt-in and limited in scope (and there are ways to override that).

Jess

> In my mental
> model, it is fully expected that you do get all the errors like
> duplicate symbols and undefined symbol references from the static
> library, even if you don't actually use much of it, because
> conceptually you're linking all of it in. That's why I was so
> surprised to see this work for i686-gnu.
> 
> That also explains why it's seemingly linking all of glibc into ld.so:
> it's only supposed to pull in some (hopefully very few) objects that
> it needs, and not all of it.
> 
> And by inspecting elf/librtld.mapT, I see that the reason it's pulling
> _exit.os in is that ld.so needs __libc_tls_initialized in a bunch of
> places, and that starts a chain of dependencies that ultimately leads
> to _exit.os. And that also explains that difference between i686-gnu
> and x86_64-gnu: i686-gnu just kind of looks at %gs to figure out if
> TLS has been initialized, but on x86_64 I had to add
> __libc_tls_initialized as an explicit flag.
> 
> I recall that ldsodefs.h/GL is the way to share some piece of data
> between ld.so and glibc proper? Should I put __libc_tls_initialized in
> there? How do I access GL() values from assembler, do I need to add it
> to some xxx-offsets.sym file? In fact, I see that there is already a
> sysdeps/x86_64/rtld-offsets.sym, that contains offsetof (struct
> rtld_global, _dl_tls_generation), so likely yes.
> 
> Alternatively, what if I made a separate weak declaration of
> __libc_tls_initialized in dl-sysdep.c, would that work & get
> automatically upgraded to the "real" one once glibc is loaded? Do I
> understand it correctly that the downside of this approach is that
> this will generate another GLIBC_PRIVATE symbol reference (or will
> it?), and that's not nice, whereas rtld_global is one symbol
> multiplexing many useful pieces of data?
> 
> Sergey
>