public inbox for glibc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug build/29456] New: missing DT_HASH section in shared objects
@ 2022-08-08  9:10 arek at hiler dot eu
  2022-08-08  9:39 ` [Bug build/29456] " mikhail.v.gavrilov at gmail dot com
                   ` (24 more replies)
  0 siblings, 25 replies; 26+ messages in thread
From: arek at hiler dot eu @ 2022-08-08  9:10 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

            Bug ID: 29456
           Summary: missing DT_HASH section in shared objects
           Product: glibc
           Version: 2.36
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: build
          Assignee: unassigned at sourceware dot org
          Reporter: arek at hiler dot eu
                CC: carlos at redhat dot com
  Target Milestone: ---

commit e47de5cb2d4dbecb58f569ed241e8e95c568f03c causes glibc shared object to
not have DT_HASH section anymore. Only the DT_GNU_HASH is still present.

What seems to be forgotten is that glibc's dlopen() is not the only thing that
may depend on it. This breaks SysV ABI compatibility and all the tools that use
DT_HASH for symbol lookup.

This breaks all the games running via Proton that are using EAC.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
@ 2022-08-08  9:39 ` mikhail.v.gavrilov at gmail dot com
  2022-08-08 10:45 ` fweimer at redhat dot com
                   ` (23 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: mikhail.v.gavrilov at gmail dot com @ 2022-08-08  9:39 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

mikhail.v.gavrilov at gmail dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mikhail.v.gavrilov at gmail dot co
                   |                            |m

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
  2022-08-08  9:39 ` [Bug build/29456] " mikhail.v.gavrilov at gmail dot com
@ 2022-08-08 10:45 ` fweimer at redhat dot com
  2022-08-08 11:33 ` arek at hiler dot eu
                   ` (22 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: fweimer at redhat dot com @ 2022-08-08 10:45 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fweimer at redhat dot com
              Flags|                            |security-

--- Comment #1 from Florian Weimer <fweimer at redhat dot com> ---
On typical distributions, none of the system libraries use DT_HASH, not even
core libraries such as libgcc_s.so.1. Any tool that performs symbol lookups
needs to support DT_GNU_HASH these days.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
  2022-08-08  9:39 ` [Bug build/29456] " mikhail.v.gavrilov at gmail dot com
  2022-08-08 10:45 ` fweimer at redhat dot com
@ 2022-08-08 11:33 ` arek at hiler dot eu
  2022-08-08 12:01 ` fweimer at redhat dot com
                   ` (21 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: arek at hiler dot eu @ 2022-08-08 11:33 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

--- Comment #2 from Arkadiusz Hiler <arek at hiler dot eu> ---
Hi, thanks for the reply. I wonder where the defaulting to having only
DT_GNU_HASH is coming from?

Quoting ld's man page:

       --hash-style=style
           Set the type of linker's hash table(s).
           style can be either "sysv" for classic
           ELF ".hash" section, "gnu" for new style
           GNU ".gnu.hash" section or "both" for
           both the classic ELF ".hash" and new
           style GNU ".gnu.hash" hash tables.  The
           default depends upon how the linker was
           configured, but for most Linux based
           systems it will be "both".


and indeed `ld --help` claims:

    --hash-style=STYLE          Set hash style to sysv/gnu/both.  Default: both

As a sidenote mold defaults to sysv style hashes[0] and the format is well
documented[1], whereas for the GNU-style one you have to read the sources of
one of the implementations or rely on blog posts.

The change also breaks existing software, not sure what glibc's stance is on
that.

[0]:
https://github.com/rui314/mold/blob/415673a49e24b1f4189000b9eb4fb1a36a697093/elf/mold.h#L1475
[1]: https://refspecs.linuxfoundation.org/elf/gabi41.pdf section 5-21

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (2 preceding siblings ...)
  2022-08-08 11:33 ` arek at hiler dot eu
@ 2022-08-08 12:01 ` fweimer at redhat dot com
  2022-08-08 12:43 ` adhemerval.zanella at linaro dot org
                   ` (20 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: fweimer at redhat dot com @ 2022-08-08 12:01 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

--- Comment #3 from Florian Weimer <fweimer at redhat dot com> ---
Distributions configure or patch GCC to pass --hash-style=gnu to the linker by
default. The linker default does not matter.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (3 preceding siblings ...)
  2022-08-08 12:01 ` fweimer at redhat dot com
@ 2022-08-08 12:43 ` adhemerval.zanella at linaro dot org
  2022-08-08 14:34 ` arek at hiler dot eu
                   ` (19 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-08-08 12:43 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

Adhemerval Zanella <adhemerval.zanella at linaro dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |adhemerval.zanella at linaro dot o
                   |                            |rg

--- Comment #4 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
(In reply to Arkadiusz Hiler from comment #0)
> commit e47de5cb2d4dbecb58f569ed241e8e95c568f03c causes glibc shared object
> to not have DT_HASH section anymore. Only the DT_GNU_HASH is still present.
> 
> What seems to be forgotten is that glibc's dlopen() is not the only thing
> that may depend on it. This breaks SysV ABI compatibility and all the tools
> that use DT_HASH for symbol lookup.

The static linker option provides objects with only one kind of hash section.
so I take it is expected from tools to handle this on GNU Linux environments. 

> 
> This breaks all the games running via Proton that are using EAC.

What EAC is doing exactly that requires a DT_HASH on glibc? What happens if you
mix DT_HASH and DT_GNU_HASH shared objects (for instance shared libraries with
only one option)? It does seem a shortcoming from EAC, although I am not sure
what exactly it is trying to enforce here.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (4 preceding siblings ...)
  2022-08-08 12:43 ` adhemerval.zanella at linaro dot org
@ 2022-08-08 14:34 ` arek at hiler dot eu
  2022-08-08 14:45 ` fweimer at redhat dot com
                   ` (18 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: arek at hiler dot eu @ 2022-08-08 14:34 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

--- Comment #5 from Arkadiusz Hiler <arek at hiler dot eu> ---
(In reply to Florian Weimer from comment #3)
> Distributions configure or patch GCC to pass --hash-style=gnu to the linker
> by default. The linker default does not matter.

Thanks, that explains a few things. My gcc -v indeed claims that:

    Configured with: /build/gcc/src/gcc/configure ...
--with-linker-hash-style=gnu ...

According to gcc/docs/install.texi the default is sysv but after browsing
the code for a bit it looks like if `--with-linker-hash-style` is not
specified no `--hash-style` is passed to the linker so it would depend on
linker's defaults.

Do you know why distros do this? Looking at package history it seems like an
artifact from the times where both was a bit problematic. There used to be
some custom patches for that years ago.


(In reply to Adhemerval Zanella from comment #4)
> > This breaks all the games running via Proton that are using EAC.
> 
> What EAC is doing exactly that requires a DT_HASH on glibc? What happens if
> you mix DT_HASH and DT_GNU_HASH shared objects (for instance shared
> libraries with only one option)? It does seem a shortcoming from EAC,
> although I am not sure what exactly it is trying to enforce here.

I can only guess but probably to look up some symbols in multiple ways to
assure no hooking has occurred.

It seems to require it only for the glibc DSOs as they were shipping with
DT_HASH for years, even on distros that default to gnu, and this is an
unexpected change.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (5 preceding siblings ...)
  2022-08-08 14:34 ` arek at hiler dot eu
@ 2022-08-08 14:45 ` fweimer at redhat dot com
  2022-08-08 14:49 ` adhemerval.zanella at linaro dot org
                   ` (17 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: fweimer at redhat dot com @ 2022-08-08 14:45 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

--- Comment #6 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Arkadiusz Hiler from comment #5)
> (In reply to Florian Weimer from comment #3)
> > Distributions configure or patch GCC to pass --hash-style=gnu to the linker
> > by default. The linker default does not matter.
> 
> Thanks, that explains a few things. My gcc -v indeed claims that:
> 
>     Configured with: /build/gcc/src/gcc/configure ...
> --with-linker-hash-style=gnu ...
> 
> According to gcc/docs/install.texi the default is sysv but after browsing
> the code for a bit it looks like if `--with-linker-hash-style` is not
> specified no `--hash-style` is passed to the linker so it would depend on
> linker's defaults.
> 
> Do you know why distros do this?

sysv hashing is much slower for symbol binding because we have to do a strcmp
even on lookup failure. The Bloom filter is missing, too. Once the glibc
dynamic linker supported it, I think distributions really wanted to use it,
even if binutils and GCC defaults had not caught up at that point yet. It fixes
real issues around startup performance.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (6 preceding siblings ...)
  2022-08-08 14:45 ` fweimer at redhat dot com
@ 2022-08-08 14:49 ` adhemerval.zanella at linaro dot org
  2022-08-08 15:08 ` arek at hiler dot eu
                   ` (16 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-08-08 14:49 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

--- Comment #7 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
(In reply to Arkadiusz Hiler from comment #5)
> (In reply to Florian Weimer from comment #3)
> > Distributions configure or patch GCC to pass --hash-style=gnu to the linker
> > by default. The linker default does not matter.
> 
> Thanks, that explains a few things. My gcc -v indeed claims that:
> 
>     Configured with: /build/gcc/src/gcc/configure ...
> --with-linker-hash-style=gnu ...
> 
> According to gcc/docs/install.texi the default is sysv but after browsing
> the code for a bit it looks like if `--with-linker-hash-style` is not
> specified no `--hash-style` is passed to the linker so it would depend on
> linker's defaults.
> 
> Do you know why distros do this? Looking at package history it seems like an
> artifact from the times where both was a bit problematic. There used to be
> some custom patches for that years ago.
> 
> 
> (In reply to Adhemerval Zanella from comment #4)
> > > This breaks all the games running via Proton that are using EAC.
> > 
> > What EAC is doing exactly that requires a DT_HASH on glibc? What happens if
> > you mix DT_HASH and DT_GNU_HASH shared objects (for instance shared
> > libraries with only one option)? It does seem a shortcoming from EAC,
> > although I am not sure what exactly it is trying to enforce here.
> 
> I can only guess but probably to look up some symbols in multiple ways to
> assure no hooking has occurred.
> 
> It seems to require it only for the glibc DSOs as they were shipping with
> DT_HASH for years, even on distros that default to gnu, and this is an
> unexpected change.

Right, however, I am not sure this characterize as an ABI break since the
symbol lookup information would be indeed provided (albeit in a different
format). And it should be transparent to most applications as well since symbol
resolution is done by the loader itself.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (7 preceding siblings ...)
  2022-08-08 14:49 ` adhemerval.zanella at linaro dot org
@ 2022-08-08 15:08 ` arek at hiler dot eu
  2022-08-08 15:29 ` adhemerval.zanella at linaro dot org
                   ` (15 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: arek at hiler dot eu @ 2022-08-08 15:08 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

--- Comment #8 from Arkadiusz Hiler <arek at hiler dot eu> ---
(In reply to Florian Weimer from comment #6)
> > Do you know why distros do this?
> 
> sysv hashing is much slower for symbol binding because we have to do a
> strcmp even on lookup failure. The Bloom filter is missing, too. Once the
> glibc dynamic linker supported it, I think distributions really wanted to
> use it, even if binutils and GCC defaults had not caught up at that point
> yet. It fixes real issues around startup performance.

I've read through both algorithms and seen the benchmarks / original
discussion around the new hash algorithm. The new one is unquestionably
superior.

I don't understand why people forced "gnu" instead of "both" though.
>From my point of view, as an outsider, it looks like people hopped on the
feature a bit early overriding upstream defaults and it has stuck.

If they would not have carried over `--with-linker-hash-style=gnu` from
that era we would respect linker's default, which is both for The GNU
Linker.

As far as I understand there's small size increase and no real performance
penalty caused by having both.

This is mostly besides the point though and doesn't affect the perceived
breakage, but I appreciate having the historical context.

(In reply to Adhemerval Zanella from comment #7)
> Right, however, I am not sure this characterize as an ABI break since the
> symbol lookup information would be indeed provided (albeit in a different
> format). And it should be transparent to most applications as well since
> symbol resolution is done by the loader itself.

The glibc loader is not the only thing that can do symbol resolution, and
software that ships with distros is not the only software out there.
Changing the section name and the format is significant. All the historical
software that uses old version of dlsym() that's linked in to use pthreads
would break.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (8 preceding siblings ...)
  2022-08-08 15:08 ` arek at hiler dot eu
@ 2022-08-08 15:29 ` adhemerval.zanella at linaro dot org
  2022-08-08 16:49 ` arek at hiler dot eu
                   ` (14 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-08-08 15:29 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

--- Comment #9 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
(In reply to Arkadiusz Hiler from comment #8)
> (In reply to Florian Weimer from comment #6)
> > > Do you know why distros do this?
> > 
> > sysv hashing is much slower for symbol binding because we have to do a
> > strcmp even on lookup failure. The Bloom filter is missing, too. Once the
> > glibc dynamic linker supported it, I think distributions really wanted to
> > use it, even if binutils and GCC defaults had not caught up at that point
> > yet. It fixes real issues around startup performance.
> 
> I've read through both algorithms and seen the benchmarks / original
> discussion around the new hash algorithm. The new one is unquestionably
> superior.
> 
> I don't understand why people forced "gnu" instead of "both" though.
> From my point of view, as an outsider, it looks like people hopped on the
> feature a bit early overriding upstream defaults and it has stuck.
> 
> If they would not have carried over `--with-linker-hash-style=gnu` from
> that era we would respect linker's default, which is both for The GNU
> Linker.
> 
> As far as I understand there's small size increase and no real performance
> penalty caused by having both.
> 
> This is mostly besides the point though and doesn't affect the perceived
> breakage, but I appreciate having the historical context.

It was done as size optimization from perceived unused features since
DT_GNU_HASH is being used as a default on most distros for a long time. 

On libc.so for x86_64, I see:

* with -Wl,--hash-style=both
$ size libc.so
   text    data     bss     dec     hex filename
1992171   20320   55120 2067611  1f8c9b libc.so

* with -Wl,--hash-style=gnu
   text    data     bss     dec     hex filename
1975923   20304   55120 2051347  1f4d13 libc.so

Roughly 1%, which is considerable for an unused feature.

> 
> (In reply to Adhemerval Zanella from comment #7)
> > Right, however, I am not sure this characterize as an ABI break since the
> > symbol lookup information would be indeed provided (albeit in a different
> > format). And it should be transparent to most applications as well since
> > symbol resolution is done by the loader itself.
> 
> The glibc loader is not the only thing that can do symbol resolution, and
> software that ships with distros is not the only software out there.
> Changing the section name and the format is significant. All the historical
> software that uses old version of dlsym() that's linked in to use pthreads
> would break.

But we do not really support loading two different C runtime environments (for
instance having libc.so with different versions in a different namespace), nor
loading libc.so by a different executable (for instance dlsym libc.so from a
different C runtime implementation), and static dlopen is being deprecated
(meaning that eventually, we will phase out its support, besides the already
know issues).

And dlopen libphtread will still work since we continue to provide a stub
libpthread.so.  Are you aware of an issue with the current scheme?

So it does not make much sense to enforce DT_HASH on libc.so, especially if
users do want to do symbol resolution it is still provided by DT_GNU_HASH. The
glibc build now uses the default set by binutils, so maybe one option is to
enforce -Wl,--hash-style=both as distro level if this is really a compatibility
issue (since it might not be specific to glibc shared objects).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (9 preceding siblings ...)
  2022-08-08 15:29 ` adhemerval.zanella at linaro dot org
@ 2022-08-08 16:49 ` arek at hiler dot eu
  2022-08-08 17:13 ` adhemerval.zanella at linaro dot org
                   ` (13 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: arek at hiler dot eu @ 2022-08-08 16:49 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

--- Comment #10 from Arkadiusz Hiler <arek at hiler dot eu> ---
(In reply to Adhemerval Zanella from comment #9)
> It was done as size optimization from perceived unused features since
> DT_GNU_HASH is being used as a default on most distros for a long time. 
> 
> On libc.so for x86_64, I see:
> 
> * with -Wl,--hash-style=both
> $ size libc.so
>    text    data     bss     dec     hex filename
> 1992171   20320   55120 2067611  1f8c9b libc.so
> 
> * with -Wl,--hash-style=gnu
>    text    data     bss     dec     hex filename
> 1975923   20304   55120 2051347  1f4d13 libc.so
> 
> Roughly 1%, which is considerable for an unused feature.

~16kB that's used by a whole class of games that are now unplayable on more
bleeding-edge distros.

See: https://github.com/ValveSoftware/Proton/issues/6051

> So it does not make much sense to enforce DT_HASH on libc.so, especially if
> users do want to do symbol resolution it is still provided by DT_GNU_HASH.
> The glibc build now uses the default set by binutils, so maybe one option is
> to enforce -Wl,--hash-style=both as distro level if this is really a
> compatibility issue (since it might not be specific to glibc shared objects).

Shifting responsibility on downstream to maintain compatibility with something
that glibc provided for years as the default and there are now users of it
is not really feasible.

I don't think I can convince you. I really hoped it can be just a simple
revert.
The best I can do is to report that a thing used to work is now broken and
point to a commit that caused it.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (10 preceding siblings ...)
  2022-08-08 16:49 ` arek at hiler dot eu
@ 2022-08-08 17:13 ` adhemerval.zanella at linaro dot org
  2022-08-08 17:32 ` adhemerval.zanella at linaro dot org
                   ` (12 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-08-08 17:13 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

--- Comment #11 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
(In reply to Arkadiusz Hiler from comment #10)
> (In reply to Adhemerval Zanella from comment #9)
> > It was done as size optimization from perceived unused features since
> > DT_GNU_HASH is being used as a default on most distros for a long time. 
> > 
> > On libc.so for x86_64, I see:
> > 
> > * with -Wl,--hash-style=both
> > $ size libc.so
> >    text    data     bss     dec     hex filename
> > 1992171   20320   55120 2067611  1f8c9b libc.so
> > 
> > * with -Wl,--hash-style=gnu
> >    text    data     bss     dec     hex filename
> > 1975923   20304   55120 2051347  1f4d13 libc.so
> > 
> > Roughly 1%, which is considerable for an unused feature.
> 
> ~16kB that's used by a whole class of games that are now unplayable on more
> bleeding-edge distros.
> 
> See: https://github.com/ValveSoftware/Proton/issues/6051
> 
> > So it does not make much sense to enforce DT_HASH on libc.so, especially if
> > users do want to do symbol resolution it is still provided by DT_GNU_HASH.
> > The glibc build now uses the default set by binutils, so maybe one option is
> > to enforce -Wl,--hash-style=both as distro level if this is really a
> > compatibility issue (since it might not be specific to glibc shared objects).
> 
> Shifting responsibility on downstream to maintain compatibility with
> something
> that glibc provided for years as the default and there are now users of it
> is not really feasible.
> 
> I don't think I can convince you. I really hoped it can be just a simple
> revert.
> The best I can do is to report that a thing used to work is now broken and
> point to a commit that caused it.

In fact, we take backward compatibility quite seriously, the main issue here is
what characterizes an ABI break and what kind of forward compatibility and
extra internal details changes we need to take care of.

I am not against revert this change, but it really odd that on all system
binaries we need to keep the sysv hash solely for glibc add eternum because a
specific tool where we do not really understand exactly why requires such
functionality, and it does not provide any extra function to glibc itself.

Does it also prevent us to add another possible HASH scheme in the future,
taking that EAC might break if it does see a DT_GNU_HASH?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (11 preceding siblings ...)
  2022-08-08 17:13 ` adhemerval.zanella at linaro dot org
@ 2022-08-08 17:32 ` adhemerval.zanella at linaro dot org
  2022-08-08 19:58 ` sam at gentoo dot org
                   ` (11 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-08-08 17:32 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

--- Comment #12 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
(In reply to Arkadiusz Hiler from comment #10)
> 
> Shifting responsibility on downstream to maintain compatibility with
> something
> that glibc provided for years as the default and there are now users of it
> is not really feasible.
> 

I send a message on libc-alpha to obtain some opinions from other maintainers
and developers [1].

[1] https://sourceware.org/pipermail/libc-alpha/2022-August/141302.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (12 preceding siblings ...)
  2022-08-08 17:32 ` adhemerval.zanella at linaro dot org
@ 2022-08-08 19:58 ` sam at gentoo dot org
  2022-08-08 19:58 ` sam at gentoo dot org
                   ` (10 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: sam at gentoo dot org @ 2022-08-08 19:58 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sam at gentoo dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (13 preceding siblings ...)
  2022-08-08 19:58 ` sam at gentoo dot org
@ 2022-08-08 19:58 ` sam at gentoo dot org
  2022-08-09 16:12 ` freswa at archlinux dot org
                   ` (9 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: sam at gentoo dot org @ 2022-08-08 19:58 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |toolchain at gentoo dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (14 preceding siblings ...)
  2022-08-08 19:58 ` sam at gentoo dot org
@ 2022-08-09 16:12 ` freswa at archlinux dot org
  2022-08-12  7:46 ` arek at hiler dot eu
                   ` (8 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: freswa at archlinux dot org @ 2022-08-09 16:12 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

freswa <freswa at archlinux dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |freswa at archlinux dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (15 preceding siblings ...)
  2022-08-09 16:12 ` freswa at archlinux dot org
@ 2022-08-12  7:46 ` arek at hiler dot eu
  2022-08-12 13:17 ` adhemerval.zanella at linaro dot org
                   ` (7 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: arek at hiler dot eu @ 2022-08-12  7:46 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

--- Comment #13 from Arkadiusz Hiler <arek at hiler dot eu> ---
> Does it also prevent us to add another possible HASH scheme in the future, taking that EAC might break if it does see a DT_GNU_HASH?

The breakage here is not related to having DT_GNU_HASH, it's related to missing
DT_HASH. I don't see how this is relevant.



There were two more breakages caused by dropping DT_HASH:

A native game - Shovel Knight:
https://github.com/ValveSoftware/Proton/issues/6051#issuecomment-1212748397

Framerate limiter for OpenGL - libstrangle:
https://bugs.gentoo.org/863863

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (16 preceding siblings ...)
  2022-08-12  7:46 ` arek at hiler dot eu
@ 2022-08-12 13:17 ` adhemerval.zanella at linaro dot org
  2022-08-14 21:33 ` hi-angel at yandex dot ru
                   ` (6 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-08-12 13:17 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

--- Comment #14 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
(In reply to Arkadiusz Hiler from comment #13)
> > Does it also prevent us to add another possible HASH scheme in the future, taking that EAC might break if it does see a DT_GNU_HASH?
> 
> The breakage here is not related to having DT_GNU_HASH, it's related to
> missing DT_HASH. I don't see how this is relevant.
> 
> 
> 
> There were two more breakages caused by dropping DT_HASH:
> 
> A native game - Shovel Knight:
> https://github.com/ValveSoftware/Proton/issues/6051#issuecomment-1212748397
> 
> Framerate limiter for OpenGL - libstrangle:
> https://bugs.gentoo.org/863863

It is relevant because we are discussing if it is reasonable to GNU ABI ELF
extension to keep DT_HASH mandatory.  From generic ABI discussion [1], it seems
that other mantainers seem reasonable to make DT_HASH optional if DT_GNU_HASH
is present on GNU objects, although some are not confortable on making it
optional for generic ABI (which is not the case anyway).

As Carlos has summarized [2], DT_GNU_HASH is the *de facto* standard hash
scheme for GNU and it has been used on distro deployments for over 16 years.
The generic ABI discussion has raises some missing spots where DT_GNU_HASH does
not cover (the total symbol list size) which generates another gABI discussion
to add a new dynamic section type [3].

>  Shifting responsibility on downstream to maintain compatibility with something
> that glibc provided for years as the default and there are now users of it
> is not really feasible.

And it is already being done anyway [5], since for such cases compatibility is
being done to revert upstream patches anyway instead of work on vendors to
adapt their code to a decade-old standard. 

So I tend to agree that if DT_HASH compatibility is really required, it would
be better to provide it as the default static linker option used to build glibc
(so you also ensure that all installed shared object does have it) as Gentoo is
doing [4].

[1] https://groups.google.com/g/generic-abi/c/th5919osPAQ
[2] https://sourceware.org/pipermail/libc-alpha/2022-August/141304.html
[3] https://groups.google.com/g/generic-abi/c/9L03yrxXPBc
[4] https://sourceware.org/pipermail/libc-alpha/2022-August/141312.html
[5] https://sourceware.org/pipermail/libc-alpha/2022-August/141313.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (17 preceding siblings ...)
  2022-08-12 13:17 ` adhemerval.zanella at linaro dot org
@ 2022-08-14 21:33 ` hi-angel at yandex dot ru
  2022-08-24  4:12 ` i at maskray dot me
                   ` (5 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: hi-angel at yandex dot ru @ 2022-08-14 21:33 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

Hi-Angel <hi-angel at yandex dot ru> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hi-angel at yandex dot ru

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (18 preceding siblings ...)
  2022-08-14 21:33 ` hi-angel at yandex dot ru
@ 2022-08-24  4:12 ` i at maskray dot me
  2022-08-24  8:28 ` rguenth at gcc dot gnu.org
                   ` (4 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: i at maskray dot me @ 2022-08-24  4:12 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

Fangrui Song <i at maskray dot me> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |i at maskray dot me

--- Comment #18 from Fangrui Song <i at maskray dot me> ---
(In reply to John Brooks from comment #17)
> I think there needs to be a much higher bar for keeping changes that break real applications, especially applications that have little or no chance of getting fixed. This is part of maintaining a stable operating system. When something as integral as glibc decides to keep breaking changes, it damages GNU/Linux as a platform. 

There was lots of misinformation about the glibc change.
I created https://maskray.me/blog/2022-08-21-glibc-and-dt-gnu-hash 

See my comment on "Easy Anti-Cheat" and the new finding about Arch Linux
package.

I think glibc should close this issue now.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (19 preceding siblings ...)
  2022-08-24  4:12 ` i at maskray dot me
@ 2022-08-24  8:28 ` rguenth at gcc dot gnu.org
  2022-08-24  8:56 ` fweimer at redhat dot com
                   ` (3 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: rguenth at gcc dot gnu.org @ 2022-08-24  8:28 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |rguenth at gcc dot gnu.org

--- Comment #19 from Richard Biener <rguenth at gcc dot gnu.org> ---
SUSE uses the linker default from GCC and configures binutils explicitely to
default to 'both'.  If glibc doesn't honor this in it's default behavior I'd
declare it broken (why should it use an explicit --hash-style not honoring the
systems default?)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (20 preceding siblings ...)
  2022-08-24  8:28 ` rguenth at gcc dot gnu.org
@ 2022-08-24  8:56 ` fweimer at redhat dot com
  2022-08-24  8:57 ` mliska at suse dot cz
                   ` (2 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: fweimer at redhat dot com @ 2022-08-24  8:56 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

--- Comment #20 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Richard Biener from comment #19)
> SUSE uses the linker default from GCC and configures binutils explicitely to
> default to 'both'.  If glibc doesn't honor this in it's default behavior I'd
> declare it broken (why should it use an explicit --hash-style not honoring
> the
> systems default?)

Current glibc sources do not override the toolchain default. Older versions
overrode it with --hash-style=both.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (21 preceding siblings ...)
  2022-08-24  8:56 ` fweimer at redhat dot com
@ 2022-08-24  8:57 ` mliska at suse dot cz
  2023-08-11 15:40 ` es20490446e at gmail dot com
  2023-08-11 15:58 ` carlos at redhat dot com
  24 siblings, 0 replies; 26+ messages in thread
From: mliska at suse dot cz @ 2022-08-24  8:57 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

Martin Liska <mliska at suse dot cz> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mliska at suse dot cz

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (22 preceding siblings ...)
  2022-08-24  8:57 ` mliska at suse dot cz
@ 2023-08-11 15:40 ` es20490446e at gmail dot com
  2023-08-11 15:58 ` carlos at redhat dot com
  24 siblings, 0 replies; 26+ messages in thread
From: es20490446e at gmail dot com @ 2023-08-11 15:40 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

Alberto Salvia Novella <es20490446e at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |es20490446e at gmail dot com

--- Comment #21 from Alberto Salvia Novella <es20490446e at gmail dot com> ---
You are overlooking the main point boyos.

People MUST be able to tell, in advance, that the ABI will be missing a
previous component.

The simple way to do that is just to increment the soname. If you can't do
that, you MUST not reduce the ABI.

You cannot rely on a component just not being used anymore, announcing the
change in a blog, or even documenting it.

The change needs to be explicit while compiling. Any other thing is a leap of
faith into the void.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug build/29456] missing DT_HASH section in shared objects
  2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
                   ` (23 preceding siblings ...)
  2023-08-11 15:40 ` es20490446e at gmail dot com
@ 2023-08-11 15:58 ` carlos at redhat dot com
  24 siblings, 0 replies; 26+ messages in thread
From: carlos at redhat dot com @ 2023-08-11 15:58 UTC (permalink / raw)
  To: glibc-bugs

https://sourceware.org/bugzilla/show_bug.cgi?id=29456

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |WONTFIX
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #22 from Carlos O'Donell <carlos at redhat dot com> ---
I'm marking this issue as RESOLVED WONTFIX, because the glibc build will
include both hashes if that's how the distribution builds the ELF . It is up to
the distributions to decide how to proceed with the availability of the hashes.
There is no intent to fix this in glibc by forcing both hashes.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2023-08-11 15:58 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-08  9:10 [Bug build/29456] New: missing DT_HASH section in shared objects arek at hiler dot eu
2022-08-08  9:39 ` [Bug build/29456] " mikhail.v.gavrilov at gmail dot com
2022-08-08 10:45 ` fweimer at redhat dot com
2022-08-08 11:33 ` arek at hiler dot eu
2022-08-08 12:01 ` fweimer at redhat dot com
2022-08-08 12:43 ` adhemerval.zanella at linaro dot org
2022-08-08 14:34 ` arek at hiler dot eu
2022-08-08 14:45 ` fweimer at redhat dot com
2022-08-08 14:49 ` adhemerval.zanella at linaro dot org
2022-08-08 15:08 ` arek at hiler dot eu
2022-08-08 15:29 ` adhemerval.zanella at linaro dot org
2022-08-08 16:49 ` arek at hiler dot eu
2022-08-08 17:13 ` adhemerval.zanella at linaro dot org
2022-08-08 17:32 ` adhemerval.zanella at linaro dot org
2022-08-08 19:58 ` sam at gentoo dot org
2022-08-08 19:58 ` sam at gentoo dot org
2022-08-09 16:12 ` freswa at archlinux dot org
2022-08-12  7:46 ` arek at hiler dot eu
2022-08-12 13:17 ` adhemerval.zanella at linaro dot org
2022-08-14 21:33 ` hi-angel at yandex dot ru
2022-08-24  4:12 ` i at maskray dot me
2022-08-24  8:28 ` rguenth at gcc dot gnu.org
2022-08-24  8:56 ` fweimer at redhat dot com
2022-08-24  8:57 ` mliska at suse dot cz
2023-08-11 15:40 ` es20490446e at gmail dot com
2023-08-11 15:58 ` carlos at redhat dot com

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).