Re: Should we make DT_HASH dynamic section for glibc?

[Fangrui Song <maskray@google.com>]

On 2022-08-08, Carlos O'Donell via Libc-alpha wrote:
>On 8/8/22 13:31, Adhemerval Zanella Netto via Libc-alpha wrote:
>> It seems that the recent change to remove the multiple hash schemes on
>> 2.36 [1] broke some specific tools used on proton games [2].  So instead
>> of explicit set the section type to use both sysv and gnu, we use the
>> toolchain default which might exclude the sysv type.
>
>Right.
>
>> The last gABI states that DT_HASH is mandatory [3], but DT_GNU_HASH works
>> a direct replacement meaning that it contains all information for symbol
>> resolution that DT_HASH provides.
>
>Correct.
>
>> It was done as size optimization from perceived unused features since
>> DT_GNU_HASH is being used as a default on most distros for a long time,
>> meaning DT_HASH might not be set.  For instance, on a Ubuntu 22.05 system
>> (GLIBC 2.35) only the glibc provided binaries (pldd, gencat, etc.) and some
>> external tools (nvidia command line) do provide DT_HASH.
>
>Correct.
>
>> So the question is whether we should enforce at least on glibc by reverting
>> e47de5cb2d4dbec.  It does sounds muddle to keep this solely on glibc, where
>> rest of the system is not enforcing it, and only for compatibility with an
>> obscure tools where there is no much information on why it requires it.
>
>The tool is EAC.
>
>It is EPICs "Easy Anti-Cheat" (https://www.easy.ac/en-us/) and like other non-standard
>consumers it has to know some specific ELF details.
>
>The game "Rogue Company" is known to use EAC and is likely broken by this change.
>
>The Nobara Project includes "changes" to make EAC-enabled Steam games work:
>https://nobaraproject.org/
>
>I have reviewed the changes that Nobara is carrying and I would not apply them upstream.
>
>The include such changes as reverting the clone3 addition because it impacts seccomp-based
>isolation.
>
>> Another option might to extend gABI to state that if DT_GNU_HASH is presents,
>> it works as DT_HASH and it should not mark as mandated.
>
>We should ask the gABI to mark DT_HASH as optional.
>
>This doesn't resolve the user issue though...
>
>> And if DT_HASH is require required, one possibility is to add a binutils
>> option to emit an empty DT_HASH just for compatibility and get the code size
>> improvement.
>
>The chain array needs to be as big as the symbol table since the presence
>of DT_HASH means it can be indexed into, therefore I don't think we can
>have an empty DT_HASH.
>
>> [1] https://sourceware.org/git/?p=glibc.git;a=commit;h=e47de5cb2d4dbecb58f569ed241e8e95c568f03c
>> [2] https://sourceware.org/bugzilla/show_bug.cgi?id=29456
>> [3] http://www.sco.com/developers/gabi/latest/ch5.dynamic.html
>
>Using only DT_GNU_HASH is a choice we *always* wanted to allow the
>downstream distributions to make, it was part of the binutils changes to
>allow just DT_GNU_HASH.
>
>Software that is an ELF consumer on Linux has had 16 years to be updated
>to handle the switch from DT_HASH to DT_GNU_HASH (OS-specific).
>
>While I'm sympathetic to application developers and their backwards
>compatibility requirements, this specific case is about an ELF consumers
>and such a consumer needs to track upstream Linux ELF developments.
>
>We aren't breaking ABI when we remove the PLT, remove the old HASH, or
>other Linux ELF changes (like the recent DT_RELR addition), but we do
>need to allow time for these changes to be absorbed by the ecosystem
>and ELF consumers (like debug information consumers).
>
>At present I would not make any changes to glibc. I would close bug 29456
>as RESOLVED/WONTFIX. I'm open to hearing from the EPIC EAC developers
>if they have a case to make about DT_HASH.
>
>In summary:
>
>- DT_GNU_HASH was added in 2006, and for the last 16 years has been the
>  modern standard on Linux. The glibc change was made to allow the
>  distributions to choose how backwards compatible they want to be with
>  ELF consumers and the hash function and section. This is not ABI, just
>  like the PLT and RELRO are not ABI.
>
>- One specific use case of "Easy Anti-Cheat" software is impacted by
>  this implementation detail change which impacts ELF consumers that
>  require DT_HASH.
>
>- The choice to have DT_HASH is with the distributions. If this breaks
>  specific applications then those developers need to engage with the
>  ecosystem or adapt their software.

Thanks for analyzing the issue!  On
https://github.com/ValveSoftware/Proton/issues/6051
nobody says this is related to DT_HASH :(

If "Easy Anti-Cheat" requires DT_HASH, I think it is a unreasonable
request.  I left a comment on
https://github.com/ValveSoftware/Proton/issues/6051#issuecomment-1208698263

Many distributions have dropped DT_HASH (e.g.
see https://github.com/llvm/llvm-project/blob/b405407/clang/lib/Driver/ToolChains/Linux.cpp#L241-L244
for the encoded Clang knowledge), it is entirely fine for glibc to drop
DT_HASH for its own shared objects, even if ELFOSABI_NONE is used.
Perhaps, as Roland suggested, clarifying the optional DT_HASH thing on gnu-gabi@ may help.