Are the pthread "compatibility" copies of symbols in libc still necessary?

Are the pthread "compatibility" copies of symbols in libc still necessary?

[Zack Weinberg]

A number of source files that properly belong to libc.so are also
compiled as part of libpthread, with a note that this is for
"compatibility for old binaries".  The exact set varies based on
architecture, but includes basic things like read, write, and fork - I
_think_ there was a difference in semantics in the distant past,
having something to do with thread cancellation.

It seems to me that these are no longer necessary.  The overall
behavior we want is for new binaries to link against the symbols in
libc.so, and old binaries that were linked against a libpthread with
those symbols to continue to work.  ELF doesn't require undefined
references to resolve to specific libraries at load time, so if we
just dropped the symbols from libpthread, shouldn't the references
from old binaries automatically resolve to the definitions in libc,
which is what we want?  Am I missing something?

zw

Re: Are the pthread "compatibility" copies of symbols in libc still necessary?

[Florian Weimer]

On 03/16/2018 03:30 PM, Zack Weinberg wrote:
> A number of source files that properly belong to libc.so are also
> compiled as part of libpthread, with a note that this is for
> "compatibility for old binaries".  The exact set varies based on
> architecture, but includes basic things like read, write, and fork - I
> _think_ there was a difference in semantics in the distant past,
> having something to do with thread cancellation.

They are still needed because versioned symbols also embed a DSO name, 
and the dynamic linker checks that if it has not been interposed.

I think it's this code in elf/dl-lookup.c:

       if (__glibc_unlikely (res < 0) && skip_map == NULL)
	{
	  /* Oh, oh.  The file named in the relocation entry does not
	     contain the needed symbol.  This code is never reached
	     for unversioned lookups.  */
	  assert (version != NULL);
	  const char *reference_name = undef_map ? undef_map->l_name : "";
	  struct dl_exception exception;
	  /* XXX We cannot translate the message.  */
	  _dl_exception_create_format
	    (&exception, DSO_FILENAME (reference_name),
	     "symbol %s version %s not defined in file %s"
	     " with link time reference%s",
	     undef_name, version->name, version->filename,
	     res == -2 ? " (no version symbols)" : "");
	  _dl_signal_cexception (0, &exception, N_("relocation error"));
	  _dl_exception_free (&exception);
	  *ref = NULL;
	  return 0;
	}

This is required by the GNU symbol versioning spec, but I don't know why.

Even new binaries use these symbols:

$ LD_DEBUG=bindings systemctl |& grep binding.*system.*libpthread.*[^_]fork
      11905:	binding file /usr/lib/systemd/libsystemd-shared-234.so [0] 
to /lib64/libpthread.so.0 [0]: normal symbol `fork' [GLIBC_2.2.5]
      11905:	binding file systemctl [0] to /lib64/libpthread.so.0 [0]: 
normal symbol `fork' [GLIBC_2.2.5]

This may be a linker bug because at least fork is a compat symbol.

Thanks,
Florian

Re: Are the pthread "compatibility" copies of symbols in libc still necessary?

[Carlos O'Donell]

On 03/16/2018 08:41 AM, Florian Weimer wrote:
> On 03/16/2018 03:30 PM, Zack Weinberg wrote:
>> A number of source files that properly belong to libc.so are also
>> compiled as part of libpthread, with a note that this is for
>> "compatibility for old binaries".  The exact set varies based on
>> architecture, but includes basic things like read, write, and fork - I
>> _think_ there was a difference in semantics in the distant past,
>> having something to do with thread cancellation.
> 
> They are still needed because versioned symbols also embed a DSO name, and the dynamic linker checks that if it has not been interposed.
> 
> I think it's this code in elf/dl-lookup.c:
> 
>       if (__glibc_unlikely (res < 0) && skip_map == NULL)
>     {
>       /* Oh, oh.  The file named in the relocation entry does not
>          contain the needed symbol.  This code is never reached
>          for unversioned lookups.  */
>       assert (version != NULL);
>       const char *reference_name = undef_map ? undef_map->l_name : "";
>       struct dl_exception exception;
>       /* XXX We cannot translate the message.  */
>       _dl_exception_create_format
>         (&exception, DSO_FILENAME (reference_name),
>          "symbol %s version %s not defined in file %s"
>          " with link time reference%s",
>          undef_name, version->name, version->filename,
>          res == -2 ? " (no version symbols)" : "");
>       _dl_signal_cexception (0, &exception, N_("relocation error"));
>       _dl_exception_free (&exception);
>       *ref = NULL;
>       return 0;
>     }
> 
> This is required by the GNU symbol versioning spec, but I don't know why.

The specification says the data must be recorded, but it doesn't say what you
have to *do* with the data?

I am of the opinion that this is simply to produce a reasonable error message
if the versioned symbol is missing completely.

We should probably reach out to Ulrich to see if has any opinion on this,
as the primary author of the specification he might have an input here.

My own opinion is that the check is overly restrictive, and that we could relax
it to allow versioned symbols to move to other shared objects.

> Even new binaries use these symbols:
> 
> $ LD_DEBUG=bindings systemctl |& grep binding.*system.*libpthread.*[^_]fork
>      11905:    binding file /usr/lib/systemd/libsystemd-shared-234.so [0] to /lib64/libpthread.so.0 [0]: normal symbol `fork' [GLIBC_2.2.5]
>      11905:    binding file systemctl [0] to /lib64/libpthread.so.0 [0]: normal symbol `fork' [GLIBC_2.2.5]
> 
> This may be a linker bug because at least fork is a compat symbol.

Is it though? libc.so.6 has fork@@GLIBC_2.2.5, which causes the binary to have
a reference to fork@GLIBC_2.2.5, and that's correct. However, now we have both
libc.so.6 and libpthread.so.0 with definitions of fork at GLIBC_2.2.5.

Also libsystemd-shared-234.so has:

000000000043f580  0000019a00000007 R_X86_64_JUMP_SLOT     0000000000000000 fork@GLIBC_2.2.5 + 0
   410: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND fork@GLIBC_2.2.5 (2)

  0x0170: Version: 1  File: libc.so.6  Cnt: 16
...
  0x0270:   Name: GLIBC_2.2.5  Flags: none  Version: 2

Is it a dynamic loader bug that the compat symbol was selected?

systemctl has libpthread.so.0 as the *first* DT_NEEDED entry, and we process them in order.

 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libsystemd-shared-234.so]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]

So libpthread.so.0 interposes the versioned symbol from libc.so.6.

     26067:     Initial object scopes
     26067:     object=systemctl [0]
     26067:      scope 0: systemctl /lib64/libpthread.so.0 /lib64/libc.so.6 /usr/lib/systemd/libsystemd-shared-234.so /lib64/libgcc_s.so.1 /lib64/ld-linux-x86-64.so.2 /lib64/librt.so.1 /lib64/libcap.so.2 /lib64/libacl.so.1 /lib64/libcryptsetup.so.4 /lib64/libgcrypt.so.20 /lib64/libip4tc.so.0 /lib64/libseccomp.so.2 /lib64/libselinux.so.1 /lib64/libidn.so.11 /lib64/liblzma.so.5 /lib64/liblz4.so.1 /lib64/libblkid.so.1 /lib64/libattr.so.1 /lib64/libuuid.so.1 /lib64/libdevmapper.so.1.02 /lib64/libdl.so.2 /lib64/libgpg-error.so.0 /lib64/libpcap.so.1 /lib64/libpcre2-8.so.0 /lib64/libsepol.so.1 /lib64/libudev.so.1 /lib64/libm.so.6
     26067:

So the search scope is libpthread *first*.

The above error never triggers because libc.so.6 *does* have the named versioned symbol,
but libpthread.so.0 also has it too.

My conclusion is this:

* We can remove fork@GLIBC_2.2.5 from libpthread.so, the shared object doesn't
  encode it as being needed and won't cause a failure per the rules that
  require the referenced shared object to have the versioned symbol that was
  bound at static link time.

Cheers,
Carlos.

Re: Are the pthread "compatibility" copies of symbols in libc still necessary?

[Florian Weimer]

On 03/16/2018 04:35 PM, Carlos O'Donell wrote:
> On 03/16/2018 08:41 AM, Florian Weimer wrote:
>> On 03/16/2018 03:30 PM, Zack Weinberg wrote:
>>> A number of source files that properly belong to libc.so are also
>>> compiled as part of libpthread, with a note that this is for
>>> "compatibility for old binaries".  The exact set varies based on
>>> architecture, but includes basic things like read, write, and fork - I
>>> _think_ there was a difference in semantics in the distant past,
>>> having something to do with thread cancellation.
>>
>> They are still needed because versioned symbols also embed a DSO name, and the dynamic linker checks that if it has not been interposed.
>>
>> I think it's this code in elf/dl-lookup.c:
>>
>>        if (__glibc_unlikely (res < 0) && skip_map == NULL)
>>      {
>>        /* Oh, oh.  The file named in the relocation entry does not
>>           contain the needed symbol.  This code is never reached
>>           for unversioned lookups.  */
>>        assert (version != NULL);
>>        const char *reference_name = undef_map ? undef_map->l_name : "";
>>        struct dl_exception exception;
>>        /* XXX We cannot translate the message.  */
>>        _dl_exception_create_format
>>          (&exception, DSO_FILENAME (reference_name),
>>           "symbol %s version %s not defined in file %s"
>>           " with link time reference%s",
>>           undef_name, version->name, version->filename,
>>           res == -2 ? " (no version symbols)" : "");
>>        _dl_signal_cexception (0, &exception, N_("relocation error"));
>>        _dl_exception_free (&exception);
>>        *ref = NULL;
>>        return 0;
>>      }
>>
>> This is required by the GNU symbol versioning spec, but I don't know why.
> 
> The specification says the data must be recorded, but it doesn't say what you
> have to *do* with the data?

It does:

“
A fatal error shall be triggered when no matching definition can be 
found in the object whose name is the one referenced by the vn_name 
element in the Elfxx_Verneed entry.
”

>> Even new binaries use these symbols:
>>
>> $ LD_DEBUG=bindings systemctl |& grep binding.*system.*libpthread.*[^_]fork
>>       11905:    binding file /usr/lib/systemd/libsystemd-shared-234.so [0] to /lib64/libpthread.so.0 [0]: normal symbol `fork' [GLIBC_2.2.5]
>>       11905:    binding file systemctl [0] to /lib64/libpthread.so.0 [0]: normal symbol `fork' [GLIBC_2.2.5]
>>
>> This may be a linker bug because at least fork is a compat symbol.

> Is it a dynamic loader bug that the compat symbol was selected?

No, I don't think the dynamic linker has to make this distinction, at 
least not if the soname matches the one in the version.

> My conclusion is this:
> 
> * We can remove fork@GLIBC_2.2.5 from libpthread.so, the shared object doesn't
>    encode it as being needed and won't cause a failure per the rules that
>    require the referenced shared object to have the versioned symbol that was
>    bound at static link time.

I already tried a while bug and failed because the system does not boot 
anymore after that, due to the check mentioned above.  I don't think the 
code has changed since then.

Thanks,
Florian

Re: Are the pthread "compatibility" copies of symbols in libc still necessary?

[Carlos O'Donell]

On 03/16/2018 09:42 AM, Florian Weimer wrote:
> On 03/16/2018 04:35 PM, Carlos O'Donell wrote:
>> On 03/16/2018 08:41 AM, Florian Weimer wrote:
>>> On 03/16/2018 03:30 PM, Zack Weinberg wrote:
>>>> A number of source files that properly belong to libc.so are also
>>>> compiled as part of libpthread, with a note that this is for
>>>> "compatibility for old binaries".  The exact set varies based on
>>>> architecture, but includes basic things like read, write, and fork - I
>>>> _think_ there was a difference in semantics in the distant past,
>>>> having something to do with thread cancellation.
>>>
>>> They are still needed because versioned symbols also embed a DSO name, and the dynamic linker checks that if it has not been interposed.
>>>
>>> I think it's this code in elf/dl-lookup.c:
>>>
>>>        if (__glibc_unlikely (res < 0) && skip_map == NULL)
>>>      {
>>>        /* Oh, oh.  The file named in the relocation entry does not
>>>           contain the needed symbol.  This code is never reached
>>>           for unversioned lookups.  */
>>>        assert (version != NULL);
>>>        const char *reference_name = undef_map ? undef_map->l_name : "";
>>>        struct dl_exception exception;
>>>        /* XXX We cannot translate the message.  */
>>>        _dl_exception_create_format
>>>          (&exception, DSO_FILENAME (reference_name),
>>>           "symbol %s version %s not defined in file %s"
>>>           " with link time reference%s",
>>>           undef_name, version->name, version->filename,
>>>           res == -2 ? " (no version symbols)" : "");
>>>        _dl_signal_cexception (0, &exception, N_("relocation error"));
>>>        _dl_exception_free (&exception);
>>>        *ref = NULL;
>>>        return 0;
>>>      }
>>>
>>> This is required by the GNU symbol versioning spec, but I don't know why.
>>
>> The specification says the data must be recorded, but it doesn't say what you
>> have to *do* with the data?
> 
> It does:
> 
> “
> A fatal error shall be triggered when no matching definition can be
> found in the object whose name is the one referenced by the vn_name
> element in the Elfxx_Verneed entry.
> ”

I don't know what drove this requirement.

From a first-principles perspective is doesn't seem to derive from any
foundational aspect of the linkage model.

Can you rationalize the requirement yourself?

Am I missing something?

>>> Even new binaries use these symbols:
>>>
>>> $ LD_DEBUG=bindings systemctl |& grep binding.*system.*libpthread.*[^_]fork
>>>       11905:    binding file /usr/lib/systemd/libsystemd-shared-234.so [0] to /lib64/libpthread.so.0 [0]: normal symbol `fork' [GLIBC_2.2.5]
>>>       11905:    binding file systemctl [0] to /lib64/libpthread.so.0 [0]: normal symbol `fork' [GLIBC_2.2.5]
>>>
>>> This may be a linker bug because at least fork is a compat symbol.
> 
>> Is it a dynamic loader bug that the compat symbol was selected?
> 
> No, I don't think the dynamic linker has to make this distinction, at
> least not if the soname matches the one in the version.

I think the problem is that the two checks are orthogonal.

You have one check to find the versioned symbol based on lookup scopes.

You have another check to determine if a DSO with the appropriate symbol and version exists.

Both of these are true in the fork case.

>> My conclusion is this:
>>
>> * We can remove fork@GLIBC_2.2.5 from libpthread.so, the shared object doesn't
>>    encode it as being needed and won't cause a failure per the rules that
>>    require the referenced shared object to have the versioned symbol that was
>>    bound at static link time.
> 
> I already tried a while bug and failed because the system does not
> boot anymore after that, due to the check mentioned above.  I don't
> think the code has changed since then.

Do you know what failed? I should try this.

Cheers,
Carlos.

Re: Are the pthread "compatibility" copies of symbols in libc still necessary?

[Florian Weimer]

On 03/16/2018 06:38 PM, Carlos O'Donell wrote:

>> It does:
>>
>> “
>> A fatal error shall be triggered when no matching definition can be
>> found in the object whose name is the one referenced by the vn_name
>> element in the Elfxx_Verneed entry.
>> ”
> 

>>> My conclusion is this:
>>>
>>> * We can remove fork@GLIBC_2.2.5 from libpthread.so, the shared object doesn't
>>>     encode it as being needed and won't cause a failure per the rules that
>>>     require the referenced shared object to have the versioned symbol that was
>>>     bound at static link time.
>>
>> I already tried a while bug and failed because the system does not
>> boot anymore after that, due to the check mentioned above.  I don't
>> think the code has changed since then.
> 
> Do you know what failed? I should try this.

I got the fatal error required by the specification.

Florian

Re: Are the pthread "compatibility" copies of symbols in libc still necessary?

[Zack Weinberg]

On Fri, Mar 16, 2018 at 1:38 PM, Carlos O'Donell <carlos@redhat.com> wrote:
> On 03/16/2018 09:42 AM, Florian Weimer wrote:
>> On 03/16/2018 04:35 PM, Carlos O'Donell wrote:
>>>
>>> The specification says the data must be recorded, but it doesn't say what you
>>> have to *do* with the data?
>>
>> It does:
>>
>> “
>> A fatal error shall be triggered when no matching definition can be
>> found in the object whose name is the one referenced by the vn_name
>> element in the Elfxx_Verneed entry.
>> ”
>
> I don't know what drove this requirement.
>
> From a first-principles perspective is doesn't seem to derive from any
> foundational aspect of the linkage model.

I don't claim to know what Ulrich was thinking at the time, but if you
start from the premise that it would be more consistent in general if
symbols always resolved at load time to the library they were found in
at link time, then adding this requirement for versioned symbols can
be seen as a way to introduce that consistency in a
backward-compatible fashion.  That said, without any way to declare
that symbol X is _supposed_ to have moved from foo.so to bar.so, it
puts us in a hole.

This is a GNU spec, so we _could_ change it.  We'd need to do a bit of
formal consultation with everyone else who might be affected, but I
don't think our hands should be tied by a decision that nobody
remembers the original rationale for anymore.

zw

Re: Are the pthread "compatibility" copies of symbols in libc still necessary?

[Carlos O'Donell]

On 03/16/2018 12:13 PM, Zack Weinberg wrote:
> On Fri, Mar 16, 2018 at 1:38 PM, Carlos O'Donell <carlos@redhat.com> wrote:
>> On 03/16/2018 09:42 AM, Florian Weimer wrote:
>>> On 03/16/2018 04:35 PM, Carlos O'Donell wrote:
>>>>
>>>> The specification says the data must be recorded, but it doesn't say what you
>>>> have to *do* with the data?
>>>
>>> It does:
>>>
>>> “
>>> A fatal error shall be triggered when no matching definition can be
>>> found in the object whose name is the one referenced by the vn_name
>>> element in the Elfxx_Verneed entry.
>>> ”
>>
>> I don't know what drove this requirement.
>>
>> From a first-principles perspective is doesn't seem to derive from any
>> foundational aspect of the linkage model.
> 
> I don't claim to know what Ulrich was thinking at the time, but if you
> start from the premise that it would be more consistent in general if
> symbols always resolved at load time to the library they were found in
> at link time, then adding this requirement for versioned symbols can
> be seen as a way to introduce that consistency in a
> backward-compatible fashion.  That said, without any way to declare
> that symbol X is _supposed_ to have moved from foo.so to bar.so, it
> puts us in a hole.
> 
> This is a GNU spec, so we _could_ change it.  We'd need to do a bit of
> formal consultation with everyone else who might be affected, but I
> don't think our hands should be tied by a decision that nobody
> remembers the original rationale for anymore.

I agree. I'd like to investigate changing this, but don't see myself
having the time to do so in the next 4-6 months, not at least until
2.28 is out the door (I'm the RM).

Cheers,
Carlos.

Re: Are the pthread "compatibility" copies of symbols in libc still necessary?

[Zack Weinberg]

On Fri, Mar 16, 2018 at 4:46 PM, Carlos O'Donell <carlos@redhat.com> wrote:
>> This is a GNU spec, so we _could_ change it.  We'd need to do a bit of
>> formal consultation with everyone else who might be affected, but I
>> don't think our hands should be tied by a decision that nobody
>> remembers the original rationale for anymore.
>
> I agree. I'd like to investigate changing this, but don't see myself
> having the time to do so in the next 4-6 months, not at least until
> 2.28 is out the door (I'm the RM).

My time is also extremely limited through September or so.