_IO_stdin_used stripped by version scripts

_IO_stdin_used stripped by version scripts

[Aurelien Jarno]

Hi all,

On some architectures the _IO_stdin_used symbol is used to determine if
the executable has been linked against the older version of libio or the
newer. More and more executables start to be linked with a version 
script in order to control the symbols they export to their plugins.
Examples of such software are firefox, network-manager or mplayer. When
using a version-script without explicitly adding an entry for the
_IO_stdin_used symbol, it appears as a local symbol in the resulting
libraries. This causes random crashes or strange behaviours. We have
observed that in Debian at least on alpha, mips* and powerpc.

People claim that adding _IO_stdin_used to the version scripts is at
best a workaround and this should be fixed in the toolchain instead.
What are the opinion of GNU libc people? How is that solved in other
distributions?

Thanks,
Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: _IO_stdin_used stripped by version scripts

[Florian Weimer]

* Aurelien Jarno:

> On some architectures the _IO_stdin_used symbol is used to determine if
> the executable has been linked against the older version of libio or the
> newer. More and more executables start to be linked with a version 
> script in order to control the symbols they export to their plugins.
> Examples of such software are firefox, network-manager or mplayer. When
> using a version-script without explicitly adding an entry for the
> _IO_stdin_used symbol, it appears as a local symbol in the resulting
> libraries. This causes random crashes or strange behaviours. We have
> observed that in Debian at least on alpha, mips* and powerpc.

The first part would be to figure out what _IO_stdin_used is actually
about.  I think it's there to preserve the libio ABI (as far as
vtables are concerned) despite the fact that the struct layout was
changed (possibly to reflect a C++ ABI change).

One fairly conservative change would version the symbols which
activate vtable compatibility, and have the new versions override the
_IO_stdin_used mechanism.  There should be room for such a flag in the
_flags2 member of struct _IO_file.

> People claim that adding _IO_stdin_used to the version scripts is at
> best a workaround and this should be fixed in the toolchain instead.

Is everyone (developers and users) willing to upgrade the toolchain?
I don't think so.  Fixings in glibc tend to need that before they
become effective.

> What are the opinion of GNU libc people? How is that solved in other
> distributions?

We haven't seen those crashes in Fedora, although the symbol is
sometimes missing there, too, and at least i386 is old enough to need
this mechanism.  We don't have any libio/stdio patches in the Fedora
glibc.

Re: _IO_stdin_used stripped by version scripts

[Aurelien Jarno]

On 2016-08-28 15:58, Florian Weimer wrote:
> * Aurelien Jarno:
> 
> > On some architectures the _IO_stdin_used symbol is used to determine if
> > the executable has been linked against the older version of libio or the
> > newer. More and more executables start to be linked with a version 
> > script in order to control the symbols they export to their plugins.
> > Examples of such software are firefox, network-manager or mplayer. When
> > using a version-script without explicitly adding an entry for the
> > _IO_stdin_used symbol, it appears as a local symbol in the resulting
> > libraries. This causes random crashes or strange behaviours. We have
> > observed that in Debian at least on alpha, mips* and powerpc.
> 
> The first part would be to figure out what _IO_stdin_used is actually
> about.  I think it's there to preserve the libio ABI (as far as
> vtables are concerned) despite the fact that the struct layout was
> changed (possibly to reflect a C++ ABI change).

For what I understood, the libio ABI has changed with glibc 2.1 in order
to support for wide-character streams and 64 bit offsets. These
changes required an incompatible change to the FILE structure. That's
why we have two versions of libio functions one tagged GLIBC_2.0 and one
tagged GLIBC_2.1. The problem is that the FILE structure can be passed
as a pointer so that the symbol versioning mechanism doesn't work.
Therefore when a binary is build against glibc >= 2.1, there will be a
reference to _IO_stdin_used add. If there are no references to
_IO_stdin_used, the compatibility layer will kick in, and make the
stdin/stdout/stderr pointers by pointers to the compatibility objects.

> One fairly conservative change would version the symbols which
> activate vtable compatibility, and have the new versions override the
> _IO_stdin_used mechanism.  There should be room for such a flag in the
> _flags2 member of struct _IO_file.

The other idea would be to simply disable this compatibility layer. This
means that binaries older than 15 years old would not work anymore, but
that *might* be acceptable for some architectures, although maybe not
for i386.

> > People claim that adding _IO_stdin_used to the version scripts is at
> > best a workaround and this should be fixed in the toolchain instead.
> 
> Is everyone (developers and users) willing to upgrade the toolchain?
> I don't think so.  Fixings in glibc tend to need that before they
> become effective.

It's true it takes time until everyone is using a new toolchain. But I
have the impression the problem is happening more and more often given
people start to use version-script on executables. So it might be the
right moment to tackle this issue.

> > What are the opinion of GNU libc people? How is that solved in other
> > distributions?
> 
> We haven't seen those crashes in Fedora, although the symbol is
> sometimes missing there, too, and at least i386 is old enough to need
> this mechanism.  We don't have any libio/stdio patches in the Fedora
> glibc.

It indeed seems that i386 currently doesn't crash one the _IO_stdin_used
symbol is missing. I guess that might change in the future or depending
on the operation done on the FILE objects.

Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: _IO_stdin_used stripped by version scripts

[Andreas Schwab]

On Aug 28 2016, Florian Weimer <fw@deneb.enyo.de> wrote:

> We haven't seen those crashes in Fedora, although the symbol is
> sometimes missing there, too, and at least i386 is old enough to need
> this mechanism.  We don't have any libio/stdio patches in the Fedora
> glibc.

I think this was only relevant while libio was shared with libstdc++.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: _IO_stdin_used stripped by version scripts

[Florian Weimer]

* Andreas Schwab:

> On Aug 28 2016, Florian Weimer <fw@deneb.enyo.de> wrote:
>
>> We haven't seen those crashes in Fedora, although the symbol is
>> sometimes missing there, too, and at least i386 is old enough to need
>> this mechanism.  We don't have any libio/stdio patches in the Fedora
>> glibc.
>
> I think this was only relevant while libio was shared with libstdc++.

But there are still old i386 binaries which use this functionality
(and they still worked a couple of months ago, with a current kernel
and all).

This still worked for the glibc 2.1 ABI a couple of months ago on
i386.  I'm not sure if we need to preserve this level of backwards
compatibility these days.  I doubt we test it systematically, and it
really constrains the kind of changes we can make to libio, not just
at the ABI level, but also in terms of applying fixes which change if
and in which order vtable methods are called.

Re: _IO_stdin_used stripped by version scripts

[Florian Weimer]

On 08/28/2016 10:01 PM, Aurelien Jarno wrote:
> On 2016-08-28 15:58, Florian Weimer wrote:
>> * Aurelien Jarno:
>>
>>> On some architectures the _IO_stdin_used symbol is used to determine if
>>> the executable has been linked against the older version of libio or the
>>> newer. More and more executables start to be linked with a version
>>> script in order to control the symbols they export to their plugins.
>>> Examples of such software are firefox, network-manager or mplayer. When
>>> using a version-script without explicitly adding an entry for the
>>> _IO_stdin_used symbol, it appears as a local symbol in the resulting
>>> libraries. This causes random crashes or strange behaviours. We have
>>> observed that in Debian at least on alpha, mips* and powerpc.
>>
>> The first part would be to figure out what _IO_stdin_used is actually
>> about.  I think it's there to preserve the libio ABI (as far as
>> vtables are concerned) despite the fact that the struct layout was
>> changed (possibly to reflect a C++ ABI change).
>
> For what I understood, the libio ABI has changed with glibc 2.1 in order
> to support for wide-character streams and 64 bit offsets. These
> changes required an incompatible change to the FILE structure. That's
> why we have two versions of libio functions one tagged GLIBC_2.0 and one
> tagged GLIBC_2.1. The problem is that the FILE structure can be passed
> as a pointer so that the symbol versioning mechanism doesn't work.
> Therefore when a binary is build against glibc >= 2.1, there will be a
> reference to _IO_stdin_used add. If there are no references to
> _IO_stdin_used, the compatibility layer will kick in, and make the
> stdin/stdout/stderr pointers by pointers to the compatibility objects.

And if this leads to crashes, the compatibility mechanism doesn't work. 
I'm not saying we should fix this, but there does seem to be a bug here, 
beyond the unintentional selection of compatibility mode.

There are a lot of ways how we could improve the hack.  But it's 
difficult to test that it works as intended because we don't have a 
large set of existing binaries to test with.

>> One fairly conservative change would version the symbols which
>> activate vtable compatibility, and have the new versions override the
>> _IO_stdin_used mechanism.  There should be room for such a flag in the
>> _flags2 member of struct _IO_file.
>
> The other idea would be to simply disable this compatibility layer. This
> means that binaries older than 15 years old would not work anymore, but
> that *might* be acceptable for some architectures, although maybe not
> for i386.

If we want to go this route, we should try it for i386 as well, and 
remove all external libio symbols (beyond those used by <stdio.h>), not 
just the existing compatibility hack.  The general libio compatibility 
has a large cost for us, and it's unclear if users still need it.

Thanks,
Florian