When to prevent execution of new binaries with old glibc

When to prevent execution of new binaries with old glibc

[Joseph Myers]

This question has come up in a couple of threads lately, but perhaps 
deserves its own thread.

When should glibc try to ensure that a binary (either a main executable or 
a shared library) built and linked against a newer version of glibc, and 
using some feature that would not work correctly when run with an older 
version of glibc, fails on startup (or shared library load) when run with 
older glibc, rather than failing later or executing incorrectly?

The main mechanism we have for this is symbol versions, although the 
EI_ABIVERSION / libc-abis mechanism might help detect some cases of using 
unsupported ELF features.  We use new symbol versions for new glibc ports, 
newly exported symbols, symbols that have changed semantics in a way 
incompatible with existing binaries, and symbols that have moved from 
another library to libc.

We rarely, however, use such a mechanism in cases where old binaries are 
expected to continue to work OK.  That includes, for example:

* Bug fixes.

* Changed function semantics where the new semantics are a refinement of 
the old semantics (e.g. a new standard imposes stricter requirements on 
the result than an old standard, and glibc changes to implement that).

* Functions taking an enumerated argument (including any case of integers 
from some enumerated list, whether or not using a C enum type) describing 
something about the operation they carry out, where support for a new 
value of that argument is added.  (In some cases, it may be the kernel not 
glibc that adds support for new values of the argument, in which case 
there is nothing much glibc can do.)

* Functions taking a flags argument, where support for a new flag is 
added.  (Again, in some cases it's actually the kernel implementing the 
argument.)

* Functions taking a string argument that they parse, where support for 
some new feature in the string is added and strings using that new feature 
would previously have been considered to result in undefined behavior 
(such as adding new format specifiers for printf, scanf, strftime, 
strptime and strfmon format strings).

In the above cases, it would be possible to use symbol versioning, making 
the new symbol version an alias of the old one, but we don't generally do 
so.  There are also other, possibly rarer, cases where such symbol version 
aliasing can be used for a similar purpose (e.g. the change long ago of 
the return type of various <fenv.h> functions from void to int in C99 TC1, 
where we did use symbol version aliasing on the affected functions).

When, if at all, should we set up symbol version aliases for such issues 
(or use some other mechanism to prevent execution of new binaries with old 
glibc)?  We can take the case of new printf specifiers as an example, 
where HJ has suggested 
<https://sourceware.org/pipermail/libc-alpha/2021-October/132150.html> 
that binaries using such features should have a dependency on the relevant 
glibc version (and in practice that would mean binaries using any affected 
printf-like function, 56 per long double variant, whether they use the new 
formats or not).  But we haven't done that for e.g. strftime/strptime 
changes (and scanf functions have only got new function names because of 
incompatible differences in the handling of formats that were already 
valid before the changes).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: When to prevent execution of new binaries with old glibc

[H.J. Lu]

On Thu, Oct 21, 2021 at 10:43 AM Joseph Myers <joseph@codesourcery.com> wrote:
>
> This question has come up in a couple of threads lately, but perhaps
> deserves its own thread.
>
> When should glibc try to ensure that a binary (either a main executable or
> a shared library) built and linked against a newer version of glibc, and
> using some feature that would not work correctly when run with an older
> version of glibc, fails on startup (or shared library load) when run with
> older glibc, rather than failing later or executing incorrectly?

This is a QOI issue.  We should do our best to ensure it.   Otherwise,
we don't need all these intricate symbol versioning schemes.

> The main mechanism we have for this is symbol versions, although the
> EI_ABIVERSION / libc-abis mechanism might help detect some cases of using
> unsupported ELF features.  We use new symbol versions for new glibc ports,
> newly exported symbols, symbols that have changed semantics in a way
> incompatible with existing binaries, and symbols that have moved from
> another library to libc.
>
> We rarely, however, use such a mechanism in cases where old binaries are
> expected to continue to work OK.  That includes, for example:
>
> * Bug fixes.
>
> * Changed function semantics where the new semantics are a refinement of
> the old semantics (e.g. a new standard imposes stricter requirements on
> the result than an old standard, and glibc changes to implement that).
>
> * Functions taking an enumerated argument (including any case of integers
> from some enumerated list, whether or not using a C enum type) describing
> something about the operation they carry out, where support for a new
> value of that argument is added.  (In some cases, it may be the kernel not
> glibc that adds support for new values of the argument, in which case
> there is nothing much glibc can do.)
>
> * Functions taking a flags argument, where support for a new flag is
> added.  (Again, in some cases it's actually the kernel implementing the
> argument.)
>
> * Functions taking a string argument that they parse, where support for
> some new feature in the string is added and strings using that new feature
> would previously have been considered to result in undefined behavior
> (such as adding new format specifiers for printf, scanf, strftime,
> strptime and strfmon format strings).
>
> In the above cases, it would be possible to use symbol versioning, making
> the new symbol version an alias of the old one, but we don't generally do
> so.  There are also other, possibly rarer, cases where such symbol version
> aliasing can be used for a similar purpose (e.g. the change long ago of
> the return type of various <fenv.h> functions from void to int in C99 TC1,
> where we did use symbol version aliasing on the affected functions).

These are our oversights, but not green lights to allowexecutables
or shared libraries to crash at random or generate incorrect results
with older glibcs.

> When, if at all, should we set up symbol version aliases for such issues
> (or use some other mechanism to prevent execution of new binaries with old
> glibc)?  We can take the case of new printf specifiers as an example,
> where HJ has suggested
> <https://sourceware.org/pipermail/libc-alpha/2021-October/132150.html>
> that binaries using such features should have a dependency on the relevant
> glibc version (and in practice that would mean binaries using any affected
> printf-like function, 56 per long double variant, whether they use the new
> formats or not).  But we haven't done that for e.g. strftime/strptime

We should do that going forward if executables or shared libraries may
crash at random or generate incorrect results with older glibcs.

We can add a glibc version reference to one of the crt files which is included
in all executables and shared libraries.

> changes (and scanf functions have only got new function names because of
> incompatible differences in the handling of formats that were already
> valid before the changes).
>

-- 
H.J.

Re: When to prevent execution of new binaries with old glibc

[Joseph Myers]

On Fri, 22 Oct 2021, H.J. Lu via Libc-alpha wrote:

> On Thu, Oct 21, 2021 at 10:43 AM Joseph Myers <joseph@codesourcery.com> wrote:
> >
> > This question has come up in a couple of threads lately, but perhaps
> > deserves its own thread.
> >
> > When should glibc try to ensure that a binary (either a main executable or
> > a shared library) built and linked against a newer version of glibc, and
> > using some feature that would not work correctly when run with an older
> > version of glibc, fails on startup (or shared library load) when run with
> > older glibc, rather than failing later or executing incorrectly?
> 
> This is a QOI issue.  We should do our best to ensure it.   Otherwise,
> we don't need all these intricate symbol versioning schemes.

I think what we've actually tried in the past is to achieve a reasonable 
balance, detecting the most obvious cases where a program is expected not 
to work with old glibc while leaving people to try running programs on 
older glibc at their own risk in other cases and avoiding introducing 
excess complexity (symbol versioning is still needed to keep proper 
compatibility with existing binaries and so not require people to rebuild 
everything when they upgrade glibc).

(Running a binary with an older glibc than that it was built with is never 
recommended, however, even when it sometimes works.)

Cf. the past discussions of how it would make sense to *remove* the 
runtime kernel version check against the version specified by 
--enable-kernel, letting programs succeed or fail when the relevant 
syscalls are executed, given that (a) they might well not actually depend 
on any affected functionality and (b) the older host kernel used in the 
affected container configurations may actually have backports of the newer 
syscalls.

> These are our oversights, but not green lights to allowexecutables
> or shared libraries to crash at random or generate incorrect results
> with older glibcs.

I don't think they are oversights; they're more part of the balance 
between catching common problem cases running new binaries on old glibcs 
and limiting the complexity involved in catching such cases.

> > When, if at all, should we set up symbol version aliases for such issues
> > (or use some other mechanism to prevent execution of new binaries with old
> > glibc)?  We can take the case of new printf specifiers as an example,
> > where HJ has suggested
> > <https://sourceware.org/pipermail/libc-alpha/2021-October/132150.html>
> > that binaries using such features should have a dependency on the relevant
> > glibc version (and in practice that would mean binaries using any affected
> > printf-like function, 56 per long double variant, whether they use the new
> > formats or not).  But we haven't done that for e.g. strftime/strptime
> 
> We should do that going forward if executables or shared libraries may
> crash at random or generate incorrect results with older glibcs.

That could cover just about any bug fix.

> We can add a glibc version reference to one of the crt files which is included
> in all executables and shared libraries.

We can, but that's a very different balance from that chosen before for 
when to stop execution of new binaries with old glibc.  (The new 
__libc_start_main symbol version means executables built with glibc 2.34 
won't run with older versions, but that's because of a more general 
compatibility issue, rather than being routine for any new version with 
any new features in any function.)

-- 
Joseph S. Myers
joseph@codesourcery.com