Compat symbols in abilist files

Compat symbols in abilist files

[Florian Weimer]

[-- Attachment #1: Type: text/plain, Size: 377 bytes --]

I've come up with the attached patch to mark compat symbols as such in 
the abilist files.

Do we want to do this?  The downside is that when building glibc in 
certain non-standard configurations, some symbols are not compat symbols 
anymore (the prime example is probably --enable-obsolete-rpc).  On the 
other hand, the additional verification is valuable.

Thanks,
Florian

[-- Attachment #2: compat.patch --]
[-- Type: text/x-patch, Size: 954 bytes --]

ABI checks: Mark compat symbols with "<compat>"

2017-04-13  Florian Weimer  <fweimer@redhat.com>

	* scripts/abilist.awk: Mark compat symbols with "<compat>."

diff --git a/scripts/abilist.awk b/scripts/abilist.awk
index bd740d4..cb17373 100644
--- a/scripts/abilist.awk
+++ b/scripts/abilist.awk
@@ -46,7 +46,10 @@ $2 == "g" || $2 == "w" && (NF == 7 || NF == 8) {
   size = " 0x" size;
   version = $6;
   symbol = $NF;
-  gsub(/[()]/, "", version);
+  weak = version ~ /^\(.*\)$/;
+  if (weak) {
+    gsub(/[()]/, "", version);
+  }
 
   # binutils versions up through at least 2.23 have some bugs that
   # caused STV_HIDDEN symbols to appear in .dynsym, though that is useless.
@@ -102,6 +105,10 @@ $2 == "g" || $2 == "w" && (NF == 7 || NF == 8) {
   if (desc == "")
     desc = symbol " " type size;
 
+  if (weak) {
+    desc = desc " <compat>";
+  }
+
   if (combine)
     version = soname " " version (combine_fullname ? " " sofullname : "");
 

Re: Compat symbols in abilist files

[Andreas Schwab]

On Apr 13 2017, Florian Weimer <fweimer@redhat.com> wrote:

> Do we want to do this?  The downside is that when building glibc in
> certain non-standard configurations, some symbols are not compat symbols
> anymore (the prime example is probably --enable-obsolete-rpc).

Does that mean check-abi fails in that case?

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: Compat symbols in abilist files

[Florian Weimer]

On 04/13/2017 03:16 PM, Andreas Schwab wrote:
> On Apr 13 2017, Florian Weimer <fweimer@redhat.com> wrote:
> 
>> Do we want to do this?  The downside is that when building glibc in
>> certain non-standard configurations, some symbols are not compat symbols
>> anymore (the prime example is probably --enable-obsolete-rpc).
> 
> Does that mean check-abi fails in that case?

Yes, it would.

Thanks,
Florian

Re: Compat symbols in abilist files

[Joseph Myers]

On Thu, 13 Apr 2017, Florian Weimer wrote:

> I've come up with the attached patch to mark compat symbols as such in the
> abilist files.
> 
> Do we want to do this?  The downside is that when building glibc in certain
> non-standard configurations, some symbols are not compat symbols anymore (the
> prime example is probably --enable-obsolete-rpc).  On the other hand, the
> additional verification is valuable.

I'd consider whether a symbol is a compat symbol to be part of the API, 
not the ABI; I don't think it belongs in these files.

It may well make sense to be able to assert that a particular symbol is 
not available for new programs to link against, for either static or 
dynamic linking.  Or that any given symbol has at most one non-compat 
version, in at most one library, except for cases (if any) explicitly 
whitelisted as deliberately having public versions in more than one 
library.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Compat symbols in abilist files

[Carlos O'Donell]

On 04/18/2017 06:18 PM, Joseph Myers wrote:
> On Thu, 13 Apr 2017, Florian Weimer wrote:
> 
>> I've come up with the attached patch to mark compat symbols as such in the
>> abilist files.
>>
>> Do we want to do this?  The downside is that when building glibc in certain
>> non-standard configurations, some symbols are not compat symbols anymore (the
>> prime example is probably --enable-obsolete-rpc).  On the other hand, the
>> additional verification is valuable.
> 
> I'd consider whether a symbol is a compat symbol to be part of the API, 
> not the ABI; I don't think it belongs in these files.
> 
> It may well make sense to be able to assert that a particular symbol is 
> not available for new programs to link against, for either static or 
> dynamic linking.  Or that any given symbol has at most one non-compat 
> version, in at most one library, except for cases (if any) explicitly 
> whitelisted as deliberately having public versions in more than one 
> library.
 
Could you expand on why you think this is not part of the ABI?

-- 
Cheers,
Carlos.

Re: Compat symbols in abilist files

[Joseph Myers]

On Wed, 19 Apr 2017, Carlos O'Donell wrote:

> > I'd consider whether a symbol is a compat symbol to be part of the API, 
> > not the ABI; I don't think it belongs in these files.
> > 
> > It may well make sense to be able to assert that a particular symbol is 
> > not available for new programs to link against, for either static or 
> > dynamic linking.  Or that any given symbol has at most one non-compat 
> > version, in at most one library, except for cases (if any) explicitly 
> > whitelisted as deliberately having public versions in more than one 
> > library.
>  
> Could you expand on why you think this is not part of the ABI?

The ABI is about what (already linked) binaries (executables and shared 
libraries) work with a given glibc installation.  Whether or not a symbol 
is a compat symbol does not affect which executables and shared libraries 
work with it.  It affects what source code can be built and linked with 
that glibc, which is an API issue, not an ABI one.

(The interface to .o files is not expected to be stable, if e.g. one glibc 
version uses an implementation-namespace function in an inline function in 
a header, but a subsequent version removes that inline function and 
changes that symbol to a compat symbol.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Compat symbols in abilist files

[Carlos O'Donell]

On 04/13/2017 09:09 AM, Florian Weimer wrote:
> I've come up with the attached patch to mark compat symbols as such in the abilist files.
> 
> Do we want to do this?  The downside is that when building glibc in certain non-standard configurations, some symbols are not compat symbols anymore (the prime example is probably --enable-obsolete-rpc).  On the other hand, the additional verification is valuable.

I think this is a good idea. We should detect changes in compat symbols as ABI breakage.

The upstream glibc abilist files should be for the default supported configuration.

Downstream distros should patch the abilist to match their configuration changes.

-- 
Cheers,
Carlos.

Re: Compat symbols in abilist files

[Florian Weimer]

On 04/19/2017 04:06 AM, Carlos O'Donell wrote:
> Downstream distros should patch the abilist to match their configuration changes.

I'm not sure if I agree with that.  I'd prefer if these files remained 
unmodified because differences threaten portability across distributions.

Is anyone except Fedora compiling current glibc versions with the 
--enable-obsolete-* flags?  Then we'll need a more intelligent ABI 
differ.  (If it's just Fedora, I'd prefer moving Fedora forward.)

Thanks,
Florian

Compat symbols in abilist files

[Florian Weimer]

I'd like to re-raise the matter of comapt symbol markers in abilist
files.

--enable-obsolete-rpc (which changed compat symbol status) is gone,
which should simplify matters.

Joseph, you objected to this in the past.  While reviewing libpthread
symbol moves, I noticed that when going from

  libc:  pread@@GLIBC_2.1   libpthread: pread@@GLIBC_2.2

to

  libc:  pread@GLIBC_2.1 pread@@GLIBC_2.2

we would actually have to switch to 

  libc:  pread@GLIBC_2.1 pread@GLIBC_2.2 pread@@GLIBC_2.34

because a newly linked binary would produce a pread@@GLIBC_2.2 reference
without linking to libpthread, which could result in a lazy binding
failure at run time.  If we swap around the roles of compat symbols,
like this

  libc:  pread@@GLIBC_2.1 pread@GLIBC_2.2

the new symbol version is not necessary.

Unfortunately this is currently not visible at all in the abilist files.
The versioned_symbol/compat_symbol constructs are also quite hard to
review.  So I'm worried that we don't get the link-time ABI we want on
all targets.

Thanks,
Florian

Re: Compat symbols in abilist files

[Joseph Myers]

On Wed, 17 Mar 2021, Florian Weimer via Libc-alpha wrote:

> I'd like to re-raise the matter of comapt symbol markers in abilist
> files.
> 
> --enable-obsolete-rpc (which changed compat symbol status) is gone,
> which should simplify matters.

Might we have other such options in future?

> Joseph, you objected to this in the past.  While reviewing libpthread
> symbol moves, I noticed that when going from

I think my argument was that whether a symbol is a compat symbol is not 
part of the ABI (it doesn't affect which executables or shared libraries 
will work with a given glibc build).

So the question is: do we want this information in the abilist files even 
though it's not part of the ABI?

I expect many patches that change this information would need to change 
abilist files anyway (the exception being a patch that purely marks 
symbols as compat symbols, without adding any new versions of those 
symbols).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Compat symbols in abilist files

[Florian Weimer]

* Joseph Myers:

> On Wed, 17 Mar 2021, Florian Weimer via Libc-alpha wrote:
>
>> I'd like to re-raise the matter of comapt symbol markers in abilist
>> files.
>> 
>> --enable-obsolete-rpc (which changed compat symbol status) is gone,
>> which should simplify matters.
>
> Might we have other such options in future?

Hard to tell.  I think the experience with --enable-obsolete-rpc wasn't
altogether positive.  The transition took far too long.

If I saw the sequence of changes, --enable-obsolete-rpc was added as a
stopgap measure after too much broke.  I suspect that won't happen
again, in that we would try hard to get distributions to move first.

>> Joseph, you objected to this in the past.  While reviewing libpthread
>> symbol moves, I noticed that when going from
>
> I think my argument was that whether a symbol is a compat symbol is not 
> part of the ABI (it doesn't affect which executables or shared libraries 
> will work with a given glibc build).

I think it's part of the link-time ABI, which in turn affects the
run-time ABI of the linked binaries.  There is definitely a
compatibility impact here, in the sense that the wrong choice of default
symbol version can cause breakage in the field (although not when
running against the same glibc to which the link happened).

> So the question is: do we want this information in the abilist files even 
> though it's not part of the ABI?

I think we have cases where compat symbols are universal because we
entirely deprecated an interface (e.g., loc1).  Those could be handled
separately.  Others are highly version-dependent, and describing that
concisely would likely encounter similar complexity than what we have in
the sources (e.g., pread).  Then there are things that are completely
target-specific, like the s390x symbol situation around glibc 2.19.
Data for that would have to live in some target-specific list in any
case.

Given that, putting a compat (C?) flag into the existing abilist files
looks like a pragmatic solution to me.

> I expect many patches that change this information would need to change 
> abilist files anyway (the exception being a patch that purely marks 
> symbols as compat symbols, without adding any new versions of those 
> symbols).

One case I mentioned in my first message is another exception.  When we
move a symbol from libc that has a higher version, like this:

  libc:  pread@@GLIBC_2.1   libpthread: pread@@GLIBC_2.2

we can get away without a new symbol version at the current glibc
version, if we go to this:

  libc:  pread@@GLIBC_2.1 pread@GLIBC_2.2

Then it is totally not apparent from the abilist whether we have the
right default symbol version.  For the usual case,

  libc:  pread@GLIBC_2.1 pread@GLIBC_2.2 pread@@GLIBC_2.34

it's more obvious.  But with the retroactive update, we could also end
up by accident with:

  libc:  pread@GLIBC_2.1 pread@@GLIBC_2.2

and that would be slightly inaccurate.

Patches that change symbol management globally (like the one I posted
today) would also benefit greatly from that little extra data in the
*.abilist files, I think.

Thanks,
Florian