the setrlimit changes in glibc 2.1.3

the setrlimit changes in glibc 2.1.3

[Cristian Gafton]

The current changes in setrlimit that were backported to glibc 2.1.3 are
causing this library to become once again binary incompatible with the
preivious releases. You you have any dynamic library linked against a
previous version of glibc that is calling setrlimit, you'll not be able to
use it after upgrading to glibc 2.1.3.

At linking stage, the linker complains about an undefined
setrlimit@@GLIBC_2.0 symbol - this is because previously the setrlimit was
not a versioned symbol.

There are anumber of vendors that are hit by this problem, and I would
like to ask the hackers from other vendors what do they think about this
change. The options are:

1. revert the change and call it good for glibc 2.1. Life sucks.
2. build a custom object file (or static archive) that gets listed in
/usr/lib/libc.so that will provide that dummy symbol version - will this
work?
3. other options?

In any case, if we leave it the way it is now, we will be screwing up
people that have put out development environments linked against previous
versions of glibc. And again we are at fault.

Cristian
--
----------------------------------------------------------------------
Cristian Gafton     --     gafton@redhat.com      --     Red Hat, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  "How could this be a problem in a country where we have Intel and 
   Microsoft?"  --Al Gore on Y2K

Re: the setrlimit changes in glibc 2.1.3

[Mark Kettenis]

   Date: Wed, 12 Jan 2000 15:30:25 -0500 (EST)
   From: Cristian Gafton <gafton@redhat.com>

   The current changes in setrlimit that were backported to glibc 2.1.3 are
   causing this library to become once again binary incompatible with the
   preivious releases. You you have any dynamic library linked against a
   previous version of glibc that is calling setrlimit, you'll not be able to
   use it after upgrading to glibc 2.1.3.

Of course we are no longer binary compatible.  We have fixed some bugs
:).  But seriously:

   At linking stage, the linker complains about an undefined
   setrlimit@@GLIBC_2.0 symbol - this is because previously the setrlimit was
   not a versioned symbol.

If this is really what is happening, that is:

 * You built a shared library, that references setrlimit, linked
   against glibc 2.1, 2.1.1 or 2.1.2.

 * You replaced glibc with version 2.1.3.

 * If you now link against your shared library you get a warning about
   setrlimit@@GLIBC_2.0 undefined.

Then either something went wrong during your built of glibc 2.1.3, or
there is a bug in the (static) linker.  To check what is really you
could do

   nm /lib/libc-2.1.3.so | grep setrlimit

This should output at least two lines looking like

   xxxxxxxx T setrlimit@@GLIBC_2.1.3
   xxxxxxxx T setrlimit@GLIBC_2.0

If this is not the case, something went wrong when buildig glibc
2.1.3.  Otherwise it is most likely a linker bug.

Mark

Re: the setrlimit changes in glibc 2.1.3

[Cristian Gafton]

On Wed, 12 Jan 2000, Mark Kettenis wrote:

>  * If you now link against your shared library you get a warning about
>    setrlimit@@GLIBC_2.0 undefined.

It's an error.

> Then either something went wrong during your built of glibc 2.1.3, or
> there is a bug in the (static) linker.  To check what is really you
> could do
> 
>    nm /lib/libc-2.1.3.so | grep setrlimit
> 
> This should output at least two lines looking like
> 
>    xxxxxxxx T setrlimit@@GLIBC_2.1.3
>    xxxxxxxx T setrlimit@GLIBC_2.0
> 
> If this is not the case, something went wrong when buildig glibc
> 2.1.3.  Otherwise it is most likely a linker bug.

I _do_ get that. The problem is that setrlimit@GLIBC_2.0 is different than
the setrlimit@@GLIBC_2.0, which is what the liker expects to find.

Cristian
--
----------------------------------------------------------------------
Cristian Gafton     --     gafton@redhat.com      --     Red Hat, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  "How could this be a problem in a country where we have Intel and 
   Microsoft?"  --Al Gore on Y2K

Re: the setrlimit changes in glibc 2.1.3

[H . J . Lu]

On Wed, Jan 12, 2000 at 06:44:33PM -0500, Cristian Gafton wrote:
> On Wed, 12 Jan 2000, Mark Kettenis wrote:
> 
> >  * If you now link against your shared library you get a warning about
> >    setrlimit@@GLIBC_2.0 undefined.
> 
> It's an error.
> 
> > Then either something went wrong during your built of glibc 2.1.3, or
> > there is a bug in the (static) linker.  To check what is really you
> > could do
> > 
> >    nm /lib/libc-2.1.3.so | grep setrlimit
> > 
> > This should output at least two lines looking like
> > 
> >    xxxxxxxx T setrlimit@@GLIBC_2.1.3
> >    xxxxxxxx T setrlimit@GLIBC_2.0
> > 
> > If this is not the case, something went wrong when buildig glibc
> > 2.1.3.  Otherwise it is most likely a linker bug.
> 
> I _do_ get that. The problem is that setrlimit@GLIBC_2.0 is different than
> the setrlimit@@GLIBC_2.0, which is what the liker expects to find.

Please send me a testcase. I will take a look. It sounds like a
linker bug to me.


H.J.

Re: the setrlimit changes in glibc 2.1.3

[Roland McGrath]

> After working some more time with the 32bit UID support, I'd like to
> apologize for proposing the 32bit UID support for glibc 2.1.x.  I was
> outvetoed - but now I'm convinced that the patches shouldn't go into
> 2.1[1].

There is no need to apologize for the proposal.  The important thing is
that you were ready to listen to those of us who thought it was a bad idea.
No harm done.  One should never apologize for being wrong the first time, 
only if one insists on staying wrong.

Re: the setrlimit changes in glibc 2.1.3

[Andreas Jaeger]

>>>>> Roland McGrath writes:

 > So I think the real question here is, what warranted this kind of change in
 > a "bug-fix" release?  I think folks around here have gotten pretty unclear
 > on the concept of maintenance releases, and everybody's playing fast and loose.

After working some more time with the 32bit UID support, I'd like to
apologize for proposing the 32bit UID support for glibc 2.1.x.  I was
outvetoed - but now I'm convinced that the patches shouldn't go into
2.1[1].

 > This is just not the way to operate.  That's why we have a development
 > branch, for pete's sake.  Are we going to have to add another level of ".x"
 > to the versions to get a truly conservative maintenance release branch?

 > And, Mark: I'm afraid the realities of packaging systems (and of humans)
 > make it hopelessly unwise to attempt to have "the right thing" for anyone
 > be to upgrade their shared libraries differently than they upgrade their
 > development environment.

Andreas

Footnotes: 
[1]  But they should go into 2.2 to get further testing.
-- 
 Andreas Jaeger
  SuSE Labs aj@suse.de
   private aj@arthur.rhein-neckar.de

Re: the setrlimit changes in glibc 2.1.3

[Thorsten Kukuk]

Hi,

On Thu, Jan 13, Roland McGrath wrote:

> So I think the real question here is, what warranted this kind of change in
> a "bug-fix" release?  I think folks around here have gotten pretty unclear
> on the concept of maintenance releases, and everybody's playing fast and loose.
> This is just not the way to operate.  That's why we have a development
> branch, for pete's sake.  Are we going to have to add another level of ".x"
> to the versions to get a truly conservative maintenance release branch?
> 
> And, Mark: I'm afraid the realities of packaging systems (and of humans)
> make it hopelessly unwise to attempt to have "the right thing" for anyone
> be to upgrade their shared libraries differently than they upgrade their
> development environment.

I think the biggest problem is another: We need to much time
for bug fix releases and the development branch. Why not make
much ofter "bug-fix only" releases ? What is the problem if we
would have glibc 2.1.14 ? It is stupid to know the bug was fixed
3 month ago, but is not released yet. The distributors start adding
patches, and nobody really know the version which is used.

And maybe it is necessary to maintain 3 branches:
1. the current "bug-fix" version
2. the next bigger release with new, smaller features
   like support for new kernel functions.
3. the version with big changes like locale support.

We have the current situation, because small, new features
go into the "bug-fix" branch, since nobody will wait 2 years
until the next major release is made. glibc 2.2 was scheduled
to come 6 month after glibc 2.1, but this was for a much longer
time then 6 month.

  Thorsten  

-- 
Thorsten Kukuk       http://www.suse.de/~kukuk/       kukuk@suse.de
SuSE GmbH            Schanzaeckerstr. 10            90443 Nuernberg
Linux is like a Vorlon.  It is incredibly powerful, gives terse,
cryptic answers and has a lot of things going on in the background.

Re: the setrlimit changes in glibc 2.1.3

[Joel Klecker]

At 14:40 -0500 2000-01-13, Roland McGrath wrote:
>So I think the real question here is, what warranted this kind of change in
>a "bug-fix" release?  I think folks around here have gotten pretty unclear
>on the concept of maintenance releases, and everybody's playing fast and
>loose.
>This is just not the way to operate.  That's why we have a development
>branch, for pete's sake.  Are we going to have to add another level of ".x"
>to the versions to get a truly conservative maintenance release branch?

<AOL/> What the hell kinda release management is this?

I don't trust any of the new rlimit or LFS stuff, so I'm avoiding it in Debian.

Stable releases should never have anything but bug fixes, except perhaps
allowing new features in things besides libc and the dynamic linker
themselves, e.g. nscd or the libthread_db library.

>And, Mark: I'm afraid the realities of packaging systems (and of humans)
>make it hopelessly unwise to attempt to have "the right thing" for anyone
>be to upgrade their shared libraries differently than they upgrade their
>development environment.

I have to agree with that. /usr/<cpu>-linux-gnulibc{2.1,2.1.1,2.1.2} is not
my idea of a good thing.
-- 
Joel Klecker (aka Espy)       <URL: mailto:espy@debian.org >
Debian Package Maintainer for the GNU C Library.

Re: the setrlimit changes in glibc 2.1.3

[Roland McGrath]

So I think the real question here is, what warranted this kind of change in
a "bug-fix" release?  I think folks around here have gotten pretty unclear
on the concept of maintenance releases, and everybody's playing fast and loose.
This is just not the way to operate.  That's why we have a development
branch, for pete's sake.  Are we going to have to add another level of ".x"
to the versions to get a truly conservative maintenance release branch?

And, Mark: I'm afraid the realities of packaging systems (and of humans)
make it hopelessly unwise to attempt to have "the right thing" for anyone
be to upgrade their shared libraries differently than they upgrade their
development environment.

Re: the setrlimit changes in glibc 2.1.3

[H . J . Lu]

On Thu, Jan 13, 2000 at 07:21:06AM +0100, Thorsten Kukuk wrote:
> On Wed, Jan 12, H . J . Lu wrote:
> 
> > On Wed, Jan 12, 2000 at 06:41:30PM -0500, Cristian Gafton wrote:
> > > 
> > > Install glibc 2.1.2 (before the setrlimit changes), do a "make libfoo.so",
> > > then upgrade to the current glibc and in the same directory do a "make
> > > test". The link will fail.
> > > 
> > 
> > Thanks. It is what I need. I will try to fix the linker.
> 
> Just curious: I have reported this problem when we have introduced 
> symbol versioning, and everybody here explained me this is the expected
> behaviour because library and main program should never call 2
> different versions of the same function. Now it is a bug ?

I was on vacation for a few weeks. I just came back. Besides, a
testcase makes a big difference.



H.J.

Re: the setrlimit changes in glibc 2.1.3

[Mark Kettenis]

   Date: Thu, 13 Jan 2000 09:35:15 -0500 (EST)
   From: Cristian Gafton <gafton@redhat.com>

   All fine and dandy, now think big-time vendors and tell me how to tell
   them that all their dveelopment libraries they have put out are now
   useless. It is quite of a challenge to deal with the big folks as it is;
   if we pull a fast one like this on them it is going to become increasingly
   difficult.

The solution is very simple.  Tell people not to upgrade their
development environment to glibc 2.1.3.

This doesn't prevent people from upgrading their runtime environment.
It is not too difficult to set things up such that the compiler and
linker keep on using the old libs and headers.  This has the
additional advantage that you'll be able to produce binaries that run
on older systems. But you'll also get the important bug-fixes
contained in glibc 2.1.3.

Mark

Re: the setrlimit changes in glibc 2.1.3

[Jakub Jelinek]

On Thu, Jan 13, 2000 at 09:35:15AM -0500, Cristian Gafton wrote:
> On Thu, 13 Jan 2000, Mark Kettenis wrote:
> 
> >    No way. I don't see why I should do that.
> > 
> > Because mixing modules that use interfaces that are binary
> > incompatible is dangerous.  In the case of setrlimit, the rlim_t type
> > changed.  If both the shared library and an application that links
> > with that library use rlim_t in their communication, bad things may
> > happen if the library is using the old type and the application is
> > using the new type.
> > 
> > By the way, you should not simply rebuild the shared lib, but also
> > check if the interfaces it provides didn't change because of the
> > changes to the interface in libc.  If the interfaces did change, you
> > must bump the lib's soname or version the symbols in your library
> > itself.
> 
> All fine and dandy, now think big-time vendors and tell me how to tell
> them that all their dveelopment libraries they have put out are now
> useless. It is quite of a challenge to deal with the big folks as it is;
> if we pull a fast one like this on them it is going to become increasingly
> difficult.

Actually, it would be probably better to keep the old definitions and signed
rlim_t type in 2.1.3 and make the interface change in 2.1.90 only to avoid
the trouble.
2.1.3 setrlimit/getrlimit should internally use the old setrlimit/getrlimit
calls.
People can use setrlimit64/getrlimit64 if they need to play with limits
between 2G and 4G and 2.1.3 setrlimit64/getrlimit64 should internally use
the new setrlimit/getrlimit calls if they are available.

If people agree on this, I can post a patch.

Cheers,
    Jakub
___________________________________________________________________
Jakub Jelinek | jakub@redhat.com | http://sunsite.mff.cuni.cz/~jj
Linux version 2.3.39 on a sparc64 machine (1343.49 BogoMips)
___________________________________________________________________

Re: the setrlimit changes in glibc 2.1.3

[Cristian Gafton]

On Thu, 13 Jan 2000, Mark Kettenis wrote:

>    No way. I don't see why I should do that.
> 
> Because mixing modules that use interfaces that are binary
> incompatible is dangerous.  In the case of setrlimit, the rlim_t type
> changed.  If both the shared library and an application that links
> with that library use rlim_t in their communication, bad things may
> happen if the library is using the old type and the application is
> using the new type.
> 
> By the way, you should not simply rebuild the shared lib, but also
> check if the interfaces it provides didn't change because of the
> changes to the interface in libc.  If the interfaces did change, you
> must bump the lib's soname or version the symbols in your library
> itself.

All fine and dandy, now think big-time vendors and tell me how to tell
them that all their dveelopment libraries they have put out are now
useless. It is quite of a challenge to deal with the big folks as it is;
if we pull a fast one like this on them it is going to become increasingly
difficult.

Cristian
--
----------------------------------------------------------------------
Cristian Gafton    --     gafton@redhat.com     --       Red Hat, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 UNIX is user friendly. It's just selective about who its friends are.

Re: the setrlimit changes in glibc 2.1.3

[Mark Kettenis]

   Date: Thu, 13 Jan 2000 07:21:06 +0100
   From: Thorsten Kukuk <kukuk@suse.de>

   Just curious: I have reported this problem when we have introduced 
   symbol versioning, and everybody here explained me this is the expected
   behaviour because library and main program should never call 2
   different versions of the same function. Now it is a bug ?

No, it's not a bug.  It was designed this way.  As soon as we
introduce a new version for a symbol, the old one will become hidden
and the (static) linker will refuse to use it.

Mark

Re: the setrlimit changes in glibc 2.1.3

[Mark Kettenis]

   Date: Thu, 13 Jan 2000 08:38:32 -0500 (EST)
   From: Cristian Gafton <gafton@redhat.com>

   On Thu, 13 Jan 2000, Andreas Schwab wrote:

   > Then you must rebuild the shared lib in question.  Binary compatibility is
   > only about *runtime* compatibility, but not *linktime* compatibility.

   No way. I don't see why I should do that.

Because mixing modules that use interfaces that are binary
incompatible is dangerous.  In the case of setrlimit, the rlim_t type
changed.  If both the shared library and an application that links
with that library use rlim_t in their communication, bad things may
happen if the library is using the old type and the application is
using the new type.

By the way, you should not simply rebuild the shared lib, but also
check if the interfaces it provides didn't change because of the
changes to the interface in libc.  If the interfaces did change, you
must bump the lib's soname or version the symbols in your library
itself.

Mark

Re: the setrlimit changes in glibc 2.1.3

[Cristian Gafton]

On Thu, 13 Jan 2000, Andreas Schwab wrote:

> Then you must rebuild the shared lib in question.  Binary compatibility is
> only about *runtime* compatibility, but not *linktime* compatibility.

No way. I don't see why I should do that.

Cristian
--
----------------------------------------------------------------------
Cristian Gafton    --     gafton@redhat.com     --       Red Hat, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 UNIX is user friendly. It's just selective about who its friends are.

Re: the setrlimit changes in glibc 2.1.3

[Andreas Schwab]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 862 bytes --]

Cristian Gafton <gafton@redhat.com> writes:

|> On Wed, 12 Jan 2000, Geoff Keating wrote:
|> 
|> > If you run 'nm libc.so.6 | grep setrlimit', do you get
|> > 
|> > 000b18b4 T setrlimit@@GLIBC_2.1.3
|> > 000b1b74 T setrlimit@GLIBC_2.0
|> > 
|> > (of course, your hex numbers will vary)?
|> 
|> The problem is that the linker complains about a missing
|> setrlimit@@GLIBC_2.0 (note the double @@), which means that the shared lib
|> in question has a reference to an unversioned setrlimit.

Then you must rebuild the shared lib in question.  Binary compatibility is
only about *runtime* compatibility, but not *linktime* compatibility.

Andreas.

-- 
Andreas Schwab                                  "And now for something
SuSE Labs                                        completely different."
schwab@suse.de
SuSE GmbH, Schanzäckerstr. 10, D-90443 Nürnberg

Re: the setrlimit changes in glibc 2.1.3

[Thorsten Kukuk]

On Wed, Jan 12, H . J . Lu wrote:

> On Wed, Jan 12, 2000 at 06:41:30PM -0500, Cristian Gafton wrote:
> > 
> > Install glibc 2.1.2 (before the setrlimit changes), do a "make libfoo.so",
> > then upgrade to the current glibc and in the same directory do a "make
> > test". The link will fail.
> > 
> 
> Thanks. It is what I need. I will try to fix the linker.

Just curious: I have reported this problem when we have introduced 
symbol versioning, and everybody here explained me this is the expected
behaviour because library and main program should never call 2
different versions of the same function. Now it is a bug ?

  Thorsten

-- 
Thorsten Kukuk       http://www.suse.de/~kukuk/       kukuk@suse.de
SuSE GmbH            Schanzaeckerstr. 10            90443 Nuernberg
Linux is like a Vorlon.  It is incredibly powerful, gives terse,
cryptic answers and has a lot of things going on in the background.

Re: the setrlimit changes in glibc 2.1.3

[H . J . Lu]

On Wed, Jan 12, 2000 at 06:41:30PM -0500, Cristian Gafton wrote:
> 
> Install glibc 2.1.2 (before the setrlimit changes), do a "make libfoo.so",
> then upgrade to the current glibc and in the same directory do a "make
> test". The link will fail.
> 

Thanks. It is what I need. I will try to fix the linker.


H.J.

Re: the setrlimit changes in glibc 2.1.3

[Cristian Gafton]

On Wed, 12 Jan 2000, Geoff Keating wrote:

> So how did the shared lib get a reference to an unversioned setrlimit?
> Was it linked with glibc 2.0?

In glibc 2.1.2 the setrlimit was not versioned. For unversioned references
the linker assumes GLIBC_2.0. Which in this case is not correct.

> I suspect it was not linked with any libc at all, which would be the
> problem.

It was linked against libc (glibc 2.1.2). The test case is attached.

Install glibc 2.1.2 (before the setrlimit changes), do a "make libfoo.so",
then upgrade to the current glibc and in the same directory do a "make
test". The link will fail.

Cristian
--
----------------------------------------------------------------------
Cristian Gafton     --     gafton@redhat.com      --     Red Hat, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  "How could this be a problem in a country where we have Intel and 
   Microsoft?"  --Al Gore on Y2K

Re: the setrlimit changes in glibc 2.1.3

[Cristian Gafton]

On Wed, 12 Jan 2000, Geoff Keating wrote:

> If you run 'nm libc.so.6 | grep setrlimit', do you get
> 
> 000b18b4 T setrlimit@@GLIBC_2.1.3
> 000b1b74 T setrlimit@GLIBC_2.0
> 
> (of course, your hex numbers will vary)?

The problem is that the linker complains about a missing
setrlimit@@GLIBC_2.0 (note the double @@), which means that the shared lib
in question has a reference to an unversioned setrlimit. But in the glibc
2.1.3 we only have versioned information. The linker then expects to have
the GLIBC_2.0 as the default version in the library, but we're out of luck
again, because the default one is GLIBC_2.1.3

Cristian
--
----------------------------------------------------------------------
Cristian Gafton     --     gafton@redhat.com      --     Red Hat, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  "How could this be a problem in a country where we have Intel and 
   Microsoft?"  --Al Gore on Y2K