Re: the setrlimit changes in glibc 2.1.3

Re: the setrlimit changes in glibc 2.1.3

[Cristian Gafton]

On Wed, 12 Jan 2000, Geoff Keating wrote:

> If you run 'nm libc.so.6 | grep setrlimit', do you get
> 
> 000b18b4 T setrlimit@@GLIBC_2.1.3
> 000b1b74 T setrlimit@GLIBC_2.0
> 
> (of course, your hex numbers will vary)?

The problem is that the linker complains about a missing
setrlimit@@GLIBC_2.0 (note the double @@), which means that the shared lib
in question has a reference to an unversioned setrlimit. But in the glibc
2.1.3 we only have versioned information. The linker then expects to have
the GLIBC_2.0 as the default version in the library, but we're out of luck
again, because the default one is GLIBC_2.1.3

Cristian
--
----------------------------------------------------------------------
Cristian Gafton     --     gafton@redhat.com      --     Red Hat, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  "How could this be a problem in a country where we have Intel and 
   Microsoft?"  --Al Gore on Y2K

Re: the setrlimit changes in glibc 2.1.3

[Andreas Schwab]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 862 bytes --]

Cristian Gafton <gafton@redhat.com> writes:

|> On Wed, 12 Jan 2000, Geoff Keating wrote:
|> 
|> > If you run 'nm libc.so.6 | grep setrlimit', do you get
|> > 
|> > 000b18b4 T setrlimit@@GLIBC_2.1.3
|> > 000b1b74 T setrlimit@GLIBC_2.0
|> > 
|> > (of course, your hex numbers will vary)?
|> 
|> The problem is that the linker complains about a missing
|> setrlimit@@GLIBC_2.0 (note the double @@), which means that the shared lib
|> in question has a reference to an unversioned setrlimit.

Then you must rebuild the shared lib in question.  Binary compatibility is
only about *runtime* compatibility, but not *linktime* compatibility.

Andreas.

-- 
Andreas Schwab                                  "And now for something
SuSE Labs                                        completely different."
schwab@suse.de
SuSE GmbH, Schanzäckerstr. 10, D-90443 Nürnberg

Re: the setrlimit changes in glibc 2.1.3

[Cristian Gafton]

On Thu, 13 Jan 2000, Andreas Schwab wrote:

> Then you must rebuild the shared lib in question.  Binary compatibility is
> only about *runtime* compatibility, but not *linktime* compatibility.

No way. I don't see why I should do that.

Cristian
--
----------------------------------------------------------------------
Cristian Gafton    --     gafton@redhat.com     --       Red Hat, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 UNIX is user friendly. It's just selective about who its friends are.

Re: the setrlimit changes in glibc 2.1.3

[Mark Kettenis]

   Date: Thu, 13 Jan 2000 08:38:32 -0500 (EST)
   From: Cristian Gafton <gafton@redhat.com>

   On Thu, 13 Jan 2000, Andreas Schwab wrote:

   > Then you must rebuild the shared lib in question.  Binary compatibility is
   > only about *runtime* compatibility, but not *linktime* compatibility.

   No way. I don't see why I should do that.

Because mixing modules that use interfaces that are binary
incompatible is dangerous.  In the case of setrlimit, the rlim_t type
changed.  If both the shared library and an application that links
with that library use rlim_t in their communication, bad things may
happen if the library is using the old type and the application is
using the new type.

By the way, you should not simply rebuild the shared lib, but also
check if the interfaces it provides didn't change because of the
changes to the interface in libc.  If the interfaces did change, you
must bump the lib's soname or version the symbols in your library
itself.

Mark

Re: the setrlimit changes in glibc 2.1.3

[Cristian Gafton]

On Thu, 13 Jan 2000, Mark Kettenis wrote:

>    No way. I don't see why I should do that.
> 
> Because mixing modules that use interfaces that are binary
> incompatible is dangerous.  In the case of setrlimit, the rlim_t type
> changed.  If both the shared library and an application that links
> with that library use rlim_t in their communication, bad things may
> happen if the library is using the old type and the application is
> using the new type.
> 
> By the way, you should not simply rebuild the shared lib, but also
> check if the interfaces it provides didn't change because of the
> changes to the interface in libc.  If the interfaces did change, you
> must bump the lib's soname or version the symbols in your library
> itself.

All fine and dandy, now think big-time vendors and tell me how to tell
them that all their dveelopment libraries they have put out are now
useless. It is quite of a challenge to deal with the big folks as it is;
if we pull a fast one like this on them it is going to become increasingly
difficult.

Cristian
--
----------------------------------------------------------------------
Cristian Gafton    --     gafton@redhat.com     --       Red Hat, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 UNIX is user friendly. It's just selective about who its friends are.

Re: the setrlimit changes in glibc 2.1.3

[Jakub Jelinek]

On Thu, Jan 13, 2000 at 09:35:15AM -0500, Cristian Gafton wrote:
> On Thu, 13 Jan 2000, Mark Kettenis wrote:
> 
> >    No way. I don't see why I should do that.
> > 
> > Because mixing modules that use interfaces that are binary
> > incompatible is dangerous.  In the case of setrlimit, the rlim_t type
> > changed.  If both the shared library and an application that links
> > with that library use rlim_t in their communication, bad things may
> > happen if the library is using the old type and the application is
> > using the new type.
> > 
> > By the way, you should not simply rebuild the shared lib, but also
> > check if the interfaces it provides didn't change because of the
> > changes to the interface in libc.  If the interfaces did change, you
> > must bump the lib's soname or version the symbols in your library
> > itself.
> 
> All fine and dandy, now think big-time vendors and tell me how to tell
> them that all their dveelopment libraries they have put out are now
> useless. It is quite of a challenge to deal with the big folks as it is;
> if we pull a fast one like this on them it is going to become increasingly
> difficult.

Actually, it would be probably better to keep the old definitions and signed
rlim_t type in 2.1.3 and make the interface change in 2.1.90 only to avoid
the trouble.
2.1.3 setrlimit/getrlimit should internally use the old setrlimit/getrlimit
calls.
People can use setrlimit64/getrlimit64 if they need to play with limits
between 2G and 4G and 2.1.3 setrlimit64/getrlimit64 should internally use
the new setrlimit/getrlimit calls if they are available.

If people agree on this, I can post a patch.

Cheers,
    Jakub
___________________________________________________________________
Jakub Jelinek | jakub@redhat.com | http://sunsite.mff.cuni.cz/~jj
Linux version 2.3.39 on a sparc64 machine (1343.49 BogoMips)
___________________________________________________________________

Re: the setrlimit changes in glibc 2.1.3

[Mark Kettenis]

   Date: Thu, 13 Jan 2000 09:35:15 -0500 (EST)
   From: Cristian Gafton <gafton@redhat.com>

   All fine and dandy, now think big-time vendors and tell me how to tell
   them that all their dveelopment libraries they have put out are now
   useless. It is quite of a challenge to deal with the big folks as it is;
   if we pull a fast one like this on them it is going to become increasingly
   difficult.

The solution is very simple.  Tell people not to upgrade their
development environment to glibc 2.1.3.

This doesn't prevent people from upgrading their runtime environment.
It is not too difficult to set things up such that the compiler and
linker keep on using the old libs and headers.  This has the
additional advantage that you'll be able to produce binaries that run
on older systems. But you'll also get the important bug-fixes
contained in glibc 2.1.3.

Mark

Re: the setrlimit changes in glibc 2.1.3

[Roland McGrath]

So I think the real question here is, what warranted this kind of change in
a "bug-fix" release?  I think folks around here have gotten pretty unclear
on the concept of maintenance releases, and everybody's playing fast and loose.
This is just not the way to operate.  That's why we have a development
branch, for pete's sake.  Are we going to have to add another level of ".x"
to the versions to get a truly conservative maintenance release branch?

And, Mark: I'm afraid the realities of packaging systems (and of humans)
make it hopelessly unwise to attempt to have "the right thing" for anyone
be to upgrade their shared libraries differently than they upgrade their
development environment.

Re: the setrlimit changes in glibc 2.1.3

[Joel Klecker]

At 14:40 -0500 2000-01-13, Roland McGrath wrote:
>So I think the real question here is, what warranted this kind of change in
>a "bug-fix" release?  I think folks around here have gotten pretty unclear
>on the concept of maintenance releases, and everybody's playing fast and
>loose.
>This is just not the way to operate.  That's why we have a development
>branch, for pete's sake.  Are we going to have to add another level of ".x"
>to the versions to get a truly conservative maintenance release branch?

<AOL/> What the hell kinda release management is this?

I don't trust any of the new rlimit or LFS stuff, so I'm avoiding it in Debian.

Stable releases should never have anything but bug fixes, except perhaps
allowing new features in things besides libc and the dynamic linker
themselves, e.g. nscd or the libthread_db library.

>And, Mark: I'm afraid the realities of packaging systems (and of humans)
>make it hopelessly unwise to attempt to have "the right thing" for anyone
>be to upgrade their shared libraries differently than they upgrade their
>development environment.

I have to agree with that. /usr/<cpu>-linux-gnulibc{2.1,2.1.1,2.1.2} is not
my idea of a good thing.
-- 
Joel Klecker (aka Espy)       <URL: mailto:espy@debian.org >
Debian Package Maintainer for the GNU C Library.

Re: the setrlimit changes in glibc 2.1.3

[Thorsten Kukuk]

Hi,

On Thu, Jan 13, Roland McGrath wrote:

> So I think the real question here is, what warranted this kind of change in
> a "bug-fix" release?  I think folks around here have gotten pretty unclear
> on the concept of maintenance releases, and everybody's playing fast and loose.
> This is just not the way to operate.  That's why we have a development
> branch, for pete's sake.  Are we going to have to add another level of ".x"
> to the versions to get a truly conservative maintenance release branch?
> 
> And, Mark: I'm afraid the realities of packaging systems (and of humans)
> make it hopelessly unwise to attempt to have "the right thing" for anyone
> be to upgrade their shared libraries differently than they upgrade their
> development environment.

I think the biggest problem is another: We need to much time
for bug fix releases and the development branch. Why not make
much ofter "bug-fix only" releases ? What is the problem if we
would have glibc 2.1.14 ? It is stupid to know the bug was fixed
3 month ago, but is not released yet. The distributors start adding
patches, and nobody really know the version which is used.

And maybe it is necessary to maintain 3 branches:
1. the current "bug-fix" version
2. the next bigger release with new, smaller features
   like support for new kernel functions.
3. the version with big changes like locale support.

We have the current situation, because small, new features
go into the "bug-fix" branch, since nobody will wait 2 years
until the next major release is made. glibc 2.2 was scheduled
to come 6 month after glibc 2.1, but this was for a much longer
time then 6 month.

  Thorsten  

-- 
Thorsten Kukuk       http://www.suse.de/~kukuk/       kukuk@suse.de
SuSE GmbH            Schanzaeckerstr. 10            90443 Nuernberg
Linux is like a Vorlon.  It is incredibly powerful, gives terse,
cryptic answers and has a lot of things going on in the background.

glibc release management

[Roland McGrath]

You will get no argument from me about shorter release cycles, but I am not
someone doing much of the work.  There are many details to bicker about,
but I'm sure that the crux of Ulrich's reply will also be simply about the
time he has available to put into libc.  

There are multiple employees of all the major distributors on this list,
and it is well past time for those monied companies to put some serious
investment explicitly into glibc maintenance and development.  Those of you
working for distributors, I don't know how much of your paid job is
supposed to have to do with glibc, but as far as I am aware there is no
company paying anyone explicitly to work on the libc team per se (as
opposed to their distribution's modified glibc package).

We can have more frequent releases, and better release management, but we
cannot get it with Ulrich in charge of it alone unless that becomes his
sole paid job (and I am presuming he would rather keep doing other things
for Cygnus/Redhat instead of just glibc).  What we got now from Ulrich is
far better than one could presume to expect from anyone without paying to
have it be job 1.  Anyone complaining, complain to your boss or your boss's
boss and don't come back here complaining until your company is paying
someone (or better, part of a multi-company detente that pays someone) to be
distribution-independent GNU libc maintainer/release-manager full time.

On the pseudotechnical front, what I would like see is the "bug-fix"
releases made much more frequently and conservatively (i.e. totally binary
compatibility with no qualifications, no new features at all, etc), these
can be 2.1.x and we can get to 2.1.937 for all it matters, and would be the
only thing on the "stable branch" of the repository.  The mainline
development version (trunk in the repository) can add features and do
reorganizations, but still do releases fairly often (2.2.x, and not be
hesitant about this x getting large too), and reasonably have both stable
and current branches going and widely used for a good while.  Major
many-month rewrites such as the new locale implementation can be their own
branch off the trunk, updated from the trunk as needed by the person
hacking that branch, and pieces merged into the trunk from there as they
become a little less volatile; other people interested in tracking the
highly volatile new implementation on the branch can track that version
too, but when it's close to working and not changing incompatibly every
other day or being wholly restructured internally every week, it can get
merged into the "unstable" mainline.

But these details of organization will get figured out easily enough
when there are enough serious resources (paid hacker time) devoted to 
getting libc done right.  Considering the current amount of support (such as I
understand it), we are doing pretty damn well already.

Re: glibc release management

[Thorsten Kukuk]

Hi,

On Thu, Jan 13, Roland McGrath wrote:

> You will get no argument from me about shorter release cycles, but I am not
> someone doing much of the work.  There are many details to bicker about,
> but I'm sure that the crux of Ulrich's reply will also be simply about the
> time he has available to put into libc.  
> 
> There are multiple employees of all the major distributors on this list,
> and it is well past time for those monied companies to put some serious
> investment explicitly into glibc maintenance and development.  Those of you
> working for distributors, I don't know how much of your paid job is
> supposed to have to do with glibc, but as far as I am aware there is no
> company paying anyone explicitly to work on the libc team per se (as
> opposed to their distribution's modified glibc package).

Wrong, SuSE has 3 developer, one is nearly fulltime working on glibc,
one if he has time or if there are bigger problems and one for
the distribuitotn modified glibc package.

We make the offer to Ulrich that Andreas Jaeger could maintain
the bug fix release, but Ulrich said no.

> 
> We can have more frequent releases, and better release management, but we
> cannot get it with Ulrich in charge of it alone unless that becomes his
> sole paid job (and I am presuming he would rather keep doing other things
> for Cygnus/Redhat instead of just glibc).  What we got now from Ulrich is
> far better than one could presume to expect from anyone without paying to
> have it be job 1.  Anyone complaining, complain to your boss or your boss's
> boss and don't come back here complaining until your company is paying
> someone (or better, part of a multi-company detente that pays someone) to be
> distribution-independent GNU libc maintainer/release-manager full time.
> 

We are not complaining because our boss will not pay someone, because he 
will do it. That is not the problem. 

> On the pseudotechnical front, what I would like see is the "bug-fix"
> releases made much more frequently and conservatively (i.e. totally binary
> compatibility with no qualifications, no new features at all, etc), these
> can be 2.1.x and we can get to 2.1.937 for all it matters, and would be the
> only thing on the "stable branch" of the repository.  The mainline
> development version (trunk in the repository) can add features and do
> reorganizations, but still do releases fairly often (2.2.x, and not be
> hesitant about this x getting large too), and reasonably have both stable
> and current branches going and widely used for a good while.  Major
> many-month rewrites such as the new locale implementation can be their own
> branch off the trunk, updated from the trunk as needed by the person
> hacking that branch, and pieces merged into the trunk from there as they
> become a little less volatile; other people interested in tracking the
> highly volatile new implementation on the branch can track that version
> too, but when it's close to working and not changing incompatibly every
> other day or being wholly restructured internally every week, it can get
> merged into the "unstable" mainline.
> 

That is exactly what I wrote in a shorter form.

> But these details of organization will get figured out easily enough
> when there are enough serious resources (paid hacker time) devoted to 
> getting libc done right.  Considering the current amount of support (such as I
> understand it), we are doing pretty damn well already.

Getting paid hacker is not the problem. 

  Thorsten

-- 
Thorsten Kukuk       http://www.suse.de/~kukuk/       kukuk@suse.de
SuSE GmbH            Schanzaeckerstr. 10            90443 Nuernberg
Linux is like a Vorlon.  It is incredibly powerful, gives terse,
cryptic answers and has a lot of things going on in the background.

Re: glibc release management

[Cristian Gafton]

On Thu, 13 Jan 2000, Roland McGrath wrote:

> But these details of organization will get figured out easily enough
> when there are enough serious resources (paid hacker time) devoted to 
> getting libc done right.  Considering the current amount of support (such as I
> understand it), we are doing pretty damn well already.

While it is true that for Red Hat we do not have somebody doing 100% glibc
hacking, there are several people within the company that are hacking on
bits and pieces from glibc. I'll not get into specifics because this is
not a contest who has more people and doing what.

The problem is not really having that much developer time spent on it as
it is finding the right people willing to do the job. You need somebody
good at it, that Ulrich can trust. But maintaining the stable branch is
*boring*. There aren't that many people really willing to do the job (or
willing to do it full time and right).

It is hard to argue whether the current state of things is good or not.
Now bug-fixes get backported from the development branch, and sometime
this means that there are new features that have to be added to support
the bug fixes. One can argue that we could seek independent fixes for the
stable branch, thus introducing incompatibilities between the stable and
the development branch. There is really no clear cut to what's best.

I think we all need to exercise better judgement when recommending
patches for the stable branch. After all, Ulrich is overworked already and
he might sometime just trust somebody else when a certain bugfix is
recommended. If I remember right, it wasn't Ulrich that proposed the
setrlimit changes for glibc 2.1.3. Regardless of whether he bought or not
into that, we all need to exercise due care when talking about patching
glibc 2.1.x, Even if Ulrich maintains sole control over glibc, he needs to
have a number of peers that he can trust on recommending the right things
when it comes to the stable branch.

And there is one more thing - when we talk about the stable branch
sometimes there are differences between what the right thing to do is and
what is expected out of a distribution. The stable branch is used by
distributions. They depend on its stability and compatibility between
snapshots. I am all for doing changes in the developmenbt code "because it
is the right thing to do", but we have to bend the rules a little bit for
the stable branch, so that people working for the Linux companies on glibc
will actually work on *developing* glibc rather than continuosly hacking
on ways to revert incompatible changes, changed behaviors, stuff that make
the ISVs go crazy.

I don't know about SuSE (or others) but at Red Hat people working on glibc
are spending more time than they should tracking down fixes that break
applications, building wrappers and doing hacks to get stuff that used to
work running again. Now, I'd love to have these resources diverted to
actual development.

Cristian
--
----------------------------------------------------------------------
Cristian Gafton    --     gafton@redhat.com     --       Red Hat, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 UNIX is user friendly. It's just selective about who its friends are.

Re: glibc release management

[Thorsten Kukuk]

Hi,

On Fri, Jan 14, Cristian Gafton wrote:

> The problem is not really having that much developer time spent on it as
> it is finding the right people willing to do the job. You need somebody
> good at it, that Ulrich can trust. But maintaining the stable branch is
> *boring*. There aren't that many people really willing to do the job (or
> willing to do it full time and right).

I think this is the result from having only a few people really working
on glibc. If we could find more people who would work on glibc, it should
be much simpler to find one who is good enough and willing to do the job.


[...]

> I think we all need to exercise better judgement when recommending
> patches for the stable branch. After all, Ulrich is overworked already and
> he might sometime just trust somebody else when a certain bugfix is
> recommended. If I remember right, it wasn't Ulrich that proposed the
> setrlimit changes for glibc 2.1.3. Regardless of whether he bought or not

No, it was Ulrich who add the setrlimit changes for glibc 2.1.3. After
this breaks a lot of appplications and architectures, Ulrichs comment
to me was "we need it" and Andreas Schwab and I tried to fix it.

[...]

> I don't know about SuSE (or others) but at Red Hat people working on glibc
> are spending more time than they should tracking down fixes that break
> applications, building wrappers and doing hacks to get stuff that used to
> work running again. Now, I'd love to have these resources diverted to
> actual development.

C++ (the various libstdc++ versions) to maintain makes much more trouble
then glibc. Often it is not enough to copy this versions from an
older release, you need to recompile them with special hacks.

Ok, SuSE avoided glibc 2.1 and started with glibc 2.1.1, but with the
step to glibc 2.1.2 we have only one real problem with linuxthreads.
This is less then I expect, but more then should.

With glibc 2.1.3 the current situation is more worse. Not from
binary compatiblity, but there are far to much patches which breaks
to many other plattforms.

  Thorsten

-- 
Thorsten Kukuk       http://www.suse.de/~kukuk/       kukuk@suse.de
SuSE GmbH            Schanzaeckerstr. 10            90443 Nuernberg
Linux is like a Vorlon.  It is incredibly powerful, gives terse,
cryptic answers and has a lot of things going on in the background.

Re: the setrlimit changes in glibc 2.1.3

[Andreas Jaeger]

>>>>> Roland McGrath writes:

 > So I think the real question here is, what warranted this kind of change in
 > a "bug-fix" release?  I think folks around here have gotten pretty unclear
 > on the concept of maintenance releases, and everybody's playing fast and loose.

After working some more time with the 32bit UID support, I'd like to
apologize for proposing the 32bit UID support for glibc 2.1.x.  I was
outvetoed - but now I'm convinced that the patches shouldn't go into
2.1[1].

 > This is just not the way to operate.  That's why we have a development
 > branch, for pete's sake.  Are we going to have to add another level of ".x"
 > to the versions to get a truly conservative maintenance release branch?

 > And, Mark: I'm afraid the realities of packaging systems (and of humans)
 > make it hopelessly unwise to attempt to have "the right thing" for anyone
 > be to upgrade their shared libraries differently than they upgrade their
 > development environment.

Andreas

Footnotes: 
[1]  But they should go into 2.2 to get further testing.
-- 
 Andreas Jaeger
  SuSE Labs aj@suse.de
   private aj@arthur.rhein-neckar.de

Re: glibc release management

[Thorsten Kukuk]

Hi,

On Fri, Jan 14, Geoff Keating wrote:

> > Date: Fri, 14 Jan 2000 09:45:41 +0100
> > From: Thorsten Kukuk <kukuk@suse.de>
> 
> > With glibc 2.1.3 the current situation is more worse. Not from
> > binary compatiblity, but there are far to much patches which breaks
> > to many other plattforms.
> 
> I looked at the ChangeLog between 2.1.2 and 2.1.3, just to check this.
> 
> There were something like 300 changes (which would sort of confirm
> previous comments about how we need to make maintenance releases more
> often...) but I only saw two (excluding Hurd patches) which I thought
> would be too extensive to be suitable for such a release in general,
> because of the likelihood they would cause problems.  One was the
> change to setrlimit(), the other was the powerpc ld.so changes (which
> I put in myself).

I don't speak about such big changes. I speak about all the little
changes which break other platforms. Ever tried to compile the
current glibc 2.1.3 cvs snapshot on PowerPC or Sparc without
patches ? Or use the current version on Intel ?
Today the libc only works on Alpha without problems and patches.
Sparc (fix from Jakub exists) and PowerPC doesn't compile, mmap
on Intel seg.faults (you need the glibc 2.2 fix).

  Thorsten

-- 
Thorsten Kukuk       http://www.suse.de/~kukuk/       kukuk@suse.de
SuSE GmbH            Schanzaeckerstr. 10            90443 Nuernberg
Linux is like a Vorlon.  It is incredibly powerful, gives terse,
cryptic answers and has a lot of things going on in the background.

Re: the setrlimit changes in glibc 2.1.3

[Roland McGrath]

> After working some more time with the 32bit UID support, I'd like to
> apologize for proposing the 32bit UID support for glibc 2.1.x.  I was
> outvetoed - but now I'm convinced that the patches shouldn't go into
> 2.1[1].

There is no need to apologize for the proposal.  The important thing is
that you were ready to listen to those of us who thought it was a bad idea.
No harm done.  One should never apologize for being wrong the first time, 
only if one insists on staying wrong.