linux libio solution

linux libio solution

[Richard Henderson]

I've just successfully built a version lib libstdc++ on Linux/Alpha
against stock Red Hat libraries.  The technique I used is applicable
to all of the Linux libio problems.

We build all of libio, ld -r it into one big .o, and use objcopy to
make all of the non-iostreams symbols local, so that (1) they are
not exported and (2) the iostreams functions are forced to use the
local copy.  The drawback is that "cout << 1" now includes a 2M
object file.  The mitigating factor is that if you use shared libs,
you won't notice.

At the moment, I've done this by hand, and so still need to turn it
into something that can be used in a Makefile.  Further, I had to
modify binutils to add the privitization option (5 lines of code).

Our choice here is either (1) require a binutils snapshot [eww], or
(2) write a special program that tinkers with ELF symbol tables
directly, and include that in the distribution.

Shall I be writing that special program, then?


r~

Re: linux libio solution

[H.J. Lu]

> 
> I've just successfully built a version lib libstdc++ on Linux/Alpha
> against stock Red Hat libraries.  The technique I used is applicable
> to all of the Linux libio problems.
> 
> We build all of libio, ld -r it into one big .o, and use objcopy to
> make all of the non-iostreams symbols local, so that (1) they are
> not exported and (2) the iostreams functions are forced to use the
> local copy.  The drawback is that "cout << 1" now includes a 2M
> object file.  The mitigating factor is that if you use shared libs,
> you won't notice.
> 
> At the moment, I've done this by hand, and so still need to turn it
> into something that can be used in a Makefile.  Further, I had to
> modify binutils to add the privitization option (5 lines of code).
> 
> Our choice here is either (1) require a binutils snapshot [eww], or
> (2) write a special program that tinkers with ELF symbol tables
> directly, and include that in the distribution.
> 
> Shall I be writing that special program, then?

I am not sure if it will work very. Have you tried mixed stdio/iostream
code? Are they in sync. For Linux, cout == stdout.

I will try to find a kludge this weekend.


H.J.

Re: linux libio solution

[Richard Henderson]

On Thu, Oct 09, 1997 at 08:15:57AM -0700, H.J. Lu wrote:
> I am not sure if it will work very. Have you tried mixed stdio/iostream
> code?

Yes.

> Are they in sync. For Linux, cout == stdout.

Well, because it was line buffered to the screen, yes.  But if
there are any problems, it will be just the same as any other host
where stdio and iostreams are separate.


r~

Re: linux libio solution

[H.J. Lu]

> 
> On Thu, Oct 09, 1997 at 08:15:57AM -0700, H.J. Lu wrote:
> > I am not sure if it will work very. Have you tried mixed stdio/iostream
> > code?
> 
> Yes.
> 
> > Are they in sync. For Linux, cout == stdout.
> 
> Well, because it was line buffered to the screen, yes.  But if
> there are any problems, it will be just the same as any other host
> where stdio and iostreams are separate.
> 

It is not that simple. For one thing, you will lose long double I/O.

Anyway, I have made a small kludge to make it to work with libc
5.3.12. But I will need someone to test it for me. I will make
it available after it passes "make check". Right now, as reported
by a few people, basic_string is broken. I will wait for a fix.


-- 
H.J. Lu (hjl@gnu.ai.mit.edu)

Re: linux libio solution

[Richard Henderson]

On Thu, Oct 09, 1997 at 03:52:45PM -0700, H.J. Lu wrote:
> It is not that simple.  For one thing, you will lose long double I/O.

Gee.  That merely sounds like a quality of implementation issue.
"If you upgrade it will work better, but if you don't it won't
completely fail."  This is perfectly acceptible.

> Anyway, I have made a small kludge to make it to work with libc 5.3.12.

Chances are high that it won't work with the glibc 1.98 961212
snapshot that is used by Red Hat on Linux/Alpha.  My solution
should work everywhere.


r~

Re: linux libio solution

[H.J. Lu]

> 
> On Thu, Oct 09, 1997 at 03:52:45PM -0700, H.J. Lu wrote:
> > It is not that simple.  For one thing, you will lose long double I/O.
> 
> Gee.  That merely sounds like a quality of implementation issue.
> "If you upgrade it will work better, but if you don't it won't
> completely fail."  This is perfectly acceptible.
> 
> > Anyway, I have made a small kludge to make it to work with libc 5.3.12.
> 
> Chances are high that it won't work with the glibc 1.98 961212
> snapshot that is used by Red Hat on Linux/Alpha.  My solution
> should work everywhere.
> 

I don't see why if it is treated as glibc 1. My kludge merely
replaces those buggy parts and leave the good ones alone.

-- 
H.J. Lu (hjl@gnu.ai.mit.edu)

Re: linux libio solution

[Robert Wilhelm]

On Thu, Oct 09, 1997 at 08:15:57AM -0700, H.J. Lu wrote:
> 
> I am not sure if it will work very. Have you tried mixed stdio/iostream
> code? Are they in sync. For Linux, cout == stdout.
>

What's so important that stdio/iostream can be mixed?

This "feature" does not work on any other platform and
has caused so much harm.

IMHO we should sacrifice it and provide to distinct implementations
for libc and libstdc++.

Robert




 

Re: linux libio solution

[Per Bothner]

> What's so important that stdio/iostream can be mixed?

Because buffered output otherwise output comes in surprising order,
which confuses or annoys people.

Because the standard actually does requires synchronized stdio/iostream
in certain cases (for stdout/cout etc), and doing so without a
tried stdio/iostream implementation is quite expensive.

Because it adds extra useful functionality to stdio (including a
docuemted way to implement features that have long existed in glibc),
and it reduces duplicated functionality.

> This "feature" does not work on any other platform 

I doubt that.  I certainly works in some situations, since the standard
requiries that.

> and has caused so much harm.

"Harm"?  You don't know what you're talking about, or at the
least are using very poorly chosen words.  "Harm" implies a
misfeature that hurts users.  Mixing stdio/iostream is a
win for users.  But, by definition, if the stdio and iostream
implementation are tied together, it will complicate trying
to update one without the other.  We are discussing whether it
OK to require users to update one if they update the other, or
if we could/should work out a way so they can only update iostream
without updating stdio.  It is a wild leap to suggest that it is
"harmful" that they are tied together.

	--Per Bothner
Cygnus Solutions     bothner@cygnus.com     http://www.cygnus.com/~bothner

Re: linux libio solution

[Robert Wilhelm]

> "Harm"?  You don't know what you're talking about, or at the
> least are using very poorly chosen words.  "Harm" implies a
> misfeature that hurts users.  Mixing stdio/iostream is a
> win for users.  But, by definition, if the stdio and iostream
> implementation are tied together, it will complicate trying
> to update one without the other.  We are discussing whether it
> OK to require users to update one if they update the other, or
> if we could/should work out a way so they can only update iostream
> without updating stdio.  It is a wild leap to suggest that it is
> "harmful" that they are tied together.
>

Sorry if this was the wrong word. I am no native english speaker.

All I wanted to express is that the mixed stdio/iostream
on linux has caused problems in the past and I think
an implementation, where libc and libstd++ are not
tied so strong together, would be a win.


On all supported commercial platforms it is possible to
install a new g++/libg++ without changing libc.

Only on linux that is not possible. 
You are always forced to upgrade libc.
Upgrading libc even between minor levels has
caused much problems in the past.
All libg++ releases did not work unmodified for Linux. 

Re: linux libio solution

[Per Bothner]

> Only on linux that is not possible. 
> You are always forced to upgrade libc.

That should not be true, and I don't think it really is.
It is just that people have not put the effort needed
to make it easy to upgrade libstdc++ without also
upgrading libc.  H.J.Lu, who has made the Linux libc and
g++ releases in the past has not seen that as a priority.
Ulrich (who is making current glibc releases), also seems
to not see it as a priority.  I tend to think that it is
important to get this to work for egcs releases.  It is not
necessarily a show-stopper (it is OK to require people to
upgrade libc if the alternative is too much work), but if
there is a reasonable solution (and there seems to be), we
should make it work.

> Upgrading libc even between minor levels has
> caused much problems in the past.
> All libg++ releases did not work unmodified for Linux. 

I agree this is not good.  The basic general solution is some variant
of what Richard Henderson proposed:  Provide a way to install iostreams,
with its private working copy of the libio code.  In some cases,
we can do better than that, which is what H.J.Lu is working on.

	--Per Bothner
Cygnus Solutions     bothner@cygnus.com     http://www.cygnus.com/~bothner

Re: linux libio solution

[Paul Koning]

>>>>> "Robert" == Robert Wilhelm <robert@physiol.med.tu-muenchen.de> writes:

 Robert> On Thu, Oct 09, 1997 at 08:15:57AM -0700, H.J. Lu wrote:
 >>  I am not sure if it will work very. Have you tried mixed
 >> stdio/iostream code? Are they in sync. For Linux, cout == stdout.
 >> 

 Robert> What's so important that stdio/iostream can be mixed?

I can think of several reasons.

1. It is documented as working.
2. It is used by many existing programs.
3. It is absolutely essential if you want to have mixed c/c++ code.

 Robert> This "feature" does not work on any other platform and has
 Robert> caused so much harm.

Huh? It absolutely DOES work on other platforms.  And what harm does
it cause?

 Robert> IMHO we should sacrifice it and provide to distinct
 Robert> implementations for libc and libstdc++.

NO.

	paul

Re: linux libio solution

[Joe Buck]

>  Robert> What's so important that stdio/iostream can be mixed?
> 
> I can think of several reasons.
> 
> 1. It is documented as working.
> 2. It is used by many existing programs.

But no conforming (to the draft standard) programs will break.

> 3. It is absolutely essential if you want to have mixed c/c++ code.

If you wish to write a portable program, it is possible to mix
stdin/out/err and cin/out/err but only if you use the sync_with_stdio
call.  In many implementations this causes a speed penalty (basically,
the iostreams operations flush all the time to make sure that the
output will mix right.  The Linux implementation has the advantage
that no speed penalty applies: cout and stdout are really the same
object.

>  Robert> This "feature" does not work on any other platform and has
>  Robert> caused so much harm.
> 
> Huh? It absolutely DOES work on other platforms.  And what harm does
> it cause?

The harm has been caused because of certain annoying details, not because
of the basic idea.  It really does make sense.  The problem has been
that libc has used a version of the libstdc++ stdio, but there have
generally been some differences; there also have been some changes in
the interface.  The result is that the FSF libg++ has never quite
worked with Linux and has always needed patches.

>  Robert> IMHO we should sacrifice it and provide to distinct
>  Robert> implementations for libc and libstdc++.
> 
> NO.

In an ideal world where libio has a fixed, immutable API, there would
be no reason to do this.  The problem is that there have been changes
in the interface (to support long double I/O, plus some other
re-architecting).

One possibility would be to have two modes of operation:

* one in which libio is shared, like now, and libstdc++ does not include
  libio but calls the Linux libio routines.  This would be suitable for
  "production" but would require recent libraries.  In the future, the
  interfaces between libstdc++ and libio would be kept fixed so that
  future libstdc++'s just "drop in".

* one where libstdc++ has its own internal libio, as it does on other
  OSes (e.g. Solaris).  Possibly some prefix would be added to the
  symbols so that they will not clash with the libio that is in libc.
  This mode would allow use of egcs on older Linux boxes (or pretty
  much any Linux box).  There would be a penalty in memory consumption
  for this, but it would only be paid once for the whole system (if
  shared libraries are used).

With the second scheme, there are separate buffers for cout and stdout.
Programs can use both if they do their own flushing and/or use
sync_with_stdio.

Re: linux libio solution

[Per Bothner]

Joe Buck <jbuck@synopsys.com> writes:
> If you wish to write a portable program, it is possible to mix
> stdin/out/err and cin/out/err but only if you use the sync_with_stdio
> call.

According to the draft standard, sync_with_stdio is initally *on*.
(There is actually some current discussion about what the draft
actually says, but it seems clear that the *intention* is that
synchronization is the default.)

	--Per Bothner
Cygnus Solutions     bothner@cygnus.com     http://www.cygnus.com/~bothner