Building binutils, egcs, gdb, etc

Building binutils, egcs, gdb, etc

[rich-paul]

I've been working on a HOWTO for how to bootstrap build a linux system
from scratch.  One annoyance that I've run into is building the gnu
tools.

I built binutils, egcs, and gdb ( Versions 2.9.1, 1.1.2, and 4.17
respectively).

Here is the issue:  Each of these packages want to build and install their
own versions of libiberty and bfd, and gdb wants to use a private libreadline,
rather than the one that I had already installed and used for bash. (It
does not appear to like the --use-installed-readline which bash uses ).

It seems to me that the way to handle this would be to have a gnu-base
distribution, which contains a libiberty and bfd, and seperate distributions
for binutils, egcs, and gdb which use them.

What do you think about this idea?  So far I am only posting to egcs
because it is the only one to which I am subscribed, but feel free to cross-
post as you think appropriate.  This would require a cooporative effort.

Also, there is one problem with configure ... it uses but does not save
the environment's LDFLAGS and CPPFLAGS ... methinks these should be treated
like CFLAGS and CXXFLAGS, since if they are set and configure is run, the
correctness of configure's output depends on them being the same.

For that matter, it might be a good idea to use /usr/bin/env to clear
environment varibles that are NOT saved, I think that this would make
configure much more reliable.

Regards,
Rich

---
There is a party that  |  Libertarian Party  |  A victimless crime is
supports the right to  |  http://www.lp.org  |     a contradiction in 
free speach and        |    The Party of     |                 terms.
encryption!!           |      Principal      |  

Re: Building binutils, egcs, gdb, etc

[Martin v. Loewis]

> I built binutils, egcs, and gdb ( Versions 2.9.1, 1.1.2, and 4.17
> respectively).
> 
> Here is the issue: Each of these packages want to build and install
> their own versions of libiberty and bfd, and gdb wants to use a
> private libreadline, rather than the one that I had already
> installed and used for bash.

I believe the standard approach is to put all packages into the same
source directory. Cygnus configure will look what sources are there,
and build everything. Then, all the packages will share libiberty and
libbfd.

Hope this helps,
Martin

Re: Building binutils, egcs, gdb, etc

[rich-paul]

I found one referance that said that this would happen if
'the sources were properly unpacked into the same directory'
but have never seen any description of how one would do that.

Takin for discussion just binutils and egcs, which configure script
should be used?  Which libiberty ( they differ ) Which libbfd?

Perhaps this is a work in progress?

---
There is a party that  |  Libertarian Party  |  A victimless crime is
supports the right to  |  http://www.lp.org  |     a contradiction in 
free speach and        |    The Party of     |                 terms.
encryption!!           |      Principal      |  

On Thu, 1 Apr 1999, Martin v. Loewis wrote:

> > I built binutils, egcs, and gdb ( Versions 2.9.1, 1.1.2, and 4.17
> > respectively).
> > 
> > Here is the issue: Each of these packages want to build and install
> > their own versions of libiberty and bfd, and gdb wants to use a
> > private libreadline, rather than the one that I had already
> > installed and used for bash.
> 
> I believe the standard approach is to put all packages into the same
> source directory. Cygnus configure will look what sources are there,
> and build everything. Then, all the packages will share libiberty and
> libbfd.
> 
> Hope this helps,
> Martin
> 

Re: Building binutils, egcs, gdb, etc

[Joe Buck]

> I found one referance that said that this would happen if
> 'the sources were properly unpacked into the same directory'
> but have never seen any description of how one would do that.

The key is that many of the GNU distributions are really subsets of
one massive distribution that knows how to build everything (pretty
much everything maintained primarily at Cygnus works this way).

Take a look at an unpacked egcs directory.  You see various files,
together with directories:

INSTALL, config, contrib, etc, gcc, include, libf2c, libiberty, libio,
libstdc++, texinfo, xiberty

Now do the same with the binutils distribution.  You see various
files, together with the directories

bfd, binutils, config, etc, gas, gprof, include, ld, libiberty, opcodes,
texinfo

If you look carefully, you'll see that many of these directories and
top-level files are the same.  What's more, the top-level Makefile is
prepared to build any set of tools underneath.  If you take the
directories bfd, binutils, gas, gprof, ld, and opcodes from the binutils
distribution and install them in the egcs tree, you can build everything
(the compiler, linker, assembler, and libraries) in one shot.

Similarly, you can combine the gdb, mmalloc, readline, sim, and utils
directory from the gdb distribution.

As I've said before, *warning*: if you combine distributions in this way,
and they were released months apart, each might have been tested with
slightly different libiberty, or bfd libraries: you can get something
labelled, say, gdb-4.17 but if it has a newer bfd library, the bugs might
be different (with some fixed and some added).

Re: Building binutils, egcs, gdb, etc

[Martin v. Loewis]

> I found one referance that said that this would happen if
> 'the sources were properly unpacked into the same directory'
> but have never seen any description of how one would do that.
> 
> Takin for discussion just binutils and egcs, which configure script
> should be used?  Which libiberty ( they differ ) Which libbfd?

These questions are exactly the reason why the instructions are so
weak. For the specific case, my guess would be
- the libiberty of egcs (it should have the latest demangler)
- the libbfd of binutils (there is none in egcs (?))
- any configure (both configures should support each other's subdirs)

> Perhaps this is a work in progress?

I guess it always is. I don't know exactly how this is maintained, but
sources tend to get out-of-synch sooner or later, and are
re-synchronized every now-and-then. This is also the reason why those
packages don't rely on pre-installed libiberty/libbfd: they can't
trust that the installed one really has all the fixes and features
that the package relies on.

I guess this feature is not used frequently enough so that anybody
really cares enough to publish maintainance information. If somebody
is building the chain for a number of platforms, I guess the make a
setup that works, then build everything, then throw their setup away.

So, if you get through, and feel like writing something up about it -
I'm sure the egcs web pages could host that information.

Regards,
Martin

Re: Building binutils, egcs, gdb, etc

[rich-paul]

OK, this makes some sense ... here are my suggestions as to where to 
go from here:

It seems to me like the best way to approach this situation would be to
factor out the common parts into a seperate distribution.  This would have
the following results ( in theory ).

Upsides:
	1) Each package would not have to maintain its own copy of these
		libraries.
	2) There could be meaningful versioning applied to these libraries
	3) One would download the libs first, and then whatever tools (s)he
		wanted to build, rather than downloading the libs with each
		tool.
	4) Bugs would be consistent
	5) The process could be disambiguated and demystified, so that perhaps
		to install a toolset you could get gdb, egcs, binutils, and base,
		and just do the following:
	tar -xzf /archive/base.tgz ( creates gnutools-baseversion/{libiberty,bfd} )
	tar -xzf /archive/egcs-core.tgz ( creates gnutools-baseversion/gcc )
	tar -xzf /archive/egcs-g++.tgz ( creates gnutools-baseversion/gcc/cp )
	tar -xzf /archive/binutils.tgz ( creates gnutools-baseversion/whatever )

	Note that this would require no duplicate files between these
	distributions, with ./configure, etc living in the base package.  It would
	also become obvious when you got a non-compatable tool ... if you unpack
	the thing, and it goes into gnutools-baseversion+1/whatever, you know
	that you need a new version of the base utils, and you can go get 'em.

Downsides:
	1) I'm not sure how this would effect people needing to add stuff to
	libiberty or libbfd.

	2) Things in libiberty would not be customizable to fit a particular
	application, they'd have to be generic. ( On second thought, maybe
	this is an upside? )

	3) Users would have to download one more package.
---
There is a party that  |  Libertarian Party  |  A victimless crime is
supports the right to  |  http://www.lp.org  |     a contradiction in 
free speach and        |    The Party of     |                 terms.
encryption!!           |      Principal      |  

On Thu, 1 Apr 1999, Joe Buck wrote:

> 
> > I found one referance that said that this would happen if
> > 'the sources were properly unpacked into the same directory'
> > but have never seen any description of how one would do that.
> 
> The key is that many of the GNU distributions are really subsets of
> one massive distribution that knows how to build everything (pretty
> much everything maintained primarily at Cygnus works this way).
> 
> Take a look at an unpacked egcs directory.  You see various files,
> together with directories:
> 
> INSTALL, config, contrib, etc, gcc, include, libf2c, libiberty, libio,
> libstdc++, texinfo, xiberty
> 
> Now do the same with the binutils distribution.  You see various
> files, together with the directories
> 
> bfd, binutils, config, etc, gas, gprof, include, ld, libiberty, opcodes,
> texinfo
> 
> If you look carefully, you'll see that many of these directories and
> top-level files are the same.  What's more, the top-level Makefile is
> prepared to build any set of tools underneath.  If you take the
> directories bfd, binutils, gas, gprof, ld, and opcodes from the binutils
> distribution and install them in the egcs tree, you can build everything
> (the compiler, linker, assembler, and libraries) in one shot.
> 
> Similarly, you can combine the gdb, mmalloc, readline, sim, and utils
> directory from the gdb distribution.
> 
> As I've said before, *warning*: if you combine distributions in this way,
> and they were released months apart, each might have been tested with
> slightly different libiberty, or bfd libraries: you can get something
> labelled, say, gdb-4.17 but if it has a newer bfd library, the bugs might
> be different (with some fixed and some added).
> 

Re: Building binutils, egcs, gdb, etc

[rich-paul]

I'll see what I can do on that score.

---
There is a party that  |  Libertarian Party  |  A victimless crime is
supports the right to  |  http://www.lp.org  |     a contradiction in 
free speach and        |    The Party of     |                 terms.
encryption!!           |      Principal      |  

On Fri, 2 Apr 1999, Martin v. Loewis wrote:

> > I found one referance that said that this would happen if
> > 'the sources were properly unpacked into the same directory'
> > but have never seen any description of how one would do that.
> > 
> > Takin for discussion just binutils and egcs, which configure script
> > should be used?  Which libiberty ( they differ ) Which libbfd?
> 
> These questions are exactly the reason why the instructions are so
> weak. For the specific case, my guess would be
> - the libiberty of egcs (it should have the latest demangler)
> - the libbfd of binutils (there is none in egcs (?))
> - any configure (both configures should support each other's subdirs)
> 
> > Perhaps this is a work in progress?
> 
> I guess it always is. I don't know exactly how this is maintained, but
> sources tend to get out-of-synch sooner or later, and are
> re-synchronized every now-and-then. This is also the reason why those
> packages don't rely on pre-installed libiberty/libbfd: they can't
> trust that the installed one really has all the fixes and features
> that the package relies on.
> 
> I guess this feature is not used frequently enough so that anybody
> really cares enough to publish maintainance information. If somebody
> is building the chain for a number of platforms, I guess the make a
> setup that works, then build everything, then throw their setup away.
> 
> So, if you get through, and feel like writing something up about it -
> I'm sure the egcs web pages could host that information.
> 
> Regards,
> Martin
> 

Re: Building binutils, egcs, gdb, etc

[Doug Semler]

> > I found one referance that said that this would happen if
> > 'the sources were properly unpacked into the same directory'
> > but have never seen any description of how one would do that.
> > 
> > Takin for discussion just binutils and egcs, which configure script
> > should be used?  Which libiberty ( they differ ) Which libbfd?
> 
> These questions are exactly the reason why the instructions are so
> weak. For the specific case, my guess would be
> - the libiberty of egcs (it should have the latest demangler)
> - the libbfd of binutils (there is none in egcs (?))
> - any configure (both configures should support each other's subdirs)
> 
> > Perhaps this is a work in progress?
> 
> I guess it always is. I don't know exactly how this is maintained, but
> sources tend to get out-of-synch sooner or later, and are
> re-synchronized every now-and-then. This is also the reason why those
> packages don't rely on pre-installed libiberty/libbfd: they can't
> trust that the installed one really has all the fixes and features
> that the package relies on.
> 
> I guess this feature is not used frequently enough so that anybody
> really cares enough to publish maintainance information. If somebody
> is building the chain for a number of platforms, I guess the make a
> setup that works, then build everything, then throw their setup away.
> 
> So, if you get through, and feel like writing something up about it -
> I'm sure the egcs web pages could host that information.
> 
> Regards,
> Martin


I don't think the feature is used much at all....I tried to 
do a complete build of the following just for the fun of it,
since we have multiple platforms which we want to keep the same
development environment on.  It used to be that you couldn't mix-and-match
the newer bfd from binutils with gdb, I don't know if that's still the
case...I just built and ran a make check-gdb.  The demangle.exp failed,
for obvious reasons (it's using a different demangler thatn it is expecting,
and the pthreads test failed as well...but other than that it passed

  autoconf 2.13, automake 1.4, binutils 2.9.1, bison 1.27, cvs 1.10.5,
  dejagnu snapshot 981026, diffutils 2.7, egcs 1.1.2, flex 2.5.4a, 
  gzip snapshot 1.2.4.4294967305, gdb 4.17, libtool 1.2g (CVS tree),
  m4 1.4, make 3.77, rcs 5.7, tar 1.12, texinfo 3.12
on:
  mips-sgi-irix6.2, mips-sgi-irix6.5,
  sparc-sun-solaris2.5.1, sparc-sun-solaris2.7,
  powerpc-ibm-aix4.2.0.0, hppa1.1-hp-hpux10.20
  i486-pc-linux-gnu

I'm a little ambitious....:)

But I ran into a couple of minor glitches
-) You have to be careful when mixing binutils with egcs trees, the headers
   from binutils/include you want to use, along with the directories bfd,
   opcodes, gas, ld, gprof and binutils.  But you don't want to use 
   binutils/texinfo or binutils/libiberty (texinfo is replaced by the 
   package, and the library in egcs uses the correct demangler).  
   And when moving dejagnu over, you only want to move the directories
   dejagnu, expect, and tcl. And right now, I only moved gdb, sim, readline,
   and utils from the gdb tree
-) Speaking of texinfo, every package includes its own version of texinfo.tex,
   but I have my own version I like to use...
-) the Makefile.in doesn't include make targets for libtool, understandable
   since it's so new.
-) The root makefile insists on setting a flag to bison: -L $srcdir/bison
   to get the bison support files from the current source rather than the
   installed ones (if any). Problem is the current version
   (1.27) doesn't have this flag...it uses environment variables
   BISON_SIMPLE and BISON_HAIRY to determine overrides to the install
   directory.
-) Binutils shouldn't be used on AIX, but they get built anyway if the 
   sources exist...
-) The reason for using the alpha gzip snapshot is that 1.2.4 is using
   a real old configure, and doesn't recognize the --with-gcc-version-trigger
   flag passed through to it.
-) The root Makefile.in specifies at least one incorrect directory for
   tools (eg M4 is set to $$r/m4/m4, but it is built in $$r/m4/src/m4,
   so when autoconf tries to make, it fails...)
-) The libtool included in egcs is bad, I have to use the alpha version
   (right now I use the one from the CVS repository on gnu.org, 1.2g)
   So I have to build a copy outside the tree first and re-libtoolize.
-) If I want to use shared libraries for all but libstdc++, I have to
   manually edit the root Makefile to remove the enable-shared from the
   CONFIG_ARGUMENTS line so that it doesn't get passed when configuring 
   the library after running configure but before running make. (The
   reason for this is we don't want to distribute a shared libstdc++ with
   our software...)
-) RCS may not get configure correctly if GNU diff is not already installed.
   (I don't know about this one though, I haven't checked it)
-) Bootstrapping w/o gnu make already installed sucks :)
-) Making the info and dvi documentation is inconsistent across all the
   packages :(


---
Doug Semler                       | doug@seaspace.com
SeaSpace Corporation              | Garbage In -- Gospel Out
Least Senior Software Developer;  | Minister of things to do Next Quarter
Low Man on the Totem Pole         | (but will Never Be Done) DNRC  O-
A closed mind is a terrible thing | Bus Error (passengers dumped)
  
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS/M d---(pu) s++:- a-- C++ UILSH+++$ P--- L++ E--- W+
N++ o-- K? w--(++$) O- M-- V- PS+ !PE Y PGP t(+) 5+++ X+
R- tv+(-) b+(++) DI++++ D G e++>++++ h!>--- r% y+>+++++**
------END GEEK CODE BLOCK------

Re: Building binutils, egcs, gdb, etc

[Jeffrey A Law]

  In message < Pine.LNX.3.96.990401143540.3398D-100000@nitrogen.colossus.net >you
 write:
  > OK, this makes some sense ... here are my suggestions as to where to 
  > go from here:
  > 
  > It seems to me like the best way to approach this situation would be to
  > factor out the common parts into a seperate distribution.  This would have
  > the following results ( in theory ).
In general this doesn't work because bfd still changes rapidly.  In fact
we do _not_ recommend trying to share bfd across binutils & gdb releases
since the bfd included in each of those releases is usually different
enough to cause problems.

If you happen to be working from snapshots sharing a bfd tree between
gdb & binutils is a lot more likely to work :-)

jeff

Re: Building binutils, egcs, gdb, etc

[Jeffrey A Law]

  In message < 199904020242.SAA17721@vortex.seaspace.com >you write:
  > I don't think the feature is used much at all....
Not within the egcs community.  But within Cygnus, we use it every day.

jeff

Re: Building binutils, egcs, gdb, etc

[David Edelsohn]

	I and most people building cross-compilers use the feature of
combining egcs, binutils, and newlib all of the time.

	As Jeff said, bfd changes too rapidly and has too much
dependencies in both gdb and binutils to combine those two packages.  I
normally build gdb separately.

David

Re: Building binutils, egcs, gdb, etc

[Martin v. Loewis]

> It seems to me like the best way to approach this situation would be to
> factor out the common parts into a seperate distribution.  This would have
> the following results ( in theory ).
[...]
> Downsides:

You forgot an important one

    4) Somebody would have to maintain it, over a period of many
       years, and be quite responsive to requests of the maintainers
       of the various packages, as well as to problems end-users have
       with it. Otherwise, maintainers will just grab a copy of the
       base package, integrate it, and modify it to their needs.

I don't see anybody stepping forward to do that. Of course, there is
also

    5) Packages are more difficult to install for typical
       end-users. End-users don't build the entire chain at one time,
       but they build gdb whenever a new version is release, and
       binutils whenver that is updated. They now download the source
       and compile. Under your scheme, they also have find out what
       else they need (i.e. the base package), and where they can get
       it.

Regards,
Martin

Re: Building binutils, egcs, gdb, etc

[rich-paul]

You're right, on both counts.  But neither of these seems inconsistant
with the requirements of current open-source projects ( my god, look at
gnome ... I had to install gobs of packages ), and it seems to
be worth pursuing from my point of view, as the Mozilla people would say,
"For the good of the code".  One problem with the current approach besides
package size and build time is that there is always for the potential for
mismatches between binutils and gcc, and a programmer working on both
projects would need to know that implementations of 'the same' function
may vary between projects.  Not to mention that projects written with 
binutils need to be read with gdb and visa-versa.

May I ask why these libraries are in constant flux?  It *seems* like their
purpose is pretty cut and dried, but it could be that I do not understand
the problem they solve well enough to know.

Regards,
Rich

P.S.  I would be willing to maintain either or both of these libraries,
if there is interest in changing to this method of distribution.  I've been
looking to find a nitch in a good opensource project, and the project seems
like a worthy one to me.  I'm a c++ guy, but I can do k&r, it's just like
watching a movie in black and white.  <G>

---
There is a party that  |  Libertarian Party  |  A victimless crime is
supports the right to  |  http://www.lp.org  |     a contradiction in 
free speach and        |    The Party of     |                 terms.
encryption!!           |      Principal      |  

On Fri, 2 Apr 1999, Martin v. Loewis wrote:

> > It seems to me like the best way to approach this situation would be to
> > factor out the common parts into a seperate distribution.  This would have
> > the following results ( in theory ).
> [...]
> > Downsides:
> 
> You forgot an important one
> 
>     4) Somebody would have to maintain it, over a period of many
>        years, and be quite responsive to requests of the maintainers
>        of the various packages, as well as to problems end-users have
>        with it. Otherwise, maintainers will just grab a copy of the
>        base package, integrate it, and modify it to their needs.
> 
> I don't see anybody stepping forward to do that. Of course, there is
> also
> 
>     5) Packages are more difficult to install for typical
>        end-users. End-users don't build the entire chain at one time,
>        but they build gdb whenever a new version is release, and
>        binutils whenver that is updated. They now download the source
>        and compile. Under your scheme, they also have find out what
>        else they need (i.e. the base package), and where they can get
>        it.
> 
> Regards,
> Martin
> 

Re: Building binutils, egcs, gdb, etc

[rich-paul]

Well, I guess I've come up with a plan of attack.  The first thing that
I'm going to do is a "liberty" project, which will attempt to create
a library that will provide the union of the functionality availiable in
the current libiberties, and be suitable for use as a shared library.

More generally, it will try to smooth over some platform differances,
providing common functionality in a way that will reduce the size of any
project that uses it be replacing code like

// example taken from xterm from xfree86, probably exists in some
// variation in expect and other projects.
#ifdef HAS_OPENPTY
	return openpty(...);
#else
	.. 70 lines of code to emulate openpty on differant systems
#endif

with always using openpty, and having it emulated in libiberty if it
is not provided by libc.  Basicly a general compatability library.

Hopefully use of this will spread to more or all opensource projects,
so they can remove their specific x-platform hacks and move them into
libiberty, reducing the amount of work requuired by configure by a great
extent, as well as the amount of preprocessor work required in client code,
and create a 'freeware common infrastructure'.

Once I get the basics from the existing libiberties, I'll be asking people
to try removing their various libiberties from various existing projects,
and testing with mine, and hopefully eventually removing libiberties from
their distributions entirely.

To those of you who have applied hacks to existing libiberties, please
let me know if there are any cases where you needed special functionality
rather than just fixed bugs in existing code.  Also, if somebody could tell
me where the original copy of egcs/libiberty came from, it would be a great
help.  Is there any repository of libiberty code that is synchronized?

Regards,
Rich Paul

P.S. I'm approaching this first, because it appears to be low-hanging fruit,
and applicable outside the projects that are it's source.  I'll probably
go ahead, when I have got this to a stable state, and attack libbfd, but that
requires a great deal of domain knowledge and possibly redesign, which I'm
not ready to do yet.

---
There is a party that  |  Libertarian Party  |  A victimless crime is
supports the right to  |  http://www.lp.org  |     a contradiction in 
free speach and        |    The Party of     |                 terms.
encryption!!           |      Principal      |  

On Fri, 2 Apr 1999 rich-paul@rich-paul.net wrote:

> You're right, on both counts.  But neither of these seems inconsistant
> with the requirements of current open-source projects ( my god, look at
> gnome ... I had to install gobs of packages ), and it seems to
> be worth pursuing from my point of view, as the Mozilla people would say,
> "For the good of the code".  One problem with the current approach besides
> package size and build time is that there is always for the potential for
> mismatches between binutils and gcc, and a programmer working on both
> projects would need to know that implementations of 'the same' function
> may vary between projects.  Not to mention that projects written with 
> binutils need to be read with gdb and visa-versa.
> 
> May I ask why these libraries are in constant flux?  It *seems* like their
> purpose is pretty cut and dried, but it could be that I do not understand
> the problem they solve well enough to know.
> 
> Regards,
> Rich
> 
> P.S.  I would be willing to maintain either or both of these libraries,
> if there is interest in changing to this method of distribution.  I've been
> looking to find a nitch in a good opensource project, and the project seems
> like a worthy one to me.  I'm a c++ guy, but I can do k&r, it's just like
> watching a movie in black and white.  <G>
> 
> ---
> There is a party that  |  Libertarian Party  |  A victimless crime is
> supports the right to  |  http://www.lp.org  |     a contradiction in 
> free speach and        |    The Party of     |                 terms.
> encryption!!           |      Principal      |  
> 
> On Fri, 2 Apr 1999, Martin v. Loewis wrote:
> 
> > > It seems to me like the best way to approach this situation would be to
> > > factor out the common parts into a seperate distribution.  This would have
> > > the following results ( in theory ).
> > [...]
> > > Downsides:
> > 
> > You forgot an important one
> > 
> >     4) Somebody would have to maintain it, over a period of many
> >        years, and be quite responsive to requests of the maintainers
> >        of the various packages, as well as to problems end-users have
> >        with it. Otherwise, maintainers will just grab a copy of the
> >        base package, integrate it, and modify it to their needs.
> > 
> > I don't see anybody stepping forward to do that. Of course, there is
> > also
> > 
> >     5) Packages are more difficult to install for typical
> >        end-users. End-users don't build the entire chain at one time,
> >        but they build gdb whenever a new version is release, and
> >        binutils whenver that is updated. They now download the source
> >        and compile. Under your scheme, they also have find out what
> >        else they need (i.e. the base package), and where they can get
> >        it.
> > 
> > Regards,
> > Martin
> > 
> 

Re: Building binutils, egcs, gdb, etc

[rich-paul]

You're right, on both counts.  But neither of these seems inconsistant
with the requirements of current open-source projects ( my god, look at
gnome ... I had to install gobs of packages ), and it seems to
be worth pursuing from my point of view, as the Mozilla people would say,
"For the good of the code".  One problem with the current approach besides
package size and build time is that there is always for the potential for
mismatches between binutils and gcc, and a programmer working on both
projects would need to know that implementations of 'the same' function
may vary between projects.  Not to mention that projects written with 
binutils need to be read with gdb and visa-versa.

May I ask why these libraries are in constant flux?  It *seems* like their
purpose is pretty cut and dried, but it could be that I do not understand
the problem they solve well enough to know.

Regards,
Rich

P.S.  I would be willing to maintain either or both of these libraries,
if there is interest in changing to this method of distribution.  I've been
looking to find a nitch in a good opensource project, and the project seems
like a worthy one to me.  I'm a c++ guy, but I can do k&r, it's just like
watching a movie in black and white.  <G>

---
There is a party that  |  Libertarian Party  |  A victimless crime is
supports the right to  |  http://www.lp.org  |     a contradiction in 
free speach and        |    The Party of     |                 terms.
encryption!!           |      Principal      |  

On Fri, 2 Apr 1999, Martin v. Loewis wrote:

> > It seems to me like the best way to approach this situation would be to
> > factor out the common parts into a seperate distribution.  This would have
> > the following results ( in theory ).
> [...]
> > Downsides:
> 
> You forgot an important one
> 
>     4) Somebody would have to maintain it, over a period of many
>        years, and be quite responsive to requests of the maintainers
>        of the various packages, as well as to problems end-users have
>        with it. Otherwise, maintainers will just grab a copy of the
>        base package, integrate it, and modify it to their needs.
> 
> I don't see anybody stepping forward to do that. Of course, there is
> also
> 
>     5) Packages are more difficult to install for typical
>        end-users. End-users don't build the entire chain at one time,
>        but they build gdb whenever a new version is release, and
>        binutils whenver that is updated. They now download the source
>        and compile. Under your scheme, they also have find out what
>        else they need (i.e. the base package), and where they can get
>        it.
> 
> Regards,
> Martin
> 

Building binutils, egcs, gdb, etc

[rich-paul]

I've been working on a HOWTO for how to bootstrap build a linux system
from scratch.  One annoyance that I've run into is building the gnu
tools.

I built binutils, egcs, and gdb ( Versions 2.9.1, 1.1.2, and 4.17
respectively).

Here is the issue:  Each of these packages want to build and install their
own versions of libiberty and bfd, and gdb wants to use a private libreadline,
rather than the one that I had already installed and used for bash. (It
does not appear to like the --use-installed-readline which bash uses ).

It seems to me that the way to handle this would be to have a gnu-base
distribution, which contains a libiberty and bfd, and seperate distributions
for binutils, egcs, and gdb which use them.

What do you think about this idea?  So far I am only posting to egcs
because it is the only one to which I am subscribed, but feel free to cross-
post as you think appropriate.  This would require a cooporative effort.

Also, there is one problem with configure ... it uses but does not save
the environment's LDFLAGS and CPPFLAGS ... methinks these should be treated
like CFLAGS and CXXFLAGS, since if they are set and configure is run, the
correctness of configure's output depends on them being the same.

For that matter, it might be a good idea to use /usr/bin/env to clear
environment varibles that are NOT saved, I think that this would make
configure much more reliable.

Regards,
Rich

---
There is a party that  |  Libertarian Party  |  A victimless crime is
supports the right to  |  http://www.lp.org  |     a contradiction in 
free speach and        |    The Party of     |                 terms.
encryption!!           |      Principal      |  

Re: Building binutils, egcs, gdb, etc

[Doug Semler]

> > I found one referance that said that this would happen if
> > 'the sources were properly unpacked into the same directory'
> > but have never seen any description of how one would do that.
> > 
> > Takin for discussion just binutils and egcs, which configure script
> > should be used?  Which libiberty ( they differ ) Which libbfd?
> 
> These questions are exactly the reason why the instructions are so
> weak. For the specific case, my guess would be
> - the libiberty of egcs (it should have the latest demangler)
> - the libbfd of binutils (there is none in egcs (?))
> - any configure (both configures should support each other's subdirs)
> 
> > Perhaps this is a work in progress?
> 
> I guess it always is. I don't know exactly how this is maintained, but
> sources tend to get out-of-synch sooner or later, and are
> re-synchronized every now-and-then. This is also the reason why those
> packages don't rely on pre-installed libiberty/libbfd: they can't
> trust that the installed one really has all the fixes and features
> that the package relies on.
> 
> I guess this feature is not used frequently enough so that anybody
> really cares enough to publish maintainance information. If somebody
> is building the chain for a number of platforms, I guess the make a
> setup that works, then build everything, then throw their setup away.
> 
> So, if you get through, and feel like writing something up about it -
> I'm sure the egcs web pages could host that information.
> 
> Regards,
> Martin


I don't think the feature is used much at all....I tried to 
do a complete build of the following just for the fun of it,
since we have multiple platforms which we want to keep the same
development environment on.  It used to be that you couldn't mix-and-match
the newer bfd from binutils with gdb, I don't know if that's still the
case...I just built and ran a make check-gdb.  The demangle.exp failed,
for obvious reasons (it's using a different demangler thatn it is expecting,
and the pthreads test failed as well...but other than that it passed

  autoconf 2.13, automake 1.4, binutils 2.9.1, bison 1.27, cvs 1.10.5,
  dejagnu snapshot 981026, diffutils 2.7, egcs 1.1.2, flex 2.5.4a, 
  gzip snapshot 1.2.4.4294967305, gdb 4.17, libtool 1.2g (CVS tree),
  m4 1.4, make 3.77, rcs 5.7, tar 1.12, texinfo 3.12
on:
  mips-sgi-irix6.2, mips-sgi-irix6.5,
  sparc-sun-solaris2.5.1, sparc-sun-solaris2.7,
  powerpc-ibm-aix4.2.0.0, hppa1.1-hp-hpux10.20
  i486-pc-linux-gnu

I'm a little ambitious....:)

But I ran into a couple of minor glitches
-) You have to be careful when mixing binutils with egcs trees, the headers
   from binutils/include you want to use, along with the directories bfd,
   opcodes, gas, ld, gprof and binutils.  But you don't want to use 
   binutils/texinfo or binutils/libiberty (texinfo is replaced by the 
   package, and the library in egcs uses the correct demangler).  
   And when moving dejagnu over, you only want to move the directories
   dejagnu, expect, and tcl. And right now, I only moved gdb, sim, readline,
   and utils from the gdb tree
-) Speaking of texinfo, every package includes its own version of texinfo.tex,
   but I have my own version I like to use...
-) the Makefile.in doesn't include make targets for libtool, understandable
   since it's so new.
-) The root makefile insists on setting a flag to bison: -L $srcdir/bison
   to get the bison support files from the current source rather than the
   installed ones (if any). Problem is the current version
   (1.27) doesn't have this flag...it uses environment variables
   BISON_SIMPLE and BISON_HAIRY to determine overrides to the install
   directory.
-) Binutils shouldn't be used on AIX, but they get built anyway if the 
   sources exist...
-) The reason for using the alpha gzip snapshot is that 1.2.4 is using
   a real old configure, and doesn't recognize the --with-gcc-version-trigger
   flag passed through to it.
-) The root Makefile.in specifies at least one incorrect directory for
   tools (eg M4 is set to $$r/m4/m4, but it is built in $$r/m4/src/m4,
   so when autoconf tries to make, it fails...)
-) The libtool included in egcs is bad, I have to use the alpha version
   (right now I use the one from the CVS repository on gnu.org, 1.2g)
   So I have to build a copy outside the tree first and re-libtoolize.
-) If I want to use shared libraries for all but libstdc++, I have to
   manually edit the root Makefile to remove the enable-shared from the
   CONFIG_ARGUMENTS line so that it doesn't get passed when configuring 
   the library after running configure but before running make. (The
   reason for this is we don't want to distribute a shared libstdc++ with
   our software...)
-) RCS may not get configure correctly if GNU diff is not already installed.
   (I don't know about this one though, I haven't checked it)
-) Bootstrapping w/o gnu make already installed sucks :)
-) Making the info and dvi documentation is inconsistent across all the
   packages :(


---
Doug Semler                       | doug@seaspace.com
SeaSpace Corporation              | Garbage In -- Gospel Out
Least Senior Software Developer;  | Minister of things to do Next Quarter
Low Man on the Totem Pole         | (but will Never Be Done) DNRC  O-
A closed mind is a terrible thing | Bus Error (passengers dumped)
  
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS/M d---(pu) s++:- a-- C++ UILSH+++$ P--- L++ E--- W+
N++ o-- K? w--(++$) O- M-- V- PS+ !PE Y PGP t(+) 5+++ X+
R- tv+(-) b+(++) DI++++ D G e++>++++ h!>--- r% y+>+++++**
------END GEEK CODE BLOCK------

Re: Building binutils, egcs, gdb, etc

[Jeffrey A Law]

  In message < 199904020242.SAA17721@vortex.seaspace.com >you write:
  > I don't think the feature is used much at all....
Not within the egcs community.  But within Cygnus, we use it every day.

jeff

Re: Building binutils, egcs, gdb, etc

[rich-paul]

OK, this makes some sense ... here are my suggestions as to where to 
go from here:

It seems to me like the best way to approach this situation would be to
factor out the common parts into a seperate distribution.  This would have
the following results ( in theory ).

Upsides:
	1) Each package would not have to maintain its own copy of these
		libraries.
	2) There could be meaningful versioning applied to these libraries
	3) One would download the libs first, and then whatever tools (s)he
		wanted to build, rather than downloading the libs with each
		tool.
	4) Bugs would be consistent
	5) The process could be disambiguated and demystified, so that perhaps
		to install a toolset you could get gdb, egcs, binutils, and base,
		and just do the following:
	tar -xzf /archive/base.tgz ( creates gnutools-baseversion/{libiberty,bfd} )
	tar -xzf /archive/egcs-core.tgz ( creates gnutools-baseversion/gcc )
	tar -xzf /archive/egcs-g++.tgz ( creates gnutools-baseversion/gcc/cp )
	tar -xzf /archive/binutils.tgz ( creates gnutools-baseversion/whatever )

	Note that this would require no duplicate files between these
	distributions, with ./configure, etc living in the base package.  It would
	also become obvious when you got a non-compatable tool ... if you unpack
	the thing, and it goes into gnutools-baseversion+1/whatever, you know
	that you need a new version of the base utils, and you can go get 'em.

Downsides:
	1) I'm not sure how this would effect people needing to add stuff to
	libiberty or libbfd.

	2) Things in libiberty would not be customizable to fit a particular
	application, they'd have to be generic. ( On second thought, maybe
	this is an upside? )

	3) Users would have to download one more package.
---
There is a party that  |  Libertarian Party  |  A victimless crime is
supports the right to  |  http://www.lp.org  |     a contradiction in 
free speach and        |    The Party of     |                 terms.
encryption!!           |      Principal      |  

On Thu, 1 Apr 1999, Joe Buck wrote:

> 
> > I found one referance that said that this would happen if
> > 'the sources were properly unpacked into the same directory'
> > but have never seen any description of how one would do that.
> 
> The key is that many of the GNU distributions are really subsets of
> one massive distribution that knows how to build everything (pretty
> much everything maintained primarily at Cygnus works this way).
> 
> Take a look at an unpacked egcs directory.  You see various files,
> together with directories:
> 
> INSTALL, config, contrib, etc, gcc, include, libf2c, libiberty, libio,
> libstdc++, texinfo, xiberty
> 
> Now do the same with the binutils distribution.  You see various
> files, together with the directories
> 
> bfd, binutils, config, etc, gas, gprof, include, ld, libiberty, opcodes,
> texinfo
> 
> If you look carefully, you'll see that many of these directories and
> top-level files are the same.  What's more, the top-level Makefile is
> prepared to build any set of tools underneath.  If you take the
> directories bfd, binutils, gas, gprof, ld, and opcodes from the binutils
> distribution and install them in the egcs tree, you can build everything
> (the compiler, linker, assembler, and libraries) in one shot.
> 
> Similarly, you can combine the gdb, mmalloc, readline, sim, and utils
> directory from the gdb distribution.
> 
> As I've said before, *warning*: if you combine distributions in this way,
> and they were released months apart, each might have been tested with
> slightly different libiberty, or bfd libraries: you can get something
> labelled, say, gdb-4.17 but if it has a newer bfd library, the bugs might
> be different (with some fixed and some added).
> 

Re: Building binutils, egcs, gdb, etc

[rich-paul]

I found one referance that said that this would happen if
'the sources were properly unpacked into the same directory'
but have never seen any description of how one would do that.

Takin for discussion just binutils and egcs, which configure script
should be used?  Which libiberty ( they differ ) Which libbfd?

Perhaps this is a work in progress?

---
There is a party that  |  Libertarian Party  |  A victimless crime is
supports the right to  |  http://www.lp.org  |     a contradiction in 
free speach and        |    The Party of     |                 terms.
encryption!!           |      Principal      |  

On Thu, 1 Apr 1999, Martin v. Loewis wrote:

> > I built binutils, egcs, and gdb ( Versions 2.9.1, 1.1.2, and 4.17
> > respectively).
> > 
> > Here is the issue: Each of these packages want to build and install
> > their own versions of libiberty and bfd, and gdb wants to use a
> > private libreadline, rather than the one that I had already
> > installed and used for bash.
> 
> I believe the standard approach is to put all packages into the same
> source directory. Cygnus configure will look what sources are there,
> and build everything. Then, all the packages will share libiberty and
> libbfd.
> 
> Hope this helps,
> Martin
> 

Re: Building binutils, egcs, gdb, etc

[Jeffrey A Law]

  In message < Pine.LNX.3.96.990401143540.3398D-100000@nitrogen.colossus.net >you
 write:
  > OK, this makes some sense ... here are my suggestions as to where to 
  > go from here:
  > 
  > It seems to me like the best way to approach this situation would be to
  > factor out the common parts into a seperate distribution.  This would have
  > the following results ( in theory ).
In general this doesn't work because bfd still changes rapidly.  In fact
we do _not_ recommend trying to share bfd across binutils & gdb releases
since the bfd included in each of those releases is usually different
enough to cause problems.

If you happen to be working from snapshots sharing a bfd tree between
gdb & binutils is a lot more likely to work :-)

jeff

Re: Building binutils, egcs, gdb, etc

[David Edelsohn]

	I and most people building cross-compilers use the feature of
combining egcs, binutils, and newlib all of the time.

	As Jeff said, bfd changes too rapidly and has too much
dependencies in both gdb and binutils to combine those two packages.  I
normally build gdb separately.

David

Re: Building binutils, egcs, gdb, etc

[rich-paul]

Well, I guess I've come up with a plan of attack.  The first thing that
I'm going to do is a "liberty" project, which will attempt to create
a library that will provide the union of the functionality availiable in
the current libiberties, and be suitable for use as a shared library.

More generally, it will try to smooth over some platform differances,
providing common functionality in a way that will reduce the size of any
project that uses it be replacing code like

// example taken from xterm from xfree86, probably exists in some
// variation in expect and other projects.
#ifdef HAS_OPENPTY
	return openpty(...);
#else
	.. 70 lines of code to emulate openpty on differant systems
#endif

with always using openpty, and having it emulated in libiberty if it
is not provided by libc.  Basicly a general compatability library.

Hopefully use of this will spread to more or all opensource projects,
so they can remove their specific x-platform hacks and move them into
libiberty, reducing the amount of work requuired by configure by a great
extent, as well as the amount of preprocessor work required in client code,
and create a 'freeware common infrastructure'.

Once I get the basics from the existing libiberties, I'll be asking people
to try removing their various libiberties from various existing projects,
and testing with mine, and hopefully eventually removing libiberties from
their distributions entirely.

To those of you who have applied hacks to existing libiberties, please
let me know if there are any cases where you needed special functionality
rather than just fixed bugs in existing code.  Also, if somebody could tell
me where the original copy of egcs/libiberty came from, it would be a great
help.  Is there any repository of libiberty code that is synchronized?

Regards,
Rich Paul

P.S. I'm approaching this first, because it appears to be low-hanging fruit,
and applicable outside the projects that are it's source.  I'll probably
go ahead, when I have got this to a stable state, and attack libbfd, but that
requires a great deal of domain knowledge and possibly redesign, which I'm
not ready to do yet.

---
There is a party that  |  Libertarian Party  |  A victimless crime is
supports the right to  |  http://www.lp.org  |     a contradiction in 
free speach and        |    The Party of     |                 terms.
encryption!!           |      Principal      |  

On Fri, 2 Apr 1999 rich-paul@rich-paul.net wrote:

> You're right, on both counts.  But neither of these seems inconsistant
> with the requirements of current open-source projects ( my god, look at
> gnome ... I had to install gobs of packages ), and it seems to
> be worth pursuing from my point of view, as the Mozilla people would say,
> "For the good of the code".  One problem with the current approach besides
> package size and build time is that there is always for the potential for
> mismatches between binutils and gcc, and a programmer working on both
> projects would need to know that implementations of 'the same' function
> may vary between projects.  Not to mention that projects written with 
> binutils need to be read with gdb and visa-versa.
> 
> May I ask why these libraries are in constant flux?  It *seems* like their
> purpose is pretty cut and dried, but it could be that I do not understand
> the problem they solve well enough to know.
> 
> Regards,
> Rich
> 
> P.S.  I would be willing to maintain either or both of these libraries,
> if there is interest in changing to this method of distribution.  I've been
> looking to find a nitch in a good opensource project, and the project seems
> like a worthy one to me.  I'm a c++ guy, but I can do k&r, it's just like
> watching a movie in black and white.  <G>
> 
> ---
> There is a party that  |  Libertarian Party  |  A victimless crime is
> supports the right to  |  http://www.lp.org  |     a contradiction in 
> free speach and        |    The Party of     |                 terms.
> encryption!!           |      Principal      |  
> 
> On Fri, 2 Apr 1999, Martin v. Loewis wrote:
> 
> > > It seems to me like the best way to approach this situation would be to
> > > factor out the common parts into a seperate distribution.  This would have
> > > the following results ( in theory ).
> > [...]
> > > Downsides:
> > 
> > You forgot an important one
> > 
> >     4) Somebody would have to maintain it, over a period of many
> >        years, and be quite responsive to requests of the maintainers
> >        of the various packages, as well as to problems end-users have
> >        with it. Otherwise, maintainers will just grab a copy of the
> >        base package, integrate it, and modify it to their needs.
> > 
> > I don't see anybody stepping forward to do that. Of course, there is
> > also
> > 
> >     5) Packages are more difficult to install for typical
> >        end-users. End-users don't build the entire chain at one time,
> >        but they build gdb whenever a new version is release, and
> >        binutils whenver that is updated. They now download the source
> >        and compile. Under your scheme, they also have find out what
> >        else they need (i.e. the base package), and where they can get
> >        it.
> > 
> > Regards,
> > Martin
> > 
> 

Re: Building binutils, egcs, gdb, etc

[rich-paul]

I'll see what I can do on that score.

---
There is a party that  |  Libertarian Party  |  A victimless crime is
supports the right to  |  http://www.lp.org  |     a contradiction in 
free speach and        |    The Party of     |                 terms.
encryption!!           |      Principal      |  

On Fri, 2 Apr 1999, Martin v. Loewis wrote:

> > I found one referance that said that this would happen if
> > 'the sources were properly unpacked into the same directory'
> > but have never seen any description of how one would do that.
> > 
> > Takin for discussion just binutils and egcs, which configure script
> > should be used?  Which libiberty ( they differ ) Which libbfd?
> 
> These questions are exactly the reason why the instructions are so
> weak. For the specific case, my guess would be
> - the libiberty of egcs (it should have the latest demangler)
> - the libbfd of binutils (there is none in egcs (?))
> - any configure (both configures should support each other's subdirs)
> 
> > Perhaps this is a work in progress?
> 
> I guess it always is. I don't know exactly how this is maintained, but
> sources tend to get out-of-synch sooner or later, and are
> re-synchronized every now-and-then. This is also the reason why those
> packages don't rely on pre-installed libiberty/libbfd: they can't
> trust that the installed one really has all the fixes and features
> that the package relies on.
> 
> I guess this feature is not used frequently enough so that anybody
> really cares enough to publish maintainance information. If somebody
> is building the chain for a number of platforms, I guess the make a
> setup that works, then build everything, then throw their setup away.
> 
> So, if you get through, and feel like writing something up about it -
> I'm sure the egcs web pages could host that information.
> 
> Regards,
> Martin
> 

Re: Building binutils, egcs, gdb, etc

[Martin v. Loewis]

> I found one referance that said that this would happen if
> 'the sources were properly unpacked into the same directory'
> but have never seen any description of how one would do that.
> 
> Takin for discussion just binutils and egcs, which configure script
> should be used?  Which libiberty ( they differ ) Which libbfd?

These questions are exactly the reason why the instructions are so
weak. For the specific case, my guess would be
- the libiberty of egcs (it should have the latest demangler)
- the libbfd of binutils (there is none in egcs (?))
- any configure (both configures should support each other's subdirs)

> Perhaps this is a work in progress?

I guess it always is. I don't know exactly how this is maintained, but
sources tend to get out-of-synch sooner or later, and are
re-synchronized every now-and-then. This is also the reason why those
packages don't rely on pre-installed libiberty/libbfd: they can't
trust that the installed one really has all the fixes and features
that the package relies on.

I guess this feature is not used frequently enough so that anybody
really cares enough to publish maintainance information. If somebody
is building the chain for a number of platforms, I guess the make a
setup that works, then build everything, then throw their setup away.

So, if you get through, and feel like writing something up about it -
I'm sure the egcs web pages could host that information.

Regards,
Martin

Re: Building binutils, egcs, gdb, etc

[Joe Buck]

> I found one referance that said that this would happen if
> 'the sources were properly unpacked into the same directory'
> but have never seen any description of how one would do that.

The key is that many of the GNU distributions are really subsets of
one massive distribution that knows how to build everything (pretty
much everything maintained primarily at Cygnus works this way).

Take a look at an unpacked egcs directory.  You see various files,
together with directories:

INSTALL, config, contrib, etc, gcc, include, libf2c, libiberty, libio,
libstdc++, texinfo, xiberty

Now do the same with the binutils distribution.  You see various
files, together with the directories

bfd, binutils, config, etc, gas, gprof, include, ld, libiberty, opcodes,
texinfo

If you look carefully, you'll see that many of these directories and
top-level files are the same.  What's more, the top-level Makefile is
prepared to build any set of tools underneath.  If you take the
directories bfd, binutils, gas, gprof, ld, and opcodes from the binutils
distribution and install them in the egcs tree, you can build everything
(the compiler, linker, assembler, and libraries) in one shot.

Similarly, you can combine the gdb, mmalloc, readline, sim, and utils
directory from the gdb distribution.

As I've said before, *warning*: if you combine distributions in this way,
and they were released months apart, each might have been tested with
slightly different libiberty, or bfd libraries: you can get something
labelled, say, gdb-4.17 but if it has a newer bfd library, the bugs might
be different (with some fixed and some added).

Re: Building binutils, egcs, gdb, etc

[Martin v. Loewis]

> It seems to me like the best way to approach this situation would be to
> factor out the common parts into a seperate distribution.  This would have
> the following results ( in theory ).
[...]
> Downsides:

You forgot an important one

    4) Somebody would have to maintain it, over a period of many
       years, and be quite responsive to requests of the maintainers
       of the various packages, as well as to problems end-users have
       with it. Otherwise, maintainers will just grab a copy of the
       base package, integrate it, and modify it to their needs.

I don't see anybody stepping forward to do that. Of course, there is
also

    5) Packages are more difficult to install for typical
       end-users. End-users don't build the entire chain at one time,
       but they build gdb whenever a new version is release, and
       binutils whenver that is updated. They now download the source
       and compile. Under your scheme, they also have find out what
       else they need (i.e. the base package), and where they can get
       it.

Regards,
Martin

Re: Building binutils, egcs, gdb, etc

[Martin v. Loewis]

> I built binutils, egcs, and gdb ( Versions 2.9.1, 1.1.2, and 4.17
> respectively).
> 
> Here is the issue: Each of these packages want to build and install
> their own versions of libiberty and bfd, and gdb wants to use a
> private libreadline, rather than the one that I had already
> installed and used for bash.

I believe the standard approach is to put all packages into the same
source directory. Cygnus configure will look what sources are there,
and build everything. Then, all the packages will share libiberty and
libbfd.

Hope this helps,
Martin