Re: Installation proposal

Re: Installation proposal

[Benjamin Kosnik]

I've long been a fan of any attempt to make the build procedure more
sane. Staging the libraries, includes, and binaries in an install
directory seems to make a great deal of sense.

Are you planning on posting the patches to implement this for comment
before they are checked in?

-benjamin

Re: Installation proposal

[Richard Henderson]

On Thu, Feb 28, 2002 at 06:43:58PM -0300, Alexandre Oliva wrote:
> Perhaps this is just the right time to move the whole bootstrap
> procedure out of gcc to the top level?  Then, we'd not only have
> separate pseudo-install trees for each stage, but also we'd get the
> added benefit of having the complete toolchain built (and optimized)
> by the toolchain itself.

That does indeed sound ideal.


r~

Re: Installation proposal

[Per Bothner]

Alexandre Oliva wrote:
> However, I think it may get the bootstrapping procedure somewhat more
> complicated.  I mean, I don't see how to fit stages into this idea.

One option is just do what we have always done, except that creating
stage1 mv $build_prefix/bin/gcc to stage1/gcc and so on.  I.e.
what we're actually doing is splitting the contents of "." into
two parts:
(a) temporary files (most .o files, gen*, stmp-* etc) remains in ".".
(b) files that will ultimately be installed are in sub-directories
of $build_prefix.

To create stage1, you can just gather together all the needed files
from both (a) and (b) and copy them to stage1.  (Of course you might
want to just delete the temporary files in "." instead.)

The better solution, once enough of the files have been converted to
use $build_prefix is to use $build_prefix for the stages.

I.e.:stage1:
   mkdir stage1
   make build_prefix=stage1
stage2:
   mkdir stage2
   make mostlyclean
   make build_prefix=stage2 PATH=`pwd`/stage1:$PATH
stage3:
   make mostlyclean
   make build_prefix=../build PATH=`pwd`/stage2:$PATH

I assume 'make mostlyclean' removes all the temporary files
in "." (such as gen* and *.o), but leave the files in
$build_prefix alone.  "make clean" also removes the files
in $build_prefix (that gcc has put there).

> Perhaps this is just the right time to move the whole bootstrap
> procedure out of gcc to the top level?  Then, we'd not only have
> separate pseudo-install trees for each stage, but also we'd get the
> added benefit of having the complete toolchain built (and optimized)
> by the toolchain itself.

That might be even better, depending on exactly what we want
bootstraping to accomplish.
-- 
	--Per Bothner
per@bothner.com   http://www.bothner.com/per/

Re: Installation proposal

[Alexandre Oliva]

On Feb 27, 2002, Daniel Jacobowitz <drow@mvista.com> wrote:

> On Wed, Feb 27, 2002 at 10:44:52PM -0300, Alexandre Oliva wrote:
>> > 3. We would test that our installations are really location-independent,
>> >    which they supposedly are.
>> 
>> Not really.  At least not cross toolchains that use a shared glibc as
>> the C library, because glibc's libc.so is a linker script that
>> references full pathnames into the install tree.  Yet another problem
>> about which we must make a conscious decision.

> Which works perfectly well if you remove the full paths and let the
> linker search this.

I wonder why glibc doesn't just do that, then...  I mean, this is a
bug in glibc, no?

> We're in the middle of fighting a losing war with libtool.  Right now,
> we edit installed .la libraries to remove the full path information
> from them.  I might add that, on GNU/Linux, this is entirely adequate;
> it would be much simpler if we could get libtool not to generate such
> things.  Ditto for DT_RPATH.

Patches to get libtool to take -L or -R flags out of link commands and
.la files would be definitely welcome.  It's been in my to-do list
forever.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist                Professional serial bug killer

Re: Installation proposal

[Alexandre Oliva]

On Feb 28, 2002, mike stump <mrs@windriver.com> wrote:

>> Date: Thu, 28 Feb 2002 09:04:06 -0800
>> From: Per Bothner <per@bothner.com>
>> To: Mark Mitchell <mark@codesourcery.com>
>> CC: "gcc@gcc.gnu.org" <gcc@gcc.gnu.org>

>> My suggestion:

>> (1) configure creates a 'build' sub-directory at the top-level,
>> parallel to 'gcc', libstdc++', etc.  The Makefiles call this
>> directory $build_prefix.

> I like this approach.

<aol>Me too</aol>

I mean, I *really* like it.  It keeps the distinction between
in-the-build-tree and in-the-install-tree that is important for
libtool to protect users from unintended RPATHs implicitly added by
stupid linkers.

However, I think it may get the bootstrapping procedure somewhat more
complicated.  I mean, I don't see how to fit stages into this idea.
Perhaps this is just the right time to move the whole bootstrap
procedure out of gcc to the top level?  Then, we'd not only have
separate pseudo-install trees for each stage, but also we'd get the
added benefit of having the complete toolchain built (and optimized)
by the toolchain itself.

Comments?

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist                Professional serial bug killer

Re: Installation proposal

[Mark Mitchell]

> they care to.  The risk of this approach is lower, and the end result
> I think it the same or superior.

I agree -- this was my original idea, and is the eventual place I
wanted to be.

> original apply to this methodology?  Any new objections against this
> methodology?

Not from me.  I just don't have enough time to do the whole thing,
and the incremental version doesn't have enough benefit for me until
it is done.

Still, I very much encourage anyone who is inclined to do this.

--
Mark Mitchell                   mark@codesourcery.com
CodeSourcery, LLC               http://www.codesourcery.com

Re: Installation proposal

[mike stump]

> Date: Thu, 28 Feb 2002 09:04:06 -0800
> From: Per Bothner <per@bothner.com>
> To: Mark Mitchell <mark@codesourcery.com>
> CC: "gcc@gcc.gnu.org" <gcc@gcc.gnu.org>

> My suggestion:

> (1) configure creates a 'build' sub-directory at the top-level,
> parallel to 'gcc', libstdc++', etc.  The Makefiles call this
> directory $build_prefix.

I like this approach.  One can do as small a job as they want, and it
will be self-consistent.  A multitude of people can work in parallel.
The semantics of make all and make install are clear and unchanged.
One can start this at any level and do as much or as little a job as
they care to.  The risk of this approach is lower, and the end result
I think it the same or superior.  Do the objections against the
original apply to this methodology?  Any new objections against this
methodology?

Re: Installation proposal

[Mark Mitchell]

--On Thursday, February 28, 2002 09:04:06 AM -0800 Per Bothner 
<per@bothner.com> wrote:

> It does look like trying to directly copy the pre-install tree
> into the real install tree is too fragile.  But I still think
> having a 'build' directory that mirrors the structure of the
> prefix directory makes sense, and we can get there incrementally.

Yes, this might work.  Sadly, it's more than I can take on right
now.

But perhaps you will have inspired someone!

--
Mark Mitchell                   mark@codesourcery.com
CodeSourcery, LLC               http://www.codesourcery.com

Re: Installation proposal

[Per Bothner]

It does look like trying to directly copy the pre-install tree
into the real install tree is too fragile.  But I still think
having a 'build' directory that mirrors the structure of the
prefix directory makes sense, and we can get there incrementally.

My suggestion:

(1) configure creates a 'build' sub-directory at the top-level,
parallel to 'gcc', libstdc++', etc.  The Makefiles call this
directory $build_prefix.

(2) Let's start with (say) gcc.  Change the gcc makefile so that
all instances of 'xgcc' instead say '$build_prefix/bin/gcc'.
Does this for all of:
(a) The -o opten when actually linking the executable.
(b) In 'install-driver'
(c) In STAGESTUFF.
(d) In the libstdc++ and libjava directories if the refer to xgcc.

At this point we have (hopefully) a working consistent system.
If we haven't broken anything, we convert more files.
(3) The remianing drivers:  g++, gcj.
(4) Other binaries: fastjar/jar -> $build_prefix/bin/jar.
(5) The actual compilers:  cc1, cc1plus, jc1, ...
(6) Other generated files.
(7) At some point we have enough converted that we can remove
the -B flags in the run-time library directories.
(8) We change the testsuite so it just uses
'PATH=$build_prefix/bin:$PATH gcc ...' instead of build_prefix/bin/gcc.
-- 
	--Per Bothner
per@bothner.com   http://www.bothner.com/per/

Re: Installation proposal

[Alexandre Oliva]

On Feb 28, 2002, "Joseph S. Myers" <jsm28@cam.ac.uk> wrote:

> On 27 Feb 2002, Alexandre Oliva wrote:
>> - on most other systems, when a libtool shared library is linked with
>> another libtool shared library, the latter still being in its build
>> tree.  Because libtool creates libraries in the build tree in such a
>> way that they can be linked with and used in-place, it encodes
>> build-tree pathnames into the library.  Then, at install time, it
>> re-links libraries such that the work in the install tree.  Again,
>> on HP-UX, DESTDIR installations don't work.

> libtool also encodes build tree paths into the .la files it installs (e.g.
> PR other/3525).  Why don't those get properly regenerated at install time?

Because libtool can't possibly know when an -L flag refers to a
build-tree pathname or to an install-tree pathname unless some
mechanism to tell it so is introduced in it.

> Is there a security issue here?

Yes.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist                Professional serial bug killer

Re: Installation proposal

[Joseph S. Myers]

On 27 Feb 2002, Alexandre Oliva wrote:

> - on most other systems, when a libtool shared library is linked with
>   another libtool shared library, the latter still being in its build
>   tree.  Because libtool creates libraries in the build tree in such a
>   way that they can be linked with and used in-place, it encodes
>   build-tree pathnames into the library.  Then, at install time, it
>   re-links libraries such that the work in the install tree.  Again,
>   on HP-UX, DESTDIR installations don't work.

libtool also encodes build tree paths into the .la files it installs (e.g.
PR other/3525).  Why don't those get properly regenerated at install time?
Is there a security issue here?

-- 
Joseph S. Myers
jsm28@cam.ac.uk

Re: Installation proposal

[Akim Demaille]

>>>>> "Per" == Per Bothner <per@bothner.com> writes:

Per> Mark Mitchell wrote:
>> What's good about this?

Per> 5. The Makefiles for builting libraries can be simpler: Just use
Per> "install/bin/g++", not "g++ -B... -nostdinc++ -I..."  (This is a
Per> variant of your point 1.)

It's funny: that's precisely what does CVS Autoconf (and Bison and CVS
M4).  Actually, we have wrappers in tests/ that point to the
non-installed executables, with all the needed magic for them to find
their peers and files.

As a result

1. the test suite runs autoconf etc. via the path, nothing is hard
   coded.

2. so when `make check', we just have PATH go into tests/ first, and
   the non installed tools are checked.

3. when `make distcheck' (i.e., dist + untar + configure
   --prefix=/somewhere + make + make check + make install + make installcheck)
   we have `installcheck' run the test suite on the installed tools
   (by the past, I *did* have installed programs that didn't work, but
   the non installed version did).

4. Bootstrapping is eased: Autoconf uses tests/autoconf,
   tests/autom4te etc. without special flags.  All the special tricks
   are located in one single place: the wrappers.
 

Re: Installation proposal

[Akim Demaille]

>>>>> "Alexandre" == Alexandre Oliva <aoliva@redhat.com> writes:

Alexandre> This will change the requirements imposed on exec_prefix.
Alexandre> According to the GNU Coding Standards, the user should be
Alexandre> able to modify prefix and exec_prefix at install time.
Alexandre> Currently, GCC already fails to follow this recommentation:
Alexandre> it depends on exec_prefix being a sub-directory of prefix,
Alexandre> and on the relative pathname from exec_prefix to prefix not
Alexandre> changing from build to install time.  By doing an
Alexandre> installation with something as simple as cp -R or the tar
Alexandre> or cpio equivalents, you lose the ability to modify
Alexandre> exec_prefix at all, even if it follows the current
Alexandre> requirements.

Alexandre> Not that I find this extremely important, but I thought I
Alexandre> point it out so that we may a conscious decision regarding
Alexandre> breaking backward compatibility with a feature that
Alexandre> probably nobody uses (namely exec_prefix from $prefix/foo
Alexandre> at build time to $prefix/bar at install time).

IMHO (but you know that :), it would be more useful to have this
recommendation withdrawn from the GCS...

Re: Installation proposal

[Richard Henderson]

On Wed, Feb 27, 2002 at 11:16:21PM -0300, Alexandre Oliva wrote:
> Ok, so if HP-UX is insane and Solaris is insane :-), are we just not
> going to support shared libraries on them?

Sure.  My patience for broken elf implementations (or reproductions
in the case of pa-som or alpha-ecoff) has grown thin.


r~

Re: Installation proposal

[Mark Mitchell]

> This will change the requirements imposed on exec_prefix.  According
> to the GNU Coding Standards, the user should be able to modify prefix
> and exec_prefix at install time.  Currently, GCC already fails to
> follow this recommentation: it depends on exec_prefix being a
> sub-directory of prefix, and on the relative pathname from exec_prefix
> to prefix not changing from build to install time.

Interesting.  If we keep that requirement, it actually makes things
easier: you can use that same relative in the "install" directory,
and then using exec_prefix should still work.

>> 3. We would test that our installations are really location-independent,
>>    which they supposedly are.
>
> Not really.  At least not cross toolchains that use a shared glibc as
> the C library, because glibc's libc.so is a linker script that
> references full pathnames into the install tree.  Yet another problem
> about which we must make a conscious decision.

Hmm, yes.  If we have post-install hooks (already pointed out as
necessary for install-info), then I suppose we could fix this.

>> We can just have the top-level configure set the prefix for all of the
>> child configures to "install".
>
> And now you can't move your build tree any longer

Not a big deal, surely.

> Also, you don't want to have installed programs looking for stuff in
> the build tree; that's a security threat.

Yes, this is a big deal.  Of course, this implies that the
location-independence isn't working, or that we're falling back
to $prefix when we can't find ourselves.  Which might be reasonable.
This seems like a serious issue.

==

Here are my conclusions:

1. You and Jim have persuaded me that this is complicated.

2. In fact, you've persuaded me that it is far too complicated.

   We support too many different build and installation modes.  We
   use too much goo to accomplish that.  The goo has hardened over
   the years to the point where it is hard to perturb it at all,
   even in such a way as to make the eventual result much simpler.

3. However, if I change things, I will probably be dealing with
   email from someone who wants to use exec_prefix when building a
   Candian cross to a PDP-11 using glibc's full-pathname linker
   script for the next six weeks.

4. Therefore, I shan't do anything.

   We'll just keep doing "-B... -I... -L..." with lots of horrible
   logic scattered around the tree to figure out whether we need
   "../../.." or just "../..", keep it hard to run
   our tests by hand (just what do I have to set LD_LIBRARY_PATH to
   again?), and so forth.

-- 
Mark Mitchell                mark@codesourcery.com
CodeSourcery, LLC            http://www.codesourcery.com

Re: Installation proposal

[Alexandre Oliva]

On Feb 27, 2002, Richard Henderson <rth@redhat.com> wrote:

> On Wed, Feb 27, 2002 at 10:28:38PM -0300, Alexandre Oliva wrote:
>> A sane system, to me, is one in which programs and libraries can find
>> their dependencies without the user having to resort to setting
>> LD_LIBRARY_PATH and *hoping* it will work, if they are installed in
>> the location they were configured for.

> IMO, there is *no* acceptible use of DT_RPATH, and the *only* 
> acceptible use of DT_RUNPATH must include the $ORIGIN token
> available on recent Solaris and Linux.  Otherwise it becomes
> impossible to move installations after the fact.

You choose not to listen to me.  I've told you already, more than
once: it does not become impossible to move installations.  It becomes
impossible to move the installation, then replace it with some other
installation, that contain shared libraries with the same SONAMEs,
then expect the moved installation to still work.  Think libc5
lossage.  If you do that, you lose.  If you just move installation,
LD_LIBRARY_PATH will do just fine, except for the caveats I mentioned
on certain insane dynamic linkers.

>> - on HPUX...

> "HPUX" and "sane" have never been uttered in the same sentence
> without accompanying negatives.

:-)

>> ... there are some dynamic linkers in which LD_LIBRARY_PATH
>> (or the equivalent) won't do what you want.

> Which is "broken" not "sane".

Ok, so if HP-UX is insane and Solaris is insane :-), are we just not
going to support shared libraries on them?

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist                Professional serial bug killer

Re: Installation proposal

[cgd]

At Thu, 28 Feb 2002 01:18:15 +0000 (UTC), "Richard Henderson" wrote:
At Thu, 28 Feb 2002 01:18:15 +0000 (UTC), "Richard Henderson" wrote:
> In which case is it then impossible to build a cross-binutils,
> install it, then build a cross-gcc?
> 
> Of course, it's almost certainly easier to do everything via
> a single-build tree, but this is a change in behaviour.

some would disagree.  8-)

it would be unfortunate to see the ability to build/install a
cross-binutils, then build an independent cross-gcc fall by the
wayside.  it would also make a lot of work for some people.  8-)



chris

Re: Installation proposal

[Daniel Jacobowitz]

On Wed, Feb 27, 2002 at 10:44:52PM -0300, Alexandre Oliva wrote:
> > 3. We would test that our installations are really location-independent,
> >    which they supposedly are.
> 
> Not really.  At least not cross toolchains that use a shared glibc as
> the C library, because glibc's libc.so is a linker script that
> references full pathnames into the install tree.  Yet another problem
> about which we must make a conscious decision.

Which works perfectly well if you remove the full paths and let the
linker search this.

I and others at MontaVista have done a significant amount of work
making HHL into a completely relocatable cross development environment. 
The only big patch left to the tools is to the linker to give
prefix-relative search directories (and GCC-style relocating based on
argv[0]).

We're in the middle of fighting a losing war with libtool.  Right now,
we edit installed .la libraries to remove the full path information
from them.  I might add that, on GNU/Linux, this is entirely adequate;
it would be much simpler if we could get libtool not to generate such
things.  Ditto for DT_RPATH.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: Installation proposal

[Mark Mitchell]

--On Wednesday, February 27, 2002 05:18:01 PM -0800 Richard Henderson 
<rth@redhat.com> wrote:

> On Wed, Feb 27, 2002 at 04:27:09PM -0800, Mark Mitchell wrote:
>> > As David Edelsohn mentioned, there could be problems if you have a
>> > previous installed tree with the same prefix.  The tree being tested
>> > may accidentally use stuff from the install tree if we aren't manually
>> > using -B/-L/-I/etc overrides.
>>
>> That would be a bug; the stuff to find the compiler and use its
>> current path should prevent this.
>
> In which case is it then impossible to build a cross-binutils,
> install it, then build a cross-gcc?

Because the cross-binutils gets installed in the final prefix, and
I'm claiming we wouldn't look there?  I see.

If look in the "install" directory first, then fall back to the
hardcoded prefix we're OK.  I don't know if we want to do that
fall-back -- that means that relocating the compiler leaves it with
some vague memory of where you said you were going to put it in the
first place.  That seems weird to me.

So, yes, I guess I'd be saying that the scenario you propose
wouldn't work, and you're right that this would be a change.

I think it's basically a design flaw to have so many different ways
of doing things.  If the right way to build a cross toolchain is
in a single tree, then that's what we should support.  There's no
reason to support every weird way of doing every weird thing; that
way lies hundreds of options and lots of weird interactions.  Oh,
wait, that's where we are now. :-)

--
Mark Mitchell                mark@codesourcery.com
CodeSourcery, LLC            http://www.codesourcery.com

Re: Installation proposal

[Alexandre Oliva]

On Feb 27, 2002, Mark Mitchell <mark@codesourcery.com> wrote:

> 1. Create a subdirectory of the objdir called "install".

> 2. Treat install as if it were the final prefix during building --
>    put everything in there.  That means that there will be directories
>    like "install/bin", "install/lib", and so forth.  So, "make" would
>    just drop everything in these directories.

> 3. At install time, run a single script which copies everything in
>    install into prefix.

This will change the requirements imposed on exec_prefix.  According
to the GNU Coding Standards, the user should be able to modify prefix
and exec_prefix at install time.  Currently, GCC already fails to
follow this recommentation: it depends on exec_prefix being a
sub-directory of prefix, and on the relative pathname from exec_prefix
to prefix not changing from build to install time.  By doing an
installation with something as simple as cp -R or the tar or cpio
equivalents, you lose the ability to modify exec_prefix at all, even
if it follows the current requirements.

Not that I find this extremely important, but I thought I point it out
so that we may a conscious decision regarding breaking backward
compatibility with a feature that probably nobody uses (namely
exec_prefix from $prefix/foo at build time to $prefix/bar at install
time).

> 3. We would test that our installations are really location-independent,
>    which they supposedly are.

Not really.  At least not cross toolchains that use a shared glibc as
the C library, because glibc's libc.so is a linker script that
references full pathnames into the install tree.  Yet another problem
about which we must make a conscious decision.

> We can just have the top-level configure set the prefix for all of the
> child configures to "install".

And now you can't move your build tree any longer (because prefix has
to be a full pathname; relative pathnames are not rejected outright as
they should, but they don't work).

Also, you don't want to have installed programs looking for stuff in
the build tree; that's a security threat.  But that's what you'll get
if you configure the default prefix as something within the build
tree.  Also, on a number of some systems, when you don't specify
dynamic search paths when linking an executable or a shared library,
they'll use -L flags or the location of libraries at build time as the
default search path for libraries which, again, is a security threat.

Modifying the configured prefix is an absolute no-no.  Doing a DESTDIR
install is definitely a much saner approach, even though it doesn't
totally defeat the security threat related with default search paths.
To do that, you have to use different linking flags on different
platforms, when building both libraries and executables linking with
them, and you sometimes have to resort to relinking at install time,
as I wrote in another message in this thread.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist                Professional serial bug killer

Re: Installation proposal

[Richard Henderson]

On Wed, Feb 27, 2002 at 10:28:38PM -0300, Alexandre Oliva wrote:
> A sane system, to me, is one in which programs and libraries can find
> their dependencies without the user having to resort to setting
> LD_LIBRARY_PATH and *hoping* it will work, if they are installed in
> the location they were configured for.

IMO, there is *no* acceptible use of DT_RPATH, and the *only* 
acceptible use of DT_RUNPATH must include the $ORIGIN token
available on recent Solaris and Linux.  Otherwise it becomes
impossible to move installations after the fact.

> - on HPUX...

"HPUX" and "sane" have never been uttered in the same sentence
without accompanying negatives.

> ... there are some dynamic linkers in which LD_LIBRARY_PATH
> (or the equivalent) won't do what you want.

Which is "broken" not "sane".


r~

Re: Installation proposal

[Alexandre Oliva]

On Feb 27, 2002, Richard Henderson <rth@redhat.com> wrote:

> On Wed, Feb 27, 2002 at 12:32:27PM -0700, Tom Tromey wrote:
>> Don't shared libraries have to be relinked with knowledge of their
>> real install location?

> Not on sane systems.

> Libtool seems to be of the opinion that this is necessary
> everywhere, however.

Please do not spread mis-information about libtool.  You may have your
reasons to dislike libtool, but what you're saying is not true.

A sane system, to me, is one in which programs and libraries can find
their dependencies without the user having to resort to setting
LD_LIBRARY_PATH and *hoping* it will work, if they are installed in
the location they were configured for.

Libtool does in fact have to resort to relinking at install time in
some situations to guarantee this property.  I list such situations
below:

- on HPUX, when installing an executable linked using libtool that is
  linked with libtool libraries in their build-tree location.  That's
  because HPUX's linker doesn't support the equivalent of -rpath, so
  the executable must be relinked after the library is installed such
  that it can store the default pathname of the library into the
  executable.  This does not work for DESTDIR installations, and
  there's no way to overcome it on HPUX.

- on most other systems, when a libtool shared library is linked with
  another libtool shared library, the latter still being in its build
  tree.  Because libtool creates libraries in the build tree in such a
  way that they can be linked with and used in-place, it encodes
  build-tree pathnames into the library.  Then, at install time, it
  re-links libraries such that the work in the install tree.  Again,
  on HP-UX, DESTDIR installations don't work.


The former is a problem for GCJ executables on HPUX.  The latter would
be a problem for GCJ because it contains inter-library dependencies.
However, except on HPUX, it should work just fine, as long as a patch
that fixes relinking in DESTDIR installations is put in.  Assuming
we're actually going to `make install' into the install-tree hold
area, it would work just fine.  Except that executables and libraries
would be set up to find their dependencies in the final install tree,
so, if you already have earlier versions in there, your results may be
incorrect.

Messing up with shared libraries with inter-dependencies is very
dangerous business.  Nothing is as simple as you'd like and, and there
are some dynamic linkers in which LD_LIBRARY_PATH (or the equivalent)
won't do what you want.  In fact, on some systems there's no way to
override the path used to search for dependencies of shared library,
so the only hope is to get them correctly encoded into the shared
library itself.  Trying to fix it up with LD_LIBRARY_PATH afterwards
just won't work.

Beware!  You were warned :-)

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist                Professional serial bug killer

Re: Installation proposal

[Richard Henderson]

On Wed, Feb 27, 2002 at 04:27:09PM -0800, Mark Mitchell wrote:
> > As David Edelsohn mentioned, there could be problems if you have a
> > previous installed tree with the same prefix.  The tree being tested may
> > accidentally use stuff from the install tree if we aren't manually using
> > -B/-L/-I/etc overrides.
> 
> That would be a bug; the stuff to find the compiler and use its
> current path should prevent this.

In which case is it then impossible to build a cross-binutils,
install it, then build a cross-gcc?

Of course, it's almost certainly easier to do everything via
a single-build tree, but this is a change in behaviour.


r~

Re: Installation proposal

[Mark Mitchell]

--On Wednesday, February 27, 2002 11:14:11 AM -0800 Jim Wilson 
<wilson@redhat.com> wrote:

> There are 3 prefixes not 1.  This scheme still needs to work even if
> prefix != exec_prefix != local_prefix.

OK.  Must we really keep all of these?

In any case, local_prefix should not be a problem; that's an absolute
path.  Wherever we drop the compiler binaries, this will be the same.

So, I think it's really a question of whether we need exec_prefix.
I can see why we wanted this, but I'm not convinced that we should
still want it.  (Sometimes, we say "I can imagine this feature being
useful" or even "I use this feature and I like it" and forget that
this is not the same as "this feature is, on balance, better than
its absence.")

If we must keep exec_prefix, we can maybe keep it by always setting
exec_prefix to "install/exec_prefix" and then at install-time,
copying the files from "install/exec_prefix" to the right place.  This
will maybe not work depending on how the driver thinks it is going
to find things in the exec_prefix.

It sounds to me like exec_prefix probably defeats the
location-independence of the compiler driver, which is not good,
in addition to the fact that it probably makes exec_prefix
incompatible with the scheme I proposed.

> As David Edelsohn mentioned, there could be problems if you have a
> previous installed tree with the same prefix.  The tree being tested may
> accidentally use stuff from the install tree if we aren't manually using
> -B/-L/-I/etc overrides.

That would be a bug; the stuff to find the compiler and use its
current path should prevent this.

It's not that it might not happen, just that it shouldn't.

> This could cause problems with builds using combined
> binutils/gcc/newlib/etc source trees.

Why?  The would get their prefix set to the same prefix ("install") as
GCC, and then they would get installed there.  Does GCC's configury
not look for "as" and such in the prefix in which it will finally be
installed?

-- 
Mark Mitchell                mark@codesourcery.com
CodeSourcery, LLC            http://www.codesourcery.com

Re: Installation proposal

[Loren James Rittle]

In article <11560000.1014832588@warlock.codesourcery.com> Mark Mitchell writes:

> Read this unless you are willing to let me change the build process around
> somewhat dramatically without knowing what I'm going to do.

As one that monkeys around with various gcc Makefiles more than is
healthy, I strongly support your goal.  I can't take credit for it,
but we have already started down the staging path with libstdc++-v3
since by arranging headers as they appear in $prefix, we could reduce
the ad hoc command line setup (you observed as similar issue but we
know it helped in actual practice).

Regards,
Loren

Re: Installation proposal

[Daniel Jacobowitz]

On Wed, Feb 27, 2002 at 11:14:11AM -0800, Jim Wilson wrote:
> There are 3 prefixes not 1.  This scheme still needs to work even if
> prefix != exec_prefix != local_prefix.
> 
> As David Edelsohn mentioned, there could be problems if you have a previous
> installed tree with the same prefix.  The tree being tested may accidentally
> use stuff from the install tree if we aren't manually using -B/-L/-I/etc
> overrides.
> 
> This could cause problems with builds using combined binutils/gcc/newlib/etc
> source trees.  This is a common method for building embedded cross compilers.
> If you build gcc into an install subdir, but don't change how binutils works,
> then the install/bin/gcc won't be able to find the just built assembler and
> linker, and pre-install testing will fail.  The binutils problem could be fixed
> by requiring people to seperately build and install binutils first, but the
> libraries newlib and libgloss are trickier.  You can't test gcc without the
> library support (e.g. crt0.o), but you can't build the libraries until after
> you have built the compiler.  This is all a little easier if everything works
> the same way and can be built from the same tree.

As a first step, IMHO, the bulk of the where-am-i logic in the GCC
driver should be moved out to libiberty.  I have patches to do the same
thing in binutils that we do in GCC, but there's no reason to duplicate
that code.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: Installation proposal

[Nathan Sidwell]

Mark Mitchell wrote:

> 1. The testing we do is much more like the testing the user will do.
oh, yes please. I've a very fragile (cos I'm lazy) script which DTRT
so I can run the the compiler more than just cc1{,plus}, binning that
would be good.

\x18\x13> the children.  Then, have the top-level "make install" do the copying
> step.  Sounds like one day's work to get it working, and another one
> to debug the fallout.
This wouldn't support
	configure --prefix=/usr/local/egcs
		--exec-prefix=/usr/local/egcs/sparc-sun-solaris2.8
to give a non-hypothetical example. I guess --exec-prefix has outlived
its usefulness in these days of stonking great disks. And, as it happens
Rob, like any sane sysadmin, automounts /usr/local from different physical
disks, depending on the OS in use, making exec-prefix redundant.

nathan

-- 
Dr Nathan Sidwell   ::   http://www.codesourcery.com   ::   CodeSourcery LLC
         'But that's a lie.' - 'Yes it is. What's your point?'
nathan@codesourcery.com : http://www.cs.bris.ac.uk/~nathan/ : nathan@acm.org

Re: Installation proposal

[Richard Henderson]

On Wed, Feb 27, 2002 at 12:32:27PM -0700, Tom Tromey wrote:
> Don't shared libraries have to be relinked with knowledge of their
> real install location?

Not on sane systems.

Libtool seems to be of the opinion that this is necessary
everywhere, however.


r~

Re: Installation proposal

[Laurent Guerby]

Jim Wilson wrote:

> If the install is overwriting /usr/bin/gcc, then it is a really good idea
> to test it before installing it.  I don't recommend doing builds that way,
> but some people do.

People installing a system compiler have probably their own
install and test system (like recompiling the kernel, then checking the
build of a few gigabytes of free software). We probably
look like testing amateurs to them :).

Who is seriously using configure with prefix /usr on a regular basis?

> If you are building something to install in a location that you don't have
> write access to, then you can still test it.  Otherwise, you need to su root
> first before you can test it.

If our accepted target is location independance, then this argument is void.

My own user policy is all root stuff is installed through binary
packaging systems, everything else is installed with user
permission (other solution I used where far worse :).

The ACT GNAT install process is location independant
and it does so by having a 10 line wrapper script that sets the needed 
env var and
then call the real bin for all of its tools (using $0).

> It makes the code-test-debug cycle faster, if you don't have to do an install
> after writing code and before testing it.  

Full C+Ada install is 65s on my machine (boot is 47 minutes, check is 28 
minutes,
ACATS is 32 minutes). Doesn't look like a real argument. Plus as far as 
I can
judge people code-test-debug on one test file (the bug) by using
gdb on xx1, they don't run the full test suite after each one line change,
they run it once it works reasonably on their test case, then bootstrap
+ check is going to take a while but install time is irrelevant then.

 > It also saves on disk space during the development cycle.

With C+Ada install takes less space than a stage, and about 25%
of a full make boostrap. If you are this short on space, then
I don't even see how you can do any development work.

> But yes, when you get to the point where you are making a release, it is
> mandatory to test the installed compiler.  

And release are where we should be good on testing (easy to automate,
testing what user will see). Why not use the same thing while developing?
If we don't then all the trouble is for our loved release manager at
release time :).

> The current testsuites can be used
> to test an installed compiler if there is no compiler in the build tree.

Is this documented? (I remember only seeing "make -k check").
I would be *really* interested.

-- 
Laurent Guerby <guerby@acm.org>

Re: Installation proposal

[mike stump]

> Date: Wed, 27 Feb 2002 20:03:05 +0100
> From: Laurent Guerby <guerby@acm.org>
> To: Mark Mitchell <mark@codesourcery.com>
> CC: gcc@gcc.gnu.org

> Testing anything but the installed compiler is a mistake,

I disagree.  When developing, being able to test early, and often and
with low latency is useful.  During normal development I only do a
make install once every few months.  After we have a system that
builds an incremental install tree directly, we can revisit this.

Re: Installation proposal

[Michael Meissner]

On Wed, Feb 27, 2002 at 01:12:29PM -0500, David Edelsohn wrote:
> 	Either you have overwritten the user's choice for prefix, which
> means that the compiler no longer can be installed where the user
> originally intended, *OR* install/bin/g++ will not work because it will
> look in /some/user/prefix/lib/gcc-lib/...
> 
> 	Or, are you saying that you somehow want GCC to recognize the
> directory into which it was installed and access all files with relative
> paths so that $prefix no longer would be built into the compiler and only
> used for installation?

Well GCC already does relative pathname lookup (ie, on UNIX type systems if you
invoke foo/bar/install/bin/gcc, it will look for the compiler components in
foo/bar/install/lib/...).  It was added in November 8th of 1999.  Maybe I'm
missing exactly what you are asking.

-- 
Michael Meissner, Red Hat, Inc.  (GCC group)
PMB 198, 174 Littleton Road #3, Westford, Massachusetts 01886, USA
Work:	  meissner@redhat.com		phone: +1 978-486-9304
Non-work: meissner@the-meissners.org	fax:   +1 978-692-4482

Re: Installation proposal

[Neil Booth]

Laurent Guerby wrote:-

> Testing anything but the installed compiler is a mistake, the testsuite
> should be completely separated from the build process, and running
> the test suite should take place after the install step IMHO.

FWIW I've always appreciated being able to test the compiler without
installing it.  I don't particularly want to install every compiler I
bootstrap.

Neil.

Re: Installation proposal

[Jim Wilson]

>Testing anything but the installed compiler is a mistake, the testsuite
>should be completely separated from the build process, and running
>the test suite should take place after the install step IMHO.

If the install is overwriting /usr/bin/gcc, then it is a really good idea
to test it before installing it.  I don't recommend doing builds that way,
but some people do.

If you are building something to install in a location that you don't have
write access to, then you can still test it.  Otherwise, you need to su root
first before you can test it.

It makes the code-test-debug cycle faster, if you don't have to do an install
after writing code and before testing it.  It also saves on disk space during
the development cycle.

But yes, when you get to the point where you are making a release, it is
mandatory to test the installed compiler.  The current testsuites can be used
to test an installed compiler if there is no compiler in the build tree.

Jim

Re: Installation proposal

[Zack Weinberg]

On Wed, Feb 27, 2002 at 08:03:05PM +0100, Laurent Guerby wrote:
> Mark Mitchell wrote:
> 
> >What's good about this?
> >
> >1. The testing we do is much more like the testing the user will do.
> >
> >  In particular, to test, say, g++, we would just run
> >  "install/bin/g++", not "g++ -B... -nostdinc++ -I..."
> 
> Testing anything but the installed compiler is a mistake, the testsuite
> should be completely separated from the build process, and running
> the test suite should take place after the install step IMHO.

But you want to be able to test the compiler *before* you install it,
don't you?

zw

Re: Installation proposal

[Joseph S. Myers]

On Wed, 27 Feb 2002, Mark Mitchell wrote:

> > Please ensure that this installs all shared libraries (most importantly
> > libgcc, but the others as well) in a safe manner.
>
> Do we actually do all this at this point?  Or are you asking for a new
> feature?  If you are, I am likely to punt.  If we already know how to do
> this somewhere, then I will copy the logic.

When investigating the safety of libgcc installation before, I found that
the shared libgcc was installed with the C install (when everything else
was installed by install-sh).  This may not be atomic, but is much safer
than install-sh which is guaranteed to fail if the system mv binary is
linked against libgcc.

http://gcc.gnu.org/ml/gcc/2001-03/msg01327.html

-- 
Joseph S. Myers
jsm28@cam.ac.uk

Re: Installation proposal

[Joseph S. Myers]

On Wed, 27 Feb 2002, Joel Sherrill wrote:

> > 4. It's much easier to ensure consistent permissions and such in the
> >    final installation if there's only one place where installations
> >    are happening.  You can make sure that everything has the same
> >    ownership, same modes, etc.  It's also easy to use something
> >    faster than install-sh when you have it because there's only one
> >    place you need to do it.
> 
> RTEMS installs with a tar piped into a cd/tar combination so I can
> see this.

GCC uses tar to install some headers in libsubdir/include.  This gets
ownerships wrong (files owned by the user who did the build rather than
the one who did the install).

-- 
Joseph S. Myers
jsm28@cam.ac.uk

Re: Installation proposal

[Mark Mitchell]

--On Wednesday, February 27, 2002 12:32 PM -0700 Tom Tromey 
<tromey@redhat.com> wrote:

>>>>>> "Mark" == Mark Mitchell <mark@codesourcery.com> writes:
>
> Mark> 3. At install time, run a single script which copies everything
> Mark> in install into prefix.
>
> It isn't quite that simple.

It never is, is it? :-)  I'm confident that there will be some follow-on
work to get the kinks out.

> Don't shared libraries have to be relinked with knowledge of their
> real install location?

I don't know; do they?  I'm not being facetious; I really don't know.

> Sometimes you need to run post-install code, like install-info.

Yes, that's true.  I'm sure there will need to be some special hooks
somewhere for this.

> This requires even more disk space than is already required.  I'm
> probably the last person alive who actually cares about this.

Yes, you probably are. :-)  If we do the bit where we build during into
the "install" directory, then you don't need any more disk space than you
do now.  You're right that until/unless that happens, more disk space will
be needed.

Do you have a bottom-line opinion on whether or not you think the change
would be a benefit or a hindrance?

--
Mark Mitchell
CodeSourcery, LLC
mark@codesourcery.com

Re: Installation proposal

[Mark Mitchell]

> Please ensure that this installs all shared libraries (most importantly
> libgcc, but the others as well) in a safe manner.

Do we actually do all this at this point?  Or are you asking for a new
feature?  If you are, I am likely to punt.  If we already know how to do
this somewhere, then I will copy the logic.

--
Mark Mitchell
CodeSourcery, LLC
mark@codesourcery.com

Re: Installation proposal

[Per Bothner]

Mark Mitchell wrote:
> What's good about this?

5. The Makefiles for builting libraries can be simpler:  Just use
"install/bin/g++", not "g++ -B... -nostdinc++ -I..."
(This is a variant of your point 1.)

6. Easier to "run in place" without installing.  I.e. don't need
to 'make install' - just add the $builddir/install to the $PATH.
(Emacs already allows this.)

7. Possibly cleaner 'make bootstrap' - use a different install
directory for each stage.

> We can just have the top-level configure set the prefix for all of the
> child configures to "install".  Then, have the top-level "make" and
> "make bootstrap" do what they do now, plus "make install" in all of
> the children.  Then, have the top-level "make install" do the copying
> step.  Sounds like one day's work to get it working, and another one
> to debug the fallout.

One disadvantage is that this would require more disk space while
building, since you need teh addition space for the install directory.
But this problem can be incrementally reduced as you gradually fix
sub-directories to leave files directly in the install directory.

I suggest chaninging the name from 'install' to 'build'.  It isn't
the real install directory.  Ideally, for sub-directories that have
been converted to this convention, you still have 'make install',
but it just copies from the 'build' directory to $prefix.  This is
not critical, but it might be useful if one wanted to install files
in a sub-directory only.
-- 
	--Per Bothner
per@bothner.com   http://www.bothner.com/per/

Re: Installation proposal

[Jim Wilson]

There are 3 prefixes not 1.  This scheme still needs to work even if
prefix != exec_prefix != local_prefix.

As David Edelsohn mentioned, there could be problems if you have a previous
installed tree with the same prefix.  The tree being tested may accidentally
use stuff from the install tree if we aren't manually using -B/-L/-I/etc
overrides.

This could cause problems with builds using combined binutils/gcc/newlib/etc
source trees.  This is a common method for building embedded cross compilers.
If you build gcc into an install subdir, but don't change how binutils works,
then the install/bin/gcc won't be able to find the just built assembler and
linker, and pre-install testing will fail.  The binutils problem could be fixed
by requiring people to seperately build and install binutils first, but the
libraries newlib and libgloss are trickier.  You can't test gcc without the
library support (e.g. crt0.o), but you can't build the libraries until after
you have built the compiler.  This is all a little easier if everything works
the same way and can be built from the same tree.

Jim

Re: Installation proposal

[Laurent Guerby]

Mark Mitchell wrote:

> What's good about this?
> 
> 1. The testing we do is much more like the testing the user will do.
> 
>   In particular, to test, say, g++, we would just run
>   "install/bin/g++", not "g++ -B... -nostdinc++ -I..."

Testing anything but the installed compiler is a mistake, the testsuite
should be completely separated from the build process, and running
the test suite should take place after the install step IMHO.

I never understood what are the advantage (if any) of tying the
testsuite run to the build process, any info? I can list
only disadvantages...


> 2. The testsuites could forget about multilibs.  They will now just
>   work the way they do when users actually use the compiler.  (Right
>   now, the testsuites have to futz around with the -I paths and -L
>   paths to find everything correctly.  Thus this logic is duplicated
>   between the compiler and the testsuite.)

One point that would be nice for test suite technology would
be a gcc flag or tool generating normalized output giving information
about what is available to test (targets, languages, etc...).

I assume most of the information is already available by
combining existing flags, but then even we don't modify the driver
a documented and maintained (accross driver changes)
script using the driver producing such information would be great.

-- 
Laurent Guerby <guerby@acm.org>

Re: Installation proposal

[Tom Tromey]

>>>>> "Mark" == Mark Mitchell <mark@codesourcery.com> writes:

Mark> 3. At install time, run a single script which copies everything
Mark> in install into prefix.

It isn't quite that simple.

Don't shared libraries have to be relinked with knowledge of their
real install location?

Sometimes you need to run post-install code, like install-info.

This requires even more disk space than is already required.  I'm
probably the last person alive who actually cares about this.

Tom

Re: Installation proposal

[Joel Sherrill]

Mark Mitchell wrote:
> 
> Read this unless you are willing to let me change the build process around
> somewhat dramatically without knowing what I'm going to do.
> 
> I got to thinking about the idea that I brought up yesterday regarding
> installation.
> 
> In particular, here's what I had in mind:
> 
> 1. Create a subdirectory of the objdir called "install".
> 
> 2. Treat install as if it were the final prefix during building --
>    put everything in there.  That means that there will be directories
>    like "install/bin", "install/lib", and so forth.  So, "make" would
>    just drop everything in these directories.
> 
> 3. At install time, run a single script which copies everything in
>    install into prefix.
> 
> What's good about this?

This is generally what RTEMS does and it does make things more
consistent between build and install images.  I am sure there
are warts to work out but it will certainly cut down on keeping
track of include paths in multiple places.  One thought is that
other packages that are usually part of the "one tree" build
might have to go this way as well.  newlib comes to mind as
one such package, I am sure there will be others.

> 1. The testing we do is much more like the testing the user will do.
> 
>    In particular, to test, say, g++, we would just run
>    "install/bin/g++", not "g++ -B... -nostdinc++ -I..."

This seems like a good thing if the details are worked out.
 
> 2. The testsuites could forget about multilibs.  They will now just
>    work the way they do when users actually use the compiler.  (Right
>    now, the testsuites have to futz around with the -I paths and -L
>    paths to find everything correctly.  Thus this logic is duplicated
>    between the compiler and the testsuite.)

This would be a great thing from a maintenance and consistency 
standpoint.

I also suspect it would fix the core dump I see when trying to
test *-rtems targets since we use a -B... -specs BOARD_specs to 
augment gcc's notion of linking.
 
> 3. We would test that our installations are really location-independent,
>    which they supposedly are.  (They certainly should be; other vendors
>    have done this with their compilers for a decade, and its very
>    handy.)

This is certainly a weakness in gcc right now.  It is hard to explain
to someone using prebuilt binaries why they shouldn't be relocated.  I
see people setting GCC_EXEC_PREFIX for their native or cross GCC and 
breaking the other.  

> 4. It's much easier to ensure consistent permissions and such in the
>    final installation if there's only one place where installations
>    are happening.  You can make sure that everything has the same
>    ownership, same modes, etc.  It's also easy to use something
>    faster than install-sh when you have it because there's only one
>    place you need to do it.

RTEMS installs with a tar piped into a cd/tar combination so I can
see this.

> Now, I thought that this would be a horrific pain because I'd have to
> change all the make rules everywhere to drop their outputs in the
> "install" directory.
> 
> But, I realized that we can put that off, even though it would be good
> to do at some point.

There are a lot of details to work out and there will be some
consistency
issues with other packages but it should be a great simplification.

> We can just have the top-level configure set the prefix for all of the
> child configures to "install".  Then, have the top-level "make" and
> "make bootstrap" do what they do now, plus "make install" in all of
> the children.  Then, have the top-level "make install" do the copying
\x1a> step.  Sounds like one day's work to get it working, and another one
> to debug the fallout.
> 
> I would like to implement this and check it in for GCC 3.2.
> 
> I can't believe there won't be debate.  Still, if nobody objects in a
> few days, this is what I'll do.
> 
> --
> Mark Mitchell                mark@codesourcery.com
> CodeSourcery, LLC            http://www.codesourcery.com

-- 
Joel Sherrill, Ph.D.             Director of Research & Development
joel@OARcorp.com                 On-Line Applications Research
Ask me about RTEMS: a free RTOS  Huntsville AL 35805
   Support Available             (256) 722-9985

Re: Installation proposal

[Joseph S. Myers]

On Wed, 27 Feb 2002, Mark Mitchell wrote:

> 3. At install time, run a single script which copies everything in
>    install into prefix.

Please ensure that this installs all shared libraries (most importantly
libgcc, but the others as well) in a safe manner.  install-sh doesn't
handle that properly at present.  Safe means that the shared library is
always available for any other processes started at any time; the
installed files need to be copied to a temporary name, that renamed to the
final name, and then if there's a symlink from the soname to the name of
the library file, that must be redirected to point to the new file
atomically.

> What's good about this?

5. Supporting DESTDIR with this new system is trivial.

-- 
Joseph S. Myers
jsm28@cam.ac.uk

Re: Installation proposal

[Stan Shebs]

Mark Mitchell wrote:
> 
> Read this unless you are willing to let me change the build process around
> somewhat dramatically without knowing what I'm going to do.
> 
> I got to thinking about the idea that I brought up yesterday regarding
> installation.
> 
> In particular, here's what I had in mind:
> 
> 1. Create a subdirectory of the objdir called "install".
> 
> 2. Treat install as if it were the final prefix during building --
>    put everything in there.  That means that there will be directories
>    like "install/bin", "install/lib", and so forth.  So, "make" would
>    just drop everything in these directories.
> 
> 3. At install time, run a single script which copies everything in
>    install into prefix.

This is actually very similar to what Apple does for Mac OS X.
The final install place is called the "dstroot", and is an image
of what will be installed.  This works well across a wide range
of projects, because then you can have a single set of tools and
scripts that scan dstroots for permissions, setuid programs, and
other potential security holes.

It's also useful for building a complete system incrementally,
by merging in dstroots, then doing a chroot to try it out.

Stan

> What's good about this?
> 
> 1. The testing we do is much more like the testing the user will do.
> 
>    In particular, to test, say, g++, we would just run
>    "install/bin/g++", not "g++ -B... -nostdinc++ -I..."
> 
> 2. The testsuites could forget about multilibs.  They will now just
>    work the way they do when users actually use the compiler.  (Right
>    now, the testsuites have to futz around with the -I paths and -L
>    paths to find everything correctly.  Thus this logic is duplicated
>    between the compiler and the testsuite.)
> 
> 3. We would test that our installations are really location-independent,
>    which they supposedly are.  (They certainly should be; other vendors
>    have done this with their compilers for a decade, and its very
>    handy.)
> 
> 4. It's much easier to ensure consistent permissions and such in the
>    final installation if there's only one place where installations
>    are happening.  You can make sure that everything has the same
>    ownership, same modes, etc.  It's also easy to use something
>    faster than install-sh when you have it because there's only one
>    place you need to do it.
> 
> Now, I thought that this would be a horrific pain because I'd have to
> change all the make rules everywhere to drop their outputs in the
> "install" directory.
> 
> But, I realized that we can put that off, even though it would be good
> to do at some point.
> 
> We can just have the top-level configure set the prefix for all of the
> child configures to "install".  Then, have the top-level "make" and
> "make bootstrap" do what they do now, plus "make install" in all of
> the children.  Then, have the top-level "make install" do the copying
> step.  Sounds like one day's work to get it working, and another one
> to debug the fallout.
> 
> I would like to implement this and check it in for GCC 3.2.
> 
> I can't believe there won't be debate.  Still, if nobody objects in a
> few days, this is what I'll do.
> 
> --
> Mark Mitchell                mark@codesourcery.com
> CodeSourcery, LLC            http://www.codesourcery.com

Re: Installation proposal

[Jim Wilson]

>	Or, are you saying that you somehow want GCC to recognize the
>directory into which it was installed and access all files with relative
>paths so that $prefix no longer would be built into the compiler and only
>used for installation?

Gcc already tries to create relative paths internally if GCC_EXEC_PREFIX is not
specified.  See the make_relative_prefix function in gcc.c.  However, prefix
is still built in, because we need to compare prefix against exec_prefix
to compute the relative paths.

Jim

Re: Installation proposal

[Paul Koning]

As a novice at GCC hackery I don't have any real opinion about the
mechanics you're proposing.  But the idea sounds sensible.

There may be another benefit, perhaps... if you actually do the more
complex change of updating where the outputs go, then in stage 2 the
invocations of the newly built compiler would look more like regular
invocations rather than the somewhat messy ones you have now.  It's
possible that a bunch of the "this compiler can't create binaries"
failures that you tend to see -- especially in cross-builds if you
don't know all the right magic words -- will go away.  That would be a
Good Thing.

     paul

Re: Installation proposal

[Mark Mitchell]

> 	Or, are you saying that you somehow want GCC to recognize the
> directory into which it was installed and access all files with relative
> paths so that $prefix no longer would be built into the compiler and only
> used for installation?

Yes -- but I think that it is a no-op.  IIRC, patches to make this work
already went in.

If not, consider my proposal to assume that this works.  If it does not,
I may try to implement it.

(The way that, in general, such schemes work is:

a) Use an environment variable.

b) OS-dependent magic to find the binary that is currently executing.

c) Search the PATH to find argv[0], if argv[0] is not absolute.

There are ways to fool this logic (various uses of links, and such), but
the environment variable gives you a fallback, and in practice such schemes
work very nicely.)

--
Mark Mitchell
CodeSourcery, LLC
mark@codesourcery.com

--
Mark Mitchell
CodeSourcery, LLC
mark@codesourcery.com

Re: Installation proposal

[David Edelsohn]

	Either you have overwritten the user's choice for prefix, which
means that the compiler no longer can be installed where the user
originally intended, *OR* install/bin/g++ will not work because it will
look in /some/user/prefix/lib/gcc-lib/...

	Or, are you saying that you somehow want GCC to recognize the
directory into which it was installed and access all files with relative
paths so that $prefix no longer would be built into the compiler and only
used for installation?

Thanks, David

Re: Installation proposal

[H . J . Lu]

On Wed, Feb 27, 2002 at 09:56:28AM -0800, Mark Mitchell wrote:
> 
> Now, I thought that this would be a horrific pain because I'd have to
> change all the make rules everywhere to drop their outputs in the
> "install" directory.
> 

If you want to go this route, which I don't have a strong opinion, I
would suggest you mandate the GNU make. It may make everything much
easier.


H.J.

Installation proposal

[Mark Mitchell]

Read this unless you are willing to let me change the build process around
somewhat dramatically without knowing what I'm going to do.

I got to thinking about the idea that I brought up yesterday regarding
installation.

In particular, here's what I had in mind:

1. Create a subdirectory of the objdir called "install".

2. Treat install as if it were the final prefix during building --
   put everything in there.  That means that there will be directories
   like "install/bin", "install/lib", and so forth.  So, "make" would
   just drop everything in these directories.

3. At install time, run a single script which copies everything in
   install into prefix.

What's good about this?

1. The testing we do is much more like the testing the user will do.

   In particular, to test, say, g++, we would just run
   "install/bin/g++", not "g++ -B... -nostdinc++ -I..."

2. The testsuites could forget about multilibs.  They will now just
   work the way they do when users actually use the compiler.  (Right
   now, the testsuites have to futz around with the -I paths and -L
   paths to find everything correctly.  Thus this logic is duplicated
   between the compiler and the testsuite.)

3. We would test that our installations are really location-independent,
   which they supposedly are.  (They certainly should be; other vendors
   have done this with their compilers for a decade, and its very
   handy.)

4. It's much easier to ensure consistent permissions and such in the
   final installation if there's only one place where installations
   are happening.  You can make sure that everything has the same
   ownership, same modes, etc.  It's also easy to use something
   faster than install-sh when you have it because there's only one
   place you need to do it.

Now, I thought that this would be a horrific pain because I'd have to
change all the make rules everywhere to drop their outputs in the
"install" directory.

But, I realized that we can put that off, even though it would be good
to do at some point.

We can just have the top-level configure set the prefix for all of the
child configures to "install".  Then, have the top-level "make" and
"make bootstrap" do what they do now, plus "make install" in all of
the children.  Then, have the top-level "make install" do the copying
step.  Sounds like one day's work to get it working, and another one
to debug the fallout.

I would like to implement this and check it in for GCC 3.2.

I can't believe there won't be debate.  Still, if nobody objects in a
few days, this is what I'll do.

-- 
Mark Mitchell                mark@codesourcery.com
CodeSourcery, LLC            http://www.codesourcery.com