Re: what should bootstrap *really* do?

Re: what should bootstrap *really* do?

[Richard Kenner]

    * build stage1

    * if anything of significance changed in stage1, delete *.[oa] in stage 2

When would anything of "significance" *not* change?  In other words, how would
this be different from what we used to have to justify the added complexity?

Re: what should bootstrap *really* do?

[DJ Delorie]

> When would anything of "significance" *not* change?

When the sources don't change.  Example: a cron job that updates the
sources from cvs and "make bootstrap"s.  If nothing changes, why waste
cycles?  The more likely case is a forgetful programmer ("Did I
bootstrap this?  Hmmm... make bootstrap... yup, I did.").

> In other words, how would this be different from what we used to
> have to justify the added complexity?

Some developers want it the old way.  Some developers want it the new
way.  Some developers want something else entirely.  Is it so bad to
want to please everyone?

I, personally, want a restartable bootstrap because I, personally,
work on systems that can't reliably complete a full bootstrap.  Other
people have other needs.  I'm trying to change the system so that more
than one option is available.

Re: what should bootstrap *really* do?

[Hans-Peter Nilsson]

> Date: Tue, 14 Nov 2000 09:58:59 -0500
> From: DJ Delorie <dj@redhat.com>

> I, personally, want a restartable bootstrap because I, personally,
> work on systems that can't reliably complete a full bootstrap.  Other
> people have other needs.  I'm trying to change the system so that more
> than one option is available.

Then why not have a *new* target with the *new* (restart-in-middle)
functionality instead of changing the "bootstrap" target?  The name
"bootstrap-restart" comes to mind. :-)

BTW, all stages should be built in the same directory (like
now), or you risk debugging info being different and failing to
compare.

brgds, H-P

Re: what should bootstrap *really* do?

[DJ Delorie]

> Then why not have a *new* target with the *new* (restart-in-middle)
> functionality instead of changing the "bootstrap" target?  The name
> "bootstrap-restart" comes to mind. :-)

Hence, my original question.  The original bootstrap really only
worked if the build area was clean and the bootstrap completed.

> BTW, all stages should be built in the same directory (like
> now), or you risk debugging info being different and failing to
> compare.

Oh, right.  Thanks!

Re: what should bootstrap *really* do?

[Hans-Peter Nilsson]

> Date: Tue, 14 Nov 2000 11:34:35 -0500
> From: DJ Delorie <dj@redhat.com>

> > Then why not have a *new* target with the *new* (restart-in-middle)
> > functionality instead of changing the "bootstrap" target?  The name
> > "bootstrap-restart" comes to mind. :-)
> 
> Hence, my original question.  The original bootstrap really only
> worked if the build area was clean and the bootstrap completed.

Oh.  So there was a bug that caused people to (reasonably) think
that the intent of bootstrap was to be able to
restart-in-middle, and at the same time the intent for most of
the developer crowd(?) was not articulated through comments,
documentation or otherwise?

brgds, H-P

Re: what should bootstrap *really* do?

[DJ Delorie]

> Oh.  So there was a bug that caused people to (reasonably) think
> that the intent of bootstrap was to be able to restart-in-middle,
> and at the same time the intent for most of the developer crowd(?)
> was not articulated through comments, documentation or otherwise?

Not quite.  If the original bootstrap failed, it left the build area
in an unknown state, and the only way to recover was either to clean
it and start again, or *really* know what you're doing.

If you're going to clean it anyway, it really doesn't matter what
"make bootstrap" does.

If you really know what you're doing, it also really doesn't matter
what "make bootstrap" does.

So the case that bootstrap needs to worry about the most is the people
who don't want to (or know to) clean it, but don't know how to recover
either.  Since the documentation advises *users* to use "make
bootstrap" to build the compiler, I expect their unvoiced expectation
that bootstrap is restartable to be significant.

As for developers, it seems they fall into two categories: those who
want bootstraps to be restartable, and those who don't.  The ones who
don't can simply run "make clean" or "rm *.o" (which they'd have to do
with the old bootstrap too), and the ones who do need lots of help
from the Makefile.

Re: what should bootstrap *really* do?

[Richard Earnshaw]

dj@redhat.com said:
> As for developers, it seems they fall into two categories: those who
> want bootstraps to be restartable, and those who don't.  The ones who
> don't can simply run "make clean" or "rm *.o" (which they'd have to do
> with the old bootstrap too), and the ones who do need lots of help
> from the Makefile. 

Since we seem to be gathering opinions....

What *I* want to do (when I'm tracking down a problem).  Is the equivalent 
of the following, which was possible with the old system.

Fix foo.c (eg combine.c):

# rebuild the bits of my compiler that have changed
make bootstrap3

# test my change -- if it works, do a full bootstrap.

Yes, I know that the compare will fail, but at this point I don't care.  I 
want to see first that it has fixed the problem I was working on.

I don't want to wait 6+ hours for a full bootstrap.  I don't want to start 
faffing with stamp files (that's error prone) and I don't want two builds, 
one made with bootstrap and one with plain make -- that just wastes disk 
space.

I could do all of those things with the old build system.  I can't do them 
with the new one.

About the only rule that I might have added to the old system was "make 
unstageX" (X=1,2,3...), which would effectively reverse the effect of 
"make stageX".

R.

Re: what should bootstrap *really* do?

[Eric W. Biederman]

Hans-Peter Nilsson <hans-peter.nilsson@axis.com> writes:

> > Date: Tue, 14 Nov 2000 09:58:59 -0500
> > From: DJ Delorie <dj@redhat.com>
> 
> > I, personally, want a restartable bootstrap because I, personally,
> > work on systems that can't reliably complete a full bootstrap.  Other
> > people have other needs.  I'm trying to change the system so that more
> > than one option is available.
> 
> Then why not have a *new* target with the *new* (restart-in-middle)
> functionality instead of changing the "bootstrap" target?  The name
> "bootstrap-restart" comes to mind. :-)
> 
> BTW, all stages should be built in the same directory (like
> now), or you risk debugging info being different and failing to
> compare.

So why not do:

stage1/foo.o : foo.c
        cc -c foo.c -o foo.o && mv foo.o stage1/foo.o

stage2/foo.o : foo.c stage1/gcc
        stage1/gcc -c foo.c -o foo.o && mv foo.o stage2/foo.o

stage3/foo.o : foo.c stage2/gcc
        stage2/gcc -c foo.c -o foo.o && mv foo.o stage3/foo.o

bootstrap: stage3/gcc

Then you can restart when ever you want.

Nice simple, builds in the normal directory, stores the files in 
the stage directories.

Then everything is simple and follows normal make rules, and doesn't
need any special files.

Eric

Recursive make harmful (was: Re: what should bootstrap *really* do?)

[Scott A Crosby]

A web-search doesn't show this paper as having been on the gcc list yet,
I'll send it along. It discusses why recursive make is harmful and other
misuses of make.

 -- http://www.pcug.org.au/~millerp/rmch/recu-make-cons-harm.html

I would suggest reading it if you're working on the build system.

Fundamentally, most of the time when recursive make is used, it's actually
lying about the dependency DAG. As workarounds, people have to insert
wierd and special rules, or have 'marker' files.  Sometimes, it can be
easier to just not lie. You can have the makefile in one directory include
makefiles in other directories if necessary.

It's been used in other places, for example, there is the thread on
linux-kernel, where it was reported to make building much faster and make 
one makefile 800 lines shorter:
   http://www.uwsg.iu.edu/hypermail/linux/kernel/0001.3/0012.html

Your explanation is perfectly reasonable. You may have makefiles in one
directory including makefiles in another directory, but there are no
recursive make's. If someone want's to do a check by doing a partial
build, do something like my sample.

Below, I give a sample.  For the most part, this sample still lets do a
'make' in a subdirectory. and it will only build that portion. I do
violate my principal of not doing recursive make only in one place: For
each module I check, I do one recursive 'make'.

I've run a benchmark of building 2.95.2 on my machine, to report where CPU
usage is going throughout the build process.  I will be sending it to the
mailing list shortly.

Scott


On 16 Nov 2000, Eric W. Biederman wrote:

> Hans-Peter Nilsson <hans-peter.nilsson@axis.com> writes:
> 
> So why not do:
> 
> stage1/foo.o : foo.c
>         cc -c foo.c -o foo.o && mv foo.o stage1/foo.o
> 
> stage2/foo.o : foo.c stage1/gcc
>         stage1/gcc -c foo.c -o foo.o && mv foo.o stage2/foo.o
> 
> stage3/foo.o : foo.c stage2/gcc
>         stage2/gcc -c foo.c -o foo.o && mv foo.o stage3/foo.o
> 
> bootstrap: stage3/gcc
> 
> Then you can restart when ever you want.
> 
> Nice simple, builds in the normal directory, stores the files in 
> the stage directories.
> 

My sample... Incomplete, but enough to give you an idea:


##
##
## In $SRC/Makefile
##
##

include $SRC/gcc/Makefile


##
##
## In  $SRC/gcc/Makefile
##
##

$build/stage1: stage1/xgcc
$build/stage1/xgcc: ${STAGE1_OBJS}
   ${CC}  _______

check-gcc-stage1: $(build)/stage1/xgcc $(deja)
   $(MAKE) -f Makefile.check check CHECK_GCC=stage1/xgcc

check-gcc-stage3: $(build)/stage3/xgcc $(deja)
   $(MAKE) -f Makefile.check check CHECK_GCC=stage3/xgcc

# Dependency for check.
check: check-gcc-stage3

Re: Recursive make harmful (was: Re: what should bootstrap *really* do?)

[DJ Delorie]

> I would suggest reading it if you're working on the build system.

Read it, a long time ago.

> Fundamentally, most of the time when recursive make is used, it's actually
> lying about the dependency DAG.

We use recursive make in bootstrap because there is no DAG to properly
reflect what we're trying to do.  I actually have a patch for the
Makefile that does a bootstrap with DAGs, but there are reasons not to
do it that way.

> As workarounds, people have to insert wierd and special rules, or
> have 'marker' files.

In our case, we're using the Makefile to automate a sequence of steps,
with each step depending on the previous being complete.  However, the
results of the steps can't coexist.

> Sometimes, it can be easier to just not lie. You can have the
> makefile in one directory include makefiles in other directories if
> necessary.

Not if you can't rely on the include functionality of make.  We can't.
But, that's just a technical limitation.

> Your explanation is perfectly reasonable.

Our explanation is that we have to build the same sources, in the same
directory, different ways.  Because of debugging information, we can't
build in different directories or with different object names and
still meet the real goal of a bootstrap - creating comparable
binaries.

We also have cases where Makefiles are created by the build process,
it's really difficult to do that with a single Makefile.

Re: what should bootstrap *really* do?

[Alexandre Oliva]

On Nov 14, 2000, kenner@vlsi1.ultra.nyu.edu (Richard Kenner) wrote:

>     This of all those folks who have flakey problems that cause the build
>     to fail at random.  

> If they have "flakey problems", how likely is the bootstrap to succeed?
> If you have flakey hardware, it seems to me the first step should be
> to fix it!

Sometimes it is the OS that is flakey (MS-Windows comes to mind), and
some of us do test GCC on MS-Windows.

>     Think of people who want to report a problem, but the error message
>     has just run off their screen.  Why should they have to restart
>     bootstrap from scratch?  

> They don't!  They can use "mae bootstrap2" or "make bootstrap3".

They could if they knew about it.  Anyway, it's a fair assumption that
repeating the same `make' command that triggered the problem at first
will result in the problem occurring again, except in case of flakey
hardware.  If `make bootstrap' doesn't restart from scratch (i.e.,
with `make clean') or proceeds from the same stage, the problem won't
repeat, and, since we already have `make clean' for those who want it,
I'd rather have `make bootstrap' just proceed from where it stopped.

> But when they do this, they know they are doing something
> "nonstandard", so will do so carefully.

`make bootstrap' is non-standard, in the first place.  `make all' is
the standard way to build a program.  It's unfortunate enough that we
recommend a non-standard build procedure for GCC.  We shouldn't demand
users to learn about additional make targets.

>     My (unproved) claim is that the number of people in this kind of
>     situation surpasses the number of people who make a minor change here
>     or there, that would require bootstrap to be restarted, so we should
>     privilege this (alleged) majority with the default behavior of `make
>     bootstrap'.

> Are you *really* claiming that the majority of users who build GCC have
> such flakey system that they can't complete a bootstrap without crashing?

No.  I'm just claiming that there are more people who get in trouble
because of flakey hardware/OSs than because of making minor changes.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: what should bootstrap *really* do?

[Richard Earnshaw]

aoliva@redhat.com said:
> This of all those folks who have flakey problems that cause the build
> to fail at random.  Think of people who build GCC on MS-Windows. Think
> of people who want to report a problem, but the error message has just
> run off their screen.  Why should they have to restart bootstrap from
> scratch?  My (unproved) claim is that the number of people in this
> kind of situation surpasses the number of people who make a minor
> change here or there, that would require bootstrap to be restarted, so
> we should privilege this (alleged) majority with the default behavior
> of `make bootstrap'.


I don't understand why the whole thing has to be so complicated.  Why 
won't the following model work?

bootstrap:
	if test ! -f stage1.completed; then \
		$MAKE_FOR_STAGE1 ; \
		make stage1; \
	fi
	if test ! -f stage2.completed; then \
		$MAKE_FOR_STAGE2 ; \
		make stage2;\
	fi
	...

stage1:
	NORMAL_MOVES
	touch stage1.completed

clean:
	NORMAL_CLEANS
	rm stage1.completed ...

Then only the stage rules create funny stamps, meaning that any step of 
the bootstrap can be repeated until the stageN rule is run.

You could then easily have

unstage1:
	if test -f stage1.completed; then \
		make mostlyclean; \
		RULES to move stage1/* back to current dir; \
		rm stage1.completed; \
	fi

This model requires two new stamp files (rather than the current 5 or 6), 
and they correspond well with the rules that create them.

R.

Re: what should bootstrap *really* do?

[Mike Stump]

> To: Hans-Peter Nilsson <hans-peter.nilsson@axis.com>
> From: Alexandre Oliva <aoliva@redhat.com>
> Date: 14 Nov 2000 16:16:17 -0200

> I already find it bad enough that the default rule is not the one we
> recommend for users.

We need two things in the tree.  One, we need a top level Makefile
that has in it:

all:
    configure && make all

all bootstrap:
    configure && make bootstrap

and we need for the recommended way to build be the default.  If what
we recommend is complex, then autoconf type mechanisms should be used
to make it simple.

The first has been on my list for the past decade or so, and the
second, for the past, oh, what's it been, many many years.  The first
lets us return to the existing net standard for software that has been
around for 15+ years. The second is just obvious.

Think of all the people that ask, how do you build gcc.  The answer
needs to be `make'.

Re: what should bootstrap *really* do?

[Richard Kenner]

    This of all those folks who have flakey problems that cause the build
    to fail at random.  

If they have "flakey problems", how likely is the bootstrap to succeed?
If you have flakey hardware, it seems to me the first step should be
to fix it!

    Think of people who want to report a problem, but the error message
    has just run off their screen.  Why should they have to restart
    bootstrap from scratch?  

They don't!  They can use "mae bootstrap2" or "make bootstrap3".  But when they
do this, they know they are doing something "nonstandard", so will do so
carefully.

    My (unproved) claim is that the number of people in this kind of
    situation surpasses the number of people who make a minor change here
    or there, that would require bootstrap to be restarted, so we should
    privilege this (alleged) majority with the default behavior of `make
    bootstrap'.

Are you *really* claiming that the majority of users who build GCC have
such flakey system that they can't complete a bootstrap without crashing?

Re: what should bootstrap *really* do?

[Geert Bosch]

On 14 Nov 2000 16:16:17 -0200, Alexandre Oliva wrote:

  I already find it bad enough that the default rule is not the one we
  recommend for users.  IMHO, adding more complexity to that is a bad
  idea.  I'd rather keep `bootstrap' simple for end users (i.e., have it
  proceed from the point in which it stopped, for whatever reason) and
  create alternate rules for developers.

If "make bootstrap" is supposed to be simple and safe, we should always
do a complete bootstrap and not try to resume an old one which is bound
to lead to nasty hard-to-reproduce problems. Indeed we can use alternate
rules for developers for whom the reduced recompilations might outweigh
the extra complexity.

Re: what should bootstrap *really* do?

[Richard Kenner]

    End users don't change something often.  And, if they do it too late,
    they'll sure get a compare failure.

Well if they're not changing anything, what are they doing to recover
from the failure?

This is my problem: if somebody runs a bootstrap and it fails, they are
going to be tempted to change something.  So I agree that end users don't
change something often, but it's precisely when they will be restarting
bootstraps that they do!

Re: what should bootstrap *really* do?

[Alexandre Oliva]

On Nov 14, 2000, Hans-Peter Nilsson <hans-peter.nilsson@axis.com> wrote:

> Anything other than nil as the initial state for end-users running
> bootstrap is adding trouble.

This of all those folks who have flakey problems that cause the build
to fail at random.  Think of people who build GCC on MS-Windows.
Think of people who want to report a problem, but the error message
has just run off their screen.  Why should they have to restart
bootstrap from scratch?  My (unproved) claim is that the number of
people in this kind of situation surpasses the number of people who
make a minor change here or there, that would require bootstrap to be
restarted, so we should privilege this (alleged) majority with the
default behavior of `make bootstrap'.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: what should bootstrap *really* do?

[Richard Kenner]

    I could do all of those things with the old build system.  I can't do
    them with the new one.

Well, actually you can by removing the right stage_? file and doing
"make bootstrap" again, but it's unwieldly and error prone.

    About the only rule that I might have added to the old system was "make 
    unstageX" (X=1,2,3...), which would effectively reverse the effect of 
    "make stageX".

I most definitely agree with that!  Indeed, this is a manual process for
me that's also very error-prone.

Re: what should bootstrap *really* do?

[Alexandre Oliva]

On Nov 14, 2000, kenner@vlsi1.ultra.nyu.edu (Richard Kenner) wrote:

>     I'd rather keep `bootstrap' simple for end users (i.e., have it
>     proceed from the point in which it stopped, for whatever reason) and
>     create alternate rules for developers.

> I disagree that's "simple"!  It leaves a serious risk of something changed
> in the middle and the bootstrap not being restarted.

End users don't change something often.  And, if they do it too late,
they'll sure get a compare failure.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: what should bootstrap *really* do?

[Hans-Peter Nilsson]

> From: Alexandre Oliva <aoliva@redhat.com>
> Date: 14 Nov 2000 16:16:17 -0200

> On Nov 14, 2000, Hans-Peter Nilsson <hans-peter.nilsson@axis.com> wrote:
> > Ok, bad name.  Not a native speaker and so on.
> > Perhaps bootstrap-continue?
> 
> I already find it bad enough that the default rule is not the one we
> recommend for users.  IMHO, adding more complexity to that is a bad
> idea.  I'd rather keep `bootstrap' simple for end users (i.e., have it
> proceed from the point in which it stopped, for whatever reason) and
> create alternate rules for developers.

I share the opinion that "bootstrap" should do from scratch and
be what is used by end users, while other targets, like
bootstrap-whatever, is for developers.  Anything other than nil
as the initial state for end-users running bootstrap is adding
trouble.

Imagine: `So your installation failed.  How many times did you
write "make bootstrap" and what were the messages at the points
of failure and the reasons?'

brgds, H-P

Re: what should bootstrap *really* do?

[Richard Kenner]

    I'd rather keep `bootstrap' simple for end users (i.e., have it
    proceed from the point in which it stopped, for whatever reason) and
    create alternate rules for developers.

I disagree that's "simple"!  It leaves a serious risk of something changed
in the middle and the bootstrap not being restarted.

Re: what should bootstrap *really* do?

[Alexandre Oliva]

On Nov 14, 2000, Hans-Peter Nilsson <hans-peter.nilsson@axis.com> wrote:

>> From: Alexandre Oliva <aoliva@redhat.com>
>> Date: 14 Nov 2000 15:54:01 -0200

>> On Nov 14, 2000, kenner@vlsi1.ultra.nyu.edu (Richard Kenner) wrote:
>> 
>> >   [No, I (H-P) wrote this, don't blame Kenner for it:]
>> >     Then why not have a *new* target with the *new* (restart-in-middle)
>> >     functionality instead of changing the "bootstrap" target?  The name
>> >     "bootstrap-restart" comes to mind. :-)
>> 
>> > That's my current thinking as well.
>> 
>> My expectation from `bootstrap-restart' is that it would start
>> bootstrap from stratch, not proceed from where it stopped last time.

> Ok, bad name.  Not a native speaker and so on.
> Perhaps bootstrap-continue?

I already find it bad enough that the default rule is not the one we
recommend for users.  IMHO, adding more complexity to that is a bad
idea.  I'd rather keep `bootstrap' simple for end users (i.e., have it
proceed from the point in which it stopped, for whatever reason) and
create alternate rules for developers.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: what should bootstrap *really* do?

[Joern Rennecke]

> My expectation from `bootstrap-restart' is that it would start
> bootstrap from stratch, not proceed from where it stopped last time.

bootstrap-resume ?

Re: what should bootstrap *really* do?

[Hans-Peter Nilsson]

> From: Alexandre Oliva <aoliva@redhat.com>
> Date: 14 Nov 2000 15:54:01 -0200

> On Nov 14, 2000, kenner@vlsi1.ultra.nyu.edu (Richard Kenner) wrote:
> 
> >   [No, I (H-P) wrote this, don't blame Kenner for it:]
> >     Then why not have a *new* target with the *new* (restart-in-middle)
> >     functionality instead of changing the "bootstrap" target?  The name
> >     "bootstrap-restart" comes to mind. :-)
> 
> > That's my current thinking as well.
> 
> My expectation from `bootstrap-restart' is that it would start
> bootstrap from stratch, not proceed from where it stopped last time.

Ok, bad name.  Not a native speaker and so on.
Perhaps bootstrap-continue?

brgds, H-P

Re: what should bootstrap *really* do?

[Alexandre Oliva]

On Nov 14, 2000, kenner@vlsi1.ultra.nyu.edu (Richard Kenner) wrote:

>     Then why not have a *new* target with the *new* (restart-in-middle)
>     functionality instead of changing the "bootstrap" target?  The name
>     "bootstrap-restart" comes to mind. :-)

> That's my current thinking as well.

My expectation from `bootstrap-restart' is that it would start
bootstrap from stratch, not proceed from where it stopped last time.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: what should bootstrap *really* do?

[Alexandre Oliva]

On Nov 14, 2000, DJ Delorie <dj@redhat.com> wrote:

>> I think we should just build the stages in their stage directories, but
>> whenever a new stage is started, make / change a symlink to point to
>> the new stage directory.

> An interesting side effect of this is that it would mark the last
> *successful* build if you did it right, which might not always be the
> last stage you tried to build.

I suppose we should arrange for this ``successful'' link/file to be
created only at the end of a `make all' or a `make bootstrap', not
after each stage is built.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: what should bootstrap *really* do?

[Richard Kenner]

    Then why not have a *new* target with the *new* (restart-in-middle)
    functionality instead of changing the "bootstrap" target?  The name
    "bootstrap-restart" comes to mind. :-)

That's my current thinking as well.

Re: what should bootstrap *really* do?

[DJ Delorie]

> I think we should just build the stages in their stage directories, but
> whenever a new stage is started, make / change a symlink to point to
> the new stage directory.

An interesting side effect of this is that it would mark the last
*successful* build if you did it right, which might not always be the
last stage you tried to build.  I'm not sure if that's best or not.

Re: what should bootstrap *really* do?

[Joern Rennecke]

> 
> On Nov 14, 2000, DJ Delorie <dj@redhat.com> wrote:
> 
> > Hmmm... This means that the "last" stage should always be built in
> > cwd, not in a stage directory, so that "make install" works.
> 
> The problem is how to define `last'.  Currently, one can do `make
> all', `make bootstrap' and `make bootstrap4' all in the same tree.  If
> we start building stages in subdirectories, we'd probably lose this
> ability, and I can see people complaining more loudly about this loss
> than about the need for deleting stage stamp files before bootstrap
> will start scratch.

I think we should just build the stages in their stage directories, but
whenever a new stage is started, make / change a symlink to point to
the new stage directory.
Or we could use a plain text file with the contents being the stage
directory to allow for systems that don't have symlinks.

Re: what should bootstrap *really* do?

[Richard Kenner]

    Thinking of someone whose build machine crashes part-way through the
    build, runs out of disk space, needs to tweak a configuration
    parameter here or there, I think `make bootstrap' should proceed from
    the stage in which the previous build stopped, not from stage1.

I don't.  Who knows how much havoc that running out of disk space or
"tweaked" parameter has or will cause?  Developers may be able to accurately
determine that, but users can't.

If *anything* goes wrong, they should be forced to start over again so
they can have confidence in the result.

RE: what should bootstrap *really* do?

[Billinghurst, David (CRTS)]

I happened to have just completed a bootstrap, so I tried "make -j 4 clean
bootstrap"  It generated a lot of activity.  

Seems to have:
 o decided the bootstrap was complete
 o deleted all the files
 o tried to compare them 
As it was a parallel make it wasn't entirely clear what was happening.

> -----Original Message-----
> From:	DJ Delorie [SMTP:dj@redhat.com]
> Sent:	Tuesday, 14 November 2000 15:29
> To:	aoliva@redhat.com
> Cc:	gcc@gcc.gnu.org
> Subject:	Re: what should bootstrap *really* do?
> 
> 
> Has anyone actually *tried* "make clean bootstrap" yet?
> 
> 

Re: what should bootstrap *really* do?

[Alexandre Oliva]

On Nov 14, 2000, DJ Delorie <dj@redhat.com> wrote:

> Hmmm... This means that the "last" stage should always be built in
> cwd, not in a stage directory, so that "make install" works.

The problem is how to define `last'.  Currently, one can do `make
all', `make bootstrap' and `make bootstrap4' all in the same tree.  If
we start building stages in subdirectories, we'd probably lose this
ability, and I can see people complaining more loudly about this loss
than about the need for deleting stage stamp files before bootstrap
will start scratch.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: what should bootstrap *really* do?

[Alexandre Oliva]

On Nov 14, 2000, DJ Delorie <dj@redhat.com> wrote:

>> I like the idea of cleanstrap,

> Has anyone actually *tried* "make clean bootstrap" yet?

It won't work reliably with -j.  You have to `make clean && make -jN
bootstrap'.  No big deal, but cleanstrap would make it at least easier
to type :-)

>> and that of cleaning the next stage in case you rebuild anything in
>> a certain stage, but I'd rather have give this a different target
>> name, such as depstrap.

> Part of my goals with this question is what should the *default* do.

I have a strong opinion that, if no sources have been changed, it
should proceed from the stage where the previous build stopped.
If sources have changed, my opinion regarding `make bootstrap' is
still the same, but it's no longer strong :-)

>> Another alternative is to have this stuff in some other directory,
>> and have each stage `cd that/directory && $(MAKE) fixinc' or
>> `libgcc1'.

> Judicious use of symlinks can make all the stages see the same real
> directories for those.

Unfortunately, symlinks aren't portable, especially symlinks of
directories.  Not that I know of a system that supports symlinks of
files but not of directories, it's just that `ln -s' is often
``emulated'' by `cp' on systems that don't have symlinks.  In the case
of directories, you'd need `cp -R', but I don't think that's portable.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: what should bootstrap *really* do?

[DJ Delorie]

> IMO, whatever works best for users, since that's the recommended
> installation procedure.  Thinking of someone whose build machine
> crashes part-way through the build, runs out of disk space, needs to
> tweak a configuration parameter here or there, I think `make
> bootstrap' should proceed from the stage in which the previous build
> stopped, not from stage1.

Hmmm... This means that the "last" stage should always be built in
cwd, not in a stage directory, so that "make install" works.  Good
tip!

> Needs of developers are certainly different.

That's why I'm asking ;-)

> I like the idea of cleanstrap,

Has anyone actually *tried* "make clean bootstrap" yet?

> and that of cleaning the next stage in case you rebuild anything in
> a certain stage, but I'd rather have give this a different target
> name, such as depstrap.

Part of my goals with this question is what should the *default* do.

> Another alternative is to have this stuff in some other directory,
> and have each stage `cd that/directory && $(MAKE) fixinc' or
> `libgcc1'.

Judicious use of symlinks can make all the stages see the same real
directories for those.

Re: what should bootstrap *really* do?

[Alexandre Oliva]

On Nov 14, 2000, DJ Delorie <dj@redhat.com> wrote:

> I've heard lots of conflicting uses for "make bootstrap".  What
> exactly *should* a bootstrap do?

IMO, whatever works best for users, since that's the recommended
installation procedure.  Thinking of someone whose build machine
crashes part-way through the build, runs out of disk space, needs to
tweak a configuration parameter here or there, I think `make
bootstrap' should proceed from the stage in which the previous build
stopped, not from stage1.

Needs of developers are certainly different.  I like the idea of
cleanstrap, and that of cleaning the next stage in case you rebuild
anything in a certain stage, but I'd rather have give this a different
target name, such as depstrap.

> Also, when I put builds in stage directories, things like libgcc1 and
> fixincludes happen for each stage.  I'm not sure if this matters or
> not to people.

Probably not, as long as bootstrap-lean still works.  Another
alternative is to have this stuff in some other directory, and have
each stage `cd that/directory && $(MAKE) fixinc' or `libgcc1'.  We
might even do it for the whole libgcc, since I doesn't see much sense
in rebuilding it for every stage.  This would be significant
build-time savings on targets with lots of multilibs.  After all,
`make compare' doesn't compare libgcc object files any more, and
comparing those of gcc itself is probably enough.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

what should bootstrap *really* do?

[DJ Delorie]

I've heard lots of conflicting uses for "make bootstrap".  What
exactly *should* a bootstrap do?  I'm working on a patch that will
build each stage in its own subdirectory, so that a source change will
cause each stage to build what's needed.  However, this really doesn't
*test* the new compiler; you really need to rebuild all of stage N if
stage N-1 changes at all.  Plus, you really don't need to rebuild all
of stage 1 once it's built; if you do you aren't testing gcc, you're
just wasting time.  However, incrementally building just stage1 with
the current scheme is tedious.

So, I see "make bootstrap" doing this:

* build stage1

* if anything of significance changed in stage1, delete *.[oa] in stage 2

* build stage2

* repeat for stage3/4

Alternatives:

"make quickstrap" - skip delete *.o step
"make cleanstrap" - rm stage*/* first

Also, when I put builds in stage directories, things like libgcc1 and
fixincludes happen for each stage.  I'm not sure if this matters or
not to people.

Comments?