Re: Makefile change to restart bootstraps

Re: Makefile change to restart bootstraps

[Richard Kenner]

    I can see it's advisable to do it when testing GCC, but remember that
    these rules are also for people who're going to build GCC to use.

Right.

    They used to get quite frustrated when `make bootstrap' re-started the
    build pretty much from scratch, and it would be quite annoying to have
    the bootstrap restarted just because you tweak something in a config.h
    file.

But remember that machines are much faster than when people "used to"
compain!  The bootstrap shouldn't *need* to be "restarted". I'd argue
quite the opposite: if a bootstrap failed for some reason, it's
necessary to understand why and fix it so that it can run completely
through.  Otherwise, how can you have confidence in the result?

And *certainly* if somebody other than a developer "teawked" something in
config.h, I'd want the bootstrap to *definitely* restart from the beginning
since he can't be sure how much it affects.

In fact the "normal user" argument makes it even stronger that we should not
have this sort of restarting bootstrap since it's much harder for normal
users to know they have to take steps to start over than for dvelopers.

Re: Makefile change to restart bootstraps

[Alexandre Oliva]

On Nov 14, 2000, kenner@vlsi1.ultra.nyu.edu (Richard Kenner) wrote:

>     They used to get quite frustrated when `make bootstrap' re-started the
>     build pretty much from scratch, and it would be quite annoying to have
>     the bootstrap restarted just because you tweak something in a config.h
>     file.

> But remember that machines are much faster than when people "used to"
> compain!

Not really.  One of the main problems with having `make bootstrap'
restarting from scratch is that, contrary to most people's
expectations, you won't get the same error you got previously.  Worse
yet: if you're using a compiler other than GCC as the bootstrap
compiler, and the previous build had already got past stage1, it's
very common that linking will fail because the bootstrap compiler
can't understand object files produced by the stage1 gcc, or because
it won't link with libgcc to get whatever auxiliary functions stage1
gcc depends on.

> The bootstrap shouldn't *need* to be "restarted". I'd argue
> quite the opposite: if a bootstrap failed for some reason, it's
> necessary to understand why and fix it so that it can run completely
> through.

So, the only way to do it safely is to have `bootstrap' depend on
`clean'.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: Makefile change to restart bootstraps

[Eric W. Biederman]

kenner@vlsi1.ultra.nyu.edu (Richard Kenner) writes:

>     I can see it's advisable to do it when testing GCC, but remember that
>     these rules are also for people who're going to build GCC to use.
> 
> Right.
> 
>     They used to get quite frustrated when `make bootstrap' re-started the
>     build pretty much from scratch, and it would be quite annoying to have
>     the bootstrap restarted just because you tweak something in a config.h
>     file.
> 
> But remember that machines are much faster than when people "used to"
> compain!  The bootstrap shouldn't *need* to be "restarted". I'd argue
> quite the opposite: if a bootstrap failed for some reason, it's
> necessary to understand why and fix it so that it can run completely
> through.  Otherwise, how can you have confidence in the result?
> 
> And *certainly* if somebody other than a developer "teawked" something in
> config.h, I'd want the bootstrap to *definitely* restart from the beginning
> since he can't be sure how much it affects.

So why aren't dependencies set up correctly in the makefile?

> In fact the "normal user" argument makes it even stronger that we should not
> have this sort of restarting bootstrap since it's much harder for normal
> users to know they have to take steps to start over than for dvelopers.

Again set the dependencies up properly and make knows what needs to be rebuilt
because that's it's job.

Eric

Re: Makefile change to restart bootstraps

[Marc Espie]

In article < 10011141126.AA15195@vlsi1.ultra.nyu.edu > you write:
>But remember that machines are much faster than when people "used to"
>compain!  

Reality check: some of us still try to do maintainance on somewhat slow
boxes.

Until gcc's cross-compiling is much safer than it currently is, you can't
really call a 68040 `fast', neither a vax...

And some of us would really like to get those boxes to stay maintainable...

Re: Makefile change to restart bootstraps

[Alexandre Oliva]

On Nov 14, 2000, Richard Earnshaw <rearnsha@arm.com> wrote:

> BTW  I've just checked RFC822.  It definitely allows multiple To: Cc: & 
> Bcc: entries.

Yep.  It seems to be a bug in sendmail 8.11.0, which is running on
one of the machines that holds my mailbox before I download it.  I'll
try to get it upgraded.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: Makefile change to restart bootstraps

[Alexandre Oliva]

On Nov 13, 2000, Branko &Ccaron;ibej <brane@xbc.nu> wrote:

> This is what I see in the raw message headers, if that helps:

Thanks.  My ``raw message'' doesn't contain the duplicate headers,
either.  There seems to be some piece of software on my end discarding
such duplicate headers.  Now I have to figure out which piece of
software it is.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: Makefile change to restart bootstraps

[Alexandre Oliva]

On Nov 13, 2000, kenner@vlsi1.ultra.nyu.edu (Richard Kenner) wrote:

>     I thought about that, but I rejected it because there may be times
>     when you don't want to restart the bootstrap, but proceed from the
>     stage in which you are.  But that may work, indeed.

> And what are those times?  Aren't every one of those cases places where you
> are asking for trouble by not recompiling everything with the new
> to-be-tested compiler?

I can see it's advisable to do it when testing GCC, but remember that
these rules are also for people who're going to build GCC to use.
They used to get quite frustrated when `make bootstrap' re-started the
build pretty much from scratch, and it would be quite annoying to have
the bootstrap restarted just because you tweak something in a config.h
file.

Since I don't like the idea of having rules for releases different
from those for CVS tree, I'd rather keep the Makefiles as
user-friendly as possible, even if this requires us developers to
slightly change our habits.  I disagree the change that was installed
is any worse for developers tan the previous set up used to be, it's
just different, and most people tend to see the way they're used to as
better than other alternatives that could work as well for them.  I
think this is one such case.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: Makefile change to restart bootstraps

[Alexandre Oliva]

On Nov 13, 2000, Raja R Harinath <harinath@cs.umn.edu> wrote:

>   build_stage1: $(ALL_BOOT_LANGUAGE_SOURCE_FILES)
>         ## there may be some things that need to be cleaned up
>         make clean
>         $(MAKE) CC="$(CC)" ... LANGUAGES="$(BOOT_LANGUAGES)"
>         touch build_stage1
        
Works for me.  But I'd rather not have the `make clean', since there's
no way to avoid it.  But I see why it might be a good idea to have it,
too.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: Makefile change to restart bootstraps

[Branko]

Richard Earnshaw wrote:

> It isn't possible to bcc the mailing lists -- the list name must appear in 
> either the to or the cc filed or the mail will be bounced by the list 
> server (part of the spam protection I guess).  So, no, I've no idea why 
> you don't see the full list (unless it's because my mails can have more 
> than one Cc: line).

This is what I see in the raw message headers, if that helps:


X-Mailer: exmh version 2.0.2 2/24/98
To: Alexandre Oliva <aoliva@redhat.com>
cc: kenner@vlsi1.ultra.nyu.edu (Richard Kenner), gcc@gcc.gnu.org
Cc: rearnsha@arm.com
Reply-To: rearnsha@arm.com


My MUA doesn't show the first cc: header, either. Why do you have two, 
anyway?

-- 
Brane &Ccaron;ibej
    home:   <brane@xbc.nu>             http://www.xbc.nu/brane/
    work:   <branko.cibej@hermes.si>   http://www.hermes-softlab.com/
     ACM:   <brane@acm.org>            http://www.acm.org/

Re: Makefile change to restart bootstraps

[Raja R Harinath]

Hi,

Alexandre Oliva <aoliva@redhat.com> writes:
> On Nov 13, 2000, Raja R Harinath <harinath@cs.umn.edu> wrote:
> > Wouldn't it be simpler if the 'stage_a' or whatever target depended
> > on all the C source files.  This would automatically restart the
> > bootstrap if any of the sources changed.
> 
> I thought about that, but I rejected it because there may be times
> when you don't want to restart the bootstrap, but proceed from the
> stage in which you are.  But that may work, indeed.

The default should be to do the right thing, which for 'make
bootstrap' would be to start from the beginning if any source
changes. 

If you want to restart in the middle even if some file changes, you
could touch the appropriate 'stage_?' file.  To skip the stage1 build,
I think you do

  touch stage_a stage_b

and to skip the stage2 build, you would do

  touch stage_c stage_d

If they were more mnemonic, you could do something like:

  touch build_stage1 # skip building stage1 compiler.  Assume xgcc is stage1
  touch done_stage1  # Assume stage1/xgcc is uptodate

Also, based on an off-line comment from D.J.Delorie, it may be useful
to change

  stage_a:
        $(MAKE) CC="$(CC)" ... LANGUAGES="$(BOOT_LANGUAGES)"
        touch stage_a

to

  build_stage1: $(ALL_BOOT_LANGUAGE_SOURCE_FILES)
        ## there may be some things that need to be cleaned up
        make clean
        $(MAKE) CC="$(CC)" ... LANGUAGES="$(BOOT_LANGUAGES)"
        touch build_stage1
        
And, maybe add a $(ALL_LANGUAGE_SOURCE_FILES) dependency to 'stage_c'
and 'stage_e' (renamed to build_stage2 and build_stage3 resp.).

- Hari
-- 
Raja R Harinath ------------------------------ harinath@cs.umn.edu
"When all else fails, read the instructions."      -- Cahn's Axiom
"Our policy is, when in doubt, do the right thing."   -- Roy L Ash

Re: Makefile change to restart bootstraps

[Richard Kenner]

    I thought about that, but I rejected it because there may be times
    when you don't want to restart the bootstrap, but proceed from the
    stage in which you are.  But that may work, indeed.

And what are those times?  Aren't every one of those cases places where you
are asking for trouble by not recompiling everything with the new
to-be-tested compiler?

Re: Makefile change to restart bootstraps

[Alexandre Oliva]

On Nov 13, 2000, Raja R Harinath <harinath@cs.umn.edu> wrote:

> Wouldn't it be simpler if the 'stage_a' or whatever target depended
> on all the C source files.  This would automatically restart the
> bootstrap if any of the sources changed.

I thought about that, but I rejected it because there may be times
when you don't want to restart the bootstrap, but proceed from the
stage in which you are.  But that may work, indeed.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: Makefile change to restart bootstraps

[Richard Earnshaw]

> If that's the problem, it's just a matter of renaming the stamp files.

Well, it would be a start, but I still don't like the way the stamp files 
break the dependency chains.

> BTW, Richard (Earnshaw), any idea why your replies don't have the
> mailing-list name in neither To: nor Cc:, even when they get delivered
> to me through the mailing list?  Have you been Bcc:ing the mailing
> list, for any reason?

It isn't possible to bcc the mailing lists -- the list name must appear in 
either the to or the cc filed or the mail will be bounced by the list 
server (part of the spam protection I guess).  So, no, I've no idea why 
you don't see the full list (unless it's because my mails can have more 
than one Cc: line).

R.

Re: Makefile change to restart bootstraps

[Alexandre Oliva]

On Nov 13, 2000, Richard Earnshaw <rearnsha@arm.com> wrote:

>> Besides, it's always been possible to restart a bootstrap in the middle if
>> you wanted to: that's what "make bootstrap2" and "make bootstrap3" are for.

> I strongly agree.  The new system relies *far* too heavily on stamp files 
> which are completely non-intuitive, since they bear no relation to what is 
> happening.  stage_[abcde] means nothing to me.

If that's the problem, it's just a matter of renaming the stamp files.


BTW, Richard (Earnshaw), any idea why your replies don't have the
mailing-list name in neither To: nor Cc:, even when they get delivered
to me through the mailing list?  Have you been Bcc:ing the mailing
list, for any reason?

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: Makefile change to restart bootstraps

[Raja R Harinath]

Alexandre Oliva <aoliva@redhat.com> writes:
> On Nov 12, 2000, Mark Mitchell <mark@codesourcery.com> wrote:
> 
> > I agree.  I don't think restarting bootstraps is likely to work very
> > often.
> 
> It is, as long as no source file has changed.  Actually, I've got very
> fond of this feature; it helps on many of the platforms to which I've
> got access.  I'd rather not see it dropped.
> 
> When I'm hunting I bug, I don't bootstrap all the time; I often just
> `make' and check that the fix appears to work before going for a full
> bootstrap.  If I had already bootstrapped that tree, I notice that
> bootstrap completes immediately and remember to remove the stamp
> files.

Wouldn't it be simpler if the 'stage_a' or whatever target depended on
all the C source files.  This would automatically restart the
bootstrap if any of the sources changed.

- Hari
-- 
Raja R Harinath ------------------------------ harinath@cs.umn.edu
"When all else fails, read the instructions."      -- Cahn's Axiom
"Our policy is, when in doubt, do the right thing."   -- Roy L Ash

Re: Makefile change to restart bootstraps

[Richard Kenner]

    I strongly agree.  The new system relies *far* too heavily on stamp files 
    which are completely non-intuitive, since they bear no relation to what is 
    happening.  stage_[abcde] means nothing to me.

Although I'm having trouble visusalizing the details, I think it might be
possible to rename those files so they are actually in the stage directory,
thus the right thing happens move often in the presence of directory copies.

But I think I'm seeing a concensus this, although this might be useful
sometimes, it shouldn't be the default.

What do people thing about renaming this to "make restartable_bootstrap",
letting somebody else do the change above, and putting back the old
"bootstrap" targets?

Re: Makefile change to restart bootstraps

[Richard Earnshaw]

>     The new `make bootstrap' continues from the point where it was
>     interrupted.  The main advantage is that it's restartable.  
> 
> This is the point: I don't see *any* case where that commonly needed. All
> the cases you mention are where it would be a *bad idea* to restart it that
> way since the previous files won't be recompiled with the changed compiler:
> this is a recipe to having undetected problems.
> 
> Besides, it's always been possible to restart a bootstrap in the middle if
> you wanted to: that's what "make bootstrap2" and "make bootstrap3" are for.

I strongly agree.  The new system relies *far* too heavily on stamp files 
which are completely non-intuitive, since they bear no relation to what is 
happening.  stage_[abcde] means nothing to me.

R.

Re: Makefile change to restart bootstraps

[Richard Kenner]

    The new `make bootstrap' continues from the point where it was
    interrupted.  The main advantage is that it's restartable.  

This is the point: I don't see *any* case where that commonly needed. All
the cases you mention are where it would be a *bad idea* to restart it that
way since the previous files won't be recompiled with the changed compiler:
this is a recipe to having undetected problems.

Besides, it's always been possible to restart a bootstrap in the middle if
you wanted to: that's what "make bootstrap2" and "make bootstrap3" are for.

Re: Makefile change to restart bootstraps

[Alexandre Oliva]

On Nov 13, 2000, kenner@vlsi1.ultra.nyu.edu (Richard Kenner) wrote:

> But which compiler do you do that with?

A pre-installed GCC.

> Don't you go back to use
> what's then in the stage1 or stage2 directory?

Nope.  I often don't have stage1 or stage2.  Whenever I bootstrap, I
generally use bootstrap-lean.

> If you're debugging the one in the stage1 directory

I pretty much always debug the one in the gcc directory.  I've only
debugged the compiler in a stage directory once, when I decided to
track down a bootstrap failure the hard way, by running two gdbs in
parallel and seeing where their behaviors diverged.  Ever since, I've
learned to identify the piece of the compiler being miscompiled first,
simplify it to a reduced testcase, and debug GCC from that, so I've
never had to debug a stage1/ or stage2/ compiler again.

> you can always do "make stage2", move
> stage1 to the current directory, fix, make, make stage1, then move
> stage2 to the current directory.  That's the same as the new "make
> bootstrap" behavior.

Huh?  In what sense is it the same as the new `make bootstrap'?  The
new `make bootstrap' continues from the point where it was
interrupted.  The main advantage is that it's restartable.  If, for
*whatever* reason, a bootstrap was not completed, typing `make
bootstrap' won't restart it, which won't work on many platforms if you
don't use GCC as the bootstrap compiler anyway.  Unless you're playing
tricks with moving stage files around, it will always do the right
thing and, if you're playing tricks with stage files, you're expected
to know what you're doing, so you can be expected to be aware of the
stage stamp files too.

> It's *much* less error-prone to always start a bootstrap from the
> beginning.

I suggest that we define a `bootstrap-restart' that cleans up all
stage stamps and does the usual bootstrap.  In fact, creating such a
target has been in my to-do list ever since you complained about it,
but I haven't found the 5 minutes needed to implement it and the other
55 minutes needed to test it yet :-)

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: Makefile change to restart bootstraps

[Richard Kenner]

    When I'm in bug-hunting mode, I use `make', then verify that the
    modified compiler generates correct code for the test case that used
    to fail.

But which compiler do you do that with?  Don't you go back to use
what's then in the stage1 or stage2 directory?  If you're debugging
the one in the stage1 directory, you can always do "make stage2", move
stage1 to the current directory, fix, make, make stage1, then move
stage2 to the current directory.  That's the same as the new "make
bootstrap" behavior.  But both have very serious problems: suppose you
fixed a bug in a later file but broke it with an earlier file?

I think it's just too easy to do something like this an not realize it.
Using the elaborate procedure above at least warns you that something
very special is being done.  With it happening automatically, it's easy
to not notice this is what happened.  It's *much* less error-prone to
always start a bootstrap from the beginning.

    Once I'm satisfied with the result, I type `make bootstrap'.  If I
    find that it completes immediately, I recall I should have cleaned the
    stamp files, so I remove them and `make bootstrap' again.

That's only two of the three cases: if the problem is that the stage1
compiler blows up, "make bootstrap" will do some work, but it may or
may not be what you want.

I think this is very confusing.

Re: Makefile change to restart bootstraps

[Alexandre Oliva]

On Nov 13, 2000, kenner@vlsi1.ultra.nyu.edu (Richard Kenner) wrote:

> I'm sorry, but I don't follow the scenario you are talking about.
> Can you be very precise?

When I'm in bug-hunting mode, I use `make', then verify that the
modified compiler generates correct code for the test case that used
to fail.

Once I'm satisfied with the result, I type `make bootstrap'.  If I
find that it completes immediately, I recall I should have cleaned the
stamp files, so I remove them and `make bootstrap' again.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: Makefile change to restart bootstraps

[Richard Kenner]

    It is, as long as no source file has changed.  Actually, I've got very
    fond of this feature; it helps on many of the platforms to which I've
    got access.  I'd rather not see it dropped.

    When I'm hunting I bug, I don't bootstrap all the time; I often just
    `make' and check that the fix appears to work before going for a full
    bootstrap.  If I had already bootstrapped that tree, I notice that
    bootstrap completes immediately and remember to remove the stamp
    files.

I'm sorry, but I don't follow the scenario you are talking about.
Can you be very precise?

Re: Makefile change to restart bootstraps

[Alexandre Oliva]

On Nov 12, 2000, Mark Mitchell <mark@codesourcery.com> wrote:

> I agree.  I don't think restarting bootstraps is likely to work very
> often.

It is, as long as no source file has changed.  Actually, I've got very
fond of this feature; it helps on many of the platforms to which I've
got access.  I'd rather not see it dropped.

When I'm hunting I bug, I don't bootstrap all the time; I often just
`make' and check that the fix appears to work before going for a full
bootstrap.  If I had already bootstrapped that tree, I notice that
bootstrap completes immediately and remember to remove the stamp
files.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: Makefile change to restart bootstraps

[Mark Mitchell]

>>>>> "Richard" == Richard Kenner <kenner@vlsi1.ultra.nyu.edu> writes:

    Richard> What I'm confused about is when you'd want to do that
    Richard> short of a hardware failure or similar.  Normally when a
    Richard> bootstrap fails, you back up, fix the problem and restart
    Richard> from the beginning.  That seems the normal operating
    Richard> mode, so having the other one seemed strange to me and
    Richard> still does.

I agree.  I don't think restarting bootstraps is likely to work very
often.

--
Mark Mitchell                   mark@codesourcery.com
CodeSourcery, LLC               http://www.codesourcery.com

Makefile change to restart bootstraps

[Richard Kenner]

Let me revisit that again. 

What I'm confused about is when you'd want to do that short of a hardware
failure or similar.  Normally when a bootstrap fails, you back up, fix the
problem and restart from the beginning.  That seems the normal operating
mode, so having the other one seemed strange to me and still does.

Thoughts?