[PATCH] Allow parallel toplevel bootstrap

[PATCH] Allow parallel toplevel bootstrap

[Richard Earnshaw]

This patch breaks building with a unified build tree.  We now get:

arm-elf-ar  rc ./libgcc.a libgcc/./_udivsi3.o libgcc/./_divsi3.o
libgcc/./_umodsi3.o libgcc/./_modsi3.o libgcc/./_dvmd_tls.o [...]
libgcc/./gthr-gnat.o libgcc/./unwind-c.o
gnumake[3]: arm-elf-ar: Command not found
gnumake[3]: *** [libgcc.a] Error 127

The problem is in the top-level makefile with the extra level of
recursion that is now used.

AR_FOR_TARGET in this case is intended to expand to ..../binutils/ar,
but the macro to do this is in RECURSE_FLAGS_TO_PASS, and that's
expanded before we've built binutils, so instead of pointing to the
newly created ar it points to the non-existant arm-elf-ar.

Restarting the make after binutils has been built should work ok, but
that's not really the point.

R.

[PATCH] Fix unified tree breakage

[Paolo Bonzini]

[-- Attachment #1: Type: text/plain, Size: 1621 bytes --]


>Restarting the make after binutils has been built should work ok, but
>that's not really the point.
>  
>
The attached patch should fix it.

With the patch, we get this sequence for bootstrap:

        @r=`${PWD_COMMAND}`; export r; \
        s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
        if [ ! -f stage_last ]; then \
          $(MAKE) $(RECURSE_FLAGS_TO_PASS) bootstrap; \
        fi
        @$(unstage)
        @r=`${PWD_COMMAND}`; export r; \
        s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
        $(MAKE) $(TARGET_FLAGS_TO_PASS) all-host
        @r=`${PWD_COMMAND}`; export r; \
        s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
        $(MAKE) $(RECURSE_FLAGS_TO_PASS) all-target
        @$(stage)

for GCC, no bootstrap:

        @$(unstage)
        @r=`${PWD_COMMAND}`; export r; \
        s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
        $(MAKE) $(RECURSE_FLAGS_TO_PASS) all-prebootstrap
        @r=`${PWD_COMMAND}`; export r; \
        s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
        $(MAKE) $(RECURSE_FLAGS_TO_PASS) all-host
        @r=`${PWD_COMMAND}`; export r; \
        s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
        $(MAKE) $(RECURSE_FLAGS_TO_PASS) all-target
        @$(stage)

And for no GCC:

        @$(unstage)
        @r=`${PWD_COMMAND}`; export r; \
        s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
        $(MAKE) $(RECURSE_FLAGS_TO_PASS) all-host
        @r=`${PWD_COMMAND}`; export r; \
        s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
        $(MAKE) $(RECURSE_FLAGS_TO_PASS) all-target
        @$(stage)

Richard, does it work for you?

Paolo

[-- Attachment #2: fix-breakage-for-rearnshaw.patch --]
[-- Type: text/plain, Size: 1266 bytes --]

2005-10-05  Paolo Bonzini  <bonzini@gnu.org>

	* Makefile.tpl (all) [gcc-no-bootstrap]: Make prebootstrap packages
	before other host packages.  Do not use stage_last to find the
	flags to pass to all-host.

Index: Makefile.tpl
===================================================================
RCS file: /cvs/gcc/gcc/Makefile.tpl,v
retrieving revision 1.143
diff -p -u -u -r1.143 Makefile.tpl
--- Makefile.tpl	5 Oct 2005 07:32:13 -0000	1.143
+++ Makefile.tpl	5 Oct 2005 13:56:11 -0000
@@ -709,13 +709,19 @@ all:
 	fi
 @endif gcc-bootstrap
 	@$(unstage)
+@if gcc-no-bootstrap
 	@r=`${PWD_COMMAND}`; export r; \
 	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
-	if [ -f stage_last ]; then \
-	  $(MAKE) $(TARGET_FLAGS_TO_PASS) all-host; \
-	else \
-	  $(MAKE) $(RECURSE_FLAGS_TO_PASS) all-host; \
-	fi
+	$(MAKE) $(RECURSE_FLAGS_TO_PASS) all-prebootstrap
+@endif gcc-no-bootstrap
+	@r=`${PWD_COMMAND}`; export r; \
+	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
+@if gcc-bootstrap
+	$(MAKE) $(TARGET_FLAGS_TO_PASS) all-host
+@endif gcc-bootstrap
+@if gcc-no-bootstrap
+	$(MAKE) $(RECURSE_FLAGS_TO_PASS) all-host
+@endif gcc-no-bootstrap
 	@r=`${PWD_COMMAND}`; export r; \
 	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
 	$(MAKE) $(RECURSE_FLAGS_TO_PASS) all-target

Re: [PATCH] Fix unified tree breakage

[Paolo Bonzini]

[-- Attachment #1: Type: text/plain, Size: 32 bytes --]

This is the correct one.

Paolo

[-- Attachment #2: fix-breakage-for-rearnshaw.patch --]
[-- Type: text/plain, Size: 787 bytes --]

2005-10-05  Paolo Bonzini  <bonzini@gnu.org>

	* Makefile.tpl (all) [gcc-no-bootstrap]: Make prebootstrap packages
	before other host packages.

Index: Makefile.tpl
===================================================================
RCS file: /cvs/gcc/gcc/Makefile.tpl,v
retrieving revision 1.143
diff -p -u -u -r1.143 Makefile.tpl
--- Makefile.tpl	5 Oct 2005 07:32:13 -0000	1.143
+++ Makefile.tpl	5 Oct 2005 14:08:40 -0000
@@ -709,6 +709,11 @@ all:
 	fi
 @endif gcc-bootstrap
 	@$(unstage)
+@if gcc-no-bootstrap
+	@r=`${PWD_COMMAND}`; export r; \
+	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
+	$(MAKE) $(RECURSE_FLAGS_TO_PASS) all-prebootstrap
+@endif gcc-no-bootstrap
 	@r=`${PWD_COMMAND}`; export r; \
 	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
 	if [ -f stage_last ]; then \

Re: [PATCH] Fix unified tree breakage

[Richard Earnshaw]

On Wed, 2005-10-05 at 15:09, Paolo Bonzini wrote:
> This is the correct one.
> 
> Paolo
> 
> ______________________________________________________________________
> 2005-10-05  Paolo Bonzini  <bonzini@gnu.org>
> 
> 	* Makefile.tpl (all) [gcc-no-bootstrap]: Make prebootstrap packages
> 	before other host packages.

That appears to work, thanks.  However, why do you assume this would
only be needed when not doing a bootstrap?  OK, it's not common to want
to use a unified tree for that, but there's no reason why it shouldn't
work.

R.

Re: [PATCH] Fix unified tree breakage

[Paolo Bonzini]

>>2005-10-05  Paolo Bonzini  <bonzini@gnu.org>
>>
>>	* Makefile.tpl (all) [gcc-no-bootstrap]: Make prebootstrap packages
>>	before other host packages.
>>    
>>
>That appears to work, thanks.  However, why do you assume this would
>only be needed when not doing a bootstrap?  OK, it's not common to want
>to use a unified tree for that, but there's no reason why it shouldn't
>work.
>  
>
I'm sorry if this is confusing.  It will go away as soon as GCC 4.1 
branches.

In 4.2, we'll have *toplevel* bootstrap enabled by default.  This is 
what you saw indicated as "@if gcc-bootstrap", and it means that the 
toplevel orchestrates a 3-stage build of libiberty, binutils, and 
everything else up to gcc.  In this case it will just work because 
stage1 will use the system assembler/linker, and stage2/3 will use the 
previous stage of the in-tree assembler/linker: there is no need to do 
"all-prebootstrap", because the packages in "all-prebootstrap" are all 
bootstrapped (together with GCC).

BTW, with toplevel bootstrap, it could make much more sense to use a 
unified tree in a native configuration.

For 4.1, you need --enable-bootstrap to enable toplevel bootstrap.  But 
the default is still non-toplevel bootstrap, and in that case making 
"all-prebootstrap" is still handled with dependencies:

    $(GCC_STRAP_TARGETS): all-prebootstrap configure-gcc
             @r=`pwd`; export r; \
             etc.

Paolo

Re: [PATCH] Fix unified tree breakage

[Richard Earnshaw]

On Wed, 2005-10-05 at 16:23, Paolo Bonzini wrote:
> >>2005-10-05  Paolo Bonzini  <bonzini@gnu.org>
> >>
> >>	* Makefile.tpl (all) [gcc-no-bootstrap]: Make prebootstrap packages
> >>	before other host packages.
> >>    
> >>
> >That appears to work, thanks.  However, why do you assume this would
> >only be needed when not doing a bootstrap?  OK, it's not common to want
> >to use a unified tree for that, but there's no reason why it shouldn't
> >work.
> >  
> >
> I'm sorry if this is confusing.  It will go away as soon as GCC 4.1 
> branches.

:-)

> In 4.2, we'll have *toplevel* bootstrap enabled by default.  This is 
> what you saw indicated as "@if gcc-bootstrap", and it means that the 
> toplevel orchestrates a 3-stage build of libiberty, binutils, and 
> everything else up to gcc.  In this case it will just work because 
> stage1 will use the system assembler/linker, and stage2/3 will use the 
> previous stage of the in-tree assembler/linker: there is no need to do 
> "all-prebootstrap", because the packages in "all-prebootstrap" are all 
> bootstrapped (together with GCC).

Ah!  I get it now.  Sorry, I was confusing gcc-bootstrap with 'make
bootstrap'

This is OK.

R.

Re: [PATCH] Fix unified tree breakage

[Alexandre Oliva]

On Oct  5, 2005, Paolo Bonzini <paolo.bonzini@lu.unisi.ch> wrote:

> In 4.2, we'll have *toplevel* bootstrap enabled by default.  This is
> what you saw indicated as "@if gcc-bootstrap", and it means that the
> toplevel orchestrates a 3-stage build of libiberty, binutils, and
> everything else up to gcc.  In this case it will just work because
> stage1 will use the system assembler/linker, and stage2/3 will use the
> previous stage of the in-tree assembler/linker:

This is not good.  It means the libgcc build uses one pair of
assembler/linker different from the one that gets installed.  Each
stage must use its own assembler/linker to build the GCC libraries.

> there is no need to do "all-prebootstrap", because the packages in
> "all-prebootstrap" are all bootstrapped (together with GCC).

I think this has to be done for all stages, and every stage must use
its own assembler/linker pair to build any target libraries (libgcc
included), otherwise the point of bootstrapping is sort of missed.

> BTW, with toplevel bootstrap, it could make much more sense to use a
> unified tree in a native configuration.

Too bad the unified tree won't be as trivial to create as checking out
uberbaum as soon as GCC moves to subversion :-(

-- 
Alexandre Oliva         http://www.lsd.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: [PATCH] Fix unified tree breakage

[Alexandre Oliva]

On Oct  5, 2005, Paolo Bonzini <paolo.bonzini@lu.unisi.ch> wrote:

> 2005-10-05  Paolo Bonzini  <bonzini@gnu.org>

> 	* Makefile.tpl (all) [gcc-no-bootstrap]: Make prebootstrap packages
> 	before other host packages.

Ok, thanks, but please consider the points in the e-mail I've just
posted downthread and try to address them.  If this means just taking
out the conditionals, it's pre-approved.

-- 
Alexandre Oliva         http://www.lsd.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: [PATCH] Fix unified tree breakage

[Alexandre Oliva]

On Oct  5, 2005, Alexandre Oliva <aoliva@redhat.com> wrote:

> On Oct  5, 2005, Paolo Bonzini <paolo.bonzini@lu.unisi.ch> wrote:
>> In 4.2, we'll have *toplevel* bootstrap enabled by default.  This is
>> what you saw indicated as "@if gcc-bootstrap", and it means that the
>> toplevel orchestrates a 3-stage build of libiberty, binutils, and
>> everything else up to gcc.  In this case it will just work because
>> stage1 will use the system assembler/linker, and stage2/3 will use the
>> previous stage of the in-tree assembler/linker:

> This is not good.  It means the libgcc build uses one pair of
> assembler/linker different from the one that gets installed.  Each
> stage must use its own assembler/linker to build the GCC libraries.

And, just to clarify why it's so important to get this right, when
bootstrapping GCC, starting without GNU binutils, but configuring GCC
to use binutils, you'd likely get assembler/linker failures while
building libgcc for stage1.

-- 
Alexandre Oliva         http://www.lsd.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: [PATCH] Fix unified tree breakage

[Alexandre Oliva]

On Oct  6, 2005, Paolo Bonzini <bonzini@gnu.org> wrote:

> The stage2 and stage3 binaries should be identical (and even the
> stage1 and stage2 binaries should behave the same).  Thus, using the
> previous stage of the in-tree assembler/linker could even be correct,
> but it is not what it is done.

Except for the stage1 case, that's exactly where relying on the
previous stage's assembler/linker could cause things to fall apart.

Anyhow, good to know it's not broken.

That said, the other day I ran into another recent breakage in a
unified tree: gas/as-new is *hard-linked* to gcc/as, as opposed to
being symlinked.  This means that, if I modify the gas sources and
rebuild them, gcc will still use the old assembler.  Oops.

Ditto for linker and binutils.

If hard-linking is a must for whatever reason (such as simplifying
stage pushing and popping), then we must add dependencies to make sure
these binaries remain up to date with respect to what they were
hard-linked to.

> run-tool --target='$$r/$(HOST_SUBDIR)/gas/as-new' \
>           --target='$$r/$(HOST_SUBDIR)/gcc/as' \
>           --host='$(AS)' \
>           '$(CONFIGURED_AS_FOR_TARGET)'"

> finds an in-tree target assembler, falling back to the system host
> assembler in a native configuration (target = host) or then to the
> $(CONFIGURED_AS_FOR_TARGET).

Neat.

> The first parameter specifies the desired triplet for the tool, so
> --host is only considered if the host and target triplets are
> identical.

I think you need some means to avoid running the wrong binary in
Canadian crosses and cross-back tools.  E.g., consider --build=x
--host=y --target=z.  How can run-tool --target=whatever distinguish
between gas/as-new, that's built for the host y and thus won't run on
the build machine x, from gcc/as, presumably copied from a pre-build
--host=x --target=z toolchain, specified in
$(CONFIGURED_AS_FOR_TARGET), or, as a potential nice improvement,
built in the build-x directory.

And then there's --build=x --host=y --target=x, for which you *could*
use the system assembler.

Just something to think about :-)

-- 
Alexandre Oliva         http://www.lsd.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: [PATCH] Fix unified tree breakage

[Bernd Jendrissek]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, Oct 07, 2005 at 11:08:38AM -0300, Alexandre Oliva wrote:
> I think you need some means to avoid running the wrong binary in
> Canadian crosses and cross-back tools.  E.g., consider --build=x
> --host=y --target=z.  How can run-tool --target=whatever distinguish
> between gas/as-new, that's built for the host y and thus won't run on
> the build machine x, from gcc/as, presumably copied from a pre-build
> --host=x --target=z toolchain, specified in
> $(CONFIGURED_AS_FOR_TARGET), or, as a potential nice improvement,
> built in the build-x directory.

I ran into something like this the other day, while building GCC as
- --build=i686-pc-linux-gnu --host=i386-mingw32 --target=arm-unknown-elf.
The resulting GCC wants to run just "as"; if I specify
- --with-as=c:/gnutools/cross/bin/arm-unknown-elf.exe (which is what I
really wanted), the configure script got confused about what exactly
that --with-as (and same for ld) is: a tool running on the build
machine, or the host machine.  At runtime GCC wants to use it as
DEFAULT_ASSEMBLER to rewrite in the place of "as" in invoke_as, so from
that perspective it is a host tool.  OTOH the configure script expects
that specified assembler to exist and be test -x, which is just
confused.

Eventually I just got my colleague to do gcc -dumpspecs >specs, edit
invoke_as, and then add -specs=specs to $(CC) in the makefile.  Does GCC
really need --with-host-as=/nonexistent/on/build/as, or do I need to
RTFM on --with-sysroot?

For the archives...
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Please fetch my new key 804177F8 from hkp://wwwkeys.eu.pgp.net/
Comment: Key D7CBA633 will soon expire, check its signature of 804177F8 now!

iD8DBQFDRtGawyMv24BBd/gRAoP5AJ0QN3cWbH7mqNRQA4nIRTJteqfBogCeO/9s
ka58VHMt8ZkshNcH+l2aaPY=
=fCj7
-----END PGP SIGNATURE-----

[PATCH] Allow parallel toplevel bootstrap

[Paolo Bonzini]

[-- Attachment #1: Type: text/plain, Size: 137 bytes --]

Here is a resubmit of the patch to make toplevel bootstrap parallel, 
this time without using MAKEOVERRIDES.

:ADDPATCH toplevel:

Paolo

[-- Attachment #2: toplevel-parallel-2.patch --]
[-- Type: text/plain, Size: 11405 bytes --]

2005-07-15  Paolo Bonzini  <bonzini@gnu.org>

	PR target/22340

	* configure.in (default_target): Remove.
	* Makefile.tpl (all): Do not use prerequisites as subroutines
	(all) [gcc-bootstrap]: Bootstrap gcc first if it was not done yet.
	(do-[+make_target+], check, install, [+bootstrap_target+]): Do not
	use prerequisites as subroutines.
	(check-host, check-target): New.
	(bootstrap configure & all targets): Do not use stage*-start
	if the directory layout is already ok.
	(non-bootstrap configure & all targets): Prepend a $(unstage).
	(stage[+id+]-bubble): Do that here.  Do not use NOTPARALLEL.
	(NOTPARALLEL): Remove.
	(unstage, stage variables): New variables.
	(unstage, stage targets): Simply expand to those variables.

Index: configure.in
===================================================================
RCS file: /cvs/gcc/gcc/configure.in,v
retrieving revision 1.362
diff -p -u -u -p -r1.362 configure.in
--- configure.in	8 Aug 2005 09:19:07 -0000	1.362
+++ configure.in	8 Aug 2005 13:00:31 -0000
@@ -1765,18 +1765,14 @@ esac
 case "$enable_bootstrap" in
   yes)
     bootstrap_lean='#'
-    default_target=bootstrap
     bootstrap_suffix=bootstrap ;;
   lean)
     bootstrap_lean=''
-    default_target=bootstrap
     bootstrap_suffix=bootstrap ;;
   no)
-    default_target=all
     bootstrap_suffix=no-bootstrap ;;
 esac
 AC_SUBST(bootstrap_lean)
-AC_SUBST(default_target)
 
 for module in ${build_configdirs} ; do
   if test -z "${no_recursion}" \
Index: Makefile.tpl
===================================================================
RCS file: /cvs/gcc/gcc/Makefile.tpl,v
retrieving revision 1.140
diff -p -u -u -p -r1.140 Makefile.tpl
--- Makefile.tpl	8 Aug 2005 09:19:17 -0000	1.140
+++ Makefile.tpl	8 Aug 2005 13:00:31 -0000
@@ -534,7 +534,7 @@ PICFLAG_FOR_TARGET = 
 
 # The first rule in the file had better be this one.  Don't put any above it.
 # This lives here to allow makefile fragments to contain dependencies.
-@default_target@:
+all:
 
 #### host and target specific makefile fragments come in here.
 @target_makefile_frag@
@@ -666,7 +666,26 @@ configure-target: [+
 
 # The target built for a native non-bootstrap build.
 .PHONY: all
-all: unstage all-host all-target stage
+all:
+@if gcc-bootstrap
+	@r=`${PWD_COMMAND}`; export r; \
+	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
+	if [ ! -f stage_last ]; then \
+	  $(MAKE) $(RECURSE_FLAGS_TO_PASS) bootstrap; \
+	fi
+@endif gcc-bootstrap
+	@$(unstage); \
+	r=`${PWD_COMMAND}`; export r; \
+	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
+	if [ -f stage_last ]; then \
+	  $(MAKE) $(TARGET_FLAGS_TO_PASS) all-host; \
+	else \
+	  $(MAKE) $(RECURSE_FLAGS_TO_PASS) all-host; \
+	fi
+	@r=`${PWD_COMMAND}`; export r; \
+	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
+	$(MAKE) $(RECURSE_FLAGS_TO_PASS) all-target
+	@$(stage)
 
 .PHONY: all-build
 all-build: [+
@@ -690,7 +709,12 @@ all-target: [+
 # but it may do additional work as well).
 [+ FOR recursive_targets +]
 .PHONY: do-[+make_target+]
-do-[+make_target+]: unstage [+make_target+]-host [+make_target+]-target stage
+do-[+make_target+]:
+	@$(unstage)
+	@$(MAKE) $(RECURSE_FLAGS_TO_PASS) [+make_target+]-host \
+	  [+make_target+]-target
+	@$(stage)
+
 
 .PHONY: [+make_target+]-host
 [+make_target+]-host: [+
@@ -770,13 +794,22 @@ clean-target-libgcc:
 check: do-check
 
 # Only include modules actually being configured and built.
-do-check: unstage [+
+.PHONY: check-host
+check-host: [+
   FOR host_modules +] \
     maybe-check-[+module+][+
-  ENDFOR host_modules +][+
+  ENDFOR host_modules +]
+
+.PHONY: check-target
+check-target: [+
   FOR target_modules +] \
     maybe-check-target-[+module+][+
-  ENDFOR target_modules +] stage
+  ENDFOR target_modules +]
+
+do-check:
+	@$(unstage)
+	@$(MAKE) $(RECURSE_FLAGS_TO_PASS) check-host check-target
+	@$(stage)
 
 # Automated reporting of test results.
 
@@ -802,7 +835,10 @@ mail-report-with-warnings.log: warning.l
 # Installation targets.
 
 .PHONY: install uninstall
-install: installdirs install-host install-target
+install:
+	@$(unstage)
+	@$(MAKE) $(RECURSE_FLAGS_TO_PASS) installdirs install-host install-target
+	@$(stage)
 
 .PHONY: install-host-nogcc
 install-host-nogcc: [+
@@ -883,7 +919,8 @@ maybe-configure-[+prefix+][+module+]:
 maybe-configure-[+prefix+][+module+]: configure-[+prefix+][+module+]
 configure-[+prefix+][+module+]: [+deps+]
 	@[+ IF bootstrap +]test -f stage_last && exit 0; \
-	[+ ENDIF bootstrap +]test ! -f [+subdir+]/[+module+]/Makefile || exit 0; \
+	[+ ELSE bootstrap +]$(unstage)
+	@[+ ENDIF bootstrap +]test ! -f [+subdir+]/[+module+]/Makefile || exit 0; \
 	$(SHELL) $(srcdir)/mkinstalldirs [+subdir+]/[+module+] ; \
 	r=`${PWD_COMMAND}`; export r; \
 	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
@@ -910,7 +947,7 @@ maybe-configure-stage[+id+]-[+prefix+][+
 @if [+module+]-bootstrap
 maybe-configure-stage[+id+]-[+prefix+][+module+]: configure-stage[+id+]-[+prefix+][+module+]
 configure-stage[+id+]-[+prefix+][+module+]: [+deps+]
-	@$(MAKE) stage[+id+]-start
+	@[ `cat stage_current` = stage[+id+] ] || $(MAKE) stage[+id+]-start
 	@[ -f [+subdir+]/[+module+]/Makefile ] && exit 0 || : ; \
 	r=`${PWD_COMMAND}`; export r; \
 	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
@@ -942,7 +979,8 @@ TARGET-[+prefix+][+module+]=[+
 maybe-all-[+prefix+][+module+]: all-[+prefix+][+module+]
 all-[+prefix+][+module+]: configure-[+prefix+][+module+]
 	@[+ IF bootstrap +]test -f stage_last && exit 0; \
-	[+ ENDIF bootstrap +]r=`${PWD_COMMAND}`; export r; \
+	[+ ELSE bootstrap +]$(unstage)
+	@[+ ENDIF bootstrap +]r=`${PWD_COMMAND}`; export r; \
 	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
 	[+exports+] \
 	(cd [+subdir+]/[+module+] && \
@@ -959,7 +997,7 @@ maybe-clean-stage[+id+]-[+prefix+][+modu
 maybe-all-stage[+id+]-[+prefix+][+module+]: all-stage[+id+]-[+prefix+][+module+]
 all-stage[+id+]: all-stage[+id+]-[+prefix+][+module+]
 all-stage[+id+]-[+prefix+][+module+]: configure-stage[+id+]-[+prefix+][+module+]
-	@$(MAKE) stage[+id+]-start
+	@[ `cat stage_current` = stage[+id+] ] || $(MAKE) stage[+id+]-start
 	@r=`${PWD_COMMAND}`; export r; \
 	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
 	[+exports+][+ IF prev +] \
@@ -974,7 +1012,7 @@ clean-stage[+id+]: clean-stage[+id+]-[+p
 clean-stage[+id+]-[+prefix+][+module+]:
 	@[ -f [+subdir+]/[+module+]/Makefile ] || [ -f [+subdir+]/stage[+id+]-[+module+]/Makefile ] \
 	  || exit 0 ; \
-	[ -f [+subdir+]/[+module+]/Makefile ] || $(MAKE) stage[+id+]-start ; \
+	@[ `cat stage_current` = stage[+id+] ] || $(MAKE) stage[+id+]-start
 	cd [+subdir+]/[+module+] && \
 	$(MAKE) [+args+] [+ IF prev +] \
 		[+poststage1_args+] [+ ENDIF prev +] \
@@ -1019,6 +1057,7 @@ check-[+module+]:
 [+ ELIF no_check_cross +]
 # This module is only tested in a native toolchain.
 check-[+module+]:
+	@$(unstage)
 	@if [ '$(host)' = '$(target)' ] ; then \
 	  r=`${PWD_COMMAND}`; export r; \
 	  s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
@@ -1028,6 +1067,7 @@ check-[+module+]:
 	fi
 [+ ELSE check +]
 check-[+module+]:
+	@$(unstage)
 	@r=`${PWD_COMMAND}`; export r; \
 	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
 	$(HOST_EXPORTS) \
@@ -1044,6 +1084,7 @@ maybe-install-[+module+]: install-[+modu
 install-[+module+]:
 [+ ELSE install +]
 install-[+module+]: installdirs
+	@$(unstage)
 	@r=`${PWD_COMMAND}`; export r; \
 	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
 	$(HOST_EXPORTS) \
@@ -1066,7 +1107,8 @@ maybe-[+make_target+]-[+module+]: [+make
   FOR depend +]\
     [+depend+]-[+module+] [+
   ENDFOR depend +]
-	@[ -f ./[+module+]/Makefile ] || exit 0; \
+	@[+ IF bootstrap +][+ ELSE +]$(unstage)
+	@[+ ENDIF bootstrap +][ -f ./[+module+]/Makefile ] || exit 0; \
 	r=`${PWD_COMMAND}`; export r; \
 	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
 	$(HOST_EXPORTS) \
@@ -1128,6 +1170,7 @@ maybe-check-target-[+module+]: check-tar
 check-target-[+module+]:
 [+ ELSE check +]
 check-target-[+module+]:
+	@$(unstage)
 	@r=`${PWD_COMMAND}`; export r; \
 	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \[+
 IF raw_cxx +]
@@ -1153,6 +1196,7 @@ maybe-install-target-[+module+]: install
 install-target-[+module+]:
 [+ ELSE install +]
 install-target-[+module+]: installdirs
+	@$(unstage)
 	@r=`${PWD_COMMAND}`; export r; \
 	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \[+
 IF raw_cxx +]
@@ -1179,7 +1223,8 @@ maybe-[+make_target+]-target-[+module+]:
   FOR depend +]\
     [+depend+]-target-[+module+] [+
   ENDFOR depend +]
-	@[ -f $(TARGET_SUBDIR)/[+module+]/Makefile ] || exit 0 ; \
+	@$(MAKE) unstage; \
+	[ -f $(TARGET_SUBDIR)/[+module+]/Makefile ] || exit 0 ; \
 	r=`${PWD_COMMAND}`; export r; \
 	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \[+
 IF raw_cxx +]
@@ -1320,17 +1365,23 @@ gcc-no-fixedincludes:
 # stage_last instead tracks the stage that was built last.  These targets
 # are dummy when toplevel bootstrap is not active.
 
-.PHONY: unstage
-unstage:
+# While making host and target tools, symlinks to the final stage must be
+# there, so $(MAKE) $(unstage) should be run at various points.  To avoid
+# excessive recursive invocations of make, we "inline" them using a variable.
+
+unstage = :
+stage = :
+
 @if gcc-bootstrap
-	@[ -f stage_current ] || $(MAKE) `cat stage_last`-start
+unstage = [ -f stage_current ] || $(MAKE) `cat stage_last`-start
+stage = $(MAKE) `cat stage_current`-end
 @endif gcc-bootstrap
 
-.PHONY: stage
+.PHONY: unstage stage
+unstage:
+	@$(unstage)
 stage:
-@if gcc-bootstrap
-	@$(MAKE) `cat stage_current`-end
-@endif gcc-bootstrap
+	@$(stage)
 
 # We name the build directories for the various stages "stage1-gcc",
 # "stage2-gcc","stage3-gcc", etc.
@@ -1343,19 +1394,6 @@ stage:
 # mv on platforms where symlinks to directories do not work or are not
 # reliable.
 
-# At the end of the bootstrap, a symlink to 'stage3-gcc' named 'gcc' must
-# be kept, so that libraries can find it.  Ick!
-
-# It would be best to preinstall gcc into a staging area (and in the
-# future, gather there all prebootstrap packages).  This would allow
-# assemblers and linkers can be bootstrapped as well as the compiler
-# (both in a combined tree, or separately).  This however requires some
-# change to the gcc driver, again in order to avoid comparison failures.
-
-# Bugs: This is crippled when doing parallel make, the `make all-host'
-# and `make all-target' phases can be parallelized.
-
-
 # 'touch' doesn't work right on some platforms.
 STAMP = echo timestamp > 
 
@@ -1440,7 +1478,8 @@ stage[+id+]-bubble:: [+ IF prev +]stage[
 	  IF prev +]|| test -f stage[+prev+]-lean [+ ENDIF prev +] ; then \
 	  echo Skipping rebuild of stage[+id+] ; \
 	else \
-	  $(MAKE) $(RECURSE_FLAGS_TO_PASS) NOTPARALLEL= all-stage[+id+]; \
+	  $(MAKE) stage[+id+]-start; \
+	  $(MAKE) $(RECURSE_FLAGS_TO_PASS) all-stage[+id+]; \
 	fi
 
 .PHONY: all-stage[+id+] clean-stage[+id+]
@@ -1480,7 +1519,9 @@ do-clean: clean-stage[+id+]
 
 [+ IF bootstrap-target +]
 .PHONY: [+bootstrap-target+]
-[+bootstrap-target+]: stage[+id+]-bubble [+compare-target+] all
+[+bootstrap-target+]: stage[+id+]-bubble
+	[+ IF compare-target +]$(MAKE) [+compare-target+]
+	[+ ENDIF compare-target +]$(MAKE) $(RECURSE_FLAGS_TO_PASS) all
 [+ ENDIF bootstrap-target +]
 
 # Rules to wipe a stage and all the following ones, also used for cleanstrap
@@ -1513,8 +1554,6 @@ stagefeedback-start::
 	done
 
 @if gcc-bootstrap
-NOTPARALLEL = .NOTPARALLEL
-$(NOTPARALLEL):
 do-distclean: distclean-stage1
 @endif gcc-bootstrap