[toplevel bootstrap PATCH] Add profiledbootstrap

[toplevel bootstrap PATCH] Add profiledbootstrap

[Paolo Bonzini]

[-- Attachment #1: Type: text/plain, Size: 497 bytes --]

This patch adds toplevel profiledbootstrap.

Since this makes stage1 have multiple descendants, I had to abandon
the prev/next scheme to link bootstrap-stages.

Instead, I either specify dependancies separately or use double-colon
rules, which allowed commands to be defined (using prev) while the
[+FOR+] has reached the successors of a stage.  This should be portable,
but I'm afraid it falls under the "may turn out to be unmaintainable"
axe.  What do you think?

Ok for mainline/src?

Paolo



[-- Attachment #2: profiledbootstrap.patch --]
[-- Type: text/plain, Size: 10021 bytes --]

2004-05-29  Paolo Bonzini  <bonzini@gnu.org>

	* Makefile.def: Add profile and feedback bootstrap stages.
	Remove next field from bootstrap stages.
	* Makefile.tpl (LN, LN_S): Substitute.
	(stageN-start, stageN-end): Use double-colon rules, to
	provide a hook for additional setup commands.
	(distclean-stageN-gcc, restageN): Create dependencies from
	[+prev+], not from [+next+].
	(stageN-bubble): Add commands for successive stages from
	[+prev+], using double-colon rules.
	(all-stageN-gcc): Fix typo.
	(stagefeedback-start, profiledbootstrap): New.
	* Makefile.in: Regenerate.
	* configure.in: Call ACX_PROG_LN.
	* configure: Regenerate.

config/ChangeLog:
2004-05-29  Paolo Bonzini  <bonzini@gnu.org>

	* config/acx.m4 (ACX_PROG_LN): New macro.

gcc/ChangeLog:
2004-05-29  Paolo Bonzini  <bonzini@gnu.org>

	* aclocal.m4 (gcc_AC_PROG_LN): Remove.
	* configure.ac: Call ACX_PROG_LN, falling back to $LN_S
	if hard links are not available.
	* configure: Regenerate.

Index: Makefile.def
===================================================================
RCS file: /cvs/gcc/gcc/Makefile.def,v
retrieving revision 1.26
diff -u -r1.26 Makefile.def
--- Makefile.def	1 Jun 2004 08:11:32 -0000	1.26
+++ Makefile.def	3 Jun 2004 07:28:56 -0000
@@ -216,25 +216,33 @@
 
 // Toplevel bootstrap
 bootstrap_stage = {
-	id=1 ; next=2 ;
+	id=1 ;
 	extra_configure_flags='--disable-intermodule \
 	  --disable-coverage --enable-languages="$(STAGE1_LANGUAGES)"' ;
 	extra_make_flags='CFLAGS="$(STAGE1_CFLAGS)"' ; };
 bootstrap_stage = {
-	id=2 ; prev=1 ; next=3 ;
+	id=2 ; prev=1 ;
 	bootstrap_target=bootstrap2 ;
 	extra_configure_flags="@stage2_werror_flag@" ;
 	extra_make_flags="" ; };
 bootstrap_stage = {
-	id=3 ; prev=2 ; next=4 ;
+	id=3 ; prev=2 ;
 	compare_target=compare ;
 	bootstrap_target=bootstrap ;
 	cleanstrap_target=cleanstrap ;
 	extra_configure_flags="@stage2_werror_flag@" ;
 	extra_make_flags="" ; };
 bootstrap_stage = {
 	id=4 ; prev=3 ;
 	compare_target=compare3 ;
 	bootstrap_target=bootstrap4 ;
 	extra_configure_flags="@stage2_werror_flag@" ;
 	extra_make_flags="" ; };
+bootstrap_stage = {
+	id=profile ; prev=1 ;
+	extra_configure_flags="@stage2_werror_flag@" ;
+	extra_make_flags='CFLAGS="$(BOOT_CFLAGS) -fprofile-generate"' ; };
+bootstrap_stage = {
+	id=feedback ; prev=1 ;
+	extra_configure_flags="@stage2_werror_flag@" ;
+	extra_make_flags='CFLAGS="$(BOOT_CFLAGS) -fprofile-use"' ; };
Index: Makefile.tpl
===================================================================
RCS file: /cvs/gcc/gcc/Makefile.tpl,v
retrieving revision 1.102
diff -u -r1.102 Makefile.tpl
--- Makefile.tpl	1 Jun 2004 08:11:33 -0000	1.102
+++ Makefile.tpl	3 Jun 2004 07:28:56 -0000
@@ -68,6 +68,8 @@
 INSTALL_PROGRAM = @INSTALL_PROGRAM@
 INSTALL_SCRIPT = @INSTALL_SCRIPT@
 INSTALL_DATA = @INSTALL_DATA@
+LN = @LN@
+LN_S = @LN_S@
 
 # -------------------------------------------------
 # Miscellaneous non-standard autoconf-set variables
@@ -1461,14 +1466,14 @@
 [+ FOR bootstrap-stage +]
 .PHONY: stage[+id+]-start stage[+id+]-end
 
-stage[+id+]-start:
+stage[+id+]-start::
 	[ -f stage_last ] && $(MAKE) `cat stage_last`-end || :
 	echo stage[+id+] > stage_last ; \
 	[ -d stage[+id+]-gcc ] || mkdir stage[+id+]-gcc; \
 	set stage[+id+]-gcc gcc ; @CREATE_LINK_TO_DIR@ [+ IF prev +] ; \
 	set stage[+prev+]-gcc prev-gcc ; @CREATE_LINK_TO_DIR@ [+ ENDIF prev +]
 
-stage[+id+]-end:
+stage[+id+]-end::
 	rm -f stage_last ; \
 	set gcc stage[+id+]-gcc ; @UNDO_LINK_TO_DIR@ [+ IF prev +] ; \
 	set prev-gcc stage[+prev+]-gcc ; @UNDO_LINK_TO_DIR@ [+ ENDIF prev +]
@@ -1477,18 +1482,19 @@
 # are remade, but not reconfigured.  The next stage (if any) will not
 # be reconfigured as well.
 .PHONY: stage[+id+]-bubble
-stage[+id+]-bubble: [+ IF prev +]stage[+prev+]-bubble[+ ENDIF +]
+stage[+id+]-bubble:: [+ IF prev +]stage[+prev+]-bubble[+ ENDIF +]
 	@if [ -f all-stage[+id+]-gcc ] ; then \
 	  echo Remaking stage [+id+] ; \
 	  rm -f all-stage[+id+]-gcc ; \
-	  $(MAKE) $(BASE_FLAGS_TO_PASS) $(RECURSE_FLAGS) \
-	    all-stage[+id+]-gcc [+ IF next +] && \
-	  if [ -f configure-stage[+next+]-gcc ] ; then \
-	    $(STAMP) configure-stage[+next+]-gcc ; \
-	  fi [+ ENDIF next +]; \
-	else \
-	  $(MAKE) $(BASE_FLAGS_TO_PASS) $(RECURSE_FLAGS) all-stage[+id+]-gcc ; \
+	fi ; \
+	$(MAKE) $(BASE_FLAGS_TO_PASS) $(RECURSE_FLAGS) all-stage[+id+]-gcc
+
+[+ IF prev +]
+stage[+prev+]-bubble::
+	@if [ -f configure-stage[+id+]-gcc ] ; then \
+	  $(STAMP) configure-stage[+id+]-gcc ; \
 	fi
+[+ ENDIF prev +]
 
 configure-stage[+id+]-gcc: [+ IF prev +] all-stage[+prev+]-gcc [+
 	  ELSE +] prebootstrap [+ ENDIF prev +]
@@ -1568,11 +1573,13 @@
 
 .PHONY: restage[+id+] distclean-stage[+id+]
 
-distclean-stage[+id+]: [+ IF next +] distclean-stage[+next+] [+ ENDIF next +]
+[+ IF prev +]distclean-stage[+prev+]: distclean-stage[+id+] [+ ENDIF prev +]
+distclean-stage[+id+]:
 	[ -f stage_last ] && $(MAKE) `cat stage_last`-end || :
 	rm -rf configure-stage[+id+]-gcc all-stage[+id+]-gcc stage[+id+]-gcc [+
 	  IF compare-target +][+compare-target+] [+ ENDIF compare-target +]
 
-restage[+id+]: [+ IF next +] distclean-stage[+next+] [+ ENDIF next +]
+[+ IF prev +]restage[+prev+]: restage[+id+] [+ ENDIF prev +]
+restage[+id+]:
 	rm -rf all-stage[+id+]-gcc [+
 	  IF compare-target +][+compare-target+] [+ ENDIF compare-target +]
 	$(MAKE) $(BASE_FLAGS_TO_PASS) $(RECURSE_FLAGS) [+
@@ -1586,6 +1593,33 @@
 [+ ENDIF cleanstrap-target +]
 
 [+ ENDFOR bootstrap-stage +]
+
+stagefeedback-start::
+	@r=`${PWD_COMMAND}`; export r; \
+	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
+	cd stageprofile-gcc && \
+	  { find . -type d | sort | sed 's,.*,$(SHELL) '"$$s"'/mkinstalldirs "../gcc/&",' | $(SHELL); } && \
+	  { find . -name '*.*da' | sed 's,.*,$(LN) -f "&" "../gcc/&",' | $(SHELL); }
+
+profiledbootstrap: all-bootstrap configure-gcc
+	@r=`${PWD_COMMAND}`; export r; \
+	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
+	$(SET_LIB_PATH) \
+	$(GCC_HOST_EXPORTS) \
+	echo "Bootstrapping the compiler"; \
+	$(MAKE) stageprofile-bubble distclean-stagefeedback stageprofile-start
+	@r=`${PWD_COMMAND}`; export r; \
+	s=`cd $(srcdir); ${PWD_COMMAND}` ; export s; \
+	$(SET_LIB_PATH) \
+	echo "Building runtime libraries and training compiler"; \
+	$(MAKE) $(BASE_FLAGS_TO_PASS) $(RECURSE_FLAGS) all
+	@r=`${PWD_COMMAND}`; export r; \
+	s=`cd $(srcdir); ${PWD_COMMAND}`; export s; \
+	$(SET_LIB_PATH) \
+	$(GCC_HOST_EXPORTS) \
+	echo "Building feedback based compiler"; \
+	$(MAKE) stagefeedback-bubble stagefeedback-end
+
 @endif gcc-bootstrap
 
 # --------------------------------------
Index: configure.in
===================================================================
RCS file: /cvs/gcc/gcc/configure.in,v
retrieving revision 1.294
diff -u -r1.294 configure.in
--- configure.in	3 Jun 2004 04:53:00 -0000	1.294
+++ configure.in	3 Jun 2004 07:28:56 -0000
@@ -18,6 +18,8 @@
 ##############################################################################
 ### WARNING: this file contains embedded tabs.  Do not run untabify on this file.
 
+sinclude(config/acx.m4)
+
 AC_INIT(move-if-change)
 AC_PREREQ(2.13)
 AC_CANONICAL_SYSTEM
@@ -25,10 +27,9 @@
 
 # Get 'install' or 'install-sh' and its variants.
 AC_PROG_INSTALL
+ACX_PROG_LN
 AC_PROG_LN_S
 
-sinclude(config/acx.m4)
-
 ### we might need to use some other shell than /bin/sh for running subshells
 ### If we are on Windows, search for the shell.  This will permit people
 ### to not have /bin/sh, but to be able to see /SOME/PATH/sh configure
Index: config/acx.m4
===================================================================
RCS file: /cvs/gcc/gcc/config/acx.m4,v
retrieving revision 1.8
diff -u -r1.8 acx.m4
--- config/acx.m4	24 May 2004 10:50:13 -0000	1.8
+++ config/acx.m4	3 Jun 2004 07:28:56 -0000
@@ -258,3 +258,27 @@
 fi
 ])
 
+dnl See if hard links work and if not, try to substitute $1 or simple copy.
+AC_DEFUN([ACX_PROG_LN],
+[AC_MSG_CHECKING(whether ln works)
+AC_CACHE_VAL(acx_cv_prog_LN,
+[rm -f conftestdata_t
+echo >conftestdata_f
+if ln conftestdata_f conftestdata_t 2>/dev/null
+then
+  acx_cv_prog_LN=ln
+else
+  acx_cv_prog_LN=no
+fi
+rm -f conftestdata_f conftestdata_t
+])dnl
+if test $acx_cv_prog_LN = no; then
+  LN="ifelse([$1],,cp,[$1])"
+  AC_MSG_RESULT([no, using $LN])
+else
+  LN="$acx_cv_prog_LN"
+  AC_MSG_RESULT(yes)
+fi
+AC_SUBST(LN)dnl
+])
+
Index: gcc/aclocal.m4
===================================================================
RCS file: /cvs/gcc/gcc/gcc/aclocal.m4,v
retrieving revision 1.90
diff -u -r1.90 aclocal.m4
--- gcc/aclocal.m4	24 May 2004 10:50:33 -0000	1.90
+++ gcc/aclocal.m4	3 Jun 2004 07:28:56 -0000
@@ -142,38 +142,6 @@
 AC_SUBST(LN_S)dnl
 ])
 
-dnl See if hard links work and if not, try to substitute either symbolic links or simple copy.
-AC_DEFUN([gcc_AC_PROG_LN],
-[AC_MSG_CHECKING(whether ln works)
-AC_CACHE_VAL(gcc_cv_prog_LN,
-[rm -f conftestdata_t
-echo >conftestdata_f
-if ln conftestdata_f conftestdata_t 2>/dev/null
-then
-  gcc_cv_prog_LN="ln"
-else
-  if ln -s conftestdata_f conftestdata_t 2>/dev/null
-  then
-    gcc_cv_prog_LN="ln -s"
-  else
-    gcc_cv_prog_LN=cp
-  fi
-fi
-rm -f conftestdata_f conftestdata_t
-])dnl
-LN="$gcc_cv_prog_LN"
-if test "$gcc_cv_prog_LN" = "ln"; then
-  AC_MSG_RESULT(yes)
-else
-  if test "$gcc_cv_prog_LN" = "ln -s"; then
-    AC_MSG_RESULT([no, using ln -s])
-  else
-    AC_MSG_RESULT([no, and neither does ln -s, so using cp])
-  fi
-fi
-AC_SUBST(LN)dnl
-])
-
 dnl Define MKDIR_TAKES_ONE_ARG if mkdir accepts only one argument instead
 dnl of the usual 2.
 AC_DEFUN([gcc_AC_FUNC_MKDIR_TAKES_ONE_ARG],
Index: gcc/configure.ac
===================================================================
RCS file: /cvs/gcc/gcc/gcc/configure.ac,v
retrieving revision 2.39
diff -u -r2.39 configure.ac
--- gcc/configure.ac	27 May 2004 19:47:36 -0000	2.39
+++ gcc/configure.ac	3 Jun 2004 07:28:56 -0000
@@ -717,8 +717,8 @@
   "") AC_MSG_ERROR([can't build without awk, bailing out]) ;;
 esac
 
-gcc_AC_PROG_LN
 gcc_AC_PROG_LN_S
+ACX_PROG_LN($LN_S)
 AC_PROG_RANLIB
 gcc_AC_PROG_INSTALL
 

Re: [toplevel bootstrap PATCH] Add profiledbootstrap

[Nathanael Nerode]

Paolo Bonzini wrote:
> This patch adds toplevel profiledbootstrap.
> 
> Since this makes stage1 have multiple descendants, I had to abandon
> the prev/next scheme to link bootstrap-stages.
> 
> Instead, I either specify dependancies separately or  use double-colon
> rules, 
The top level isn't allowed to use GNU make, IIRC -- aren't double-colon 
rules a GNU makeism?  :-P  Or are they portable?

(And isn't this a nightmare?  Thanks for your work.)

Re: [toplevel bootstrap PATCH] Add profiledbootstrap

[Daniel Jacobowitz]

On Tue, Jun 08, 2004 at 10:21:25PM -0400, Nathanael Nerode wrote:
> Paolo Bonzini wrote:
> >This patch adds toplevel profiledbootstrap.
> >
> >Since this makes stage1 have multiple descendants, I had to abandon
> >the prev/next scheme to link bootstrap-stages.
> >
> >Instead, I either specify dependancies separately or  use double-colon
> >rules, 
> The top level isn't allowed to use GNU make, IIRC -- aren't double-colon 
> rules a GNU makeism?  :-P  Or are they portable?
> 
> (And isn't this a nightmare?  Thanks for your work.)

Even if they are a GNU makeism, the question is whether they'll cause
parse errors in other Make implementations - the bootstrap rules belong
to GCC.

The GNU make manual does not list them as an extension, so I assume
they are at least a little portable...

-- 
Daniel Jacobowitz

Re: [toplevel bootstrap PATCH] Add profiledbootstrap

[Ian Lance Taylor]

Daniel Jacobowitz <drow@false.org> writes:

> > The top level isn't allowed to use GNU make, IIRC -- aren't double-colon 
> > rules a GNU makeism?  :-P  Or are they portable?
> > 
> > (And isn't this a nightmare?  Thanks for your work.)
> 
> Even if they are a GNU makeism, the question is whether they'll cause
> parse errors in other Make implementations - the bootstrap rules belong
> to GCC.
> 
> The GNU make manual does not list them as an extension, so I assume
> they are at least a little portable...

Double colon rules in Makefiles are portable.  To be portable, a
particular target must have either only single colon rules or only
double colon rules.

(Not that I know what this is about.)

Ian

Re: [toplevel bootstrap PATCH] Add profiledbootstrap

[Paolo Bonzini]

> Even if they are a GNU makeism, the question is whether they'll cause
> parse errors in other Make implementations - the bootstrap rules belong
> to GCC.

Anyway, the bootstrap rules are under @if gcc-bootstrap so they could 
use GNU make rules -- not that this is a good idea IMHO.

> The GNU make manual does not list them as an extension, so I assume
> they are at least a little portable...

Yes, they are.

Paolo

Re: [toplevel bootstrap PATCH] Add profiledbootstrap

[Paolo Bonzini]

> Even if they are a GNU makeism, the question is whether they'll cause
> parse errors in other Make implementations - the bootstrap rules belong
> to GCC.

Anyway, the bootstrap rules are under @if gcc-bootstrap so they could 
use GNU make rules -- not that this is a good idea IMHO.

> The GNU make manual does not list them as an extension, so I assume
> they are at least a little portable...

Yes, they are.

Paolo

Re: [toplevel bootstrap PATCH] Add profiledbootstrap

[Paolo Bonzini]

> (And isn't this a nightmare?  Thanks for your work.)

No, it's quite fun so far...  The nightmare will come when things must 
be made ready for defaulting to --enable-bootstrap... :-)

Paolo

Re: [toplevel bootstrap PATCH] Add profiledbootstrap

[Nathanael Nerode]

With the information given about the portability of double-colon rules,
I can approve this.  So I do.  Please commit.

 
2004-05-29  Paolo Bonzini  <bonzini@gnu.org>

        * Makefile.def: Add profile and feedback bootstrap stages.
        Remove next field from bootstrap stages.
        * Makefile.tpl (LN, LN_S): Substitute.
        (stageN-start, stageN-end): Use double-colon rules, to
        provide a hook for additional setup commands.
        (distclean-stageN-gcc, restageN): Create dependencies from
        [+prev+], not from [+next+].
        (stageN-bubble): Add commands for successive stages from
        [+prev+], using double-colon rules.
        (all-stageN-gcc): Fix typo.
        (stagefeedback-start, profiledbootstrap): New.
        * Makefile.in: Regenerate.
        * configure.in: Call ACX_PROG_LN.
        * configure: Regenerate.

config/ChangeLog:
2004-05-29  Paolo Bonzini  <bonzini@gnu.org>

        * config/acx.m4 (ACX_PROG_LN): New macro.

gcc/ChangeLog:
2004-05-29  Paolo Bonzini  <bonzini@gnu.org>

        * aclocal.m4 (gcc_AC_PROG_LN): Remove.
        * configure.ac: Call ACX_PROG_LN, falling back to $LN_S
        if hard links are not available.
        * configure: Regenerate.

Sometime I'll probably run some more cosmetic cleanup passes on the Makefile.
In case you want to preempt me, here's what I'd do:

(1)  Separate out all the bootstrap targets which call Make recursively on
the same toplevel Makefile into one list, so they can easily be distinguished
from those which don't.  This way, they effectively form a 'higher level'
Makefile; all rules are either at the 'upper level' or the regular level,
never both, and there are only a fixed number of levels.  This increases
maintainability by a surprisingly large factor.

(2) Document all shell variables which are expected to be set when various
Make macros or autoconf substitutions are used in a Make rule.

(3) Make some of the nastier stuff more readable, one way or another.
For instance, this is a readability nightmare:
+         { find . -type d | sort | sed 's,.*,$(SHELL) '"$$s"'/mkinstalldirs "../gcc/&",' | $(SHELL); } && \
+         { find . -name '*.*da' | sed 's,.*,$(LN) -f "&" "../gcc/&",' | $(SHELL); }
but the logic isn't really that complicated.  If it can't be transformed into
a more readable form, it deserves a comment explaining what it's doing.
("Copy all *.*da files from the old directory tree to the new one")

(4) Clean up the stylistic issues with the various LN checks in configure;
ideally there should be one argument-free checking macro for each functional
thing we might try to do, and it isn't that way right now.

-- 
There are none so blind as those who will not see.