[Bug bootstrap/17817] New: restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

[Bug bootstrap/17817] New: restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

[dberlin at gcc dot gnu dot org]

Before we switched over to building gen* in the build subdir, doing something like:

make bootstrap
<hit bug during stage2 compile, fix bug modify source>
make restage1
make bootstrap
would start again from the end of stage1, including rebuilding the build dir
programs with the stage1 compiler.
This is no longer the case.

-- 
           Summary: restage[1,2,3] and then bootstrap no longer cause build
                    dir to be rebuilt
           Product: gcc
           Version: 4.0.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: bootstrap
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: dberlin at gcc dot gnu dot org
                CC: gcc-bugs at gcc dot gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17817

[Bug bootstrap/17817] restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

[pinskia at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bonzini at gcc dot gnu dot
                   |                            |org
            Summary|restage[1,2,3] and then     |restage[1,2,3] and then
                   |bootstrap no longer cause   |bootstrap no longer cause
                   |build dir to be rebuilt     |build dir to be rebuilt
   Target Milestone|---                         |4.0.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17817

[Bug bootstrap/17817] [4.0 Regression] restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

[pinskia at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|restage[1,2,3] and then     |[4.0 Regression]
                   |bootstrap no longer cause   |restage[1,2,3] and then
                   |build dir to be rebuilt     |bootstrap no longer cause
                   |                            |build dir to be rebuilt


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17817

[Bug bootstrap/17817] [4.0 Regression] restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

[bonzini at gcc dot gnu dot org]

------- Additional Comments From bonzini at gcc dot gnu dot org  2004-10-04 07:37 -------
I can look into this, but the build/ directory is Kelley's pet :-)

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kcook at gcc dot gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17817

[Bug bootstrap/17817] [4.0 Regression] restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

[kcook at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |ASSIGNED
     Ever Confirmed|                            |1
   Last reconfirmed|0000-00-00 00:00:00         |2004-10-04 13:06:07
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17817

[Bug bootstrap/17817] [4.0 Regression] restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

[kcook at gcc dot gnu dot org]

------- Additional Comments From kcook at gcc dot gnu dot org  2004-10-04 16:54 -------
Subject: Re:  [4.0 Regression] restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

I believe that this is currently working as intended.

The files in build/* are only built by the build compiler and do not
get rebuilt between stages unless one of its dependendent source files
changed.  The idea was to speed up bootstraps by not requiring the
generator programs, some of which on some (*cough* darwin) platforms
take a half hour to run, to be needlessly rerun multiple times only to
produce identical output.  For a long winded explanation, see
http://gcc.gnu.org/ml/gcc-patches/2004-09/msg00091.html.

In current CVS, the generator programs are no longer being rebuilt, yet
they are still being rerun.  I have an, as yet unsubmitted, three line
patch which would change the latter behavior to no longer erase the
stamp files between stages for the generated files such as
insn-attrtab.c.  I've been piecemealing the changes in, waiting a week
or two for some negative feedback from the each piece, which you have
now provided.

Do you have a specific example where the new behaviour you have noticed
is causing a problem?

If there is a real problem with this, then restoring previous behavior
of rebuilding the gen* programs each time would be a simple change:
either a one-liner to also stage bconfig.h or with some slightly more
complication properly stage the entire build directory.

Though I may be overlooking something, I'm under the impression that
the benefits of a significantly quicker bootstrap will outweigh the
risks of a missing a hypothetical bug in the stage2 compiler which
causes a problem while generating our generated *.c files, yet not
causing a bootstrap miscompare nor a problem overlooked by the
testsuite.

Also it is noteworthy that none of the behavior you are seeing will
occur with the Paolo's top level bootstrap which I was under the
impression was supposed to become the default RSN.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17817

[Bug bootstrap/17817] [4.0 Regression] restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

[dberlin at dberlin dot org]

------- Additional Comments From dberlin at dberlin dot org  2004-10-04 17:01 -------
Subject: Re:  [4.0 Regression] restage[1,2,3] and then
	bootstrap no longer cause build dir to be rebuilt

On Mon, 2004-10-04 at 16:54 +0000, kcook at gcc dot gnu dot org wrote:
> ------- Additional Comments From kcook at gcc dot gnu dot org  2004-10-04 16:54 -------
> Subject: Re:  [4.0 Regression] restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt
> 
> I believe that this is currently working as intended.
> 
> The files in build/* are only built by the build compiler and do not
> get rebuilt between stages unless one of its dependendent source files
> changed.  The idea was to speed up bootstraps by not requiring the
> generator programs, some of which on some (*cough* darwin) platforms
> take a half hour to run, to be needlessly rerun multiple times only to
> produce identical output.  For a long winded explanation, see
> http://gcc.gnu.org/ml/gcc-patches/2004-09/msg00091.html.

Why not instead fix the programs to run faster?

> 
> In current CVS, the generator programs are no longer being rebuilt, yet
> they are still being rerun.  I have an, as yet unsubmitted, three line
> patch which would change the latter behavior to no longer erase the
> stamp files between stages for the generated files such as
> insn-attrtab.c.  I've been piecemealing the changes in, waiting a week
> or two for some negative feedback from the each piece, which you have
> now provided.
> 
> Do you have a specific example where the new behaviour you have noticed
> is causing a problem?
> 

The real problem is that if you are testing an optimization, and it
causes the generator programs to fail, you can no longer do

make restage1
<fix>
make bootstrap
and have it work, because it will no longer rebuild the generator 
programs with stage1, and will still try to run the broken programs.

I have to actually rm -rf build/* 
make restage1 (which rebuilds them now, and didn't before)
rm -rf build/*
make bootstrap
to get it to do what it used to.

This is somewhat of a pain in the ass to find and fix bugs that you
cause in the generator programs, which is not an uncommon occurrence
when testing new optimizations.

It'd be nice if there was at least some flag controlling this behavior
that i could set in configure or the makefile.


> Though I may be overlooking something, I'm under the impression that
> the benefits of a significantly quicker bootstrap will outweigh the
> risks of a missing a hypothetical bug in the stage2 compiler which
> causes a problem while generating our generated *.c files, yet not
> causing a bootstrap miscompare nor a problem overlooked by the
> testsuite.
I don't see why speeding up the gen* programs isn't an option here.  I
do see that they often produce identical output, and that this is
pointless.
I just wanted to register my complaint that something that used to be a
valuable source of testing (I've had entire optimizations have all their
bugs exposed by gengtype and genattrtab, so that when those two were
working, it bootstrapped and passed the entire testsuite on the first
try).

I guess i'll just do what i do now, or go to start running the testsuite
intsead of also relying upon reubilding generator programs to find bugs.


> 
> Also it is noteworthy that none of the behavior you are seeing will
> occur with the Paolo's top level bootstrap which I was under the
> impression was supposed to become the default RSN.
> 
Oh.

> 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17817

[Bug bootstrap/17817] [4.0 Regression] restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

[bonzini at gnu dot org]

------- Additional Comments From bonzini at gnu dot org  2004-10-04 17:16 -------
Subject: Re:  [4.0 Regression] restage[1,2,3] and then
 bootstrap no longer cause build dir to be rebuilt

> Also it is noteworthy that none of the behavior you are seeing will
> occur with the Paolo's top level bootstrap which I was under the
> impression was supposed to become the default RSN.

No, it will be in the 4.1 timeframe.  There are still problems with 
combined trees.

Also, note that for toplevel bootstrap you'd want the 
STAGEMOVE/STAGECOPY patch you worked on, because it is not as simple as 
keeping stamp files around: in that case you have to really copy the 
STAGECOPY stuff while configuring successive stages (while STAGEMOVE can 
die).

Paolo



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17817

[Bug bootstrap/17817] [4.0 Regression] restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

[kcook at gcc dot gnu dot org]

------- Additional Comments From kcook at gcc dot gnu dot org  2004-10-04 20:16 -------
Subject: PATCH:  restage[1,2,3] and then bootstrap no
 longer cause build dir to be rebuilt

 > The real problem is that if you are testing an optimization, and it
 > causes the generator programs to fail, you can no longer do
 >
 > make restage1
 > <fix>
 > make bootstrap
 > and have it work, because it will no longer rebuild the generator
 > programs with stage1, and will still try to run the broken programs.
 >
 > I have to actually rm -rf build/*
 > make restage1 (which rebuilds them now, and didn't before)
 > rm -rf build/*
 > make bootstrap
 > to get it to do what it used to.
 >
 > This is somewhat of a pain in the ass to find and fix bugs that you
 > cause in the generator programs, which is not an uncommon occurrence
 > when testing new optimizations.

Clearly a valid reason to rebuild the generator executables at each 
stage.  This patch stages the build directory.

Lightly tested to see that requested functionality is restored.

OK to install if full bootstrap succeeds?


2004-10-04  Kelley Cook  <kcook@gcc.gnu.org>

	* Makefile.in: Stage the build directory too.

--- gcc-orig/gcc/Makefile.in	2004-10-01 03:42:49.000000000 -0400
+++ gcc-snapshot/gcc/Makefile.in	2004-10-04 14:27:19.506704700 -0400
@@ -119,7 +119,7 @@ T_ADAFLAGS =
 # See below for how to change them for certain systems.
 
 # List of language subdirectories.
-SUBDIRS =@subdirs@
+SUBDIRS =@subdirs@ build
 
 # Selection of languages to be made.
 CONFIG_LANGUAGES = @all_languages@
@@ -3899,6 +3899,7 @@ stage1-start:
 # If SPECS is overridden, make sure it is `installed' as specs.
 	-mv $(SPECS) stage1/specs
 	-mv $(STAGEMOVESTUFF) stage1
+	-mv build/* stage1/build
 	-cp -p $(STAGECOPYSTUFF) stage1
 # Copy as/ld if they exist to stage dir, so that running xgcc from the stage
 # dir will work properly.
@@ -3937,6 +3938,7 @@ stage2-start:
 # If SPECS is overridden, make sure it is `installed' as specs.
 	-mv $(SPECS) stage2/specs
 	-mv $(STAGEMOVESTUFF) stage2
+	-mv build/* stage2/build
 	-cp -p $(STAGECOPYSTUFF) stage2
 # Copy as/ld if they exist to stage dir, so that running xgcc from the stage
 # dir will work properly.
@@ -3971,6 +3973,7 @@ stage3-start:
 # If SPECS is overridden, make sure it is `installed' as specs.
 	-mv $(SPECS) stage3/specs
 	-mv $(STAGEMOVESTUFF) stage3
+	-mv build/* stage3/build
 	-cp -p $(STAGECOPYSTUFF) stage3
 # Copy as/ld if they exist to stage dir, so that running xgcc from the stage
 # dir will work properly.
@@ -4005,6 +4008,7 @@ stage4-start:
 # If SPECS is overridden, make sure it is `installed' as specs.
 	-mv $(SPECS) stage4/specs
 	-mv $(STAGEMOVESTUFF) stage4
+	-mv build/* stage4/build
 	-cp -p $(STAGECOPYSTUFF) stage4
 # Copy as/ld if they exist to stage dir, so that running xgcc from the stage
 # dir will work properly.
@@ -4037,6 +4041,7 @@ stageprofile-start:
 	   if [ -d stageprofile/$$dir ] ; then true ; else mkdir stageprofile/$$dir ; fi ; \
 	 done
 	-mv $(STAGEMOVESTUFF) stageprofile
+	-mv build/* stageprofile/build
 	-cp -p $(STAGECOPYSTUFF) stageprofile
 # Copy as/ld if they exist to stage dir, so that running xgcc from the stage
 # dir will work properly.
@@ -4069,6 +4074,7 @@ stagefeedback-start:
 	   if [ -d stagefeedback/$$dir ] ; then true ; else mkdir stagefeedback/$$dir ; fi ; \
 	 done
 	-mv $(STAGEMOVESTUFF) stagefeedback
+	-mv build/* stagefeedback/build
 	-cp -p $(STAGECOPYSTUFF) stagefeedback
 # Copy as/ld if they exist to stage dir, so that running xgcc from the stage
 # dir will work properly.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17817

[Bug bootstrap/17817] [4.0 Regression] restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

[paolo dot bonzini at polimi dot it]

------- Additional Comments From paolo dot bonzini at polimi dot it  2004-10-05 06:44 -------
Subject: Re:  [4.0 Regression] restage[1,2,3] and then
 bootstrap no longer cause build dir to be rebuilt

> I don't see why speeding up the gen* programs isn't an option here.  I
> do see that they often produce identical output, and that this is
> pointless.
> I just wanted to register my complaint that something that used to be a
> valuable source of testing (I've had entire optimizations have all their
> bugs exposed by gengtype and genattrtab, so that when those two were
> working, it bootstrapped and passed the entire testsuite on the first
> try).

I was about to suggest to use $(CC_FOR_BUILD) for generator programs, so 
that it is useless to rerun them on each stage, but this is definitely a 
point.

Paolo



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17817

[Bug bootstrap/17817] [4.0 Regression] restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

[cvs-commit at gcc dot gnu dot org]

------- Additional Comments From cvs-commit at gcc dot gnu dot org  2004-10-05 21:51 -------
Subject: Bug 17817

CVSROOT:	/cvs/gcc
Module name:	gcc
Changes by:	kcook@gcc.gnu.org	2004-10-05 21:51:24

Modified files:
	gcc            : Makefile.in ChangeLog 

Log message:
	2004-10-05  Kelley Cook  <kcook@gcc.gnu.org>
	
	PR bootstrap/17817
	* Makefile.in: Stage the build directory too.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/Makefile.in.diff?cvsroot=gcc&r1=1.1406&r2=1.1407
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ChangeLog.diff?cvsroot=gcc&r1=2.5755&r2=2.5756



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17817

[Bug bootstrap/17817] [4.0 Regression] restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2004-10-05 22:15 -------
Fixed.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17817

[Bug bootstrap/17817] [4.0 Regression] restage[1,2,3] and then bootstrap no longer cause build dir to be rebuilt

[bonzini at gcc dot gnu dot org]

------- Additional Comments From bonzini at gcc dot gnu dot org  2004-10-06 07:35 -------
When 4.1 opens, I intend to restore the "buggy" behavior for the toplevel
bootstrap framework since it saves potentially a lot of time, making the full
bootstrap optional for the benefit of those writing optimization passes.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17817