Re: native cc re-used after 3-stage now

Re: native cc re-used after 3-stage now

[Kaveh R. Ghazi]

 > From: "Kaveh R. Ghazi" <ghazi@caip.rutgers.edu>
 > 
 >  > From: Robert Lipe <robertl@sco.com>
 >  > 
 >  > After yesterday's commit (Thank you, Philipp) the gcc directory now does
 >  > successfully complete a three-stage from native.  But there is a catch.
 >  > 
 >  > The top level make bootstrap goes all through the build of GCC itself,
 >  > then when it decides to move along, it goes BACK into the GCC directory
 >  > and decides to rebuild cpp.  Of course, it does this with native cc
 >  > which wont' work becuase many of the objects have been build with the
 >  > generated cc.  As another possible hint, it seems to be rerunning
 >  > fixincludes on each pass now.
 >  > 
 >  > Does anyone recognize this?
 >  > make[2]: Entering directory `/home2/negcs/gcc'
 >  > /bin/cc  -DIN_GCC    -g  -DHAVE_CONFIG_H  -o cpp cppmain.o \
 >  > intl.o libcpp.a obstack.o alloca.o  ./intl/libintl.a     ../libiberty/libiberty.a
 >  > Undefined                       first referenced
 >  >  symbol                             in file
 >  > __divdi3                            libcpp.a(cppexp.o)
 >  > __udivdi3                           libcpp.a(cppexp.o)
 >  > __umoddi3                           libcpp.a(cppexp.o)
 >  > __moddi3                            libcpp.a(cppexp.o)
 >  > UX:ld: ERROR: cpp: fatal error: Symbol referencing errors. No output written to cpp
 > 
 > Having used native cc for stage1, I can confirm I am seeing the same
 > problem on sunos4 and solaris2.7.
 > 
 > Relinking cpp also happened to me on x86-linux, but because I had used
 > the system's gcc for stage1, it found the libgcc.a symbols in the
 > system copy instead of the CVS copy.
 > 
 > Of course that's still wrong, but the build proceeded so people using
 > gcc for stage1 may not notice it.
 > 
 > 		--Kaveh

I looked at this a bit.  For some reason, in stage2 and stage3 cpp is
being linked with libintl.a first and then libintl.a is rebuilt.  This
causes cpp's dependency to be out of date later on.

It might be related to HJ's intl parallel make problem:
http://gcc.gnu.org/ml/gcc-bugs/2000-06/msg00264.html

However I can't tell if the patch posted below for that is the right
approach.  Although I see an explanation of the symptoms in the
posting above, the patch below didn't explain how/why the proposed fix
addressed the problem.
http://gcc.gnu.org/ml/gcc-patches/2000-06/msg00297.html


		--Kaveh
--
Kaveh R. Ghazi			Engagement Manager / Project Services
ghazi@caip.rutgers.edu		Qwest Internet Solutions

Re: native cc re-used after 3-stage now

[H . J . Lu]

On Sat, Jun 10, 2000 at 09:01:19AM -0400, Kaveh R. Ghazi wrote:
> 
> I looked at this a bit.  For some reason, in stage2 and stage3 cpp is
> being linked with libintl.a first and then libintl.a is rebuilt.  This
> causes cpp's dependency to be out of date later on.
> 
> It might be related to HJ's intl parallel make problem:
> http://gcc.gnu.org/ml/gcc-bugs/2000-06/msg00264.html
> 
> However I can't tell if the patch posted below for that is the right
> approach.  Although I see an explanation of the symptoms in the
> posting above, the patch below didn't explain how/why the proposed fix
> addressed the problem.
> http://gcc.gnu.org/ml/gcc-patches/2000-06/msg00297.html
> 
> 

This patch is for the parallel make problem I saw. The problem is
the missing dependency in the rule for libintl.a, btw, it is not the
only place in gcc. You cannot do it with parallel build. In my case,
GNU make starts building both libintl.a and intl.all at the same time
since GNU make sees them as independent, unrelated targets. My patch
adds the dependency for libintl.a to make it depend on intl.intl.all
so that GNU make will starts building intl.intl.all first, knowing
libintl.a will be there when intl.intl.all is done.


-- 
H.J. Lu (hjl@gnu.org)

Re: native cc re-used after 3-stage now

[Gerald Pfeifer]

On Sun, 11 Jun 2000, H . J . Lu wrote:
> I guess not many people believe paralell build should be fixed. I will
> shut up now.

Many do, as far as I can see. Please provide descriptions with your
patches and I'm confident that problems you notice will be addressed.

Gerald
-- 
Gerald "Jerry" pfeifer@dbai.tuwien.ac.at http://www.dbai.tuwien.ac.at/~pfeifer/

Re: native cc re-used after 3-stage now

[H . J . Lu]

On Mon, Jun 12, 2000 at 02:55:54PM +0200, Toon Moene wrote:
> H . J . Lu wrote:
> 
> > Thanks for taking it up. In fact, you don't need parallel build on
> > a SMP machine to see the bug. Since there are no dependencies on
> > s-libi77, s-libf77, s-libu77 and s-libe77, with "make -j 100000" in
> > libf2c, GNU make will start to build s-libi77, s-libf77, s-libu77 and
> > s-libe77 at the same time along with other targets. Since there are
> > no rules for them, the build will fail.
> 
> Well, the build will fail if s-lib[ifu]77 are *absent*, because there
> are no rules to build them.  They are supposed to be present when you
> hit the target (from .../egcs/libf2c/Makefile.in):
> 
> $(LIBG2C): s-libi77 s-libf77 s-libu77 s-libe77
> 
> because the sequence of events is as follows:
> 
> all: all-unilib
>         $(MULTIDO) $(FLAGS_TO_PASS) multi-do DO="all-unilib"
> 
> all-unilib: i77 f77 u77 s-libe77
>         $(MAKE) $(FLAGS_TO_PASS) $(LIBG2C)
> 
> i77:
>         cd libI77; $(MAKE) $(FLAGS_TO_PASS) all
> 
> f77:
>         cd libF77; $(MAKE) $(FLAGS_TO_PASS) all
> 
> u77:
>         cd libU77; $(MAKE) $(FLAGS_TO_PASS) all
> 
> s-libe77: f2cext.c
> 	blah...blah...blah
> 
> and it is the responsibility of targets [ifu]77 to generate the stamp
> files s-lib[ifu]77.
> 
> So by the time $MAKE ... $(LIBG2C) is called, [ifu]77 and s-libe77 have
> been satisfied, which means that s-lib[ifu]77 and s-libe77 should be
> present.
> 

You are right and I cannot reproduce it myself. I will keep an eye
on it.

H.J.

Re: native cc re-used after 3-stage now

[Toon Moene]

H . J . Lu wrote:

> Thanks for taking it up. In fact, you don't need parallel build on
> a SMP machine to see the bug. Since there are no dependencies on
> s-libi77, s-libf77, s-libu77 and s-libe77, with "make -j 100000" in
> libf2c, GNU make will start to build s-libi77, s-libf77, s-libu77 and
> s-libe77 at the same time along with other targets. Since there are
> no rules for them, the build will fail.

Well, the build will fail if s-lib[ifu]77 are *absent*, because there
are no rules to build them.  They are supposed to be present when you
hit the target (from .../egcs/libf2c/Makefile.in):

$(LIBG2C): s-libi77 s-libf77 s-libu77 s-libe77

because the sequence of events is as follows:

all: all-unilib
        $(MULTIDO) $(FLAGS_TO_PASS) multi-do DO="all-unilib"

all-unilib: i77 f77 u77 s-libe77
        $(MAKE) $(FLAGS_TO_PASS) $(LIBG2C)

i77:
        cd libI77; $(MAKE) $(FLAGS_TO_PASS) all

f77:
        cd libF77; $(MAKE) $(FLAGS_TO_PASS) all

u77:
        cd libU77; $(MAKE) $(FLAGS_TO_PASS) all

s-libe77: f2cext.c
	blah...blah...blah

and it is the responsibility of targets [ifu]77 to generate the stamp
files s-lib[ifu]77.

So by the time $MAKE ... $(LIBG2C) is called, [ifu]77 and s-libe77 have
been satisfied, which means that s-lib[ifu]77 and s-libe77 should be
present.

PS: Parallel builds are really fun !  I managed to get the load on my
laptop as high as 14.x while building three frontends 4-way parallel :-)
Fortunately, the build just managed to miss my swap file limit by about
8 % ...

-- 
Toon Moene - mailto:toon@moene.indiv.nluug.nl - phoneto: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
GNU Fortran 77: http://gcc.gnu.org/onlinedocs/g77_news.html
GNU Fortran 95: http://g95.sourceforge.net/ (under construction)

Re: native cc re-used after 3-stage now

[Robert Lipe]

Kaveh R. Ghazi wrote:

> PS: Robert, is the cpp relink problem solved for you too?

I see that this morning's (6/11) three-stage built for the first time
since 5/31.  On 6/10 it failed becuase of the problem that inspired this
thread.  So while I don't know precisely which patch cured it, something
committed this weekend cured it.

Thanx to whomever cleaned this up.

RJL

Re: native cc re-used after 3-stage now

[H . J . Lu]

On Sun, Jun 11, 2000 at 06:10:46PM +0200, Toon Moene wrote:
> 
> However, I do think your message is enough for me to start building gcc
> using -j 4 on my machines - even if they're only single CPU - as this
> would give me sufficient chance to catch the real flukes (I hope).
> 
> In the end, we're also building our NWP code four-way parallel, and it
> doesn't hurt - even on the laptop I'm typing this on.
> 

Thanks for taking it up. In fact, you don't need parallel build on
a SMP machine to see the bug. Since there are no dependencies on
s-libi77, s-libf77, s-libu77 and s-libe77, with "make -j 100000" in
libf2c, GNU make will start to build s-libi77, s-libf77, s-libu77 and
s-libe77 at the same time along with other targets. Since there are
no rules for them, the build will fail.



H.J.

Re: native cc re-used after 3-stage now

[Toon Moene]

H . J . Lu wrote:
> 
> On Sun, Jun 11, 2000 at 05:36:24PM +0200, Toon Moene wrote:

> > H . J . Lu wrote:

> > >
> > > Isn't it amazing that there are targets without rules.
> >
> > No.
> >
> > See subdirectories (of the egcs/libf2c directory) libF77, libI77 and
> > libU77 and the Makefile.in's contained therein.
> >
> 
> I know. Do you seriously think it will work every time with paralell
> build on a very fast n-way SMP machine? It is similiar to libintl.a.
> Makefile in libf2c lies. It just works by chance. In my case, GNU make
> starts to build s-libi77 before the builds in subdirectories finish.
> BTW, there are other paralell build bugs in libf2c.  I guess not many
> people believe paralell build should be fixed. I will shut up now.

Well, as always, it would help if you indicate exactly at what point,
and why, the build fails.

There's a rationale (in comments) in the libf2c/Makefile.in for the
setup as it is now - but of course, there could be bugs.

We would like to deal with the bugs in parallel makes, but we need a
more descriptive bug report than the cryptogram you just sent in.

However, I do think your message is enough for me to start building gcc
using -j 4 on my machines - even if they're only single CPU - as this
would give me sufficient chance to catch the real flukes (I hope).

In the end, we're also building our NWP code four-way parallel, and it
doesn't hurt - even on the laptop I'm typing this on.

-- 
Toon Moene - mailto:toon@moene.indiv.nluug.nl - phoneto: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
GNU Fortran 77: http://gcc.gnu.org/onlinedocs/g77_news.html
GNU Fortran 95: http://g95.sourceforge.net/ (under construction)

Re: native cc re-used after 3-stage now

[H . J . Lu]

On Sun, Jun 11, 2000 at 05:36:24PM +0200, Toon Moene wrote:
> H . J . Lu wrote:
> 
> > BTW, while you are on it, you can try
> > 
> > # cd ..../egcs/libf2c
> > # grep s-libu77 Makefile.in
> > 
> > I got
> > 
> > # grep s-libu77 Makefile.in
> > $(LIBG2C): s-libi77 s-libf77 s-libu77 s-libe77
> > 
> > Isn't it amazing that there are targets without rules.
> 
> No.
> 
> See subdirectories (of the egcs/libf2c directory) libF77, libI77 and
> libU77 and the Makefile.in's contained therein.
> 

I know. Do you seriously think it will work every time with paralell
build on a very fast n-way SMP machine? It is similiar to libintl.a.
Makefile in libf2c lies. It just works by chance. In my case, GNU make
starts to build s-libi77 before the builds in subdirectories finish. 
BTW, there are other paralell build bugs in libf2c.  I guess not many
people believe paralell build should be fixed. I will shut up now.



-- 
H.J. Lu (hjl@gnu.org)

Re: native cc re-used after 3-stage now

[Toon Moene]

H . J . Lu wrote:

> BTW, while you are on it, you can try
> 
> # cd ..../egcs/libf2c
> # grep s-libu77 Makefile.in
> 
> I got
> 
> # grep s-libu77 Makefile.in
> $(LIBG2C): s-libi77 s-libf77 s-libu77 s-libe77
> 
> Isn't it amazing that there are targets without rules.

No.

See subdirectories (of the egcs/libf2c directory) libF77, libI77 and
libU77 and the Makefile.in's contained therein.

Cheers,

-- 
Toon Moene - mailto:toon@moene.indiv.nluug.nl - phoneto: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
GNU Fortran 77: http://gcc.gnu.org/onlinedocs/g77_news.html
GNU Fortran 95: http://g95.sourceforge.net/ (under construction)

Re: native cc re-used after 3-stage now

[H . J . Lu]

On Sun, Jun 11, 2000 at 09:42:04AM -0400, Kaveh R. Ghazi wrote:
>  > From: "H . J . Lu" <hjl@valinux.com>
>  > 
>  > This patch is for the parallel make problem I saw. The problem is the
>  > missing dependency in the rule for libintl.a, btw, it is not the only
>  > place in gcc. You cannot do it with parallel build. In my case, GNU
>  > make starts building both libintl.a and intl.all at the same time
>  > since GNU make sees them as independent, unrelated targets. My patch
>  > adds the dependency for libintl.a to make it depend on intl.intl.all
>  > so that GNU make will starts building intl.intl.all first, knowing
>  > libintl.a will be there when intl.intl.all is done.
>  > H.J. Lu (hjl@gnu.org)
> 
> HJ, thanks for the explanation.  Mark's shorter alternative seems to
> have fixed my problem also.  But I appreciate your clarification.
> 

Thanks, Mark. That was my first version. But I got the wrong idea
from the comments in Makefile.in :-).

BTW, while you are on it, you can try

# cd ..../egcs/libf2c
# grep s-libu77 Makefile.in

I got

# grep s-libu77 Makefile.in
$(LIBG2C): s-libi77 s-libf77 s-libu77 s-libe77

Isn't it amazing that there are targets without rules.


-- 
H.J. Lu (hjl@gnu.org)

Re: native cc re-used after 3-stage now

[Kaveh R. Ghazi]

 > From: "H . J . Lu" <hjl@valinux.com>
 > 
 > This patch is for the parallel make problem I saw. The problem is the
 > missing dependency in the rule for libintl.a, btw, it is not the only
 > place in gcc. You cannot do it with parallel build. In my case, GNU
 > make starts building both libintl.a and intl.all at the same time
 > since GNU make sees them as independent, unrelated targets. My patch
 > adds the dependency for libintl.a to make it depend on intl.intl.all
 > so that GNU make will starts building intl.intl.all first, knowing
 > libintl.a will be there when intl.intl.all is done.
 > H.J. Lu (hjl@gnu.org)

HJ, thanks for the explanation.  Mark's shorter alternative seems to
have fixed my problem also.  But I appreciate your clarification.

		--Kaveh

PS: Robert, is the cpp relink problem solved for you too?

--
Kaveh R. Ghazi			Engagement Manager / Project Services
ghazi@caip.rutgers.edu		Qwest Internet Solutions

Re: native cc re-used after 3-stage now

[H . J . Lu]

I checked my log. With my patch, intl/libintl.a in gcc was built
3 times, first with the native cc, second with stage1/xgcc and
third with stage2/xgcc. It makes senses to me.


H.J.

Re: native cc re-used after 3-stage now

[Kaveh R. Ghazi]

 > From: Robert Lipe <robertl@sco.com>
 > 
 > After yesterday's commit (Thank you, Philipp) the gcc directory now does
 > successfully complete a three-stage from native.  But there is a catch.
 > 
 > The top level make bootstrap goes all through the build of GCC itself,
 > then when it decides to move along, it goes BACK into the GCC directory
 > and decides to rebuild cpp.  Of course, it does this with native cc
 > which wont' work becuase many of the objects have been build with the
 > generated cc.  As another possible hint, it seems to be rerunning
 > fixincludes on each pass now.
 > 
 > Does anyone recognize this?
 > make[2]: Entering directory `/home2/negcs/gcc'
 > /bin/cc  -DIN_GCC    -g  -DHAVE_CONFIG_H  -o cpp cppmain.o \
 > intl.o libcpp.a obstack.o alloca.o  ./intl/libintl.a     ../libiberty/libiberty.a
 > Undefined                       first referenced
 >  symbol                             in file
 > __divdi3                            libcpp.a(cppexp.o)
 > __udivdi3                           libcpp.a(cppexp.o)
 > __umoddi3                           libcpp.a(cppexp.o)
 > __moddi3                            libcpp.a(cppexp.o)
 > UX:ld: ERROR: cpp: fatal error: Symbol referencing errors. No output written to cpp

Having used native cc for stage1, I can confirm I am seeing the same
problem on sunos4 and solaris2.7.

Relinking cpp also happened to me on x86-linux, but because I had used
the system's gcc for stage1, it found the libgcc.a symbols in the
system copy instead of the CVS copy.

Of course that's still wrong, but the build proceeded so people using
gcc for stage1 may not notice it.

		--Kaveh
--
Kaveh R. Ghazi			Engagement Manager / Project Services
ghazi@caip.rutgers.edu		Qwest Internet Solutions

Re: native cc re-used after 3-stage now

[Robert Lipe]

David Edelsohn>         Ae you using GNU Make?  This looks like the sort
David Edelsohn> of problems that people encounter when they try to build
David Edelsohn> GCC using the native AIX Make.

Good thought.

On all systems where this fails, it's GNU make, though the precise
version does vary somewhat.  In all cases, it's the same GNU make that
successfully bootstrapped GCC prior to last week.

RJL

Re: native cc re-used after 3-stage now

[David Edelsohn]

	Ae you using GNU Make?  This looks like the sort of problems that
people encounter when they try to build GCC using the native AIX Make.
Also, given that Make 3.79 is available, is this possibly a bug limited to
a specific version of GNU Make (older OR newer?).

David

native cc re-used after 3-stage now

[Robert Lipe]

After yesterday's commit (Thank you, Philipp) the gcc directory now does
successfully complete a three-stage from native.  But there is a catch.

The top level make bootstrap goes all through the build of GCC itself,
then when it decides to move along, it goes BACK into the GCC directory
and decides to rebuild cpp.  Of course, it does this with native cc
which wont' work becuase many of the objects have been build with the
generated cc.  As another possible hint, it seems to be rerunning
fixincludes on each pass now.

Does anyone recognize this?


..
else \
  case "compare" in \
    *-lean ) rm -rf stage$stage ;; \
    *) ;; \
  esac; true; \
fi
make[1]: Leaving directory `/home2/negcs/gcc'
Building runtime libraries
make[1]: Entering directory `/home2/negcs'
make[2]: Entering directory `/home2/negcs/libiberty'
make[3]: Entering directory `/home2/negcs/libiberty/testsuite'
make[3]: Nothing to be done for `all'.
make[3]: Leaving directory `/home2/negcs/libiberty/testsuite'
make[2]: Leaving directory `/home2/negcs/libiberty'
make[2]: Entering directory `/home2/negcs/texinfo'
make all-recursive
make[3]: Entering directory `/home2/negcs/texinfo'
Making all in intl
make[4]: Entering directory `/home2/negcs/texinfo/intl'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory `/home2/negcs/texinfo/intl'
Making all in lib
make[4]: Entering directory `/home2/negcs/texinfo/lib'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory `/home2/negcs/texinfo/lib'
Making all in makeinfo
make[4]: Entering directory `/home2/negcs/texinfo/makeinfo'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory `/home2/negcs/texinfo/makeinfo'
Making all in util
make[4]: Entering directory `/home2/negcs/texinfo/util'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory `/home2/negcs/texinfo/util'
make[3]: Leaving directory `/home2/negcs/texinfo'
make[2]: Leaving directory `/home2/negcs/texinfo'
make[2]: Entering directory `/home2/negcs/etc'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/home2/negcs/etc'
make[2]: Entering directory `/home2/negcs/gcc'
/bin/cc  -DIN_GCC    -g  -DHAVE_CONFIG_H  -o cpp cppmain.o \
intl.o libcpp.a obstack.o alloca.o  ./intl/libintl.a     ../libiberty/libiberty.a
Undefined                       first referenced
 symbol                             in file
__divdi3                            libcpp.a(cppexp.o)
__udivdi3                           libcpp.a(cppexp.o)
__umoddi3                           libcpp.a(cppexp.o)
__moddi3                            libcpp.a(cppexp.o)
UX:ld: ERROR: cpp: fatal error: Symbol referencing errors. No output written to cpp