public inbox for gcc@gcc.gnu.org
 help / color / mirror / Atom feed
* testing consistency
@ 1997-09-04 18:48 Joe Buck
  1997-09-04 20:13 ` Dave Avery
                   ` (2 more replies)
  0 siblings, 3 replies; 70+ messages in thread
From: Joe Buck @ 1997-09-04 18:48 UTC (permalink / raw)
  To: egcs team

I think that I'd like to see a bit more discipline in how we do testing.
Many of us are doing various tests now, but I do want to make sure that
we cover the cases that users are likely to use.

This means that we should make sure to test the compilers that are built
by following the (yet to be written) instructions exactly, that is,
bootstrap the compiler and build with "-O2 -g", then doing tests with
"clean" environments (nothing else installed: e.g. no libg++ if it isn't
part of egcs).  Additional tests with higher optimization levels and
extra headers are welcome but should be considered "extra".

Ideally, it should be extremely simple to run a "standard test" and
do a full report of results, and mail it back to a standard address.
e.g.

./configure
make
make test_and_report

The idea is that this runs all the test suites, collects all of the
relevant data, and sends it in a mail message.  It would be nice if
cruft is deleted so the mail message isn't full of garbage.

Relevant data that we wouldn't want to miss includes:
	configure options (including platform)
	libc version, if relevant
	assembler and linker versions, if relevant
	gcc options used to compile the final gcc (e.g. -O2 -g)

It may be hard to collect that automatically -- maybe we can just have
a standard "test report template".  It could even be a WWW form.

This process shouldn't be too bureaucratic, but I'd like to be able to
say with confidence that "the following list of platforms passes all
tests" (OK, maybe there is a small set of known failures).



^ permalink raw reply	[flat|nested] 70+ messages in thread
* Re: testing consistency
@ 1997-09-05 18:09 meissner
  1997-09-05 18:55 ` Joel Sherrill
  0 siblings, 1 reply; 70+ messages in thread
From: meissner @ 1997-09-05 18:09 UTC (permalink / raw)
  To: joel, law; +Cc: egcs, jbuck

| On Fri, 5 Sep 1997, Jeffrey A Law wrote:
| 
| >   > This would also need to include a "compile but don't execute and report" 
| >   > for cross environments.  It would certainly save me a lot of setup effort. :)
| > This should work if you've got an assembler, linker and libraries available.
| 
| This will always be the case for rtems configurations since I use binutils
| and do a one-tree build every time.
| 
| > The testsuite will build the executable, then fail trying to run it.
| 
| I guess that is OK as long as it is clear which are compile time failures
| and which are blowing up on running it.

You could always configure it so that it uses a simulator of /bin/true to run
the program.

^ permalink raw reply	[flat|nested] 70+ messages in thread
* Re: testing consistency
@ 1997-09-10 22:37 Aaron Jackson
  1997-09-11  0:40 ` Jim Wilson
                   ` (3 more replies)
  0 siblings, 4 replies; 70+ messages in thread
From: Aaron Jackson @ 1997-09-10 22:37 UTC (permalink / raw)
  To: law; +Cc: egcs

I have tried to use the bootstrap target you posted several times, both
on the 970904 and the 970907 snapshot and was NEVER able to get past the
first stage build.  I finally gave up on egcs and just compiled and
installed gcc2.7.2.2 and g770.5.20.  I tried on both ultrix 4.4 and
digital unix 2.0 systems.  I would like to stay current with all these
many patches and improvements posted so far, and I'll give it another try
with the next snapshot, although I loath the four hour compile times and
I get really bummed when it doesn't work.

Aaron Jackson		jackson@msrce.howard.edu

>The first step in this process is to get everyone to bootstrap in the
>same manner.  Is anyone else using the rough toplevel "bootstrap"
>target I posted last week?  Comments and improvements are definitely
>welcome!

^ permalink raw reply	[flat|nested] 70+ messages in thread
* Re: testing consistency
@ 1997-09-11 11:36 Mike Stump
  1997-09-11 12:47 ` Jim Wilson
  0 siblings, 1 reply; 70+ messages in thread
From: Mike Stump @ 1997-09-11 11:36 UTC (permalink / raw)
  To: jackson, wilson; +Cc: egcs

> To: Aaron Jackson <jackson@negril.msrce.howard.edu>
> cc: egcs@cygnus.com
> Date: Thu, 11 Sep 1997 00:39:56 -0700
> From: Jim Wilson <wilson@cygnus.com>

> I see that you did report a problem once earlier, but that was due
> to the fact that you tried to build the Fortran front end in the
> first stage build.  This does not work unless you are compiling with
> gcc.

autoconf (configure) should be used to determine if the C compiler one
is using will in fact be capable of compiling the Fortran part, if
not, it should arrange to bootstrap all by itself.

^ permalink raw reply	[flat|nested] 70+ messages in thread
* Re: testing consistency
@ 1997-09-11 15:27 meissner
  1997-09-11 16:06 ` Doug Evans
  1997-09-11 17:08 ` Joe Buck
  0 siblings, 2 replies; 70+ messages in thread
From: meissner @ 1997-09-11 15:27 UTC (permalink / raw)
  To: law, wilson; +Cc: egcs, hjl

Speaking as a Linux user, I tend to prefer it not be installed into /usr
unless I explicitly add --prefix=/usr.  None of the other tools, such
as ld, as, etc. install themselves into /usr, and in fact when I started
using Linux, I tended to get burned by it installing itself into /usr
and the assembler and friends installing themself into /usr/local, until
I started using --prefix all of the time.

^ permalink raw reply	[flat|nested] 70+ messages in thread
[parent not found: <199709112227.SAA01970cygnus.egcs@tweedledumb.cygnus.com>]
* Re: testing consistency
@ 1997-09-11 21:33 Peter Seebach
  0 siblings, 0 replies; 70+ messages in thread
From: Peter Seebach @ 1997-09-11 21:33 UTC (permalink / raw)
  To: egcs

I have, for many years now, used almost nothing but systems where gcc has been
"the compiler".  On Amiga Unix, BSD/OS, NetBSD, Lynx/OS, the NeXT, and
Linux, gcc has been "the compiler" for quite some time.

I would be *very* upset if I downloaded another version, compiled it,
and found that it installed by default into /usr.

I mean, *very* upset.  It would make me very distrustful.

I had never, before this, even *considered* the possibility that something
I grab separately from my OS would try to install itself in the normal /usr;
/usr is for *unified distributions*.  I do not want anything going in /usr
unless that exact version, as built, is *exactly* the one the rest of /usr
is tested with.

I would, in any event, never want a compiler or other "base" tool to
overwrite "the system default", unless I went out of my way to tell it to.

When I build a new version of a compiler, I expect it to keep itself nicely
isolated from "the system" unless explicitly told to do otherwise; this
prevents all manner of unfortunate mishaps.

Saying "gcc is the system compiler" is deeply misleading.  Bugs creep in and
out.  Code may, however wrong this is, *depend* on a given set of bugs, or
a given set of implementation decisions.  gcc is not always the same as, or
even compatible with, gcc.  I think it's great to be *able* to replace the
system compiler with a more current version of gcc, but I think it is very
bad for that to be the default behavior.

My vote, as a regular user of several systems where gcc is "the compiler",
would be very strongly that gcc should continue to, by default, install into
/usr/local, and deciding that it should be installed into /usr should remain
an act of explicit volition, either by the user running configure, or the
person preparing a package for distribution.

Your milage may vary.  Some compilers not for use with some programs.  Your
sysadmin helps you put it together.

-s


^ permalink raw reply	[flat|nested] 70+ messages in thread
* Re: testing consistency
@ 1997-09-15 14:44 Alexandre Oliva
  0 siblings, 0 replies; 70+ messages in thread
From: Alexandre Oliva @ 1997-09-15 14:44 UTC (permalink / raw)
  To: law; +Cc: egcs

> The first step in this process is to get everyone to bootstrap in the
> same manner.  Is anyone else using the rough toplevel "bootstrap"
> target I posted last week?  Comments and improvements are definitely
> welcome!

I'm not using that bootstrap yet because it lacks some features I
really need:

1) the ability to specify additional flags to be used to build gcc.  I
currently have to append a strtoul-related flag to CFLAGS and
BOOT_CFLAGS so that I can build g77 on SunOS 4.1.3, and I can only do
that by using the bootstrap target in the gcc directory.

2) the ability to continue a build from a given stage on, as I can do
with bootstrap2 & bootstrap3.

Also, I don't think bootstrap should make compare.  Maybe I'm just
used to the ol' gcc bootstrap behavior, but I don't think building gcc
has anything to do with comparing it.  Maybe make check should do
it...

Here's a patch that provides bootstrap[23] targets, and removes the
BOOT_CFLAGS from the make invocation.  Doing this, BOOT_CFLAGS may be
overridden, and, if it is not, the default BOOT_CFLAGS from the gcc
Makefile will be used.

-- 
Alexandre Oliva
mailto:oliva@dcc.unicamp.br mailto:aoliva@acm.org
Universidade Estadual de Campinas, SP, Brasil

^ permalink raw reply	[flat|nested] 70+ messages in thread
[parent not found: <oren6qtfsg.fsf@sunsite.dcc.unicamp.br>]
* Re: testing consistency
@ 1997-09-18 21:42 Mike Stump
  1997-09-18 21:50 ` Jeffrey A Law
  1997-09-18 21:55 ` Joe Buck
  0 siblings, 2 replies; 70+ messages in thread
From: Mike Stump @ 1997-09-18 21:42 UTC (permalink / raw)
  To: law, oliva; +Cc: egcs

> As far as I'm concerned, _building_ gcc means 3-staging it and comparing
> it (native platforms).  We need not be restricted by the old way of doing
> things.

I think it is reasonable to consider comparing as part of the checking
process.  Right now, make doesn't imply make check.  I can envision a
change in make, so that check is in fact implied.  make && make
install then has neat interesting semantics.

I think I would rather see comparing as part of the check process, and
we should just encourage people to do make check.  Maybe make check is
appropriate to add to the end of a bootstrap?

^ permalink raw reply	[flat|nested] 70+ messages in thread
* Re: testing consistency
@ 1997-09-19  9:52 Kaveh R. Ghazi
  1997-09-19 10:20 ` Joe Buck
  0 siblings, 1 reply; 70+ messages in thread
From: Kaveh R. Ghazi @ 1997-09-19  9:52 UTC (permalink / raw)
  To: law, oliva; +Cc: egcs

 > Jeffrey A Law writes:
 > 
 > > I'd really prefer not to make it too easy to override BOOT_CFLAGS;
 > > remember we want both ourselves and the average joe using egcs to build
 > > in the same manner -- making BOOT_CFLAGS easy to override defeats that
 > > purpose to some extent.
 > 
 > OTOH, it makes it harder to test alternate configurations, and there
 > are certainly people who'd like to perform tests by compiling gcc with
 > look unrolling (Kaveh Ghazi does, as far as I remember), higher
 > optimization levels (as I do), etc.  The average joe would not set
 > BOOT_CFLAGS at all; I think anyone who intends to perform more
 > complete automated testing (like me) would be interested in being able
 > to bootstrap gcc with different flags.
 > 
 > I understand your concern about making sure gcc is tested in standard
 > conditions, but there's no reason for making it harder to test it more
 > completely, is there?
 > -- 
 > Alexandre Oliva
 
	I completely agree.

	I've always considered it a bad practice that up until
recently configuring any package with autoconf passed only -g -O to
gcc.  Ditto for BOOT_CFLAGS when building gcc.  Cheers to whoever had
enough faith in gcc to up the default CFLAGS produced by autoconf. :-)

	Its great that -O2 is used by default now.  But if we don't
test the >O2 opts more, we can't expect them to get safe enough to
use.  Okay, maybe they're avoided because they don't always produce
better code, but I think they're also marginalized because of the
perception that they aren't ready for prime time.  We can't change
this except by using them.

	I'd urge _more_ testing with higher BOOTCFLAGS.  Eg, inline
functions are very important in c++.  We should be testing -O3 or
-finline-functions more.

	To make testing easier and encourage more usage, I'd like to
see a -Oall with every possible optimization flag thrown in.  (I can
agree that choosing -funroll-loops vs. -funroll-all-loops is
debatable.)  Maybe a new -O4 contains -funroll-loops and -Oall
contains -funroll-all-loops.  I can provide a patch if this is desired.

		Comments?
		--Kaveh
--
Kaveh R. Ghazi				Project Manager
ghazi@caip.rutgers.edu			ICon CMT Corp.

^ permalink raw reply	[flat|nested] 70+ messages in thread

end of thread, other threads:[~1997-09-19 17:12 UTC | newest]

Thread overview: 70+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
1997-09-04 18:48 testing consistency Joe Buck
1997-09-04 20:13 ` Dave Avery
1997-09-04 21:11   ` Jim Wilson
1997-09-05  8:56     ` Joe Buck
1997-09-05  8:58   ` Joe Buck
1997-09-05  7:26 ` Joel Sherrill
1997-09-05 12:44   ` Jeffrey A Law
1997-09-05 16:40     ` Joel Sherrill
1997-09-10 15:21 ` Jeffrey A Law
1997-09-10 16:15   ` Joe Buck
1997-09-10 16:22     ` Jeffrey A Law
1997-09-10 22:23     ` Alexandre Oliva
1997-09-11  9:23       ` Jeffrey A Law
1997-09-11 10:01     ` Jeffrey A Law
1997-09-05 18:09 meissner
1997-09-05 18:55 ` Joel Sherrill
1997-09-10 22:37 Aaron Jackson
1997-09-11  0:40 ` Jim Wilson
1997-09-11  9:23 ` Jeffrey A Law
1997-09-11 10:17 ` H.J. Lu
1997-09-11 11:05   ` Jeffrey A Law
1997-09-11 11:05     ` H.J. Lu
1997-09-11 11:05       ` Jeffrey A Law
1997-09-11 11:36         ` H.J. Lu
1997-09-11 11:36           ` Jeffrey A Law
1997-09-11 12:54             ` H.J. Lu
1997-09-11 13:20               ` Jeffrey A Law
1997-09-11 14:01                 ` Per Bothner
1997-09-11 15:09                   ` Richard Henderson
1997-09-18 12:57                   ` Bob Glickstein
1997-09-11 13:37             ` Jim Wilson
1997-09-11 14:11               ` Per Bothner
1997-09-11 15:01               ` Jeffrey A Law
1997-09-11 16:13                 ` Joe Buck
1997-09-11 17:08                 ` Jim Wilson
1997-09-11 18:39                   ` Joe Buck
1997-09-11 13:20           ` Per Bothner
1997-09-12  7:27         ` Paul Koning
1997-09-11 12:57   ` Ian Lance Taylor
1997-09-11 13:51     ` H.J. Lu
1997-09-11 13:54       ` Ian Lance Taylor
1997-09-11 13:59         ` H.J. Lu
1997-09-11 14:04           ` Ian Lance Taylor
1997-09-11 19:01   ` Jim Wilson
1997-09-14 12:06 ` Joel Sherrill
1997-09-14 12:19   ` Jeffrey A Law
1997-09-14 12:44     ` Joel Sherrill
1997-09-18 22:56       ` Jeffrey A Law
1997-09-19  8:04         ` H.J. Lu
1997-09-19  8:15           ` Jeffrey A Law
1997-09-19 12:19           ` Jim Wilson
1997-09-19 13:14             ` Joe Buck
1997-09-19 14:51               ` Dave Love
1997-09-19 17:12               ` Jeffrey A Law
1997-09-11 11:36 Mike Stump
1997-09-11 12:47 ` Jim Wilson
1997-09-11 15:27 meissner
1997-09-11 16:06 ` Doug Evans
1997-09-11 17:08 ` Joe Buck
     [not found] <199709112227.SAA01970cygnus.egcs@tweedledumb.cygnus.com>
1997-09-11 16:31 ` not-for-mail
1997-09-11 21:33 Peter Seebach
1997-09-15 14:44 Alexandre Oliva
     [not found] <oren6qtfsg.fsf@sunsite.dcc.unicamp.br>
1997-09-18 20:37 ` Jeffrey A Law
1997-09-18 21:32   ` Alexandre Oliva
1997-09-18 22:24     ` Jeffrey A Law
1997-09-18 21:42 Mike Stump
1997-09-18 21:50 ` Jeffrey A Law
1997-09-18 21:55 ` Joe Buck
1997-09-19  9:52 Kaveh R. Ghazi
1997-09-19 10:20 ` Joe Buck

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).