Re: [GCC 3.x] Performance testing for QA

[dewar@gnat.com (Robert Dewar)]

<<   I only reason I think its nice to have the compiler bootstrap as part
of the benchmark is to make sure that the compiler is fast.  Instead, I
think it would be more interesting to see how fast the compiler could
build all the other benchmarks -- not how fast it could build itself :)
>>

I disagree, gcc is a very interesting benchmark because of its locality
patterns. It precisely is NOT a collection of tight loops, and unlike
some of the other tests in SPEC, it is very hard to tune a compiler 
to do artifically well on the gcc benchmark.

Of course you want a big collection of benchmarks, and actually we advise
users of GNAT to always test on their own code rather than benchmarks,
but GCC is probably one of the better benchmarks around. 
The trouble with the encoder/decoder examples is that they tend
to concentrate on specific inner loops. That means that they are
much more open to subversion (although interestingly, GCC is probably
the one compiler where there is NOT a lot of effort spent specifically
on getting benchmarks to work better).