From mboxrd@z Thu Jan 1 00:00:00 1970 From: Arvind Sankar To: Marc Lehmann Cc: egcs@cygnus.com Subject: Re: announce: egcs benchmark suite, alpha release Date: Wed, 20 Jan 1999 18:54:00 -0000 Message-id: <19990120215431.A18367@anjala.mit.edu> References: <19990116065208.A17513@cerebro.laendle> X-SW-Source: 1999-01/msg00002.html hi! I tried checking out the benchmarks a while back. When I do make check, the first test (c++/oopack) works properly, but then everything else bums with LINK ERROR. If I try to run make check again, make bums with this queer error: gcc bench.o bench.sum -o bench /usr/bin/ld:bench.sum: file format not recognized; treating as linker script /usr/bin/ld:bench.sum:2: parse error collect2: ld returned 1 exit status make: *** [bench] Error 1 Apparently, make finds an implicit rule bench: bench.o, and that causes it do stuff like this. Shouldn't a lot of the targets be made .PHONY? Right now, bench.sum has no dependents, so it is always up-to-date. On Sat, Jan 16, 1999 at 06:52:08AM +0100, Marc Lehmann wrote: > I'm pleased to announce the first public release of the egcs benchmark suite. > > WHAT IS IT? > > The egcs benchmark suite is a collection of individual tests used to > determine relative performance changes of specific optimizations. There are > tests that check performance of different datatypes on the same problem, > tests that measure the C++ abstraction penalty, tests measuring function > inlining/call overhead and more... > > Regular benchmarking will also allow to detect optimization problems. > > WHAT IS IT _NOT_? > > It's not a general purpose benchmark library. It's unsuitable for comparing > performance of one machine/os to another, and it's also unsuitable to compare > performance between different compilers. Its sole purpose is to measure speed > enhancements (or deficiencies) between egcs compiler revisions. > > In short, what the testsuite is for compiler correctness, is the benchmark > suite for optimizer quality. > > WHAT WE NEED > > at the moment are people that alpha test the release, give compatibility > hints, detect problems with the setup. > > if the suite turns out to be useful, we need regular testers so we can fill > the benchmark database and write cute perl programs that draw even cuter > graphs ;) > > Also we need input on how to handle compiler switches, i.e. should > we run a single test with -O2, or multiple tests with -O2, -O3, -O3 > -funroll-loops etc. (and how do we keep the results comparable in the > latter case) > > WHERE TO GET IT? > > At the moment its "only" available via cvs. Instead of checking out the > "egcs" directory you can check out the "benchmarks" directory. (Just follow > the instructions for checking out "egcs" but replace it by "benchmarks") > > HOW TO USE IT? > > Yes, the README is not quite up-to-date, but everything you should have to do is to type: > > make > > which will compile & run all benchmarks and automatically mail me the > results, testing everything in the package. > > THANKS > > OH YES, thanks a lot to Jan Hubicka, who not only had the original idea to > create the benchmark suite, but also delivered all the initial tests and > programmed the initial testing framework. > > > -----==- | > ----==-- _ | > ---==---(_)__ __ ____ __ Marc Lehmann +-- > --==---/ / _ \/ // /\ \/ / pcg@goof.com |e| > -=====/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ > The choice of a GNU generation | > | From mboxrd@z Thu Jan 1 00:00:00 1970 From: Arvind Sankar To: Marc Lehmann Cc: egcs@cygnus.com Subject: Re: announce: egcs benchmark suite, alpha release Date: Sun, 31 Jan 1999 23:58:00 -0000 Message-ID: <19990120215431.A18367@anjala.mit.edu> References: <19990116065208.A17513@cerebro.laendle> X-SW-Source: 1999-01n/msg00766.html Message-ID: <19990131235800.50mk1p7u9xqHQ-HkeSnqUuhxDFfbvq15Ax78T6tg8JI@z> hi! I tried checking out the benchmarks a while back. When I do make check, the first test (c++/oopack) works properly, but then everything else bums with LINK ERROR. If I try to run make check again, make bums with this queer error: gcc bench.o bench.sum -o bench /usr/bin/ld:bench.sum: file format not recognized; treating as linker script /usr/bin/ld:bench.sum:2: parse error collect2: ld returned 1 exit status make: *** [bench] Error 1 Apparently, make finds an implicit rule bench: bench.o, and that causes it do stuff like this. Shouldn't a lot of the targets be made .PHONY? Right now, bench.sum has no dependents, so it is always up-to-date. On Sat, Jan 16, 1999 at 06:52:08AM +0100, Marc Lehmann wrote: > I'm pleased to announce the first public release of the egcs benchmark suite. > > WHAT IS IT? > > The egcs benchmark suite is a collection of individual tests used to > determine relative performance changes of specific optimizations. There are > tests that check performance of different datatypes on the same problem, > tests that measure the C++ abstraction penalty, tests measuring function > inlining/call overhead and more... > > Regular benchmarking will also allow to detect optimization problems. > > WHAT IS IT _NOT_? > > It's not a general purpose benchmark library. It's unsuitable for comparing > performance of one machine/os to another, and it's also unsuitable to compare > performance between different compilers. Its sole purpose is to measure speed > enhancements (or deficiencies) between egcs compiler revisions. > > In short, what the testsuite is for compiler correctness, is the benchmark > suite for optimizer quality. > > WHAT WE NEED > > at the moment are people that alpha test the release, give compatibility > hints, detect problems with the setup. > > if the suite turns out to be useful, we need regular testers so we can fill > the benchmark database and write cute perl programs that draw even cuter > graphs ;) > > Also we need input on how to handle compiler switches, i.e. should > we run a single test with -O2, or multiple tests with -O2, -O3, -O3 > -funroll-loops etc. (and how do we keep the results comparable in the > latter case) > > WHERE TO GET IT? > > At the moment its "only" available via cvs. Instead of checking out the > "egcs" directory you can check out the "benchmarks" directory. (Just follow > the instructions for checking out "egcs" but replace it by "benchmarks") > > HOW TO USE IT? > > Yes, the README is not quite up-to-date, but everything you should have to do is to type: > > make > > which will compile & run all benchmarks and automatically mail me the > results, testing everything in the package. > > THANKS > > OH YES, thanks a lot to Jan Hubicka, who not only had the original idea to > create the benchmark suite, but also delivered all the initial tests and > programmed the initial testing framework. > > > -----==- | > ----==-- _ | > ---==---(_)__ __ ____ __ Marc Lehmann +-- > --==---/ / _ \/ // /\ \/ / pcg@goof.com |e| > -=====/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ > The choice of a GNU generation | > |