From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jan Hubicka To: egcs@cygnus.com Subject: Benchmark suite... Date: Tue, 20 Oct 1998 05:46:00 -0000 Message-id: <19981020144547.21146@atrey.karlin.mff.cuni.cz> X-SW-Source: 1998-10/msg00762.html Hi After a week spent by tunning my patch I decided that I really need some testsuite. It is quite time consuming to test changes at real programs, since lots of thinks needs to be compiled. I would like to have some testsuite to verify my changes. So I've put together few programs and made a script to compile/test them. Because others seemsed to like benchmark suite idea too, I've decided to make something more definitive, wich should possibly become part of egcs, if it turns out to be usefull. I've made benchmarking library that uses interupt to stop test, so you might change time of test as you wish and benchmarking is quite accurate since speed of system calls is not significant. Also I've made script able to automatically run tests at various data types (long long, long, short and char), display results and compare them immediately with saved files. So you might see the changes before test is finished. It is also able to run specified tests, so when you see some problem in one test, you don't need to run whole suite etc... My goal is not to make comparsion benchmark for various compilers, so I am not doing any overall results. It is targeted only for developpers who want to see behaviour of their changes. Thats why I want to make it consisting mainly of simple tests, where you may browse the assembly result and see whats is going wrong and so on. So far I've implemented just very few tests mostly grabbed from my programs - Mset calcualtion loop, quicksort and recursive hanoi. Jeffrey pointed out that complex tests are also neccesary so I've added also mathing algorithm from XaoS and palette approximation from Allegro. I would love to have more tests. Both simple loops wich needs some tricks to optimize and complex code (like complression algorithm or whatever else) Current version is avalable at my ftp site ftp://ftp.ta.jcu.cz/pub/linux/hubicka/experimental/egcs-bench.tgz I have only gcc2.7.2.3 and egcc here, but results seems to be very interesting. I will send comparsions of current snapshot tommorow. So here is place for yours ideas :) I am open to all suggestions.... PS: here is comparsion of gcc2.7.2 and egcs1.0 at Pentium/120. For every test the number of loops per socond is printed and comparsion to gcc (100%=no change, 50% = twice as slow, 200% = twice as fast). First line is signed types, second unsigned. The types are sorted by size, so for integer test it is: long long, long, short, char and for fp it is long double,double,char. Benchmarks Tue Oct 20 14:36:33 MET DST 1998 egcc -O2 -pipe Integer tests Hanoi (tests/hanoi.c) 836.6337 101% 1270.2970 96% 854.0000 107% 1036.0000 106% 827.0000 100% 1265.0000 96% 905.0000 110% 1204.9020 116% Mandelbrot set calculation loop (tests/mset.c) 764000.0000 92% 2093137.2549 90% 1726732.6733 111% 1666000.0000 99% 767647.0588 92% 2100990.0990 91% 1826732.6733 116% 1902970.2970 113% Unrolled mandelbrot set calculation loop (tests/umset.c) 864000.0000 95% 2406930.6931 91% 2190099.0099 116% 2080198.0198 102% 856435.6436 96% 2428431.3725 91% 2244554.4554 119% 2310891.0891 113% Quicksort (tests/qsort.c) 37.8641 84% 96.0396 128% 67.3267 89% 109.9010 102% 38.2353 84% 95.0495 125% 67.3267 89% 110.8911 99% Bressemham line drawing algorithm (tests/bressemham.c) 6554613.8614 100% 8417720.0000 103% 7112316.8317 101% 8328500.0000 99% 6642680.0000 102% 8444198.0198 103% 7191702.9703 103% 8421333.3333 100% Palette approximation (tests/pal.c) -------.---- ---% 7.2072 112% -------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---% XaoS internal loop (tests/xaos.c) -------.---- ---% 449.0196 91% -------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---% Floating point tests Mandelbrot set calculation loop (tests/mset.c) 2853465.3465 107% 2888118.8119 109% 2903960.3960 109% Unrolled mandelbrot set calculation loop (tests/umset.c) 6158415.8416 136% 6052475.2475 134% 5890196.0784 131% Quicksort (tests/qsort.c) 38.6139 77% 32.6733 66% 28.1553 78% Complex type tests Mandelbrot set calculation loop using complex numbers (tests/cmset.c) 2947000.0000 88% 3051485.1485 92% 2975247.5248 89% Unrolled mandelbrot set calculation loop using complex numbers (tests/cumset.c) 5826732.6733 97% 5884158.4158 99% 5399009.9010 91% Honza