* Benchmark suite - first results...
@ 1998-10-21 13:01 Jan Hubicka
0 siblings, 0 replies; only message in thread
From: Jan Hubicka @ 1998-10-21 13:01 UTC (permalink / raw)
To: egcs
Hi
I've done some tests with my suite, so you might find them interesting.
Also please send me new tests you think they are usefull. At least I would
love to see effect my changes have. Today I will add some new tests and
make new release tommorow.
Thinks I want to add in future (except new tests) are:
1) run every test more times and print result after three (or specified
number) of tests have same result +- 0.5% (or similar number).
I hope it will improve reliability. My experience is that constetuctive
runs of suite prints almost in 99% 100%, sometimes 99% or 101% wich is
acceptable but sometimes (very ralery) junk results like 60% or so
especially when server is busy. This should fix that.
2) add option to run tests just for specified types
3) separate "complex" tests to separate part (like fp or integer) and
print just one result for them. It looks strange to have to see
those -----.-- for unsupported tests.
4) sort tests by complexity so if you receive some performance loss you
might use first test and get shortest example
5) some tool for comparing the sum files is needed (it is boring to run
testsuite again) maybe someone should be interested to write it :))
6) write some readme :)
Well, thats all for today. Here are the results:
(quite interesting as far as I can tell. egcs1.0 has quite often noticeable
slowdowns. Interesting is also 9% speed in fp test in haifa disabled current
snapshot) or half speed in quicksort compared to 1.0.
One of the purposes I've tracked down but I am unable to fix is that
following code:
int p[10];
t (int i)
{
return p[i];
}
compiled in 2.7.2 into:
t:
movl 4(%esp),%eax
movl p(,%eax,4),%eax
ret
Egcs 1.0 now compiles to:
movl 4(%esp),%edx
leal 0(,%edx,4),%eax
movl p(%eax),%eax
ret
And current snapshot to:
movl 4(%esp),%eax
sall $2,%eax
movl p(%eax),%eax
ret
Well, I would preffer 2.7.2's version :)), but I am unable to track down,
who is responsibe for conversion of addressing mode and why it don't do
that now. This disappears for -m386, so it seems to do something with
prices of operations and CSE?.
Results are printed in format <number of loops> <difference>
where difference is number of % of speed of previous run. 100% means no
change, 50% half speed, 200% twice as fast.
they are in two rows. First is for signed, second for unsigned.
columns are types (long long, long, short, char) for integer and
(long double,double,float) for FP - sorted by size.
Hope this benchmarks will help to unhide weak places for egcs at simple
tests and make development much easier and funny :)
At least it helped me to tune my patch. you might see results at the end
of file. They are sometimes quite bad. Current version has fixed most
slowdowns. Interesing that this change caused great impact of performance
in crafty.
Changes from gcc 2.7.2.3 to egcs1.0 withhout scheduling and optimizing for
486
Benchmarks Tue Oct 20 08:44:37 CEST 1998
egcc -m486 -O2 -fno-schedule-insns2
Integer tests
Hanoi (tests/hanoi.c)
993.4132 92% 1528.6000 87% 1045.4000 95% 1256.6000 97%
995.4092 92% 1529.4000 87% 1146.6000 103% 1482.4351 111%
Mandelbrot set calculation loop (tests/mset.c)
976047.9042 90% 2791000.0000 93% 2181836.3273 107% 2143200.0000 98%
976047.9042 90% 2790818.3633 93% 2449301.3972 120% 2448800.0000 112%
Unrolled mandelbrot set calculation loop (tests/umset.c)
1086400.0000 93% 3332135.7285 97% 2750499.0020 113% 2643712.5749 100%
1086427.1457 93% 3333333.3333 97% 2916966.0679 120% 2926400.0000 111%
Quicksort (tests/qsort.c)
58.6826 100% 140.8000 146% 140.0398 146% 141.4000 100%
58.8000 101% 140.8000 146% 140.1198 146% 141.4000 100%
Bressemham line drawing algorithm (tests/bressemham.c)
6013180.0000 98% 10588436.0000 97% 9452240.0000 102% 11060095.8084 100%
6932264.0000 109% 10597928.1437 97% 9452906.1876 102% 11065256.0000 100%
Palette approximation (tests/pal.c)
-------.---- ---% 8.6444 106% -------.---- ---% -------.---- ---%
-------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---%
XaoS internal loop (tests/xaos.c)
-------.---- ---% 525.7485 87% -------.---- ---% -------.---- ---%
-------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---%
\f
Floating point tests
Mandelbrot set calculation loop (tests/mset.c)
3427400.0000 100% 3425600.0000 100% 3423000.0000 100%
Unrolled mandelbrot set calculation loop (tests/umset.c)
6106387.2255 104% 6028600.0000 104% 5837400.0000 104%
Quicksort (tests/qsort.c)
49.7006 78% 49.5010 94% 37.1769 80%
\f
Complex type tests
Mandelbrot set calculation loop using complex numbers (tests/cmset.c)
4281437.1257 100% 4280638.7226 100% 4276800.0000 100%
Unrolled mandelbrot set calculation loop using complex numbers (tests/cumset.c)
7725200.0000 100% 7600600.0000 100% 7302994.0120 100%
\f
-------------------------------------------------------------------------------
Comparsions with scheduling and pentium optimizations
Benchmarks Tue Oct 20 08:52:22 CEST 1998
egcc -mpentium -O2
Integer tests
Hanoi (tests/hanoi.c)
1065.8000 98% 1684.6000 96% 1079.2000 98% 1322.0000 102%
1069.9203 99% 1685.6000 96% 1170.8000 105% 1531.8000 115%
Mandelbrot set calculation loop (tests/mset.c)
983200.0000 91% 2791200.0000 93% 2105189.6208 103% 2143313.3733 98%
984031.9361 91% 2790818.3633 93% 2400400.0000 118% 2553000.0000 117%
Unrolled mandelbrot set calculation loop (tests/umset.c)
1112774.4511 96% 3188223.5529 92% 2882435.1297 119% 2658400.0000 101%
1112800.0000 96% 3189820.3593 92% 2916733.0677 120% 2935528.9421 111%
Quicksort (tests/qsort.c)
58.8000 100% 140.1198 145% 140.2000 146% 141.7166 100%
58.4830 100% 141.0000 146% 140.3194 146% 141.9162 100%
Bressemham line drawing algorithm (tests/bressemham.c)
6014514.9701 98% 10932822.3553 100% 9443976.0479 102% 10773808.3832 98%
6014311.3772 94% 11013392.0000 101% 9449229.5409 102% 10763748.0000 97%
Palette approximation (tests/pal.c)
-------.---- ---% 9.1270 112% -------.---- ---% -------.---- ---%
-------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---%
XaoS internal loop (tests/xaos.c)
-------.---- ---% 556.4000 93% -------.---- ---% -------.---- ---%
-------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---%
\f
Floating point tests
Mandelbrot set calculation loop (tests/mset.c)
4135528.9421 121% 4133333.3333 121% 3743200.0000 109%
Unrolled mandelbrot set calculation loop (tests/umset.c)
8190600.0000 140% 8049400.0000 139% 7590618.7625 136%
Quicksort (tests/qsort.c)
49.7006 78% 49.7006 95% 37.2000 80%
\f
Complex type tests
Mandelbrot set calculation loop using complex numbers (tests/cmset.c)
3868200.0000 90% 3867330.6773 90% 4130600.0000 97%
Unrolled mandelbrot set calculation loop using complex numbers (tests/cumset.c)
6224151.6966 81% 6142914.1717 81% 6512200.0000 89%
\f
-------------------------------------------------------------------------------
Changes from egcs1.0 to current snapshot:
Benchmarks Tue Oct 20 09:02:20 CEST 1998
gcc -mpentium -O2
Integer tests
Hanoi (tests/hanoi.c)
1051.4000 99% 1634.8606 97% 1092.2000 101% 1286.8263 97%
1051.8962 98% 1635.8000 97% 1226.1477 105% 1506.7864 98%
Mandelbrot set calculation loop (tests/mset.c)
1008782.4351 103% 2926546.9062 105% 2307968.1275 110% 2105988.0240 98%
1017200.0000 103% 2926800.0000 105% 2552800.0000 106% 2353400.0000 92%
Unrolled mandelbrot set calculation loop (tests/umset.c)
1126946.1078 101% 3490000.0000 109% 2857400.0000 99% 2532934.1317 95%
1128600.0000 101% 3489620.7585 109% 2962400.0000 102% 2774000.0000 94%
Quicksort (tests/qsort.c)
46.2151 79% 105.4000 75% 62.8743 45% 122.3553 86%
46.3221 79% 105.4000 75% 62.6747 45% 123.0000 87%
Bressemham line drawing algorithm (tests/bressemham.c)
6105304.0000 102% 10006950.0998 92% 9293620.7585 98% 9972410.3586 93%
6400744.0000 106% 9990880.0000 91% 9295153.6926 98% 9956948.0000 93%
Palette approximation (tests/pal.c)
-------.---- ---% 8.3984 92% -------.---- ---% -------.---- ---%
-------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---%
XaoS internal loop (tests/xaos.c)
-------.---- ---% 599.6000 108% -------.---- ---% -------.---- ---%
-------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---%
\f
Floating point tests
Mandelbrot set calculation loop (tests/mset.c)
4441916.1677 107% 4441400.0000 107% 3865000.0000 103%
Unrolled mandelbrot set calculation loop (tests/umset.c)
8408383.2335 103% 8402200.0000 104% 8036327.3453 106%
Quicksort (tests/qsort.c)
55.8000 112% 55.6886 112% 37.0518 100%
\f
Complex type tests
Mandelbrot set calculation loop using complex numbers (tests/cmset.c)
3528600.0000 91% 3526546.9062 91% 257085.8283 6%
Unrolled mandelbrot set calculation loop using complex numbers (tests/cumset.c)
5881836.3273 95% 5882634.7305 96% 268862.2754 4%
\f
-------------------------------------------------------------------------------
Changes after enabling haifa
Benchmarks Tue Oct 20 09:11:36 CEST 1998
gcc -mpentium -O2
Integer tests
Hanoi (tests/hanoi.c)
1089.6208 104% 1633.2669 100% 1136.6000 104% 1346.4000 105%
1091.0180 104% 1631.1377 100% 1266.2675 103% 1548.2000 103%
Mandelbrot set calculation loop (tests/mset.c)
1000598.8024 99% 3076400.0000 105% 2264400.0000 98% 2223153.6926 106%
1008800.0000 99% 3076800.0000 105% 2499400.0000 98% 2608982.0359 111%
Unrolled mandelbrot set calculation loop (tests/umset.c)
1140400.0000 101% 3515400.0000 101% 2857285.4291 100% 2622954.0918 104%
1141800.0000 101% 3514970.0599 101% 2980600.0000 101% 2882200.0000 104%
Quicksort (tests/qsort.c)
48.8048 106% 105.3892 100% 62.8743 100% 123.0000 101%
48.8000 105% 105.4000 100% 62.8743 100% 122.9541 100%
Bressemham line drawing algorithm (tests/bressemham.c)
6104806.3872 100% 10075872.0000 101% 9289928.1437 100% 9947936.0000 100%
6153352.0000 96% 10050003.9920 101% 9275032.0000 100% 10016852.0000 101%
Palette approximation (tests/pal.c)
-------.---- ---% 8.2677 98% -------.---- ---% -------.---- ---%
-------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---%
XaoS internal loop (tests/xaos.c)
-------.---- ---% 608.6000 102% -------.---- ---% -------.---- ---%
-------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---%
\f
Floating point tests
Mandelbrot set calculation loop (tests/mset.c)
3869800.0000 87% 3869000.0000 87% 3865069.8603 100%
Unrolled mandelbrot set calculation loop (tests/umset.c)
8192415.1697 97% 8187848.6056 97% 8103393.2136 101%
Quicksort (tests/qsort.c)
55.8000 100% 55.5777 100% 37.0518 100%
\f
Complex type tests
Mandelbrot set calculation loop using complex numbers (tests/cmset.c)
3241317.3653 92% 3242000.0000 92% 3072800.0000 1195%
Unrolled mandelbrot set calculation loop using complex numbers (tests/cumset.c)
5742000.0000 98% 5740918.1637 98% 5631400.0000 2095%
\f
-------------------------------------------------------------------------------
Changes from current snapshot to my patch
Benchmarks Tue Oct 20 09:18:13 CEST 1998
gcc -mpentium -O2
Integer tests
Hanoi (tests/hanoi.c)
1124.6000 103% 1715.1697 105% 1120.6000 99% 1323.4000 98%
1125.1497 103% 1714.7705 105% 1242.5150 98% 1589.6208 103%
Mandelbrot set calculation loop (tests/mset.c)
1034800.0000 103% 3157884.2315 103% 2264200.0000 100% 2308200.0000 104%
1043800.0000 103% 3157800.0000 103% 2666000.0000 107% 2727345.3094 105%
Unrolled mandelbrot set calculation loop (tests/umset.c)
1171200.0000 103% 3501996.0080 100% 3037600.0000 106% 2703792.4152 103%
1172600.0000 103% 3502195.6088 100% 3264271.4571 110% 2953600.0000 102%
Quicksort (tests/qsort.c)
48.7026 100% 105.2000 100% 78.8423 125% 123.2000 100%
48.8000 100% 104.7904 99% 78.8423 125% 123.1537 100%
Bressemham line drawing algorithm (tests/bressemham.c)
6307016.0000 103% 10674251.4970 106% 9382982.0359 101% 9830036.0000 99%
6371169.6607 104% 10670475.0499 106% 9374251.4970 101% 9818880.0000 98%
Palette approximation (tests/pal.c)
-------.---- ---% 9.1188 110% -------.---- ---% -------.---- ---%
-------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---%
XaoS internal loop (tests/xaos.c)
-------.---- ---% 610.9780 100% -------.---- ---% -------.---- ---%
-------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---%
\f
Floating point tests
Mandelbrot set calculation loop (tests/mset.c)
3870059.8802 100% 3869061.8762 100% 3865139.4422 100%
Unrolled mandelbrot set calculation loop (tests/umset.c)
8485400.0000 104% 8477245.5090 104% 8432000.0000 104%
Quicksort (tests/qsort.c)
55.6886 100% 55.6000 100% 37.1257 100%
\f
Complex type tests
Mandelbrot set calculation loop using complex numbers (tests/cmset.c)
3158400.0000 97% 3157600.0000 97% 3154000.0000 103%
Unrolled mandelbrot set calculation loop using complex numbers (tests/cumset.c)
6849600.0000 119% 6845309.3812 119% 6838323.3533 121%
\f
-------------------------------------------------------------------------------
Changes after enabling -fschedule-insns
Benchmarks Tue Oct 20 09:25:48 CEST 1998
gcc -mpentium -O2 -fschedule-insns
Integer tests
Hanoi (tests/hanoi.c)
1078.0439 96% 1814.3713 106% 1151.8000 103% 1307.1856 99%
1079.0419 96% 1814.1717 106% 1316.6000 106% 1517.8000 95%
Mandelbrot set calculation loop (tests/mset.c)
1008800.0000 97% 3076294.8207 97% 2353200.0000 104% 2264200.0000 98%
1017365.2695 97% 3076247.5050 97% 2608383.2335 98% 2553800.0000 94%
Unrolled mandelbrot set calculation loop (tests/umset.c)
1054000.0000 90% 3477400.0000 99% 3047000.0000 100% 2891417.1657 107%
1054291.4172 90% 3477045.9082 99% 3253200.0000 100% 3008582.8343 102%
Quicksort (tests/qsort.c)
48.7026 100% 105.3892 100% 72.6547 92% 122.6000 100%
48.6056 100% 105.3892 101% 72.6547 92% 122.5549 100%
Bressemham line drawing algorithm (tests/bressemham.c)
6713724.5509 106% 10954191.6168 103% 9439560.0000 101% 11213676.6467 114%
6551289.4212 103% 10617388.0000 100% 9393440.0000 100% 11197016.0000 114%
Palette approximation (tests/pal.c)
-------.---- ---% 7.4803 92% -------.---- ---% -------.---- ---%
-------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---%
XaoS internal loop (tests/xaos.c)
-------.---- ---% 602.3952 99% -------.---- ---% -------.---- ---%
-------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---%
\f
Floating point tests
Mandelbrot set calculation loop (tests/mset.c)
4612400.0000 119% 4611800.0000 119% 4605988.0240 119%
Unrolled mandelbrot set calculation loop (tests/umset.c)
8485229.5409 100% 8477844.3114 100% 8460279.4411 100%
Quicksort (tests/qsort.c)
55.5777 100% 55.6886 100% 46.6135 126%
\f
Complex type tests
Mandelbrot set calculation loop using complex numbers (tests/cmset.c)
3157000.0000 100% 3157485.0299 100% 2498602.7944 79%
Unrolled mandelbrot set calculation loop using complex numbers (tests/cumset.c)
8054800.0000 118% 8053000.0000 118% 7475600.0000 109%
\f
-------------------------------------------------------------------------------
And for comparsions changes after enabling -fschedule-insns in unpatched
last snapshot
Benchmarks Tue Oct 20 09:32:39 CEST 1998
gcc -mpentium -O2 -fschedule-insns
Integer tests
Hanoi (tests/hanoi.c)
1064.4711 98% 1811.1776 111% 1174.8503 103% 1323.4000 98%
1066.6667 98% 1812.7745 111% 1309.5808 103% 1508.3832 97%
Mandelbrot set calculation loop (tests/mset.c)
968263.4731 97% 2926200.0000 95% 2264400.0000 100% 2143113.7725 96%
968400.0000 96% 2926200.0000 95% 2500000.0000 100% 2608782.4351 100%
Unrolled mandelbrot set calculation loop (tests/umset.c)
1057600.0000 93% 3391235.0598 96% 2826000.0000 99% 2815200.0000 107%
1057600.0000 93% 3391000.0000 96% 2953400.0000 99% 3027800.0000 105%
Quicksort (tests/qsort.c)
48.7026 100% 105.3892 100% 62.8743 100% 123.2000 100%
48.7078 100% 104.7904 99% 62.8743 100% 123.2000 100%
Bressemham line drawing algorithm (tests/bressemham.c)
6426934.1317 105% 10145073.8523 101% 9353049.9002 101% 10015556.8862 101%
6369604.7904 104% 10160884.0000 101% 9372176.0000 101% 10038215.5689 100%
Palette approximation (tests/pal.c)
-------.---- ---% 8.4980 103% -------.---- ---% -------.---- ---%
-------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---%
XaoS internal loop (tests/xaos.c)
-------.---- ---% 622.1557 102% -------.---- ---% -------.---- ---%
-------.---- ---% -------.---- ---% -------.---- ---% -------.---- ---%
\f
Floating point tests
Mandelbrot set calculation loop (tests/mset.c)
3999400.0000 103% 3997011.9522 103% 3865800.0000 100%
Unrolled mandelbrot set calculation loop (tests/umset.c)
8191200.0000 100% 8189600.0000 100% 8102594.8104 100%
Quicksort (tests/qsort.c)
55.7769 100% 55.7769 100% 46.5070 126%
\f
Complex type tests
Mandelbrot set calculation loop using complex numbers (tests/cmset.c)
3242400.0000 100% 3241434.2629 100% 2263600.0000 74%
Unrolled mandelbrot set calculation loop using complex numbers (tests/cumset.c)
7314570.8583 127% 7313772.4551 127% 7248200.0000 129%
\f
--
OK. Lets make a signature file.
+-------------------------------------------------------------------------+
| Jan Hubicka (Jan Hubi\v{c}ka in TeX) hubicka@freesoft.cz |
| Czech free software foundation: http://www.freesoft.cz |
|AA project - the new way for computer graphics - http://www.ta.jcu.cz/aa |
| homepage: http://www.paru.cas.cz/~hubicka/ , games koules, Xonix, fast |
| fractal zoomer XaoS, index of Czech GNU/Linux/UN*X documentation etc. |
+-------------------------------------------------------------------------+
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~1998-10-21 13:01 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
1998-10-21 13:01 Benchmark suite - first results Jan Hubicka
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).