Linux Number Crunching: Benchmarking Compilers and Languages on ia32

Linux Number Crunching: Benchmarking Compilers and Languages on ia32

[Scott Robert Ladd]

Hello,

I've posted a new benchmark article; it is the first in a series that looks
at numerical applications and language/compiler performance.

http://www.coyotegulch.com/reviews/almabench.html

The article has been mentioned on the front page of Slashdot, but my web
site seems to be suriving. ;)

I'm developing an entire series of benchmarks for investigating different
aspects of high-performance computing. The current article will be extended
in the coming weeks with more benchmark tests and new compilers.

To get the whole story, please read the entire article.

Thanks.

..Scott

--
Scott Robert Ladd
Coyote Gulch Productions,  http://www.coyotegulch.com
No ads -- just very free (and somewhat unusual) code.

RE: Linux Number Crunching: Benchmarking Compilers and Languages on ia32

[Scott Robert Ladd]

Steven Anderson wrote
> After reviewing the code, I found that all the times
> are done using the clock system call. The problem is
> clock returns an approximation of the processor time.
> The test should use the times system call to determine
> how long each function ran.

I'm not certain which article you are referencing; the article in the
subject line uses the GNU "time" command to calculate run-times for a single
benchmark, not clock. I've had numerous debates (especially with Java folk)
over the validity of such a timing method; rather than argue, I'll be
including metrics from several timing sources in my next round of tests.

In an earlier article, I compared Intel and gcc using a number of different
benchmarks; gcc performed admirably, winning several benchmarks.

The "best" compiler is very application specific. For example, one of my own
applications implements an evolutionary algorithm for state machines. Using
finite state machines, gcc produces code that is marginally faster; when
using fuzzy state machines, Intel's code runs faster than does gcc's. Such
differences are one reason I prefer to have multiple compilers available.

I've had a lot of suggestions and "demands" from people; I've learned some
things, and also discovered several popular misconceptions about code
generation. My most recent benchmark tested 2300 combinations of switches
for generating code with gcc, giving me a better idea of how options
interact on different benchmarks. When time permits, I'll publish this
information, along with analysis of generated code, which I think will be of
value both to users and implementors of gcc.

--
Scott Robert Ladd
Coyote Gulch Productions (http://www.coyotegulch.com)
Professional programming for science and engineering;
Interesting and unusual bits of very free code.

RE: Linux Number Crunching: Benchmarking Compilers and Languages on ia32

[Steven Anderson]

> To get the whole story, please read the entire
article.
I read the entire article and decided to see the
results on my Athlon-XP box. So I downloaded the tests
and ran them. GCC did not fair well which surprised
me. Also, I have a program that does Gaussian
elimination where g++ beats the Intel compiler with
the proper optimization setting. So I was surprise how
poorly GCC did in the tests. 

After reviewing the code, I found that all the times
are done using the clock system call. The problem is
clock returns an approximation of the processor time.
The test should use the times system call to determine
how long each function ran.

Thanks

__________________________________________________
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now.
http://mailplus.yahoo.com

RE: Linux Number Crunching: Benchmarking Compilers and Languages on ia32

[Scott Robert Ladd]

> It's the last bit, "superior intrinsic implementations of trigonometric
> functions" that strikes me as unlikely. Can you provide more information
> on this, with specific C++ code and generated asm, so that the GNU C++
> developers can see what is going on here, or how you reached these
> conclusions? I know that Gaby and Roger Sayle did some work/patching in
> this area, so if there are still issues I'd like to see them fixed
> soonish.

I based the statement above on experience from a couple of years ago, where
gcc did not emit solid code for intrinsic trigonometric functions.

I'll generate assembly-language output for both compilers, and see what
they're actually producing. That'll have to come later today or tongight, as
my plate overfloweth at the moment...

..Scott

Re: Linux Number Crunching: Benchmarking Compilers and Languages on ia32

[Benjamin Kosnik]

In this article, you state that 

Finally, I'll point out that Intel's C++ compiler produces code that is
more than twice as fast as that emitted by GNU's g++. The Intel compiler
performs automatic vectorization of loops, and it appears to have
superior intrinsic implementations of trigonometric functions.

It's the last bit, "superior intrinsic implementations of trigonometric
functions" that strikes me as unlikely. Can you provide more information
on this, with specific C++ code and generated asm, so that the GNU C++
developers can see what is going on here, or how you reached these
conclusions? I know that Gaby and Roger Sayle did some work/patching in
this area, so if there are still issues I'd like to see them fixed
soonish.

thanks,
benjamin