Comparing Linux C and C++ Compilers: Benchmarks and Analysis

Comparing Linux C and C++ Compilers: Benchmarks and Analysis

[Scott Robert Ladd]

Hello,

I've posted a comparison of recent GCC versions (3.3, 3.4, and the
coming 4.0) with Intel C++ 8.1, including several benchmarks and
"state-of-the-product" reviews. You can find the article at:

     http://www.coyotegulch.com/reviews/linux_compilers/

The above article replaces an older piece I published in late 2002. This
new comparison marks what I hope will be an ongoing series that tracks 
the quality of Linux compilers.

..Scott


-- 
Scott Robert Ladd
site: http://www.coyotegulch.com
blog: http://chaoticcoyote.blogspot.com

Re: Comparing Linux C and C++ Compilers: Benchmarks and Analysis

[Richard Guenther]

On Sat, 18 Sep 2004 13:10:38 -0400, Scott Robert Ladd
<coyote@coyotegulch.com> wrote:
> Hello,
> 
> I've posted a comparison of recent GCC versions (3.3, 3.4, and the
> coming 4.0) with Intel C++ 8.1, including several benchmarks and
> "state-of-the-product" reviews. You can find the article at:
> 
>      http://www.coyotegulch.com/reviews/linux_compilers/
> 
> The above article replaces an older piece I published in late 2002. This
> new comparison marks what I hope will be an ongoing series that tracks
> the quality of Linux compilers.

You may want to check gcc 4.0s auto-vectorizing capabilities by using
-ftree-vectorize.  I believe Intel compiler does this by default with your
optimization settings, and also unrolls loops where profitable (-funroll-loops
for all gcc versions).  While I personally do not have success with
auto-vectorizing (neither with Intel nor with gcc), loop unrolling
shows very positive effects on my numerical codes.

Richard.

Re: Comparing Linux C and C++ Compilers: Benchmarks and Analysis

[Scott Robert Ladd]

Richard Guenther wrote:
> You may want to check gcc 4.0s auto-vectorizing capabilities by using
> -ftree-vectorize.  I believe Intel compiler does this by default with your
> optimization settings, and also unrolls loops where profitable (-funroll-loops
> for all gcc versions).  While I personally do not have success with
> auto-vectorizing (neither with Intel nor with gcc), loop unrolling
> shows very positive effects on my numerical codes.

I've had mixed success with unrolling loops in GCC, though I haven't 
tested it thoroughly with 4.0.

I'll likely publish an addendum to the article sometime next week, after 
people's feedback comes in.

Thanks.


-- 
Scott Robert Ladd
site: http://www.coyotegulch.com
blog: http://chaoticcoyote.blogspot.com

Re: Comparing Linux C and C++ Compilers: Benchmarks and Analysis

[Andi Kleen]

Scott Robert Ladd <coyote@coyotegulch.com> writes:

> Hello,
>
> I've posted a comparison of recent GCC versions (3.3, 3.4, and the
> coming 4.0) with Intel C++ 8.1, including several benchmarks and
> "state-of-the-product" reviews. You can find the article at:
>
>      http://www.coyotegulch.com/reviews/linux_compilers/
>
> The above article replaces an older piece I published in late 2002. This
> new comparison marks what I hope will be an ongoing series that tracks
> the quality of Linux compilers.

Would it be possible to test with profile feedback too? 

More and more interesting optimizations seem to be only enabled
with that.

-Andi

Re: Comparing Linux C and C++ Compilers: Benchmarks and Analysis

[Scott Robert Ladd]

Andi Kleen wrote:
> Would it be possible to test with profile feedback too? 
> 
> More and more interesting optimizations seem to be only enabled
> with that.

Intel has profile feedback, too; I suspect I need to do a separate 
article, explaining what "profile feedback" is, and look at how it 
affects code generation.

-- 
Scott Robert Ladd
site: http://www.coyotegulch.com
blog: http://chaoticcoyote.blogspot.com

Re: Comparing Linux C and C++ Compilers: Benchmarks and Analysis

[Natros]

"This change caused quite a bit of annoyance when trying to run 8.1 on
my Gentoo-based machines, because Gentoo's GCC is somehow incompatible
with ICC's expectations. I'm investigating this further; for now, just
be aware that Intel's 8.1 compiler requires a "standard" GCC install."

You can use Dinkumware libraries with icc -cxxlib-icc, so you don't
need a "standard" gcc install. On my gentoo I exported the
CPLUS_INCLUDE_PATH

$ export CPLUS_INCLUDE_PATH=/usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/include/g++-v3:/usr/lib/gcc-lib/i6
86-pc-linux-gnu/3.3.4/include/g++-v3/i686-pc-linux-gnu


On Sat, 18 Sep 2004 13:10:38 -0400, Scott Robert Ladd
<coyote@coyotegulch.com> wrote:
> Hello,
> 
> I've posted a comparison of recent GCC versions (3.3, 3.4, and the
> coming 4.0) with Intel C++ 8.1, including several benchmarks and
> "state-of-the-product" reviews. You can find the article at:
> 
>      http://www.coyotegulch.com/reviews/linux_compilers/
> 
> The above article replaces an older piece I published in late 2002. This
> new comparison marks what I hope will be an ongoing series that tracks
> the quality of Linux compilers.
> 
> ..Scott
> 
> --
> Scott Robert Ladd
> site: http://www.coyotegulch.com
> blog: http://chaoticcoyote.blogspot.com
> 



-- 
Natros

Re: Comparing Linux C and C++ Compilers: Benchmarks and Analysis

[Scott Robert Ladd]

Natros wrote:
> You can use Dinkumware libraries with icc -cxxlib-icc, so you don't
> need a "standard" gcc install. On my gentoo I exported the
> CPLUS_INCLUDE_PATH
> 
> $ export CPLUS_INCLUDE_PATH=/usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/include/g++-v3:/usr/lib/gcc-lib/i6
> 86-pc-linux-gnu/3.3.4/include/g++-v3/i686-pc-linux-gnu

I'd prefer not to address Intel questions in the GCC forum, and this 
will be the last one I answer herein.

I tried using the Dinkum libraries, and still received linker errors in 
some cases. Rather than fight the problem, I figured out what Intel 
wanted from GCC. I'll add a note about this in next week's addendum.

-- 
Scott Robert Ladd
site: http://www.coyotegulch.com
blog: http://chaoticcoyote.blogspot.com

Re: Comparing Linux C and C++ Compilers: Benchmarks and Analysis

[Kaveh R. Ghazi]

 > I've posted a comparison of recent GCC versions (3.3, 3.4, and the
 > coming 4.0) with Intel C++ 8.1, including several benchmarks and
 > "state-of-the-product" reviews. You can find the article at:
 > 
 > http://www.coyotegulch.com/reviews/linux_compilers/

Were you able to figure out the discrepancies posted here?
http://gcc.gnu.org/ml/gcc/2004-08/msg00901.html

Also:
"Performance on alma was dramatically improved by disabling the inline
 mathematical functions (sin, cos) in glibc, and replacing them with
 inline code generated directly by GCC. This is how the Intel compiler
 operates at higher optimization levels; while GCC has not quite
 reached the performance of its commercial competitor, results on
 these benchmarks suggest that the new "tree-ssa" architecture holds
 great promise for improved code generation."

It's not clear from this whether the numbers you posted include
disabling the math functions or not.

(I really need to get access to an x86-linux box again.)

		--Kaveh
--
Kaveh R. Ghazi			ghazi@caip.rutgers.edu

Re: Comparing Linux C and C++ Compilers: Benchmarks and Analysis

[Scott Robert Ladd]

Kaveh R. Ghazi wrote:
> It's not clear from this whether the numbers you posted include
> disabling the math functions or not.

Let's put it this way: including -D__NO_MATH_INLINES -D__NO_INLINE
-D__NO_STRING_INLINES in my compile command lines does not change the
compiler output.

-- 
Scott Robert Ladd
site: http://www.coyotegulch.com
blog: http://chaoticcoyote.blogspot.com

Re: Comparing Linux C and C++ Compilers: Benchmarks and Analysis

[Jeremy Sanders]

On Sat, 18 Sep 2004, Richard Guenther wrote:

> You may want to check gcc 4.0s auto-vectorizing capabilities by using 
> -ftree-vectorize.  I believe Intel compiler does this by default with 
> your optimization settings, and also unrolls loops where profitable 
> (-funroll-loops for all gcc versions).  While I personally do not have 
> success with auto-vectorizing (neither with Intel nor with gcc), loop 
> unrolling shows very positive effects on my numerical codes.

I tried compiling scimark with gcc 4 with -ftree-vectorize -msse2 
-fdump-tree-vect-stats and again with the lno branch. This was with a P4 
on Fedora Core 2. Looking at the generated .vect files, it looked like gcc 
wasn't able to vectorize any of the loops in scimark, which is a bit 
diappointing. It's possible I didn't use it correctly I suppose.

Jeremy

-- 
Jeremy Sanders <jss@ast.cam.ac.uk>   http://www-xray.ast.cam.ac.uk/~jss/
X-Ray Group, Institute of Astronomy, University of Cambridge, UK.
Public Key Server PGP Key ID: E1AAE053

Re: Comparing Linux C and C++ Compilers: Benchmarks and Analysis

[Kaveh R. Ghazi]

 > Let's put it this way: including -D__NO_MATH_INLINES -D__NO_INLINE
 > -D__NO_STRING_INLINES in my compile command lines does not change the
 > compiler output.

Well, that's not what your article says.  That's why I provided the
quote in your own words.
http://gcc.gnu.org/ml/gcc/2004-09/msg01119.html


Also, you were going to address discrepancies from previous reports:
http://gcc.gnu.org/ml/gcc/2004-08/msg00902.html

Did you figure out why your results changed?

		--Kaveh
--
Kaveh R. Ghazi			ghazi@caip.rutgers.edu

Re: Comparing Linux C and C++ Compilers: Benchmarks and Analysis

[Scott Robert Ladd]

Kaveh R. Ghazi wrote:
> Well, that's not what your article says.  That's why I provided the
> quote in your own words.
> http://gcc.gnu.org/ml/gcc/2004-09/msg01119.html

I've clarified the sentence in question; thanks for noting that.

> Also, you were going to address discrepancies from previous reports:
> http://gcc.gnu.org/ml/gcc/2004-08/msg00902.html

I need to take a closer look at where various compilers are getting 
their libraries, just to be certain that everyone is linking to the same 
places.

..Scott

-- 
Scott Robert Ladd
site: http://www.coyotegulch.com
blog: http://chaoticcoyote.blogspot.com

Re: Comparing Linux C and C++ Compilers: Benchmarks and Analysis

[Devang Patel]

On Sep 19, 2004, at 7:02 AM, Jeremy Sanders wrote:

> On Sat, 18 Sep 2004, Richard Guenther wrote:
>
>> You may want to check gcc 4.0s auto-vectorizing capabilities by using 
>> -ftree-vectorize.  I believe Intel compiler does this by default with 
>> your optimization settings, and also unrolls loops where profitable 
>> (-funroll-loops for all gcc versions).  While I personally do not 
>> have success with auto-vectorizing (neither with Intel nor with gcc), 
>> loop unrolling shows very positive effects on my numerical codes.
>
> I tried compiling scimark with gcc 4 with -ftree-vectorize -msse2 
> -fdump-tree-vect-stats and again with the lno branch. This was with a 
> P4 on Fedora Core 2. Looking at the generated .vect files, it looked 
> like gcc wasn't able to vectorize any of the loops in scimark, which 
> is a bit diappointing. It's possible I didn't use it correctly I 
> suppose.

Would it be possible to post what GCC emits in .vect file, using 
-fdump-tree-vect-details option, for loops that you expect to get 
vectorized (or loops that are vectorized by other compilers)?. If you 
can provide preprocessed source file, I can try it on powerpc-darwin.

Thanks,
-
Devang