Re: Sun compares their compiler tools to GNU tools

Re: Sun compares their compiler tools to GNU tools

[N8TM]

In a message dated 11/6/98 7:10:15 PM Pacific Standard Time,
mal@bewoner.dma.be writes:

> g77 is still
>  not up to commercial standards wrt FORTRAN.
On every platform I've tried, there are some tests which g77 performs better
than any other compiler, and there are more which go the other way.  To look
at it from another angle, I'm trying to adapt the g77 testsuite to use with
other compilers.  The egcs testsuite has done good things for g77, bring its
reliability up to commercial standards IMO.

Re: Sun compares their compiler tools to GNU tools

[Toon Moene]

>  Those modest improvements will hopefully get much better
>  one I figure out what has caused Alpha's slow decline
>  over the past few months.  Peak performance was in July,
>  and we've lost about two points since then.

Well, to put up some food for thought, recently I got the following  
for the second loop in the Fortran source below (left is ~ month  
old, right is yesterdays snapshot):

$L432:                                | $L206:
        ldt $f10,-16($4)              |         s8subq $23,16,$2
        ldt $f13,0($3)                |         s8subq $23,8,$5
                                      >         addq $2,$22,$2
                                      >         addq $5,$12,$4
                                      >         ldq $26,544($30)
                                      >         addq $2,$8,$2
                                      >         addq $4,$27,$4
                                      >         addq $10,$2,$2
                                      >         addq $26,$4,$1
                                      >         ldt $f10,0($2)
                                      >         addq $5,$22,$6
                                      >         ldt $f11,0($1)
                                      >         ldq $2,600($30)
                                      >         addq $6,$8,$7
                                      >         fnop
                                      >         fnop
                                      >         s8addq $23,$22,$3
                                      >         ldq $26,552($30)
                                      >         mult $f11,$f10,$f11
                                      >         fnop
                                      >         addq $2,$7,$1
                                      >         addq $3,$8,$3
                                      >         addq $26,$4,$2
                                      >         addq $10,$3,$3
                                      >         ldt $f10,0($3)
                                      >         ldt $f13,0($1)
                                      >         ldt $f12,0($2)
                                      >         addq $5,$14,$1
                                      >         ldq $2,560($30)
                                      >         subt $f13,$f11,$f13
                                      >         addq $1,$8,$1
                                      >         fnop
                                      >         mult $f12,$f10,$f12
                                      >         addq $10,$1,$1
                                      >         addq $5,$11,$3
                                      >         fnop
                                      >         addq $2,$4,$4
                                      >         ldt $f10,0($1)
        ldt $f11,0($4)                          ldt $f11,0($4)
        ldt $f12,0($7)                |         addq $5,$19,$2
        mult $f13,$f10,$f13           |         ldq $26,568($30)
        ldt $f22,0($1)                |         addq $3,$16,$3
        ldt $f25,0($5)                |         fnop
        fnop                          |         fnop
        ldt $f10,0($6)                |         mult $f11,$f10,$f11
        ldt $f26,0($8)                |         subt $f13,$f12,$f13
        fnop                          |         addq $2,$8,$2
        fnop                          |         addq $6,$21,$1
        mult $f22,$f11,$f22           |         addq $26,$3,$3
        ldt $f14,0($22)               |         addq $10,$2,$2
        ldt $f24,0($2)                |         ldt $f10,0($2)
        fnop                          |         ldt $f12,0($3)
        mult $f25,$f10,$f25           |         ldq $2,584($30)
        ldt $f15,0($23)               |         addq $10,$1,$1
        ldt $f23,0($28)               |         fnop
        fnop                          |         fnop
        subt $f12,$f13,$f12           |         mult $f12,$f10,$f12
        mult $f26,$f14,$f26           |         subt $f13,$f11,$f13
        ldt $f10,0($24)               |         addq $6,$0,$6
        ldt $f11,-8($4)               |         addq $5,$13,$5
        mult $f24,$f15,$f24           |         addq $2,$7,$3
        ldt $f13,0($25)               |         ldt $f10,0($1)
        addq $24,8,$24                |         ldt $f15,0($3)
        fnop                          |         addq $10,$6,$6
        mult $f23,$f10,$f23           |         ldq $3,592($30)
        addq $25,8,$25                |         ldt $f11,0($6)
        addq $28,8,$28                |         fnop
        fnop                          |         fnop
        mult $f13,$f11,$f13           |         mult $f15,$f10,$f15
        addq $23,8,$23                |         subt $f13,$f12,$f13
        addq $2,8,$2                  |         ldq $26,576($30)
        fnop                          |         addq $5,$28,$5
        subt $f12,$f22,$f12           |         addq $3,$7,$1
        addq $22,8,$22                |         addl $23,1,$23
        addq $8,8,$8                  |         ldt $f14,0($1)
        fnop                          |         addq $10,$7,$7
        addq $6,8,$6                  |         addq $26,$5,$5
        addq $5,8,$5                  |         ldt $f10,0($7)
        addq $4,8,$4                  |         fnop
        addq $1,8,$1                  |         fnop
        addq $7,8,$7                  |         mult $f14,$f11,$f14
        addq $3,8,$3                  |         subt $f13,$f15,$f13
        subt $f12,$f25,$f12           |         ldt $f12,0($5)
        subl $9,1,$9                  |         subl $20,1,$20
        subt $f12,$f26,$f12           |         mult $f12,$f10,$f12
        subt $f12,$f24,$f12           |         subt $f13,$f14,$f13
        subt $f12,$f23,$f12           |         subt $f13,$f12,$f11
        subt $f12,$f13,$f11           <
        cpys $f31,$f11,$f11                     cpys $f31,$f11,$f11
        cmptle $f27,$f11,$f10         |         cmptle $f22,$f11,$f10
        fcmovne $f10,$f11,$f27        |         fcmovne $f10,$f11,$f22
        bge $9,$L432                  |         fnop
                                      >         bge $20,$L206

[ Obviously, something's wrong with strength reduction, because the
  s8subq's at the top are a result of the -1, 0 and +1 elements below
  - after strength reduction these instructions should be simple
  offsets in addressing modes, not separate instructions ... ]

for the following Fortran code (second loop):

      subroutine sl3d(itmax,n1,n2,n3,alfa,eps,method,impcv,
     & cmnn,cpnn,cnmn,cnpn,cnnn,cnnm,cnnp,
     & src,u,u1,resi,resf)
      implicit double precision (a-h,o-z)
      dimension cmnn(n1,n2,n3),cpnn(n1,n2,n3),cnmn(n1,n2,n3),
     & cnpn(n1,n2,n3),cnnn(n1,n2,n3),src(n1,n2,n3),u(n1,n2,n3),
     & cnnm(n1,n2,n3),cnnp(n1,n2,n3),u1(n1,n2,n3)
      character*(*) method
      alfa1=1.0-alfa
      do k=1,n3
      do j=1,n2
      do i=1,n1
      u1(i,j,k)=u(i,j,k)
      end do
      end do
      end do
      resi=0.0
      do k=2,n3-1
      do j=2,n2-1
      do i=2,n1-1
      res=src(i,j,k)-cmnn(i,j,k)*u(i-1,j,k)-cpnn(i,j,k)*u(i+1,j,k)
     &              -cnmn(i,j,k)*u(i,j-1,k)-cnpn(i,j,k)*u(i,j+1,k)
     &              -cnnm(i,j,k)*u(i,j,k-1)-cnnp(i,j,k)*u(i,j,k+1)
     &              -cnnn(i,j,k)*u(i,j,k)
      resi=max(resi,abs(res))
      end do
      end do
      end do
      if (resi.le.1.0e-20) return
      if (method.eq.'SOR'.or.method.eq.'sor') then
      do it=1,itmax
      err=0.0
      do k=2,n3-1
      do j=2,n2-1
      do i=2,n1-1
      u0=u(i,j,k)
      u(i,j,k)=(src(i,j,k)-cmnn(i,j,k)*u(i-1,j,k)-cpnn(i,j,k)*u(i+1,j,k)
     &              -cnmn(i,j,k)*u(i,j-1,k)-cnpn(i,j,k)*u(i,j+1,k)
     &              -cnnm(i,j,k)*u(i,j,k-1)-cnnp(i,j,k)*u(i,j,k+1)
     &       )/cnnn(i,j,k)*alfa+alfa1*u(i,j,k)
      err=err+abs(u(i,j,k)-u0)
      end do
      end do
      end do
      err=err/float((n1-2)*(n2-2)*(n3-2))
      if (it.eq.1) err0=err
      if (err.le.1.0e-20) return
      if (impcv.ne.0) write(impcv,*) it,err/err0
      if (err0/err.ge.eps) goto 10
      end do
10    continue
      else if (method.eq.'DJACO'.or.method.eq.'djaco') then
       call jaco3d(itmax,n1,n2,n3,alfa,eps,impcv,
     & cmnn,cpnn,cnmn,cnpn,cnnn,cnnm,cnnp,
     & src,u,u1,resi,resf)
      else if (method.eq.'JACOBI'.or.method.eq.'jacobi') then
      do it=1,itmax
      err=0.0
      do k=2,n3-1
      do j=2,n2-1
      do i=2,n1-1
      u(i,j,k)=(src(i,j,k)
     &              -cmnn(i,j,k)*u1(i-1,j,k)-cpnn(i,j,k)*u1(i+1,j,k)
     &              -cnmn(i,j,k)*u1(i,j-1,k)-cnpn(i,j,k)*u1(i,j+1,k)
     &              -cnnm(i,j,k)*u1(i,j,k-1)-cnnp(i,j,k)*u1(i,j,k+1)
     &       )/cnnn(i,j,k)
      err=err+abs(u(i,j,k)-u1(i,j,k))
      end do
      end do
      end do
      err=err/float((n1-2)*(n2-2)*(n3-2))
      if (it.eq.1) err0=err
      if (err.le.1.0e-20) return
      if (impcv.ne.0) write(impcv,*) it,err/err0
      if (err0/err.ge.eps) goto 20
      do k=2,n3-1
      do j=2,n2-1
      do i=2,n1-1
      u1(i,j,k)=u(i,j,k)
      end do
      end do
      end do
      end do
20    continue
      else if (method.eq.'ZEBRA'.or.method.eq.'zebra') then
      do it=1,itmax
      err=0.0
      do kl=2,3
      do jl=2,3
      do il=2,3
      do k=kl,n3-1,2
      do j=jl,n2-1,2
      do i=il,n1-1,2
      u0=u(i,j,k)
      u(i,j,k)=(src(i,j,k)-cmnn(i,j,k)*u(i-1,j,k)-cpnn(i,j,k)*u(i+1,j,k)
     &              -cnmn(i,j,k)*u(i,j-1,k)-cnpn(i,j,k)*u(i,j+1,k)
     &              -cnnm(i,j,k)*u(i,j,k-1)-cnnp(i,j,k)*u(i,j,k+1)
     &       )/cnnn(i,j,k)*alfa+alfa1*u(i,j,k)
      err=err+abs(u(i,j,k)-u0)
      end do
      end do
      end do
      end do
      end do
      end do
      err=err/float((n1-2)*(n2-2)*(n3-2))
      if (it.eq.1) err0=err
      if (err.le.1.0e-20) return
      if (impcv.ne.0) write(impcv,*) it,err/err0
      if (err0/err.ge.eps) goto 30
      end do
30    continue
      else
          print *,'Methode de resolution inconnue: ',method
          stop
      end if
      resf=0.0
      do k=2,n3-1
      do j=2,n2-1
      do i=2,n1-1
      res=src(i,j,k)-cmnn(i,j,k)*u(i-1,j,k)-cpnn(i,j,k)*u(i+1,j,k)
     &              -cnmn(i,j,k)*u(i,j-1,k)-cnpn(i,j,k)*u(i,j+1,k)
     &              -cnnm(i,j,k)*u(i,j,k-1)-cnnp(i,j,k)*u(i,j,k+1)
     &              -cnnn(i,j,k)*u(i,j,k)
      resf=max(resf,abs(res))
      end do
      end do
      end do
      end

Hope this helps,
Toon.

RE: Sun compares their compiler tools to GNU tools

[James Mansion]

> From: Ruslan Shevchenko [ mailto:Ruslan@Shevchenko.Kiev.UA ]
> Subject: Re: Sun compares their compiler tools to GNU tools
> ...
> The test suite must consists of:
> 
>   1. standart implementation of well-known algorithms.
>    (for example, map<int,less<int> >::lexicographical_compare in loop)

Absolutely.  I'd look for a lot of STL issues, also things that exercise
new and
delete in all forms.  Something that works like Tools.H++ usage wouldbe
useful too.

>  
>   2. tupical C++ scientific application.
>    (LINPACK or blass; or some well known simmulation algorithms)
> 
>   3. tupical C++ business application.
>    (for example, some CORBA server.)

The problem here will be to avoid being opsys specific. I'd like to know
where
comilers stand from the point of valuing options and I don't care about
the OS.
It *seems* with rather unscientific test that VC++ and Watcom C++ work
well,
as does the Intel reference compiler, and that Borland, gcc-based
compilers,
and so on are 'also ran' at best.  I was pleasantly susrprised by the
performance
of VC++.

But the tests I did were limited, and I'd like to see a better test.

SpecInt doesn't really work well sincce it relies too much on a
Unix-like IO
library and OS performance.

> 
> 
> -- 
>     @=                                   
>      //RSSH                             
> mailto:Ruslan@Shevchenko.Kiev.UA
> 
> CORBA in Ukraine & ex-USSR: http://www.corbadev.kiev.ua
> 

Re: Sun compares their compiler tools to GNU tools

[Richard Henderson]

On Sat, Nov 07, 1998 at 04:22:36AM -0800, David S. Miller wrote:
> Maybe you did not read it Mike.  Consider that in Appendix B they
> don't even list the GCC spec run detailed listings, or have I missed it?

Appendix B, page 35 in the PDF file.  What is confusing is that the
software tables and flags notes are wrong -- they were not updated
from the sunpro build.

I spent a good part of the weekend running Spec95 to see how we are
doing.  I'll post other bits elsewhere; most of it is depressing.

The bits that relate specifically to this document are the results I
got from running egcs 1.0.3 on similar hardware to that in Sun's report. 
Both mine and theirs were done on a 167 MHz Ultra 1 workstation.  I
don't know how to make Solaris yield up the cache sizes, but I presume
they aren't too far off; ours is a stock machine.

Since my peak numbers were slower than base numbers, I'll compare
Sun's peak and my base:

		Sun		Me
SPECfp		3.87		5.32
SPECint		4.84		4.93

I'd consider SPECint to be within measurement error, as their
individual numbers fluctuate on either side of my median. 

I've no idea what's up with their fp numbers; I'm not aware of any
real changes that should have affected that across the two minor
releases but I suppose it is possible.

More interesting than this whole line of argument, though, is that
current cvs fails 103.su2cor, 104.hydro2d, 141.apsi, 145.fpppp,
and 146.wave5. 

SPECint is only marginally improved at 5.02, though this is with
the jump.c bug that disables conditional move.


r~


--------------------------------------------------------------------------
                        SPEC Benchmark CINT95 Summary
                                       

                     Base      Base      Base      Peak      Peak      Peak
   Benchmarks    Ref Time  Run Time     Ratio  Ref Time  Run Time     Ratio
   ------------  --------  --------  --------  --------  --------  --------
   099.go            4600       759      6.06      4600       759      6.06 
   124.m88ksim       1900       508      3.74      1900       578      3.29 
   126.gcc           1700       308      5.51      1700       347      4.90 
   129.compress      1800       317      5.68      1800       372      4.84 
   130.li            1900       482      3.94      1900       549      3.46 
   132.ijpeg         2400       520      4.62      2400       570      4.21 
   134.perl          1900       342      5.56      1900       342      5.56 
   147.vortex        2700       556      4.85      2700       561      4.81 
   SPECint_base95 (Geom. Mean)           4.93                              
   SPECint95 (Geom. Mean)                                              4.55

                         SPEC Benchmark CFP95 Summary
                                       

                     Base      Base      Base      Peak      Peak      Peak
   Benchmarks    Ref Time  Run Time     Ratio  Ref Time  Run Time     Ratio
   ------------  --------  --------  --------  --------  --------  --------
   101.tomcatv       3700       446      8.30      3700       410      9.02 
   102.swim          8600       807     10.7       8600       794     10.8  
   103.su2cor        1400       382      3.67      1400       390      3.59 
   104.hydro2d       2400       622      3.86      2400       647      3.71 
   107.mgrid         2500       674      3.71      2500       704      3.55 
   110.applu         2200       565      3.90      2200       867      2.54 
   125.turb3d        4100      1034      3.97      4100      1109      3.70 
   141.apsi          2100       403      5.21      2100       367      5.73 
   145.fpppp         9600      1260      7.62      9600      1310      7.33 
   146.wave5         3000       468      6.41      3000       456      6.58 
   SPECfp_base95 (Geom. Mean)            5.32                              
   SPECfp95 (Geom. Mean)                                               5.10


                                   HARDWARE
                                   --------
         Model Name: UltraI
                CPU: 167 MHz Ultra-1
                FPU: Integrated
   Number of CPU(s): 1
      Primary Cache: ?
    Secondary Cache: ?
        Other Cache: 
             Memory: 128MB
     Disk Subsystem: 3 x 2GB
                     
     Other Hardware: Ethernet


                                   SOFTWARE
                                   --------
   Operating System: Solaris 2.5.1
           Compiler: egcs-2.90.29 980515 (egcs-1.0.3 release)
        File System: ufs
       System State: Multi User

               Base: -O2

               Peak: -O3 -funroll-loops
                     145: -O3 -funroll-loops -fno-schedule-insns

Re: Sun compares their compiler tools to GNU tools

[job bogan]

Your message dated: Tue, 10 Nov 1998 12:58:04 PST
>   Date: Tue, 10 Nov 1998 12:27:38 -0800
>   From: Richard Henderson <rth@cygnus.com>
>
>   I don't know how to make Solaris yield up the cache sizes, but I
>   presume they aren't too far off; ours is a stock machine.
> 
>"prtconf -pv" should spit it out somewhere.

nope, you want prtdiag.  

> /usr/platform/sun4u/sbin/prtdiag 
System Configuration:  Sun Microsystems  sun4u Sun Ultra 1 UPA/SBus (UltraSPARC 167MHz)
System clock frequency: 83 MHz
Memory size: 128 Megabytes

========================= CPUs =========================

                    Run   Ecache   CPU    CPU
Brd  CPU   Module   MHz     MB    Impl.   Mask
---  ---  -------  -----  ------  ------  ----
 0     0     0      167     0.5   US-I     4.0


========================= IO Cards =========================

     Bus   Freq
Brd  Type  MHz   Slot  Name                              Model
---  ----  ----  ----  --------------------------------  ----------------------
 0   SBus   25     0   cgsix                             SUNW,501-2325         
 0   SBus   25     1   SUNW,rtvc                         SUNW,501-2232-03REV50 
 0   SBus   25    14   SUNW,fas/sd (block)                                     
 0   SBus   25    14   SUNW,hme                                                
 0   SBus   25    14   SUNW,bpp                                                
 0   UPA    83    30   FFB, Single Buffered              SUNW,501-2634         

No failures found in System
===========================
> 

job

Re: Sun compares their compiler tools to GNU tools

[Richard Henderson]

On Tue, Nov 10, 1998 at 09:40:42PM +0100, Gerald Pfeifer wrote:
> `/usr/platform/sun4u/sbin/prtdiag -v` should do the job:

Ok, here's the output for lisa then,

System Configuration:  Sun Microsystems  sun4u Sun Ultra 1 SBus \
(UltraSPARC 167MHz)
System clock frequency: 84 MHz
Memory size: 128 Megabytes
       CPU Units: Frequency Cache-Size Version
            A: MHz  MB  Impl. Mask  B: MHz  MB  Impl. Mask
            ----------  ----- ----  ----------  ----- ----
               167 0.5   10    4.0                          

> PS: It might be interesting to see those numbers for 1.1 and current-cvs;
>     David's rewrote should result in some improvements, I guess?

As mentioned in the other mail, half of SPECfp fails, and SPECint
only has modest improvements.

Those modest improvements will hopefully get much better one I 
figure out what has caused Alpha's slow decline over the past
few months.  Peak performance was in July, and we've lost about
two points since then.


r~

Re: Sun compares their compiler tools to GNU tools

[David S. Miller]

   Date: Tue, 10 Nov 1998 13:13:59 -0800
   From: Richard Henderson <rth@cygnus.com>

   On Tue, Nov 10, 1998 at 12:58:04PM -0800, David S. Miller wrote:
   > BTW, did you remember to enable -mtune=ultrasparc?

   Oh yes.  I forgot to mention that the toolchain was built
   with --with-cpu=ultrasparc.

Don't use "--with-cpu=ultrasparc" it's deadly, "--mtune=" is the
only safe Ultra optimization enabling method with current cvs at the
moment.  I am not surprised half of specfp failed now that I know you
did this.

Later,
David S. Miller
davem@dm.cobaltmicro.com

Re: Sun compares their compiler tools to GNU tools

[David S. Miller]

   Date: Tue, 10 Nov 1998 12:27:38 -0800
   From: Richard Henderson <rth@cygnus.com>

   I don't know how to make Solaris yield up the cache sizes, but I
   presume they aren't too far off; ours is a stock machine.
 
"prtconf -pv" should spit it out somewhere.

BTW, did you remember to enable -mtune=ultrasparc?  They certainly
used the equilavent option in the sunpro runs (actually even if they
didn't, sunpro uses various system calls etc. to find out what kind of
cpu the machine is and enables some relevant switches based upon that
information).

Later,
David S. Miller
davem@dm.cobaltmicro.com

Re: Sun compares their compiler tools to GNU tools

[Gerald Pfeifer]

On Tue, 10 Nov 1998, Richard Henderson wrote:
> I don't know how to make Solaris yield up the cache sizes, but I
> presume they aren't too far off; ours is a stock machine.

`/usr/platform/sun4u/sbin/prtdiag -v` should do the job:

  :
                      Run   Ecache   CPU    CPU
  Brd  CPU   Module   MHz     MB    Impl.   Mask
  ---  ---  -------  -----  ------  ------  ----
   0     0     0      143     0.5   US-I     2.2
  :

Gerald

PS: It might be interesting to see those numbers for 1.1 and current-cvs;
    David's rewrote should result in some improvements, I guess?
-- 
Gerald Pfeifer (Jerry)      Vienna University of Technology
pfeifer@dbai.tuwien.ac.at   http://www.dbai.tuwien.ac.at/~pfeifer/

Re: Sun compares their compiler tools to GNU tools

[Richard Henderson]

On Tue, Nov 10, 1998 at 12:58:04PM -0800, David S. Miller wrote:
> BTW, did you remember to enable -mtune=ultrasparc?

Oh yes.  I forgot to mention that the toolchain was built
with --with-cpu=ultrasparc.


r~

Re: Sun compares their compiler tools to GNU tools

[Linus Torvalds]

In article < k2u30abm8b.fsf@zero.aec.at >, Andi Kleen  <ak@muc.de> wrote:
>
>The problem seems to be that inline functions generate worse code than
>hand expanded macros: KAI C++ expands the inline functions before gcc
>sees it and gets better code.

Gcc inline functions have fairly bad performance, which is sad (and
contrasts against the documentation which claims that inline functions
are as fast as macros - which is definitely not true). 

The reason for gcc inline functions being slow _appears_ to be that
inlining is done after CSE and some register allocation.  Which is good,
because it means that CSE will nicely CSE inline functions too.  I've
used that to good effect to force gcc to CSE stuff it wouldn't otherwise
have found. 

However, it's also bad, because it means that CSE will _not_ find common
code inside and outside of a inline function - which is fairly common. 

(Note: the above is just from looking at gcc generated code - not from
actually looking at gcc sources.  Maybe the code generation issues I've
seen are for some other reason than CSE and regalloc, so take the above
with a grain of salt). 

			Linus

Re: Sun compares their compiler tools to GNU tools

[Joe Buck]

> KAI C++ seems to do better than egcs on the Stepanov Abstraction benchmark.
> ...

> The problem seems to be that inline functions generate worse code than
> hand expanded macros: KAI C++ expands the inline functions before gcc
> sees it and gets better code.

egcs has made considerable improvements in particular cases; ADDRESSOF
can substantially reduce the abstraction penalty for passing struct
or class objects consisting of a single field.

There's been some discussion of solving the problem by doing more with
trees, eg do the inlining while we still have trees before creating the
RTL.  Jim Wilson has looked at this, I don't know the status.

Re: Sun compares their compiler tools to GNU tools

[N8TM]

In a message dated 11/7/98 10:45:26 PM Pacific Standard Time, pcg@goof.com
writes:

> time to make the egcs-benchmark suite better by adding spec95 to it ;->
>  
I run a version of Livermore Kernels on each snapshot.
It takes about 90 seconds on Irix or W95, not over 40 minutes on any OS.  I'm
sure others do some simple benchmarking.  Given the close tie-in between gcc
and g77, it's not necessary to run the same benchmark regularly on more than
one language.

Re: Sun compares their compiler tools to GNU tools

[Andi Kleen]

In article < 199811071823.KAA07463@yamato.synopsys.com >,
Joe Buck <jbuck@synopsys.com> writes:
>> KAI C++ have better performance than Sun CC (near 30%) and
>> Sun CC near 30% better gcc,

> Such figures are meaningless.  Better performance on what?

> There are cases where KAI C++ will have about 5x the performance of
> either Sun CC or g++, and other cases where the performance is
> exactly the same (KAI does C++ -> C so it is limited by the performance
> of the C compiler you choose, which might be gcc or Sun C).

KAI C++ seems to do better than egcs on the Stepanov Abstraction benchmark.
This benchmark implements a simple algorithm in various abstraction
levels (e.g. hand coded, with STL, without, with classes and members etc.)
Then it tests how much slower the abstracted versions run compared to the
handcoded version. 

The problem seems to be that inline functions generate worse code than
hand expanded macros: KAI C++ expands the inline functions before gcc
sees it and gets better code.

The Stepanov benchmark is available from ftp.kai.com.


-Andi

Re: Sun compares their compiler tools to GNU tools

[Gerald Pfeifer]

David Miller already responded in quite some detail (Thanks!), so I'm
only adding some 0.2 Schilling.

On Fri, 6 Nov 1998, Mike Stump wrote:
>> Shall we add some official reply from EGCS developers and testers) to
>> our web page?
> If we can't say we're better now, what point is it?

First of all, we *are* better now! egcs-1.1 and egcs-mainline generate
more efficient code than egcs-1.0.3 and gcc 2.7, at least for those
experiments that I have conducted.

Then, there is more than benchmark data. Sun claims that their environment
allow for faster development, ignoring free Tools like DDD for example
( http://www.cs.tu-bs.de/softech/ddd/ ).

Then, there is cross-platform availability, something which Sun cannot
offer (except for x86 Solaris).


> It looked detailed enough for me.  Maybe you don't play in the
> benchmark circles?  Maybe you can back up your statement better. What
> detail did you miss?

The third-party benchmarks from Vividata are not reproducable, which
means that they are absolutely not scientifically founded.  (And yes,
I am in that business.)

I am sure one can up with a special benchmark for any compiler, where
this very compiler is twice as fast as any other compiler. (Cf. what
Sun has done when ``improving'' there Java benchmark performance.)

Gerald
-- 
Gerald Pfeifer (Jerry)      Vienna University of Technology
pfeifer@dbai.tuwien.ac.at   http://www.dbai.tuwien.ac.at/~pfeifer/

Re: Sun compares their compiler tools to GNU tools

[Ruslan Shevchenko]

Joe Buck wrote:
> 
> ? KAI C++ have better performance than Sun CC (near 30%) and
> ? Sun CC near 30% better gcc,
> 
> Such figures are meaningless.  Better performance on what?
> 

The description of tests is situated on KAI C++ web site.

All tests are published. (You will find it during browsing)

They are writeln with idea of illustrating  optimizations
tehnicues.

> There are cases where KAI C++ will have about 5x the performance of
> either Sun CC or g++, and other cases where the performance is
> exactly the same (KAI does C++ -? C so it is limited by the performance
> of the C compiler you choose, which might be gcc or Sun C).
> 
> There are cases where egcs beats Sun CC and vice versa.  And how
> should we measure the ratio of the vast number of valid C++ programs
> that egcs compiles and Sun does not?

IMHO, it is possible.

The test suite must consists of:

  1. standart implementation of well-known algorithms.
   (for example, map<int,less<int> >::lexicographical_compare in loop)
 
  2. tupical C++ scientific application.
   (LINPACK or blass; or some well known simmulation algorithms)

  3. tupical C++ business application.
   (for example, some CORBA server.)


-- 
    @=                                   
     //RSSH                             
mailto:Ruslan@Shevchenko.Kiev.UA

CORBA in Ukraine & ex-USSR: http://www.corbadev.kiev.ua

Re: Sun compares their compiler tools to GNU tools

[Marc Lehmann]

On Fri, Nov 06, 1998 at 03:06:06PM -0800, Mike Stump wrote:
> > latest compiler at the time, I found my particular application ran
> > about 30% faster using egcs.
> 
> Maybe we should start a page with user experiences with egcs?  Real
> customers with real applications who are willing to benchmark fairly
> between the two compilers, the latest compilers from each.

Collecting experiences like this would be helpful, but I fear it
will be difficult to collect.

> We then only need to decide, do we list all, or just the ones that are

I think listing all is better: we don't have to "sell" a product for money,
nor do we need be better - being honest (and we are not too bad) will
actually give us a better reputation, and this also reminds us of where we
stand.

      -----==-                                              |
      ----==-- _                                            |
      ---==---(_)__  __ ____  __       Marc Lehmann       +--
      --==---/ / _ \/ // /\ \/ /       pcg@goof.com       |e|
      -=====/_/_//_/\_,_/ /_/\_\                          --+
    The choice of a GNU generation                        |
                                                          |

Re: Sun compares their compiler tools to GNU tools

[Toon Moene]

>  WRT the chapter on performance, 34% difference on specint
>  is not all that bad. Specfp that needs fortran is a known
>  problem. g77 is still not up to commercial standards wrt
>  FORTRAN. On the other hand their hint that the difference
>  carries over to C because they share the same backend is
>  tenuous and doesn't show in their own data.

If the esteemed review committee of Linux Expo deems my paper [to  
be written] on the issue of "Fortran related optimisations in egcs"  
(that's not the title - the real one is better) worthy of a  
presentation at Linux Expo, I'm probably the only one presenting  
interesting work that isn't "original" in the sense that it hasn't  
been written up / implemented elsewhere (see  
http://www.cygnus.com/ml/egcs/1997-Dec/0396.html and  
http://www.cygnus.com/ml/egcs/1998-Jul/1034.html )

We still have a long way to go ...

--
Toon Moene ( mailto:toon@moene.indiv.nluug.nl )
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
g77 Support: mailto:fortran@gnu.org ; egcs: mailto:egcs-bugs@cygnus.com

Re: Sun compares their compiler tools to GNU tools

[Joe Buck]

> KAI C++ have better performance than Sun CC (near 30%) and
> Sun CC near 30% better gcc,

Such figures are meaningless.  Better performance on what?

There are cases where KAI C++ will have about 5x the performance of
either Sun CC or g++, and other cases where the performance is
exactly the same (KAI does C++ -> C so it is limited by the performance
of the C compiler you choose, which might be gcc or Sun C).

There are cases where egcs beats Sun CC and vice versa.  And how
should we measure the ratio of the vast number of valid C++ programs
that egcs compiles and Sun does not?

Re: Sun compares their compiler tools to GNU tools

[Marc Lehmann]

On Fri, Nov 06, 1998 at 01:41:50PM -0800, Joe Buck wrote:
> 
> > When I compared egcs to Sun's C compiler about a year ago, using their
> > latest compiler at the time, I found my particular application ran
> > about 30% faster using egcs.  This was on a 300mhz ultrasparc.
> 
> Different people have reported quite different ratios.  We need
> better data, I think.

time to make the egcs-benchmark suite better by adding spec95 to it ;->

*sigh*, why can't it be that easy in reality.

      -----==-                                              |
      ----==-- _                                            |
      ---==---(_)__  __ ____  __       Marc Lehmann       +--
      --==---/ / _ \/ // /\ \/ /       pcg@goof.com       |e|
      -=====/_/_//_/\_,_/ /_/\_\                          --+
    The choice of a GNU generation                        |
                                                          |

Re: Sun compares their compiler tools to GNU tools

[Joe Buck]

> Maybe we should start a page with user experiences with egcs?  Real
> customers with real applications who are willing to benchmark fairly
> between the two compilers, the latest compilers from each.
> 
> We then only need to decide, do we list all, or just the ones that are
> better?

We list everything, of course, and treat any cases where we lose as
challenges to do better.  

Re: Sun compares their compiler tools to GNU tools

[Doug Brown]

And, of course, the Sun comparison completely ignores two chief advantages of
gcc/egcs: portability to a wide range of platforms, and much greater ISO/IEC
14882 compliance.

So if you're looking for a c++ compiler that only runs on Sun platforms, that
doesn't support bool types, or namespaces, or a host of other ANSI standard
features, and doesn't come bundled with STD/STL, but runs somewhat faster
than a three-year old version of gcc in carefully selected tests using a
statistically flawed testing procedure, then I guess Sun CC is the way to go!
:-)


Doug

mal@bewoner.dma.be wrote:

> Colin Douglas Howell writes:
>  > Sun has published a marketing comparison of their compiler suite,
>  > Visual WorkShop C++ 3.0, with the GNU compiler tools.  See the URL:
>  >
>  >      http://www.sun.com/workshop/visual/
>  >
>  > The comparison includes performance test results on UltraSPARC.  Of
>  > course, this is designed to sell Sun's compiler tools, and is thus
>  > rather biased.  Also, their performance tests were done against old
>  > versions of gcc; they compare version 4.2 of their C, C++, and FORTRAN
>  > compilers with egcs 1.0.1 (and in one case with gcc 2.7.2.2).  So the
>  > recent egcs optimizations and SPARC backend improvements are not
>  > tested.  (On the other hand, Sun is currently beta-testing version 5.0
>  > of their compilers.)
>  >
>  > I figured people might want to know what the competition is up to.  Is
>  > anyone in a position to comment on the specific tests or do
>  > comparisons with the current egcs versions?
>  >
>
> Around here we have the SunSoft Visual Workshop and egcs. I'll add a
> few comments on their text.
>
> In the introduction one of their points is "Specific code, such as C++
> exception handling, ran up to 10 times faster." If EH is on your
> critical path, you have problems no compiler can solve.
>
> WRT the chapter on performance, 34% difference on specint is not all
> that bad. Specfp that needs fortran is a known problem. g77 is still
> not up to commercial standards wrt FORTRAN. On the other hand their
> hint that the difference carries over to C because they share the same
> backend is tenuous and doesn't show in their own data.
>
> Another distortion is their claim that they used the "enhanced version
> of Cygnus' gcc, which is usually a version ahead of the freeware
> version in terms of performance and functionality" while they were
> using egcs which is a "freeware version".
>
> Their case study with Vividata gave numbers between 7 and 30%
> difference on runs where the numbers represented the *best times* for
> each compiler. This is methodologically unsound since you have to take
> into account scheduling issues etc. They should take the means or
> publish the whole data. Further for this case study they used gcc
> 2.7.2.2 instead of egcs.
>
> Their most touted advantages are not compiler related. They hype their
> Workshop integrated tools (of which an incredibly buggy version of
> Xemacs 20.0 is the core ;-) ). There, the comparison is decidedly
> unfair since they compare their debug session with tooltalk integrated
> dbx and emacs to a command line gdb session. I personally find gud in
> Xemacs more usable than the Sun stuff. And for some really complex
> debugging sessions ddd is even better.
>
> Other tools include an enhanced performance profiler tcov and
> automatic profile feedback-based optimisations. This is an area in
> which egcs is lacking.
>
> They also have a class browser. Since OOBR is no longer distributed
> with XEmacs there is also a lacune there. Although of our 8 developers
> here nobody uses the thing. YMMV
>
> Their Runtime Checking can be done with a special malloc library and
> things like efence are more advanced. It's something of a poor man's
> Purify.
>
> Their appendix B claims to publish the full SPEC test results but
> their is some error in it since they don't seem to publish the gcc
> results. I also couldn't find which flags they were using since they
> only publish the flags used for Sun cc.
>
> It would also be interesting to test the difference something like GNU
> rope would make.
>
> Arguing about preferences between tools ends pretty fast in a
> religious war but I find their comparison very flawed.
>
>  > Also, this comparison is currently being discussed at slashdot.org,
>  > just in case anyone wants to add a voice of reason into that
>  > discussion.
>  >
>
> Life is too short to take up the task of inserting reason in
> /. discussions.
>
>  > --
>  > Colin Douglas Howell                    Systems Administrator
>  > e-mail:  howell@cs.stanford.edu         Computer Facilities Group
>  > office:  (650) 723-2491                 Computer Science Department
>  >                                         Stanford University

Re: Sun compares their compiler tools to GNU tools

[David S. Miller]

   Date: Fri, 6 Nov 1998 14:04:00 -0800
   From: mrs@wrs.com (Mike Stump)

   > BTW, the page at the URL above is ridiculous,

   No, it is not.  It is perfectly reasonable and valid.

It lacks internal consistency at least.  They say they use egcs as
their example of the latest and greatest, in this document, and then
they proceed to use gcc-2.7.2 for one of the primary benchmarks.

I think using a 4 year old example of gcc's optimization technology
is nothing short of rediculious.

Next, they pass off all of gdb's features, and for one that graphical
GUI's are available for it and run just fine.  The impression left by
this memo is that it is command line only.  This simply is not true.

   It looked detailed enough for me.  Maybe you don't play in the
   benchmark circles?  Maybe you can back up your statement better.  What
   detail did you miss?  Did you follow the link and read the whole
   thing, or just the 2 paragraphs?

Maybe you did not read it Mike.  Consider that in Appendix B they
don't even list the GCC spec run detailed listings, or have I missed
it?  If I have, please show me where in the document it is.  As far as
I am concerned, as far as detail goes, this memo was completely
lacking.  It's a marketing prop, nothing more, and it was designed and
played out as one.

   >  . What's ``the latest from GNU''

   What part of ``egcs-2.90.23 980102 (egcs-1.0.1 release)'' was unclear
   to you?

Correct, but latest GNU is not gcc-2.7.2

   >  . Customer application (C code)  7%-30%
   >    No source code available?

   You seem to know know or understand specint or benchmarking, please
   read comp.benchmarks for a year.

This may be how things are done, but it still does not mean that using
such tactics is utter crap.  One nice thing for vendors and these
sorts of benchmarks is that nobody can then refute their claims.  Open
benchmarks would kill this problem.  And thats the sort of thing the
EGCS project should use, so people could easily check up on any claims
that we make about gcc.

Later,
David S. Miller
davem@dm.cobaltmicro.com

Re: Sun compares their compiler tools to GNU tools

[Colin Douglas Howell]

Mike Stump writes:
> 
> > Date: Fri, 6 Nov 1998 00:49:59 +0100 (MET)
> > From: Gerald Pfeifer <pfeifer@dbai.tuwien.ac.at>
> > To: egcs@cygnus.com
> > cc: Colin Douglas Howell <howell@cs.stanford.edu>
> 
> > On Wed, 4 Nov 1998, Colin Douglas Howell wrote:
> > > Sun has published a marketing comparison of their compiler suite,
> > > Visual WorkShop C++ 3.0, with the GNU compiler tools.  See the URL:
> > > 
> > > 	http://www.sun.com/workshop/visual/
> 
> > Shall we add some official reply from EGCS developers and testers) to our
> > web page?
> 
> If we can't say we're better now, what point is it?
> 
> > BTW, the page at the URL above is ridiculous,
> 
> No, it is not.  It is perfectly reasonable and valid.
> 
> > and this is the main page of that product! No link to more detailed
> > information is evident.
> 
> It looked detailed enough for me.  Maybe you don't play in the
> benchmark circles?  Maybe you can back up your statement better.  What
> detail did you miss?  Did you follow the link and read the whole
> thing, or just the 2 paragraphs?
> 
> >  . What's ``the latest from GNU''
> 
> What part of ``egcs-2.90.23 980102 (egcs-1.0.1 release)'' was unclear
> to you?
> 
> >    in ``We examined the performance of the Sun C and C++ compilers,
> >    comparing them with the latest from GNU''?
> 
> >  . Customer application (C code)  7%-30%
> >    No source code available?
> 
> You seem to know know or understand specint or benchmarking, please
> read comp.benchmarks for a year.
> 
> >    What does the range mean?
> 
> A range means that for some specific inputs to the application, the
> performance was only 7% better, for other inputs, it was 30% faster.
> I think this was perfectly clear and I think it was fully explained in
> the article.  I'm begining to think that you didn't even spend the
> time to read what they wrote.

Calm down.  Of course he hadn't read the article, because he couldn't
find the link to it.  ("No link to more detailed information is
evident.")  I pointed the link out to him in a reply.  Clearly Sun's
web page design could use some work.

We're all on the same team, OK?

-- 
Colin Douglas Howell                    Systems Administrator
e-mail:  howell@cs.stanford.edu         Computer Facilities Group
office:  (650) 723-2491                 Computer Science Department
                                        Stanford University

Re: Sun compares their compiler tools to GNU tools

[Mike Stump]

> Date: Fri, 6 Nov 1998 00:49:59 +0100 (MET)
> From: Gerald Pfeifer <pfeifer@dbai.tuwien.ac.at>
> To: egcs@cygnus.com
> cc: Colin Douglas Howell <howell@cs.stanford.edu>

> On Wed, 4 Nov 1998, Colin Douglas Howell wrote:
> > Sun has published a marketing comparison of their compiler suite,
> > Visual WorkShop C++ 3.0, with the GNU compiler tools.  See the URL:
> > 
> > 	http://www.sun.com/workshop/visual/

> Shall we add some official reply from EGCS developers and testers) to our
> web page?

If we can't say we're better now, what point is it?

> BTW, the page at the URL above is ridiculous,

No, it is not.  It is perfectly reasonable and valid.

> and this is the main page of that product! No link to more detailed
> information is evident.

It looked detailed enough for me.  Maybe you don't play in the
benchmark circles?  Maybe you can back up your statement better.  What
detail did you miss?  Did you follow the link and read the whole
thing, or just the 2 paragraphs?

>  . What's ``the latest from GNU''

What part of ``egcs-2.90.23 980102 (egcs-1.0.1 release)'' was unclear
to you?

>    in ``We examined the performance of the Sun C and C++ compilers,
>    comparing them with the latest from GNU''?

>  . Customer application (C code)  7%-30%
>    No source code available?

You seem to know know or understand specint or benchmarking, please
read comp.benchmarks for a year.

>    What does the range mean?

A range means that for some specific inputs to the application, the
performance was only 7% better, for other inputs, it was 30% faster.
I think this was perfectly clear and I think it was fully explained in
the article.  I'm begining to think that you didn't even spend the
time to read what they wrote.

Re: Sun compares their compiler tools to GNU tools

[Ruslan Shevchenko]

Gerald Pfeifer wrote:
> 
> On Wed, 4 Nov 1998, Colin Douglas Howell wrote:
> ? Sun has published a marketing comparison of their compiler suite,
> ? Visual WorkShop C++ 3.0, with the GNU compiler tools.  See the URL:
> ?
> ?       http://www.sun.com/workshop/visual/
> 
> Shall we add some official reply from EGCS developers and testers) to our
> web page?
> 
> Gerald
> 
> BTW, the page at the URL above is ridiculous, and this is the main page
> of that product! No link to more detailed information is evident.
> 
>  . What's ``the latest from GNU'' in ``We examined the performance of the
>    Sun C and C++ compilers, comparing them with the latest from GNU''?
> 
>  . Customer application (C code)  7%-30%
>    No source code available? What does the range mean?
>  :

I have Sun CC 4.2 on Sun Solaris and can run few tests
  (and my tupical "customer" application) and publish results.

IMHO, best time for this is after 1.1.1 release.

P.S. Interestion olso comparasion KAI C++ and Sun CC / gcc.

KAI C++ have better performance than Sun CC (near 30%) and
Sun CC near 30% better gcc,

>  :
> --
> Gerald Pfeifer (Jerry)      Vienna University of Technology
> pfeifer@dbai.tuwien.ac.at   http://www.dbai.tuwien.ac.at/~pfeifer/

-- 
    @=                                   
     //RSSH                             
mailto:Ruslan@Shevchenko.Kiev.UA

CORBA in Ukraine & ex-USSR: http://www.corbadev.kiev.ua

Re: Sun compares their compiler tools to GNU tools

[Mike Stump]

> To: Colin Douglas Howell <howell@cs.stanford.edu>
> From: hjstein@bfr.co.il (Harvey J. Stein)
> Date: 06 Nov 1998 14:36:04 +0200

> When I compared egcs to Sun's C compiler about a year ago, using their
> latest compiler at the time, I found my particular application ran
> about 30% faster using egcs.

Maybe we should start a page with user experiences with egcs?  Real
customers with real applications who are willing to benchmark fairly
between the two compilers, the latest compilers from each.

We then only need to decide, do we list all, or just the ones that are
better?

Re: Sun compares their compiler tools to GNU tools

[Joe Buck]

> When I compared egcs to Sun's C compiler about a year ago, using their
> latest compiler at the time, I found my particular application ran
> about 30% faster using egcs.  This was on a 300mhz ultrasparc.

Different people have reported quite different ratios.  We need
better data, I think.

Sun compares their compiler tools to GNU tools

[mal]

Colin Douglas Howell writes:
 > Sun has published a marketing comparison of their compiler suite,
 > Visual WorkShop C++ 3.0, with the GNU compiler tools.  See the URL:
 > 
 > 	http://www.sun.com/workshop/visual/
 > 
 > The comparison includes performance test results on UltraSPARC.  Of
 > course, this is designed to sell Sun's compiler tools, and is thus
 > rather biased.  Also, their performance tests were done against old
 > versions of gcc; they compare version 4.2 of their C, C++, and FORTRAN
 > compilers with egcs 1.0.1 (and in one case with gcc 2.7.2.2).  So the
 > recent egcs optimizations and SPARC backend improvements are not
 > tested.  (On the other hand, Sun is currently beta-testing version 5.0
 > of their compilers.)
 > 
 > I figured people might want to know what the competition is up to.  Is
 > anyone in a position to comment on the specific tests or do
 > comparisons with the current egcs versions?
 > 

Around here we have the SunSoft Visual Workshop and egcs. I'll add a
few comments on their text.

In the introduction one of their points is "Specific code, such as C++ 
exception handling, ran up to 10 times faster." If EH is on your
critical path, you have problems no compiler can solve.

WRT the chapter on performance, 34% difference on specint is not all
that bad. Specfp that needs fortran is a known problem. g77 is still
not up to commercial standards wrt FORTRAN. On the other hand their
hint that the difference carries over to C because they share the same 
backend is tenuous and doesn't show in their own data.

Another distortion is their claim that they used the "enhanced version 
of Cygnus' gcc, which is usually a version ahead of the freeware
version in terms of performance and functionality" while they were
using egcs which is a "freeware version".

Their case study with Vividata gave numbers between 7 and 30%
difference on runs where the numbers represented the *best times* for
each compiler. This is methodologically unsound since you have to take 
into account scheduling issues etc. They should take the means or
publish the whole data. Further for this case study they used gcc
2.7.2.2 instead of egcs.

Their most touted advantages are not compiler related. They hype their 
Workshop integrated tools (of which an incredibly buggy version of
Xemacs 20.0 is the core ;-) ). There, the comparison is decidedly
unfair since they compare their debug session with tooltalk integrated 
dbx and emacs to a command line gdb session. I personally find gud in
Xemacs more usable than the Sun stuff. And for some really complex
debugging sessions ddd is even better. 

Other tools include an enhanced performance profiler tcov and
automatic profile feedback-based optimisations. This is an area in
which egcs is lacking.

They also have a class browser. Since OOBR is no longer distributed
with XEmacs there is also a lacune there. Although of our 8 developers 
here nobody uses the thing. YMMV

Their Runtime Checking can be done with a special malloc library and
things like efence are more advanced. It's something of a poor man's
Purify. 

Their appendix B claims to publish the full SPEC test results but
their is some error in it since they don't seem to publish the gcc
results. I also couldn't find which flags they were using since they
only publish the flags used for Sun cc.

It would also be interesting to test the difference something like GNU 
rope would make.

Arguing about preferences between tools ends pretty fast in a
religious war but I find their comparison very flawed.

 > Also, this comparison is currently being discussed at slashdot.org,
 > just in case anyone wants to add a voice of reason into that
 > discussion.
 > 

Life is too short to take up the task of inserting reason in
/. discussions. 

 > -- 
 > Colin Douglas Howell                    Systems Administrator
 > e-mail:  howell@cs.stanford.edu         Computer Facilities Group
 > office:  (650) 723-2491                 Computer Science Department
 >                                         Stanford University

Re: Sun compares their compiler tools to GNU tools

[Joe Buck]

> On Wed, 4 Nov 1998, Colin Douglas Howell wrote:
> > Sun has published a marketing comparison of their compiler suite,
> > Visual WorkShop C++ 3.0, with the GNU compiler tools.  See the URL:
> > 
> > 	http://www.sun.com/workshop/visual/
> 
> Shall we add some official reply from EGCS developers and testers) to our
> web page?

Before we do, we should obtain more data: do we have Spec95 numbers for
egcs-1.1, for example?  We want to be very careful to avoid descending
to Sun's level -- anything we write should be precise and defendable.

Re: Sun compares their compiler tools to GNU tools

[Harvey J. Stein]

N8TM@aol.com writes:

 > In a message dated 11/5/98 11:34:24 AM Pacific Standard Time,
 > howell@cs.stanford.edu writes:
 > 
 > >  Is anyone in a position to comment on the specific tests or do
 > >  comparisons with the current egcs versions?
 > I personally don't find the Sun claims overly impressive or
 > exaggerated, other than the biased view of which egcs versions are
 > "latest."  The vendors' compilers show much bigger margins than
 > this on other platforms, but that doesn't prevent egcs from being
 > seen as a useful option.

*Why* is everyone so quick to assume that commercial compilers produce
better code than GCC/EGCS?  The answer is *it* *depends*.  GCC & EGCS
often do better than vendor compilers.  *Test* your own applications.
*Profile* them to see which complier does better on each subroutine.
You *will* find subroutines where each compiler beats the other.

-- 
Harvey J. Stein
BFM Financial Research
hjstein@bfr.co.il

Re: Sun compares their compiler tools to GNU tools

[Colin Douglas Howell]

Gerald Pfeifer writes:
> 
> On Wed, 4 Nov 1998, Colin Douglas Howell wrote:
> > Sun has published a marketing comparison of their compiler suite,
> > Visual WorkShop C++ 3.0, with the GNU compiler tools.  See the URL:
> > 
> > 	http://www.sun.com/workshop/visual/
> 
> Shall we add some official reply from EGCS developers and testers) to our
> web page?
> 
> Gerald
> 
> BTW, the page at the URL above is ridiculous, and this is the main page
> of that product! No link to more detailed information is evident.
> 
>  . What's ``the latest from GNU'' in ``We examined the performance of the
>    Sun C and C++ compilers, comparing them with the latest from GNU''?
> 
>  . Customer application (C code)  7%-30%
>    No source code available? What does the range mean?

To be fair to Sun, they do give more information.  I gave that URL
because it was where the initial blurb for "Sun vs. GNU" was made.
The detailed white paper is accessible from the 3 links just under the
"What's Hot!" heading:

		NEW Moving to Sun Visual WorkShop: A
                How and Why Transition Guide for GNU
                Developers
                HTML | PS | PDF

The words "HTML | PS | PDF" are the links and provide the white paper
in three different formats.

The customer application mentioned is described in the white paper.

Also the phrase "latest from GNU" is a bit disingenuous, if not
downright wrong.  Most of their comparisons are against egcs 1.0.1;
one is against gcc 2.7.2.2.  Hardly the latest.

-- 
Colin Douglas Howell                    Systems Administrator
e-mail:  howell@cs.stanford.edu         Computer Facilities Group
office:  (650) 723-2491                 Computer Science Department
                                        Stanford University

Re: Sun compares their compiler tools to GNU tools

[Harvey J. Stein]

Colin Douglas Howell <howell@CS.Stanford.EDU> writes:

 > Sun has published a marketing comparison of their compiler suite,
 > Visual WorkShop C++ 3.0, with the GNU compiler tools.  See the URL:
 > 
 > 	http://www.sun.com/workshop/visual/
 > 
 > The comparison includes performance test results on UltraSPARC.  Of
 > course, this is designed to sell Sun's compiler tools, and is thus
 > rather biased.  Also, their performance tests were done against old
 > versions of gcc; they compare version 4.2 of their C, C++, and FORTRAN
 > compilers with egcs 1.0.1 (and in one case with gcc 2.7.2.2).  So the
 > recent egcs optimizations and SPARC backend improvements are not
 > tested.  (On the other hand, Sun is currently beta-testing version 5.0
 > of their compilers.)
 > 
 > I figured people might want to know what the competition is up to.  Is
 > anyone in a position to comment on the specific tests or do
 > comparisons with the current egcs versions?
 > 
 > Also, this comparison is currently being discussed at slashdot.org,
 > just in case anyone wants to add a voice of reason into that
 > discussion.

When I compared egcs to Sun's C compiler about a year ago, using their
latest compiler at the time, I found my particular application ran
about 30% faster using egcs.  This was on a 300mhz ultrasparc.

The sun article is misleading in a number of ways.  One is (as you pointed
out) that they using gcc 2.7.2.2 in some tests.  Another is that they
use Sun's profiling optimization in other tests but not gcc's
equivalent feature.  A third is the compiler switches they use.  A
forth is the kinds of tests they do.  I posted comments on slashdot
about it.

-- 
Harvey J. Stein
BFM Financial Research
hjstein@bfr.co.il

Re: Sun compares their compiler tools to GNU tools

[Gerald Pfeifer]

On Wed, 4 Nov 1998, Colin Douglas Howell wrote:
> Sun has published a marketing comparison of their compiler suite,
> Visual WorkShop C++ 3.0, with the GNU compiler tools.  See the URL:
> 
> 	http://www.sun.com/workshop/visual/

Shall we add some official reply from EGCS developers and testers) to our
web page?

Gerald

BTW, the page at the URL above is ridiculous, and this is the main page
of that product! No link to more detailed information is evident.

 . What's ``the latest from GNU'' in ``We examined the performance of the
   Sun C and C++ compilers, comparing them with the latest from GNU''?

 . Customer application (C code)  7%-30%
   No source code available? What does the range mean?
 :
 :
-- 
Gerald Pfeifer (Jerry)      Vienna University of Technology
pfeifer@dbai.tuwien.ac.at   http://www.dbai.tuwien.ac.at/~pfeifer/

Re: Sun compares their compiler tools to GNU tools

[N8TM]

In a message dated 11/5/98 11:34:24 AM Pacific Standard Time,
howell@cs.stanford.edu writes:

>  Is
>  anyone in a position to comment on the specific tests or do
>  comparisons with the current egcs versions?
I personally don't find the Sun claims overly impressive or exaggerated, other
than the biased view of which egcs versions are "latest."  The vendors'
compilers show much bigger margins than this on other platforms, but that
doesn't prevent egcs from being seen as a useful option.

Sun compares their compiler tools to GNU tools

[Colin Douglas Howell]

Sun has published a marketing comparison of their compiler suite,
Visual WorkShop C++ 3.0, with the GNU compiler tools.  See the URL:

	http://www.sun.com/workshop/visual/

The comparison includes performance test results on UltraSPARC.  Of
course, this is designed to sell Sun's compiler tools, and is thus
rather biased.  Also, their performance tests were done against old
versions of gcc; they compare version 4.2 of their C, C++, and FORTRAN
compilers with egcs 1.0.1 (and in one case with gcc 2.7.2.2).  So the
recent egcs optimizations and SPARC backend improvements are not
tested.  (On the other hand, Sun is currently beta-testing version 5.0
of their compilers.)

I figured people might want to know what the competition is up to.  Is
anyone in a position to comment on the specific tests or do
comparisons with the current egcs versions?

Also, this comparison is currently being discussed at slashdot.org,
just in case anyone wants to add a voice of reason into that
discussion.

-- 
Colin Douglas Howell                    Systems Administrator
e-mail:  howell@cs.stanford.edu         Computer Facilities Group
office:  (650) 723-2491                 Computer Science Department
                                        Stanford University