Optimization

Optimization

[Brian D. McGrew]

Good morning,

We have a software tree that we compile for Solaris Sparc and Linux 
X86.  There are several versions this software that we support, 4.5.3 
and 4.7 are still our current versions.  We compile up 4.5.3 for 
Solaris and Linux with gcc-295.  4.7 is compiled with gcc-32.

The problem that we're seeing is this:  4.5.3 and 4.7 run almost the 
same speed on Linux.  However, on Solaris Sparc 4.5.3 runs about 10x 
faster than the Linux version.  All the compile lines and complier 
options are the same (and quoted at the bottom of this message).

Can someone shed some light on to why me might be seeing this problem?  
The compile line that I pasted is from our Linux build.  The only 
differences between Solaris and Linux is the -mtune=ultrasparc and 
-mcpu-v8 options on Sparc.

Any ideas???


-brian

Brian D. McGrew		{ brian@doubledimension.com || 
pacemakertaker@yahoo.com }
--
 > YOU!  Off my planet!

Begin forwarded message:

> From: "Brian D. McGrew" <brian@ultra-trix.visionpro.com>
> Date: April 27, 2004 6:53:24 AM PDT
> To: brian@ultra-trix.visionpro.com, brian@visionpro.com, 
> brian@doubledimension.com
> Subject: GCC
>
> /usr/local/gnu/gcc-332/bin/g++ -c -fPIC -g -O0 -DALGODEBUG  
> -D__EXTENSIONS__ -I/
> usr/X11R6/include -DHAS_SYS_IOCTL_H -UHAS_SYS_FILIO_H -DHAS_STDINT_H 
> -DHAS_STRIN
> G_H -DHAS_ASM_IOCTL_H -DHAS_SYS_STROPTS_H -DHAS_TIME_H 
> -DHAS_SYS_WAIT_H -DUNIXHO
> ST -DTHREAD -D_PTHREADS -D_REENTRANT -D_POSIX_C_SOURCE=199506L  -pipe 
> -D_G_NO_EX
> TERN_TEMPLATES -funroll-loops -finline-limit=2750 -pipe -ansi 
> -pedantic -Wimplic
> it -Wconversion -Wsign-compare -Wno-long-long -Wreorder 
> -Woverloaded-virtual -Ws
> ign-promo -DIN_IS_LIBRARY -DDPC_DEBUG -DINTERNAL_UNDEFINED_RELEASE 
> -DX86 -DLINUX
>  -D_BSD_SOURCE -D_XOPEN_SOURCE   -I../.././include 
> -I../.././include/libinc -I..
>  /.././include/localinc -DXMSTRINGDEFINES  SharedPool.c -o 
> SharedPool.sh
>

Re: Optimization

[Eljay Love-Jensen]

Hi Brian,

I am unclear:  what's the problem?

My first inkling is that your Solaris SPARC platform is x10 faster than 
your Linux x86 platform.

My second inkling is...  how come you are not performance comparing 
optimized code (-O2 or -O3)?  Unoptimized code can be notoriously ... er, 
umm ... unoptimized.

--Eljay

Re: Optimization

[Brian D. McGrew]

Doh!  I goofed and posted the wrong trace file ... The exceprt from the 
file that I posted was from our internal builds, not the 'release' 
tree.  The release tree for Sparc and Linux, 4.5.3 and 4.7 are both 
compiled -O2 -g.  We have so many problems with our code that we have 
to send it out with the -g flag turned on.

Upon profiling the code, I've found that:

1)  4.5.3 built with gcc-295 and 4.7 built with gcc-32 on Linux run 
pretty close to the same speed when compiled -O2 -g.  The differences 
in speed are neglegable, within a few miliseconds.

2)  4.5.3 built with gcc-295 and 4.7 built with gcc-32 on Solaris 
exhibit huge performance differences.  4.7 is about 10x slower than 
4.5.3.  Again, in the release tree compiled with -O2 -g.  I do not see 
this problem on Linux, so I'm inclined to believe it is not something 
in my code.  Also,  the Linux tree will compile up -O2 -g 
-finline-limit=2750 in about 90 minutes.  The Solaris tree takes about 
16 hours for the same build.

I'll post the correct compile line at the bottom of this message.

Thanks for the help!

---[ Compile line from 4.7-SPARC using gcc-32 ]---
/opt/mvplocal/gcc-31/bin/g++  -c -fPIC -O2 -g -finline-limit=2750  
-mcpu=v8 -mtune=ultrasparc -D__EXTENSIONS__  -DHAS_SYS_IOCTL_H 
-DHAS_SYS_FILIO_H -DHAS_STRINGS_H -DHAS_SYS_IOCCOM_H -DHAS_SYS_TIME_H 
-DHAS_SYS_WAIT_H -DUNIXHOST -DTHREAD -D_PTHREADS -D_REENTRANT 
-D_POSIX_C_SOURCE=199506L  -D_G_NO_EXTERN_TEMPLATES -funroll-loops 
-pipe -ansi -pedantic -Wimplicit -Wconversion -Wsign-compare 
-Wno-long-long -Wreorder -Woverloaded-virtual -Wsign-promo 
-Wredundant-decls -DIN_IS_LIBRARY -DDPC_DEBUG 
-DNO_ARRAY_BOUNDS_CHECKING -DSVR4   -I../.././include 
-I../.././include/libinc -I../.././include/localinc -DXMSTRINGDEFINES  
SharedPool.c -o SharedPool.sh

---[ Compile line from 4.5.3-SPARC using gcc-295 ]---
/opt/mvplocal/bin/g++ -c -fPIC -O2 -g -mcpu=v8 -mtune=ultrasparc -DSVR4 
-D_POSIX_C_SOURCE=199506L -DTHREAD -D_REENTRANT -D__EXTENSIONS__ 
-I/usr/dt/include -I/usr/openwin/include -DUNIXHOST -D_PTHREADS   -pipe 
-D_G_NO_EXTERN_TEMPLATES -funroll-loops -ansi -pedantic -Wimplicit 
-Wconversion -Wsign-compare -Wno-long-long -Wreorder 
-Woverloaded-virtual -Wsign-promo -DNEWRESULTS -DIN_IS_LIBRARY 
-DDPC_DEBUG   -I../.././include -I../.././include/libinc 
-I../.././include/localinc RemoteCommseg.c -o RemoteCommseg.sh

-brian

Brian D. McGrew		{ brian@doubledimension.com || 
pacemakertaker@yahoo.com }
--
 > YOU!  Off my planet!
On Apr 27, 2004, at 7:06 AM, Eljay Love-Jensen wrote:

> Hi Brian,
>
> I am unclear:  what's the problem?
>
> My first inkling is that your Solaris SPARC platform is x10 faster 
> than your Linux x86 platform.
>
> My second inkling is...  how come you are not performance comparing 
> optimized code (-O2 or -O3)?  Unoptimized code can be notoriously ... 
> er, umm ... unoptimized.
>
> --Eljay
>

Re: Optimization

[llewelly]

Brian D. McGrew <brian@visionpro.com> writes:

> Good morning,
> 
> We have a software tree that we compile for Solaris Sparc and Linux
> X86.  There are several versions this software that we support, 4.5.3
> and 4.7 are still our current versions.  We compile up 4.5.3 for
> Solaris and Linux with gcc-295.  4.7 is compiled with gcc-32.
> 
> The problem that we're seeing is this:  4.5.3 and 4.7 run almost the
> same speed on Linux.  However, on Solaris Sparc 4.5.3 runs about 10x
> faster than the Linux version.

Could you rephrase this? I can't tell whether you mean '4.5.3 on
    solaris is 10 times faster than 4.7 on solaris', or '4.5.3 on
    solaris is 10 times faster than 4.5.3 on linux'.


> All the compile lines and complier
> options are the same (and quoted at the bottom of this message).
> 
> Can someone shed some light on to why me might be seeing this
> problem?

Maybe you should try a profiler?

> The compile line that I pasted is from our Linux build.  The only
> differences between Solaris and Linux is the -mtune=ultrasparc and
> -mcpu-v8 options on Sparc.
> 
> Any ideas???
> 
> 
> -brian
> 
> Brian D. McGrew		{ brian@doubledimension.com ||
> pacemakertaker@yahoo.com }
> --
>  > YOU!  Off my planet!
> 
> Begin forwarded message:
> 
> > From: "Brian D. McGrew" <brian@ultra-trix.visionpro.com>
> > Date: April 27, 2004 6:53:24 AM PDT
> > To: brian@ultra-trix.visionpro.com, brian@visionpro.com,
> > brian@doubledimension.com
> > Subject: GCC
> >
> > /usr/local/gnu/gcc-332/bin/g++ -c -fPIC -g -O0 -DALGODEBUG
                                               ^^^ If performance is
                                               important this could be
                                               a problem.
                                               

Re: Optimization

[Brian D. McGrew]

Yes, 4.5.3-SOLARIS is 10x faster than 4.7 on Solaris.  4.5.3-LINUX and 
4.7-LINUX run about the same.

As for the -O0 flag, I posted the wrong trace file.  In the release 
trees, we use -O2 -g -finline-limit=2750 on all versions, all 
platforms.  My bad.

-brian

Brian D. McGrew		{ brian@doubledimension.com || 
pacemakertaker@yahoo.com }
--
 > YOU!  Off my planet!
On Apr 27, 2004, at 7:45 AM, llewelly@xmission.com wrote:

> Brian D. McGrew <brian@visionpro.com> writes:
>
>> Good morning,
>>
>> We have a software tree that we compile for Solaris Sparc and Linux
>> X86.  There are several versions this software that we support, 4.5.3
>> and 4.7 are still our current versions.  We compile up 4.5.3 for
>> Solaris and Linux with gcc-295.  4.7 is compiled with gcc-32.
>>
>> The problem that we're seeing is this:  4.5.3 and 4.7 run almost the
>> same speed on Linux.  However, on Solaris Sparc 4.5.3 runs about 10x
>> faster than the Linux version.
>
> Could you rephrase this? I can't tell whether you mean '4.5.3 on
>     solaris is 10 times faster than 4.7 on solaris', or '4.5.3 on
>     solaris is 10 times faster than 4.5.3 on linux'.
>
>
>> All the compile lines and complier
>> options are the same (and quoted at the bottom of this message).
>>
>> Can someone shed some light on to why me might be seeing this
>> problem?
>
> Maybe you should try a profiler?
>
>> The compile line that I pasted is from our Linux build.  The only
>> differences between Solaris and Linux is the -mtune=ultrasparc and
>> -mcpu-v8 options on Sparc.
>>
>> Any ideas???
>>
>>
>> -brian
>>
>> Brian D. McGrew		{ brian@doubledimension.com ||
>> pacemakertaker@yahoo.com }
>> --
>>> YOU!  Off my planet!
>>
>> Begin forwarded message:
>>
>>> From: "Brian D. McGrew" <brian@ultra-trix.visionpro.com>
>>> Date: April 27, 2004 6:53:24 AM PDT
>>> To: brian@ultra-trix.visionpro.com, brian@visionpro.com,
>>> brian@doubledimension.com
>>> Subject: GCC
>>>
>>> /usr/local/gnu/gcc-332/bin/g++ -c -fPIC -g -O0 -DALGODEBUG
>                                                ^^^ If performance is
>                                                important this could be
>                                                a problem.
>
>

Re: Optimization

[Ian Lance Taylor]

Brian D.McGrew <brian@visionpro.com> writes:

> As for the -O0 flag, I posted the wrong trace file.  In the release
> trees, we use -O2 -g -finline-limit=2750 on all versions, all
> platforms.  My bad.

I doubt this is your problem, but -finline-limit will have a different
effect on i386 and SPARC, since the SPARC will normally require more
instructions.  Although -finline-limit is expressed in terms of
pseudo-instructions, gcc pseudo-instructions are different on i386 and
SPARC.  It would normally be appropriate to use a larger
-finline-limit on SPARC.

As I say, I doubt this is your problem.  I have no idea what your
problem is.  It's fairly unlikely that anybody can tell you what is
wrong here.

As was already suggested, try using a profiler to see where the hot
spots are in your slow version.  Perhaps gcc is egregiously
miscompiling something on SPARC.

Ian

Re: Optimization

[Brian D. McGrew]

so what flag can I give gcc to tell it to inline everything, or as much 
as possible???  Our code already takes 16 hours to build on sparc, 
using -O2 -g -finline-limit=2750.

I've profiled the code but I'm not exactly sure what to do with it now 
that I've got it.  Someone sail something about gproff but I don't know 
exactly what I'm looking for.

-brian

Brian D. McGrew		{ brian@doubledimension.com || 
pacemakertaker@yahoo.com }
--
 > YOU!  Off my planet!
On Apr 27, 2004, at 8:02 AM, Ian Lance Taylor wrote:

> Brian D.McGrew <brian@visionpro.com> writes:
>
>> As for the -O0 flag, I posted the wrong trace file.  In the release
>> trees, we use -O2 -g -finline-limit=2750 on all versions, all
>> platforms.  My bad.
>
> I doubt this is your problem, but -finline-limit will have a different
> effect on i386 and SPARC, since the SPARC will normally require more
> instructions.  Although -finline-limit is expressed in terms of
> pseudo-instructions, gcc pseudo-instructions are different on i386 and
> SPARC.  It would normally be appropriate to use a larger
> -finline-limit on SPARC.
>
> As I say, I doubt this is your problem.  I have no idea what your
> problem is.  It's fairly unlikely that anybody can tell you what is
> wrong here.
>
> As was already suggested, try using a profiler to see where the hot
> spots are in your slow version.  Perhaps gcc is egregiously
> miscompiling something on SPARC.
>
> Ian
>

Re: Optimization

[llewelly]

Brian D. McGrew <brian@visionpro.com> writes:

> Yes, 4.5.3-SOLARIS is 10x faster than 4.7 on Solaris.  4.5.3-LINUX and
> 4.7-LINUX run about the same.
[snip]

This is a shot in the dark, but if you do a lot of output to cout and
    friends, try:

    std::ios_base::sync_with_stdio(false);

and see http://gcc.gnu.org/onlinedocs/libstdc++/27_io/howto.html#8 on
    the same topic.

Re: Optimization

[Ian Lance Taylor]

Brian D.McGrew <brian@visionpro.com> writes:

> so what flag can I give gcc to tell it to inline everything, or as
> much as possible???  Our code already takes 16 hours to build on
> sparc, using -O2 -g -finline-limit=2750.

I suppose you could use a larger -finline-limit value.

> I've profiled the code but I'm not exactly sure what to do with it now
> that I've got it.  Someone sail something about gproff but I don't
> know exactly what I'm looking for.

If you haven't run gprof, or some other profiling program, then you
haven't yet profiled the code.  I'm sure there is documentation out
there which can help.  I'm sorry, but this is not something which I am
prepared to walk you through.

Ian

Re: Optimization.

[Sriharsha]

Mikael Johansson wrote:

> Hello.
> When optimizing a program using Gcc is it common to use different 
> compiler flags
> for the executive than for the other parts of the program.

I dont think you cannot optimize an executable (if that is what you meant).

>
> Thanks beforehand.
>
> Mikael
>
> _________________________________________________________________
> Hitta rätt på nätet med MSN Sök http://search.msn.se/
>
>

-- 
 *****************************
 * Sriharsha Vedurmudi			
 * Software Engineer		
 * 
 * Redpine Signals Inc.	
 * Gate #395, Plot 87,88			
 * Sagar Society, Road #2, 
 * Banjara Hills,		
 * Hyderabad - 500 034			
 * www.redpinesignals.com	
 *							
 * +91-40-23559911  (Office)
 * +91-98491-33133  (Mobile)
 *****************************

Optimization.

[Mikael Johansson]

Hello.
When optimizing a program using Gcc is it common to use different compiler 
flags
for the executive than for the other parts of the program.

Thanks beforehand.

Mikael

_________________________________________________________________
Hitta rätt på nätet med MSN Sök http://search.msn.se/

Re: optimization

[Paolo Carlini]

> What do you think about that?

Hi Andrey,

indeed, I agree that, in principle, a compiler should be able to optimize this.
Perhaps soon GCC will do better than now: all the work being done on the AST
branch (http://gcc.gnu.org/projects/ast-optimizer.html) and cfg/mid level RTL
branch (http://gcc.gnu.org/projects/cfg.html) should lead to new optmization
passes, working at abstraction levels different from those currently involved
(basically, at the tree-level today is done only inlining and everything else at
the low RTL level)

If you want to understand more of such issues I suggest you to begin studying at
least a couple of books:

S.S. Muchnick
Advanced Compiler Desing and Implementation
Morgan Kaufmann, 1997.

K. Czarnecki and U.W. Eisenecker
Generative Programming
Addison Wesley, 2000.

The former deals with advanced compiler optimizations, the latter with "static
metaprogramming" (among other things), which can allow to write a templatized
maxRpkLen() equivalent which *today's*  compiler will evaluate to a constant at
compile time, as part of the standard template instantiation mechanism.

From a practical point of view, it would be nice if you could track how new
versions of GCC will optimize the testcase you presented, for instance posting
and discussing the corresponding assembler produced.

Cheers,
Paolo.

optimization

[Andrey R. Urazov]

[-- Attachment #1: Type: text/plain, Size: 2216 bytes --]

Hello everybody,

I wonder if this is possible to implement an optimizer that would
produce just one statement returning constant for the following fragment
of code:

---
#include <cstddef>

inline unsigned strlen(const char* s)
{
    unsigned len = 0;
    while (*s++) len++;
    return len;
}

struct RPK {
    const char* rpk;
    const char* table;
};

// referenced primary keys and corresponding tables
const RPK rpks[] = {
    {"section_id", "section"},
    {"group_id", "group"},
    {"person_id", "person"},
    {"professional_id", "professional"},
    {"coach_id", "coach"},
    {"trip_id", "trip"},
    {"trip_category_id", "trip_category"},
    {"route_id", "route"},
    {"place_id", "place"},
    {"qualification", "qualification"},
    {"position_id", "position"}
};
const size_t n_rpks = sizeof(rpks)/sizeof(rpks[0]);

inline unsigned maxRpkLen()
{
    unsigned max = strlen(rpks[0].rpk);
    for (size_t i = 1; i<n_rpks; i++) {
        unsigned len = strlen(rpks[i].rpk);
        if (max<len) max = len;
    }
    return max;
}

int main()
{
    return maxRpkLen();
}
---

It seems to me that there is no problem for an optimizer to reveal that
at the moment of inlining of maxRpkLen() corresponding code does not
depend on variables that can change values and calculate the value
during compilation so that a constant value be inserted in the
application code.

Actually this does not happen with whatever level of optimization in
GCC. I think that it makes sense to provide a user with the possibility
of code fragment execution during compilation stage since that would
solve the problem. Of course, if there was such an option, it should be
disabled by default and only used by the programmer at his/her own risk
because the code corresponding to "constant fragment" can contain
indefinite loop or just run for a long time.

What do you think about that?

                        Yours sincerely, Andrey Urazov
-- 
We were so poor we couldn't afford a watchdog.  If we heard a noise at night,
we'd bark ourselves.
		-- Crazy Jimmy
--
Tuesday, January 29, 2002, 16:49:04 +0600 - Andrey R. Urazov (mailto:coola@ngs.ru)


[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: Optimization

[Martin v. Loewis]

> Can anyone briefly explain to me what the gcc does when compiling with the
> -O3 flag for maximal optimization?

Please see (gcc)Optimize Options in the manual.

> What increase in performance can I expect?

It depends on your application. Most likely, you won't notice a
difference.

Regards,
Martin

Optimization

[Andersson, Per-Ola]

Can anyone briefly explain to me what the gcc does when compiling with the
-O3 flag for maximal optimization?
What increase in performance can I expect?

Thnx
// Per-Ola