Need some clarification about optimization flags, what "exactly" does -O1 do?

Need some clarification about optimization flags, what "exactly" does -O1 do?

[Andrzej Giniewicz]

Hi - it's my first time on that list, I hope I choose good one as gcc
have really quite number of different mailing lists :)

lately I hit very strange behaviour, i.e. the app I wanted to build
works when I build it with -O0 but crash at -O1, I wanted to eliminate
flags that work as I usually do, by doing -O0 and adding all flags
that are on -O1 and aren't on -O0, I got their list using standard
recommended by manual way:

g++ -c -Q -O0 --help=optimize > 0.flags
g++ -c -Q -O1 --help=optimize > 1.flags
diff 0.flags 1.flags | grep enabled

the problem is that I was able to add all flags returned by above
commands! Is there some secret things that -O1 do instead of only
lighting those -f*** flags listened by above commands? If above
returned me:

-fcprop-registers -fdefer-pop -fguess-branch-probability
-fif-conversion -fif-conversion2 -fipa-pure-const -fipa-reference
-fmerge-constants -fomit-frame-pointer -fsplit-wide-types -ftree-ccp
-ftree-ch -ftree-copy-prop -ftree-copyrename -ftree-dce
-ftree-dominator-opts -ftree-dse -ftree-fre -ftree-salias -ftree-sink
-ftree-sra -ftree-ter -funit-at-a-time

is there any difference / reason why in my case:

-O1

fails but:

-O0 -fcprop-registers -fdefer-pop -fguess-branch-probability
-fif-conversion -fif-conversion2 -fipa-pure-const -fipa-reference
-fmerge-constants -fomit-frame-pointer -fsplit-wide-types -ftree-ccp
-ftree-ch -ftree-copy-prop -ftree-copyrename -ftree-dce
-ftree-dominator-opts -ftree-dse -ftree-fre -ftree-salias -ftree-sink
-ftree-sra -ftree-ter -funit-at-a-time

works?... I build with athlon-xp architecture but same happens when I
do i686 (with tune=generic)... it's of course Linux, and I use GCC
4.3.2

cheers,
Andrzej.

btw, the problem I have is during run of OGRE3D engine so quite huge
app, I managed to track the problem up to this place:

            HardwareIndexBuffer::IndexType indexType =
mCurrentSection->get32BitIndices()? HardwareIndexBuffer::IT_32BIT :
HardwareIndexBuffer::IT_16BIT;

there IndexType is enum for only those two values that in any other
context are 0/1, and get32BitIndices is function returning bool, this
makes it kind of:

            int x = boolValue ? 0:1;

the problem is that any diagnostic, stwich/case/if for that value
equal HardwareIndexBuffer::IT_32BIT or HardwareIndexBuffer::IT_16BIT
fail, simple cout on it says it's 104... always, every run - every
build with anything more than -O0... it's working on -O0 thought and
-O0 with above flags... anyway for now question is rather about what
more than flags returned by those 3 mentioned lines of shell does -O1
do because I can just go on with disabling one feature and leaving
rest in place

Re: Need some clarification about optimization flags, what "exactly" does -O1 do?

[Brian Dessent]

Andrzej Giniewicz wrote:

> g++ -c -Q -O0 --help=optimize > 0.flags
> g++ -c -Q -O1 --help=optimize > 1.flags
> diff 0.flags 1.flags | grep enabled
> 
> the problem is that I was able to add all flags returned by above
> commands! Is there some secret things that -O1 do instead of only
> lighting those -f*** flags listened by above commands? If above
> returned me:

When you specify -O0 (or no -O at all) you are telling the compiler to
perform no optimization, and therefore all those -f options become
no-ops -- the code whose behavior would be modified by their presence
has been bypassed entirely.

Moreover, as the manual explains, not all optimizations have a
corresponding -f flag, some are controlled directly by the -O level.  So
you absolutely cannot treat -O as being composed of a bunch of -f
options.

> the problem is that any diagnostic, stwich/case/if for that value
> equal HardwareIndexBuffer::IT_32BIT or HardwareIndexBuffer::IT_16BIT
> fail, simple cout on it says it's 104... always, every run - every
> build with anything more than -O0... it's working on -O0 thought and
> -O0 with above flags... anyway for now question is rather about what
> more than flags returned by those 3 mentioned lines of shell does -O1
> do because I can just go on with disabling one feature and leaving
> rest in place

The only way that anyone's going to be able to offer any concrete help
is with a testcase that reproduces the problem, something that is
standalone and ideally reduced as much as possible.  There is a small
chance that there is a genuine compiler bug, but a larger chance that
some aspect of your code is invoking undefined behavior which results in
a program that exhibits seemingly irrational and completely nonsensical
behavior.  For instance recent versions of gcc can be very sensitive to
aliasing violations and signed integer overflow; however that level of
strictness is not enabled by default at -O1 so I suspect that it's
something else.

Brian

Re: Need some clarification about optimization flags, what "exactly" does -O1 do?

[Andrew Haley]

Andrzej Giniewicz wrote:
> Hi - it's my first time on that list, I hope I choose good one as gcc
> have really quite number of different mailing lists :)
> 
> lately I hit very strange behaviour, i.e. the app I wanted to build
> works when I build it with -O0 but crash at -O1, I wanted to eliminate
> flags that work as I usually do, by doing -O0 and adding all flags
> that are on -O1 and aren't on -O0, I got their list using standard
> recommended by manual way:
> 
> g++ -c -Q -O0 --help=optimize > 0.flags
> g++ -c -Q -O1 --help=optimize > 1.flags
> diff 0.flags 1.flags | grep enabled
> 
> the problem is that I was able to add all flags returned by above
> commands! Is there some secret things that -O1 do instead of only
> lighting those -f*** flags listened by above commands? If above
> returned me:
> 
> -fcprop-registers -fdefer-pop -fguess-branch-probability
> -fif-conversion -fif-conversion2 -fipa-pure-const -fipa-reference
> -fmerge-constants -fomit-frame-pointer -fsplit-wide-types -ftree-ccp
> -ftree-ch -ftree-copy-prop -ftree-copyrename -ftree-dce
> -ftree-dominator-opts -ftree-dse -ftree-fre -ftree-salias -ftree-sink
> -ftree-sra -ftree-ter -funit-at-a-time
> 
> is there any difference / reason why in my case:
> 
> -O1
> 
> fails but:
> 
> -O0 -fcprop-registers -fdefer-pop -fguess-branch-probability
> -fif-conversion -fif-conversion2 -fipa-pure-const -fipa-reference
> -fmerge-constants -fomit-frame-pointer -fsplit-wide-types -ftree-ccp
> -ftree-ch -ftree-copy-prop -ftree-copyrename -ftree-dce
> -ftree-dominator-opts -ftree-dse -ftree-fre -ftree-salias -ftree-sink
> -ftree-sra -ftree-ter -funit-at-a-time
> 
> works?... 

With -O0 you get no optimization, regardless of the other options.

> I build with athlon-xp architecture but same happens when I
> do i686 (with tune=generic)... it's of course Linux, and I use GCC
> 4.3.2

> btw, the problem I have is during run of OGRE3D engine so quite huge
> app, I managed to track the problem up to this place:
> 
>             HardwareIndexBuffer::IndexType indexType =
> mCurrentSection->get32BitIndices()? HardwareIndexBuffer::IT_32BIT :
> HardwareIndexBuffer::IT_16BIT;
> 
> there IndexType is enum for only those two values that in any other
> context are 0/1, and get32BitIndices is function returning bool, this
> makes it kind of:
> 
>             int x = boolValue ? 0:1;

What does the declaration of IndexType look like?

> the problem is that any diagnostic, stwich/case/if for that value
> equal HardwareIndexBuffer::IT_32BIT or HardwareIndexBuffer::IT_16BIT
> fail, simple cout on it says it's 104... always, every run - every
> build with anything more than -O0... it's working on -O0 thought and
> -O0 with above flags... anyway for now question is rather about what
> more than flags returned by those 3 mentioned lines of shell does -O1
> do because I can just go on with disabling one feature and leaving
> rest in place

The bug is probably in your app, not in gcc.
I think you'd be much better off fixing the bug than trying to
discover which optimization pass it breaks.

Andrew.

Re: Need some clarification about optimization flags, what "exactly" does -O1 do?

[Andrzej Giniewicz]

Hi,

> What does the declaration of IndexType look like?

nothing special:

       enum IndexType {
         IT_16BIT,
         IT_32BIT
       };

> The bug is probably in your app, not in gcc.
> I think you'd be much better off fixing the bug than trying to
> discover which optimization pass it breaks.

not quite my app just app I try to build, i.e. developer seems to not
have gcc 4.3 yet and I am just trying to get it on :) To be more
precise - the value that is defined as:

... ? HardwareIndexBuffer::IT_32BIT : HardwareIndexBuffer::IT_16BIT

is then checked with:

       switch (mIndexType)
       {
       case IT_16BIT:
           mIndexSize = sizeof(unsigned short);
           break;
       case IT_32BIT:
           mIndexSize = sizeof(unsigned int);
           break;
       }

but it don't trigger any of those in anything more than -O0 (if I add
default with cerr<< it is triggered) and works on -O0... I didn't
thought such funny thing can be caused by some bug where code is quite
easy, simple switch on enumerated value that is set to one of two and
not used/changed anywhere else...

cheers,
Andrzej.

Re: Need some clarification about optimization flags, what "exactly" does -O1 do?

[Andrew Haley]

Andrzej Giniewicz wrote:

>> What does the declaration of IndexType look like?
> 
> nothing special:
> 
>         enum IndexType {
>           IT_16BIT,
>           IT_32BIT
>         };
> 
>> The bug is probably in your app, not in gcc.
>> I think you'd be much better off fixing the bug than trying to
>> discover which optimization pass it breaks.
> 
> not quite my app just app I try to build, i.e. developer seems to not
> have gcc 4.3 yet and I am just trying to get it on :) To be more
> precise - the value that is defined as:
> 
> ... ? HardwareIndexBuffer::IT_32BIT : HardwareIndexBuffer::IT_16BIT
> 
> is then checked with:
> 
>         switch (mIndexType)
>         {
>         case IT_16BIT:
>             mIndexSize = sizeof(unsigned short);
>             break;
>         case IT_32BIT:
>             mIndexSize = sizeof(unsigned int);
>             break;
>         }
> 
> but it don't trigger any of those in anything more than -O0 (if I add
> default with cerr<< it is triggered) and works on -O0... I didn't
> thought such funny thing can be caused by some bug where code is quite
> easy, simple switch on enumerated value that is set to one of two and
> not used/changed anywhere else...

Yeah, but that bug almost certainly isn't in the code you're looking at.

I've recently seen a bug that was triggered by -O in an enum where the
declaration was:

   enum Cell {
    Cell_0
   };

and gcc quite correctly assumed that a Cell could only have one value.

I'm not saying I'm absolutely certain this isn't a bug in gcc.  It's just
that it probably isn't.

Andrew.

Re: Need some clarification about optimization flags, what "exactly" does -O1 do?

[Andrzej Giniewicz]

Hi,

> Yeah, but that bug almost certainly isn't in the code you're looking at.
>
> I've recently seen a bug that was triggered by -O in an enum where the
> declaration was:
>
>   enum Cell {
>    Cell_0
>   };
>
> and gcc quite correctly assumed that a Cell could only have one value.
>
> I'm not saying I'm absolutely certain this isn't a bug in gcc.  It's just
> that it probably isn't.

interesting, seems like something similar, any hints how to start
tracking real issue?

thanks in advance for your help,
Andrzej.

btw... I found the flag... - well, kind of... -O0 and -O0 -ftree-dce
and -O1 -fno-tree-dce all works, but -O1 as stated before don't... so
I can workaround the issue just for now with -fno-tree-dce until real
one is found - this gives me some more time :)

Re: Need some clarification about optimization flags, what "exactly" does -O1 do?

[Andrew Haley]

Andrzej Giniewicz wrote:

>> Yeah, but that bug almost certainly isn't in the code you're looking at.
>>
>> I've recently seen a bug that was triggered by -O in an enum where the
>> declaration was:
>>
>>   enum Cell {
>>    Cell_0
>>   };
>>
>> and gcc quite correctly assumed that a Cell could only have one value.
>>
>> I'm not saying I'm absolutely certain this isn't a bug in gcc.  It's just
>> that it probably isn't.
> 
> interesting, seems like something similar, any hints how to start
> tracking real issue?
> 
> thanks in advance for your help,
> Andrzej.
> 
> btw... I found the flag... - well, kind of... -O0 and -O0 -ftree-dce
> and -O1 -fno-tree-dce all works, but -O1 as stated before don't... so
> I can workaround the issue just for now with -fno-tree-dce until real
> one is found - this gives me some more time :)

That's interesting.  tree-dce is Dead Code Elimination on trees, where
gcc removes code that can't possibly be reached.  It's possible that
there's a bug in DCE, but I bet it's something like the enum bug above.
If you're interested in finding the real bug I can tell you how to read
the compiler dumps.

Andrew.

Re: Need some clarification about optimization flags, what "exactly" does -O1 do?

[Michael Meissner]

On Thu, Nov 27, 2008 at 11:21:38AM +0100, Andrzej Giniewicz wrote:
> Hi - it's my first time on that list, I hope I choose good one as gcc
> have really quite number of different mailing lists :)
> 
> lately I hit very strange behaviour, i.e. the app I wanted to build
> works when I build it with -O0 but crash at -O1, I wanted to eliminate
> flags that work as I usually do, by doing -O0 and adding all flags
> that are on -O1 and aren't on -O0, I got their list using standard
> recommended by manual way:
> 
> g++ -c -Q -O0 --help=optimize > 0.flags
> g++ -c -Q -O1 --help=optimize > 1.flags
> diff 0.flags 1.flags | grep enabled

Just for reference, there are optimizations that are done based on the
optimization level, and not on particular -f switches.

-- 
Michael Meissner, IBM
4 Technology Place Drive, MS 2203A, Westford, MA, 01886, USA
meissner@linux.vnet.ibm.com

Re: Need some clarification about optimization flags, what "exactly" does -O1 do?

[Andrzej Giniewicz]

Hi,

> That's interesting.  tree-dce is Dead Code Elimination on trees, where
> gcc removes code that can't possibly be reached.  It's possible that
> there's a bug in DCE, but I bet it's something like the enum bug above.
> If you're interested in finding the real bug I can tell you how to read
> the compiler dumps.

that would be sweet, finding real bugs is always better than workaround :)

cheers,
Andrzej.

Re: Need some clarification about optimization flags, what "exactly" does -O1 do?

[Diego Novillo]

On Fri, Nov 28, 2008 at 05:58, Andrew Haley <aph@redhat.com> wrote:

> That's interesting.  tree-dce is Dead Code Elimination on trees, where
> gcc removes code that can't possibly be reached.

Strictly speaking, DCE removes code that can't possibly affect the
output of the program.  Unreachable code is removed by the CFG cleanup
helper.  Apologies for nitpicking, but this distinction is handled in
two different spots in the compiler, so it may affect what files/dumps
you need to look at.


Diego.

Re: Need some clarification about optimization flags, what "exactly" does -O1 do?

[Andrew Haley]

Andrzej Giniewicz wrote:

>> That's interesting.  tree-dce is Dead Code Elimination on trees, where
>> gcc removes code that can't possibly be reached.  It's possible that
>> there's a bug in DCE, but I bet it's something like the enum bug above.
>> If you're interested in finding the real bug I can tell you how to read
>> the compiler dumps.
> 
> that would be sweet, finding real bugs is always better than workaround :)

Compile with -fdump-tree-all.  You'll see dozens of dump files like

hello.c.037t.addressables2
hello.c.036t.dce1
hello.c.035t.mergephi1
hello.c.034t.copyprop1
hello.c.033t.esra
hello.c.032t.sdse1
hello.c.031t.addressables1
hello.c.030t.forwprop1

Diff the pass before dce and the post-dce dump and you'll see what dce did.
There are several dce passes.

Also, you can diff the final optimized code with and without dce.

Andrew.

Re: Need some clarification about optimization flags, what "exactly" does -O1 do?

[Andrew Haley]

Diego Novillo wrote:
> On Fri, Nov 28, 2008 at 05:58, Andrew Haley <aph@redhat.com> wrote:
> 
>> That's interesting.  tree-dce is Dead Code Elimination on trees, where
>> gcc removes code that can't possibly be reached.
> 
> Strictly speaking, DCE removes code that can't possibly affect the
> output of the program.  Unreachable code is removed by the CFG cleanup
> helper.  Apologies for nitpicking, but this distinction is handled in
> two different spots in the compiler, so it may affect what files/dumps
> you need to look at.

Thanks for the nitpick -- input always appreciated! -- but we already
know this is dce, having discovered that disabling dce fixes the problem,
so the dce dumps are what we need to inspect.  I guess it's possible,
though, that a dce change causes a later pass to do something different.

Andrew.

Re: Need some clarification about optimization flags, what "exactly" does -O1 do?

[Andrzej Giniewicz]

Hi,

> Compile with -fdump-tree-all.  You'll see dozens of dump files like
>
> hello.c.037t.addressables2
> hello.c.036t.dce1
> hello.c.035t.mergephi1
> hello.c.034t.copyprop1
> hello.c.033t.esra
> hello.c.032t.sdse1
> hello.c.031t.addressables1
> hello.c.030t.forwprop1
>
> Diff the pass before dce and the post-dce dump and you'll see what dce did.
> There are several dce passes.
>
> Also, you can diff the final optimized code with and without dce.

I did as you said, was trying to track it step by step at dce, but
finally checked how optimized version look... the code:

HardwareIndexBuffer::IndexType indexType =
mCurrentSection->get32BitIndices()?HardwareIndexBuffer::IT_32BIT :
HardwareIndexBuffer::IT_16BIT;

where IndexType is:

       enum IndexType {
         IT_16BIT,
         IT_32BIT
       };

and mCurrentSection is pointer field in current object of which
get32BitIndices() return m32BitIndices field, was optimized to:

indexType = (IndexType) this->mCurrentSection->m32BitIndices;

in above version the version without DCE had this in form of:

if (this->mCurrentSection->m32BitIndices) {
   indexType = 1;
} else {
  indexType = 0;
}

or exactly:

  D.288376 = this->mCurrentSection->m32BitIndices;
  if (D.288376)
    goto <bb 29>;
  else
    goto <bb 30>;

<bb 29>:
  indexType.4866 = 1;
  indexType = 1;
  goto <bb 31>;

<bb 30>:
  indexType.4867 = 0;
  indexType = 0;

I have not much experience, but it seems like it's not that bad...
maybe I should check how m32BitIndicies is initialized? (as reminder -
in optimized version indexType end to be 104 by some reason...)

cheers,
Andrzej.

Re: Need some clarification about optimization flags, what "exactly" does -O1 do?

[Andrzej Giniewicz]

Hi,

> I have not much experience, but it seems like it's not that bad...
> maybe I should check how m32BitIndicies is initialized?

and I did it - and it turned out there was case when it was
uninitialized... looking for this without all your help would be
really hard :)... the issue is solved and you were right - it's not
gcc issue :)

thanks again for great new technique of looking for bugs, for sure
will come in handy few times!

cheers,
Andrzej.

Re: Need some clarification about optimization flags, what "exactly" does -O1 do?

[Andrew Haley]

Andrzej Giniewicz wrote:

>> I have not much experience, but it seems like it's not that bad...
>> maybe I should check how m32BitIndicies is initialized?
> 
> and I did it - and it turned out there was case when it was
> uninitialized... looking for this without all your help would be
> really hard :)... the issue is solved and you were right - it's not
> gcc issue :)
> 
> thanks again for great new technique of looking for bugs, for sure
> will come in handy few times!

Excellent.  Have fun.

Andrew.

RE: Need some clarification about optimization flags, what "exactly" does -O1 do?

[John (Eljay) Love-Jensen]

Hi Andrzej Giniewicz,

> interesting, seems like something similar, any hints how to start tracking real issue?

Well, if that IS the real issue, I'd start looking for enum abuse.

For example, in some old code I had to maintain, there was this situation (simplified):

enum Bar { uno, dos, tres };
int Foo(enum Bar bar)
{
  int value = bar;
  switch(value)
  {
  case uno: return 100;
  case dos: return 200;
  case tres: return 300;
  case 4: return 400;
  case -1: return 0;
  default: return -1;
  }
}

The last three cases could happen in the code, due to sledgehammer casting.  That's undefined behavior.  The int value = bar; did not fix the undefined behavior.

Ultimately, the code was fixed to not have undefined behavior.

HTH,
--Eljay