Weird optimization bug...?

Weird optimization bug...?

[Adrian Bentley]

I have this crash bug in a templatized 2d blurring function.  It does
not occur in debug mode and if I put a print statement in the loop to
tell me what iteration it is on, it does not occur either.  I cannot see
any reason it seg faults when looking at dumped core or using gdb
manually, both using debug info + optimizations.  It seems like an
optimization issue (i.e. when I force the print statement it doesn't do
the optimization that borks it...).

I just upgraded to gcc 3.3.2 (most recent "stable" version on gentoo)
and have been trying to get a company project to build correctly.

My first inclination is to upgrade to the lastest "unstable" gcc version
on portage (3.3.3 I think).

Any suggestions?
Adruab

Re: Weird optimization bug...?

[Eljay Love-Jensen]

Hi Adruab,

A bug like that is usually indicative of a buffer overrun.

int rgb[3];
rgb[4] = 7;

Or of cavalier casting.
short hue;
int* p = (int*)&hue;
*p = 0;

Without seeing the code (or some sample code that replicates the bug), it 
would be difficult to guess less likely scenarios.

HTH,
--Eljay

Re: Weird optimization bug...?

[Adrian Bentley]

Yeah, that would make sense.  And buffer overruns can sometimes appear
only in release mode (not 0 initializing something or the like).... 
However, if that was the case, inserting a random function into the code
certain shouldn't fix the problem :P.

Here's a snippet of code for all to see, I'll just explain some of the
dependent stuff:

//not compilable unless you have all the other classes... which is a lot

template<typename T1,typename T2> void
vbox_blur(T1 pen,int w, int h, int length,T2 outpen)
{	
	int x,y;
   	typename T1::iterator_y iter, end;
	
	printf("V Blur (%d,%d) - %d size\n",w,h,length); //debug crap
	
	const int div = (length*2+1);

	length=min(h,length);
	for(x=0;x<w;x++,pen.inc_x(),outpen.inc_x())
	{
		printf("\nx line %d\n",x); //debugging crud
		
		iter=pen.y();
		end=pen.end_y();
		const typename T1::value_type val=pen.get_value();
		typename T1::accumulator_type tot(val*(length+1));
		//tot=val*(length+1);

		for (y=0;y<length && iter!=end;y++,++iter) tot+=*iter;
		iter=pen.y();

		for (y=0;y<h && iter!=end;y++,++iter,outpen.inc_y())
		{
			//calling either of these functions here fixes the crash... WTF!!!!!!
			//printf("%d\n",y);
			//stupid(y);
			
			/*if (y>length) tot-=iter[-length-1];
			else tot-=val;

			if (y+length<h) tot+=iter[length];
			else tot+=end[-1];*/
				
			tot -= (y>length) ? iter[-length-1] : val;
			tot += (y+length<h) ? iter[length] : end[-1];
			
			outpen.put_value(tot/div);
		}
		outpen.dec_y(y);
	}
}

The pen is basically just a random-access iterator for a 2 dimensional
array (the inc_x and inc_y functions behave exactly as you'd expect). 
The value type it's looking at is basically a vector (actually a color)
with overloaded functions +=, -=, / (among others).  Iterator_y is
basically just a specialized iterator to increment using a stride rather
than 1.

In any case, I'm pretty sure this was not a buffer overrun (at least of
the fault of the pen/iterators), because it worked just fine in previous
versions of gcc, and it's pretty simple and has been tested (I'm going
to disect the pen class in a bit).  Plus that certainly doesn't explain
why forcing a function call will make it not crash.

I would submit a bug, except this certainly isn't very straight forward,
and I can't submit that big of a chunk of my project for testing.... 
Any suggestions?

Adruab

On Thu, 2004-06-03 at 04:26, Eljay Love-Jensen wrote:
> Hi Adruab,
> 
> A bug like that is usually indicative of a buffer overrun.
> 
> int rgb[3];
> rgb[4] = 7;
> 
> Or of cavalier casting.
> short hue;
> int* p = (int*)&hue;
> *p = 0;
> 
> Without seeing the code (or some sample code that replicates the bug), it 
> would be difficult to guess less likely scenarios.
> 
> HTH,
> --Eljay
> 
> 
> 

Re: Weird optimization bug...?

[Eljay Love-Jensen]

Hi Adruab.

 >However, if that was the case, inserting a random function into the code 
certain shouldn't fix the problem :P.

Inserting a random function into the code can affect the optimizer, which 
can mask the problem.

I recommend taking a look at the assembly file to see what's different 
between having and not having the extra function present, in the optimized 
code.

You can use the "-save-temps" to keep the assembly file around, and 
"-fverbose-asm" for extra blather.

HTH,
--Eljay

Re: Weird optimization bug...?

[Adrian Bentley]

Sorry, I probably wasn't clear.  I was just referring to why I didn't
think it was a buffer overrun.  It makes perfect sense why it would
change things with the optimizer involved :) (thought perhaps I am
missing something wrt the buffer overrun...).

Thanks for the pointers on the assembly generation.  I'm having some
trouble building with those options, so I'll get back to you when I've
figured it out (or hit a brick wall :P).

Thanks again,
Adruab > Adrian Bentley

On Thu, 2004-06-03 at 11:36, Eljay Love-Jensen wrote:
> Hi Adruab.
> 
>  >However, if that was the case, inserting a random function into the code 
> certain shouldn't fix the problem :P.
> 
> Inserting a random function into the code can affect the optimizer, which 
> can mask the problem.
> 
> I recommend taking a look at the assembly file to see what's different 
> between having and not having the extra function present, in the optimized 
> code.
> 
> You can use the "-save-temps" to keep the assembly file around, and 
> "-fverbose-asm" for extra blather.
> 
> HTH,
> --Eljay
> 
> 
> 

Re: Weird optimization bug...?

[Adrian Bentley]

So I looked at the assembly and didn't find anything super glaring.  The
compiled code was very different, so I'm not surprised the result is
different.  I haven't gotten any closer to figuring out what kind of
mis-optimization it's doing (or if it's something else totally
bizarre).  Any other pointers for narrowing it down, or should I just
hack it and wait for a new version of gcc to come out :).

Thanks again,
Adruab

On Thu, 2004-06-03 at 11:36, Eljay Love-Jensen wrote:
> Hi Adruab.
> 
>  >However, if that was the case, inserting a random function into the code 
> certain shouldn't fix the problem :P.
> 
> Inserting a random function into the code can affect the optimizer, which 
> can mask the problem.
> 
> I recommend taking a look at the assembly file to see what's different 
> between having and not having the extra function present, in the optimized 
> code.
> 
> You can use the "-save-temps" to keep the assembly file around, and 
> "-fverbose-asm" for extra blather.
> 
> HTH,
> --Eljay
> 
> 
> 

Re: Weird optimization bug...?

[Adrian Bentley]

Ok ok... I think I found why it's crashing (although it doesn't actually
crash when I run it because I'm calling a print function).  Anyways, I
re-expanded the if statement to check the bounds on the array
indirections....

//see other code for the rest...
if (y>length) 
{
	typename T1::value_type &v = iter[-length-1];
	if( (char*)&v < beginptr || (char*)&v >= endptr)
		printf("crap! %d (%p off %p)\n",y,&v,&*iter);
	tot-=iter[-length-1];
}
else
{
	tot-=val;
}

And I'm indirecting off of iter which is being incremented every
iteration.  But for some reason it looks like gcc decided to use the
original value of iter for the 10th iteration (1 greater than the value
for length my example is using) inside the sample case I have... (i.e.
rather than using the incremented value).  So it looks like it was
trying to optimize out an expression that wasn't constant (either in
attempting to optimize the first if statement out of the loop).  It
doesn't do this when I turn optimization off... so it definitely looks
like an optimizer bug.

Anyways, I'm not sure why it would do this as iter is being changed
every iteration.  Oh well, I'm gonna manually do some unrolling and
hopefully it won't break any more.  Hope that helps anyone interested in
the issue.

Thanks again all,
Adruab


On Thu, 2004-06-03 at 11:36, Eljay Love-Jensen wrote:
> Hi Adruab.
> 
>  >However, if that was the case, inserting a random function into the code 
> certain shouldn't fix the problem :P.
> 
> Inserting a random function into the code can affect the optimizer, which 
> can mask the problem.
> 
> I recommend taking a look at the assembly file to see what's different 
> between having and not having the extra function present, in the optimized 
> code.
> 
> You can use the "-save-temps" to keep the assembly file around, and 
> "-fverbose-asm" for extra blather.
> 
> HTH,
> --Eljay
> 
> 
> 

Re: Weird optimization bug...?

[Ian Lance Taylor]

Adrian Bentley <adruab@voria.com> writes:

> And I'm indirecting off of iter which is being incremented every
> iteration.  But for some reason it looks like gcc decided to use the
> original value of iter for the 10th iteration (1 greater than the value
> for length my example is using) inside the sample case I have... (i.e.
> rather than using the incremented value).  So it looks like it was
> trying to optimize out an expression that wasn't constant (either in
> attempting to optimize the first if statement out of the loop).  It
> doesn't do this when I turn optimization off... so it definitely looks
> like an optimizer bug.
> 
> Anyways, I'm not sure why it would do this as iter is being changed
> every iteration.  Oh well, I'm gonna manually do some unrolling and
> hopefully it won't break any more.  Hope that helps anyone interested in
> the issue.

If you have time, please submit a bug report.  See
    http://gcc.gnu.org/bugs.html

Thanks.

Ian

Re: Weird optimization bug...?

[Adrian Bentley]

Yes, I certainly will.  

Is there any pragma or the like to disable gcc optimizations around a
certain part of a file?  Trying to get around this is taking way too
long :).

Thanks,
Adruab


On Thu, 2004-06-03 at 19:59, Ian Lance Taylor wrote:
> Adrian Bentley <adruab@voria.com> writes:
> 
> > And I'm indirecting off of iter which is being incremented every
> > iteration.  But for some reason it looks like gcc decided to use the
> > original value of iter for the 10th iteration (1 greater than the value
> > for length my example is using) inside the sample case I have... (i.e.
> > rather than using the incremented value).  So it looks like it was
> > trying to optimize out an expression that wasn't constant (either in
> > attempting to optimize the first if statement out of the loop).  It
> > doesn't do this when I turn optimization off... so it definitely looks
> > like an optimizer bug.
> > 
> > Anyways, I'm not sure why it would do this as iter is being changed
> > every iteration.  Oh well, I'm gonna manually do some unrolling and
> > hopefully it won't break any more.  Hope that helps anyone interested in
> > the issue.
> 
> If you have time, please submit a bug report.  See
>     http://gcc.gnu.org/bugs.html
> 
> Thanks.
> 
> Ian
> 
> 

Re: Weird optimization bug...?

[Adrian Bentley]

Ok, it's posted (#15837).  Hope it helps.

Adruab

On Thu, 2004-06-03 at 19:59, Ian Lance Taylor wrote:
> Adrian Bentley <adruab@voria.com> writes:
> 
> > And I'm indirecting off of iter which is being incremented every
> > iteration.  But for some reason it looks like gcc decided to use the
> > original value of iter for the 10th iteration (1 greater than the value
> > for length my example is using) inside the sample case I have... (i.e.
> > rather than using the incremented value).  So it looks like it was
> > trying to optimize out an expression that wasn't constant (either in
> > attempting to optimize the first if statement out of the loop).  It
> > doesn't do this when I turn optimization off... so it definitely looks
> > like an optimizer bug.
> > 
> > Anyways, I'm not sure why it would do this as iter is being changed
> > every iteration.  Oh well, I'm gonna manually do some unrolling and
> > hopefully it won't break any more.  Hope that helps anyone interested in
> > the issue.
> 
> If you have time, please submit a bug report.  See
>     http://gcc.gnu.org/bugs.html
> 
> Thanks.
> 
> Ian
> 
> 

Re: Weird optimization bug...?

[Ian Lance Taylor]

Adrian Bentley <adruab@voria.com> writes:

> Is there any pragma or the like to disable gcc optimizations around a
> certain part of a file?

No, I'm afraid not.  The optimization options affect the entire file.

Ian

Re: Weird optimization bug...?

[Ken Foskey]

On Sun, 2004-06-06 at 05:09, Adrian Bentley wrote:

> Is there any pragma or the like to disable gcc optimizations around a
> certain part of a file?  Trying to get around this is taking way too
> long :).

Create a separate file with that specific subroutine in it and compile
without optimisations.

The simple solution often eludes us :-)

-- 
Thanks
KenF
OpenOffice.org developer

Re: Weird optimization bug...?

[Adrian Bentley]

Heheh, ok, so the bug I posted for gcc didn't seem to be received very well
(I was really trying to be polite and not confuse people).  Any suggestions
on bug posting technique, so they will be usefully received in the future?

Thanks,
Adruab

----- Original Message ----- 
From: "Ian Lance Taylor" <ian@wasabisystems.com>
To: "Adrian Bentley" <adruab@voria.com>
Sent: Saturday, June 05, 2004 4:31 PM
Subject: Re: Weird optimization bug...?


> Adrian Bentley <adruab@voria.com> writes:
>
> > Ok, it's posted (#15837).  Hope it helps.
>
> Thanks.
>
> Ian
>
>

Re: Weird optimization bug...?

[Ian Lance Taylor]

"Adrian Bentley" <adruab@voria.com> writes:

> Heheh, ok, so the bug I posted for gcc didn't seem to be received very well
> (I was really trying to be polite and not confuse people).  Any suggestions
> on bug posting technique, so they will be usefully received in the future?

I dunno, from my perspective I think the bug was received reasonably
well.  People told you that your code relied on undefined behaviour.
I can see that that is not the best answer from your perspective, but
to me it does not seem like an unreasonable answer.

If you would like a specific warning, then present a specific case of
code which is undefined, and suggest what warning it should issue.
Also indicate which warning options you tried; for example, did you
try -Wstrict-aliasing?

Ian

RE: Weird optimization bug...?

[lrtaylor]

If not a buffer overrun, perhaps a buffer/variable initialization (or
lack thereof) issue?  Where the variable ends up in memory could affect
whether or not things work the way you expect, and I suppose that
optimization changes could affect that.

Just a thought.

Thanks,
Lyle


-----Original Message-----
From: gcc-help-owner@gcc.gnu.org [mailto:gcc-help-owner@gcc.gnu.org] On
Behalf Of Adrian Bentley
Sent: Thursday, June 03, 2004 1:11 PM
To: Eljay Love-Jensen
Cc: gcc-help@gcc.gnu.org
Subject: Re: Weird optimization bug...?

Sorry, I probably wasn't clear.  I was just referring to why I didn't
think it was a buffer overrun.  It makes perfect sense why it would
change things with the optimizer involved :) (thought perhaps I am
missing something wrt the buffer overrun...).

Thanks for the pointers on the assembly generation.  I'm having some
trouble building with those options, so I'll get back to you when I've
figured it out (or hit a brick wall :P).

Thanks again,
Adruab > Adrian Bentley

On Thu, 2004-06-03 at 11:36, Eljay Love-Jensen wrote:
> Hi Adruab.
> 
>  >However, if that was the case, inserting a random function into the
code 
> certain shouldn't fix the problem :P.
> 
> Inserting a random function into the code can affect the optimizer,
which 
> can mask the problem.
> 
> I recommend taking a look at the assembly file to see what's different

> between having and not having the extra function present, in the
optimized 
> code.
> 
> You can use the "-save-temps" to keep the assembly file around, and 
> "-fverbose-asm" for extra blather.
> 
> HTH,
> --Eljay
> 
> 
>