[Bug c++/38625] New: Segmentation fault when dereferencing valid pointer, probably REGRESSION

[Bug c++/38625] New: Segmentation fault when dereferencing valid pointer, probably REGRESSION

[l dot jirkovsky at gmail dot com]

First, I'd like to apologize that I wasn't able to extract the problematic
code. I hope somebody more experienced will find the way how to do it.

Problematic code is in enblend, which can be downloaded from cvs using cvs:
cvs -z3 -d :pserver:anonymous@enblend.cvs.sourceforge.net:/cvsroot/enblend -f
enblend

The problem is in src/mask.h:228 in (*mxLeft != *mx). It seems that it crashes
when dereferencing pointer (which should be valid because it's used before),
because it even doesn't get to overloaded operator!= function in
include/vigra/diff2d.hxx:1093

I've tested it with gcc 3.4.6 and it works without segfaults, so I think that
it can be some regression. Moreover, windows version compiled with MSVC seems
do be unaffected too.

I think it can be somewhat connected to bug #32896.

It has already been reported in hugin's bugtracker (enblend is a part of it):
https://sourceforge.net/tracker2/?func=detail&aid=2121647&group_id=77506&atid=550441

Steps to reproduce:
-------------------
compile enblend:
  make -f Makefile.cvs
  ./configure --prefix=/usr --enable-debug=no --with-x
  sed -i 's|#define\ malloc\ rpl_malloc|/* & */|' config.h # sometimes maloc is
not detected corectly
  make
Download test files which cause segfault and unzip them from:
  http://blender6xx.ic.cz/pub/DEBUG.zip
run enblend:
  enblend kladruby00* -o test.tif

After some time it segfaults

System information
------------------
OS: Archlinux current (but it affect wide spectrum of distributions, so it's
not problem in only one distro)

uname -a: Linux red_dragon 2.6.27.10 #2 Sat Dec 20 09:47:07 CET 2008 i686
Intel(R) Pentium(R) 4 CPU 2.00GHz GenuineIntel GNU/Linux

gcc -v:
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: ../configure --prefix=/usr --enable-shared
--enable-languages=c,c++,fortran,objc,obj-c++,treelang --enable-threads=posix
--mandir=/usr/share/man --infodir=/usr/share/info --enable-__cxa_atexit
--disable-multilib --libdir=/usr/lib --libexecdir=/usr/lib --enable-clocale=gnu
--disable-libstdcxx-pch --with-tune=generic
Thread model: posix
gcc version 4.3.2 (GCC)

gcc-3.4 -v
Reading specs from /usr/lib/gcc/i686-pc-linux-gnu/3.4.6/specs
Configured with: ../gcc-3.4.6/configure --prefix=/usr --enable-shared
--enable-languages=c,c++ --enable-threads=posix --mandir=/usr/share/man
--libexecdir=/usr/lib --enable-__cxa_atexit --disable-multilib
--libdir=/usr/lib --enable-clocale=gnu --program-suffix=-3.4
Thread model: posix
gcc version 3.4.6


-- 
           Summary: Segmentation fault when dereferencing valid pointer,
                    probably REGRESSION
           Product: gcc
           Version: 4.3.2
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c++
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: l dot jirkovsky at gmail dot com


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38625

[Bug c++/38625] Segmentation fault when dereferencing valid pointer, probably REGRESSION

[pinskia at gcc dot gnu dot org]

------- Comment #1 from pinskia at gcc dot gnu dot org  2008-12-25 15:36 -------
>I think it can be somewhat connected to bug #32896.

Unlikely.  

Anyways does it segfault when compiled at -O0.  How about -O2
-fno-strict-aliasing ?


-- 

pinskia at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38625

[Bug c++/38625] Segmentation fault when dereferencing valid pointer, probably REGRESSION

[l dot jirkovsky at gmail dot com]

------- Comment #2 from l dot jirkovsky at gmail dot com  2008-12-25 17:25 -------
I've already tested it with -O2 -fno-strict-aliasing without success. I'll test
it with -O0.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38625

[Bug c++/38625] Segmentation fault when dereferencing valid pointer, probably REGRESSION

[l dot jirkovsky at gmail dot com]

------- Comment #3 from l dot jirkovsky at gmail dot com  2008-12-25 18:07 -------
with -O0 no segfault


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38625

[Bug c++/38625] Segmentation fault when dereferencing valid pointer, probably REGRESSION

[pinskia at gcc dot gnu dot org]

------- Comment #4 from pinskia at gcc dot gnu dot org  2008-12-25 18:09 -------
How about -O2 -fno-strict-overflow ?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38625

[Bug c++/38625] Segmentation fault when dereferencing valid pointer, probably REGRESSION

[l dot jirkovsky at gmail dot com]

------- Comment #5 from l dot jirkovsky at gmail dot com  2008-12-25 18:51 -------
-O2 -fno-strict-overflow also segfaults


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38625

[Bug c++/38625] Segmentation fault when dereferencing valid pointer, probably REGRESSION

[l dot jirkovsky at gmail dot com]

------- Comment #6 from l dot jirkovsky at gmail dot com  2009-01-10 16:42 -------
I've tried it with gcc 4.2.4 and it works perfectly, so it have to be caused by
some change between 4.2.4 and 4.3.2.

I'll try to use svn to find out which commit causes this.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38625

[Bug c++/38625] Segmentation fault when dereferencing valid pointer, probably REGRESSION

[l dot jirkovsky at gmail dot com]

------- Comment #7 from l dot jirkovsky at gmail dot com  2009-01-10 16:47 -------
I've forgot to post info about gcc 4.2.4:
$ gcc -v
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: ../gcc-4.2.4/configure --prefix=/home/lukas/gcc
--enable-shared --enable-languages=c,c++ --enable-threads=posix
--enable-__cxa_atexit
Thread model: posix
gcc version 4.2.4


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38625

[Bug c++/38625] Segmentation fault when dereferencing valid pointer, probably REGRESSION

[hjl dot tools at gmail dot com]

------- Comment #8 from hjl dot tools at gmail dot com  2009-01-11 23:32 -------
This patch

http://gcc.gnu.org/ml/gcc-patches/2007-02/msg00886.html

triggers this crash. It failed at -O1. With gcc 4.4, it failed at -O2.


-- 

hjl dot tools at gmail dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hjl dot tools at gmail dot
                   |                            |com
   Last reconfirmed|0000-00-00 00:00:00         |2009-01-11 23:32:10
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38625

[Bug c++/38625] Segmentation fault when dereferencing valid pointer, probably REGRESSION

[hjl dot tools at gmail dot com]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1407 bytes --]



------- Comment #9 from hjl dot tools at gmail dot com  2009-01-11 23:33 -------
There are warnings like

../include/vigra/accessor.hxx: In function âvoid
vigra::read_bands(vigra::Decoder*, ImageIterator, Accessor, SrcValueType) [with
ImageIterator = vigra::CachedFileImageIterator<vigra::RGBValue<double, 0u, 1u,
2u> >, Accessor = vigra::RGBAccessor<vigra::RGBValue<double, 0u, 1u, 2u> >,
SrcValueType = double]â:
../include/vigra/accessor.hxx:813: warning: array subscript is above array
bounds
In file included from enblend.h:39,
                 from enblend.cc:124:
../include/vigra/diff2d.hxx: In function âvoid enblend::maskBounds(MaskType*,
vigra::Rect2D&, vigra::Rect2D&) [with MaskType =
enblend::enblendMain(std::list<vigra::ImageImportInfo*,
std::allocator<vigra::ImageImportInfo*> >&, vigra::ImageExportInfo&,
vigra::Rect2D&) [with ImagePixelType = vigra::RGBValue<unsigned char, 0u, 1u,
2u>]::MaskType]â:
../include/vigra/diff2d.hxx:1108: warning: assuming signed overflow does not
occur when assuming that (X - c) > X is always false


-- 

hjl dot tools at gmail dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|2009-01-11 23:32:10         |2009-01-11 23:33:14
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38625

[Bug c++/38625] Segmentation fault when dereferencing valid pointer, probably REGRESSION

[hjl dot tools at gmail dot com]

------- Comment #10 from hjl dot tools at gmail dot com  2009-01-11 23:46 -------
Adding "--param inline-unit-growth=60" fixed gcc 4.4 revision 143274
at -O2.


-- 

hjl dot tools at gmail dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jh at suse dot cz
             Status|WAITING                     |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|2009-01-11 23:33:14         |2009-01-11 23:46:52
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38625

[Bug c++/38625] Segmentation fault when dereferencing valid pointer, probably REGRESSION

[l dot jirkovsky at gmail dot com]

------- Comment #11 from l dot jirkovsky at gmail dot com  2009-01-29 11:19 -------
First, I'd like to thank you for doing this hard work and for finding out which
patch causes this problem.

Anyway I've done more investigation to the problematic code.

The problem actually begins in
CachedFileImageIteratorBase::operator*()

In correct build (without optimizations, with debugging enabled or with
"--param inline-unit-growth=60") the currentRow pointer is pointer to
"ordinary" array, I'm guessing it's array of unsigned shorts.

But in segfaulting build my debugger (gdb) shows me, that currentRow is:
vigra::TinyVectorBase<unsigned char, 3, unsigned char [3],
vigra::TinyVector<unsigned char, 3> >
which _data structure doesn't exist in memory. Because it deems really weird
I'm not sure the debugger was right (it was run with higly optimized code when
only some parts of enblend actually had debugging information on).

However if I'm wrong in previous statement, the currentRow should still be
valid. I'd took if I was trying to access, lets say, currentRow[1000] which
could be out of array bounds, but this code segfaults when I'm trying to access
currentRow[0].


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38625

[Bug c++/38625] Segmentation fault when dereferencing valid pointer, probably REGRESSION

[rguenth at gcc dot gnu dot org]

------- Comment #12 from rguenth at gcc dot gnu dot org  2010-01-02 19:40 -------
Looks like invalid code in the first place.


-- 

rguenth at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |INVALID


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38625