[Bug tree-optimization/36524] New: regression: crash when compiling IdentifierTable.cpp

[Bug tree-optimization/36524] New: regression: crash when compiling IdentifierTable.cpp

[edwintorok at gmail dot com]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 7432 bytes --]

When I compile IdentifierTable.cpp from clang using gcc 4.3.1 I get a crash,
compiling with gcc 4.2.4 doesn't crash.

If I preprocess it first, and then compile it doesn't crash.
If I move the the llvm-svn/llvm/include files to a different folder it doesn't
crash.
So I am not even able to provide an archive of .cpp and .h files to reproduce
since it depends on folder names.

I will rebuild g++ with debug symbols and see where it crashes.
What --enable-checking setting should I use?

Do you have any other suggestions how to get some files that I could submit
that would reproduce the bug?

$ /usr/bin/g++  -I/home/edwin/llvm-svn/llvm/include
-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic  -D_DEBUG
-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/../../include -D_GNU_SOURCE
-D__STDC_LIMIT_MACROS   IdentifierTable.cpp -o IdentifierTable.o -c -O1
In file included from IdentifierTable.cpp:161:
/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/../../include/clang/Basic/TokenKinds.def:272:
internal compiler error: Segmentation fault
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.3/README.Bugs> for instructions.


Valgrind shows:
==17940== Invalid write of size 1
==17940==    at 0x9B4D40: (within /usr/lib/gcc/x86_64-linux-gnu/4.3/cc1plus)
==17940==    by 0x9B4FFF: (within /usr/lib/gcc/x86_64-linux-gnu/4.3/cc1plus)
==17940==    by 0x4D1ACD: (within /usr/lib/gcc/x86_64-linux-gnu/4.3/cc1plus)
==17940==    by 0x4570CF: (within /usr/lib/gcc/x86_64-linux-gnu/4.3/cc1plus)
==17940==    by 0x469544: (within /usr/lib/gcc/x86_64-linux-gnu/4.3/cc1plus)
==17940==    by 0x4D615C: (within /usr/lib/gcc/x86_64-linux-gnu/4.3/cc1plus)
==17940==    by 0x68DFAA: (within /usr/lib/gcc/x86_64-linux-gnu/4.3/cc1plus)
==17940==    by 0x4C4F1A5: (below main) (libc-start.c:222)
==17940==  Address 0x227505b36fb6 is not stack'd, malloc'd or (recently) free'd


$ g++ -v
Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 4.3.1-2'
--with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared
--with-system-zlib --libexecdir=/usr/lib --without-included-gettext
--enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.3
--program-suffix=-4.3 --enable-clocale=gnu --enable-libstdcxx-debug
--enable-objc-gc --enable-mpfr --enable-cld --enable-checking=release
--build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.3.1 (Debian 4.3.1-2)

Doesn't crash:
$ /usr/bin/g++-4.2  -I/home/edwin/llvm-svn/llvm/include
-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic  -D_DEBUG
-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/../../include -D_GNU_SOURCE
-D__STDC_LIMIT_MACROS IdentifierTable.cpp -o IdentifierTable.o -c -O1

$ g++-4.2 -v
Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared
--with-system-zlib --libexecdir=/usr/lib --without-included-gettext
--enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.2
--program-suffix=-4.2 --enable-clocale=gnu --enable-libstdcxx-debug
--enable-objc-gc --enable-mpfr --enable-checking=release
--build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.2.4 (Debian 4.2.4-2)

I am unable to reproduce the crash if I first preprocess and then I compile, I
tried:
$ /usr/bin/g++ -v -I/home/edwin/llvm-svn/llvm/include
-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic  -D_DEBUG
-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/../../include -D_GNU_SOURCE
-D__STDC_LIMIT_MACROS   IdentifierTable.cpp -o IdentifierTable.o -c -O1
-save-temps
Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 4.3.1-2'
--with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared
--with-system-zlib --libexecdir=/usr/lib --without-included-gettext
--enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.3
--program-suffix=-4.3 --enable-clocale=gnu --enable-libstdcxx-debug
--enable-objc-gc --enable-mpfr --enable-cld --enable-checking=release
--build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.3.1 (Debian 4.3.1-2)
COLLECT_GCC_OPTIONS='-v' '-I/home/edwin/llvm-svn/llvm/include'
'-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic' '-D_DEBUG'
'-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/../../include'
'-D_GNU_SOURCE' '-D__STDC_LIMIT_MACROS' '-o' 'IdentifierTable.o' '-c'
'-save-temps' '-O1' '-shared-libgcc' '-mtune=generic'
 /usr/lib/gcc/x86_64-linux-gnu/4.3.1/cc1plus -E -quiet -v
-I/home/edwin/llvm-svn/llvm/include
-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic
-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/../../include -D_GNU_SOURCE
-D_DEBUG -D_GNU_SOURCE -D__STDC_LIMIT_MACROS IdentifierTable.cpp -mtune=generic
-O1 -fpch-preprocess -o IdentifierTable.ii
ignoring nonexistent directory "/usr/local/include/x86_64-linux-gnu"
ignoring nonexistent directory
"/usr/lib/gcc/x86_64-linux-gnu/4.3.1/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/usr/include/x86_64-linux-gnu"
#include "..." search starts here:
#include <...> search starts here:
 /home/edwin/llvm-svn/llvm/include
 /home/edwin/llvm-svn/llvm/tools/clang/lib/Basic
 /home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/../../include
 /usr/include/c++/4.3
 /usr/include/c++/4.3/x86_64-linux-gnu
 /usr/include/c++/4.3/backward
 /usr/local/include
 /usr/lib/gcc/x86_64-linux-gnu/4.3.1/include
 /usr/lib/gcc/x86_64-linux-gnu/4.3.1/include-fixed
 /usr/include
End of search list.
In file included from <built-in>:0:
<built-in>:0: internal compiler error: Segmentation fault
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.3/README.Bugs> for instructions.

$ /usr/bin/g++ -c -O1 IdentifierTable.ii
In file included from IdentifierTable.cpp:162:
/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/../../include/clang/Basic/TokenKinds.def:
In member function ‘void clang::IdentifierTable::AddKeywords(const
clang::LangOptions&)’:
/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/../../include/clang/Basic/TokenKinds.def:271:
error: expected `}' at end of input

Ok, so the preprocessed file got truncated, trying again without -O1:
$ /usr/bin/g++  -I/home/edwin/llvm-svn/llvm/include
-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic  -D_DEBUG
-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/../../include -D_GNU_SOURCE
-D__STDC_LIMIT_MACROS   IdentifierTable.cpp -o IdentifierTable.o -c 
-save-temps

However the crash won't reproduce with this preprocessed file:
$ /usr/bin/g++ -c -O1 IdentifierTable.ii


-- 
           Summary: regression: crash when compiling IdentifierTable.cpp
           Product: gcc
           Version: 4.3.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: edwintorok at gmail dot com
 GCC build triplet: x86_64-linux-gnu
  GCC host triplet: x86_64-linux-gnu
GCC target triplet: x86_64-linux-gnu


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36524

[Bug tree-optimization/36524] regression: crash when compiling IdentifierTable.cpp

[rguenth at gcc dot gnu dot org]

------- Comment #1 from rguenth at gcc dot gnu dot org  2008-06-13 11:24 -------
-fpch-preprocess is probably one of the keys?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36524

[Bug tree-optimization/36524] regression: crash when compiling IdentifierTable.cpp

[edwintorok at gmail dot com]

------- Comment #2 from edwintorok at gmail dot com  2008-06-13 11:34 -------
(In reply to comment #1)
> -fpch-preprocess is probably one of the keys?
> 

Running cc1plus without -fpch-preprocess still leads to a crash.

edwin@lightspeed2:~/llvm-svn/llvm/tools/clang/lib/Basic$
/usr/lib/gcc/x86_64-linux-gnu/4.3.1/cc1plus -E -quiet -v
-I/home/edwin/llvm-svn/llvm/include
-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic
-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/../../include -MMD
/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/Release/IdentifierTable.d -MF
/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/Release/IdentifierTable.d.tmp
-MP -MT
/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/Release/IdentifierTable.lo -MT
/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/Release/IdentifierTable.o -MT
/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/Release/IdentifierTable.d
-D_GNU_SOURCE -D_DEBUG -D_GNU_SOURCE -D__STDC_LIMIT_MACROS IdentifierTable.cpp
-mtune=generic -Woverloaded-virtual -pedantic -Wall -W -Wwrite-strings
-Wno-long-long -Wunused -Wno-unused-parameter -fomit-frame-pointer
-fno-exceptions -fno-rtti -O1  -o IdentifierTable.ii
ignoring nonexistent directory "/usr/local/include/x86_64-linux-gnu"
ignoring nonexistent directory
"/usr/lib/gcc/x86_64-linux-gnu/4.3.1/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/usr/include/x86_64-linux-gnu"
#include "..." search starts here:
#include <...> search starts here:
 /home/edwin/llvm-svn/llvm/include
 /home/edwin/llvm-svn/llvm/tools/clang/lib/Basic
 /home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/../../include
 /usr/include/c++/4.3
 /usr/include/c++/4.3/x86_64-linux-gnu
 /usr/include/c++/4.3/backward
 /usr/local/include
 /usr/lib/gcc/x86_64-linux-gnu/4.3.1/include
 /usr/lib/gcc/x86_64-linux-gnu/4.3.1/include-fixed
 /usr/include
End of search list.
In file included from <built-in>:0:
<built-in>:0: internal compiler error: Segmentation fault
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.3/README.Bugs> for instructions.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36524

[Bug tree-optimization/36524] regression: crash when compiling IdentifierTable.cpp

[rguenth at gcc dot gnu dot org]

------- Comment #3 from rguenth at gcc dot gnu dot org  2008-06-13 11:45 -------
Can you delete the preprocessed libstdc++ includes from /usr/lib?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36524

[Bug tree-optimization/36524] regression: crash when compiling IdentifierTable.cpp

[edwintorok at gmail dot com]

------- Comment #4 from edwintorok at gmail dot com  2008-06-13 12:01 -------
(In reply to comment #3)
> Can you delete the preprocessed libstdc++ includes from /usr/lib?
> 

Sure, I only had these:
$ find  /usr/lib/ /usr/local/lib -name *.gch
/usr/lib/gcc/i586-mingw32msvc/4.2.1-sjlj/include/c++/i586-mingw32msvc/bits/stdtr1c++.h.gch
/usr/lib/gcc/i586-mingw32msvc/4.2.1-sjlj/include/c++/i586-mingw32msvc/bits/stdtr1c++.h.gch/O2g.gch
/usr/lib/gcc/i586-mingw32msvc/4.2.1-sjlj/include/c++/i586-mingw32msvc/bits/stdc++.h.gch
/usr/lib/gcc/i586-mingw32msvc/4.2.1-sjlj/include/c++/i586-mingw32msvc/bits/stdc++.h.gch/O0g.gch
/usr/lib/gcc/i586-mingw32msvc/4.2.1-sjlj/include/c++/i586-mingw32msvc/bits/stdc++.h.gch/O2g.gch

I deleted them, however I am still getting teh crash.

Meanwhile, I am building gcc 4.3.1 from sources, with:
--enable-checking=all --disable-bootstrap --disable-multilib
--enable-languages=c,c++


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36524

[Bug tree-optimization/36524] regression: crash when compiling IdentifierTable.cpp

[edwintorok at gmail dot com]

------- Comment #5 from edwintorok at gmail dot com  2008-06-13 13:37 -------
I have built with --enable-checking=yes (instead of all), because all was
taking too long.

It looks like a value of a pointer is invalid.
See the gdb session below.

What should I do next to help debug the problem?

Starting program: /usr/local/libexec/gcc/x86_64-unknown-linux-gnu/4.3.1/cc1plus
-E -quiet -v -I/home/edwin/llvm-svn/llvm/include
-I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic
 -I/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/../../include -MMD I.d -MF
/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/Release/IdentifierTable.d.tmp
-MP -MT /home/edwin/ll
vm-svn/llvm/tools/clang/lib/Basic/Release/IdentifierTable.lo -MT
/home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/Release/IdentifierTable.o -MT
/home/edwin/llvm-svn/llvm/tools/cla
ng/lib/Basic/Release/IdentifierTable.d -D_GNU_SOURCE -D_DEBUG -D_GNU_SOURCE
-D__STDC_LIMIT_MACROS IdentifierTable.cpp -o I.ii -mtune=generic
-Woverloaded-virtual -pedantic -Wall
-W -Wwrite-strings -Wno-long-long -Wunused -Wno-unused-parameter -Winvalid-pch
-fomit-frame-pointer -fno-exceptions -fno-rtti -O1
ignoring nonexistent directory
"/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.3.1/../../../../x86_64-unknown-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
/home/edwin/llvm-svn/llvm/include
 /home/edwin/llvm-svn/llvm/tools/clang/lib/Basic
 /home/edwin/llvm-svn/llvm/tools/clang/lib/Basic/../../include

/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.3.1/../../../../include/c++/4.3.1

/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.3.1/../../../../include/c++/4.3.1/x86_64-unknown-linux-gnu

/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.3.1/../../../../include/c++/4.3.1/backward
 /usr/local/include
/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.3.1/include
 /usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.3.1/include-fixed
 /usr/include
End of search list.

Program received signal SIGSEGV, Segmentation fault.
0x0000000000c111d3 in new_string_token (pfile=0x2ae4280, text=0x22745f323372
<Address 0x22745f323372 out of bounds>, len=2743223540) at
../../gcc-4.3.1/libcpp/macro.c:100
(gdb) bt
#0  0x0000000000c111d3 in new_string_token (pfile=0x2ae4280,
text=0x22745f323372 <Address 0x22745f323372 out of bounds>, len=2743223540) at
../../gcc-4.3.1/libcpp/macro.c:100
#1  0x0000000000c11cbb in stringify_arg (pfile=0x2ae4280, arg=0x2b439a0) at
../../gcc-4.3.1/libcpp/macro.c:434
#2  0x0000000000c12a61 in replace_args (pfile=0x2ae4280, node=0x7f8657b614a0,
macro=0x7f8657a15140, args=0x2b439a0) at ../../gcc-4.3.1/libcpp/macro.c:921
#3  0x0000000000c12888 in enter_macro_context (pfile=0x2ae4280,
node=0x7f8657b614a0, result=0x2ae4710) at ../../gcc-4.3.1/libcpp/macro.c:850
#4  0x0000000000c133e7 in cpp_get_token (pfile=0x2ae4280) at
../../gcc-4.3.1/libcpp/macro.c:1234
#5  0x00000000005a4a28 in preprocess_file (pfile=0x2ae4280) at
../../gcc-4.3.1/gcc/c-ppoutput.c:148
#6  0x00000000005a08a9 in c_common_init () at ../../gcc-4.3.1/gcc/c-opts.c:1244
#7  0x00000000004bf843 in cxx_init () at ../../gcc-4.3.1/gcc/cp/lex.c:421
#8  0x0000000000807698 in toplev_main (argc=<value optimized out>, argv=<value
optimized out>) at ../../gcc-4.3.1/gcc/toplev.c:2123
#9  0x0000003cbb81e1a6 in __libc_start_main () from /lib/libc.so.6
#10 0x0000000000404269 in _start ()

(gdb) p text
$1 = (unsigned char *) 0x22745f323372 <Address 0x22745f323372 out of bounds>
(gdb) p len
$2 = 2743223540
(gdb) up
(gdb) p dest
$3 = (unsigned char *) 0x2b47866 ""
(gdb) p len
$4 = 18446706190546257140
^^^^^^^^^^^^^^^^ looks like a negative number to me

(gdb) p pfile->u_buff
$5 = (_cpp_buff *) 0x2b47850

432   len = dest - BUFF_FRONT (pfile->u_buff);
433   BUFF_FRONT (pfile->u_buff) = dest + 1;
434   return new_string_token (pfile, dest - len, len);

(gdb) display arg->first[0]
(gdb) b 433
Breakpoint 6 at 0xc11c8a: file ../../gcc-4.3.1/libcpp/macro.c, line 433.
(gdb) cond 6 len>100000
(gdb) r
...
Breakpoint 6, stringify_arg (pfile=0x2a0f280, arg=0x2a6e9a0) at
../../gcc-4.3.1/libcpp/macro.c:433
1: *arg->first[0] = {src_loc = 9522344, type = CPP_NAME, flags = 0 '\0', val =
{node = 0x7f472c117020, source = 0x7f472c117020, str = {len = 739340320, text =
0x2a72512 "\"clang/
Basic/TokenKinds.def\""}, arg_no = 739340320, pragma = 739340320}}
(gdb) p pfile->u_buff->cur
$30 = (unsigned char *) 0x22745f323372 <Address 0x22745f323372 out of bounds>


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36524

[Bug preprocessor/36524] regression: crash when compiling IdentifierTable.cpp

[jsm28 at gcc dot gnu dot org]

------- Comment #6 from jsm28 at gcc dot gnu dot org  2009-03-30 00:22 -------
Does the code in question stringize any of the C++ operators and_eq, bitand,
bitor, compl, not_eq, or_eq, xor_eq, or use them in any other way?  If so,
this is likely to be the bug I fixed with
<http://gcc.gnu.org/ml/gcc-patches/2008-11/msg01480.html> (and so would
be fixed for 4.4 and 4.5).  If not, I don't think we can do anything without
at least a tarball of sources that reproduce the problem.


-- 

jsm28 at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36524

[Bug preprocessor/36524] regression: crash when compiling IdentifierTable.cpp

[jsm28 at gcc dot gnu dot org]

------- Comment #7 from jsm28 at gcc dot gnu dot org  2009-11-22 19:26 -------
Presuming fixed by the patch I identified since no other comments.


-- 

jsm28 at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
      Known to work|                            |4.4.0
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36524