public inbox for gcc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug c/38136]  New: vim crashes on startup when compiled with -O3 but works with -O2
@ 2008-11-15 18:06 dominique dot pelle at gmail dot com
  2008-11-15 18:43 ` Andrew Thomas Pinski
                   ` (8 more replies)
  0 siblings, 9 replies; 10+ messages in thread
From: dominique dot pelle at gmail dot com @ 2008-11-15 18:06 UTC (permalink / raw)
  To: gcc-bugs

Vim (at least current version vim-7.2.40) crashes on start up when compiled
with optimization option -O3 using gcc-4.3.2 (that's the default version that
comes with Ubuntu-8.10). Vim works fine when compiled with -O0, -O1 or -O2.  

Narrowing it down further, compilation options "-O2 -finline-functions" are
enough to trigger the crash.

Steps to reproduce:

1/ download vim-7.2.40

  $ cvs  -z3 -d:pserver:anonymous@vim.cvs.sf.net:/cvsroot/vim checkout
-rv7-2-040 vim7

2/ configure and build vim with -O3 optimization level:

  $ cd vim7
  $ ./configure
  $ make CFLAGS=-O3

3/ run vim and observe crash

  $ cd src
  $ ./vim
*** buffer overflow detected ***: ./vim terminated
======= Backtrace: =========
/lib/tls/i686/cmov/libc.so.6(__fortify_fail+0x48)[0xb77aa558]
/lib/tls/i686/cmov/libc.so.6[0xb77a8680]
/lib/tls/i686/cmov/libc.so.6(__strcpy_chk+0x44)[0xb77a7944]
./vim[0x8081e8d]
./vim[0x8083521]
./vim[0x808abec]
./vim[0x80950e1]
./vim[0x80b4ba8]
./vim[0x80b72bb]
./vim[0x80a6ffb]
./vim[0x80a7661]
./vim[0x80a4b14]
./vim[0x80a4c18]
./vim[0x80fa68c]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb76c6685]
./vim[0x8052b31]
======= Memory map: ========
08048000-0823c000 r-xp 00000000 08:04 879079     /tmp/vim7/src/vim
0823c000-0823d000 r--p 001f3000 08:04 879079     /tmp/vim7/src/vim
0823d000-08249000 rw-p 001f4000 08:04 879079     /tmp/vim7/src/vim
08249000-08251000 rw-p 08249000 00:00 0 
0930f000-09393000 rw-p 0930f000 00:00 0          [heap]
b731e000-b744d000 rw-p b731e000 00:00 0 
b744d000-b7462000 r-xp 00000000 08:04 1910133   
/lib/tls/i686/cmov/libpthread-2.8.90.so
b7462000-b7463000 r--p 00014000 08:04 1910133   
/lib/tls/i686/cmov/libpthread-2.8.90.so
b7463000-b7464000 rw-p 00015000 08:04 1910133   
/lib/tls/i686/cmov/libpthread-2.8.90.so
b7464000-b7466000 rw-p b7464000 00:00 0 
b7466000-b746a000 r-xp 00000000 08:04 2181398    /usr/lib/libXdmcp.so.6.0.0
b746a000-b746b000 rw-p 00003000 08:04 2181398    /usr/lib/libXdmcp.so.6.0.0
b746b000-b746c000 rw-p b746b000 00:00 0 
b746c000-b746e000 r-xp 00000000 08:04 2180573    /usr/lib/libXau.so.6.0.0
b746e000-b746f000 rw-p 00001000 08:04 2180573    /usr/lib/libXau.so.6.0.0
b746f000-b7470000 r-xp 00000000 08:04 2181509    /usr/lib/libxcb-xlib.so.0.0.0
b7470000-b7471000 r--p 00000000 08:04 2181509    /usr/lib/libxcb-xlib.so.0.0.0
b7471000-b7472000 rw-p 00001000 08:04 2181509    /usr/lib/libxcb-xlib.so.0.0.0
b7472000-b7475000 r-xp 00000000 08:04 1892947    /lib/libattr.so.1.1.0
b7475000-b7477000 rw-p 00003000 08:04 1892947    /lib/libattr.so.1.1.0
b7477000-b749f000 r-xp 00000000 08:04 1892979    /lib/libpcre.so.3.12.1
b749f000-b74a0000 r--p 00027000 08:04 1892979    /lib/libpcre.so.3.12.1
b74a0000-b74a1000 rw-p 00028000 08:04 1892979    /lib/libpcre.so.3.12.1
b74a1000-b74c5000 r-xp 00000000 08:04 2181526    /usr/lib/libexpat.so.1.5.2
b74c5000-b74c7000 r--p 00023000 08:04 2181526    /usr/lib/libexpat.so.1.5.2
b74c7000-b74c8000 rw-p 00025000 08:04 2181526    /usr/lib/libexpat.so.1.5.2
b74c8000-b74c9000 rw-p b74c8000 00:00 0 
b74c9000-b74e0000 r-xp 00000000 08:04 2181505    /usr/lib/libxcb.so.1.0.0
b74e0000-b74e1000 r--p 00016000 08:04 2181505    /usr/lib/libxcb.so.1.0.0
b74e1000-b74e2000 rw-p 00017000 08:04 2181505    /usr/lib/libxcb.so.1.0.0
b74e2000-b74e8000 r-xp 00000000 08:04 2182517   
/usr/lib/libxcb-render.so.0.0.0
b74e8000-b74e9000 r--p 00005000 08:04 2182517   
/usr/lib/libxcb-render.so.0.0.0
b74e9000-b74ea000 rw-p 00006000 08:04 2182517   
/usr/lib/libxcb-render.so.0.0.0
b74ea000-b74ed000 r-xp 00000000 08:04 2182519   
/usr/lib/libxcb-render-util.so.0.0.0
b74ed000-b74ee000 r--p 00002000 08:04 2182519   
/usr/lib/libxcb-render-util.so.0.0.0
b74ee000-b74ef000 rw-p 00003000 08:04 2182519   
/usr/lib/libxcb-render-util.so.0.0.0
b74ef000-b7513000 r-xp 00000000 08:04 2261047    /usr/lib/libpng12.so.0.27.0
b7513000-b7515000 rw-p 00023000 08:04 2261047    /usr/lib/libpng12.so.0.27.0
b7515000-b7554000 r-xp 00000000 08:04 2179338    /usr/lib/libpixman-1.so.0.12.0
b7554000-b7556000 r--p 0003e000 08:04 2179338    /usr/lib/libpixman-1.so.0.12.0
b7556000-b7557000 rw-p 00040000 08:04 2179338    /usr/lib/libpixman-1.so.0.12.0
b7557000-b7558000 rw-p b7557000 00:00 0 
b7558000-b7560000 r-xp 00000000 08:04 2180407    /usr/lib/libXcursor.so.1.0.2
b7560000-b7561000 rw-p 00007000 08:04 2180407    /usr/lib/libXcursor.so.1.0.2
b7561000-b7566000 r-xp 00000000 08:04 2181117    /usr/lib/libXrandr.so.2.1.0
b7566000-b7567000 r--p 00005000 08:04 2181117    /usr/lib/libXrandr.so.2.1.0
b7567000-b7568000 rw-p 00006000 08:04 2181117    /usr/lib/libXrandr.so.2.1.0
b7568000-b7570000 r-xp 00000000 08:04 2180995    /usr/lib/libXi.so.6.0.0
b7570000-b7571000 r--p 00007000 08:04 2180995    /usr/lib/libXi.so.6.0.0
b7571000-b7572000 rw-p 00008000 08:04 2180995    /usr/lib/libXi.so.6.0.0
b7572000-b7574000 r-xp 00000000 08:04 2180450    /usr/lib/libXinerama.so.1.0.0
b7574000-b7575000 rw-p 00001000 08:04 2180450    /usr/lib/libXinerama.so.1.0.0
b7575000-b757d000 r-xp 00000000 08:04 2181518    /usr/lib/libXrender.so.1.3.0
b757d000-b757e000 r--p 00007000 08:04 2181518    /usr/lib/libXrender.so.1.3.0
b757e000-b757f000 rw-p 00008000 08:04 2181518    /usr/lib/libXrender.so.1.3.0
b757f000-b7580000 rw-p b757f000 00:00 0 
b7580000-b758d000 r-xp 00000000 08:04 2180024    /usr/lib/libXext.so.6.4.0
b758d000-b758f000 rw-p 0000c000 08:04 2180024    /usr/lib/libXext.so.6.4.0
b758f000-b7593000 r-xp 00000000 08:04 2180417    /usr/lib/libXfixes.so.3.1.0
b7593000-b7594000 rw-p 00003000 08:04 2180417    /usr/lib/libXfixes.so.3.1.0
b7594000-b7596000 r-xp 00000000 08:04 2180844    /usr/lib/libXdamage.so.1.1.0
b7596000-b7597000 rw-p 00001000 08:04 2180844    /usr/lib/libXdamage.so.1.1.0
b7597000-b7599000 r-xp 00000000 08:04 2181516   
/usr/lib/libXcomposite.so.1.0.0
b7599000-b759a000 r--p 00001000 08:04 2181516   
/usr/lib/libXcomposite.so.1.0.0
b759a000-b759b000 rw-p 00002000 08:04 2181516   
/usr/lib/libXcomposite.so.1.0.0
b759b000-b75b0000 r-xp 00000000 08:04 2180364    /usr/lib/libICE.so.6.3.0
b75b0000-b75b1000 rw-p 00014000 08:04 2180364    /usr/lib/libICE.so.6.3.0
b75b1000-b75b4000 rw-p b75b1000 00:00 0 
b75b4000-b75bb000 r-xp 00000000 08:04 2179449    /usr/lib/libSM.so.6.0.0
b75bb000-b75bc000 r--p 00006000 08:04 2179449    /usr/lib/libSM.so.6.0.0
b75bc000-b75bd000 rw-p 00007000 08:04 2179449    /usr/lib/libSM.so.6.0.0
b75bd000-b75bf000 r-xp 00000000 08:04 1910111   
/lib/tls/i686/cmov/libdl-2.8.90.so
b75bf000-b75c0000 r--p 00001000 08:04 1910111   
/lib/tls/i686/cmov/libdl-2.8.90.so
b75c0000-b75c1000 rw-p 00002000 08:04 1910111   
/lib/tls/i686/cmov/libdl-2.8.90.so
b75c1000-b76ac000 r-xp 00000000 08:04 2181512    /usr/lib/libX11.so.6.2.0
b76ac000-b76ad000 r--p 000ea000 08:04 2181512    /usr/lib/libX11.so.6.2.0
b76ad000-b76af000 rw-p 000eb000 08:04 2181512    /usr/lib/libX11.so.6.2.0
b76af000-b76b0000 rw-p b76af000 00:00 0 
b76b0000-b7808000 r-xp 00000000 08:04 1910105   
/lib/tls/i686/cmov/libc-2.8.90.so
b7808000-b780a000 r--p 00158000 08:04 1910105   
/lib/tls/i686/cmov/libc-2.8.90.so
b780a000-b780b000 rw-p 0015a000 08:04 1910105   
/lib/tls/i686/cmov/libc-2.8.90.so
b780b000-b780e000 rw-p b780b000 00:00 0 
b780e000-b7813000 r-xp 00000000 08:04 2179465    /usr/lib/libgpm.so.2.0.0
b7813000-b7814000 r--p 00004000 08:04 2179465    /usr/lib/libgpm.so.2.0.0
b7814000-b7815000 rw-p 00005000 08:04 2179465    /usr/lib/libgpm.so.2.0.0
b7815000-b781b000 r-xp 00000000 08:04 1892950    /lib/libacl.so.1.1.0
b781b000-b781d000 rw-p 00005000 08:04 1892950    /lib/libacl.so.1.1.0
b781d000-b781e000 rw-p b781d000 00:00 0 
b781e000-b7836000 r-xp 00000000 08:04 1892478    /lib/libselinux.so.1
b7836000-b7837000 r--p 00017000 08:04 1892478    /lib/libselinux.so.1
b7837000-b7838000 rw-p 00018000 08:04 1892478    /lib/libselinux.so.1
b7838000-b7865000 r-xp 00000000 08:04 1892432    /lib/libncurses.so.5.6
b7865000-b7868000 rw-p 0002c000 08:04 1892432    /lib/libncurses.so.5.6
b7868000-b788c000 r-xp 00000000 08:04 1910113   
/lib/tls/i686/cmov/libm-2.8.90.so
b788c000-b788d000 r--p 00023000 08:04 1910113   
/lib/tls/i686/cmov/libm-2.8.90.so
b788d000-b788e000 rw-p 00024000 08:04 1910113   
/lib/tls/i686/cmov/libm-2.8.90.so
b788e000-b78db000 r-xp 00000000 08:04 2180441    /usr/lib/libXt.so.6.0.0
b78db000-b78df000 rw-p 0004c000 08:04 2180441    /usr/lib/libXt.so.6.0.0
b78df000-b7994000 r-xp 00000000 08:04 2180146   
/usr/lib/libglib-2.0.so.0.1800.2
b7994000-b7995000 r--p 000b4000 08:04 2180146   
/usr/lib/libglib-2.0.so.0.1800.2
b7995000-b7996000 rw-p 000b5000 08:04 2180146   
/usr/lib/libglib-2.0.so.0.1800.2
b7996000-b7999000 r-xp 00000000 08:04 2180155   
/usr/lib/libgmodule-2.0.so.0.1800.2
b7999000-b799a000 r--p 00002000 08:04 2180155   
/usr/lib/libgmodule-2.0.so.0.1800.2
b799a000-b799b000 rw-p 00003000 08:04 2180155   
/usr/lib/libgmodule-2.0.so.0.1800.2
b799b000-b799c000 rw-p b799b000 00:00 0 
b799c000-b79d8000 r-xp 00000000 08:04 2180173   
/usr/lib/libgobject-2.0.so.0.1800.2
b79d8000-b79d9000 r--p 0003b000 08:04 2180173   
/usr/lib/libgobject-2.0.so.0.1800.2
b79d9000-b79da000 rw-p 0003c000 08:04 2180173   
/usr/lib/libgobject-2.0.so.0.1800.2
b79da000-b7a05000 r-xp 00000000 08:04 2179260   
/usr/lib/libfontconfig.so.1.3.0
b7a05000-b7a06000 r--p 0002a000 08:04 2179260   
/usr/lib/libfontconfig.so.1.3.0
Vim: Caught deadly signal ABRT0 08:04 2179260    /usr/lib/lib
Vim: Finished.
Aborted (core dumped)


When running vim under valgrind, the the following error is reported
*only* when compiling with -O3  (or -O2 -finline-functions):

$ valgrind ./vim
==16167== Memcheck, a memory error detector.
==16167== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.
==16167== Using LibVEX rev 1854, a library for dynamic binary translation.
==16167== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.
==16167== Using valgrind-3.3.1-Debian, a dynamic binary instrumentation
framework.
==16167== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.
==16167== For more details, rerun with: -v
==16167== 
**16167** *** strcpy_chk: buffer overflow detected ***: program terminated
==16167==    at 0x4027871: VALGRIND_PRINTF_BACKTRACE (valgrind.h:3695)
==16167==    by 0x4027A37: __strcpy_chk (mc_replace_strmem.c:614)
==16167==    by 0x8081E8C: call_user_func (in /tmp/vim7/src/vim)
==16167==    by 0x8083520: call_func (in /tmp/vim7/src/vim)
==16167==    by 0x808ABEB: get_func_tv (in /tmp/vim7/src/vim)
==16167==    by 0x80950E0: ex_call (in /tmp/vim7/src/vim)
==16167==    by 0x80B4BA7: do_one_cmd (in /tmp/vim7/src/vim)
==16167==    by 0x80B72BA: do_cmdline (in /tmp/vim7/src/vim)
==16167==    by 0x80A6FFA: do_source (in /tmp/vim7/src/vim)
==16167==    by 0x80A7660: source_callback (in /tmp/vim7/src/vim)
==16167==    by 0x80A4B13: do_in_runtimepath (in /tmp/vim7/src/vim)
==16167==    by 0x80A4C17: source_runtime (in /tmp/vim7/src/vim)
==16167== 
==16167== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 103 from 1)
==16167== malloc/free: in use at exit: 1,688,545 bytes in 11,584 blocks.
==16167== malloc/free: 23,040 allocs, 11,456 frees, 3,315,246 bytes allocated.
==16167== For counts of detected errors, rerun with: -v
==16167== searching for pointers to 11,584 not-freed blocks.
==16167== checked 2,073,324 bytes.
==16167== 
==16167== LEAK SUMMARY:
==16167==    definitely lost: 0 bytes in 0 blocks.
==16167==      possibly lost: 6,006 bytes in 323 blocks.
==16167==    still reachable: 1,682,539 bytes in 11,261 blocks.
==16167==         suppressed: 0 bytes in 0 blocks.
==16167== Rerun with --leak-check=full to see details of leaked memory.


Notice also that gcc reports the following warning when compiling eval.c which
match the location of the crash:

gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_GTK  -I/usr/include/gtk-2.0
-I/usr/lib/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo
-I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include
-I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12    
-O3         -o objects/eval.o eval.c
In function 'strcpy',
    inlined from 'add_nr_var' at eval.c:21343,
    inlined from 'call_user_func' at eval.c:21097:
/usr/include/bits/string3.h:106: warning: call to __builtin___strcpy_chk will
always overflow destination buffer
In function 'strcpy',
    inlined from 'add_nr_var' at eval.c:21343,
    inlined from 'call_user_func' at eval.c:21115:
/usr/include/bits/string3.h:106: warning: call to __builtin___strcpy_chk will
always overflow destination buffer
In function 'strcpy',
    inlined from 'add_nr_var' at eval.c:21343,
    inlined from 'call_user_func' at eval.c:21117:
/usr/include/bits/string3.h:106: warning: call to __builtin___strcpy_chk will
always overflow destination buffer


Vim crashes in strcpy(...) in vim7/src/eval.c:21343

21097     add_nr_var(&fc.l_avars, &fc.fixvar[fixvar_idx++].var, "0",
21098                                 (varnumber_T)(argcount -
fp->uf_args.ga_len));

.....

21336     static void
21337 add_nr_var(dp, v, name, nr)
21338     dict_T      *dp;
21339     dictitem_T  *v;
21340     char        *name;
21341     varnumber_T nr;
21342 {
21343     STRCPY(v->di_key, name);

The warning happens because v->di_key seems to be pointing
to a buffer of 1 char, but more than 1 char are being written.

But looking at the code, gcc warnings is spurious: Vim code
is just using the classic pattern described in:

http://gcc.gnu.org/onlinedocs/gcc-4.3.2/gcc/Zero-Length.html#Zero-Length


I found that the following patch to the vim code makes it work
when compiled with -O3, but I think it's just a workaround for
a bug in gcc.


RCS file: /cvsroot/vim/vim7/src/structs.h,v
retrieving revision 1.81
diff -c -r1.81 structs.h
*** structs.h   9 Nov 2008 12:45:25 -0000       1.81
--- structs.h   15 Nov 2008 09:55:01 -0000
***************
*** 1093,1099 ****
  {
      typval_T  di_tv;          /* type and value of the variable */
      char_u    di_flags;       /* flags (only used for variable) */
!     char_u    di_key[1];      /* key (actually longer!) */
  };

  typedef struct dictitem_S dictitem_T;
--- 1093,1105 ----
  {
      typval_T  di_tv;          /* type and value of the variable */
      char_u    di_flags;       /* flags (only used for variable) */
! #ifdef __GNUC__
!     /* Declaring di_key[] instead of di_key[1] prevents crashes when
!      * compiling with gcc -O3 */
!     char_u    di_key[];       /* key (actually longer!) */
! #else
!     char_u    di_key[1];      /* key (actually longer!) */
! #endif
  };

  typedef struct dictitem_S dictitem_T;



The bug is also discussed in vim_use mailing list:

http://groups.google.com/group/vim_dev/browse_thread/thread/1d4e5a65f0afe223


-- 
           Summary: vim crashes on startup when compiled with -O3 but works
                    with -O2
           Product: gcc
           Version: 4.3.2
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: dominique dot pelle at gmail dot com


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38136


^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2008-11-16 14:08 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-11-15 18:06 [Bug c/38136] New: vim crashes on startup when compiled with -O3 but works with -O2 dominique dot pelle at gmail dot com
2008-11-15 18:43 ` Andrew Thomas Pinski
2008-11-15 18:44 ` [Bug c/38136] " pinskia at gmail dot com
2008-11-15 20:35 ` rguenth at gcc dot gnu dot org
2008-11-15 20:42 ` rguenth at gcc dot gnu dot org
2008-11-15 21:23 ` dominique dot pelle at gmail dot com
2008-11-15 21:42 ` rguenth at gcc dot gnu dot org
2008-11-15 23:01 ` jakub at gcc dot gnu dot org
2008-11-16 13:49 ` dominique dot pelle at gmail dot com
2008-11-16 14:08 ` dominique dot pelle at gmail dot com

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).