[Bug sanitizer/61095] New: tsan is broken in gcc trunk, works in 4.9

[Bug sanitizer/61095] New: tsan is broken in gcc trunk, works in 4.9

[kcc at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

            Bug ID: 61095
           Summary: tsan is broken in gcc trunk, works in 4.9
           Product: gcc
           Version: 4.10.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: sanitizer
          Assignee: unassigned at gcc dot gnu.org
          Reporter: kcc at gcc dot gnu.org
                CC: dodji at gcc dot gnu.org, dvyukov at gcc dot gnu.org,
                    jakub at gcc dot gnu.org, kcc at gcc dot gnu.org

tsan seems to be broken in the current gcc trunk
(Tested r210145 on Linux ubuntu 12.04). 
gcc 4.9.0 works fine. 

There have been no recent libsanitizer merges from upstream so the problem
was likely introduced somewhere in the buildfiles, etc


% /usr/local/gcc-4.9.0/bin/g++ -g -fsanitize=thread -fPIE -pie -static-libtsan
simple_race.cc  && ./a.out 
==================
WARNING: ThreadSanitizer: data race (pid=31025)
...

% ../gcc-inst/bin/g++ -g -fsanitize=thread -fPIE -pie -static-libtsan
simple_race.cc  && ./a.out 
FATAL: ThreadSanitizer CHECK failed:
../../../../gcc/libsanitizer/tsan/tsan_rtl.cc:587 "((IsShadowMem((uptr)(p +
size * kShadowCnt / kShadowCell - 1)))) != (0)" (0x0, 0x0)
    #0 __tsan::PrintCurrentStackSlow()
../../../../gcc/libsanitizer/tsan/tsan_rtl_report.cc:710 (a.out+0x00000005840c)
    #1 __tsan::TsanCheckFailed(char const*, int, char const*, unsigned long
long, unsigned long long)
../../../../gcc/libsanitizer/tsan/tsan_rtl_report.cc:39 (a.out+0x0000000584e2)
    #2 __sanitizer::CheckFailed(char const*, int, char const*, unsigned long
long, unsigned long long)
../../../../gcc/libsanitizer/sanitizer_common/sanitizer_common.cc:74
(a.out+0x00000005ea93)
    #3 MemoryRangeSet ../../../../gcc/libsanitizer/tsan/tsan_rtl.cc:587
(a.out+0x00000001e1d6)
    #4 __tsan::user_alloc(__tsan::ThreadState*, unsigned long, unsigned long,
unsigned long) ../../../../gcc/libsanitizer/tsan/tsan_mman.cc:113
(a.out+0x00000005403a)
    #5 calloc ../../../../gcc/libsanitizer/tsan/tsan_interceptors.cc:500
(a.out+0x00000003d4d4)
    #6 _dl_allocate_tls <null>:0 (ld-linux-x86-64.so.2+0x000000012074)
    #7 __pthread_create_2_1 <null>:0 (libpthread.so.0+0x000000008abc)
    #8 pthread_create
../../../../gcc/libsanitizer/tsan/tsan_interceptors.cc:891
(a.out+0x00000003b758)
    #9 main /home/kcc/tmp/simple_race.cc:23 (a.out+0x000000079473)
    #10 __libc_start_main <null>:0 (libc.so.6+0x00000002176c)
    #11 <null> <null>:0 (a.out+0x000000010694)


% cat simple_race.cc 
#include <pthread.h>
int Global;
void *Thread1(void *x) {
  Global++;
  return NULL;
}
int main() {
  pthread_t t;
  pthread_create(&t, NULL, Thread1, NULL);
  Global++;
  pthread_join(t, NULL);
}
%

[Bug sanitizer/61095] tsan is broken in gcc trunk, works in 4.9

[kcc at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

--- Comment #1 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
I suspect a miscompile,
at least the following patch to the libsanitizer sources fixes the problem:

--- tsan_rtl.cc (revision 210145)
+++ tsan_rtl.cc (working copy)
@@ -584,7 +584,8 @@
   if (kGoMode || size < 64*1024) {
     u64 *p = (u64*)MemToShadow(addr);
     CHECK(IsShadowMem((uptr)p));
-    CHECK(IsShadowMem((uptr)(p + size * kShadowCnt / kShadowCell - 1)));
+    volatile uptr xxx = (uptr)(p + size * kShadowCnt / kShadowCell - 1);
+    CHECK(IsShadowMem(xxx));
     // FIXME: may overwrite a part outside the region
     for (uptr i = 0; i < size / kShadowCell * kShadowCnt;) {
       p[i++] = val;


Note the "volatile". W/o volatile tsan still crashes.

[Bug sanitizer/61095] [4.10 Regression] tsan is broken in gcc trunk, works in 4.9

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Target|                            |x86_64-*-*
   Target Milestone|---                         |4.10.0
            Summary|tsan is broken in gcc       |[4.10 Regression] tsan is
                   |trunk, works in 4.9         |broken in gcc trunk, works
                   |                            |in 4.9

[Bug sanitizer/61095] miscompile: tsan is broken in gcc trunk, works in 4.9

[kcc at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

Kostya Serebryany <kcc at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Target|x86_64-*-*                  |
           Priority|P3                          |P1
   Target Milestone|4.10.0                      |---
            Summary|[4.10 Regression] tsan is   |miscompile: tsan is broken
                   |broken in gcc trunk, works  |in gcc trunk, works in 4.9
                   |in 4.9                      |

--- Comment #2 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
fresh gcc optimizes away a piece of code. minimal repro:

int FOO(unsigned long addr) {
    unsigned long *p = (unsigned long*)((addr & 0xffff83fffffffff8UL) * 4);
    unsigned long xxx = (unsigned long)(p + 1);
    return xxx >= 0x3c000000000UL;
}


gcc 4.9: 
/usr/local/gcc-4.9.0/bin/gcc -S z.c -O2  -o -  | grep -A 10 FOO:
FOO:
.LFB0:
        .cfi_startproc
        movabsq $-136339441844232, %rax
        andq    %rax, %rdi
        movabsq $4123168604159, %rax
        leaq    8(,%rdi,4), %rdx
        cmpq    %rax, %rdx
        seta    %al
        movzbl  %al, %eax
        ret


trunk: 
../gcc-inst/bin/gcc -S z.c -O2  -o -  | grep -A 10 FOO:
FOO:
.LFB0:
        .cfi_startproc
        xorl    %eax, %eax
        ret

[Bug sanitizer/61095] [4.10 Regression] miscompile: tsan is broken in gcc trunk, works in 4.9

[kcc at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

Kostya Serebryany <kcc at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Target|                            |x86_64-*-*
   Target Milestone|---                         |4.10.0
            Summary|miscompile: tsan is broken  |[4.10 Regression]
                   |in gcc trunk, works in 4.9  |miscompile: tsan is broken
                   |                            |in gcc trunk, works in 4.9

[Bug sanitizer/61095] [4.10 Regression] miscompile: tsan is broken in gcc trunk, works in 4.9

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2014-05-07
     Ever confirmed|0                           |1

--- Comment #3 from Richard Biener <rguenth at gcc dot gnu.org> ---
Confirmed.

[Bug sanitizer/61095] [4.10 Regression] miscompile: tsan is broken in gcc trunk, works in 4.9

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

--- Comment #4 from Richard Biener <rguenth at gcc dot gnu.org> ---
CCP2 does this to

  <bb 2>:
  _2 = addr_1(D) & 18446607734267707384;
  _3 = _2 * 4;
  p_4 = (long unsigned int *) _3;
  _5 = p_4 + 8;
  xxx_6 = (long unsigned int) _5;
  _7 = xxx_6 > 4123168604159;
  _8 = (int) _7;
  return _8;

but not CCP1 to

  <bb 2>:
  _2 = addr_1(D) & 18446607734267707384;
  _3 = _2 * 4;
  p_4 = (long unsigned int *) _3;
  _5 = p_4 + 8;
  xxx_6 = (long unsigned int) _5;
  _7 = xxx_6 > 4123168604159;
  _8 = (int) _7;
  return _8;

that's weird.

[Bug sanitizer/61095] [4.10 Regression] miscompile: tsan is broken in gcc trunk, works in 4.9

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

--- Comment #5 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Started with r210113.

[Bug sanitizer/61095] [4.10 Regression] miscompile: tsan is broken in gcc trunk, works in 4.9

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

--- Comment #6 from Richard Biener <rguenth at gcc dot gnu.org> ---
Diffs:

@@ -53,31 +53,31 @@
 Visiting statement:
 p_4 = (long unsigned int *) _3;
 which is likely CONSTANT
-Lattice value changed to CONSTANT 0x0 (0x0ffffffffffffffe0).  Adding SSA edges 
to worklist.
+Lattice value changed to CONSTANT 0x0 (0xffffffe0).  Adding SSA edges to
workli
st.
 adding stmt 3 to worklist

...
 Visiting statement:
 xxx_6 = (long unsigned int) _5;
 which is likely CONSTANT
-Lattice value changed to CONSTANT 0x8 (0x0ffffffffffffffe0).  Adding SSA edges
to worklist.
+Lattice value changed to CONSTANT 0x8 (0xffffffe0).  Adding SSA edges to
worklist.
 adding stmt 5 to worklist
...
 Visiting statement:
 _7 = xxx_6 > 4123168604159;
 which is likely CONSTANT
-Lattice value changed to CONSTANT 0x0 (0x1).  Adding SSA edges to worklist.
+Lattice value changed to CONSTANT 0.  Adding SSA edges to worklist.
 adding stmt 6 to worklist


weird.

[Bug sanitizer/61095] [4.10 Regression] miscompile: tsan is broken in gcc trunk, works in 4.9

[rsandifo at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

rsandifo at gcc dot gnu.org <rsandifo at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
                 CC|                            |rsandifo at gcc dot gnu.org
           Assignee|unassigned at gcc dot gnu.org      |rsandifo at gcc dot gnu.org

--- Comment #7 from rsandifo at gcc dot gnu.org <rsandifo at gcc dot gnu.org> ---
Patch part way through testing.  It was a silly
"int | unsigned int -> HOST_WIDE_INT" extension bug.

[Bug sanitizer/61095] [4.10 Regression] miscompile: tsan is broken in gcc trunk, works in 4.9

[rsandifo at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

--- Comment #8 from rsandifo at gcc dot gnu.org <rsandifo at gcc dot gnu.org> ---
Author: rsandifo
Date: Wed May  7 18:00:59 2014
New Revision: 210181

URL: http://gcc.gnu.org/viewcvs?rev=210181&root=gcc&view=rev
Log:
gcc/
    PR tree-optimization/61095
    * tree-ssanames.c (get_nonzero_bits): Fix type extension in wi::shwi.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/tree-ssanames.c

[Bug sanitizer/61095] [4.10 Regression] miscompile: tsan is broken in gcc trunk, works in 4.9

[kcc at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

--- Comment #9 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
I confirm that the minimized reproducer is fixed, but tsan is still crashing on
the same CHECK:

../gcc-inst/bin/g++ -g -fsanitize=thread -fPIE -pie -static-libtsan
simple_race.cc  && ./a.out 
FATAL: ThreadSanitizer CHECK failed:
../../../../gcc/libsanitizer/tsan/tsan_rtl.cc:587 "((IsShadowMem((uptr)(p +
size * kShadowCnt / kShadowCell - 1)))) != (0)" (0x0, 0x0)
    #0 __tsan::PrintCurrentStackSlow()
../../../../gcc/libsanitizer/tsan/tsan_rtl_report.cc:710 (a.out+0x00000005840c)
    #1 __tsan::TsanCheckFailed(char const*, int, char const*, unsigned long
long, unsigned long long)
../../../../gcc/libsanitizer/tsan/tsan_rtl_report.cc:39 (a.out+0x0000000584e2)
    #2 __sanitizer::CheckFailed(char const*, int, char const*, unsigned long
long, unsigned long long)
../../../../gcc/libsanitizer/sanitizer_common/sanitizer_common.cc:74
(a.out+0x00000005ea93)
    #3 MemoryRangeSet ../../../../gcc/libsanitizer/tsan/tsan_rtl.cc:587
(a.out+0x00000001e1d6)

Maybe there is another similar issue somewhere nearby

[Bug sanitizer/61095] [4.10 Regression] miscompile: tsan is broken in gcc trunk, works in 4.9

[kcc at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

--- Comment #10 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
also, just curious: are such fixes supposed to be accompanied with regression
tests?

[Bug sanitizer/61095] [4.10 Regression] miscompile: tsan is broken in gcc trunk, works in 4.9

[kcc at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

--- Comment #11 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
ignore comment #9, everything seems to work now. 
This passes: 

make -j 40 -C gcc check-g{cc,++} 
RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} tsan.exp'

[Bug sanitizer/61095] [4.10 Regression] miscompile: tsan is broken in gcc trunk, works in 4.9

[rsandifo at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

--- Comment #12 from rsandifo at gcc dot gnu.org <rsandifo at gcc dot gnu.org> ---
(In reply to Kostya Serebryany from comment #10)
> also, just curious: are such fixes supposed to be accompanied with
> regression tests?

You're right of course.  I'll send a testsuite patch soon.

[Bug sanitizer/61095] [4.10 Regression] miscompile: tsan is broken in gcc trunk, works in 4.9

[rsandifo at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

--- Comment #13 from rsandifo at gcc dot gnu.org <rsandifo at gcc dot gnu.org> ---
Author: rsandifo
Date: Thu May  8 08:00:21 2014
New Revision: 210203

URL: http://gcc.gnu.org/viewcvs?rev=210203&root=gcc&view=rev
Log:
gcc/testsuite/
    PR tree-optimization/61095
    * gcc.dg/torture/pr61095.c: New test.

Added:
    trunk/gcc/testsuite/gcc.dg/torture/pr61095.c
Modified:
    trunk/gcc/testsuite/ChangeLog

[Bug sanitizer/61095] [4.10 Regression] miscompile: tsan is broken in gcc trunk, works in 4.9

[rsandifo at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61095

rsandifo at gcc dot gnu.org <rsandifo at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #14 from rsandifo at gcc dot gnu.org <rsandifo at gcc dot gnu.org> ---
Fixed.