[Bug sanitizer/59302] New: tsan: Unexpected mmap in InternalAllocator!

[Bug sanitizer/59302] New: tsan: Unexpected mmap in InternalAllocator!

[Joost.VandeVondele at mat dot ethz.ch]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59302

            Bug ID: 59302
           Summary: tsan: Unexpected mmap in InternalAllocator!
           Product: gcc
           Version: 4.9.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: sanitizer
          Assignee: unassigned at gcc dot gnu.org
          Reporter: Joost.VandeVondele at mat dot ethz.ch
                CC: dodji at gcc dot gnu.org, dvyukov at gcc dot gnu.org,
                    jakub at gcc dot gnu.org, kcc at gcc dot gnu.org

One tsan test I'm doing fails with:

Unexpected mmap in InternalAllocator

apart from expecting a somewhat more useful output (e.g. 
Sanitizer internal error: Unexpected mmap in InternalAllocator  
)

Maybe this shouldn't happen ?

I'm getting this with the following backtrace:

[Switching to Thread 0x7ffff03ce100 (LWP 17517)]

Breakpoint 1, Allocate (stat=<optimized out>, alignment=<optimized out>,
size=<optimized out>, this=<optimized out>)
    at ../../../../gcc/libsanitizer/sanitizer_common/sanitizer_allocator.h:948
948        MapUnmapCallback().OnMap(map_beg, map_size);
(gdb) bt
#0  Allocate (stat=<optimized out>, alignment=<optimized out>, size=<optimized
out>, this=<optimized out>)
    at ../../../../gcc/libsanitizer/sanitizer_common/sanitizer_allocator.h:948
#1  Allocate (cleared=<optimized out>, alignment=<optimized out>,
size=<optimized out>, cache=<optimized out>, this=<optimized out>)
    at ../../../../gcc/libsanitizer/sanitizer_common/sanitizer_allocator.h:1187
#2  RawInternalAlloc (cache=0x7ffff039e4c8, size=131080) at
../../../../gcc/libsanitizer/sanitizer_common/sanitizer_allocator.cc:75
#3  __sanitizer::InternalAlloc (size=131072, cache=0x7ffff039e4c8) at
../../../../gcc/libsanitizer/sanitizer_common/sanitizer_allocator.cc:93
#4  0x00007ffff35a6c6f in EnsureSize (size=4097, this=0x7ffff37dc310
<__tsan::ctx_placeholder+64656>) at
../../../../gcc/libsanitizer/tsan/tsan_vector.h:98
#5  PushBack (v=..., this=0x7ffff37dc310 <__tsan::ctx_placeholder+64656>) at
../../../../gcc/libsanitizer/tsan/tsan_vector.h:60
#6  HandleRacyStacks (thr=<optimized out>, addr_max=<optimized out>,
addr_min=<optimized out>, traces=...)
    at ../../../../gcc/libsanitizer/tsan/tsan_rtl_report.cc:487
#7  __tsan::ReportRace (thr=<optimized out>) at
../../../../gcc/libsanitizer/tsan/tsan_rtl_report.cc:676
#8  0x00007ffff35a9e82 in __tsan_report_race_thunk () at
../../../../gcc/libsanitizer/tsan/tsan_rtl_amd64.S:122
#9  0x00007ffff3577a48 in HandleRace (old=..., cur=..., shadow_mem=<optimized
out>, thr=<optimized out>) at ../../../../gcc/libsanitizer/tsan/tsan_rtl.cc:376
#10 MemoryAccessImpl (cur=..., shadow_mem=<optimized out>, kIsAtomic=<optimized
out>, kAccessIsWrite=<optimized out>, kAccessSizeLog=<optimized out>, 
    addr=<optimized out>, thr=<optimized out>) at
../../../../gcc/libsanitizer/tsan/tsan_rtl.cc:460
#11 __tsan::MemoryAccess (thr=0x7ffff035b780, pc=542014296,
addr=3378151234108248, kAccessSizeLog=8, kAccessIsWrite=true, kIsAtomic=true)
    at ../../../../gcc/libsanitizer/tsan/tsan_rtl.cc:531
#12 0x0004006f23be9d58 in ?? ()
#13 0x00000fffc0d69de0 in ?? ()
#14 0x0000000000000008 in ?? ()
#15 0x0000000000000000 in ?? ()
(gdb) cont
Continuing.
Unexpected mmap in InternalAllocator![Thread 0x7ffff03ce100 (LWP 17517) exited]
[Thread 0x7ffff0462100 (LWP 17516) exited]
[Thread 0x7ffff4cf6100 (LWP 17515) exited]
[Thread 0x7fffeefff700 (LWP 17514) exited]
[Inferior 1 (process 17510) exited with code 01]

[Bug sanitizer/59302] tsan: Unexpected mmap in InternalAllocator!

[kcc at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59302

--- Comment #1 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
This isn't expected to happen and so we did not write a better warning message.
We may be able to fix it, but the underlying problem is in your tests: 
it has a race that tsan is trying to report and which is suppressed
(probably because a similar races was reported before).

How many race reports do you see before tsan crashes? 
Did you suppress any of those?

[Bug sanitizer/59302] tsan: Unexpected mmap in InternalAllocator!

[Joost.VandeVondele at mat dot ethz.ch]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59302

Joost VandeVondele <Joost.VandeVondele at mat dot ethz.ch> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |Joost.VandeVondele at mat dot ethz
                   |                            |.ch

--- Comment #2 from Joost VandeVondele <Joost.VandeVondele at mat dot ethz.ch> ---
(In reply to Kostya Serebryany from comment #1)
> This isn't expected to happen and so we did not write a better warning
> message.
> We may be able to fix it, but the underlying problem is in your tests: 
> it has a race that tsan is trying to report and which is suppressed
> (probably because a similar races was reported before).
> 
> How many race reports do you see before tsan crashes? 
> Did you suppress any of those?

yes, I have a suppression in place to workaround PR59194. As it is an atomic
update in a hot loop, it will be triggered millions of times.

[Bug sanitizer/59302] tsan: Unexpected mmap in InternalAllocator!

[kcc at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59302

--- Comment #3 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
(In reply to Joost VandeVondele from comment #2)
> (In reply to Kostya Serebryany from comment #1)
> > This isn't expected to happen and so we did not write a better warning
> > message.
> > We may be able to fix it, but the underlying problem is in your tests: 
> > it has a race that tsan is trying to report and which is suppressed
> > (probably because a similar races was reported before).
> > 
> > How many race reports do you see before tsan crashes? 
> > Did you suppress any of those?
> 
> yes, I have a suppression in place to workaround PR59194. As it is an atomic
> update in a hot loop, it will be triggered millions of times.

Suppressing a race that happens so many times is no good. 
Even if we fix the crash above (which I'd prefer not to; instead we should emit
a more descriptive message and die; we'll do that) tsan will remain very slow. 
The right solution is of course to fix the code to not have that race.
The next good solution is to annotate the function with
__attribute__((no_sanitize_thread)) -- but I don't know if fortran has anything
like this. 
The next somewhat good solution is to blacklist the function
(https://code.google.com/p/thread-sanitizer/wiki/Flags?ts=1385538776&updated=Flags#Blacklist_Format)
but that is not supported by GCC and will have to be implemented first.

[Bug sanitizer/59302] tsan: Unexpected mmap in InternalAllocator!

[dvyukov at google dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59302

Dmitry Vyukov <dvyukov at google dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dvyukov at google dot com

--- Comment #4 from Dmitry Vyukov <dvyukov at google dot com> ---
The referenced issue https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59194
is marked as fixed. I propose to fix this issue, but I don't see an option to
change status.