Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Bernd Edlinger]

On 15/02/16 09:07, Tom de Vries wrote:
>>On 15/02/16 08:24, Dmitry Vyukov wrote:
>>
>>    If we are talking about pr 68580, then I would try:
>>    https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68580#c2
>>    first.
>
> As I tried to explain in the follow-up comment ( https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68580#c3 ),
> since unfortunately I have no reliable way of reproducing the failure, there's no defined way to 'try' something.

But your proposed patch is also only guessing.
Sure pthread_create can fail, as malloc and mmap.
But if that is the reason for the failure it would happen
just randomly, everywhere.

Why do you think that only this test case shows the problem?

I think Dmitry's comment may be right on the point.

In pr65400-1.c we have
int v; int q; int o; 

be 4 byte aligned integers.
and at least
v and q share the same 8-byte tsan shadow memory slot.

v and q are modified simultaniously, and each update
can be lost.

The barrier wont help here, as it only synchronizes
accesses on v.

So I think either we change int => long long
or add __attribute__((aligned(8))) to the variable declarations,
to make sure that each of them goes into a different memory
slot.


Regards,
Bernd.

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Dmitry Vyukov]

On Mon, Feb 15, 2016 at 10:07 AM, Bernd Edlinger
<bernd.edlinger@hotmail.de> wrote:
> On 15/02/16 09:07, Tom de Vries wrote:
>>>On 15/02/16 08:24, Dmitry Vyukov wrote:
>>>
>>>    If we are talking about pr 68580, then I would try:
>>>    https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68580#c2
>>>    first.
>>
>> As I tried to explain in the follow-up comment ( https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68580#c3 ),
>> since unfortunately I have no reliable way of reproducing the failure, there's no defined way to 'try' something.
>
> But your proposed patch is also only guessing.

Yeah, that's what I thought.
Agree that general pthread_create failures should affect more tests.

We can do both I guess.

> Sure pthread_create can fail, as malloc and mmap.
> But if that is the reason for the failure it would happen
> just randomly, everywhere.
>
> Why do you think that only this test case shows the problem?
>
> I think Dmitry's comment may be right on the point.
>
> In pr65400-1.c we have
> int v; int q; int o;
>
> be 4 byte aligned integers.
> and at least
> v and q share the same 8-byte tsan shadow memory slot.
>
> v and q are modified simultaniously, and each update
> can be lost.
>
> The barrier wont help here, as it only synchronizes
> accesses on v.
>
> So I think either we change int => long long
> or add __attribute__((aligned(8))) to the variable declarations,
> to make sure that each of them goes into a different memory
> slot.
>
>
> Regards,
> Bernd.
>
>

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Tom de Vries]

On 15/02/16 10:07, Bernd Edlinger wrote:
> On 15/02/16 09:07, Tom de Vries wrote:
>>> >>On 15/02/16 08:24, Dmitry Vyukov wrote:
>>> >>
>>> >>    If we are talking about pr 68580, then I would try:
>>> >>    https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68580#c2
>>> >>    first.
>> >
>> >As I tried to explain in the follow-up comment (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68580#c3  ),
>> >since unfortunately I have no reliable way of reproducing the failure, there's no defined way to 'try' something.

> But your proposed patch is also only guessing.

I've tried to be as clear as possible in the RFC submission that I'm not 
certain about the cause of the failure, and that the patch is proposing 
a fix that would make that guessed failure cause explicit.

> Sure pthread_create can fail, as malloc and mmap.
> But if that is the reason for the failure it would happen
> just randomly, everywhere.
>
> Why do you think that only this test case shows the problem?
>

As I explained in the RFC submission, my reasoning there was that the 
test is one of the very few test cases that tests the result of 
pthread_create and then returns 0, which causes the failure in 
combination with dg-shouldfail.

But thinking about it some more, even if pthread_create would fail, 
causing the testcase to fail in execution, allowing the execution test 
to pass due to dg-shouldfail, presumably the dg-output test would still 
fail in that case, so my reasoning was not sound.

So I suppose you're right, indeed the pthread_create fail hypothesis is 
not the most logical one.

Still, the patch is an improvement irrespective of the PR that inspired 
it, and perhaps a lot more library calls should be checked for errors 
that just pthread_create.

> I think Dmitry's comment may be right on the point.

If someone proposes that as a patch for the testcase, great. I'm more 
that willing to test that in my setup to be able to claim 'bootstrapped 
and reg-tested on x86_64' in the submission.

I'm just trying to point out that I cannot 'try' out that patch and come 
back with the conformation that 'the patch fixes the failure', given the 
nature of the failure.

Thanks,
- Tom

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Dmitry Vyukov]

On Mon, Feb 15, 2016 at 11:45 AM, Tom de Vries <Tom_deVries@mentor.com> wrote:
> On 15/02/16 10:07, Bernd Edlinger wrote:
>>
>> On 15/02/16 09:07, Tom de Vries wrote:
>>>>
>>>> >>On 15/02/16 08:24, Dmitry Vyukov wrote:
>>>> >>
>>>> >>    If we are talking about pr 68580, then I would try:
>>>> >>    https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68580#c2
>>>> >>    first.
>>>
>>> >
>>> >As I tried to explain in the follow-up comment
>>> > (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68580#c3  ),
>>> >since unfortunately I have no reliable way of reproducing the failure,
>>> > there's no defined way to 'try' something.
>
>
>> But your proposed patch is also only guessing.
>
>
> I've tried to be as clear as possible in the RFC submission that I'm not
> certain about the cause of the failure, and that the patch is proposing a
> fix that would make that guessed failure cause explicit.
>
>> Sure pthread_create can fail, as malloc and mmap.
>> But if that is the reason for the failure it would happen
>> just randomly, everywhere.
>>
>> Why do you think that only this test case shows the problem?
>>
>
> As I explained in the RFC submission, my reasoning there was that the test
> is one of the very few test cases that tests the result of pthread_create
> and then returns 0, which causes the failure in combination with
> dg-shouldfail.
>
> But thinking about it some more, even if pthread_create would fail, causing
> the testcase to fail in execution, allowing the execution test to pass due
> to dg-shouldfail, presumably the dg-output test would still fail in that
> case, so my reasoning was not sound.
>
> So I suppose you're right, indeed the pthread_create fail hypothesis is not
> the most logical one.
>
> Still, the patch is an improvement irrespective of the PR that inspired it,
> and perhaps a lot more library calls should be checked for errors that just
> pthread_create.
>
>> I think Dmitry's comment may be right on the point.
>
>
> If someone proposes that as a patch for the testcase, great. I'm more that
> willing to test that in my setup to be able to claim 'bootstrapped and
> reg-tested on x86_64' in the submission.
>
> I'm just trying to point out that I cannot 'try' out that patch and come
> back with the conformation that 'the patch fixes the failure', given the
> nature of the failure.

Yes, we can't directly test a fix. But s/int/long long/ is still the
right thing to do. We do it in other tests for similar reasons. We can
submit it and see if flakes remain or go away.

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Bernd Edlinger]

[-- Attachment #1: Type: text/plain, Size: 3046 bytes --]

On 15/02/16 08:18, Dmitry Vyukov wrote: 
> llvm tsan tests contain test.h file (probably what's called
> test_barrier.h in gcc), you can put the macro there. test.h should
> already be included into all tests.

Hmm.. as the person who introduced test_barrer.h (well before llvm had a test.h ;)
I must say, that although if gcc was first here, we will  probably change that to
match llvm's implementation for gcc-7.

I would not like to add more differences here without a very good reason.
I'd say, if Dmitry sees a reason to improve the error handling in test.h, that
is a good thing, and should go into llvm's tree first.

And independently of that I am looking at using llvm's test.h framework instead
of gcc's test_barrier.h for gcc-7 soon.

On 15/02/16 11:56, Tom de Vries wrote:
> On Mon, Feb 15, 2016 at 11:45 AM, Tom de Vries <Tom_deVries@mentor.com> wrote:
>>
>> I've tried to be as clear as possible in the RFC submission that I'm not
>> certain about the cause of the failure, and that the patch is proposing a
>> fix that would make that guessed failure cause explicit.
>>
>>> Sure pthread_create can fail, as malloc and mmap.
>>> But if that is the reason for the failure it would happen
>>> just randomly, everywhere.
>>>
>>> Why do you think that only this test case shows the problem?
>>>
>>
>> As I explained in the RFC submission, my reasoning there was that the test
>> is one of the very few test cases that tests the result of pthread_create
>> and then returns 0, which causes the failure in combination with
>> dg-shouldfail.
>>
>> But thinking about it some more, even if pthread_create would fail, causing
>> the testcase to fail in execution, allowing the execution test to pass due
>> to dg-shouldfail, presumably the dg-output test would still fail in that
>> case, so my reasoning was not sound.
>>
>> So I suppose you're right, indeed the pthread_create fail hypothesis is not
>> the most logical one.
>>
>> Still, the patch is an improvement irrespective of the PR that inspired it,
>> and perhaps a lot more library calls should be checked for errors that just
>> pthread_create.
>>
>>> I think Dmitry's comment may be right on the point.
>>
>>
>> If someone proposes that as a patch for the testcase, great. I'm more that
>> willing to test that in my setup to be able to claim 'bootstrapped and
>> reg-tested on x86_64' in the submission.
>>

No problem.  PR65400 was a GCC wrong code bug, so it makes no
sense to have the same test in llvm's tree, thus we are free to fix it on
our own, as we like.

Here is a patch that puts each value on it's own 8-byte aligned memory
location.  From my experience with tsan tests, sharing shadow memory
slots between v and q or o is the most likely explanation for the occasional
inability to spot the race condition on v, thus the test case fails, because
the return code is 0, and the expected output is not found.


Boot-strapped/regression tested on x86_64-linux-gnu.

OK for trunk?


Thanks
Bernd.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch-pr68580.diff --]
[-- Type: text/x-patch; name="patch-pr68580.diff", Size: 575 bytes --]

2016-02-15  Bernd Edlinger  <bernd.edlinger@hotmail.de>

	* c-c++-common/tsan/pr65400-1.c (v, q, o): Make 8-byte aligned.

--- gcc/testsuite/c-c++-common/tsan/pr65400-1.c.jj	2015-03-19 08:53:38.000000000 +0100
+++ gcc/testsuite/c-c++-common/tsan/pr65400-1.c	2016-02-15 11:09:18.852320827 +0100
@@ -7,9 +7,9 @@
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
-int v;
-int q;
-int o;
+int v __attribute__((aligned(8)));
+int q __attribute__((aligned(8)));
+int o __attribute__((aligned(8)));
 extern void baz4 (int *);
 
 __attribute__((noinline, noclone)) int

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Dmitry Vyukov]

On Mon, Feb 15, 2016 at 12:29 PM, Bernd Edlinger
<bernd.edlinger@hotmail.de> wrote:
> On 15/02/16 08:18, Dmitry Vyukov wrote:
>> llvm tsan tests contain test.h file (probably what's called
>> test_barrier.h in gcc), you can put the macro there. test.h should
>> already be included into all tests.
>
> Hmm.. as the person who introduced test_barrer.h (well before llvm had a test.h ;)
> I must say, that although if gcc was first here, we will  probably change that to
> match llvm's implementation for gcc-7.
>
> I would not like to add more differences here without a very good reason.
> I'd say, if Dmitry sees a reason to improve the error handling in test.h, that
> is a good thing, and should go into llvm's tree first.
>
> And independently of that I am looking at using llvm's test.h framework instead
> of gcc's test_barrier.h for gcc-7 soon.
>
> On 15/02/16 11:56, Tom de Vries wrote:
>> On Mon, Feb 15, 2016 at 11:45 AM, Tom de Vries <Tom_deVries@mentor.com> wrote:
>>>
>>> I've tried to be as clear as possible in the RFC submission that I'm not
>>> certain about the cause of the failure, and that the patch is proposing a
>>> fix that would make that guessed failure cause explicit.
>>>
>>>> Sure pthread_create can fail, as malloc and mmap.
>>>> But if that is the reason for the failure it would happen
>>>> just randomly, everywhere.
>>>>
>>>> Why do you think that only this test case shows the problem?
>>>>
>>>
>>> As I explained in the RFC submission, my reasoning there was that the test
>>> is one of the very few test cases that tests the result of pthread_create
>>> and then returns 0, which causes the failure in combination with
>>> dg-shouldfail.
>>>
>>> But thinking about it some more, even if pthread_create would fail, causing
>>> the testcase to fail in execution, allowing the execution test to pass due
>>> to dg-shouldfail, presumably the dg-output test would still fail in that
>>> case, so my reasoning was not sound.
>>>
>>> So I suppose you're right, indeed the pthread_create fail hypothesis is not
>>> the most logical one.
>>>
>>> Still, the patch is an improvement irrespective of the PR that inspired it,
>>> and perhaps a lot more library calls should be checked for errors that just
>>> pthread_create.
>>>
>>>> I think Dmitry's comment may be right on the point.
>>>
>>>
>>> If someone proposes that as a patch for the testcase, great. I'm more that
>>> willing to test that in my setup to be able to claim 'bootstrapped and
>>> reg-tested on x86_64' in the submission.
>>>
>
> No problem.  PR65400 was a GCC wrong code bug, so it makes no
> sense to have the same test in llvm's tree, thus we are free to fix it on
> our own, as we like.
>
> Here is a patch that puts each value on it's own 8-byte aligned memory
> location.  From my experience with tsan tests, sharing shadow memory
> slots between v and q or o is the most likely explanation for the occasional
> inability to spot the race condition on v, thus the test case fails, because
> the return code is 0, and the expected output is not found.
>
>
> Boot-strapped/regression tested on x86_64-linux-gnu.
>
> OK for trunk?


I don't know whether it will fire or not, but 4-byte variables that
are 8-byte aligned can still be collocated with something else. Making
vars 8-byte should be safer.

AW: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Bernd Edlinger]

On 15/02/16 13:05, Dmitry Vyukov wrote:
> On Mon, Feb 15, 2016 at 12:29 PM, Bernd Edlinger
> <bernd.edlinger@hotmail.de> wrote:
>>
>> No problem.  PR65400 was a GCC wrong code bug, so it makes no
>> sense to have the same test in llvm's tree, thus we are free to fix it on
>> our own, as we like.
>>
>> Here is a patch that puts each value on it's own 8-byte aligned memory
>> location.  From my experience with tsan tests, sharing shadow memory
>> slots between v and q or o is the most likely explanation for the occasional
>> inability to spot the race condition on v, thus the test case fails, because
>> the return code is 0, and the expected output is not found.
>>
>>
>> Boot-strapped/regression tested on x86_64-linux-gnu.
>>
>> OK for trunk?
>
>
> I don't know whether it will fire or not, but 4-byte variables that
> are 8-byte aligned can still be collocated with something else. Making
> vars 8-byte should be safer.

Yes, but as PR65400 is a wrong code bug, I would not like to change more
than absolutely necessary to the test case, in order not to loose the ability
to check the original regression.

The test case does not have more than the 3 global values that you see.
nm a.out | sort shows that these are now separate:

0000000000601400 b barrier
0000000000601420 b barrier_wait
0000000000601428 B v
0000000000601430 B o
0000000000601438 B q
0000000000601440 B _end



Regards,
Bernd.

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Dmitry Vyukov]

On Mon, Feb 15, 2016 at 1:44 PM, Bernd Edlinger
<bernd.edlinger@hotmail.de> wrote:
> On 15/02/16 13:05, Dmitry Vyukov wrote:
>> On Mon, Feb 15, 2016 at 12:29 PM, Bernd Edlinger
>> <bernd.edlinger@hotmail.de> wrote:
>>>
>>> No problem.  PR65400 was a GCC wrong code bug, so it makes no
>>> sense to have the same test in llvm's tree, thus we are free to fix it on
>>> our own, as we like.
>>>
>>> Here is a patch that puts each value on it's own 8-byte aligned memory
>>> location.  From my experience with tsan tests, sharing shadow memory
>>> slots between v and q or o is the most likely explanation for the occasional
>>> inability to spot the race condition on v, thus the test case fails, because
>>> the return code is 0, and the expected output is not found.
>>>
>>>
>>> Boot-strapped/regression tested on x86_64-linux-gnu.
>>>
>>> OK for trunk?
>>
>>
>> I don't know whether it will fire or not, but 4-byte variables that
>> are 8-byte aligned can still be collocated with something else. Making
>> vars 8-byte should be safer.
>
> Yes, but as PR65400 is a wrong code bug, I would not like to change more
> than absolutely necessary to the test case, in order not to loose the ability
> to check the original regression.
>
> The test case does not have more than the 3 global values that you see.
> nm a.out | sort shows that these are now separate:
>
> 0000000000601400 b barrier
> 0000000000601420 b barrier_wait
> 0000000000601428 B v
> 0000000000601430 B o
> 0000000000601438 B q
> 0000000000601440 B _end


Looks good to me then.

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Mike Stump]

On Feb 15, 2016, at 3:29 AM, Bernd Edlinger <bernd.edlinger@hotmail.de> wrote:
> And independently of that I am looking at using llvm's test.h framework instead
> of gcc's test_barrier.h for gcc-7 soon.

Here’s to hoping that we don’t back slide on:

  https://gcc.gnu.org/ml/gcc-patches/2015-01/msg00436.html

Did they ever adopt a reliable scheme to test?

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Dmitry Vyukov]

On Mon, Feb 15, 2016 at 8:22 PM, Mike Stump <mikestump@comcast.net> wrote:
> On Feb 15, 2016, at 3:29 AM, Bernd Edlinger <bernd.edlinger@hotmail.de> wrote:
>> And independently of that I am looking at using llvm's test.h framework instead
>> of gcc's test_barrier.h for gcc-7 soon.
>
> Here’s to hoping that we don’t back slide on:
>
>   https://gcc.gnu.org/ml/gcc-patches/2015-01/msg00436.html
>
> Did they ever adopt a reliable scheme to test?

Yes, they did.
Btw, the pthread_barrier solution did not work on macos as it does not
implement pthread_barrier and libpthread.so.0 does not exist. That was
replaced with spin loop with usleep(100). That caused spurious "as if
synchronized via sleep" messages that broke tests. That was replaced
with busy loop. That caused timeouts of weak test bots. That was
replaced with loop with sched_yield. That caused tsan trace overflows
and "failed to restore stack trace" error messages. And that was
replaced with special support in tsan runtime which seems to work in
all contexts so far...

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Tom de Vries]

On 15/02/16 12:29, Bernd Edlinger wrote:
> Here is a patch that puts each value on it's own 8-byte aligned memory
> location.  From my experience with tsan tests, sharing shadow memory
> slots between v and q or o is the most likely explanation for the occasional
> inability to spot the race condition on v, thus the test case fails, because
> the return code is 0, and the expected output is not found.
>
> Boot-strapped/regression tested on x86_64-linux-gnu.
>
> OK for trunk?
>

Hi,

Could you add 'PR testsuite/68580' to the log entry when committing?

Thanks,
- Tom

> patch-pr68580.diff
>
>
> 2016-02-15  Bernd Edlinger<bernd.edlinger@hotmail.de>
>
> 	* c-c++-common/tsan/pr65400-1.c (v, q, o): Make 8-byte aligned.
>
> --- gcc/testsuite/c-c++-common/tsan/pr65400-1.c.jj	2015-03-19 08:53:38.000000000 +0100
> +++ gcc/testsuite/c-c++-common/tsan/pr65400-1.c	2016-02-15 11:09:18.852320827 +0100
> @@ -7,9 +7,9 @@
>   #include "tsan_barrier.h"
>
>   static pthread_barrier_t barrier;
> -int v;
> -int q;
> -int o;
> +int v __attribute__((aligned(8)));
> +int q __attribute__((aligned(8)));
> +int o __attribute__((aligned(8)));
>   extern void baz4 (int *);
>
>   __attribute__((noinline, noclone)) int

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Bernd Edlinger]

On 18.02.2016 12:36, Tom de Vries wrote:
> On 15/02/16 12:29, Bernd Edlinger wrote:
>> Here is a patch that puts each value on it's own 8-byte aligned memory
>> location.  From my experience with tsan tests, sharing shadow memory
>> slots between v and q or o is the most likely explanation for the
>> occasional
>> inability to spot the race condition on v, thus the test case fails,
>> because
>> the return code is 0, and the expected output is not found.
>>
>> Boot-strapped/regression tested on x86_64-linux-gnu.
>>
>> OK for trunk?
>>
>
> Hi,
>
> Could you add 'PR testsuite/68580' to the log entry when committing?
>

Yes, of course, thanks.

Could someone take the time and review this patch?
I don't think it can cause any trouble for gcc-6 and/or gcc-5
even at stage 4.

Is it OK for trunk and gcc-5-branch?

Thanks
Bernd.

> Thanks,
> - Tom
>
>> patch-pr68580.diff
>>
>>
>> 2016-02-15  Bernd Edlinger<bernd.edlinger@hotmail.de>
>>
>>     * c-c++-common/tsan/pr65400-1.c (v, q, o): Make 8-byte aligned.
>>
>> --- gcc/testsuite/c-c++-common/tsan/pr65400-1.c.jj    2015-03-19
>> 08:53:38.000000000 +0100
>> +++ gcc/testsuite/c-c++-common/tsan/pr65400-1.c    2016-02-15
>> 11:09:18.852320827 +0100
>> @@ -7,9 +7,9 @@
>>   #include "tsan_barrier.h"
>>
>>   static pthread_barrier_t barrier;
>> -int v;
>> -int q;
>> -int o;
>> +int v __attribute__((aligned(8)));
>> +int q __attribute__((aligned(8)));
>> +int o __attribute__((aligned(8)));
>>   extern void baz4 (int *);
>>
>>   __attribute__((noinline, noclone)) int
>

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Jakub Jelinek]

On Thu, Feb 18, 2016 at 08:19:22PM +0000, Bernd Edlinger wrote:
> > Could you add 'PR testsuite/68580' to the log entry when committing?
> >
> 
> Yes, of course, thanks.
> 
> Could someone take the time and review this patch?
> I don't think it can cause any trouble for gcc-6 and/or gcc-5
> even at stage 4.
> 
> Is it OK for trunk and gcc-5-branch?

Ok.

> >> 2016-02-15  Bernd Edlinger<bernd.edlinger@hotmail.de>
> >>
> >>     * c-c++-common/tsan/pr65400-1.c (v, q, o): Make 8-byte aligned.
> >>
> >> --- gcc/testsuite/c-c++-common/tsan/pr65400-1.c.jj    2015-03-19
> >> 08:53:38.000000000 +0100
> >> +++ gcc/testsuite/c-c++-common/tsan/pr65400-1.c    2016-02-15
> >> 11:09:18.852320827 +0100
> >> @@ -7,9 +7,9 @@
> >>   #include "tsan_barrier.h"
> >>
> >>   static pthread_barrier_t barrier;
> >> -int v;
> >> -int q;
> >> -int o;
> >> +int v __attribute__((aligned(8)));
> >> +int q __attribute__((aligned(8)));
> >> +int o __attribute__((aligned(8)));
> >>   extern void baz4 (int *);
> >>
> >>   __attribute__((noinline, noclone)) int
> >

	Jakub

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Tom de Vries]

On 15/02/16 08:24, Dmitry Vyukov wrote:
> If we are talking about pr 68580, then I would try:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68580#c2
> first.

As I tried to explain in the follow-up comment ( 
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68580#c3 ), since 
unfortunately I have no reliable way of reproducing the failure, there's 
no defined way to 'try' something.

Thanks,
- Tom

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Dmitry Vyukov]

If we are talking about pr 68580, then I would try:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68580#c2
first.



On Mon, Feb 15, 2016 at 8:18 AM, Dmitry Vyukov <dvyukov@google.com> wrote:
> On Mon, Feb 15, 2016 at 7:00 AM, Tom de Vries <Tom_deVries@mentor.com> wrote:
>> Hi,
>>
>> Occasionally, I've been running into this failure while running the tsan
>> testsuite:
>> ...
>> FAIL: c-c++-common/tsan/pr65400-1.c -O0 execution test
>> ...
>>
>> I've also observed a potential similar occurrence here (
>> https://gcc.gnu.org/ml/gcc-regression/2015-08/msg00147.html ).
>>
>> Initially, I couldn't reproduce it on the command line.
>>
>> Then I tried mimicking heavy load situations that might arise while running
>> the testsuite (I test with -j12 on a server), by running the test in
>> parallel, and found that in that case pthread_create fails, and the test
>> returns 0, which combined with the dg-shouldfail, makes the execution test
>> fail.
>>
>> So I suspect the failure I see while running the testsuite is the result of
>> pthread_create failure.
>>
>> The problem is that I can't be sure, given that in the testcase we do not
>> distinguish between:
>> - return 0, case pthread_create failed, and
>> - return 0 at end of main, case -fsanitize=thread failed to catch the
>>   race condition.
>>
>> Note also that it's possible that many other tsan tests are failing in
>> pthread_create, but that this goes unnoticed because we don't test for the
>> result of pthread_create in most tests.
>>
>> This untested patch is an attempt at structurally testing and handling the
>> result of pthread_create in the tsan testsuite, using this macro:
>> ...
>> #define PTHREAD_CREATE(thread, attr, start_routine, arg)        \
>>   {                                                             \
>>     int res = pthread_create (thread, attr, start_routine, arg);\
>>     if (res)                                                    \
>>       {                                                         \
>>         error (0, res, "pthread_create failed with");           \
>>         exit (0);                                               \
>>       }                                                         \
>>   }
>> ...
>>
>> If this patch is committed, I should at least be able to find out if indeed
>> the failure I observe is related to resource exhaustion.
>>
>> Good idea? Any other comments?
>
>
> Hi Tom,
>
> Yes, I guess we need to check return values. I am not sure how we can
> handle ENOMEMs in tests, but at least knowing the failure reason is
> valuable. We also suspect failing mmap's in some tests on some test
> bots.
>
> But changes to tsan runtime and tests should go into llvm repository
> first and then be integrated into gcc repository. If you never sent
> changes to llvm we can help.
>
> llvm tsan tests contain test.h file (probably what's called
> test_barrier.h in gcc), you can put the macro there. test.h should
> already be included into all tests.
>
> Thanks

Re: [RFC, PR68580] Handle pthread_create error in tsan testsuite

[Dmitry Vyukov]

On Mon, Feb 15, 2016 at 7:00 AM, Tom de Vries <Tom_deVries@mentor.com> wrote:
> Hi,
>
> Occasionally, I've been running into this failure while running the tsan
> testsuite:
> ...
> FAIL: c-c++-common/tsan/pr65400-1.c -O0 execution test
> ...
>
> I've also observed a potential similar occurrence here (
> https://gcc.gnu.org/ml/gcc-regression/2015-08/msg00147.html ).
>
> Initially, I couldn't reproduce it on the command line.
>
> Then I tried mimicking heavy load situations that might arise while running
> the testsuite (I test with -j12 on a server), by running the test in
> parallel, and found that in that case pthread_create fails, and the test
> returns 0, which combined with the dg-shouldfail, makes the execution test
> fail.
>
> So I suspect the failure I see while running the testsuite is the result of
> pthread_create failure.
>
> The problem is that I can't be sure, given that in the testcase we do not
> distinguish between:
> - return 0, case pthread_create failed, and
> - return 0 at end of main, case -fsanitize=thread failed to catch the
>   race condition.
>
> Note also that it's possible that many other tsan tests are failing in
> pthread_create, but that this goes unnoticed because we don't test for the
> result of pthread_create in most tests.
>
> This untested patch is an attempt at structurally testing and handling the
> result of pthread_create in the tsan testsuite, using this macro:
> ...
> #define PTHREAD_CREATE(thread, attr, start_routine, arg)        \
>   {                                                             \
>     int res = pthread_create (thread, attr, start_routine, arg);\
>     if (res)                                                    \
>       {                                                         \
>         error (0, res, "pthread_create failed with");           \
>         exit (0);                                               \
>       }                                                         \
>   }
> ...
>
> If this patch is committed, I should at least be able to find out if indeed
> the failure I observe is related to resource exhaustion.
>
> Good idea? Any other comments?


Hi Tom,

Yes, I guess we need to check return values. I am not sure how we can
handle ENOMEMs in tests, but at least knowing the failure reason is
valuable. We also suspect failing mmap's in some tests on some test
bots.

But changes to tsan runtime and tests should go into llvm repository
first and then be integrated into gcc repository. If you never sent
changes to llvm we can help.

llvm tsan tests contain test.h file (probably what's called
test_barrier.h in gcc), you can put the macro there. test.h should
already be included into all tests.

Thanks

[RFC, PR68580] Handle pthread_create error in tsan testsuite

[Tom de Vries]

[-- Attachment #1: Type: text/plain, Size: 1762 bytes --]

Hi,

Occasionally, I've been running into this failure while running the tsan 
testsuite:
...
FAIL: c-c++-common/tsan/pr65400-1.c -O0 execution test
...

I've also observed a potential similar occurrence here ( 
https://gcc.gnu.org/ml/gcc-regression/2015-08/msg00147.html ).

Initially, I couldn't reproduce it on the command line.

Then I tried mimicking heavy load situations that might arise while 
running the testsuite (I test with -j12 on a server), by running the 
test in parallel, and found that in that case pthread_create fails, and 
the test returns 0, which combined with the dg-shouldfail, makes the 
execution test fail.

So I suspect the failure I see while running the testsuite is the result 
of pthread_create failure.

The problem is that I can't be sure, given that in the testcase we do 
not distinguish between:
- return 0, case pthread_create failed, and
- return 0 at end of main, case -fsanitize=thread failed to catch the
   race condition.

Note also that it's possible that many other tsan tests are failing in 
pthread_create, but that this goes unnoticed because we don't test for 
the result of pthread_create in most tests.

This untested patch is an attempt at structurally testing and handling 
the result of pthread_create in the tsan testsuite, using this macro:
...
#define PTHREAD_CREATE(thread, attr, start_routine, arg)	\
   {								\
     int res = pthread_create (thread, attr, start_routine, arg);\
     if (res)							\
       {								\
	error (0, res, "pthread_create failed with");		\
	exit (0);						\
       }								\
   }
...

If this patch is committed, I should at least be able to find out if 
indeed the failure I observe is related to resource exhaustion.

Good idea? Any other comments?

Thanks,
- Tom


[-- Attachment #2: 0001-Handle-pthread_create-error-in-tsan-testsuite.patch --]
[-- Type: text/x-patch, Size: 25190 bytes --]

Handle pthread_create error in tsan testsuite

2016-02-15  Tom de Vries  <tom@codesourcery.com>

	PR testsuite/68580
	* c-c++-common/tsan/pthread_safe.h: New header file.
	* g++.dg/tsan/pthread_safe.h: Same.
	* c-c++-common/tsan/atomic_stack.c: Include pthread_safe.h.  Use
	PTHREAD_CREATE.
	* c-c++-common/tsan/bitfield_race.c: Same.
	* c-c++-common/tsan/fd_pipe_race.c: Same.
	* c-c++-common/tsan/mutexset1.c: Same.
	* c-c++-common/tsan/pr65400-1.c: Same.
	* c-c++-common/tsan/pr65400-3.c: Same.
	* c-c++-common/tsan/race_on_barrier.c: Same.
	* c-c++-common/tsan/race_on_barrier2.c: Same.
	* c-c++-common/tsan/race_on_mutex.c: Same.
	* c-c++-common/tsan/race_on_mutex2.c: Same.
	* c-c++-common/tsan/simple_race.c: Same.
	* c-c++-common/tsan/simple_stack.c: Same.
	* c-c++-common/tsan/sleep_sync.c: Same.
	* c-c++-common/tsan/thread_leak.c: Same.
	* c-c++-common/tsan/thread_leak1.c: Same.
	* c-c++-common/tsan/thread_leak2.c: Same.
	* c-c++-common/tsan/tiny_race.c: Same.
	* c-c++-common/tsan/tls_race.c: Same.
	* c-c++-common/tsan/write_in_reader_lock.c: Same.
	* g++.dg/tsan/aligned_vs_unaligned_race.C: Same.
	* g++.dg/tsan/atomic_free.C: Same.
	* g++.dg/tsan/atomic_free2.C: Same.
	* g++.dg/tsan/benign_race.C: Same.
	* g++.dg/tsan/cond_race.C: Same.
	* g++.dg/tsan/default_options.C: Same.
	* g++.dg/tsan/fd_close_norace.C: Same.
	* g++.dg/tsan/fd_close_norace2.C: Same.
	* g++.dg/tsan/pr64265.C: Same.
	* g++.dg/tsan/vptr_benign_race.C: Same.
	* g++.dg/tsan/vptr_harmful_race.C: Same.

---
 gcc/testsuite/c-c++-common/tsan/atomic_stack.c         |  5 +++--
 gcc/testsuite/c-c++-common/tsan/bitfield_race.c        |  3 ++-
 gcc/testsuite/c-c++-common/tsan/fd_pipe_race.c         |  5 +++--
 gcc/testsuite/c-c++-common/tsan/mutexset1.c            |  5 +++--
 gcc/testsuite/c-c++-common/tsan/pr65400-1.c            |  4 ++--
 gcc/testsuite/c-c++-common/tsan/pr65400-3.c            |  3 ++-
 gcc/testsuite/c-c++-common/tsan/pthread_safe.h         | 12 ++++++++++++
 gcc/testsuite/c-c++-common/tsan/race_on_barrier.c      |  3 ++-
 gcc/testsuite/c-c++-common/tsan/race_on_barrier2.c     |  3 ++-
 gcc/testsuite/c-c++-common/tsan/race_on_mutex.c        |  5 +++--
 gcc/testsuite/c-c++-common/tsan/race_on_mutex2.c       |  3 ++-
 gcc/testsuite/c-c++-common/tsan/simple_race.c          |  5 +++--
 gcc/testsuite/c-c++-common/tsan/simple_stack.c         |  3 ++-
 gcc/testsuite/c-c++-common/tsan/sleep_sync.c           |  3 ++-
 gcc/testsuite/c-c++-common/tsan/thread_leak.c          |  3 ++-
 gcc/testsuite/c-c++-common/tsan/thread_leak1.c         |  3 ++-
 gcc/testsuite/c-c++-common/tsan/thread_leak2.c         |  3 ++-
 gcc/testsuite/c-c++-common/tsan/tiny_race.c            |  3 ++-
 gcc/testsuite/c-c++-common/tsan/tls_race.c             |  3 ++-
 gcc/testsuite/c-c++-common/tsan/write_in_reader_lock.c |  3 ++-
 gcc/testsuite/g++.dg/tsan/aligned_vs_unaligned_race.C  |  5 +++--
 gcc/testsuite/g++.dg/tsan/atomic_free.C                |  3 ++-
 gcc/testsuite/g++.dg/tsan/atomic_free2.C               |  3 ++-
 gcc/testsuite/g++.dg/tsan/benign_race.C                |  3 ++-
 gcc/testsuite/g++.dg/tsan/cond_race.C                  |  3 ++-
 gcc/testsuite/g++.dg/tsan/default_options.C            |  5 +++--
 gcc/testsuite/g++.dg/tsan/fd_close_norace.C            |  5 +++--
 gcc/testsuite/g++.dg/tsan/fd_close_norace2.C           |  3 ++-
 gcc/testsuite/g++.dg/tsan/pr64265.C                    |  4 ++--
 gcc/testsuite/g++.dg/tsan/pthread_safe.h               | 12 ++++++++++++
 gcc/testsuite/g++.dg/tsan/vptr_benign_race.C           |  5 +++--
 gcc/testsuite/g++.dg/tsan/vptr_harmful_race.C          |  5 +++--
 32 files changed, 94 insertions(+), 42 deletions(-)

diff --git a/gcc/testsuite/c-c++-common/tsan/atomic_stack.c b/gcc/testsuite/c-c++-common/tsan/atomic_stack.c
index 746afa7..dee8f36 100644
--- a/gcc/testsuite/c-c++-common/tsan/atomic_stack.c
+++ b/gcc/testsuite/c-c++-common/tsan/atomic_stack.c
@@ -2,6 +2,7 @@
 /* { dg-additional-options "-ldl" } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -22,8 +23,8 @@ void *Thread2(void *x) {
 int main() {
   barrier_init(&barrier, 2);
   pthread_t t[2];
-  pthread_create(&t[0], NULL, Thread1, NULL);
-  pthread_create(&t[1], NULL, Thread2, NULL);
+  PTHREAD_CREATE(&t[0], NULL, Thread1, NULL);
+  PTHREAD_CREATE(&t[1], NULL, Thread2, NULL);
   pthread_join(t[0], NULL);
   pthread_join(t[1], NULL);
   return 0;
diff --git a/gcc/testsuite/c-c++-common/tsan/bitfield_race.c b/gcc/testsuite/c-c++-common/tsan/bitfield_race.c
index 4268115..ca4ce15 100644
--- a/gcc/testsuite/c-c++-common/tsan/bitfield_race.c
+++ b/gcc/testsuite/c-c++-common/tsan/bitfield_race.c
@@ -2,6 +2,7 @@
 /* { dg-additional-options "-ldl" } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -20,7 +21,7 @@ void *Thread1(void *x) {
 int main() {
   barrier_init(&barrier, 2);
   pthread_t t;
-  pthread_create(&t, 0, Thread1, 0);
+  PTHREAD_CREATE(&t, 0, Thread1, 0);
   Global.b = 43;
   barrier_wait(&barrier);
   pthread_join(t, 0);
diff --git a/gcc/testsuite/c-c++-common/tsan/fd_pipe_race.c b/gcc/testsuite/c-c++-common/tsan/fd_pipe_race.c
index e2176da..00c35c7 100644
--- a/gcc/testsuite/c-c++-common/tsan/fd_pipe_race.c
+++ b/gcc/testsuite/c-c++-common/tsan/fd_pipe_race.c
@@ -3,6 +3,7 @@
 
 #include <pthread.h>
 #include <unistd.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -25,8 +26,8 @@ int main() {
   barrier_init(&barrier, 2);
   pipe(fds);
   pthread_t t[2];
-  pthread_create(&t[0], NULL, Thread1, NULL);
-  pthread_create(&t[1], NULL, Thread2, NULL);
+  PTHREAD_CREATE(&t[0], NULL, Thread1, NULL);
+  PTHREAD_CREATE(&t[1], NULL, Thread2, NULL);
   pthread_join(t[0], NULL);
   pthread_join(t[1], NULL);
   return 0;
diff --git a/gcc/testsuite/c-c++-common/tsan/mutexset1.c b/gcc/testsuite/c-c++-common/tsan/mutexset1.c
index 3462ec4..e17222b 100644
--- a/gcc/testsuite/c-c++-common/tsan/mutexset1.c
+++ b/gcc/testsuite/c-c++-common/tsan/mutexset1.c
@@ -2,6 +2,7 @@
 /* { dg-additional-options "-ldl" } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -27,8 +28,8 @@ int main() {
   barrier_init(&barrier, 2);
   pthread_mutex_init(&mtx, 0);
   pthread_t t[2];
-  pthread_create(&t[0], NULL, Thread1, NULL);
-  pthread_create(&t[1], NULL, Thread2, NULL);
+  PTHREAD_CREATE(&t[0], NULL, Thread1, NULL);
+  PTHREAD_CREATE(&t[1], NULL, Thread2, NULL);
   pthread_join(t[0], NULL);
   pthread_join(t[1], NULL);
   pthread_mutex_destroy(&mtx);
diff --git a/gcc/testsuite/c-c++-common/tsan/pr65400-1.c b/gcc/testsuite/c-c++-common/tsan/pr65400-1.c
index 96fbbfd..df0f643 100644
--- a/gcc/testsuite/c-c++-common/tsan/pr65400-1.c
+++ b/gcc/testsuite/c-c++-common/tsan/pr65400-1.c
@@ -4,6 +4,7 @@
 /* { dg-additional-sources pr65400-2.c } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -74,8 +75,7 @@ main ()
 {
   pthread_t th;
   barrier_init (&barrier, 2);
-  if (pthread_create (&th, NULL, tf, NULL))
-    return 0;
+  PTHREAD_CREATE (&th, NULL, tf, NULL);
   v++;
   barrier_wait (&barrier);
   pthread_join (th, NULL);
diff --git a/gcc/testsuite/c-c++-common/tsan/pr65400-3.c b/gcc/testsuite/c-c++-common/tsan/pr65400-3.c
index 9483bb6..9e48ab3 100644
--- a/gcc/testsuite/c-c++-common/tsan/pr65400-3.c
+++ b/gcc/testsuite/c-c++-common/tsan/pr65400-3.c
@@ -3,6 +3,7 @@
 /* { dg-additional-options "-fno-omit-frame-pointer -ldl" } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -63,7 +64,7 @@ main ()
 {
   pthread_t th;
   barrier_init (&barrier, 2);
-  if (pthread_create (&th, NULL, tf, NULL))
+  if (PTHREAD_CREATE (&th, NULL, tf, NULL))
     return 0;
   barrier_wait (&barrier);
   v++;
diff --git a/gcc/testsuite/c-c++-common/tsan/pthread_safe.h b/gcc/testsuite/c-c++-common/tsan/pthread_safe.h
new file mode 100644
index 0000000..07d273a
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/tsan/pthread_safe.h
@@ -0,0 +1,12 @@
+#include <error.h>
+#include <stdlib.h>
+
+#define PTHREAD_CREATE(thread, attr, start_routine, arg)		\
+  {									\
+    int res = pthread_create (thread, attr, start_routine, arg);	\
+    if (res)								\
+      {									\
+	error (0, res, "pthread_create failed with");			\
+	exit (0);							\
+      }									\
+  }
diff --git a/gcc/testsuite/c-c++-common/tsan/race_on_barrier.c b/gcc/testsuite/c-c++-common/tsan/race_on_barrier.c
index 3de3ff2..b96004d 100644
--- a/gcc/testsuite/c-c++-common/tsan/race_on_barrier.c
+++ b/gcc/testsuite/c-c++-common/tsan/race_on_barrier.c
@@ -2,6 +2,7 @@
 /* { dg-additional-options "-ldl" } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -24,7 +25,7 @@ void *Thread2(void *x) {
 int main() {
   barrier_init(&barrier, 2);
   pthread_t t;
-  pthread_create(&t, NULL, Thread1, NULL);
+  PTHREAD_CREATE(&t, NULL, Thread1, NULL);
   Thread2(0);
   pthread_join(t, NULL);
   pthread_barrier_destroy(&B);
diff --git a/gcc/testsuite/c-c++-common/tsan/race_on_barrier2.c b/gcc/testsuite/c-c++-common/tsan/race_on_barrier2.c
index b01a5cc..b15eacf 100644
--- a/gcc/testsuite/c-c++-common/tsan/race_on_barrier2.c
+++ b/gcc/testsuite/c-c++-common/tsan/race_on_barrier2.c
@@ -4,6 +4,7 @@
 #include <stdio.h>
 #include <stddef.h>
 #include <unistd.h>
+#include "pthread_safe.h"
 
 pthread_barrier_t B;
 int Global;
@@ -23,7 +24,7 @@ void *Thread2(void *x) {
 int main() {
   pthread_barrier_init(&B, 0, 2);
   pthread_t t;
-  pthread_create(&t, NULL, Thread1, NULL);
+  PTHREAD_CREATE(&t, NULL, Thread1, NULL);
   Thread2(0);
   pthread_join(t, NULL);
   return 0;
diff --git a/gcc/testsuite/c-c++-common/tsan/race_on_mutex.c b/gcc/testsuite/c-c++-common/tsan/race_on_mutex.c
index ae30d05..e2e1f00 100644
--- a/gcc/testsuite/c-c++-common/tsan/race_on_mutex.c
+++ b/gcc/testsuite/c-c++-common/tsan/race_on_mutex.c
@@ -2,6 +2,7 @@
 /* { dg-additional-options "-ldl" } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -28,8 +29,8 @@ void *Thread2(void *x) {
 int main() {
   barrier_init(&barrier, 2);
   pthread_t t[2];
-  pthread_create(&t[0], NULL, Thread1, NULL);
-  pthread_create(&t[1], NULL, Thread2, NULL);
+  PTHREAD_CREATE(&t[0], NULL, Thread1, NULL);
+  PTHREAD_CREATE(&t[1], NULL, Thread2, NULL);
   pthread_join(t[0], NULL);
   pthread_join(t[1], NULL);
   pthread_mutex_destroy(&Mtx);
diff --git a/gcc/testsuite/c-c++-common/tsan/race_on_mutex2.c b/gcc/testsuite/c-c++-common/tsan/race_on_mutex2.c
index 57d7e21..8bc508d 100644
--- a/gcc/testsuite/c-c++-common/tsan/race_on_mutex2.c
+++ b/gcc/testsuite/c-c++-common/tsan/race_on_mutex2.c
@@ -2,6 +2,7 @@
 /* { dg-additional-options "-ldl" } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -18,7 +19,7 @@ int main() {
   pthread_mutex_t Mtx;
   pthread_mutex_init(&Mtx, 0);
   pthread_t t;
-  pthread_create(&t, 0, Thread, &Mtx);
+  PTHREAD_CREATE(&t, 0, Thread, &Mtx);
   barrier_wait(&barrier);
   pthread_mutex_destroy(&Mtx);
   pthread_join(t, 0);
diff --git a/gcc/testsuite/c-c++-common/tsan/simple_race.c b/gcc/testsuite/c-c++-common/tsan/simple_race.c
index c1a369b..da614bb 100644
--- a/gcc/testsuite/c-c++-common/tsan/simple_race.c
+++ b/gcc/testsuite/c-c++-common/tsan/simple_race.c
@@ -4,6 +4,7 @@
 
 #include <pthread.h>
 #include <unistd.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 #define MAX_ITERATIONS_NUMBER 1
@@ -45,8 +46,8 @@ void *Thread2(void *x) {
 
 int main_1() {
   pthread_t t[2];
-  pthread_create(&t[0], NULL, Thread1, NULL);
-  pthread_create(&t[1], NULL, Thread2, NULL);
+  PTHREAD_CREATE(&t[0], NULL, Thread1, NULL);
+  PTHREAD_CREATE(&t[1], NULL, Thread2, NULL);
   pthread_join(t[0], NULL);
   pthread_join(t[1], NULL);
   return 0;
diff --git a/gcc/testsuite/c-c++-common/tsan/simple_stack.c b/gcc/testsuite/c-c++-common/tsan/simple_stack.c
index a4d0aba..b20087d 100644
--- a/gcc/testsuite/c-c++-common/tsan/simple_stack.c
+++ b/gcc/testsuite/c-c++-common/tsan/simple_stack.c
@@ -2,6 +2,7 @@
 /* { dg-additional-options "-ldl" } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -38,7 +39,7 @@ void *Thread2(void *x) {
 }
 
 void StartThread(pthread_t *t, void *(*f)(void*)) {
-  pthread_create(t, NULL, f, NULL);
+  PTHREAD_CREATE(t, NULL, f, NULL);
 }
 
 int main() {
diff --git a/gcc/testsuite/c-c++-common/tsan/sleep_sync.c b/gcc/testsuite/c-c++-common/tsan/sleep_sync.c
index c681dce..7c0cada 100644
--- a/gcc/testsuite/c-c++-common/tsan/sleep_sync.c
+++ b/gcc/testsuite/c-c++-common/tsan/sleep_sync.c
@@ -3,6 +3,7 @@
 
 #include <pthread.h>
 #include <unistd.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -22,7 +23,7 @@ void *Thread(void *p) {
 int main() {
   barrier_init(&barrier, 2);
   pthread_t t;
-  pthread_create(&t, 0, Thread, 0);
+  PTHREAD_CREATE(&t, 0, Thread, 0);
   X = 43;
   barrier_wait(&barrier);
   pthread_join(t, 0);
diff --git a/gcc/testsuite/c-c++-common/tsan/thread_leak.c b/gcc/testsuite/c-c++-common/tsan/thread_leak.c
index 02deaba..644dd1d 100644
--- a/gcc/testsuite/c-c++-common/tsan/thread_leak.c
+++ b/gcc/testsuite/c-c++-common/tsan/thread_leak.c
@@ -1,5 +1,6 @@
 #include <pthread.h>
 #include <stdio.h>
+#include "pthread_safe.h"
 
 void *Thread(void *x) {
   return 0;
@@ -7,7 +8,7 @@ void *Thread(void *x) {
 
 int main() {
   pthread_t t;
-  pthread_create(&t, 0, Thread, 0);
+  PTHREAD_CREATE(&t, 0, Thread, 0);
   pthread_join(t, 0);
   printf("PASS\n");
   return 0;
diff --git a/gcc/testsuite/c-c++-common/tsan/thread_leak1.c b/gcc/testsuite/c-c++-common/tsan/thread_leak1.c
index ce28ee4..dde4abb 100644
--- a/gcc/testsuite/c-c++-common/tsan/thread_leak1.c
+++ b/gcc/testsuite/c-c++-common/tsan/thread_leak1.c
@@ -2,6 +2,7 @@
 
 #include <pthread.h>
 #include <unistd.h>
+#include "pthread_safe.h"
 
 void *Thread(void *x) {
   return 0;
@@ -9,7 +10,7 @@ void *Thread(void *x) {
 
 int main() {
   pthread_t t;
-  pthread_create(&t, 0, Thread, 0);
+  PTHREAD_CREATE(&t, 0, Thread, 0);
   sleep(1);
   return 0;
 }
diff --git a/gcc/testsuite/c-c++-common/tsan/thread_leak2.c b/gcc/testsuite/c-c++-common/tsan/thread_leak2.c
index c9b8046..24a871e 100644
--- a/gcc/testsuite/c-c++-common/tsan/thread_leak2.c
+++ b/gcc/testsuite/c-c++-common/tsan/thread_leak2.c
@@ -2,6 +2,7 @@
 
 #include <pthread.h>
 #include <unistd.h>
+#include "pthread_safe.h"
 
 void *Thread(void *x) {
   return 0;
@@ -11,7 +12,7 @@ int main() {
   int i;
   for (i = 0; i < 5; i++) {
     pthread_t t;
-    pthread_create(&t, 0, Thread, 0);
+    PTHREAD_CREATE(&t, 0, Thread, 0);
   }
   sleep(1);
   return 0;
diff --git a/gcc/testsuite/c-c++-common/tsan/tiny_race.c b/gcc/testsuite/c-c++-common/tsan/tiny_race.c
index 10a3feb..4c432f4 100644
--- a/gcc/testsuite/c-c++-common/tsan/tiny_race.c
+++ b/gcc/testsuite/c-c++-common/tsan/tiny_race.c
@@ -2,6 +2,7 @@
 /* { dg-additional-options "-ldl" } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -16,7 +17,7 @@ void *Thread1(void *x) {
 int main() {
   barrier_init(&barrier, 2);
   pthread_t t;
-  pthread_create(&t, 0, Thread1, 0);
+  PTHREAD_CREATE(&t, 0, Thread1, 0);
   Global = 43;
   barrier_wait(&barrier);
   pthread_join(t, 0);
diff --git a/gcc/testsuite/c-c++-common/tsan/tls_race.c b/gcc/testsuite/c-c++-common/tsan/tls_race.c
index 4dd6506..fd306f2 100644
--- a/gcc/testsuite/c-c++-common/tsan/tls_race.c
+++ b/gcc/testsuite/c-c++-common/tsan/tls_race.c
@@ -2,6 +2,7 @@
 /* { dg-additional-options "-ldl" } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -16,7 +17,7 @@ int main() {
   barrier_init(&barrier, 2);
   static __thread int Var = 42;
   pthread_t t;
-  pthread_create(&t, 0, Thread, &Var);
+  PTHREAD_CREATE(&t, 0, Thread, &Var);
   Var = 43;
   barrier_wait(&barrier);
   pthread_join(t, 0);
diff --git a/gcc/testsuite/c-c++-common/tsan/write_in_reader_lock.c b/gcc/testsuite/c-c++-common/tsan/write_in_reader_lock.c
index df32632..cff1761 100644
--- a/gcc/testsuite/c-c++-common/tsan/write_in_reader_lock.c
+++ b/gcc/testsuite/c-c++-common/tsan/write_in_reader_lock.c
@@ -2,6 +2,7 @@
 /* { dg-additional-options "-ldl" } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -23,7 +24,7 @@ int main(int argc, char *argv[]) {
   pthread_rwlock_init(&rwlock, NULL);
   pthread_rwlock_rdlock(&rwlock);
   pthread_t t;
-  pthread_create(&t, 0, Thread1, 0);
+  PTHREAD_CREATE(&t, 0, Thread1, 0);
   volatile int x = GLOB;
  (void)x;
   pthread_rwlock_unlock(&rwlock);
diff --git a/gcc/testsuite/g++.dg/tsan/aligned_vs_unaligned_race.C b/gcc/testsuite/g++.dg/tsan/aligned_vs_unaligned_race.C
index 1facadc..9d6fd63 100644
--- a/gcc/testsuite/g++.dg/tsan/aligned_vs_unaligned_race.C
+++ b/gcc/testsuite/g++.dg/tsan/aligned_vs_unaligned_race.C
@@ -4,6 +4,7 @@
 #include <pthread.h>
 #include <stdio.h>
 #include <stdint.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -27,8 +28,8 @@ void *Thread2(void *x) {
 int main() {
   barrier_init(&barrier, 2);
   pthread_t t[2];
-  pthread_create(&t[0], NULL, Thread1, NULL);
-  pthread_create(&t[1], NULL, Thread2, NULL);
+  PTHREAD_CREATE(&t[0], NULL, Thread1, NULL);
+  PTHREAD_CREATE(&t[1], NULL, Thread2, NULL);
   pthread_join(t[0], NULL);
   pthread_join(t[1], NULL);
   printf("Pass\n");
diff --git a/gcc/testsuite/g++.dg/tsan/atomic_free.C b/gcc/testsuite/g++.dg/tsan/atomic_free.C
index 20429f1..93c5c59 100644
--- a/gcc/testsuite/g++.dg/tsan/atomic_free.C
+++ b/gcc/testsuite/g++.dg/tsan/atomic_free.C
@@ -2,6 +2,7 @@
 /* { dg-additional-options "-ldl" } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -16,7 +17,7 @@ int main() {
   barrier_init(&barrier, 2);
   int *a = new int(0);
   pthread_t t;
-  pthread_create(&t, 0, Thread, a);
+  PTHREAD_CREATE(&t, 0, Thread, a);
   barrier_wait(&barrier);
   delete a;
   pthread_join(t, 0);
diff --git a/gcc/testsuite/g++.dg/tsan/atomic_free2.C b/gcc/testsuite/g++.dg/tsan/atomic_free2.C
index 3b6a8e3..b1f1fe7 100644
--- a/gcc/testsuite/g++.dg/tsan/atomic_free2.C
+++ b/gcc/testsuite/g++.dg/tsan/atomic_free2.C
@@ -2,6 +2,7 @@
 /* { dg-additional-options "-ldl" } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -16,7 +17,7 @@ int main() {
   barrier_init(&barrier, 2);
   int *a = new int(0);
   pthread_t t;
-  pthread_create(&t, 0, Thread, a);
+  PTHREAD_CREATE(&t, 0, Thread, a);
   delete a;
   barrier_wait(&barrier);
   pthread_join(t, 0);
diff --git a/gcc/testsuite/g++.dg/tsan/benign_race.C b/gcc/testsuite/g++.dg/tsan/benign_race.C
index b5f1720..295b41a 100644
--- a/gcc/testsuite/g++.dg/tsan/benign_race.C
+++ b/gcc/testsuite/g++.dg/tsan/benign_race.C
@@ -1,6 +1,7 @@
 #include <pthread.h>
 #include <stdio.h>
 #include <unistd.h>
+#include "pthread_safe.h"
 
 int Global;
 int WTFGlobal;
@@ -27,7 +28,7 @@ int main() {
                              &WTFGlobal, sizeof(WTFGlobal),
                              "Race on WTFGlobal");
   pthread_t t;
-  pthread_create(&t, 0, Thread, 0);
+  PTHREAD_CREATE(&t, 0, Thread, 0);
   sleep(1);
   Global = 43;
   WTFGlobal = 143;
diff --git a/gcc/testsuite/g++.dg/tsan/cond_race.C b/gcc/testsuite/g++.dg/tsan/cond_race.C
index d72d0fb..2fa9d27 100644
--- a/gcc/testsuite/g++.dg/tsan/cond_race.C
+++ b/gcc/testsuite/g++.dg/tsan/cond_race.C
@@ -4,6 +4,7 @@
 /* { dg-output "pthread_cond_signal.*" } */
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -30,7 +31,7 @@ int main() {
   pthread_mutex_init(&c->m, 0);
   pthread_cond_init(&c->c, 0);
   pthread_t th;
-  pthread_create(&th, 0, thr, c);
+  PTHREAD_CREATE(&th, 0, thr, c);
   pthread_mutex_lock(&c->m);
   while (!c->done)
     pthread_cond_wait(&c->c, &c->m);
diff --git a/gcc/testsuite/g++.dg/tsan/default_options.C b/gcc/testsuite/g++.dg/tsan/default_options.C
index 13e1984..35e113c 100644
--- a/gcc/testsuite/g++.dg/tsan/default_options.C
+++ b/gcc/testsuite/g++.dg/tsan/default_options.C
@@ -1,5 +1,6 @@
 #include <pthread.h>
 #include <stdio.h>
+#include "pthread_safe.h"
 
 extern "C" const char *__tsan_default_options() {
   return "report_bugs=0";
@@ -19,8 +20,8 @@ void *Thread2(void *x) {
 
 int main() {
   pthread_t t[2];
-  pthread_create(&t[0], NULL, Thread1, NULL);
-  pthread_create(&t[1], NULL, Thread2, NULL);
+  PTHREAD_CREATE(&t[0], NULL, Thread1, NULL);
+  PTHREAD_CREATE(&t[1], NULL, Thread2, NULL);
   pthread_join(t[0], NULL);
   pthread_join(t[1], NULL);
   fprintf(stderr, "DONE\n");
diff --git a/gcc/testsuite/g++.dg/tsan/fd_close_norace.C b/gcc/testsuite/g++.dg/tsan/fd_close_norace.C
index 9babb6a..08c4c0a 100644
--- a/gcc/testsuite/g++.dg/tsan/fd_close_norace.C
+++ b/gcc/testsuite/g++.dg/tsan/fd_close_norace.C
@@ -4,6 +4,7 @@
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <fcntl.h>
+#include "pthread_safe.h"
 
 void *Thread1(void *x) {
   int f = open("/dev/random", O_RDONLY);
@@ -20,8 +21,8 @@ void *Thread2(void *x) {
 
 int main() {
   pthread_t t[2];
-  pthread_create(&t[0], NULL, Thread1, NULL);
-  pthread_create(&t[1], NULL, Thread2, NULL);
+  PTHREAD_CREATE(&t[0], NULL, Thread1, NULL);
+  PTHREAD_CREATE(&t[1], NULL, Thread2, NULL);
   pthread_join(t[0], NULL);
   pthread_join(t[1], NULL);
   printf("OK\n");
diff --git a/gcc/testsuite/g++.dg/tsan/fd_close_norace2.C b/gcc/testsuite/g++.dg/tsan/fd_close_norace2.C
index 56f00f8..103e6ec 100644
--- a/gcc/testsuite/g++.dg/tsan/fd_close_norace2.C
+++ b/gcc/testsuite/g++.dg/tsan/fd_close_norace2.C
@@ -1,6 +1,7 @@
 #include <pthread.h>
 #include <stdio.h>
 #include <unistd.h>
+#include "pthread_safe.h"
 
 int pipes[2];
 
@@ -17,7 +18,7 @@ int main() {
   if (pipe(pipes))
     return 1;
   pthread_t t;
-  pthread_create(&t, 0, Thread, 0);
+  PTHREAD_CREATE(&t, 0, Thread, 0);
   // send shutdown signal
   while (write(pipes[1], &t, 1) != 1) {
   }
diff --git a/gcc/testsuite/g++.dg/tsan/pr64265.C b/gcc/testsuite/g++.dg/tsan/pr64265.C
index fde32e7..9a08246 100644
--- a/gcc/testsuite/g++.dg/tsan/pr64265.C
+++ b/gcc/testsuite/g++.dg/tsan/pr64265.C
@@ -3,6 +3,7 @@
 // { dg-additional-options "-fno-omit-frame-pointer -ldl" }
 
 #include <pthread.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -43,8 +44,7 @@ main ()
 {
   pthread_t th;
   barrier_init (&barrier, 2);
-  if (pthread_create (&th, NULL, tf, NULL))
-    return 0;
+  PTHREAD_CREATE (&th, NULL, tf, NULL);
   v++;
   barrier_wait (&barrier);
   pthread_join (th, NULL);
diff --git a/gcc/testsuite/g++.dg/tsan/pthread_safe.h b/gcc/testsuite/g++.dg/tsan/pthread_safe.h
new file mode 100644
index 0000000..07d273a
--- /dev/null
+++ b/gcc/testsuite/g++.dg/tsan/pthread_safe.h
@@ -0,0 +1,12 @@
+#include <error.h>
+#include <stdlib.h>
+
+#define PTHREAD_CREATE(thread, attr, start_routine, arg)		\
+  {									\
+    int res = pthread_create (thread, attr, start_routine, arg);	\
+    if (res)								\
+      {									\
+	error (0, res, "pthread_create failed with");			\
+	exit (0);							\
+      }									\
+  }
diff --git a/gcc/testsuite/g++.dg/tsan/vptr_benign_race.C b/gcc/testsuite/g++.dg/tsan/vptr_benign_race.C
index 283684b..2cf0301 100644
--- a/gcc/testsuite/g++.dg/tsan/vptr_benign_race.C
+++ b/gcc/testsuite/g++.dg/tsan/vptr_benign_race.C
@@ -1,6 +1,7 @@
 #include <pthread.h>
 #include <semaphore.h>
 #include <stdio.h>
+#include "pthread_safe.h"
 
 struct A {
   A() {
@@ -40,8 +41,8 @@ void *Thread2(void *x) {
 
 int main() {
   pthread_t t[2];
-  pthread_create(&t[0], NULL, Thread1, NULL);
-  pthread_create(&t[1], NULL, Thread2, NULL);
+  PTHREAD_CREATE(&t[0], NULL, Thread1, NULL);
+  PTHREAD_CREATE(&t[1], NULL, Thread2, NULL);
   pthread_join(t[0], NULL);
   pthread_join(t[1], NULL);
   fprintf(stderr, "PASS\n");
diff --git a/gcc/testsuite/g++.dg/tsan/vptr_harmful_race.C b/gcc/testsuite/g++.dg/tsan/vptr_harmful_race.C
index 1473f93..c43b51f 100644
--- a/gcc/testsuite/g++.dg/tsan/vptr_harmful_race.C
+++ b/gcc/testsuite/g++.dg/tsan/vptr_harmful_race.C
@@ -5,6 +5,7 @@
 #include <semaphore.h>
 #include <stdio.h>
 #include <unistd.h>
+#include "pthread_safe.h"
 #include "tsan_barrier.h"
 
 static pthread_barrier_t barrier;
@@ -49,8 +50,8 @@ void *Thread2(void *x) {
 int main() {
   barrier_init(&barrier, 2);
   pthread_t t[2];
-  pthread_create(&t[0], NULL, Thread1, NULL);
-  pthread_create(&t[1], NULL, Thread2, NULL);
+  PTHREAD_CREATE(&t[0], NULL, Thread1, NULL);
+  PTHREAD_CREATE(&t[1], NULL, Thread2, NULL);
   pthread_join(t[0], NULL);
   pthread_join(t[1], NULL);
 }