nptl/tst-eintr1.c test question

nptl/tst-eintr1.c test question

[Steve Ellcey]

I was wondering if anyone else is having problems with nptl/tst-eintr1?
This test is known to fail on some kernels and there is a comment about
it on the glibc wiki.

https://sourceware.org/glibc/wiki/Testing/Testsuite#Known_testsuite_failures

The problem I have is that when this test fails for me on my aarch64 box
it doesn't just result in a failed test, it aborts the whole 'make check'
process and I don't get any results printed out.  I was wondering if there
was some way to make this test not run on some platforms (and how I might
identify what those platforms were).

Maybe something like:

#define BAD_KERNEL (defined (__ARM_ARCH_8A))
#if BAD_KERNEL
static int do_fail (void) { return 1; }
# define TEST_FUNCTION do_fail ()
#else
# define TEST_FUNCTION do_test ()
#endif

I don't know if this test fails on all aarch8/8A platforms, I just know
that it fails for me, so I am not sure what a good way to set BAD_KERNEL
would be.

Any thoughts?

FYI: The failure I get is due to to many forks causing the make process(es) to
abort:

../scripts/evaluate-test.sh nptl/tst-eintr1 $? false false > /home/ubuntu/sellcey/glibc-patched/obj-glibc64/nptl/tst-eintr1.test-result
/bin/sh: 2: Cannot fork
../Rules:241: recipe for target '/home/ubuntu/sellcey/glibc-patched/obj-glibc64/nptl/tst-eintr1.out' failed
make[2]: *** [/home/ubuntu/sellcey/glibc-patched/obj-glibc64/nptl/tst-eintr1.out] Error 2
make[2]: Leaving directory '/home/ubuntu/sellcey/glibc-patched/src/glibc/nptl'
Makefile:215: recipe for target 'nptl/tests' failed
make[1]: *** [nptl/tests] Error 2
make[1]: Leaving directory '/home/ubuntu/sellcey/glibc-patched/src/glibc'
Makefile:9: recipe for target 'check' failed
make: *** [check] Error 2


Steve Ellcey
sellcey@caviumnetworks.com

Re: nptl/tst-eintr1.c test question

[Florian Weimer]

On 12/16/2016 12:02 AM, Steve Ellcey wrote:

> FYI: The failure I get is due to to many forks causing the make process(es) to
> abort:
>
> ../scripts/evaluate-test.sh nptl/tst-eintr1 $? false false > /home/ubuntu/sellcey/glibc-patched/obj-glibc64/nptl/tst-eintr1.test-result
> /bin/sh: 2: Cannot fork
> ../Rules:241: recipe for target '/home/ubuntu/sellcey/glibc-patched/obj-glibc64/nptl/tst-eintr1.out' failed
> make[2]: *** [/home/ubuntu/sellcey/glibc-patched/obj-glibc64/nptl/tst-eintr1.out] Error 2
> make[2]: Leaving directory '/home/ubuntu/sellcey/glibc-patched/src/glibc/nptl'
> Makefile:215: recipe for target 'nptl/tests' failed
> make[1]: *** [nptl/tests] Error 2
> make[1]: Leaving directory '/home/ubuntu/sellcey/glibc-patched/src/glibc'
> Makefile:9: recipe for target 'check' failed
> make: *** [check] Error 2

So the failure happens outside of the test harness.  This means we 
cannot intercept it.

Some kernel developers have suggested we should just retry the fork if 
we get EAGAIN (and do the same for thread creation).  But I don't think 
this is a good idea—we would either have to do it in glibc, or any shell 
on the planet.  It's also not clear how many attempts are are needed to 
cover up the race.

A few more details are available here:

   https://bugzilla.kernel.org/show_bug.cgi?id=154011

It may help to disable cgroups restrictions and avoid configuring low 
per-user process limits.  Or you might be observing a different issue, 
not related to what we saw earlier this year on Fedora builders.

In the end, the kernel's fork/clone implementation is buggy and has to 
be fixed.

Florian

Re: nptl/tst-eintr1.c test question

[Stefan Liebler]

On 12/16/2016 12:02 AM, Steve Ellcey wrote:
> I was wondering if anyone else is having problems with nptl/tst-eintr1?
> This test is known to fail on some kernels and there is a comment about
> it on the glibc wiki.
>
> https://sourceware.org/glibc/wiki/Testing/Testsuite#Known_testsuite_failures
>
> The problem I have is that when this test fails for me on my aarch64 box
> it doesn't just result in a failed test, it aborts the whole 'make check'
> process and I don't get any results printed out.  I was wondering if there
> was some way to make this test not run on some platforms (and how I might
> identify what those platforms were).
>
> Maybe something like:
>
> #define BAD_KERNEL (defined (__ARM_ARCH_8A))
> #if BAD_KERNEL
> static int do_fail (void) { return 1; }
> # define TEST_FUNCTION do_fail ()
> #else
> # define TEST_FUNCTION do_test ()
> #endif
>
> I don't know if this test fails on all aarch8/8A platforms, I just know
> that it fails for me, so I am not sure what a good way to set BAD_KERNEL
> would be.
>
> Any thoughts?
>
> FYI: The failure I get is due to to many forks causing the make process(es) to
> abort:
>
> ../scripts/evaluate-test.sh nptl/tst-eintr1 $? false false > /home/ubuntu/sellcey/glibc-patched/obj-glibc64/nptl/tst-eintr1.test-result
> /bin/sh: 2: Cannot fork
> ../Rules:241: recipe for target '/home/ubuntu/sellcey/glibc-patched/obj-glibc64/nptl/tst-eintr1.out' failed
> make[2]: *** [/home/ubuntu/sellcey/glibc-patched/obj-glibc64/nptl/tst-eintr1.out] Error 2
> make[2]: Leaving directory '/home/ubuntu/sellcey/glibc-patched/src/glibc/nptl'
> Makefile:215: recipe for target 'nptl/tests' failed
> make[1]: *** [nptl/tests] Error 2
> make[1]: Leaving directory '/home/ubuntu/sellcey/glibc-patched/src/glibc'
> Makefile:9: recipe for target 'check' failed
> make: *** [check] Error 2
>
>
> Steve Ellcey
> sellcey@caviumnetworks.com
>
Hi Steve,

I recognize such EAGAIN failures on s390, too.
In my case, I can only see them if cgroups configuration has limited 
pids.max. Otherwise, there is no issue.

If pthread_create fails with EAGAIN and I then query the number of 
processes via e.g. sysinfo(), I recognize a small number of processes. 
If I afterwards try pthread_create() again, it fails with EAGAIN, too.

The threads have been exited, but the kernel releases the PIDs only 
interval-based. On at least s390 this interval is longer than on e.g. x86.

Bye
Stefan