Re: How to skip test when building glibc

[Adhemerval Zanella <adhemerval.zanella@linaro.org>]

> On 17 Jun 2022, at 05:12, Letu Ren <fantasquex@gmail.com> wrote:
> 
>> Yes I think it worth to set a higher bond for tests that require a lot of time,
>> the issue is usually we tests with faster machines where defaults are good
>> enough. If you may please open bugs with the required time on the platform
>> you are testing, it would require to adjust on each test code to increase the
>> required timeout.
> 
> I think I should report bugs to glibc. The problem is where to know
> the time limit for each failed unit test set by glibc and how to
> measure the time consumed by tests. I think using bash time is not
> precise. Is there something I can refer to?

The default configuration for all testcases is to create a helper parent 
process to handle timeout and spurious failures (such as segfault). You 
can disable it by using the —direct command line with testrun.sh (which 
runs the test using the built glibc loader and libraries):

  builddir$ time ./testrun testcase —direct

Some tests might require additional environment variables or arguments,
easiest way to check is to get them is to copy from make check log.

> 
>> This might be a bug either in the compiler or in the environment you are using.
>> The commit log for riscv32 libm-test-ulps (b2d175cdb755277ef5579fdac914768003bfbc5c)
>> states that values does not match the RV32 on QEMU, so it might what is
>> happening.
>> 
>> If you are running on real hardware, it might be case where floating-point
>> in non default rounding mode has higher ULPs than expected (I am assuming
>> the RV64 ones were done in read hardware). You can
>> regenerate the libm-test-ulps with `make regen-ulp`, this will show some
>> information on how to copy back the file on source tree.
>> 
>> We will need to increase the generic threshold what we consider an error
>> (as you can see any math function that shows ULPs larger than 8).
> 
> I use a HiFive Unmatched board to test. The hardware baseline is
> RV64GC and the ABI is lp64d. I don't think I have the ability to solve
> this problem currently. May need some time to learn how libm works. :(

So you are indeed testing against the real hardware.  Could you open a
bug report [1] with the output of ‘make regen-ulps’ ? I think we will
need to update the minimum required ULPs.

It can be also a regression on compiler that is generating larger error
bounds, since the current ULP baseline seems also to be generated on
read hardware.

> 
>> This is more worrisome because it means the NAN signal is not being
>> propagated by some FP operation. I think you will need to debug to
>> understand better that is happening, since strfromf code does a lot of FP
>> operations.
> 
> I don't think I have the ability to solve this problem currently. :(
> Anyway, thanks for replying and analyzing the reason why unit tests
> fail.  I will try to investigate them.

Could you also open another bug report with the output of the testcases?

> 
> Letu Ren

[1] https://sourceware.org/bugzilla/ <https://sourceware.org/bugzilla/>