Re: state of the ports

[Adhemerval Zanella Netto <adhemerval.zanella@linaro.org>]

On 25/02/24 19:47, Mark Wielaard wrote:
> Hi,
> 
> On Fri, Feb 02, 2024 at 03:35:04AM +0100, Andreas K. Huettel wrote:
>> So... after all the testing gone into the release I thought I'd
>> summarize a bit for the list here - by sorting the glibc ports into
>> "excellent", "good", "medium", "bad", and "nobody even tried".
> 
> We have several buildbots at builder.sourceware.org for glibc which
> probably should start sending build failure emails when things break.
> https://builder.sourceware.org/buildbot/#/builders?tags=glibc

It would be really helpful to proper document this instead of just throwing
a comment on https://sourceware.org/glibc/wiki/Buildbot saying the current
information is outdated. The wiki the main source of developer information
and I don't think linking to a blog post if the best way to document it.

And although the buildbot is a good improvement for glibc development,
this does seems like disjoint initiative that did not get much feedback
from glibc community and it is not really tied to current practice. 

On the weekly call [1], we focus most of patchwork [2] results, which has 
results for pre-commit buildbot setup by RedHat and extended by Linaro 
for ARM.  And on each release and on the following weeks, we check and 
discuss the release wiki information to check for potential regressions 
and such. Joseph also keeps a buildbot using build-many-glibcs.py to keep 
track of build for all supported architectures [4].

So I personally think it would be more valuable to hook the buildbots on 
the pre-commit CI instead of doing pos-commit checks.

[1] https://sourceware.org/glibc/wiki/PatchworkReviewMeetings
[2] https://patchwork.sourceware.org/project/glibc/list/
[3] https://sourceware.org/glibc/wiki/Release/2.39
[4] https://sourceware.org/pipermail/libc-testresults/

> 
>> 1. EXCELLENT 
>> ============
>>
>> aarch64
>>   - tests pass nearly everywhere
>>   - on raspi, likely timeouts
> 
> https://builder.sourceware.org/buildbot/#/builders/glibc-debian-arm64
> Used to have some tests time out, but since we are running with
> TIMEOUTFACTOR=4 is all green:
> 
>    4807 PASS
>      22 UNSUPPORTED
>      16 XFAIL
>       2 XPASS
> 
>> powerpc64
>> powerpc64le
>>   - tests pass
> 
> https://builder.sourceware.org/buildbot/#/builders/glibc-debian-ppc64
> https://builder.sourceware.org/buildbot/#/builders/glibc-fedora-ppc64le
> 
> We only do a make subdirs=elf check but those all pass:
> 
> glibc-debian-ppc64:
>     341 PASS
>       5 UNSUPPORTED
>       2 XPASS
> 
> glibc-fedora-ppc64le:
>     338 PASS
>       2 UNSUPPORTED
>       2 XPASS
> 
>> s390x
>>   - tests pass
> 
> https://builder.sourceware.org/buildbot/#/builders/glibc-fedora-s390x
> Likewise, just subdirs=elf:
> 
>     336 PASS
>       2 UNSUPPORTED
>       2 XPASS
> 
> For ppc64, ppc64le and s390x we could run a different subset if that
> is more useful. The issue really is the nptl subdir, which just takes
> a very long time (because it doesn't run any test in parallel?)
> 
>> x86-64
>>   - tests pass
> 
> Fully green on glibc-fedora-x86_64
> https://builder.sourceware.org/buildbot/#/builders/glibc-fedora-x86_64
> 
>    4945 PASS
>     274 UNSUPPORTED
>      16 XFAIL
>       4 XPASS
> 
> But on glibc-debian-testing-x86_64 and glibc-rawhide-x86_64 it
> occassionally sees one FAIL: elf/tst-audit10 which times out even with
> TIMEOUTFACTOR=4
> 
>> riscv64
>>   - ~5 failures, but there may be timeouts among them
> 
> https://builder.sourceware.org/buildbot/#/builders/glibc-ubuntu-riscv
> 
> With TIMEOUTFACTOR=4 there are no (timeout) failures anymore.
> 
>    4699 PASS
>      24 UNSUPPORTED
>      16 XFAIL
>       2 XPASS
> 
> This is the slowest of them all (takes 2 hours 15 minutes to 3 hours,
> depending on how good the ccache is). But we have multiple riscv
> boards now thanks to StarFive and one of them is dedicated to glibc.
> 
>> sparc
>>   - ~5 failures (!)
> 
> Seeing even more failures:
> 
>       9 FAIL
>    4725 PASS
>      23 UNSUPPORTED
>      17 XFAIL
>       2 XPASS
> 
> FAIL: elf/tst-audit18
> FAIL: elf/tst-pldd
> FAIL: elf/tst-ptrguard1
> FAIL: math/test-float64x-float128-mul
> FAIL: nptl/tst-mutex9
> FAIL: nptl/tst-oddstacklimit
> FAIL: posix/tst-spawn7-pidfd
> FAIL: socket/tst-socket-timestamp
> FAIL: stdlib/tst-setcontext2
> 
>> x86 (32bit)
>>   - in general tests pass
>>   - there may be breakage on old hw (pentium2)
> 
> Really a i386 VM on x86_64:
> https://builder.sourceware.org/buildbot/#/builders/glibc-debian-i386
> 
>    4851 PASS
>      40 UNSUPPORTED
>      16 XFAIL
>       6 XPASS
> 
> Cheers,
> 
> Mark