riscv glibc testing

riscv glibc testing

[Vineet Gupta]

Hi,

Trying to understand how folks working on riscv glibc run the testsuite.

One could run native on Unmatched (but doesn't allow testing new 
extensions) or native on qemu with linux (but that will likely be too 
slow) specially the building part.

My preferred solution would be to avoid linux altogether, do a host 
cross-build and test with User mode qemu. And then wire it into 
riscv-gnu-toolchain [1] just like it currently tests gcc dejagnu. 
However glibc cross testing from my prior ARC work required linux+ssh etc.

This email is to see if someone has such as setup already.

Thx,
-Vineet

[1] https://github.com/riscv-collab/riscv-gnu-toolchain

Re: riscv glibc testing

[Philipp Tomsich]

Vineet,

We run all testing using qemu userland through binfmt_misc by
replacing the system-installed qemu with our own (containing the
latest extensions).
This supports even the strangest cases (such as fork + exec that
alternates host and target binaries): e.g., even the test-workload of
perlbench_r is no problem.

Philipp.


On Fri, 11 Nov 2022 at 20:48, Vineet Gupta <vineetg@rivosinc.com> wrote:
>
> Hi,
>
> Trying to understand how folks working on riscv glibc run the testsuite.
>
> One could run native on Unmatched (but doesn't allow testing new
> extensions) or native on qemu with linux (but that will likely be too
> slow) specially the building part.
>
> My preferred solution would be to avoid linux altogether, do a host
> cross-build and test with User mode qemu. And then wire it into
> riscv-gnu-toolchain [1] just like it currently tests gcc dejagnu.
> However glibc cross testing from my prior ARC work required linux+ssh etc.
>
> This email is to see if someone has such as setup already.
>
> Thx,
> -Vineet
>
> [1] https://github.com/riscv-collab/riscv-gnu-toolchain

Re: riscv glibc testing

[Palmer Dabbelt]

On Fri, 11 Nov 2022 11:51:21 PST (-0800), philipp.tomsich@vrull.eu wrote:
> Vineet,
>
> We run all testing using qemu userland through binfmt_misc by
> replacing the system-installed qemu with our own (containing the
> latest extensions).
> This supports even the strangest cases (such as fork + exec that
> alternates host and target binaries): e.g., even the test-workload of
> perlbench_r is no problem.

THat's OK for the GCC and binutils test suites, but last I checked 
QEMU's user-mode emulation couldn't handle the glibc test suite and 
wasn't accepted as a target for saying "this patch passes the tests".  
It's kind of a bummer because it makes running the glibc test suite 
without real hardware a lot more of a headache, but I think it's just 
the nature of the beast here: the glibc test suite exercises a lot of 
edge case behavior in the Linux uABI, trying to get that all matched 
perfectly in QEMU would be a lot of work (and IIUC come with some 
performance issues).

I don't think any of that has changed, but it's not really a 
RISC-V-specific problem so one of the global maintainers would know for 
sure.  I know DJ's been doing a lot of work on the patchwork testing 
stuff, so it's entirely possible something's changed?

> Philipp.
>
>
> On Fri, 11 Nov 2022 at 20:48, Vineet Gupta <vineetg@rivosinc.com> wrote:
>>
>> Hi,
>>
>> Trying to understand how folks working on riscv glibc run the testsuite.
>>
>> One could run native on Unmatched (but doesn't allow testing new
>> extensions) or native on qemu with linux (but that will likely be too
>> slow) specially the building part.
>>
>> My preferred solution would be to avoid linux altogether, do a host
>> cross-build and test with User mode qemu. And then wire it into
>> riscv-gnu-toolchain [1] just like it currently tests gcc dejagnu.
>> However glibc cross testing from my prior ARC work required linux+ssh etc.
>>
>> This email is to see if someone has such as setup already.
>>
>> Thx,
>> -Vineet
>>
>> [1] https://github.com/riscv-collab/riscv-gnu-toolchain

Re: riscv glibc testing

[Christoph Müllner]

[-- Attachment #1: Type: text/plain, Size: 1144 bytes --]

Hi Vineet,

did you see my PR for improved testing capabilities for the
riscv-gnu-toolchain repo:
  https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1139

Among the changes is support for testing glibc.
Tests run very long and produce incredibly much output (triggers an
out-of-memory of tmux on my 32GiB machine).
Executed everything on a Fedora machine.

BR
Christoph


On Fri, Nov 11, 2022 at 8:48 PM Vineet Gupta <vineetg@rivosinc.com> wrote:

> Hi,
>
> Trying to understand how folks working on riscv glibc run the testsuite.
>
> One could run native on Unmatched (but doesn't allow testing new
> extensions) or native on qemu with linux (but that will likely be too
> slow) specially the building part.
>
> My preferred solution would be to avoid linux altogether, do a host
> cross-build and test with User mode qemu. And then wire it into
> riscv-gnu-toolchain [1] just like it currently tests gcc dejagnu.
> However glibc cross testing from my prior ARC work required linux+ssh etc.
>
> This email is to see if someone has such as setup already.
>
> Thx,
> -Vineet
>
> [1] https://github.com/riscv-collab/riscv-gnu-toolchain
>

Re: riscv glibc testing

[Vineet Gupta]

[-- Attachment #1: Type: text/plain, Size: 1383 bytes --]

Hi Christoph,

Awesome, that's exactly what I was looking for :-)
I'll give it a try.

Thx,
-Vineet

On 11/11/22 12:46, Christoph Müllner wrote:
> Hi Vineet,
>
> did you see my PR for improved testing capabilities for the 
> riscv-gnu-toolchain repo:
> https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1139
>
> Among the changes is support for testing glibc.
> Tests run very long and produce incredibly much output (triggers an 
> out-of-memory of tmux on my 32GiB machine).
> Executed everything on a Fedora machine.
>
> BR
> Christoph
>
>
> On Fri, Nov 11, 2022 at 8:48 PM Vineet Gupta <vineetg@rivosinc.com> wrote:
>
>     Hi,
>
>     Trying to understand how folks working on riscv glibc run the
>     testsuite.
>
>     One could run native on Unmatched (but doesn't allow testing new
>     extensions) or native on qemu with linux (but that will likely be too
>     slow) specially the building part.
>
>     My preferred solution would be to avoid linux altogether, do a host
>     cross-build and test with User mode qemu. And then wire it into
>     riscv-gnu-toolchain [1] just like it currently tests gcc dejagnu.
>     However glibc cross testing from my prior ARC work required
>     linux+ssh etc.
>
>     This email is to see if someone has such as setup already.
>
>     Thx,
>     -Vineet
>
>     [1] https://github.com/riscv-collab/riscv-gnu-toolchain
>

Re: riscv glibc testing

[DJ Delorie]

Palmer Dabbelt <palmer@rivosinc.com> writes:
> I don't think any of that has changed, but it's not really a 
> RISC-V-specific problem so one of the global maintainers would know for 
> sure.  I know DJ's been doing a lot of work on the patchwork testing 
> stuff, so it's entirely possible something's changed?

Our patchwork-based CI/CD system allows anyone to add a "trybot" for any
testing purposes.  I've still got on my todo list to add in my unmatched
board as a trybot for risc-v testing although I haven't decided if it
will test every patch, or only the ones that touch paths in
sysdeps/riscv.