RISC-V Test Errors and Failures

RISC-V Test Errors and Failures

[juzhe.zhong]

[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

Hi, Palmer.
I saw your patch showed there are a lot of run time fail (execution fail) of C++.
bug-*.C

These tests are RVV api intrinsics tests coming from Kito's that I have already fixed all of them.
I just double checked again they all passed.
I think it may be your regression environment does not set up simulator (QEMU or SPIKE or GEM5) correctly.
For example, did not enable vector extension in simulator, I don't you may try.

Thanks.


juzhe.zhong@rivai.ai

Re: RISC-V Test Errors and Failures

[Andrew Pinski]

On Tue, May 16, 2023 at 6:58 PM juzhe.zhong@rivai.ai
<juzhe.zhong@rivai.ai> wrote:
>
> Hi, Palmer.
> I saw your patch showed there are a lot of run time fail (execution fail) of C++.
> bug-*.C
>
> These tests are RVV api intrinsics tests coming from Kito's that I have already fixed all of them.
> I just double checked again they all passed.
> I think it may be your regression environment does not set up simulator (QEMU or SPIKE or GEM5) correctly.
> For example, did not enable vector extension in simulator, I don't you may try.

So on x86_64, we test to see if you have the right vector unit before
running those tests? The same thing was true on powerpc (and I think
aarch64 does the same for SVE now too). The reason why I am asking is
that I would need to run the testsuite using the simulator as setup
for the RISCV ISA I am using rather than the one with everything on.
So does the RVV runtime testsuite tests to see if you can run RVV
before running them (or running them and return they passed)?

Thanks,
Andrew Pinski

>
> Thanks.
>
>
> juzhe.zhong@rivai.ai

Re: Re: RISC-V Test Errors and Failures

[juzhe.zhong]

[-- Attachment #1: Type: text/plain, Size: 1520 bytes --]

Oh, I see. Kito has add /* { dg-do run { target { riscv_vector } } } */
But not all RVV tests has use this and I not sure whether it can work.
I think Kito can answer it.
If yes, I think we should add all of them.

Thanks.


juzhe.zhong@rivai.ai
 
From: Andrew Pinski
Date: 2023-05-17 10:02
To: juzhe.zhong@rivai.ai
CC: gcc-patches; palmer; Kito.cheng
Subject: Re: RISC-V Test Errors and Failures
On Tue, May 16, 2023 at 6:58 PM juzhe.zhong@rivai.ai
<juzhe.zhong@rivai.ai> wrote:
>
> Hi, Palmer.
> I saw your patch showed there are a lot of run time fail (execution fail) of C++.
> bug-*.C
>
> These tests are RVV api intrinsics tests coming from Kito's that I have already fixed all of them.
> I just double checked again they all passed.
> I think it may be your regression environment does not set up simulator (QEMU or SPIKE or GEM5) correctly.
> For example, did not enable vector extension in simulator, I don't you may try.
 
So on x86_64, we test to see if you have the right vector unit before
running those tests? The same thing was true on powerpc (and I think
aarch64 does the same for SVE now too). The reason why I am asking is
that I would need to run the testsuite using the simulator as setup
for the RISCV ISA I am using rather than the one with everything on.
So does the RVV runtime testsuite tests to see if you can run RVV
before running them (or running them and return they passed)?
 
Thanks,
Andrew Pinski
 
>
> Thanks.
>
>
> juzhe.zhong@rivai.ai
 

Re: Re: RISC-V Test Errors and Failures

[Palmer Dabbelt]

On Tue, 16 May 2023 19:07:01 PDT (-0700), juzhe.zhong@rivai.ai wrote:
> Oh, I see. Kito has add /* { dg-do run { target { riscv_vector } } } */
> But not all RVV tests has use this and I not sure whether it can work.
> I think Kito can answer it.
> If yes, I think we should add all of them.

Unless I'm missing something, it looks like that only checks if GCC is 
compiling for V.  Nothing appears to be checking if the system the tests 
are running on supports V.

    # Return 1 if the target has RISC-V vector extension, 0 otherwise.
    # Cache the result.
    
    proc check_effective_target_riscv_vector { } {
        # Check that we are compiling for v by checking the __riscv_v marco.
        return [check_no_compiler_messages riscv_vector assembly {
           #if !defined(__riscv_v)
           #error "__riscv_v not defined!"
           #endif
        }]
    }

Those are really just two different things.

It seems pretty reasonably to me to just avoid running the tests when 
the DUT lacks V, but I'm never great with DG.  We should probably add 
similar checks for the other ISA extensions, there's going to be a bunch 
of this.

> 
> Thanks.
> 
> 
> juzhe.zhong@rivai.ai
>  
> From: Andrew Pinski
> Date: 2023-05-17 10:02
> To: juzhe.zhong@rivai.ai
> CC: gcc-patches; palmer; Kito.cheng
> Subject: Re: RISC-V Test Errors and Failures
> On Tue, May 16, 2023 at 6:58 PM juzhe.zhong@rivai.ai
> <juzhe.zhong@rivai.ai> wrote:
>>
>> Hi, Palmer.
>> I saw your patch showed there are a lot of run time fail (execution fail) of C++.
>> bug-*.C
>>
>> These tests are RVV api intrinsics tests coming from Kito's that I have already fixed all of them.
>> I just double checked again they all passed.
>> I think it may be your regression environment does not set up simulator (QEMU or SPIKE or GEM5) correctly.
>> For example, did not enable vector extension in simulator, I don't you may try.
>  
> So on x86_64, we test to see if you have the right vector unit before
> running those tests? The same thing was true on powerpc (and I think
> aarch64 does the same for SVE now too). The reason why I am asking is
> that I would need to run the testsuite using the simulator as setup
> for the RISCV ISA I am using rather than the one with everything on.
> So does the RVV runtime testsuite tests to see if you can run RVV
> before running them (or running them and return they passed)?
>  
> Thanks,
> Andrew Pinski
>  
>>
>> Thanks.
>>
>>
>> juzhe.zhong@rivai.ai
>  

Re: Re: RISC-V Test Errors and Failures

[Kito Cheng]

Currently we are highly rely on simulator can setup correctly by ELF
attribute or -march setting, but seems not true for everyone, for
longer term we need something like
check_effective_target_aarch64_sve_hw, but as Palmer point out, we
might need...bunch of that for different extensions....

On Wed, May 17, 2023 at 10:13 AM Palmer Dabbelt <palmer@rivosinc.com> wrote:
>
> On Tue, 16 May 2023 19:07:01 PDT (-0700), juzhe.zhong@rivai.ai wrote:
> > Oh, I see. Kito has add /* { dg-do run { target { riscv_vector } } } */
> > But not all RVV tests has use this and I not sure whether it can work.
> > I think Kito can answer it.
> > If yes, I think we should add all of them.
>
> Unless I'm missing something, it looks like that only checks if GCC is
> compiling for V.  Nothing appears to be checking if the system the tests
> are running on supports V.
>
>     # Return 1 if the target has RISC-V vector extension, 0 otherwise.
>     # Cache the result.
>
>     proc check_effective_target_riscv_vector { } {
>         # Check that we are compiling for v by checking the __riscv_v marco.
>         return [check_no_compiler_messages riscv_vector assembly {
>            #if !defined(__riscv_v)
>            #error "__riscv_v not defined!"
>            #endif
>         }]
>     }
>
> Those are really just two different things.
>
> It seems pretty reasonably to me to just avoid running the tests when
> the DUT lacks V, but I'm never great with DG.  We should probably add
> similar checks for the other ISA extensions, there's going to be a bunch
> of this.
>
> >
> > Thanks.
> >
> >
> > juzhe.zhong@rivai.ai
> >
> > From: Andrew Pinski
> > Date: 2023-05-17 10:02
> > To: juzhe.zhong@rivai.ai
> > CC: gcc-patches; palmer; Kito.cheng
> > Subject: Re: RISC-V Test Errors and Failures
> > On Tue, May 16, 2023 at 6:58 PM juzhe.zhong@rivai.ai
> > <juzhe.zhong@rivai.ai> wrote:
> >>
> >> Hi, Palmer.
> >> I saw your patch showed there are a lot of run time fail (execution fail) of C++.
> >> bug-*.C
> >>
> >> These tests are RVV api intrinsics tests coming from Kito's that I have already fixed all of them.
> >> I just double checked again they all passed.
> >> I think it may be your regression environment does not set up simulator (QEMU or SPIKE or GEM5) correctly.
> >> For example, did not enable vector extension in simulator, I don't you may try.
> >
> > So on x86_64, we test to see if you have the right vector unit before
> > running those tests? The same thing was true on powerpc (and I think
> > aarch64 does the same for SVE now too). The reason why I am asking is
> > that I would need to run the testsuite using the simulator as setup
> > for the RISCV ISA I am using rather than the one with everything on.
> > So does the RVV runtime testsuite tests to see if you can run RVV
> > before running them (or running them and return they passed)?
> >
> > Thanks,
> > Andrew Pinski
> >
> >>
> >> Thanks.
> >>
> >>
> >> juzhe.zhong@rivai.ai
> >

Re: Re: RISC-V Test Errors and Failures

[Kito Cheng]

Palmer:

For short-term, this should help your internal test:
https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233

On Wed, May 17, 2023 at 10:20 AM Kito Cheng <kito.cheng@sifive.com> wrote:
>
> Currently we are highly rely on simulator can setup correctly by ELF
> attribute or -march setting, but seems not true for everyone, for
> longer term we need something like
> check_effective_target_aarch64_sve_hw, but as Palmer point out, we
> might need...bunch of that for different extensions....
>
> On Wed, May 17, 2023 at 10:13 AM Palmer Dabbelt <palmer@rivosinc.com> wrote:
> >
> > On Tue, 16 May 2023 19:07:01 PDT (-0700), juzhe.zhong@rivai.ai wrote:
> > > Oh, I see. Kito has add /* { dg-do run { target { riscv_vector } } } */
> > > But not all RVV tests has use this and I not sure whether it can work.
> > > I think Kito can answer it.
> > > If yes, I think we should add all of them.
> >
> > Unless I'm missing something, it looks like that only checks if GCC is
> > compiling for V.  Nothing appears to be checking if the system the tests
> > are running on supports V.
> >
> >     # Return 1 if the target has RISC-V vector extension, 0 otherwise.
> >     # Cache the result.
> >
> >     proc check_effective_target_riscv_vector { } {
> >         # Check that we are compiling for v by checking the __riscv_v marco.
> >         return [check_no_compiler_messages riscv_vector assembly {
> >            #if !defined(__riscv_v)
> >            #error "__riscv_v not defined!"
> >            #endif
> >         }]
> >     }
> >
> > Those are really just two different things.
> >
> > It seems pretty reasonably to me to just avoid running the tests when
> > the DUT lacks V, but I'm never great with DG.  We should probably add
> > similar checks for the other ISA extensions, there's going to be a bunch
> > of this.
> >
> > >
> > > Thanks.
> > >
> > >
> > > juzhe.zhong@rivai.ai
> > >
> > > From: Andrew Pinski
> > > Date: 2023-05-17 10:02
> > > To: juzhe.zhong@rivai.ai
> > > CC: gcc-patches; palmer; Kito.cheng
> > > Subject: Re: RISC-V Test Errors and Failures
> > > On Tue, May 16, 2023 at 6:58 PM juzhe.zhong@rivai.ai
> > > <juzhe.zhong@rivai.ai> wrote:
> > >>
> > >> Hi, Palmer.
> > >> I saw your patch showed there are a lot of run time fail (execution fail) of C++.
> > >> bug-*.C
> > >>
> > >> These tests are RVV api intrinsics tests coming from Kito's that I have already fixed all of them.
> > >> I just double checked again they all passed.
> > >> I think it may be your regression environment does not set up simulator (QEMU or SPIKE or GEM5) correctly.
> > >> For example, did not enable vector extension in simulator, I don't you may try.
> > >
> > > So on x86_64, we test to see if you have the right vector unit before
> > > running those tests? The same thing was true on powerpc (and I think
> > > aarch64 does the same for SVE now too). The reason why I am asking is
> > > that I would need to run the testsuite using the simulator as setup
> > > for the RISCV ISA I am using rather than the one with everything on.
> > > So does the RVV runtime testsuite tests to see if you can run RVV
> > > before running them (or running them and return they passed)?
> > >
> > > Thanks,
> > > Andrew Pinski
> > >
> > >>
> > >> Thanks.
> > >>
> > >>
> > >> juzhe.zhong@rivai.ai
> > >

Re: RISC-V Test Errors and Failures

[Vineet Gupta]

On 5/16/23 19:21, Kito Cheng wrote:
> Palmer:
>
> For short-term, this should help your internal test:
> https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233

That only helps if using bleeding edge toolchain scripts (which I 
regularly do and so did Patrick).

Palmer has a fork of toolchain scripts and I'm assuming he hasn't caught 
up to that point ;-)

-Vineet

Re: RISC-V Test Errors and Failures

[Palmer Dabbelt]

On Tue, 16 May 2023 19:46:28 PDT (-0700), Vineet Gupta wrote:
> On 5/16/23 19:21, Kito Cheng wrote:
>> Palmer:
>>
>> For short-term, this should help your internal test:
>> https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233
>
> That only helps if using bleeding edge toolchain scripts (which I
> regularly do and so did Patrick).
>
> Palmer has a fork of toolchain scripts and I'm assuming he hasn't caught
> up to that point ;-)

I'm fine dropping the fork if the bugs have been fixed.  IIRC last week 
we were still waiting for them to merge something?

> -Vineet

Re: RISC-V Test Errors and Failures

[Patrick O'Neill]

On 5/16/23 19:47, Palmer Dabbelt wrote:
> On Tue, 16 May 2023 19:46:28 PDT (-0700), Vineet Gupta wrote:
>> On 5/16/23 19:21, Kito Cheng wrote:
>>> Palmer:
>>>
>>> For short-term, this should help your internal test:
>>> https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233
>>
>> That only helps if using bleeding edge toolchain scripts (which I
>> regularly do and so did Patrick).
>>
>> Palmer has a fork of toolchain scripts and I'm assuming he hasn't caught
>> up to that point ;-)
>
> I'm fine dropping the fork if the bugs have been fixed.  IIRC last 
> week we were still waiting for them to merge something?
The testsuite was broken last week, but was fixed by 
https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1247 which was 
merged last Friday.

That might be the thing you were thinking about?

Re: RISC-V Test Errors and Failures

[Palmer Dabbelt]

On Tue, 16 May 2023 19:51:48 PDT (-0700), Patrick O'Neill wrote:
>
> On 5/16/23 19:47, Palmer Dabbelt wrote:
>> On Tue, 16 May 2023 19:46:28 PDT (-0700), Vineet Gupta wrote:
>>> On 5/16/23 19:21, Kito Cheng wrote:
>>>> Palmer:
>>>>
>>>> For short-term, this should help your internal test:
>>>> https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233
>>>
>>> That only helps if using bleeding edge toolchain scripts (which I
>>> regularly do and so did Patrick).
>>>
>>> Palmer has a fork of toolchain scripts and I'm assuming he hasn't caught
>>> up to that point ;-)
>>
>> I'm fine dropping the fork if the bugs have been fixed.  IIRC last
>> week we were still waiting for them to merge something?
> The testsuite was broken last week, but was fixed by
> https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1247 which was
> merged last Friday.
>
> That might be the thing you were thinking about?

Probably, I'll go try and bump stuff and see if it works...

Thanks!

Re: RISC-V Test Errors and Failures

[Vineet Gupta]

On 5/16/23 19:53, Palmer Dabbelt wrote:
>
> Probably, I'll go try and bump stuff and see if it works... 

Word of caution: Best to not disturb your existing setup, a try a fresh 
checkout first

Re: RISC-V Test Errors and Failures

[Palmer Dabbelt]

On Tue, 16 May 2023 20:08:26 PDT (-0700), Vineet Gupta wrote:
>
> On 5/16/23 19:53, Palmer Dabbelt wrote:
>>
>> Probably, I'll go try and bump stuff and see if it works...
>
> Word of caution: Best to not disturb your existing setup, a try a fresh
> checkout first

Even easier, I think I can get away with just

diff --git a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
index 94d6ec5..efc3a80 100755
--- a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
+++ b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
@@ -12,4 +12,4 @@ done
 
 xlen="$(readelf -h $1 | grep 'Class' | cut -d: -f 2 | xargs echo | sed 's/^ELF//')"
 
-qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu rv$xlen,zba=on,zbb=on,zbc=on,zbs=on "$@"
+qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu rv$xlen,zba=on,zbb=on,zbc=on,zbs=on,v=on "$@"

for now.  I'm going to throw together hwprobe for qemu-user, from looking at
the AVX stuff it should be pretty easy to plumb that into DG and then get the
detection going.

Re: RISC-V Test Errors and Failures

[Kito Cheng]

> diff --git a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
> index 94d6ec5..efc3a80 100755
> --- a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
> +++ b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
> @@ -12,4 +12,4 @@ done
>
>  xlen="$(readelf -h $1 | grep 'Class' | cut -d: -f 2 | xargs echo | sed 's/^ELF//')"
>
> -qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu rv$xlen,zba=on,zbb=on,zbc=on,zbs=on "$@"
> +qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu rv$xlen,zba=on,zbb=on,zbc=on,zbs=on,v=on "$@"

This not work when you testing some combination e.g. Z*inx and zve*,
but anyway I guess those configurations are not matter for you guys :P

>
> for now.  I'm going to throw together hwprobe for qemu-user, from looking at
> the AVX stuff it should be pretty easy to plumb that into DG and then get the
> detection going.

Re: RISC-V Test Errors and Failures

[Jeff Law]

On 5/16/23 21:33, Kito Cheng via Gcc-patches wrote:
>> diff --git a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
>> index 94d6ec5..efc3a80 100755
>> --- a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
>> +++ b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
>> @@ -12,4 +12,4 @@ done
>>
>>   xlen="$(readelf -h $1 | grep 'Class' | cut -d: -f 2 | xargs echo | sed 's/^ELF//')"
>>
>> -qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu rv$xlen,zba=on,zbb=on,zbc=on,zbs=on "$@"
>> +qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu rv$xlen,zba=on,zbb=on,zbc=on,zbs=on,v=on "$@"
> 
> This not work when you testing some combination e.g. Z*inx and zve*,
> but anyway I guess those configurations are not matter for you guys :P
What you could do is install a suitable binfmt handler, then you don't 
need the wrappers at all.  That's how we're handling this stuff in 
Ventana.  It also means you don't need magic dejagnu baseboard files or 
anything like that.  In fact from dejagnu's standpoint it looks native.

jeff