RISC-V Test Errors and Failures

RISC-V Test Errors and Failures

[juzhe.zhong]

[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

Hi, Palmer.
I saw your patch showed there are a lot of run time fail (execution fail) of C++.
bug-*.C

These tests are RVV api intrinsics tests coming from Kito's that I have already fixed all of them.
I just double checked again they all passed.
I think it may be your regression environment does not set up simulator (QEMU or SPIKE or GEM5) correctly.
For example, did not enable vector extension in simulator, I don't you may try.

Thanks.


juzhe.zhong@rivai.ai

Re: RISC-V Test Errors and Failures

[Andrew Pinski]

On Tue, May 16, 2023 at 6:58 PM juzhe.zhong@rivai.ai
<juzhe.zhong@rivai.ai> wrote:
>
> Hi, Palmer.
> I saw your patch showed there are a lot of run time fail (execution fail) of C++.
> bug-*.C
>
> These tests are RVV api intrinsics tests coming from Kito's that I have already fixed all of them.
> I just double checked again they all passed.
> I think it may be your regression environment does not set up simulator (QEMU or SPIKE or GEM5) correctly.
> For example, did not enable vector extension in simulator, I don't you may try.

So on x86_64, we test to see if you have the right vector unit before
running those tests? The same thing was true on powerpc (and I think
aarch64 does the same for SVE now too). The reason why I am asking is
that I would need to run the testsuite using the simulator as setup
for the RISCV ISA I am using rather than the one with everything on.
So does the RVV runtime testsuite tests to see if you can run RVV
before running them (or running them and return they passed)?

Thanks,
Andrew Pinski

>
> Thanks.
>
>
> juzhe.zhong@rivai.ai

Re: Re: RISC-V Test Errors and Failures

[juzhe.zhong]

[-- Attachment #1: Type: text/plain, Size: 1520 bytes --]

Oh, I see. Kito has add /* { dg-do run { target { riscv_vector } } } */
But not all RVV tests has use this and I not sure whether it can work.
I think Kito can answer it.
If yes, I think we should add all of them.

Thanks.


juzhe.zhong@rivai.ai
 
From: Andrew Pinski
Date: 2023-05-17 10:02
To: juzhe.zhong@rivai.ai
CC: gcc-patches; palmer; Kito.cheng
Subject: Re: RISC-V Test Errors and Failures
On Tue, May 16, 2023 at 6:58 PM juzhe.zhong@rivai.ai
<juzhe.zhong@rivai.ai> wrote:
>
> Hi, Palmer.
> I saw your patch showed there are a lot of run time fail (execution fail) of C++.
> bug-*.C
>
> These tests are RVV api intrinsics tests coming from Kito's that I have already fixed all of them.
> I just double checked again they all passed.
> I think it may be your regression environment does not set up simulator (QEMU or SPIKE or GEM5) correctly.
> For example, did not enable vector extension in simulator, I don't you may try.
 
So on x86_64, we test to see if you have the right vector unit before
running those tests? The same thing was true on powerpc (and I think
aarch64 does the same for SVE now too). The reason why I am asking is
that I would need to run the testsuite using the simulator as setup
for the RISCV ISA I am using rather than the one with everything on.
So does the RVV runtime testsuite tests to see if you can run RVV
before running them (or running them and return they passed)?
 
Thanks,
Andrew Pinski
 
>
> Thanks.
>
>
> juzhe.zhong@rivai.ai
 

Re: Re: RISC-V Test Errors and Failures

[Palmer Dabbelt]

On Tue, 16 May 2023 19:07:01 PDT (-0700), juzhe.zhong@rivai.ai wrote:
> Oh, I see. Kito has add /* { dg-do run { target { riscv_vector } } } */
> But not all RVV tests has use this and I not sure whether it can work.
> I think Kito can answer it.
> If yes, I think we should add all of them.

Unless I'm missing something, it looks like that only checks if GCC is 
compiling for V.  Nothing appears to be checking if the system the tests 
are running on supports V.

    # Return 1 if the target has RISC-V vector extension, 0 otherwise.
    # Cache the result.
    
    proc check_effective_target_riscv_vector { } {
        # Check that we are compiling for v by checking the __riscv_v marco.
        return [check_no_compiler_messages riscv_vector assembly {
           #if !defined(__riscv_v)
           #error "__riscv_v not defined!"
           #endif
        }]
    }

Those are really just two different things.

It seems pretty reasonably to me to just avoid running the tests when 
the DUT lacks V, but I'm never great with DG.  We should probably add 
similar checks for the other ISA extensions, there's going to be a bunch 
of this.

> 
> Thanks.
> 
> 
> juzhe.zhong@rivai.ai
>  
> From: Andrew Pinski
> Date: 2023-05-17 10:02
> To: juzhe.zhong@rivai.ai
> CC: gcc-patches; palmer; Kito.cheng
> Subject: Re: RISC-V Test Errors and Failures
> On Tue, May 16, 2023 at 6:58 PM juzhe.zhong@rivai.ai
> <juzhe.zhong@rivai.ai> wrote:
>>
>> Hi, Palmer.
>> I saw your patch showed there are a lot of run time fail (execution fail) of C++.
>> bug-*.C
>>
>> These tests are RVV api intrinsics tests coming from Kito's that I have already fixed all of them.
>> I just double checked again they all passed.
>> I think it may be your regression environment does not set up simulator (QEMU or SPIKE or GEM5) correctly.
>> For example, did not enable vector extension in simulator, I don't you may try.
>  
> So on x86_64, we test to see if you have the right vector unit before
> running those tests? The same thing was true on powerpc (and I think
> aarch64 does the same for SVE now too). The reason why I am asking is
> that I would need to run the testsuite using the simulator as setup
> for the RISCV ISA I am using rather than the one with everything on.
> So does the RVV runtime testsuite tests to see if you can run RVV
> before running them (or running them and return they passed)?
>  
> Thanks,
> Andrew Pinski
>  
>>
>> Thanks.
>>
>>
>> juzhe.zhong@rivai.ai
>  

Re: Re: RISC-V Test Errors and Failures

[Kito Cheng]

Currently we are highly rely on simulator can setup correctly by ELF
attribute or -march setting, but seems not true for everyone, for
longer term we need something like
check_effective_target_aarch64_sve_hw, but as Palmer point out, we
might need...bunch of that for different extensions....

On Wed, May 17, 2023 at 10:13 AM Palmer Dabbelt <palmer@rivosinc.com> wrote:
>
> On Tue, 16 May 2023 19:07:01 PDT (-0700), juzhe.zhong@rivai.ai wrote:
> > Oh, I see. Kito has add /* { dg-do run { target { riscv_vector } } } */
> > But not all RVV tests has use this and I not sure whether it can work.
> > I think Kito can answer it.
> > If yes, I think we should add all of them.
>
> Unless I'm missing something, it looks like that only checks if GCC is
> compiling for V.  Nothing appears to be checking if the system the tests
> are running on supports V.
>
>     # Return 1 if the target has RISC-V vector extension, 0 otherwise.
>     # Cache the result.
>
>     proc check_effective_target_riscv_vector { } {
>         # Check that we are compiling for v by checking the __riscv_v marco.
>         return [check_no_compiler_messages riscv_vector assembly {
>            #if !defined(__riscv_v)
>            #error "__riscv_v not defined!"
>            #endif
>         }]
>     }
>
> Those are really just two different things.
>
> It seems pretty reasonably to me to just avoid running the tests when
> the DUT lacks V, but I'm never great with DG.  We should probably add
> similar checks for the other ISA extensions, there's going to be a bunch
> of this.
>
> >
> > Thanks.
> >
> >
> > juzhe.zhong@rivai.ai
> >
> > From: Andrew Pinski
> > Date: 2023-05-17 10:02
> > To: juzhe.zhong@rivai.ai
> > CC: gcc-patches; palmer; Kito.cheng
> > Subject: Re: RISC-V Test Errors and Failures
> > On Tue, May 16, 2023 at 6:58 PM juzhe.zhong@rivai.ai
> > <juzhe.zhong@rivai.ai> wrote:
> >>
> >> Hi, Palmer.
> >> I saw your patch showed there are a lot of run time fail (execution fail) of C++.
> >> bug-*.C
> >>
> >> These tests are RVV api intrinsics tests coming from Kito's that I have already fixed all of them.
> >> I just double checked again they all passed.
> >> I think it may be your regression environment does not set up simulator (QEMU or SPIKE or GEM5) correctly.
> >> For example, did not enable vector extension in simulator, I don't you may try.
> >
> > So on x86_64, we test to see if you have the right vector unit before
> > running those tests? The same thing was true on powerpc (and I think
> > aarch64 does the same for SVE now too). The reason why I am asking is
> > that I would need to run the testsuite using the simulator as setup
> > for the RISCV ISA I am using rather than the one with everything on.
> > So does the RVV runtime testsuite tests to see if you can run RVV
> > before running them (or running them and return they passed)?
> >
> > Thanks,
> > Andrew Pinski
> >
> >>
> >> Thanks.
> >>
> >>
> >> juzhe.zhong@rivai.ai
> >

Re: Re: RISC-V Test Errors and Failures

[Kito Cheng]

Palmer:

For short-term, this should help your internal test:
https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233

On Wed, May 17, 2023 at 10:20 AM Kito Cheng <kito.cheng@sifive.com> wrote:
>
> Currently we are highly rely on simulator can setup correctly by ELF
> attribute or -march setting, but seems not true for everyone, for
> longer term we need something like
> check_effective_target_aarch64_sve_hw, but as Palmer point out, we
> might need...bunch of that for different extensions....
>
> On Wed, May 17, 2023 at 10:13 AM Palmer Dabbelt <palmer@rivosinc.com> wrote:
> >
> > On Tue, 16 May 2023 19:07:01 PDT (-0700), juzhe.zhong@rivai.ai wrote:
> > > Oh, I see. Kito has add /* { dg-do run { target { riscv_vector } } } */
> > > But not all RVV tests has use this and I not sure whether it can work.
> > > I think Kito can answer it.
> > > If yes, I think we should add all of them.
> >
> > Unless I'm missing something, it looks like that only checks if GCC is
> > compiling for V.  Nothing appears to be checking if the system the tests
> > are running on supports V.
> >
> >     # Return 1 if the target has RISC-V vector extension, 0 otherwise.
> >     # Cache the result.
> >
> >     proc check_effective_target_riscv_vector { } {
> >         # Check that we are compiling for v by checking the __riscv_v marco.
> >         return [check_no_compiler_messages riscv_vector assembly {
> >            #if !defined(__riscv_v)
> >            #error "__riscv_v not defined!"
> >            #endif
> >         }]
> >     }
> >
> > Those are really just two different things.
> >
> > It seems pretty reasonably to me to just avoid running the tests when
> > the DUT lacks V, but I'm never great with DG.  We should probably add
> > similar checks for the other ISA extensions, there's going to be a bunch
> > of this.
> >
> > >
> > > Thanks.
> > >
> > >
> > > juzhe.zhong@rivai.ai
> > >
> > > From: Andrew Pinski
> > > Date: 2023-05-17 10:02
> > > To: juzhe.zhong@rivai.ai
> > > CC: gcc-patches; palmer; Kito.cheng
> > > Subject: Re: RISC-V Test Errors and Failures
> > > On Tue, May 16, 2023 at 6:58 PM juzhe.zhong@rivai.ai
> > > <juzhe.zhong@rivai.ai> wrote:
> > >>
> > >> Hi, Palmer.
> > >> I saw your patch showed there are a lot of run time fail (execution fail) of C++.
> > >> bug-*.C
> > >>
> > >> These tests are RVV api intrinsics tests coming from Kito's that I have already fixed all of them.
> > >> I just double checked again they all passed.
> > >> I think it may be your regression environment does not set up simulator (QEMU or SPIKE or GEM5) correctly.
> > >> For example, did not enable vector extension in simulator, I don't you may try.
> > >
> > > So on x86_64, we test to see if you have the right vector unit before
> > > running those tests? The same thing was true on powerpc (and I think
> > > aarch64 does the same for SVE now too). The reason why I am asking is
> > > that I would need to run the testsuite using the simulator as setup
> > > for the RISCV ISA I am using rather than the one with everything on.
> > > So does the RVV runtime testsuite tests to see if you can run RVV
> > > before running them (or running them and return they passed)?
> > >
> > > Thanks,
> > > Andrew Pinski
> > >
> > >>
> > >> Thanks.
> > >>
> > >>
> > >> juzhe.zhong@rivai.ai
> > >

Re: RISC-V Test Errors and Failures

[Vineet Gupta]

On 5/16/23 19:21, Kito Cheng wrote:
> Palmer:
>
> For short-term, this should help your internal test:
> https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233

That only helps if using bleeding edge toolchain scripts (which I 
regularly do and so did Patrick).

Palmer has a fork of toolchain scripts and I'm assuming he hasn't caught 
up to that point ;-)

-Vineet

Re: RISC-V Test Errors and Failures

[Palmer Dabbelt]

On Tue, 16 May 2023 19:46:28 PDT (-0700), Vineet Gupta wrote:
> On 5/16/23 19:21, Kito Cheng wrote:
>> Palmer:
>>
>> For short-term, this should help your internal test:
>> https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233
>
> That only helps if using bleeding edge toolchain scripts (which I
> regularly do and so did Patrick).
>
> Palmer has a fork of toolchain scripts and I'm assuming he hasn't caught
> up to that point ;-)

I'm fine dropping the fork if the bugs have been fixed.  IIRC last week 
we were still waiting for them to merge something?

> -Vineet

Re: RISC-V Test Errors and Failures

[Patrick O'Neill]

On 5/16/23 19:47, Palmer Dabbelt wrote:
> On Tue, 16 May 2023 19:46:28 PDT (-0700), Vineet Gupta wrote:
>> On 5/16/23 19:21, Kito Cheng wrote:
>>> Palmer:
>>>
>>> For short-term, this should help your internal test:
>>> https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233
>>
>> That only helps if using bleeding edge toolchain scripts (which I
>> regularly do and so did Patrick).
>>
>> Palmer has a fork of toolchain scripts and I'm assuming he hasn't caught
>> up to that point ;-)
>
> I'm fine dropping the fork if the bugs have been fixed.  IIRC last 
> week we were still waiting for them to merge something?
The testsuite was broken last week, but was fixed by 
https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1247 which was 
merged last Friday.

That might be the thing you were thinking about?

Re: RISC-V Test Errors and Failures

[Palmer Dabbelt]

On Tue, 16 May 2023 19:51:48 PDT (-0700), Patrick O'Neill wrote:
>
> On 5/16/23 19:47, Palmer Dabbelt wrote:
>> On Tue, 16 May 2023 19:46:28 PDT (-0700), Vineet Gupta wrote:
>>> On 5/16/23 19:21, Kito Cheng wrote:
>>>> Palmer:
>>>>
>>>> For short-term, this should help your internal test:
>>>> https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1233
>>>
>>> That only helps if using bleeding edge toolchain scripts (which I
>>> regularly do and so did Patrick).
>>>
>>> Palmer has a fork of toolchain scripts and I'm assuming he hasn't caught
>>> up to that point ;-)
>>
>> I'm fine dropping the fork if the bugs have been fixed.  IIRC last
>> week we were still waiting for them to merge something?
> The testsuite was broken last week, but was fixed by
> https://github.com/riscv-collab/riscv-gnu-toolchain/pull/1247 which was
> merged last Friday.
>
> That might be the thing you were thinking about?

Probably, I'll go try and bump stuff and see if it works...

Thanks!

Re: RISC-V Test Errors and Failures

[Vineet Gupta]

On 5/16/23 19:53, Palmer Dabbelt wrote:
>
> Probably, I'll go try and bump stuff and see if it works... 

Word of caution: Best to not disturb your existing setup, a try a fresh 
checkout first

Re: RISC-V Test Errors and Failures

[Palmer Dabbelt]

On Tue, 16 May 2023 20:08:26 PDT (-0700), Vineet Gupta wrote:
>
> On 5/16/23 19:53, Palmer Dabbelt wrote:
>>
>> Probably, I'll go try and bump stuff and see if it works...
>
> Word of caution: Best to not disturb your existing setup, a try a fresh
> checkout first

Even easier, I think I can get away with just

diff --git a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
index 94d6ec5..efc3a80 100755
--- a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
+++ b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
@@ -12,4 +12,4 @@ done
 
 xlen="$(readelf -h $1 | grep 'Class' | cut -d: -f 2 | xargs echo | sed 's/^ELF//')"
 
-qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu rv$xlen,zba=on,zbb=on,zbc=on,zbs=on "$@"
+qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu rv$xlen,zba=on,zbb=on,zbc=on,zbs=on,v=on "$@"

for now.  I'm going to throw together hwprobe for qemu-user, from looking at
the AVX stuff it should be pretty easy to plumb that into DG and then get the
detection going.

Re: RISC-V Test Errors and Failures

[Kito Cheng]

> diff --git a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
> index 94d6ec5..efc3a80 100755
> --- a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
> +++ b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
> @@ -12,4 +12,4 @@ done
>
>  xlen="$(readelf -h $1 | grep 'Class' | cut -d: -f 2 | xargs echo | sed 's/^ELF//')"
>
> -qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu rv$xlen,zba=on,zbb=on,zbc=on,zbs=on "$@"
> +qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu rv$xlen,zba=on,zbb=on,zbc=on,zbs=on,v=on "$@"

This not work when you testing some combination e.g. Z*inx and zve*,
but anyway I guess those configurations are not matter for you guys :P

>
> for now.  I'm going to throw together hwprobe for qemu-user, from looking at
> the AVX stuff it should be pretty easy to plumb that into DG and then get the
> detection going.

Re: RISC-V Test Errors and Failures

[Jeff Law]

On 5/16/23 21:33, Kito Cheng via Gcc-patches wrote:
>> diff --git a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
>> index 94d6ec5..efc3a80 100755
>> --- a/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
>> +++ b/scripts/wrapper/qemu/riscv64-unknown-linux-gnu-run
>> @@ -12,4 +12,4 @@ done
>>
>>   xlen="$(readelf -h $1 | grep 'Class' | cut -d: -f 2 | xargs echo | sed 's/^ELF//')"
>>
>> -qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu rv$xlen,zba=on,zbb=on,zbc=on,zbs=on "$@"
>> +qemu-riscv$xlen -r 5.10 "${qemu_args[@]}" -L ${RISC_V_SYSROOT} -cpu rv$xlen,zba=on,zbb=on,zbc=on,zbs=on,v=on "$@"
> 
> This not work when you testing some combination e.g. Z*inx and zve*,
> but anyway I guess those configurations are not matter for you guys :P
What you could do is install a suitable binfmt handler, then you don't 
need the wrappers at all.  That's how we're handling this stuff in 
Ventana.  It also means you don't need magic dejagnu baseboard files or 
anything like that.  In fact from dejagnu's standpoint it looks native.

jeff

Re: RISC-V Test Errors and Failures

[Vineet Gupta]

On 5/25/23 13:29, Thomas Schwinge wrote:
> Hi!
>
> On 2023-05-17T09:52:13+0200, Andreas Schwab via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
>> On Mai 16 2023, Vineet Gupta wrote:
>>
>>> Yes I was seeing similar tcl errors and such - and in my case an even
>>> higher count.
>> They are coming from commit d6654a4be3b.
> I call FUD.  Until you prove otherwise, of coures.
>
>
> Grüße
>   Thomas

Just as a data point, with those patches reverted, I don't see the tcl 
errors.

2023-05-25 7a3c9f8e8362 Revert "Let each 'lto_init' determine the 
default 'LTO_OPTIONS', and 'torture-init' the 'LTO_TORTURE_OPTIONS'"
2023-05-25 22206cb760ee Revert "Testsuite: Add missing 
'torture-init'/'torture-finish' around 'LTO_TORTURE_OPTIONS' usage"
2023-05-25 db46b946dd6d Revert "Testsuite: Add 'torture-init-done', and 
use it to conditionalize implicit 'torture-init'"
2023-05-25 bd412162fd0d Revert "xxx vineet fixup"
2023-05-22 97a5e2241d33 xxx vineet fixup
2023-05-24 ec2e86274427 Fortran: reject bad DIM argument of SIZE 
intrinsic in simplification [PR104350]
...

                ========= Summary of gcc testsuite =========
                             | # of unexpected case / # of unique 
unexpected case
                             |          gcc |          g++ | gfortran |
  rv64imafdc/  lp64d/ medlow |   25 /     4 |    1 /     1 |   72 /    12 |
  rv32imafdc/ ilp32d/ medlow |   26 /     5 |    3 /     2 |   72 /    12 |
    rv32imac/  ilp32/ medlow |   25 /     4 |    3 /     2 |  109 /    19 |
    rv64imac/   lp64/ medlow |   26 /     5 |    1 /     1 |  109 /    19 |

Re: RISC-V Test Errors and Failures

[Thomas Schwinge]

Hi!

On 2023-05-17T09:52:13+0200, Andreas Schwab via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
> On Mai 16 2023, Vineet Gupta wrote:
>
>> Yes I was seeing similar tcl errors and such - and in my case an even
>> higher count.
>
> They are coming from commit d6654a4be3b.

I call FUD.  Until you prove otherwise, of coures.


Grüße
 Thomas

Re: RISC-V Test Errors and Failures

[Vineet Gupta]

On 5/17/23 00:52, Andreas Schwab wrote:
> On Mai 16 2023, Vineet Gupta wrote:
>
>> Yes I was seeing similar tcl errors and such - and in my case an even
>> higher count.
> They are coming from commit d6654a4be3b.
>

As of a726d007f197 today I get a gazzilion splat for riscv multilib 
dejagnu runs and over 5k fails

ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error code NONE
...
...
                ========= Summary of gcc testsuite =========
                             | # of unexpected case / # of unique 
unexpected case
                             |          gcc |          g++ | gfortran |
  rv64imafdc/  lp64d/ medlow | 5033 /     4 |    1 /     1 |   72 /    12 |
  rv32imafdc/ ilp32d/ medlow | 5032 /     3 |    3 /     2 |   72 /    12 |
    rv32imac/  ilp32/ medlow |    1 /     1 |    3 /     2 |  109 /    19 |
    rv64imac/   lp64/ medlow | 5034 /     5 |    1 /     1 |  109 /    19 |

For a non multilib run things are sane:

                ========= Summary of gcc testsuite =========
                             | # of unexpected case / # of unique 
unexpected case
                             |          gcc |          g++ | gfortran |
  rv64imafdc/  lp64d/ medlow |   11 /     4 |    1 /     1 |   72 /    12 |

It is really hard to test anything on upstream ATM.

-Vineet

Re: RISC-V Test Errors and Failures

[Andreas Schwab]

On Mai 16 2023, Vineet Gupta wrote:

> Yes I was seeing similar tcl errors and such - and in my case an even
> higher count.

They are coming from commit d6654a4be3b.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: RISC-V Test Errors and Failures

[Jeff Law]

On 5/16/23 20:39, Palmer Dabbelt wrote:

> 
> By "chroot environment" you mean something like a 
> debootstrap-into-chroot with qemu-user/binfmt-misc?  I don't have that 
> setup right now, but it wouldn't be a big lift.
Essentially, yes.  I actually have a home built ones for the various 
targets.  There was a time when they needed to be <100M so they could be 
stored on github.  Going forward I just want them to be a docker 
container for Fedora or Ubuntu.  With binfmt you can run non-native 
containers trivially.

jeff

Re: RISC-V Test Errors and Failures

[Palmer Dabbelt]

On Tue, 16 May 2023 19:32:21 PDT (-0700), jeffreyalaw@gmail.com wrote:
>
>
> On 5/16/23 20:05, Palmer Dabbelt wrote:
>> On Tue, 16 May 2023 19:00:12 PDT (-0700), Jeff Law wrote:
>>>
>>>
>>> On 5/16/23 19:29, Palmer Dabbelt wrote:
>>>
>>>>
>>>>> I think the most pressing need is bleeding edge gcc regression
>>>>> tracking.
>>>>>   @Jeff is anything setup on sourceware and/or usable ? I thought they
>>>>> do have existing bots for some arches to spin up build / run - perhaps
>>>>> runs are native and not qemu.
>>>>
>>>> IIRC Jeff said his builders were hanging right now.
>>> Correct.  More precisely, the riscv64 builds hang.  Not sure if it's
>>> stage2 or stage3 of the bootstrap.  Been happening for the last couple
>>> weeks.  I suspect some codegen bug in the riscv port.  I'll have to
>>> bisect it which will be quite painful.
>>
>> Can anyone else do it?  If the only blocker for having an upstream
>> regression CI thing is just sorting out why it broke over the last few
>> weeks then I'm happy to try and trick someone around here into doing
>> some work...
> Probably easiest for me unless someone else has a chroot environment
> handy.  It's not hard to do the bisection, it just involves a lot of
> waiting.

By "chroot environment" you mean something like a 
debootstrap-into-chroot with qemu-user/binfmt-misc?  I don't have that 
setup right now, but it wouldn't be a big lift.

> I've just about got the my problem from earlier today under control,
> then I can probably start bisection.

That's fine with me, I have plenty of other stuff to do ;)

Re: RISC-V Test Errors and Failures

[Jeff Law]

On 5/16/23 20:05, Palmer Dabbelt wrote:
> On Tue, 16 May 2023 19:00:12 PDT (-0700), Jeff Law wrote:
>>
>>
>> On 5/16/23 19:29, Palmer Dabbelt wrote:
>>
>>>
>>>> I think the most pressing need is bleeding edge gcc regression 
>>>> tracking.
>>>>   @Jeff is anything setup on sourceware and/or usable ? I thought they
>>>> do have existing bots for some arches to spin up build / run - perhaps
>>>> runs are native and not qemu.
>>>
>>> IIRC Jeff said his builders were hanging right now.
>> Correct.  More precisely, the riscv64 builds hang.  Not sure if it's
>> stage2 or stage3 of the bootstrap.  Been happening for the last couple
>> weeks.  I suspect some codegen bug in the riscv port.  I'll have to
>> bisect it which will be quite painful.
> 
> Can anyone else do it?  If the only blocker for having an upstream 
> regression CI thing is just sorting out why it broke over the last few 
> weeks then I'm happy to try and trick someone around here into doing 
> some work...
Probably easiest for me unless someone else has a chroot environment 
handy.  It's not hard to do the bisection, it just involves a lot of 
waiting.

I've just about got the my problem from earlier today under control, 
then I can probably start bisection.

Jeff

Re: RISC-V Test Errors and Failures

[Jeff Law]

On 5/16/23 20:08, Vineet Gupta wrote:

>>
>>> I think the most pressing need is bleeding edge gcc regression tracking.
>>>   @Jeff is anything setup on sourceware and/or usable ? I thought they
>>> do have existing bots for some arches to spin up build / run - perhaps
>>> runs are native and not qemu.
>>
>> IIRC Jeff said his builders were hanging right now.
> 
> Jeff it seems has his own test infra. I was ask
Mine is the closest we've got for project-wide testing.  Various orgs 
have their own servers/bots.

jeff

Re: RISC-V Test Errors and Failures

[Vineet Gupta]

On 5/16/23 18:29, Palmer Dabbelt wrote:
> On Tue, 16 May 2023 18:04:37 PDT (-0700), Vineet Gupta wrote:
>> + Christoph, Jiawei
>>
>> On 5/16/23 17:20, Palmer Dabbelt wrote:
>>>> We really need to add some CI around RV toolchains to trip on these
>>>> sooner !
>>>
>>> Sounds like you're volunteering to set one up?
>>
>> Patrick's github CI patch seems to be a great start. Lets wait for it to
>> get merged, that will at least catch rv toolchain snafus: although the
>> granularity of testing is not ideal (tc changes are not so frequent)
>
> You mean riscv-gnu-toolchain changes?  That's not super useful for GCC 
> development, they're on a fork.

Well they are still useful to catch various snafus in toolchain plumbing 
itself - I've run into 2 of those and Patcrick 2 himself, when trying to 
use latest upstream toolchain scripts. But sure they are not testing 
bleeding edge gcc.


>
>> I think the most pressing need is bleeding edge gcc regression tracking.
>>   @Jeff is anything setup on sourceware and/or usable ? I thought they
>> do have existing bots for some arches to spin up build / run - perhaps
>> runs are native and not qemu.
>
> IIRC Jeff said his builders were hanging right now.

Jeff it seems has his own test infra. I was asking if sourceware (or 
whatever the custodian of gcc project has).
I'd be really surprised if primary arches such as x86/aarch64 don't have 
any test bots there ?

>
>> FWIW rivos gitlab CI (not public) has capability to track upstream gcc
>> (Kevin almost has it working), but there is no easy way to publish it
>> for rest of the world and I'd rather that be done in a public infra.
>
> +Kevin
>
> At least having the failure lists public would be a must-have, and I 
> think that's tricky to do with gitlab.

Yep.

> Bjorn and Conor have something glued to the kernel patchwork that 
> uploads test results to github as snippits, but IIRC we're trying to 
> replace it with something more directly visible.
>
>> Didn't ISCAS/PLCT have such infra - sorry Kito asked the same question
>> this morning, but I was not fully awoke so don't remember what Jiawei
>> replied.
>
> I didn't even remember he asked ;)

Re: RISC-V Test Errors and Failures

[Palmer Dabbelt]

On Tue, 16 May 2023 19:00:12 PDT (-0700), Jeff Law wrote:
>
>
> On 5/16/23 19:29, Palmer Dabbelt wrote:
>
>>
>>> I think the most pressing need is bleeding edge gcc regression tracking.
>>>   @Jeff is anything setup on sourceware and/or usable ? I thought they
>>> do have existing bots for some arches to spin up build / run - perhaps
>>> runs are native and not qemu.
>>
>> IIRC Jeff said his builders were hanging right now.
> Correct.  More precisely, the riscv64 builds hang.  Not sure if it's
> stage2 or stage3 of the bootstrap.  Been happening for the last couple
> weeks.  I suspect some codegen bug in the riscv port.  I'll have to
> bisect it which will be quite painful.

Can anyone else do it?  If the only blocker for having an upstream 
regression CI thing is just sorting out why it broke over the last few 
weeks then I'm happy to try and trick someone around here into doing 
some work...

Re: RISC-V Test Errors and Failures

[Jeff Law]

On 5/16/23 19:29, Palmer Dabbelt wrote:

> 
>> I think the most pressing need is bleeding edge gcc regression tracking.
>>   @Jeff is anything setup on sourceware and/or usable ? I thought they
>> do have existing bots for some arches to spin up build / run - perhaps
>> runs are native and not qemu.
> 
> IIRC Jeff said his builders were hanging right now.
Correct.  More precisely, the riscv64 builds hang.  Not sure if it's 
stage2 or stage3 of the bootstrap.  Been happening for the last couple 
weeks.  I suspect some codegen bug in the riscv port.  I'll have to 
bisect it which will be quite painful.

jeff

Re: RISC-V Test Errors and Failures

[Palmer Dabbelt]

On Tue, 16 May 2023 18:04:37 PDT (-0700), Vineet Gupta wrote:
> + Christoph, Jiawei
>
> On 5/16/23 17:20, Palmer Dabbelt wrote:
>>> We really need to add some CI around RV toolchains to trip on these
>>> sooner !
>>
>> Sounds like you're volunteering to set one up?
>
> Patrick's github CI patch seems to be a great start. Lets wait for it to
> get merged, that will at least catch rv toolchain snafus: although the
> granularity of testing is not ideal (tc changes are not so frequent)

You mean riscv-gnu-toolchain changes?  That's not super useful for GCC 
development, they're on a fork.

> I think the most pressing need is bleeding edge gcc regression tracking.
>   @Jeff is anything setup on sourceware and/or usable ? I thought they
> do have existing bots for some arches to spin up build / run - perhaps
> runs are native and not qemu.

IIRC Jeff said his builders were hanging right now.

> FWIW rivos gitlab CI (not public) has capability to track upstream gcc
> (Kevin almost has it working), but there is no easy way to publish it
> for rest of the world and I'd rather that be done in a public infra.

+Kevin

At least having the failure lists public would be a must-have, and I 
think that's tricky to do with gitlab.  Bjorn and Conor have something 
glued to the kernel patchwork that uploads test results to github as 
snippits, but IIRC we're trying to replace it with something more 
directly visible.

> Didn't ISCAS/PLCT have such infra - sorry Kito asked the same question
> this morning, but I was not fully awoke so don't remember what Jiawei
> replied.

I didn't even remember he asked ;)

Re: RISC-V Test Errors and Failures

[Vineet Gupta]

+ Christoph, Jiawei

On 5/16/23 17:20, Palmer Dabbelt wrote:
>> We really need to add some CI around RV toolchains to trip on these 
>> sooner !
>
> Sounds like you're volunteering to set one up? 

Patrick's github CI patch seems to be a great start. Lets wait for it to 
get merged, that will at least catch rv toolchain snafus: although the 
granularity of testing is not ideal (tc changes are not so frequent)

I think the most pressing need is bleeding edge gcc regression tracking.
  @Jeff is anything setup on sourceware and/or usable ? I thought they 
do have existing bots for some arches to spin up build / run - perhaps 
runs are native and not qemu.

FWIW rivos gitlab CI (not public) has capability to track upstream gcc 
(Kevin almost has it working), but there is no easy way to publish it 
for rest of the world and I'd rather that be done in a public infra.

Didn't ISCAS/PLCT have such infra - sorry Kito asked the same question 
this morning, but I was not fully awoke so don't remember what Jiawei 
replied.

Thx,
-Vineet

Re: RISC-V Test Errors and Failures

[Palmer Dabbelt]

On Tue, 16 May 2023 17:16:11 PDT (-0700), Vineet Gupta wrote:
> On 5/16/23 16:06, Palmer Dabbelt wrote:
>> A few of us were talking about test-related issues in the patchwork
>> meeting
>> this morning.  I bumped to trunk and did a full rebuild, I'm getting the
>> following (it's in riscv-systems-ci/riscv-gnu-toolchain).  This is
>> about what I
>> remember seeing last time I ran the tests, which was a week or so ago.  I
>> figured it'd be best to just blast the lists, as Jeff said his test
>> running had
>> been hanging so there might be some issue preventing folks from seeing
>> the
>> failures.
>>
>> I guess I didn't get time to look last time and I doubt things are
>> looking any
>> better right now.  I'll try and take a look at some point, but any
>> help would
>> of course be appreciated.
>
> Yes I was seeing similar tcl errors and such - and in my case an even
> higher count.
> Also for posterity, what was your configure cmdline ? multilibs or no

If only I'd saved those in the build somewhere... :)

It's all in github.com/palmer-dabbelt/riscv-systems-ci, which points to 
riscv-gnu-toolchain.  I've always got uncommitted diff in my various 
local checkous, but I think this would only be

    toolchain: toolchain/install.stamp
    
    toolchain/install.stamp: toolchain/Makefile
            $(MAKE) -C $(dir $<)
            date > $@
    
    toolchain/Makefile: riscv-gnu-toolchain/configure
            mkdir -p $(dir $@)
            env -C $(dir $@) $(abspath $<) --prefix="$(abspath $(dir $@)/install)" --enable-linux --enable-multilib --enable-gcc-checking=yes
    
    toolchain/check.log: toolchain/install.stamp
            $(MAKE) -C $(dir $<) check \
                    GLIBC_TARGET_BOARDS_EXTRA="riscv-sim/-march=rv64gczba_zbb_zbc_zbs/-mabi=lp64d riscv-sim/-march=rv64imafdcv/-mabi=lp64d riscv-sim/-march=rv32imafdcv/-mabi=ilp32d" |& tee $@
            touch -c $@
    
    toolchain/report: toolchain/check.log
            $(MAKE) -C $(dir $<) report \
                    GLIBC_TARGET_BOARDS_EXTRA="riscv-sim/-march=rv64gczba_zbb_zbc_zbs/-mabi=lp64d riscv-sim/-march=rv64imafdcv/-mabi=lp64d riscv-sim/-march=rv32imafdcv/-mabi=ilp32d" |& tee $@
            touch -c $@

> We really need to add some CI around RV toolchains to trip on these sooner !

Sounds like you're volunteering to set one up?

>> $ cat toolchain/report
>> make[1]: Entering directory '/scratch/merges/rgt-gcc-trunk/toolchain'
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/scripts/testsuite-filter
>> gcc glibc
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/test/allowlist `find
>> build-gcc-linux-stage2/gcc/testsuite/ -name *.sum |paste -sd "," -`
>>         === g++: Unexpected fails for rv64imac lp64 medlow ===
>> FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
>>         === g++: Unexpected fails for rv32imac ilp32 medlow ===
>> FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
>> FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess
>> errors)
>> FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess
>> errors)
>>         === g++: Unexpected fails for rv64imafdc lp64d medlow ===
>> FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
>>         === g++: Unexpected fails for rv32imafdc ilp32d medlow ===
>> FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
>> FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess
>> errors)
>> FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess
>> errors)
>>         === g++: Unexpected fails for rv64imafdcv lp64d  ===
>> FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
>> FAIL: g++.target/riscv/rvv/base/bug-10.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-11.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-12.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-13.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-14.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-15.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-16.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-17.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-2.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-23.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-3.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-4.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-5.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-6.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-7.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-8.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-9.C execution test
>>         === g++: Unexpected fails for rv32imafdcv ilp32d  ===
>> FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
>> FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess
>> errors)
>> FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess
>> errors)
>> FAIL: g++.target/riscv/rvv/base/bug-10.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-11.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-12.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-13.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-14.C (test for excess errors)
>> FAIL: g++.target/riscv/rvv/base/bug-15.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-16.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-17.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-18.C (test for excess errors)
>> FAIL: g++.target/riscv/rvv/base/bug-19.C (test for excess errors)
>> FAIL: g++.target/riscv/rvv/base/bug-2.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-20.C (test for excess errors)
>> FAIL: g++.target/riscv/rvv/base/bug-21.C (test for excess errors)
>> FAIL: g++.target/riscv/rvv/base/bug-22.C (test for excess errors)
>> FAIL: g++.target/riscv/rvv/base/bug-23.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-3.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-4.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-5.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-6.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-7.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-8.C execution test
>> FAIL: g++.target/riscv/rvv/base/bug-9.C (test for excess errors)
>>         === g++: Unexpected fails for rv64gczba_zbb_zbc_zbs lp64d ===
>> FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
>>         === gcc: Unexpected fails for rv64imac lp64 medlow ===
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages,
>> line 14)
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>>         === gcc: Unexpected fails for rv32imac ilp32 medlow ===
>> XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages,
>> line 14)
>>         === gcc: Unexpected fails for rv64imafdc lp64d medlow ===
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages,
>> line 14)
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>>         === gcc: Unexpected fails for rv32imafdc ilp32d medlow ===
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages,
>> line 14)
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>>         === gcc: Unexpected fails for rv64imafdcv lp64d  ===
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages,
>> line 14)
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> FAIL: gcc.target/riscv/rvv/autovec/binop/shift-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/shift-scalar-run.c execution
>> test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vadd-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vand-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vdiv-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vmax-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vmin-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vmul-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vor-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vrem-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vsub-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vxor-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/series_run-1.c -O3 -ftree-vectorize
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/insert_run-1.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/insert_run-2.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-1.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-2.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-3.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-4.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-5.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-6.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vmv-imm-run.c -O3 -ftree-vectorize
>> (test for excess errors)
>> FAIL: gcc.target/riscv/rvv/base/bug-3.c execution test
>> FAIL: gcc.target/riscv/rvv/base/bug-4.c execution test
>> FAIL: gcc.target/riscv/rvv/base/bug-5.c execution test
>>         === gcc: Unexpected fails for rv32imafdcv ilp32d  ===
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages,
>> line 14)
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> FAIL: gcc.target/riscv/rvv/autovec/binop/shift-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/shift-scalar-run.c execution
>> test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vadd-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vand-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vdiv-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vmax-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vmin-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vmul-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vor-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vrem-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vsub-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/binop/vxor-run.c execution test
>> FAIL: gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/series_run-1.c -O3 -ftree-vectorize
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/insert_run-1.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/insert_run-2.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-1.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-2.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-3.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-4.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-5.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-6.c -std=c99
>> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax
>> execution test
>> FAIL: gcc.target/riscv/rvv/autovec/vmv-imm-run.c -O3 -ftree-vectorize
>> (test for excess errors)
>> FAIL: gcc.target/riscv/rvv/base/bug-3.c execution test
>> FAIL: gcc.target/riscv/rvv/base/bug-4.c execution test
>> FAIL: gcc.target/riscv/rvv/base/bug-5.c execution test
>>         === gcc: Unexpected fails for rv64gczba_zbb_zbc_zbs lp64d ===
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> FAIL: gcc.dg/optimize-bswapsi-1.c scan-tree-dump-times optimized "=
>> __builtin_bswap32 \\(" 6
>> FAIL: gcc.dg/optimize-bswapsi-2.c scan-tree-dump-times bswap "32 bit
>> bswap implementation found at" 4
>> FAIL: gcc.dg/optimize-bswapsi-3.c scan-tree-dump-times optimized "=
>> __builtin_bswap32 \\(" 1
>> FAIL: gcc.dg/optimize-bswapsi-4.c scan-tree-dump bswap "32 bit bswap
>> implementation found at"
>> FAIL: gcc.dg/optimize-bswapsi-5.c scan-tree-dump-times optimized "=
>> __builtin_bswap32 \\(" 2
>> FAIL: gcc.dg/optimize-bswapsi-6.c scan-tree-dump store-merging "32 bit
>> bswap implementation found at"
>> FAIL: gcc.dg/optimize-bswapsi-7.c scan-tree-dump-times optimized "=
>> __builtin_bswap32 \\(" 4
>> XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages,
>> line 14)
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>> ERROR: tcl error sourcing
>> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
>> ERROR: torture-init: torture_without_loops is not empty as expected
>>
>>               ========= Summary of gcc testsuite =========
>>                            | # of unexpected case / # of unique
>> unexpected case
>>                            |          gcc |          g++ | gfortran |
>> rv32imafdc/ ilp32d/ medlow |  833 /     3 |    3 /     2 |    0 /     0 |
>>   rv32imac/  ilp32/ medlow |    1 /     1 |    3 /     2 |    0 /     0 |
>> rv32imafdcv/ ilp32d/        |  950 /    29 |   25 /    24 |    0 /    
>> 0 |
>> rv64imafdcv/  lp64d/        |  950 /    29 |   18 /    18 |    0 /    
>> 0 |
>> rv64imafdc/  lp64d/ medlow |  833 /     3 |    1 /     1 |    0 /     0 |
>> rv64gczba_zbb_zbc_zbs/  lp64d/        |  840 /    10 |    1 / 1 |    0
>> /     0 |
>>   rv64imac/   lp64/ medlow |  833 /     3 |    1 /     1 |    0 /     0 |
>> make[1]: *** [Makefile:951: report-gcc-linux] Error 1
>> make[1]: Leaving directory '/scratch/merges/rgt-gcc-trunk/toolchain'

Re: RISC-V Test Errors and Failures

[Vineet Gupta]

On 5/16/23 16:06, Palmer Dabbelt wrote:
> A few of us were talking about test-related issues in the patchwork 
> meeting
> this morning.  I bumped to trunk and did a full rebuild, I'm getting the
> following (it's in riscv-systems-ci/riscv-gnu-toolchain).  This is 
> about what I
> remember seeing last time I ran the tests, which was a week or so ago.  I
> figured it'd be best to just blast the lists, as Jeff said his test 
> running had
> been hanging so there might be some issue preventing folks from seeing 
> the
> failures.
>
> I guess I didn't get time to look last time and I doubt things are 
> looking any
> better right now.  I'll try and take a look at some point, but any 
> help would
> of course be appreciated.

Yes I was seeing similar tcl errors and such - and in my case an even 
higher count.
Also for posterity, what was your configure cmdline ? multilibs or no
We really need to add some CI around RV toolchains to trip on these sooner !

>
> $ cat toolchain/report
> make[1]: Entering directory '/scratch/merges/rgt-gcc-trunk/toolchain'
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/scripts/testsuite-filter 
> gcc glibc 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/test/allowlist `find 
> build-gcc-linux-stage2/gcc/testsuite/ -name *.sum |paste -sd "," -`
>         === g++: Unexpected fails for rv64imac lp64 medlow ===
> FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
>         === g++: Unexpected fails for rv32imac ilp32 medlow ===
> FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
> FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess 
> errors)
> FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess 
> errors)
>         === g++: Unexpected fails for rv64imafdc lp64d medlow ===
> FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
>         === g++: Unexpected fails for rv32imafdc ilp32d medlow ===
> FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
> FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess 
> errors)
> FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess 
> errors)
>         === g++: Unexpected fails for rv64imafdcv lp64d  ===
> FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
> FAIL: g++.target/riscv/rvv/base/bug-10.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-11.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-12.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-13.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-14.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-15.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-16.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-17.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-2.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-23.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-3.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-4.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-5.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-6.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-7.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-8.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-9.C execution test
>         === g++: Unexpected fails for rv32imafdcv ilp32d  ===
> FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
> FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess 
> errors)
> FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess 
> errors)
> FAIL: g++.target/riscv/rvv/base/bug-10.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-11.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-12.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-13.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-14.C (test for excess errors)
> FAIL: g++.target/riscv/rvv/base/bug-15.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-16.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-17.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-18.C (test for excess errors)
> FAIL: g++.target/riscv/rvv/base/bug-19.C (test for excess errors)
> FAIL: g++.target/riscv/rvv/base/bug-2.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-20.C (test for excess errors)
> FAIL: g++.target/riscv/rvv/base/bug-21.C (test for excess errors)
> FAIL: g++.target/riscv/rvv/base/bug-22.C (test for excess errors)
> FAIL: g++.target/riscv/rvv/base/bug-23.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-3.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-4.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-5.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-6.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-7.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-8.C execution test
> FAIL: g++.target/riscv/rvv/base/bug-9.C (test for excess errors)
>         === g++: Unexpected fails for rv64gczba_zbb_zbc_zbs lp64d ===
> FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
>         === gcc: Unexpected fails for rv64imac lp64 medlow ===
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages, 
> line 14)
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
>         === gcc: Unexpected fails for rv32imac ilp32 medlow ===
> XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages, 
> line 14)
>         === gcc: Unexpected fails for rv64imafdc lp64d medlow ===
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages, 
> line 14)
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
>         === gcc: Unexpected fails for rv32imafdc ilp32d medlow ===
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages, 
> line 14)
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
>         === gcc: Unexpected fails for rv64imafdcv lp64d  ===
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages, 
> line 14)
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> FAIL: gcc.target/riscv/rvv/autovec/binop/shift-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/shift-scalar-run.c execution 
> test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vadd-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vand-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vdiv-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vmax-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vmin-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vmul-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vor-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vrem-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vsub-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vxor-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/series_run-1.c -O3 -ftree-vectorize 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/insert_run-1.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/insert_run-2.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-1.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-2.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-3.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-4.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-5.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-6.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vmv-imm-run.c -O3 -ftree-vectorize 
> (test for excess errors)
> FAIL: gcc.target/riscv/rvv/base/bug-3.c execution test
> FAIL: gcc.target/riscv/rvv/base/bug-4.c execution test
> FAIL: gcc.target/riscv/rvv/base/bug-5.c execution test
>         === gcc: Unexpected fails for rv32imafdcv ilp32d  ===
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages, 
> line 14)
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> FAIL: gcc.target/riscv/rvv/autovec/binop/shift-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/shift-scalar-run.c execution 
> test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vadd-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vand-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vdiv-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vmax-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vmin-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vmul-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vor-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vrem-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vsub-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/binop/vxor-run.c execution test
> FAIL: gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/series_run-1.c -O3 -ftree-vectorize 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/insert_run-1.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/insert_run-2.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-1.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-2.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-3.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-4.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-5.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-6.c -std=c99 
> -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax 
> execution test
> FAIL: gcc.target/riscv/rvv/autovec/vmv-imm-run.c -O3 -ftree-vectorize 
> (test for excess errors)
> FAIL: gcc.target/riscv/rvv/base/bug-3.c execution test
> FAIL: gcc.target/riscv/rvv/base/bug-4.c execution test
> FAIL: gcc.target/riscv/rvv/base/bug-5.c execution test
>         === gcc: Unexpected fails for rv64gczba_zbb_zbc_zbs lp64d ===
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> FAIL: gcc.dg/optimize-bswapsi-1.c scan-tree-dump-times optimized "= 
> __builtin_bswap32 \\(" 6
> FAIL: gcc.dg/optimize-bswapsi-2.c scan-tree-dump-times bswap "32 bit 
> bswap implementation found at" 4
> FAIL: gcc.dg/optimize-bswapsi-3.c scan-tree-dump-times optimized "= 
> __builtin_bswap32 \\(" 1
> FAIL: gcc.dg/optimize-bswapsi-4.c scan-tree-dump bswap "32 bit bswap 
> implementation found at"
> FAIL: gcc.dg/optimize-bswapsi-5.c scan-tree-dump-times optimized "= 
> __builtin_bswap32 \\(" 2
> FAIL: gcc.dg/optimize-bswapsi-6.c scan-tree-dump store-merging "32 bit 
> bswap implementation found at"
> FAIL: gcc.dg/optimize-bswapsi-7.c scan-tree-dump-times optimized "= 
> __builtin_bswap32 \\(" 4
> XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages, 
> line 14)
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
> ERROR: tcl error sourcing 
> /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
> ERROR: torture-init: torture_without_loops is not empty as expected
>
>               ========= Summary of gcc testsuite =========
>                            | # of unexpected case / # of unique 
> unexpected case
>                            |          gcc |          g++ | gfortran |
> rv32imafdc/ ilp32d/ medlow |  833 /     3 |    3 /     2 |    0 /     0 |
>   rv32imac/  ilp32/ medlow |    1 /     1 |    3 /     2 |    0 /     0 |
> rv32imafdcv/ ilp32d/        |  950 /    29 |   25 /    24 |    0 /     
> 0 |
> rv64imafdcv/  lp64d/        |  950 /    29 |   18 /    18 |    0 /     
> 0 |
> rv64imafdc/  lp64d/ medlow |  833 /     3 |    1 /     1 |    0 /     0 |
> rv64gczba_zbb_zbc_zbs/  lp64d/        |  840 /    10 |    1 / 1 |    0 
> /     0 |
>   rv64imac/   lp64/ medlow |  833 /     3 |    1 /     1 |    0 /     0 |
> make[1]: *** [Makefile:951: report-gcc-linux] Error 1
> make[1]: Leaving directory '/scratch/merges/rgt-gcc-trunk/toolchain'

RISC-V Test Errors and Failures

[Palmer Dabbelt]

A few of us were talking about test-related issues in the patchwork meeting
this morning.  I bumped to trunk and did a full rebuild, I'm getting the
following (it's in riscv-systems-ci/riscv-gnu-toolchain).  This is about what I
remember seeing last time I ran the tests, which was a week or so ago.  I
figured it'd be best to just blast the lists, as Jeff said his test running had
been hanging so there might be some issue preventing folks from seeing the
failures.

I guess I didn't get time to look last time and I doubt things are looking any
better right now.  I'll try and take a look at some point, but any help would
of course be appreciated.

$ cat toolchain/report
make[1]: Entering directory '/scratch/merges/rgt-gcc-trunk/toolchain'
/scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/scripts/testsuite-filter gcc glibc /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/test/allowlist `find build-gcc-linux-stage2/gcc/testsuite/ -name *.sum |paste -sd "," -`
		=== g++: Unexpected fails for rv64imac lp64 medlow ===
FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
		=== g++: Unexpected fails for rv32imac ilp32 medlow ===
FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess errors)
FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess errors)
		=== g++: Unexpected fails for rv64imafdc lp64d medlow ===
FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
		=== g++: Unexpected fails for rv32imafdc ilp32d medlow ===
FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess errors)
FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess errors)
		=== g++: Unexpected fails for rv64imafdcv lp64d  ===
FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
FAIL: g++.target/riscv/rvv/base/bug-10.C execution test
FAIL: g++.target/riscv/rvv/base/bug-11.C execution test
FAIL: g++.target/riscv/rvv/base/bug-12.C execution test
FAIL: g++.target/riscv/rvv/base/bug-13.C execution test
FAIL: g++.target/riscv/rvv/base/bug-14.C execution test
FAIL: g++.target/riscv/rvv/base/bug-15.C execution test
FAIL: g++.target/riscv/rvv/base/bug-16.C execution test
FAIL: g++.target/riscv/rvv/base/bug-17.C execution test
FAIL: g++.target/riscv/rvv/base/bug-2.C execution test
FAIL: g++.target/riscv/rvv/base/bug-23.C execution test
FAIL: g++.target/riscv/rvv/base/bug-3.C execution test
FAIL: g++.target/riscv/rvv/base/bug-4.C execution test
FAIL: g++.target/riscv/rvv/base/bug-5.C execution test
FAIL: g++.target/riscv/rvv/base/bug-6.C execution test
FAIL: g++.target/riscv/rvv/base/bug-7.C execution test
FAIL: g++.target/riscv/rvv/base/bug-8.C execution test
FAIL: g++.target/riscv/rvv/base/bug-9.C execution test
		=== g++: Unexpected fails for rv32imafdcv ilp32d  ===
FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2a (test for excess errors)
FAIL: g++.dg/modules/xtreme-header-5_c.C -std=c++2b (test for excess errors)
FAIL: g++.target/riscv/rvv/base/bug-10.C execution test
FAIL: g++.target/riscv/rvv/base/bug-11.C execution test
FAIL: g++.target/riscv/rvv/base/bug-12.C execution test
FAIL: g++.target/riscv/rvv/base/bug-13.C execution test
FAIL: g++.target/riscv/rvv/base/bug-14.C (test for excess errors)
FAIL: g++.target/riscv/rvv/base/bug-15.C execution test
FAIL: g++.target/riscv/rvv/base/bug-16.C execution test
FAIL: g++.target/riscv/rvv/base/bug-17.C execution test
FAIL: g++.target/riscv/rvv/base/bug-18.C (test for excess errors)
FAIL: g++.target/riscv/rvv/base/bug-19.C (test for excess errors)
FAIL: g++.target/riscv/rvv/base/bug-2.C execution test
FAIL: g++.target/riscv/rvv/base/bug-20.C (test for excess errors)
FAIL: g++.target/riscv/rvv/base/bug-21.C (test for excess errors)
FAIL: g++.target/riscv/rvv/base/bug-22.C (test for excess errors)
FAIL: g++.target/riscv/rvv/base/bug-23.C execution test
FAIL: g++.target/riscv/rvv/base/bug-3.C execution test
FAIL: g++.target/riscv/rvv/base/bug-4.C execution test
FAIL: g++.target/riscv/rvv/base/bug-5.C execution test
FAIL: g++.target/riscv/rvv/base/bug-6.C execution test
FAIL: g++.target/riscv/rvv/base/bug-7.C execution test
FAIL: g++.target/riscv/rvv/base/bug-8.C execution test
FAIL: g++.target/riscv/rvv/base/bug-9.C (test for excess errors)
		=== g++: Unexpected fails for rv64gczba_zbb_zbc_zbs lp64d  ===
FAIL: g++.dg/contracts/contracts-tmpl-spec2.C   output pattern test
		=== gcc: Unexpected fails for rv64imac lp64 medlow ===
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages, line 14)
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
		=== gcc: Unexpected fails for rv32imac ilp32 medlow ===
XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages, line 14)
		=== gcc: Unexpected fails for rv64imafdc lp64d medlow ===
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages, line 14)
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
		=== gcc: Unexpected fails for rv32imafdc ilp32d medlow ===
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages, line 14)
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
		=== gcc: Unexpected fails for rv64imafdcv lp64d  ===
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages, line 14)
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
FAIL: gcc.target/riscv/rvv/autovec/binop/shift-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/shift-scalar-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vadd-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vand-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vdiv-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vmax-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vmin-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vmul-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vor-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vrem-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vsub-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vxor-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c execution test
FAIL: gcc.target/riscv/rvv/autovec/series_run-1.c -O3 -ftree-vectorize execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/insert_run-1.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/insert_run-2.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-1.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-2.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-3.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-4.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-5.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-6.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vmv-imm-run.c -O3 -ftree-vectorize (test for excess errors)
FAIL: gcc.target/riscv/rvv/base/bug-3.c execution test
FAIL: gcc.target/riscv/rvv/base/bug-4.c execution test
FAIL: gcc.target/riscv/rvv/base/bug-5.c execution test
		=== gcc: Unexpected fails for rv32imafdcv ilp32d  ===
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages, line 14)
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
FAIL: gcc.target/riscv/rvv/autovec/binop/shift-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/shift-scalar-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vadd-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vand-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vdiv-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vmax-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vmin-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vmul-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vor-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vrem-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vsub-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/binop/vxor-run.c execution test
FAIL: gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c execution test
FAIL: gcc.target/riscv/rvv/autovec/series_run-1.c -O3 -ftree-vectorize execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/insert_run-1.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/insert_run-2.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-1.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-2.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-3.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-4.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-5.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vls-vlmax/repeat_run-6.c -std=c99 -O3 -ftree-vectorize --param riscv-autovec-preference=fixed-vlmax execution test
FAIL: gcc.target/riscv/rvv/autovec/vmv-imm-run.c -O3 -ftree-vectorize (test for excess errors)
FAIL: gcc.target/riscv/rvv/base/bug-3.c execution test
FAIL: gcc.target/riscv/rvv/base/bug-4.c execution test
FAIL: gcc.target/riscv/rvv/base/bug-5.c execution test
		=== gcc: Unexpected fails for rv64gczba_zbb_zbc_zbs lp64d  ===
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/builtins/builtins.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/execute/ieee/ieee.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.c-torture/unsorted/unsorted.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
FAIL: gcc.dg/optimize-bswapsi-1.c scan-tree-dump-times optimized "= __builtin_bswap32 \\(" 6
FAIL: gcc.dg/optimize-bswapsi-2.c scan-tree-dump-times bswap "32 bit bswap implementation found at" 4
FAIL: gcc.dg/optimize-bswapsi-3.c scan-tree-dump-times optimized "= __builtin_bswap32 \\(" 1
FAIL: gcc.dg/optimize-bswapsi-4.c scan-tree-dump bswap "32 bit bswap implementation found at"
FAIL: gcc.dg/optimize-bswapsi-5.c scan-tree-dump-times optimized "= __builtin_bswap32 \\(" 2
FAIL: gcc.dg/optimize-bswapsi-6.c scan-tree-dump store-merging "32 bit bswap implementation found at"
FAIL: gcc.dg/optimize-bswapsi-7.c scan-tree-dump-times optimized "= __builtin_bswap32 \\(" 4
XPASS: gcc.dg/uninit-pr101912.c pr101912 (test for bogus messages, line 14)
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/format/format.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/guality/guality.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/pch/pch.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sancov/sancov.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/simulate-thread/simulate-thread.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/sso/sso.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/stackalign/stackalign.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/torture/tls/tls.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.dg/tsan/tsan.exp.
ERROR: torture-init: torture_without_loops is not empty as expected
ERROR: tcl error sourcing /scratch/merges/rgt-gcc-trunk/riscv-gnu-toolchain/gcc/gcc/testsuite/gcc.misc-tests/i386-prefetch.exp.
ERROR: torture-init: torture_without_loops is not empty as expected

               ========= Summary of gcc testsuite =========
                            | # of unexpected case / # of unique unexpected case
                            |          gcc |          g++ |     gfortran |
 rv32imafdc/ ilp32d/ medlow |  833 /     3 |    3 /     2 |    0 /     0 |
   rv32imac/  ilp32/ medlow |    1 /     1 |    3 /     2 |    0 /     0 |
 rv32imafdcv/ ilp32d/        |  950 /    29 |   25 /    24 |    0 /     0 |
 rv64imafdcv/  lp64d/        |  950 /    29 |   18 /    18 |    0 /     0 |
 rv64imafdc/  lp64d/ medlow |  833 /     3 |    1 /     1 |    0 /     0 |
 rv64gczba_zbb_zbc_zbs/  lp64d/        |  840 /    10 |    1 /     1 |    0 /     0 |
   rv64imac/   lp64/ medlow |  833 /     3 |    1 /     1 |    0 /     0 |
make[1]: *** [Makefile:951: report-gcc-linux] Error 1
make[1]: Leaving directory '/scratch/merges/rgt-gcc-trunk/toolchain'