[testsuite] skip ARM tests if no thumb2 support

[testsuite] skip ARM tests if no thumb2 support

[Janis Johnson]

[-- Attachment #1: Type: text/plain, Size: 118 bytes --]

These tests apparently require thumb2 support (I don't yet know much
about ARM).  OK for trunk, and later 4.6?

Janis

[-- Attachment #2: gcc-20110614-4 --]
[-- Type: text/plain, Size: 1132 bytes --]

2011-06-14  Janis Johnson  <janisjo@codesourcery.com>

	* gcc.target/arm/pr42879.c: Skip if no thumb2 support, ignore
	compiler warning about switch conflicts.
	* gcc.target/arm/pr45701-3.c: Likewise.

Index: gcc.target/arm/pr42879.c
===================================================================
--- gcc.target/arm/pr42879.c	(revision 175047)
+++ gcc.target/arm/pr42879.c	(working copy)
@@ -1,4 +1,6 @@
+/* { dg-require-effective-target arm_thumb2_ok } */
 /* { dg-options "-march=armv7-a -mthumb -Os" }  */
+/* { dg-prune-output "switch .* conflicts with" } */
 /* { dg-final { scan-assembler "lsls" } } */
 
 struct A
Index: gcc.target/arm/pr45701-3.c
===================================================================
--- gcc.target/arm/pr45701-3.c	(revision 175047)
+++ gcc.target/arm/pr45701-3.c	(working copy)
@@ -1,5 +1,7 @@
 /* { dg-do compile } */
+/* { dg-require-effective-target arm_thumb2_ok } */
 /* { dg-options "-march=armv7-a -mthumb -Os" }  */
+/* { dg-prune-output "switch .* conflicts with" } */
 /* { dg-final { scan-assembler "push\t.*r8" } } */
 /* { dg-final { scan-assembler-not "push\t*r3" } } */
 

Re: [testsuite] skip ARM tests if no thumb2 support

[Ramana Radhakrishnan]

On 14 June 2011 21:35, Janis Johnson <janisjo@codesourcery.com> wrote:
> These tests apparently require thumb2 support (I don't yet know much
> about ARM).  OK for trunk, and later 4.6?

OK - The -march=armv7-a is redundant in these tests. You should be
able to lose them if arm_thumb2_ok returns true.

cheers
Ramana

Re: [testsuite] skip ARM tests if no thumb2 support

[Janis Johnson]

On 06/15/2011 01:13 AM, Ramana Radhakrishnan wrote:
> On 14 June 2011 21:35, Janis Johnson <janisjo@codesourcery.com> wrote:
>> These tests apparently require thumb2 support (I don't yet know much
>> about ARM).  OK for trunk, and later 4.6?
> 
> OK - The -march=armv7-a is redundant in these tests. You should be
> able to lose them if arm_thumb2_ok returns true.

That brings up an important point.  Without any changes, this test
passes scan-assembler when run with multilibs that don't override the
-march= option, but fails the scan when -march from multilibs
overrides the one from dg-options and doesn't support thumb2.  With
"dg-require-effective-target arm_thumb2_ok" the test is skipped for
multilibs that don't support thumb2, including those that default to an
arch that doesn't support it.

If we want this and similar tests to continue to be run for default
multilibs with default support for older values of -march then instead
of using ""dg-require-effective-target arm_thumb2_ok" the test can skip
multilibs that use -march that don't support thumb2, e.g.

/* { dg-skip-if "need thumb2" { arm*-*-* } { "-march=*" } { "-march=armv6t2" "-march=armv[7-9]*" "-march=armv[1-9][0-9]*" } } */

In summary, the choice for a test like this is:

1. Provide options needed to test particular functionality, and skip the
   test if multilib flags that override those options would not test the
   relevant functionality.
2. Skip the test if multilib options on their own do not provide the
   functionality that the test covers.

Option 2 assumes that there is enough test coverage for later arch
versions, either with default support or multilib options.  This choice
applies to several tests in gcc.target/arm.

Janis

Re: [testsuite] skip ARM tests if no thumb2 support

[Richard Earnshaw]

On 15/06/11 17:38, Janis Johnson wrote:
> On 06/15/2011 01:13 AM, Ramana Radhakrishnan wrote:
>> On 14 June 2011 21:35, Janis Johnson <janisjo@codesourcery.com> wrote:
>>> These tests apparently require thumb2 support (I don't yet know much
>>> about ARM).  OK for trunk, and later 4.6?
>>
>> OK - The -march=armv7-a is redundant in these tests. You should be
>> able to lose them if arm_thumb2_ok returns true.
> 
> That brings up an important point.  Without any changes, this test
> passes scan-assembler when run with multilibs that don't override the
> -march= option, but fails the scan when -march from multilibs
> overrides the one from dg-options and doesn't support thumb2.  With
> "dg-require-effective-target arm_thumb2_ok" the test is skipped for
> multilibs that don't support thumb2, including those that default to an
> arch that doesn't support it.
> 
> If we want this and similar tests to continue to be run for default
> multilibs with default support for older values of -march then instead
> of using ""dg-require-effective-target arm_thumb2_ok" the test can skip
> multilibs that use -march that don't support thumb2, e.g.
> 
> /* { dg-skip-if "need thumb2" { arm*-*-* } { "-march=*" } { "-march=armv6t2" "-march=armv[7-9]*" "-march=armv[1-9][0-9]*" } } */
> 
> In summary, the choice for a test like this is:
> 
> 1. Provide options needed to test particular functionality, and skip the
>    test if multilib flags that override those options would not test the
>    relevant functionality.
> 2. Skip the test if multilib options on their own do not provide the
>    functionality that the test covers.
> 

I'd posit a third option:

 3. Add a new dg directive (perhaps dg-ignore-multilib) that instructs
the framework to ignore the multilib options entirely (only supported
for compile/assembly tests).

There are a lot of target-specific tests that are not really testing
execution, just that the compiler generates the right instruction under
specific compilation options.  It seems silly to force all this into the
multi-lib framework.

R.

> Option 2 assumes that there is enough test coverage for later arch
> versions, either with default support or multilib options.  This choice
> applies to several tests in gcc.target/arm.
> 
> Janis
> 
> 
> 

Re: [testsuite] skip ARM tests if no thumb2 support

[Janis Johnson]

On 06/29/2011 06:55 AM, Richard Earnshaw wrote:
> On 15/06/11 17:38, Janis Johnson wrote:
>> On 06/15/2011 01:13 AM, Ramana Radhakrishnan wrote:
>>> On 14 June 2011 21:35, Janis Johnson <janisjo@codesourcery.com> wrote:
>>>> These tests apparently require thumb2 support (I don't yet know much
>>>> about ARM).  OK for trunk, and later 4.6?
>>>
>>> OK - The -march=armv7-a is redundant in these tests. You should be
>>> able to lose them if arm_thumb2_ok returns true.
>>
>> That brings up an important point.  Without any changes, this test
>> passes scan-assembler when run with multilibs that don't override the
>> -march= option, but fails the scan when -march from multilibs
>> overrides the one from dg-options and doesn't support thumb2.  With
>> "dg-require-effective-target arm_thumb2_ok" the test is skipped for
>> multilibs that don't support thumb2, including those that default to an
>> arch that doesn't support it.
>>
>> If we want this and similar tests to continue to be run for default
>> multilibs with default support for older values of -march then instead
>> of using ""dg-require-effective-target arm_thumb2_ok" the test can skip
>> multilibs that use -march that don't support thumb2, e.g.
>>
>> /* { dg-skip-if "need thumb2" { arm*-*-* } { "-march=*" } { "-march=armv6t2" "-march=armv[7-9]*" "-march=armv[1-9][0-9]*" } } */
>>
>> In summary, the choice for a test like this is:
>>
>> 1. Provide options needed to test particular functionality, and skip the
>>    test if multilib flags that override those options would not test the
>>    relevant functionality.
>> 2. Skip the test if multilib options on their own do not provide the
>>    functionality that the test covers.
>>
> 
> I'd posit a third option:
> 
>  3. Add a new dg directive (perhaps dg-ignore-multilib) that instructs
> the framework to ignore the multilib options entirely (only supported
> for compile/assembly tests).
> 
> There are a lot of target-specific tests that are not really testing
> execution, just that the compiler generates the right instruction under
> specific compilation options.  It seems silly to force all this into the
> multi-lib framework.
> 
> R.

The MIPS tests have support for something like that that, but it seems
to me that if tests are being run for lots of multilibs then it's
better to test things like code generation for a variety of relevant
options instead of using the same options every time.  

Having said that, I'm happy that you and other ARM maintainers and
developers are now thinking about how multilibs and dg-options interact
and how you want the tests to be run.  My primary goal is to get rid of
spurious failures when we run tests internally with 10 or 12 multilibs.
So far I've been stumbling about trying to skip tests when multilibs
don't support them, but I'll follow whatever approach you guys prefer.

Janis

>> Option 2 assumes that there is enough test coverage for later arch
>> versions, either with default support or multilib options.  This choice
>> applies to several tests in gcc.target/arm.
>>
>> Janis
>>
> 

Re: [testsuite] skip ARM tests if no thumb2 support

[Richard Sandiford]

Janis Johnson <janisjo@codesourcery.com> writes:
> On 06/29/2011 06:55 AM, Richard Earnshaw wrote:
>> I'd posit a third option:
>> 
>>  3. Add a new dg directive (perhaps dg-ignore-multilib) that instructs
>> the framework to ignore the multilib options entirely (only supported
>> for compile/assembly tests).
>> 
>> There are a lot of target-specific tests that are not really testing
>> execution, just that the compiler generates the right instruction under
>> specific compilation options.  It seems silly to force all this into the
>> multi-lib framework.
>> 
>> R.
>
> The MIPS tests have support for something like that that, but it seems
> to me that if tests are being run for lots of multilibs then it's
> better to test things like code generation for a variety of relevant
> options instead of using the same options every time.  

I might be misunderstanding what you mean, but the MIPS version tries
to keep as many of the original options as possible, only overriding
those that are known to be incompatible with the test.  So e.g. a
test for a MIPS IV feature will run normally on a MIPS IV-compatible
target, but will override the architecture on other targets.

That said, I wouldn't recommend the mips.exp approach to anyone.
It's been useful, and it seems to do what it's meant to, but it's
a very... heavy-weight approach.

Richard

Re: [testsuite] skip ARM tests if no thumb2 support

[Janis Johnson]

On 06/29/2011 06:55 AM, Richard Earnshaw wrote:

>  3. Add a new dg directive (perhaps dg-ignore-multilib) that instructs
> the framework to ignore the multilib options entirely (only supported
> for compile/assembly tests).
> 
> There are a lot of target-specific tests that are not really testing
> execution, just that the compiler generates the right instruction under
> specific compilation options.  It seems silly to force all this into the
> multi-lib framework.

If it's true that most testsuite runs on ARM include a default multilib,
then some tests can be skipped for everything other than the default
multilib and specify all the flags that are needed for the test, e.g.
with

  /* { dg-require-effective-target default_multilib } */

That's much simpler than ignoring multilib flags, and avoids running
the same test multiple times with exactly the same options.

Janis