Re: [testsuite] skip ARM tests with conflicting options

[Janis Johnson <janisjo@codesourcery.com>]

On 06/08/2011 03:39 AM, Richard Earnshaw wrote:
> On 08/06/11 03:14, Janis Johnson wrote:
>> On 06/07/2011 06:25 PM, Mike Stump wrote:
>>> On Jun 7, 2011, at 4:24 PM, Janis Johnson wrote:
>>>> On 06/07/2011 02:07 PM, Joseph S. Myers wrote:
>>>>> On Tue, 7 Jun 2011, Janis Johnson wrote:
>>>>>
>>>>>> Several tests in gcc.target/arm use dg-options with -mcpu=xxxx, which
>>>>>> causes compiler warnings or errors when the multilib flags include
>>>>>> -march=yyyy.  This patch causes those tests to be skipped.  It also
>>>>>> prevents gcc.target/arm/20090811-1.c from running with multilibs that
>>>>>> would override -mcpu or -mfloat-abi options specified for the test.
>>>>>
>>>>> I think you should allow compatible -march options - for example, if 
>>>>> dg-options has -mcpu=cortex-a8, allow -march=armv7-a but disallow all 
>>>>> other -march options.
>>>>>
>>>> Is this one OK?
>>>
>>> Not sure if the arm people want to review this or would rather I review it...
>>>
>>> Let's give the arm folks a couple days to comment, if no objections, Ok.
>>>
>>> A point of warning, eventually, you'll discover that when a compiler defaults to the argument you want to skip, that you'll needs slightly more power to skip them.  darwin ran into this with things like -m64, and eventually had to do something like lp64.  configure options like --with-cpu=arm9 are the sort that can change the default.
>>
>> Yes, I hope to hear from ARM people.
>>
>> On ARM, the default from --with-cpu= is overridden by -march at
>> compile so it's not a problem for this particular set of tests.
>> As I said originally, this set is the tip of the iceberg and they
>> get more difficult.
>>
>> Janis
>>
>>
> 
> 
> I'm worried by this whole approach of command-line checking.  It works,
> just about, for testsuite variations set with target_list, but it won't
> work with options used to configure the compiler (eg --with-mode=thumb,
> or --with-cpu=...).  Perhaps a better approach would be a new dg- test
> that built a trivial file with all the options and disabled the test if
> that test failed for any reason.  Something like:
> 
> dg-target-compatible (target, <compile|link>, additional-opts)
> 
> The test is only performed if target matches the current target.
> 
> I'm not sure if this is something that can be easily cached (well, it
> might be possible if you could index off additional-opts and the default
> opts), so it might be that this test has to be re-run every time there
> is a test that needs it.
> 
> R.

There's a similar functionality now with effective targets that end with
"_ok"; they test with options that would be added with a later directive.
The problem is that they can't be specific enough for what the test is
looking for; arm_neon_fp16_ok, for example, currently passes when the
multilib options include -mfpu=neon, which overrides the options provided
by dg-options, and that leads to problems.

I think that a test that requires a specific option should provide that
option, but be skipped if multilib options include a conflict.  This
includes -mcpu, -mfpu, -march, -mfloat-abi, -mbig-endian, and possibly
more.  Those options override the defaults for the configuration, so we
shouldn't have to worry about the defaults.

The big question is whether such a test should be run for all multilibs
that might possibly pass the test, or only for default and for mulitlibs
that provide the same options.  There are a lot of arm tests that use
-march but pass for a great many other -march options.  In most cases
they use -march with the value for which the problem was reported.
Should those tests be run for all multilibs, with the multilib options
overriding the "defaults" for the test from dg-options, or should they
be skipped multilibs that use other values?  The answer might depend on
the individual test; maybe some should be run for a large number of
multilib options to find problems with specific ones, while others can
be limited to run just once or a few times.

Janis