Re: Aw: Re: Invalid "dg-do run" in the testsuite (with 2 blanks)

[Jerry D <jvdelisle2@gmail.com>]

On 3/25/24 3:30 PM, Harald Anlauf wrote:
>> On 3/25/24 12:53 PM, Harald Anlauf wrote:
>>>>>>> I noticed by chance that we have quite a lot of improperly specified do-do
>>>>>>> directives in the testsuite.
>>>>>>>
>>>>>>> % grep "dg-do  run" gcc/testsuite/gfortran.dg/ -rl|wc -l
>>>>>>> 83
>>>>>>>
>>>>>
>>>>> I think this is on purpose.
>>>>> The idea to use a "feature" in dejagnu to only iterate once and not
>>>>> over all possible options. So execution time can be lowered a bit.
>>>>>
>>>>> But I don't know if this hack still works, it definitely did work some years ago.
>>>>>
>>>>> Cheers,
>>>>> Manfred
>>>
>>> Is this "feature" documented somewhere?  I don't see it on
>>>
>>> https://gcc.gnu.org/onlinedocs/gcc-13.2.0/gccint/Directives.html
>>>
>>> Given that the dg-directives are important and possibly fragile,
>>> and since we had issues in the past, can we check that a test
>>> that was added works the way intended?
>>>
>>>>>>> Note that with two blanks instead of just one a testcase does not get executed.
>>>>>>>
>>>>>>> Does anybody want to earn the honors to change the directives and
>>>>>>> check for "fallout" in the testsuite?
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Harald
>>>>>>>
>>>>>>
>>>>>> One failure after fixing all the spaces ( sed is our friend ).
>>>>>>
>>>>>> FAIL: gfortran.dg/inline_matmul_1.f90   -O0   scan-tree-dump-times optimized "_gfortran_matmul" 0
>>>
>>> This does actually point to an issue with the testcase:
>>> it only works properly with optimization enabled.
>>>
>>> Manual inspection of this test and the expected dump suggests
>>> that e.g. -O1 could have been added to the dg-options directive.
>>>
>>> Shouldn't we fix at least the dg-options of that testcase?
>>>
>>> Cheers,
>>> Harald
>>>
>>
>> I restored the one test that appeared to fail so that it had the two
>> spaces 'trick'.  When run in the test suite, it is compiled with -O
>> which does invoke the optimization.  I manually checked the tree dump
>> with this option and it indeed has all the _gfortran_matmul calls removed.
>>
>> I am inclined to leave these all untouched with the two spaces in place.
>>
>>   From the test log:
>>
>> PASS: gfortran.dg/inline_matmul_1.f90   -O  execution test
>> PASS: gfortran.dg/inline_matmul_1.f90   -O   scan-tree-dump-times
>> optimized "_gfortran_matmul" 0
> 
> Alright, then leave it that way.
> 
> I find it somewhat unsatisfactory though, to have a behavior of the
> testsuite harness that is so intranparent.
> 
> If it is a simple oversight that the behavior of double space was
> never documented, it could simply be fixed, for the sake of everybody.
> 
> The take-home message for me is - whenever I write a testcase that
> relies on this behavior - to add a comment in the header that this
> is intended behavior, and set all compiler flags appropriately...
> 
> Cheers,
> Harald
> 
> 

With two spaces there are 9 tests run at -O, with one space there are 54 
variations executed. Specifying the optimization option in the 
dg-options also runs 54 variations.  It appears that the "feature" was 
done on purpose, but it is frustrating that this is not documented anywhere.

I do remember discussions way back about some tests taking a long time 
to run and needing a way to speed up the overall testing.  I will look 
over the documentation and see if we can get this addressed.

For my own test cases, I have always specified the options I want and 
for front-end work, optimizations don't usually matter except in this 
case.  I don't have time to go through all 83 tests.

 From my system here:

Testing of trunk complete..... one space

real	4m56.294s
user	35m51.082s
sys	12m51.283s


Testing of trunk complete..... two spaces

real	4m44.421s
user	34m3.215s
sys	12m38.778s

Not a huge difference.

Jerry