Re: Aw: Re: Invalid "dg-do run" in the testsuite (with 2 blanks)

[Richard Biener <richard.guenther@gmail.com>]

On Tue, Mar 26, 2024 at 2:24 AM Jerry D <jvdelisle2@gmail.com> wrote:
>
> On 3/25/24 3:30 PM, Harald Anlauf wrote:
> >> On 3/25/24 12:53 PM, Harald Anlauf wrote:
> >>>>>>> I noticed by chance that we have quite a lot of improperly specified do-do
> >>>>>>> directives in the testsuite.
> >>>>>>>
> >>>>>>> % grep "dg-do  run" gcc/testsuite/gfortran.dg/ -rl|wc -l
> >>>>>>> 83
> >>>>>>>
> >>>>>
> >>>>> I think this is on purpose.
> >>>>> The idea to use a "feature" in dejagnu to only iterate once and not
> >>>>> over all possible options. So execution time can be lowered a bit.
> >>>>>
> >>>>> But I don't know if this hack still works, it definitely did work some years ago.
> >>>>>
> >>>>> Cheers,
> >>>>> Manfred
> >>>
> >>> Is this "feature" documented somewhere?  I don't see it on
> >>>
> >>> https://gcc.gnu.org/onlinedocs/gcc-13.2.0/gccint/Directives.html
> >>>
> >>> Given that the dg-directives are important and possibly fragile,
> >>> and since we had issues in the past, can we check that a test
> >>> that was added works the way intended?
> >>>
> >>>>>>> Note that with two blanks instead of just one a testcase does not get executed.
> >>>>>>>
> >>>>>>> Does anybody want to earn the honors to change the directives and
> >>>>>>> check for "fallout" in the testsuite?
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>> Harald
> >>>>>>>
> >>>>>>
> >>>>>> One failure after fixing all the spaces ( sed is our friend ).
> >>>>>>
> >>>>>> FAIL: gfortran.dg/inline_matmul_1.f90   -O0   scan-tree-dump-times optimized "_gfortran_matmul" 0
> >>>
> >>> This does actually point to an issue with the testcase:
> >>> it only works properly with optimization enabled.
> >>>
> >>> Manual inspection of this test and the expected dump suggests
> >>> that e.g. -O1 could have been added to the dg-options directive.
> >>>
> >>> Shouldn't we fix at least the dg-options of that testcase?
> >>>
> >>> Cheers,
> >>> Harald
> >>>
> >>
> >> I restored the one test that appeared to fail so that it had the two
> >> spaces 'trick'.  When run in the test suite, it is compiled with -O
> >> which does invoke the optimization.  I manually checked the tree dump
> >> with this option and it indeed has all the _gfortran_matmul calls removed.
> >>
> >> I am inclined to leave these all untouched with the two spaces in place.
> >>
> >>   From the test log:
> >>
> >> PASS: gfortran.dg/inline_matmul_1.f90   -O  execution test
> >> PASS: gfortran.dg/inline_matmul_1.f90   -O   scan-tree-dump-times
> >> optimized "_gfortran_matmul" 0
> >
> > Alright, then leave it that way.
> >
> > I find it somewhat unsatisfactory though, to have a behavior of the
> > testsuite harness that is so intranparent.
> >
> > If it is a simple oversight that the behavior of double space was
> > never documented, it could simply be fixed, for the sake of everybody.
> >
> > The take-home message for me is - whenever I write a testcase that
> > relies on this behavior - to add a comment in the header that this
> > is intended behavior, and set all compiler flags appropriately...
> >
> > Cheers,
> > Harald
> >
> >
>
> With two spaces there are 9 tests run at -O, with one space there are 54
> variations executed. Specifying the optimization option in the
> dg-options also runs 54 variations.  It appears that the "feature" was
> done on purpose, but it is frustrating that this is not documented anywhere.

It would be nice to document it in sourcebuild.texi and to put a comment before
the dg-do with two spaces indicating that two spaces are on purpose.

I also don't remember this being implemented ...

Richard.

> I do remember discussions way back about some tests taking a long time
> to run and needing a way to speed up the overall testing.  I will look
> over the documentation and see if we can get this addressed.
>
> For my own test cases, I have always specified the options I want and
> for front-end work, optimizations don't usually matter except in this
> case.  I don't have time to go through all 83 tests.
>
>  From my system here:
>
> Testing of trunk complete..... one space
>
> real    4m56.294s
> user    35m51.082s
> sys     12m51.283s
>
>
> Testing of trunk complete..... two spaces
>
> real    4m44.421s
> user    34m3.215s
> sys     12m38.778s
>
> Not a huge difference.
>
> Jerry
>
>