[PATCH] testsuite: outputs.exp: test for skip_atsave more thoroughly

[PATCH] testsuite: outputs.exp: test for skip_atsave more thoroughly

[Alexandre Oliva]

The presence of -I or -L flags in link command lines changes the
driver's, and thus the linker's behavior, WRT naming files with
command-line options.  With such flags, the driver creates .args.0 and
.args.1 files, whereas without them it's the linker (collect2, really)
that creates .ld1_args.

I've hit some fails on a target system that doesn't have -I or -L
flags in the board config file, but it does add some of them
implicitly with configured-in driver self specs.  Alas, the test in
outputs.exp doesn't catch that, so we proceed to run rather than
skip_atsave tests.

I've reworked the outest procedure to allow dry runs and to return
would-have-been pass/fail results as lists, so we can now test whether
certain files are created and use that to configure the actual test
runs.

Regstrapped on x86_64-linux-gnu, also tested with a cross to
aarch64-rtems6.  Ok to install?


for  gcc/testsuite/ChangeLog

	* gcc.misc-tests/outputs.exp (outest): Introduce quiet mode,
	create and return lists of passes and fails.  Use it to catch
	skip_atsave cases where -L flags are implicitly added by
	driver self specs.
---
 gcc/testsuite/gcc.misc-tests/outputs.exp |   49 ++++++++++++++++++++++++++----
 1 file changed, 42 insertions(+), 7 deletions(-)

diff --git a/gcc/testsuite/gcc.misc-tests/outputs.exp b/gcc/testsuite/gcc.misc-tests/outputs.exp
index afae735e92d76..a63ce66693b97 100644
--- a/gcc/testsuite/gcc.misc-tests/outputs.exp
+++ b/gcc/testsuite/gcc.misc-tests/outputs.exp
@@ -116,8 +116,23 @@ if [info exists env(MAKEFLAGS)] {
 # it weren't for
 # https://core.tcl-lang.org/tcl/tktview?name=5bbd044812), but .{i,s,o}
 # and .[iso] will pass even if only the .o is present.
+
+# Return a list containing two lists, the first naming the passes, the
+# second naming the fails.  If test ends with a question mark, the
+# test is taken as a preparatory test or cleanup, and no pass or fail
+# results will be logged, though the lists will still be built and
+# returned.
 array unset outests *
 proc outest { test sources opts dirs outputs } {
+    if { [string index $test end] == "?" } {
+	set quiet 1
+    } else {
+	set quiet 0
+    }
+
+    set passes {}
+    set fails {}
+
     global b srcdir subdir
     global outests
 
@@ -182,15 +197,15 @@ proc outest { test sources opts dirs outputs } {
 		set o "$og"
 	    }
 	    if { [file exists $d$o] } then {
-		pass "$test: $d$o"
+		lappend passes "$d$o"
 		file delete $d$o
 	    } else {
 	        set ogl [glob -nocomplain -path $d -- $o]
 		if { $ogl != {} } {
-		    pass "$test: $d$o"
+		    lappend passes "$d$o"
 		    file delete $ogl
 		} else {
-		    fail "$test: $d$o"
+		    lappend fails "$d$o"
 		}
 	    }
 	}
@@ -219,17 +234,27 @@ proc outest { test sources opts dirs outputs } {
     }
 
     if { [llength $outb] == 0 } then {
-	pass "$test: extra"
+	lappend passes "extra"
     } else {
-	fail "$test: extra\n$outb"
+	lappend fails "extra\n$outb"
     }
 
     if { [string equal "$gcc_output" ""] } then {
-	pass "$test: std out"
+	lappend passes "std out"
     } else {
-	fail "$test: std out\n$gcc_output"
+	lappend fails "std out\n$gcc_output"
     }
 
+    if !$quiet {
+	foreach p $passes {
+	    pass "$test: $p"
+	}
+	foreach f $fails {
+	    fail "$test: $f"
+	}
+    }
+
+    return [list $passes $fails]
 }
 
 set sing {-0.c}
@@ -279,6 +304,16 @@ if { "$aout" != "" } then {
     set oaout "-o $aout"
 }
 
+# Sometimes the -I or -L flags that cause the compiler driver to save
+# .args.[01], instead of leaving it for the linker to save .ld1_args,
+# is hiding in driver self specs.
+if !$skip_atsave {
+    set atsave_test_out [outest "$b-skip-atsave?" $sing "@/dev/null -o $b.exe -save-temps" {} {{.args.1}}]
+    if { [lindex [lindex $atsave_test_out 0] 0] == "$b.args.1" } {
+	set skip_atsave 1
+    }
+}
+
 # Driver-chosen outputs.
 outest "$b-1 asm default 1" $sing "-S" {} {{-0.s}}
 outest "$b-2 asm default 2" $mult "-S" {} {{-1.s -2.s}}

-- 
Alexandre Oliva, happy hacker                https://FSFLA.org/blogs/lxo/
   Free Software Activist                       GNU Toolchain Engineer
Disinformation flourishes because many people care deeply about injustice
but very few check the facts.  Ask me about <https://stallmansupport.org>

[PATCH] testsuite: outputs.exp: cleanup before running tests (was: Re: [PATCH] testsuite: outputs.exp: test for skip_atsave more thoroughly)

[Alexandre Oliva]

On Jun 21, 2022, Alexandre Oliva <oliva@adacore.com> wrote:

> 	* gcc.misc-tests/outputs.exp (outest): Introduce quiet mode,

Use the just-added dry-run infrastructure to clean up files that may
have been left over by interrupted runs of outputs.exp, which used to
lead to spurious non-repeatable (self-fixing) failures.

Regstrapped on x86_64-linux-gnu, also tested with a cross to
aarch64-rtems6.  Ok to install?


for  gcc/testsuite/ChangeLog

	* gcc.misc-tests/outputs.exp: Clean up left-overs first.
---
 gcc/testsuite/gcc.misc-tests/outputs.exp |    3 +++
 1 file changed, 3 insertions(+)

diff --git a/gcc/testsuite/gcc.misc-tests/outputs.exp b/gcc/testsuite/gcc.misc-tests/outputs.exp
index a63ce66693b97..ab919db1ccb2d 100644
--- a/gcc/testsuite/gcc.misc-tests/outputs.exp
+++ b/gcc/testsuite/gcc.misc-tests/outputs.exp
@@ -304,6 +304,9 @@ if { "$aout" != "" } then {
     set oaout "-o $aout"
 }
 
+# Clean up any left-overs from an earlier interrupted run.
+outest "$b-cleanup?" $sing "$oaout" {alt/ dir/ o/ od/ obj/} {{} {} {} {} {} {$aout}}
+
 # Sometimes the -I or -L flags that cause the compiler driver to save
 # .args.[01], instead of leaving it for the linker to save .ld1_args,
 # is hiding in driver self specs.


-- 
Alexandre Oliva, happy hacker                https://FSFLA.org/blogs/lxo/
   Free Software Activist                       GNU Toolchain Engineer
Disinformation flourishes because many people care deeply about injustice
but very few check the facts.  Ask me about <https://stallmansupport.org>

Re: [PATCH] testsuite: outputs.exp: test for skip_atsave more thoroughly

[Richard Biener]

On Tue, Jun 21, 2022 at 7:45 AM Alexandre Oliva via Gcc-patches
<gcc-patches@gcc.gnu.org> wrote:
>
>
> The presence of -I or -L flags in link command lines changes the
> driver's, and thus the linker's behavior, WRT naming files with
> command-line options.  With such flags, the driver creates .args.0 and
> .args.1 files, whereas without them it's the linker (collect2, really)
> that creates .ld1_args.
>
> I've hit some fails on a target system that doesn't have -I or -L
> flags in the board config file, but it does add some of them
> implicitly with configured-in driver self specs.  Alas, the test in
> outputs.exp doesn't catch that, so we proceed to run rather than
> skip_atsave tests.
>
> I've reworked the outest procedure to allow dry runs and to return
> would-have-been pass/fail results as lists, so we can now test whether
> certain files are created and use that to configure the actual test
> runs.
>
> Regstrapped on x86_64-linux-gnu, also tested with a cross to
> aarch64-rtems6.  Ok to install?

OK.

Thanks

>
> for  gcc/testsuite/ChangeLog
>
>         * gcc.misc-tests/outputs.exp (outest): Introduce quiet mode,
>         create and return lists of passes and fails.  Use it to catch
>         skip_atsave cases where -L flags are implicitly added by
>         driver self specs.
> ---
>  gcc/testsuite/gcc.misc-tests/outputs.exp |   49 ++++++++++++++++++++++++++----
>  1 file changed, 42 insertions(+), 7 deletions(-)
>
> diff --git a/gcc/testsuite/gcc.misc-tests/outputs.exp b/gcc/testsuite/gcc.misc-tests/outputs.exp
> index afae735e92d76..a63ce66693b97 100644
> --- a/gcc/testsuite/gcc.misc-tests/outputs.exp
> +++ b/gcc/testsuite/gcc.misc-tests/outputs.exp
> @@ -116,8 +116,23 @@ if [info exists env(MAKEFLAGS)] {
>  # it weren't for
>  # https://core.tcl-lang.org/tcl/tktview?name=5bbd044812), but .{i,s,o}
>  # and .[iso] will pass even if only the .o is present.
> +
> +# Return a list containing two lists, the first naming the passes, the
> +# second naming the fails.  If test ends with a question mark, the
> +# test is taken as a preparatory test or cleanup, and no pass or fail
> +# results will be logged, though the lists will still be built and
> +# returned.
>  array unset outests *
>  proc outest { test sources opts dirs outputs } {
> +    if { [string index $test end] == "?" } {
> +       set quiet 1
> +    } else {
> +       set quiet 0
> +    }
> +
> +    set passes {}
> +    set fails {}
> +
>      global b srcdir subdir
>      global outests
>
> @@ -182,15 +197,15 @@ proc outest { test sources opts dirs outputs } {
>                 set o "$og"
>             }
>             if { [file exists $d$o] } then {
> -               pass "$test: $d$o"
> +               lappend passes "$d$o"
>                 file delete $d$o
>             } else {
>                 set ogl [glob -nocomplain -path $d -- $o]
>                 if { $ogl != {} } {
> -                   pass "$test: $d$o"
> +                   lappend passes "$d$o"
>                     file delete $ogl
>                 } else {
> -                   fail "$test: $d$o"
> +                   lappend fails "$d$o"
>                 }
>             }
>         }
> @@ -219,17 +234,27 @@ proc outest { test sources opts dirs outputs } {
>      }
>
>      if { [llength $outb] == 0 } then {
> -       pass "$test: extra"
> +       lappend passes "extra"
>      } else {
> -       fail "$test: extra\n$outb"
> +       lappend fails "extra\n$outb"
>      }
>
>      if { [string equal "$gcc_output" ""] } then {
> -       pass "$test: std out"
> +       lappend passes "std out"
>      } else {
> -       fail "$test: std out\n$gcc_output"
> +       lappend fails "std out\n$gcc_output"
>      }
>
> +    if !$quiet {
> +       foreach p $passes {
> +           pass "$test: $p"
> +       }
> +       foreach f $fails {
> +           fail "$test: $f"
> +       }
> +    }
> +
> +    return [list $passes $fails]
>  }
>
>  set sing {-0.c}
> @@ -279,6 +304,16 @@ if { "$aout" != "" } then {
>      set oaout "-o $aout"
>  }
>
> +# Sometimes the -I or -L flags that cause the compiler driver to save
> +# .args.[01], instead of leaving it for the linker to save .ld1_args,
> +# is hiding in driver self specs.
> +if !$skip_atsave {
> +    set atsave_test_out [outest "$b-skip-atsave?" $sing "@/dev/null -o $b.exe -save-temps" {} {{.args.1}}]
> +    if { [lindex [lindex $atsave_test_out 0] 0] == "$b.args.1" } {
> +       set skip_atsave 1
> +    }
> +}
> +
>  # Driver-chosen outputs.
>  outest "$b-1 asm default 1" $sing "-S" {} {{-0.s}}
>  outest "$b-2 asm default 2" $mult "-S" {} {{-1.s -2.s}}
>
> --
> Alexandre Oliva, happy hacker                https://FSFLA.org/blogs/lxo/
>    Free Software Activist                       GNU Toolchain Engineer
> Disinformation flourishes because many people care deeply about injustice
> but very few check the facts.  Ask me about <https://stallmansupport.org>

Re: [PATCH] testsuite: outputs.exp: cleanup before running tests (was: Re: [PATCH] testsuite: outputs.exp: test for skip_atsave more thoroughly)

[Richard Biener]

On Tue, Jun 21, 2022 at 7:47 AM Alexandre Oliva via Gcc-patches
<gcc-patches@gcc.gnu.org> wrote:
>
> On Jun 21, 2022, Alexandre Oliva <oliva@adacore.com> wrote:
>
> >       * gcc.misc-tests/outputs.exp (outest): Introduce quiet mode,
>
> Use the just-added dry-run infrastructure to clean up files that may
> have been left over by interrupted runs of outputs.exp, which used to
> lead to spurious non-repeatable (self-fixing) failures.
>
> Regstrapped on x86_64-linux-gnu, also tested with a cross to
> aarch64-rtems6.  Ok to install?

OK

>
> for  gcc/testsuite/ChangeLog
>
>         * gcc.misc-tests/outputs.exp: Clean up left-overs first.
> ---
>  gcc/testsuite/gcc.misc-tests/outputs.exp |    3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/gcc/testsuite/gcc.misc-tests/outputs.exp b/gcc/testsuite/gcc.misc-tests/outputs.exp
> index a63ce66693b97..ab919db1ccb2d 100644
> --- a/gcc/testsuite/gcc.misc-tests/outputs.exp
> +++ b/gcc/testsuite/gcc.misc-tests/outputs.exp
> @@ -304,6 +304,9 @@ if { "$aout" != "" } then {
>      set oaout "-o $aout"
>  }
>
> +# Clean up any left-overs from an earlier interrupted run.
> +outest "$b-cleanup?" $sing "$oaout" {alt/ dir/ o/ od/ obj/} {{} {} {} {} {} {$aout}}
> +
>  # Sometimes the -I or -L flags that cause the compiler driver to save
>  # .args.[01], instead of leaving it for the linker to save .ld1_args,
>  # is hiding in driver self specs.
>
>
> --
> Alexandre Oliva, happy hacker                https://FSFLA.org/blogs/lxo/
>    Free Software Activist                       GNU Toolchain Engineer
> Disinformation flourishes because many people care deeply about injustice
> but very few check the facts.  Ask me about <https://stallmansupport.org>