[PATCH 0/4] Fix library testsuite compilation for build sysroot

[PATCH 0/4] Fix library testsuite compilation for build sysroot

[Maciej W. Rozycki]

Hi,

 This patch series addresses a problem with the testsuite compiler being 
set up across libatomic, libffi, libgo, libgomp with no correlation 
whatsoever to the target compiler being used in GCC compilation.  
Consequently there in no arrangement made to set up the compilation 
sysroot according to the build sysroot specified for GCC compilation, 
causing a catastrophic failure across the testsuites affected from the 
inability to link executables.

 The fix is based on a similar arrangement already made for passing 
autoconf output variables in libgomp, and uses the GCC_UNDER_TEST (or 
GOC_UNDER_TEST, as applicable) TCL variable already used across several 
GCC library testsuites.

 Verified with a cross-compiler configured for the `riscv-linux-gnu' 
target and the `x86_64-linux-gnu' host and using RISC-V/Linux QEMU in the 
user emulation mode as the target board.  Also no change in results with 
`x86_64-linux-gnu' native regression testing.

 See individual change descriptions for details.

 OK to apply to the GCC repo (for libraries maintained externally I'll be 
happy to assist with any merging required, although given that these 
changes are confined to autoconf/automake scriptery they should be 
straightforward to apply, barring any conflicts in generated files)?

  Maciej

[PATCH 1/4] libatomic/test: Fix compilation for build sysroot

[Maciej W. Rozycki]

Fix a problem with the libatomic testsuite using a method to determine 
the compiler to use resulting in the tool being different from one the 
library has been built with, and causing a catastrophic failure from the 
lack of a suitable `--sysroot=' option where the `--with-build-sysroot=' 
configuration option has been used to build the compiler resulting in 
the inability to link executables.

Address this problem by providing a DejaGNU configuration file defining 
the compiler to use, via the GCC_UNDER_TEST TCL variable, set from $CC 
by autoconf, which will have all the required options set for the target 
compiler to build executables in the environment configured, removing 
failures like:

.../bin/riscv64-linux-gnu-ld: cannot find crt1.o: No such file or directory
.../bin/riscv64-linux-gnu-ld: cannot find -lm
collect2: error: ld returned 1 exit status
compiler exited with status 1
FAIL: libatomic.c/atomic-compare-exchange-1.c (test for excess errors)
Excess errors:
.../bin/riscv64-linux-gnu-ld: cannot find crt1.o: No such file or directory
.../bin/riscv64-linux-gnu-ld: cannot find -lm

UNRESOLVED: libatomic.c/atomic-compare-exchange-1.c compilation failed to produce executable

and bringing overall test results for the `riscv64-linux-gnu' target 
(here with the `x86_64-linux-gnu' host and RISC-V QEMU in the Linux user 
emulation mode as the target board) from:

		=== libatomic Summary ===

# of unexpected failures	27
# of unresolved testcases	27

to:

		=== libatomic Summary ===

# of expected passes		54

	libatomic/
	* configure.ac: Add testsuite/libatomic-test-support.exp to 
	output files.
	* configure: Regenerate.
	* testsuite/libatomic-test-support.exp.in: New file.
	* testsuite/Makefile.am (EXTRA_DEJAGNU_SITE_CONFIG): New 
	variable.
	* testsuite/Makefile.in: Regenerate.
---
 libatomic/configure                               |    3 +++
 libatomic/configure.ac                            |    1 +
 libatomic/testsuite/Makefile.am                   |    2 ++
 libatomic/testsuite/Makefile.in                   |    5 ++++-
 libatomic/testsuite/libatomic-test-support.exp.in |    1 +
 5 files changed, 11 insertions(+), 1 deletion(-)

gcc-test-libatomic-gcc-under-test.diff
Index: gcc/libatomic/configure
===================================================================
--- gcc.orig/libatomic/configure
+++ gcc/libatomic/configure
@@ -15701,6 +15701,8 @@ fi
 
 ac_config_files="$ac_config_files Makefile testsuite/Makefile"
 
+ac_config_files="$ac_config_files testsuite/libatomic-test-support.exp"
+
 cat >confcache <<\_ACEOF
 # This file is a shell script that caches the results of configure
 # tests run on this system so they can be shared between configure
@@ -16772,6 +16774,7 @@ do
     "gstdint.h") CONFIG_COMMANDS="$CONFIG_COMMANDS gstdint.h" ;;
     "Makefile") CONFIG_FILES="$CONFIG_FILES Makefile" ;;
     "testsuite/Makefile") CONFIG_FILES="$CONFIG_FILES testsuite/Makefile" ;;
+    "testsuite/libatomic-test-support.exp") CONFIG_FILES="$CONFIG_FILES testsuite/libatomic-test-support.exp" ;;
 
   *) as_fn_error $? "invalid argument: \`$ac_config_target'" "$LINENO" 5;;
   esac
Index: gcc/libatomic/configure.ac
===================================================================
--- gcc.orig/libatomic/configure.ac
+++ gcc/libatomic/configure.ac
@@ -279,4 +279,5 @@ else
 fi
 
 AC_CONFIG_FILES(Makefile testsuite/Makefile)
+AC_CONFIG_FILES(testsuite/libatomic-test-support.exp)
 AC_OUTPUT
Index: gcc/libatomic/testsuite/Makefile.am
===================================================================
--- gcc.orig/libatomic/testsuite/Makefile.am
+++ gcc/libatomic/testsuite/Makefile.am
@@ -11,3 +11,5 @@ EXPECT = $(shell if test -f $(top_buildd
 _RUNTEST = $(shell if test -f $(top_srcdir)/../dejagnu/runtest; then \
 	     echo $(top_srcdir)/../dejagnu/runtest; else echo runtest; fi)
 RUNTEST = $(_RUNTEST) $(AM_RUNTESTFLAGS)
+
+EXTRA_DEJAGNU_SITE_CONFIG = libatomic-test-support.exp
Index: gcc/libatomic/testsuite/Makefile.in
===================================================================
--- gcc.orig/libatomic/testsuite/Makefile.in
+++ gcc/libatomic/testsuite/Makefile.in
@@ -108,7 +108,7 @@ am__configure_deps = $(am__aclocal_m4_de
 DIST_COMMON = $(srcdir)/Makefile.am
 mkinstalldirs = $(SHELL) $(top_srcdir)/../mkinstalldirs
 CONFIG_HEADER = $(top_builddir)/auto-config.h
-CONFIG_CLEAN_FILES =
+CONFIG_CLEAN_FILES = libatomic-test-support.exp
 CONFIG_CLEAN_VPATH_FILES =
 AM_V_P = $(am__v_P_@AM_V@)
 am__v_P_ = $(am__v_P_@AM_DEFAULT_V@)
@@ -277,6 +277,7 @@ _RUNTEST = $(shell if test -f $(top_srcd
 	     echo $(top_srcdir)/../dejagnu/runtest; else echo runtest; fi)
 
 RUNTEST = $(_RUNTEST) $(AM_RUNTESTFLAGS)
+EXTRA_DEJAGNU_SITE_CONFIG = libatomic-test-support.exp
 all: all-am
 
 .SUFFIXES:
@@ -309,6 +310,8 @@ $(top_srcdir)/configure: @MAINTAINER_MOD
 $(ACLOCAL_M4): @MAINTAINER_MODE_TRUE@ $(am__aclocal_m4_deps)
 	cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh
 $(am__aclocal_m4_deps):
+libatomic-test-support.exp: $(top_builddir)/config.status $(srcdir)/libatomic-test-support.exp.in
+	cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@
 
 mostlyclean-libtool:
 	-rm -f *.lo
Index: gcc/libatomic/testsuite/libatomic-test-support.exp.in
===================================================================
--- /dev/null
+++ gcc/libatomic/testsuite/libatomic-test-support.exp.in
@@ -0,0 +1 @@
+set GCC_UNDER_TEST {@CC@}

[PATCH 2/4] libffi/test: Fix compilation for build sysroot

[Maciej W. Rozycki]

Fix a problem with the libffi testsuite using a method to determine the 
compiler to use resulting in the tool being different from one the 
library has been built with, and causing a catastrophic failure from the 
inability to actually choose any compiler at all in a cross-compilation
configuration.

Address this problem by providing a DejaGNU configuration file defining 
the compiler to use, via the GCC_UNDER_TEST TCL variable, set from $CC 
by autoconf, which will have all the required options set for the target 
compiler to build executables in the environment configured, removing 
failures like:

FAIL: libffi.call/closure_fn0.c -W -Wall -Wno-psabi -O0 (test for excess errors)
Excess errors:
default_target_compile: No compiler to compile with
UNRESOLVED: libffi.call/closure_fn0.c -W -Wall -Wno-psabi -O0 compilation failed to produce executable

and bringing overall test results for the `riscv64-linux-gnu' target 
(here with the `x86_64-linux-gnu' host and RISC-V QEMU in the Linux user 
emulation mode as the target board) from:

		=== libffi Summary ===

# of unexpected failures	708
# of unresolved testcases	708
# of unsupported tests		30

to:

		=== libffi Summary ===

# of expected passes		1934
# of unsupported tests		28

Also respect the TOOL_EXECUTABLE TCL variable for a standalone run via 
`runtest' and remove an unused TOOL_OPTIONS TCL variable instance.

	libffi/
	* configure.ac: Add testsuite/libffi-test-support.exp to output 
	files.
	* configure: Regenerate.
	* testsuite/libffi-test-support.exp.in: New file.
	* testsuite/Makefile.am (EXTRA_DEJAGNU_SITE_CONFIG): New 
	variable.
	* testsuite/Makefile.in: Regenerate.
	* testsuite/lib/libffi.exp (libffi-init): Handle GCC_UNDER_TEST.
	(libffi_target_compile): Likewise.
---
 libffi/configure                            |    3 ++-
 libffi/configure.ac                         |    2 +-
 libffi/testsuite/Makefile.am                |    2 ++
 libffi/testsuite/Makefile.in                |    5 ++++-
 libffi/testsuite/lib/libffi.exp             |   16 ++++++++++++++--
 libffi/testsuite/libffi-test-support.exp.in |    1 +
 6 files changed, 24 insertions(+), 5 deletions(-)

gcc-test-libffi-gcc-under-test.diff
Index: gcc/libffi/configure
===================================================================
--- gcc.orig/libffi/configure
+++ gcc/libffi/configure
@@ -16635,7 +16635,7 @@ ac_config_commands="$ac_config_commands
 ac_config_links="$ac_config_links include/ffitarget.h:src/$TARGETDIR/ffitarget.h"
 
 
-ac_config_files="$ac_config_files include/Makefile include/ffi.h Makefile testsuite/Makefile man/Makefile libffi.pc"
+ac_config_files="$ac_config_files include/Makefile include/ffi.h Makefile testsuite/Makefile testsuite/libffi-test-support.exp man/Makefile libffi.pc"
 
 
 cat >confcache <<\_ACEOF
@@ -17802,6 +17802,7 @@ do
     "include/ffi.h") CONFIG_FILES="$CONFIG_FILES include/ffi.h" ;;
     "Makefile") CONFIG_FILES="$CONFIG_FILES Makefile" ;;
     "testsuite/Makefile") CONFIG_FILES="$CONFIG_FILES testsuite/Makefile" ;;
+    "testsuite/libffi-test-support.exp") CONFIG_FILES="$CONFIG_FILES testsuite/libffi-test-support.exp" ;;
     "man/Makefile") CONFIG_FILES="$CONFIG_FILES man/Makefile" ;;
     "libffi.pc") CONFIG_FILES="$CONFIG_FILES libffi.pc" ;;
 
Index: gcc/libffi/configure.ac
===================================================================
--- gcc.orig/libffi/configure.ac
+++ gcc/libffi/configure.ac
@@ -368,6 +368,6 @@ test -d src/$TARGETDIR || mkdir src/$TAR
 
 AC_CONFIG_LINKS(include/ffitarget.h:src/$TARGETDIR/ffitarget.h)
 
-AC_CONFIG_FILES(include/Makefile include/ffi.h Makefile testsuite/Makefile man/Makefile libffi.pc)
+AC_CONFIG_FILES(include/Makefile include/ffi.h Makefile testsuite/Makefile testsuite/libffi-test-support.exp man/Makefile libffi.pc)
 
 AC_OUTPUT
Index: gcc/libffi/testsuite/Makefile.am
===================================================================
--- gcc.orig/libffi/testsuite/Makefile.am
+++ gcc/libffi/testsuite/Makefile.am
@@ -11,6 +11,8 @@ RUNTEST = `if [ -f $(top_srcdir)/../deja
 	       echo $(top_srcdir)/../dejagnu/runtest ; \
 	    else echo runtest; fi`
 
+EXTRA_DEJAGNU_SITE_CONFIG = libffi-test-support.exp
+
 AM_RUNTESTFLAGS =
 
 CLEANFILES = *.exe core* *.log *.sum
Index: gcc/libffi/testsuite/Makefile.in
===================================================================
--- gcc.orig/libffi/testsuite/Makefile.in
+++ gcc/libffi/testsuite/Makefile.in
@@ -105,7 +105,7 @@ am__configure_deps = $(am__aclocal_m4_de
 DIST_COMMON = $(srcdir)/Makefile.am
 mkinstalldirs = $(SHELL) $(top_srcdir)/../mkinstalldirs
 CONFIG_HEADER = $(top_builddir)/fficonfig.h
-CONFIG_CLEAN_FILES =
+CONFIG_CLEAN_FILES = libffi-test-support.exp
 CONFIG_CLEAN_VPATH_FILES =
 AM_V_P = $(am__v_P_@AM_V@)
 am__v_P_ = $(am__v_P_@AM_DEFAULT_V@)
@@ -278,6 +278,7 @@ RUNTEST = `if [ -f $(top_srcdir)/../deja
 	       echo $(top_srcdir)/../dejagnu/runtest ; \
 	    else echo runtest; fi`
 
+EXTRA_DEJAGNU_SITE_CONFIG = libffi-test-support.exp
 CLEANFILES = *.exe core* *.log *.sum
 EXTRA_DIST = config/default.exp libffi.call/cls_19byte.c		\
 libffi.call/cls_align_longdouble_split.c				\
@@ -389,6 +390,8 @@ $(top_srcdir)/configure: @MAINTAINER_MOD
 $(ACLOCAL_M4): @MAINTAINER_MODE_TRUE@ $(am__aclocal_m4_deps)
 	cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh
 $(am__aclocal_m4_deps):
+libffi-test-support.exp: $(top_builddir)/config.status $(srcdir)/libffi-test-support.exp.in
+	cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@
 
 mostlyclean-libtool:
 	-rm -f *.lo
Index: gcc/libffi/testsuite/lib/libffi.exp
===================================================================
--- gcc.orig/libffi/testsuite/lib/libffi.exp
+++ gcc/libffi/testsuite/lib/libffi.exp
@@ -99,7 +99,8 @@ proc libffi-init { args } {
     global blddirffi
     global objdir
     global blddircxx
-    global TOOL_OPTIONS
+    global TOOL_EXECUTABLE
+    global GCC_UNDER_TEST
     global tool
     global libffi_include
     global libffi_link_flags
@@ -123,7 +124,15 @@ proc libffi-init { args } {
     set ld_library_path "."
     append ld_library_path ":${gccdir}"
 
-    set compiler "${gccdir}/xgcc"
+    if ![info exists GCC_UNDER_TEST] then {
+	if [info exists TOOL_EXECUTABLE] {
+	    set GCC_UNDER_TEST $TOOL_EXECUTABLE
+	} else {
+	    set GCC_UNDER_TEST "[find_gcc]"
+	}
+    }
+
+    set compiler [lindex $GCC_UNDER_TEST 0]
     if { [is_remote host] == 0 && [which $compiler] != 0 } {
 	foreach i "[exec $compiler --print-multi-lib]" {
 	    set mldir ""
@@ -175,11 +184,14 @@ proc libffi_target_compile { source dest
     global srcdir
     global blddirffi
     global TOOL_OPTIONS
+    global GCC_UNDER_TEST
     global libffi_link_flags
     global libffi_include
     global target_triplet
 
 
+    lappend options "compiler=$GCC_UNDER_TEST"
+
     if { [target_info needs_status_wrapper]!="" && [info exists gluefile] } {
 	lappend options "libs=${gluefile}"
 	lappend options "ldflags=$wrap_flags"
Index: gcc/libffi/testsuite/libffi-test-support.exp.in
===================================================================
--- /dev/null
+++ gcc/libffi/testsuite/libffi-test-support.exp.in
@@ -0,0 +1 @@
+set GCC_UNDER_TEST {@CC@}

Re: [golang-dev] [PATCH 0/4] Fix library testsuite compilation for build sysroot

[Ulderico Cirello]

Hi Maciej,


Go's project doesn't take mail patches for changes. Please use gerrit (
https://go-review.googlesource.com/ ).


Thanks





        - ccf


On Mon, Nov 11, 2019 at 10:12 AM Maciej W. Rozycki <macro@wdc.com> wrote:

> Hi,
>
>  This patch series addresses a problem with the testsuite compiler being
> set up across libatomic, libffi, libgo, libgomp with no correlation
> whatsoever to the target compiler being used in GCC compilation.
> Consequently there in no arrangement made to set up the compilation
> sysroot according to the build sysroot specified for GCC compilation,
> causing a catastrophic failure across the testsuites affected from the
> inability to link executables.
>
>  The fix is based on a similar arrangement already made for passing
> autoconf output variables in libgomp, and uses the GCC_UNDER_TEST (or
> GOC_UNDER_TEST, as applicable) TCL variable already used across several
> GCC library testsuites.
>
>  Verified with a cross-compiler configured for the `riscv-linux-gnu'
> target and the `x86_64-linux-gnu' host and using RISC-V/Linux QEMU in the
> user emulation mode as the target board.  Also no change in results with
> `x86_64-linux-gnu' native regression testing.
>
>  See individual change descriptions for details.
>
>  OK to apply to the GCC repo (for libraries maintained externally I'll be
> happy to assist with any merging required, although given that these
> changes are confined to autoconf/automake scriptery they should be
> straightforward to apply, barring any conflicts in generated files)?
>
>   Maciej
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-dev+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-dev/alpine.LFD.2.21.1911081714120.13542%40redsun52.ssa.fujisawa.hgst.com
> .
>

Re: [golang-dev] [PATCH 0/4] Fix library testsuite compilation for build sysroot

[Kaz Kylheku (libffi)]

On 2019-11-11 10:15, Ulderico Cirello wrote:
> Hi Maciej,
> 
> Go's project doesn't take mail patches for changes.

Is it that they'd have to read man pages and learn how to use common 
utilities?

Or that nobody has written a "patch in Go" yet?

Re: [golang-dev] [PATCH 0/4] Fix library testsuite compilation for build sysroot

[Ian Lance Taylor]

On Mon, Nov 11, 2019 at 10:31 AM Kaz Kylheku (libffi)
<382-725-6798@kylheku.com> wrote:
>
> On 2019-11-11 10:15, Ulderico Cirello wrote:
> > Hi Maciej,
> >
> > Go's project doesn't take mail patches for changes.
>
> Is it that they'd have to read man pages and learn how to use common
> utilities?
>
> Or that nobody has written a "patch in Go" yet?

Please be polite; thanks.  There are many advantages to using Gerrit
for code review.  It has nothing to do with reading man pages or using
the patch program.

Ian

Re: [golang-dev] [PATCH 0/4] Fix library testsuite compilation for build sysroot

[Kaz Kylheku (libffi)]

On 2019-11-11 10:34, Ian Lance Taylor wrote:
> On Mon, Nov 11, 2019 at 10:31 AM Kaz Kylheku (libffi)
> <382-725-6798@kylheku.com> wrote:
>> 
>> On 2019-11-11 10:15, Ulderico Cirello wrote:
>> > Hi Maciej,
>> >
>> > Go's project doesn't take mail patches for changes.
>> 
>> Is it that they'd have to read man pages and learn how to use common
>> utilities?
>> 
>> Or that nobody has written a "patch in Go" yet?
> 
> Please be polite; thanks.

Ah, that would be, "thank you for your bug report and patch".

Re: [golang-dev] [PATCH 0/4] Fix library testsuite compilation for build sysroot

[Ian Lance Taylor]

On Mon, Nov 11, 2019 at 10:15 AM Ulderico Cirello
<uldericofilho@gmail.com> wrote:
>
> Go's project doesn't take mail patches for changes. Please use gerrit ( https://go-review.googlesource.com/ ).

These patches are for gccgo, not the gc toolchain.  They should
probably have been sent to gofrontend-dev rather than golang-dev.  The
gccgo repo does take patches via e-mail; I route them through Gerrit
as needed.

Ian

Re: [golang-dev] [PATCH 0/4] Fix library testsuite compilation for build sysroot

[Maciej W. Rozycki]

On Mon, 11 Nov 2019, Ian Lance Taylor wrote:

> > Go's project doesn't take mail patches for changes. Please use gerrit ( https://go-review.googlesource.com/ ).
> 
> These patches are for gccgo, not the gc toolchain.  They should
> probably have been sent to gofrontend-dev rather than golang-dev.  The
> gccgo repo does take patches via e-mail; I route them through Gerrit
> as needed.

 I may have misinterpreted this paragraph[1]:

"Submitting Changes

   Changes to the Go frontend should follow the same process as for the
   main Go repository, only for the gofrontend project and the
   gofrontend-dev@googlegroups.com mailing list rather than the go project
   and the golang-dev@googlegroups.com mailing list. Those changes will
   then be merged into the GCC sources."

Sorry about that; I think it might benefit from a rewrite for clarity 
though.

References:

[1] "Contributing to the gccgo frontend - The Go Programming Language", 
    <https://golang.org/doc/gccgo_contribute.html>

  Maciej

Re: [golang-dev] [PATCH 0/4] Fix library testsuite compilation for build sysroot

[Ian Lance Taylor]

On Mon, Nov 11, 2019 at 10:44 AM Maciej W. Rozycki <macro@wdc.com> wrote:
>
> On Mon, 11 Nov 2019, Ian Lance Taylor wrote:
>
> > > Go's project doesn't take mail patches for changes. Please use gerrit ( https://go-review.googlesource.com/ ).
> >
> > These patches are for gccgo, not the gc toolchain.  They should
> > probably have been sent to gofrontend-dev rather than golang-dev.  The
> > gccgo repo does take patches via e-mail; I route them through Gerrit
> > as needed.
>
>  I may have misinterpreted this paragraph[1]:
>
> "Submitting Changes
>
>    Changes to the Go frontend should follow the same process as for the
>    main Go repository, only for the gofrontend project and the
>    gofrontend-dev@googlegroups.com mailing list rather than the go project
>    and the golang-dev@googlegroups.com mailing list. Those changes will
>    then be merged into the GCC sources."
>
> Sorry about that; I think it might benefit from a rewrite for clarity
> though.
>
> References:
>
> [1] "Contributing to the gccgo frontend - The Go Programming Language",
>     <https://golang.org/doc/gccgo_contribute.html>


The paragraph seems reasonable clear to me, so I'm obviously missing
something.  Can you suggest a clearer rewrite?  Thanks.

Ian

Re: [golang-dev] [PATCH 0/4] Fix library testsuite compilation for build sysroot

[Maciej W. Rozycki]

On Mon, 11 Nov 2019, Ulderico Cirello wrote:

> Go's project doesn't take mail patches for changes. Please use gerrit (
> https://go-review.googlesource.com/ ).

 Thanks for your reply; this is however too much effort for my limited 
resources and a one-off change.

 The reason is I'm not actively working on Go and I have only enabled Go 
frontend compilation/verification for my RISC-V effort in case there is a 
regression caused by a machine backend change that happens to only trigger 
for the Go frontend so that it does not go unnoticed.

 I have provided this change in a hope it is useful to the community and 
in these circumstances hopefully someone actually interested in Go will 
pick up and merge this change; otherwise I will drop my local change and 
consequently Go verification once GCC 10 has been released.

 Thank you for your understanding.

  Maciej