[PATCH v4 GCC 2/5] libffi/test: Fix compilation for build sysroot

[PATCH v4 GCC 2/5] libffi/test: Fix compilation for build sysroot

[Maciej W. Rozycki]

Fix a problem with the libffi testsuite using a method to determine the 
compiler to use resulting in the tool being different from one the 
library has been built with, and causing a catastrophic failure from the 
inability to actually choose any compiler at all in a cross-compilation 
configuration.

Address this problem by providing a DejaGNU configuration file defining 
the compiler to use, via the CC_FOR_TARGET TCL variable, set from $CC by 
autoconf, which will have all the required options set for the target 
compiler to build executables in the environment configured, removing 
failures like:

FAIL: libffi.call/closure_fn0.c -W -Wall -Wno-psabi -O0 (test for excess errors)
Excess errors:
default_target_compile: No compiler to compile with
UNRESOLVED: libffi.call/closure_fn0.c -W -Wall -Wno-psabi -O0 compilation failed to produce executable

and bringing overall test results for the `riscv64-linux-gnu' target 
(here with the `x86_64-linux-gnu' host and RISC-V QEMU in the Linux user 
emulation mode as the target board) from:

		=== libffi Summary ===

# of unexpected failures	708
# of unresolved testcases	708
# of unsupported tests		30

to:

		=== libffi Summary ===

# of expected passes		1934
# of unsupported tests		28

	libffi/
	* configure.ac: Add testsuite/local.exp to output files.
	* configure: Regenerate.
	* testsuite/local.exp.in: New file.
	* testsuite/Makefile.am (EXTRA_DEJAGNU_SITE_CONFIG): New 
	variable.
	* testsuite/Makefile.in: Regenerate.
---
This is a backport of upstream libffi change as recorded here: 
<https://sourceware.org/pipermail/libffi-discuss/2020/002503.html>.

New change in v4.
---
 libffi/configure              |    3 ++-
 libffi/configure.ac           |    2 +-
 libffi/testsuite/Makefile.am  |    2 ++
 libffi/testsuite/Makefile.in  |    5 ++++-
 libffi/testsuite/local.exp.in |    2 ++
 5 files changed, 11 insertions(+), 3 deletions(-)

gcc-test-libffi-cc-for-target-template.diff
Index: gcc/libffi/configure
===================================================================
--- gcc.orig/libffi/configure
+++ gcc/libffi/configure
@@ -16662,7 +16662,7 @@ ac_config_commands="$ac_config_commands
 ac_config_links="$ac_config_links include/ffitarget.h:src/$TARGETDIR/ffitarget.h"
 
 
-ac_config_files="$ac_config_files include/Makefile include/ffi.h Makefile testsuite/Makefile man/Makefile libffi.pc"
+ac_config_files="$ac_config_files include/Makefile include/ffi.h Makefile testsuite/Makefile testsuite/local.exp man/Makefile libffi.pc"
 
 
 cat >confcache <<\_ACEOF
@@ -17829,6 +17829,7 @@ do
     "include/ffi.h") CONFIG_FILES="$CONFIG_FILES include/ffi.h" ;;
     "Makefile") CONFIG_FILES="$CONFIG_FILES Makefile" ;;
     "testsuite/Makefile") CONFIG_FILES="$CONFIG_FILES testsuite/Makefile" ;;
+    "testsuite/local.exp") CONFIG_FILES="$CONFIG_FILES testsuite/local.exp" ;;
     "man/Makefile") CONFIG_FILES="$CONFIG_FILES man/Makefile" ;;
     "libffi.pc") CONFIG_FILES="$CONFIG_FILES libffi.pc" ;;
 
Index: gcc/libffi/configure.ac
===================================================================
--- gcc.orig/libffi/configure.ac
+++ gcc/libffi/configure.ac
@@ -377,6 +377,6 @@ test -d src/$TARGETDIR || mkdir src/$TAR
 
 AC_CONFIG_LINKS(include/ffitarget.h:src/$TARGETDIR/ffitarget.h)
 
-AC_CONFIG_FILES(include/Makefile include/ffi.h Makefile testsuite/Makefile man/Makefile libffi.pc)
+AC_CONFIG_FILES(include/Makefile include/ffi.h Makefile testsuite/Makefile testsuite/local.exp man/Makefile libffi.pc)
 
 AC_OUTPUT
Index: gcc/libffi/testsuite/Makefile.am
===================================================================
--- gcc.orig/libffi/testsuite/Makefile.am
+++ gcc/libffi/testsuite/Makefile.am
@@ -13,6 +13,8 @@ RUNTEST = `if [ -f $(top_srcdir)/../deja
 
 AM_RUNTESTFLAGS =
 
+EXTRA_DEJAGNU_SITE_CONFIG = local.exp
+
 CLEANFILES = *.exe core* *.log *.sum
 
 EXTRA_DIST = config/default.exp libffi.call/cls_19byte.c		\
Index: gcc/libffi/testsuite/Makefile.in
===================================================================
--- gcc.orig/libffi/testsuite/Makefile.in
+++ gcc/libffi/testsuite/Makefile.in
@@ -106,7 +106,7 @@ am__configure_deps = $(am__aclocal_m4_de
 DIST_COMMON = $(srcdir)/Makefile.am
 mkinstalldirs = $(SHELL) $(top_srcdir)/../mkinstalldirs
 CONFIG_HEADER = $(top_builddir)/fficonfig.h
-CONFIG_CLEAN_FILES =
+CONFIG_CLEAN_FILES = local.exp
 CONFIG_CLEAN_VPATH_FILES =
 AM_V_P = $(am__v_P_@AM_V@)
 am__v_P_ = $(am__v_P_@AM_DEFAULT_V@)
@@ -279,6 +279,7 @@ RUNTEST = `if [ -f $(top_srcdir)/../deja
 	       echo $(top_srcdir)/../dejagnu/runtest ; \
 	    else echo runtest; fi`
 
+EXTRA_DEJAGNU_SITE_CONFIG = local.exp
 CLEANFILES = *.exe core* *.log *.sum
 EXTRA_DIST = config/default.exp libffi.call/cls_19byte.c		\
 libffi.call/cls_align_longdouble_split.c				\
@@ -390,6 +391,8 @@ $(top_srcdir)/configure: @MAINTAINER_MOD
 $(ACLOCAL_M4): @MAINTAINER_MODE_TRUE@ $(am__aclocal_m4_deps)
 	cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh
 $(am__aclocal_m4_deps):
+local.exp: $(top_builddir)/config.status $(srcdir)/local.exp.in
+	cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@
 
 mostlyclean-libtool:
 	-rm -f *.lo
Index: gcc/libffi/testsuite/local.exp.in
===================================================================
--- /dev/null
+++ gcc/libffi/testsuite/local.exp.in
@@ -0,0 +1,2 @@
+set CC_FOR_TARGET "@CC@"
+set CXX_FOR_TARGET "@CXX@"

[PATCH v4 GCC 3/5] libffi/test: Make `libffi-init' use $CC_FOR_TARGET

[Maciej W. Rozycki]

Update code in `libffi-init' to use $CC_FOR_TARGET in determining the 
value of $ld_library_path, as using a different compiler location from 
one actually used in testing may have odd consequences.

As this obviously loses the setting of $gccdir provide a replacement way 
to determine the directory if feasible, however prefer the location of 
shared libgcc over static libgcc.  This helps in configurations where 
shared libgcc is, unlike libgcc, a location that is not specific to the 
compiler version, a common scenario.  It does not address the scenario 
however where there is a shared libgcc symlink installed pointing to the 
actual run-time library installed elsewhere; this would have to be dealt 
with in a board description file specific to the installation.

Use `remote_exec host' rather than `exec' to invoke the compiler so as 
to support remote configurations and also avoid the latter procedure's 
path length limitation that prevents execution in some actual scenarios.

As using `remote_exec host' precludes the use of our existing file name 
globbing to examine directory contents reuse, with minor modifications 
needed to adjust to our structure, the piece I previously contributed to 
GCC with commit d42b84f427e4 ("testsuite: Fix run-time tracking down 
of `libgcc_s'").

	libffi/
	* testsuite/lib/libffi.exp (libffi-init): Use CC_FOR_TARGET.
	Update the determination of `ld_library_path' accordingly.
---
This is a backport of combined upstream libffi changes as recorded here: 
<https://sourceware.org/pipermail/libffi-discuss/2020/002505.html> and 
here: <https://sourceware.org/pipermail/libffi-discuss/2020/002506.html> 
(there's no point to introduce indentation breakage only to fix it with 
the next change).

New change in v4.
---
 libffi/testsuite/lib/libffi.exp |   72 ++++++++++++++++++++++++++++++----------
 1 file changed, 54 insertions(+), 18 deletions(-)

gcc-test-libffi-init-compiler.diff
Index: gcc/libffi/testsuite/lib/libffi.exp
===================================================================
--- gcc.orig/libffi/testsuite/lib/libffi.exp
+++ gcc/libffi/testsuite/lib/libffi.exp
@@ -99,7 +99,7 @@ proc libffi-init { args } {
     global blddirffi
     global objdir
     global blddircxx
-    global TOOL_OPTIONS
+    global CC_FOR_TARGET
     global tool
     global libffi_include
     global libffi_link_flags
@@ -114,26 +114,62 @@ proc libffi-init { args } {
 
     set compiler_vendor "gnu"
 
-    set gccdir [lookfor_file $tool_root_dir gcc/libgcc.a]
-    if {$gccdir != ""} {
-	set gccdir [file dirname $gccdir]
-    }
-    verbose "gccdir $gccdir"
+    if { [string match $compiler_vendor "gnu"] } {
+	if [info exists CC_FOR_TARGET] then {
+	    set compiler "$CC_FOR_TARGET"
+	    set libgcc_a_x [remote_exec host "$compiler" \
+			    "-print-file-name=libgcc.a"]
+	    if { [lindex $libgcc_a_x 0] == 0 } {
+		set gccdir [file dirname [lindex $libgcc_a_x 1]]
+	    } else {
+		set gccdir ""
+	    }
+	} else {
+	    set gccdir [lookfor_file $tool_root_dir gcc/libgcc.a]
+	    if {$gccdir != ""} {
+		set gccdir [file dirname $gccdir]
+	    }
+	    set compiler "${gccdir}/xgcc"
+	}
+	verbose "gccdir $gccdir"
 
-    set ld_library_path "."
-    append ld_library_path ":${gccdir}"
+	set shlib_ext [get_shlib_extension]
+	set libgcc_s_x [remote_exec host "$compiler" \
+			"-print-file-name=libgcc_s.${shlib_ext}"]
+	if { [lindex $libgcc_s_x 0] == 0 } {
+	    set libgcc_dir [file dirname [lindex $libgcc_s_x 1]]
+	} else {
+	    set libgcc_dir $gccdir
+	}
+	verbose "libgcc_dir $libgcc_dir"
 
-    set compiler "${gccdir}/xgcc"
-    if { [is_remote host] == 0 && [which $compiler] != 0 } {
-	foreach i "[exec $compiler --print-multi-lib]" {
-	    set mldir ""
-	    regexp -- "\[a-z0-9=_/\.-\]*;" $i mldir
-	    set mldir [string trimright $mldir "\;@"]
-	    if { "$mldir" == "." } {
-		continue
+	set ld_library_path "."
+	append ld_library_path ":${gccdir}"
+
+	set multi_dir_x [remote_exec host "$compiler" "-print-multi-directory"]
+	set multi_lib_x [remote_exec host "$compiler" "-print-multi-lib"]
+	if { [lindex $multi_dir_x 0] == 0 && [lindex $multi_lib_x 0] == 0 } {
+	    set multi_dir [string trim [lindex $multi_dir_x 1]]
+	    set multi_lib [string trim [lindex $multi_lib_x 1]]
+	    if { "$multi_dir" == "." } {
+		set multi_root "$libgcc_dir"
+	    } else {
+		set multi_match [string last "/$multi_dir" "$libgcc_dir"]
+		if { "$multi_match" >= 0 } {
+		    set multi_root [string range "$libgcc_dir" \
+				    0 [expr $multi_match - 1]]
+		} else {
+		    set multi_lib ""
+		}
 	    }
-	    if { [llength [glob -nocomplain ${gccdir}/${mldir}/libgcc_s*.so.*]] >= 1 } {
-		append ld_library_path ":${gccdir}/${mldir}"
+	    foreach i "$multi_lib" {
+		set mldir ""
+		regexp -- "\[a-z0-9=_/\.-\]*;" $i mldir
+		set mldir [string trimright $mldir "\;@"]
+		if { "$mldir" == "$multi_dir" } {
+		    continue
+		}
+		append ld_library_path ":${multi_root}/${mldir}"
 	    }
 	}
     }

Re: [PATCH v4 GCC 2/5] libffi/test: Fix compilation for build sysroot

[Jeff Law]

On Sat, 2020-04-04 at 00:01 +0100, Maciej W. Rozycki wrote:
> Fix a problem with the libffi testsuite using a method to determine the 
> compiler to use resulting in the tool being different from one the 
> library has been built with, and causing a catastrophic failure from the 
> inability to actually choose any compiler at all in a cross-compilation 
> configuration.
> 
> Address this problem by providing a DejaGNU configuration file defining 
> the compiler to use, via the CC_FOR_TARGET TCL variable, set from $CC by 
> autoconf, which will have all the required options set for the target 
> compiler to build executables in the environment configured, removing 
> failures like:
> 
> FAIL: libffi.call/closure_fn0.c -W -Wall -Wno-psabi -O0 (test for excess
> errors)
> Excess errors:
> default_target_compile: No compiler to compile with
> UNRESOLVED: libffi.call/closure_fn0.c -W -Wall -Wno-psabi -O0 compilation
> failed to produce executable
> 
> and bringing overall test results for the `riscv64-linux-gnu' target 
> (here with the `x86_64-linux-gnu' host and RISC-V QEMU in the Linux user 
> emulation mode as the target board) from:
> 
> 		=== libffi Summary ===
> 
> # of unexpected failures	708
> # of unresolved testcases	708
> # of unsupported tests		30
> 
> to:
> 
> 		=== libffi Summary ===
> 
> # of expected passes		1934
> # of unsupported tests		28
> 
> 	libffi/
> 	* configure.ac: Add testsuite/local.exp to output files.
> 	* configure: Regenerate.
> 	* testsuite/local.exp.in: New file.
> 	* testsuite/Makefile.am (EXTRA_DEJAGNU_SITE_CONFIG): New 
> 	variable.
> 	* testsuite/Makefile.in: Regenerate.
Oh, I see it now.  THe patches I ack'd were actually for upstream libffi.

You should actually wait for a libffi maintainer to ack those, not me :-)  Sorry
for the confusion.

Both backports are OK once they're upstreamed.

jeff
> 

Re: [PATCH v4 GCC 2/5] libffi/test: Fix compilation for build sysroot

[Maciej W. Rozycki]

On Mon, 6 Apr 2020, Jeff Law wrote:

> > 	libffi/
> > 	* configure.ac: Add testsuite/local.exp to output files.
> > 	* configure: Regenerate.
> > 	* testsuite/local.exp.in: New file.
> > 	* testsuite/Makefile.am (EXTRA_DEJAGNU_SITE_CONFIG): New 
> > 	variable.
> > 	* testsuite/Makefile.in: Regenerate.
> Oh, I see it now.  THe patches I ack'd were actually for upstream libffi.
> 
> You should actually wait for a libffi maintainer to ack those, not me :-)  Sorry
> for the confusion.

 Sorry to make it unclear.  I chose to cc the other mailing list with the 
libffi part of both submissions so as to give a chance to chime in to 
members of both communities.  We are quite tightly coupled with each other 
here after all and my experience over the many years I have been involved 
has been that the bits related to Autotools are often quite tricky and 
hard to get right (Autotools are fine tools in my experience, it's just 
you need to be thorough with them as they tend not to forgive a cursory 
approach).

> Both backports are OK once they're upstreamed.

 Thanks for your ack; it was my intent to get these upstream first, and I 
realise there can be changes requested that will require the GCC backport 
to be adjusted accordingly (and reviewed again).

  Maciej