Re: [PATCH v3 2/4] libffi/test: Fix compilation for build sysroot

Re: [PATCH v3 2/4] libffi/test: Fix compilation for build sysroot

[Maciej W. Rozycki]

On Tue, 24 Mar 2020, Mike Stump wrote:

> > Have we made any conclusions WRT the way to move forward with this stuff?  
> > Things remain broken and I'd prefer to get the issues off the plate while 
> > the stuff is hot, or at least mildly warm.  I'm about to get distracted 
> > with other work.
> 
> It's unfortunate that upstream has anything that prevents it from us 
> just importing it all and calling it done.
> 
> Anyway, if you see a path forward for grabbing all the Makefile/config 
> stuff and leaving the abi changing stuff out, and just important that, 
> we can do a partial import.  I say this without reviewing the diffs from 
> upstream and how many there are and what's in them.  I'm hoping things 
> are nicely segregated and reasonably small otherwise.

 Thank you for your input.

 I have actually considered extracting the bits already, but I hesitated 
putting that forward that as having looked at the part that we require I 
have thought it to be very messy: the .exp file is handcrafted with an 
inline piece of scriptery buried in `configure.ac' and never cleaned, not 
even with `make distclean', nor equipped with any Makefile dependencies.  
Clearly it must have been written by someone who hasn't been accustomed to 
working with GNU autotools.

 The ultimate change is very small, but it has only been created gradually 
with four commits in the upstream libffi repository, none of which applies
cleanly to ours and most of which include unrelated stuff.  They are:

- commit 8308984e479e ("[PATCH] Make sure we're running dejagnu tests with 
  the right compiler."),

- commit 2d9b3939751b ("[PATCH] Fix for closures with sunpro compiler"),

- commit 0c3824702d3d ("[PATCH] Always set CC_FOR_TARGET for dejagnu, to 
  make the testsuite respect $CC"),

- commit 7d698125b1f0 ("[PATCH] Use the proper C++ compiler to run C++ 
  tests") -- not yet needed in our libffi version as no tests are marked
  C++.

-- at <git://github.com/libffi/libffi.git>.  I have now extracted the 
relevant bits from the four commits and the result is below.

 I have pushed it through my testing and it fixes the test results just 
like my earlier proposal; in fact libffi.log files are the same modulo 
timestamps and one number that is randomly generated.  It is worth noting 
however that the multilib discovery logic in `libffi-init' has not been 
updated unlike with my proposal and it continues using the compiler 
hardcoded within rather than one set with CC_FOR_TARGET/CXX_FOR_TARGET.

 That uses a mechanism (*_FOR_TARGET, interpreted within `target_compile') 
different from one we do (*_UNDER_TEST, used to set `compiler=' in the 
invocation of `target_compile'), and ignores TOOL_EXECUTABLE altogether.  
It makes sense however semantically to me for a standalone library test 
suite to consider the compiler just a tool in testing and not the object 
under test.  Plus it makes it easy to define compilers for the various 
languages required.

 So I am in favour of retaining the mechanism rather than using my earlier 
proposal, however I'm in two minds as to how to proceed.  Integrating the 
change as it is will make us having clutter left in the tree after `make 
distclean', but we can do it right away.  Fixing the problems with the 
change upstream in libffi first and then merging the result back into our 
tree will avoid getting the clutter, but will likely take time.

 I'll sleep on it yet, and meanwhile I'll be happy to hear suggestions.  
I have also cc-ed the libffi mailing list for a possible further insight.

  Maciej

---
 libffi/configure             |    5 +++++
 libffi/configure.ac          |    5 +++++
 libffi/testsuite/Makefile.am |    2 ++
 libffi/testsuite/Makefile.in |    1 +
 4 files changed, 13 insertions(+)

Index: gcc/libffi/configure
===================================================================
--- gcc.orig/libffi/configure
+++ gcc/libffi/configure
@@ -14961,6 +14961,11 @@ _ACEOF
 
 
 
+cat > local.exp <<EOF
+set CC_FOR_TARGET "$CC"
+set CXX_FOR_TARGET "$CXX"
+EOF
+
 
 { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to enable maintainer-specific portions of Makefiles" >&5
 $as_echo_n "checking whether to enable maintainer-specific portions of Makefiles... " >&6; }
Index: gcc/libffi/configure.ac
===================================================================
--- gcc.orig/libffi/configure.ac
+++ gcc/libffi/configure.ac
@@ -61,6 +61,11 @@ AC_PROG_LIBTOOL
 # Test for 64-bit build.
 AC_CHECK_SIZEOF([size_t])
 
+cat > local.exp <<EOF
+set CC_FOR_TARGET "$CC"
+set CXX_FOR_TARGET "$CXX"
+EOF
+
 AM_MAINTAINER_MODE
 
 AC_CHECK_HEADERS(sys/mman.h)
Index: gcc/libffi/testsuite/Makefile.am
===================================================================
--- gcc.orig/libffi/testsuite/Makefile.am
+++ gcc/libffi/testsuite/Makefile.am
@@ -13,6 +13,8 @@ RUNTEST = `if [ -f $(top_srcdir)/../deja
 
 AM_RUNTESTFLAGS =
 
+EXTRA_DEJAGNU_SITE_CONFIG=../local.exp
+
 CLEANFILES = *.exe core* *.log *.sum
 
 EXTRA_DIST = config/default.exp libffi.call/cls_19byte.c		\
Index: gcc/libffi/testsuite/Makefile.in
===================================================================
--- gcc.orig/libffi/testsuite/Makefile.in
+++ gcc/libffi/testsuite/Makefile.in
@@ -279,6 +279,7 @@ RUNTEST = `if [ -f $(top_srcdir)/../deja
 	       echo $(top_srcdir)/../dejagnu/runtest ; \
 	    else echo runtest; fi`
 
+EXTRA_DEJAGNU_SITE_CONFIG = ../local.exp
 CLEANFILES = *.exe core* *.log *.sum
 EXTRA_DIST = config/default.exp libffi.call/cls_19byte.c		\
 libffi.call/cls_align_longdouble_split.c				\

Re: [PATCH v3 2/4] libffi/test: Fix compilation for build sysroot

[Mike Stump]

On Mar 26, 2020, at 3:00 PM, Maciej W. Rozycki <macro@wdc.com> wrote:
> 
> I have actually considered extracting the bits already, but I hesitated 
> putting that forward that as having looked at the part that we require I 
> have thought it to be very messy:

Yeah, sometimes it's like that.  I glanced at the work, if you think it's a step forward, I'd support importing it, my take, import from upstream isn't a bad way to go.

> So I am in favour of retaining the mechanism rather than using my earlier 
> proposal, however I'm in two minds as to how to proceed.  Integrating the 
> change as it is will make us having clutter left in the tree after `make 
> distclean', but we can do it right away.

I'd support this.  distclean is one rm -rf away from being clean enough.  I'd not let that gate or hold up the import.

If there is work that we want that's more to do with in tree building and testing (sys root fun, multilibs), while not ideal, we can deviate from upstream to support that.  Though, if there is a way to naturally support that, that upstream can accept, that'd be better.

Re: [PATCH v3 2/4] libffi/test: Fix compilation for build sysroot

[Maciej W. Rozycki]

On Mon, 30 Mar 2020, Mike Stump wrote:

> > I have actually considered extracting the bits already, but I hesitated 
> > putting that forward that as having looked at the part that we require I 
> > have thought it to be very messy:
> 
> Yeah, sometimes it's like that.  I glanced at the work, if you think 
> it's a step forward, I'd support importing it, my take, import from 
> upstream isn't a bad way to go.

 So I looked into it some more and interestingly enough all the commits I 
have listed in the previous message have already been made as of libffi's 
commit c82cc159426d ("Merge pull request #166 from chevah/master") that we 
imported with our commit b1760f7f915a ("Merge libffi to upstream commit 
c82cc159426d8d4402375fa1ae3f045b9cf82e16").

 That merge was extensively discussed starting from: 
<https://gcc.gnu.org/ml/gcc-patches/2015-01/msg00705.html>, however no 
mention was as to why the local.exp part had been omitted from the merge.  
Perhaps it was considered not necessary for integrating with the GCC tree, 
although I would think that keeping the divergence to the minimum is 
preferable, and it looks to me that our requirements boil down essentially 
to adding multilib support and making some further, minor tweaks to match 
the rest of the tree.  Whereas the diff between the Makefile systems as at 
libffi's commit c82cc159426d and our commit b1760f7f915a looks to me quite 
substantial.

 Perhaps Richard will be able to provide some input here.

> > So I am in favour of retaining the mechanism rather than using my earlier 
> > proposal, however I'm in two minds as to how to proceed.  Integrating the 
> > change as it is will make us having clutter left in the tree after `make 
> > distclean', but we can do it right away.
> 
> I'd support this.  distclean is one rm -rf away from being clean enough.  
> I'd not let that gate or hold up the import.

 While doing further work on finding a solution that would be acceptable 
(to me anyway), I actually found two further upstream libffi commits:

- commit 6b6df1a7bb37 ("[PATCH] Adds `local.exp` to CLEANFILES"),

- commit 6cf0dea78a5a ("[PATCH] Change CLEANFILES to DISTCLEANFILES"),

both beyond our merge point, that fix this shortcoming.  Still there's no 
Makefile dependency, so if configure.ac is patched or local.exp removed, 
then it is not regenerated, and all that would not be required if what 
automake provides was used.

> If there is work that we want that's more to do with in tree building 
> and testing (sys root fun, multilibs), while not ideal, we can deviate 
> from upstream to support that.  Though, if there is a way to naturally 
> support that, that upstream can accept, that'd be better.

 I did some work now to reduce the divergence and will be posting patch 
series shortly to both upstream libffi and our version.  Hopefully that'll 
be acceptable, at least the initial, minimal change from each series.

 If not, for a reference, here's an updated version of the patch I posted 
last time.  It includes the two upstream libffi commits I have mentioned 
above.

 Let's see how it goes.  Thank you for your input.

  Maciej
---
 libffi/Makefile.am           |    3 +++
 libffi/Makefile.in           |    4 ++++
 libffi/configure             |    5 +++++
 libffi/configure.ac          |    5 +++++
 libffi/testsuite/Makefile.am |    2 ++
 libffi/testsuite/Makefile.in |    1 +
 6 files changed, 20 insertions(+)

gcc-test-libffi-cc-for-target.diff
Index: gcc/libffi/Makefile.am
===================================================================
--- gcc.orig/libffi/Makefile.am
+++ gcc/libffi/Makefile.am
@@ -15,6 +15,9 @@ EXTRA_DIST = LICENSE ChangeLog.v1 Change
 	libffi.xcodeproj/project.pbxproj				\
 	libtool-ldflags
 
+# local.exp is generated by configure
+DISTCLEANFILES = local.exp
+
 # Automake Documentation:
 # If your package has Texinfo files in many directories, you can use the
 # variable TEXINFO_TEX to tell Automake where to find the canonical
Index: gcc/libffi/Makefile.in
===================================================================
--- gcc.orig/libffi/Makefile.in
+++ gcc/libffi/Makefile.in
@@ -454,6 +454,9 @@ EXTRA_DIST = LICENSE ChangeLog.v1 Change
 	libtool-ldflags
 
 
+# local.exp is generated by configure
+DISTCLEANFILES = local.exp
+
 # Automake Documentation:
 # If your package has Texinfo files in many directories, you can use the
 # variable TEXINFO_TEX to tell Automake where to find the canonical
@@ -1674,6 +1677,7 @@ installcheck: installcheck-recursive
 	-rm -f src/x86/$(am__dirstamp)
 	-rm -f src/xtensa/$(DEPDIR)/$(am__dirstamp)
 	-rm -f src/xtensa/$(am__dirstamp)
+	-test -z "$(DISTCLEANFILES)" || rm -f $(DISTCLEANFILES)
 
 maintainer-clean-generic:
 	@echo "This command is intended for maintainers to use"
Index: gcc/libffi/configure
===================================================================
--- gcc.orig/libffi/configure
+++ gcc/libffi/configure
@@ -14961,6 +14961,11 @@ _ACEOF
 
 
 
+cat > local.exp <<EOF
+set CC_FOR_TARGET "$CC"
+set CXX_FOR_TARGET "$CXX"
+EOF
+
 
 { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to enable maintainer-specific portions of Makefiles" >&5
 $as_echo_n "checking whether to enable maintainer-specific portions of Makefiles... " >&6; }
Index: gcc/libffi/configure.ac
===================================================================
--- gcc.orig/libffi/configure.ac
+++ gcc/libffi/configure.ac
@@ -61,6 +61,11 @@ AC_PROG_LIBTOOL
 # Test for 64-bit build.
 AC_CHECK_SIZEOF([size_t])
 
+cat > local.exp <<EOF
+set CC_FOR_TARGET "$CC"
+set CXX_FOR_TARGET "$CXX"
+EOF
+
 AM_MAINTAINER_MODE
 
 AC_CHECK_HEADERS(sys/mman.h)
Index: gcc/libffi/testsuite/Makefile.am
===================================================================
--- gcc.orig/libffi/testsuite/Makefile.am
+++ gcc/libffi/testsuite/Makefile.am
@@ -13,6 +13,8 @@ RUNTEST = `if [ -f $(top_srcdir)/../deja
 
 AM_RUNTESTFLAGS =
 
+EXTRA_DEJAGNU_SITE_CONFIG=../local.exp
+
 CLEANFILES = *.exe core* *.log *.sum
 
 EXTRA_DIST = config/default.exp libffi.call/cls_19byte.c		\
Index: gcc/libffi/testsuite/Makefile.in
===================================================================
--- gcc.orig/libffi/testsuite/Makefile.in
+++ gcc/libffi/testsuite/Makefile.in
@@ -279,6 +279,7 @@ RUNTEST = `if [ -f $(top_srcdir)/../deja
 	       echo $(top_srcdir)/../dejagnu/runtest ; \
 	    else echo runtest; fi`
 
+EXTRA_DEJAGNU_SITE_CONFIG = ../local.exp
 CLEANFILES = *.exe core* *.log *.sum
 EXTRA_DIST = config/default.exp libffi.call/cls_19byte.c		\
 libffi.call/cls_align_longdouble_split.c				\