Re: [PATCH v2 0/4] libffi: Sync with upstream

[David Edelsohn <dje.gcc@gmail.com>]

Hi, H.J.

My colleague responded that GCC Go builds and works on AIX, but it
currently requires a special, custom version of GNU objcopy that adds
support for the types of features that Go requires to operate on AIX
XCOFF files.  Those changes have not yet been updated and contributed
to GNU Binutils.

I will see if I can install that version of objcopy standalone.  We
also can ask Clement and ATOS to test GCC Go build with your proposed
libffi patch, or is it vanilla libffi trunk?

Thanks, David


On Sat, Oct 16, 2021 at 3:59 PM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> On Sat, Oct 16, 2021 at 12:53 PM David Edelsohn <dje.gcc@gmail.com> wrote:
> >
> > On Sat, Oct 16, 2021 at 1:13 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> > >
> > > On Sat, Oct 16, 2021 at 10:04 AM David Edelsohn <dje.gcc@gmail.com> wrote:
> > > >
> > > > On Sat, Oct 16, 2021 at 7:48 AM H.J. Lu <hjl.tools@gmail.com> wrote:
> > > > >
> > > > > On Fri, Oct 15, 2021 at 5:22 PM David Edelsohn <dje.gcc@gmail.com> wrote:
> > > > > >
> > > > > > On Fri, Oct 15, 2021 at 8:06 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> > > > > > >
> > > > > > > On Wed, Oct 13, 2021 at 6:42 AM H.J. Lu <hjl.tools@gmail.com> wrote:
> > > > > > > >
> > > > > > > > On Wed, Oct 13, 2021 at 6:03 AM Richard Biener
> > > > > > > > <richard.guenther@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > On Wed, Oct 13, 2021 at 2:56 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > On Wed, Oct 13, 2021 at 5:45 AM Richard Biener
> > > > > > > > > > <richard.guenther@gmail.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Sep 2, 2021 at 5:50 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Change in the v2 patch:
> > > > > > > > > > > >
> > > > > > > > > > > > 1. Disable static trampolines by default.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > GCC maintained a copy of libffi snapshot from 2009 and cherry-picked fixes
> > > > > > > > > > > > from upstream over the last 10+ years.  In the meantime, libffi upstream
> > > > > > > > > > > > has been changed significantly with new features, bug fixes and new target
> > > > > > > > > > > > support.  Here is a set of patches to sync with libffi 3.4.2 release and
> > > > > > > > > > > > make it easier to sync with libffi upstream:
> > > > > > > > > > > >
> > > > > > > > > > > > 1. Document how to sync with upstream.
> > > > > > > > > > > > 2. Add scripts to help sync with upstream.
> > > > > > > > > > > > 3. Sync with libffi 3.4.2. This patch is quite big.  It is availale at
> > > > > > > > > > > >
> > > > > > > > > > > > https://gitlab.com/x86-gcc/gcc/-/commit/15e80c879c571f79a0e57702848a9df5fba5be2f
> > > > > > > > > > > > 4. Integrate libffi build and testsuite with GCC.
> > > > > > > > > > >
> > > > > > > > > > > How did you test this?  It looks like libgo is the only consumer of
> > > > > > > > > > > libffi these days.
> > > > > > > > > > > In particular go/libgo seems to be supported on almost all targets besides
> > > > > > > > > > > darwin/windows - did you test cross and canadian configurations?
> > > > > > > > > >
> > > > > > > > > > I only tested it on Linux/i686 and Linux/x86-64.   My understanding is that
> > > > > > > > > > the upstream libffi works on Darwin and Windows.
> > > > > > > > > >
> > > > > > > > > > > I applaud the attempt to sync to upsteam but I fear you won't get any "review"
> > > > > > > > > > > of this massive diff.
> > > > > > > > > >
> > > > > > > > > > I believe that it should just work.  Our libffi is very much out of date.
> > > > > > > > >
> > > > > > > > > Yes, you can hope.  And yes, our libffi is out of date.
> > > > > > > > >
> > > > > > > > > Can you please do the extra step to test one weird architecture, namely
> > > > > > > > > powerpc64-aix which is available on the compile-farm?
> > > > > > > >
> > > > > > > > I will give it a try and report back.
> > > > > > > >
> > > > > > > > > If that goes well I think it's good to "hope" at this point (and plenty of
> > > > > > > > > time to fix fallout until the GCC 12 release).
> > > > > > > > >
> > > > > > > > > Thus OK after the extra testing dance and waiting until early next
> > > > > > > > > week so others can throw in a veto.
> > > > > > >
> > > > > > > I tried to bootstrap GCC master branch on  gcc119.fsffrance.org:
> > > > > > >
> > > > > > > *  MT/MODEL: 8284-22A                                                         *
> > > > > > > * Partition: gcc119                                                           *
> > > > > > > *    System: power8-aix.osuosl.org                                            *
> > > > > > > *       O/S: AIX V7.2 7200-04-03-2038
> > > > > > >
> > > > > > > I configured GCC with
> > > > > > >
> > > > > > > --with-as=/usr/bin/as --with-ld=/usr/bin/ld
> > > > > > > --enable-version-specific-runtime-libs --disable-nls
> > > > > > > --enable-decimal-float=dpd --disable-libstdcxx-pch --disable-werror
> > > > > > > --enable-__cxa_atexit --with-gmp=/opt/cfarm --with-mpfr=/opt/cfarm
> > > > > > > --with-mpc=/opt/cfarm --with-isl=/opt/cfarm --prefix=/opt/freeware
> > > > > > > --with-local-prefix=/opt/freeware --enable-languages=c,c++,go
> > > > > > >
> > > > > > > I got
> > > > > > >
> > > > > > > g++   -g -DIN_GCC     -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W
> > > > > > > -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wno-format -Wmissing-format-at
> > > > > > > tribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-
> > > > > > > overlength-strings -fno-common  -DHAVE_CONFIG_H  -DGENERATOR_FILE -static-libstd
> > > > > > > c++ -static-libgcc -Wl,-bbigtoc -Wl,-bmaxdata:0x40000000 -o build/genenums \
> > > > > > >     build/genenums.o build/read-md.o build/errors.o ../build-powerpc-ibm-aix7.2.
> > > > > > > 4.0/libiberty/libiberty.a
> > > > > > > ld: 0711-317 ERROR: Undefined symbol: lexer_line
> > > > > > > ld: 0711-317 ERROR: Undefined symbol: .yylex(char const**)
> > > > > > > ld: 0711-317 ERROR: Undefined symbol: .yybegin(char const*)
> > > > > > > ld: 0711-317 ERROR: Undefined symbol: lexer_toplevel_done
> > > > > > > ld: 0711-317 ERROR: Undefined symbol: .yyend()
> > > > > > > ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information.
> > > > > > > collect2: error: ld returned 8 exit status
> > > > > > > Makefile:3000: recipe for target 'build/gengtype' failed
> > > > > > > gmake[5]: *** [build/gengtype] Error 1
> > > > > > >
> > > > > > > David, is there an instruction to bootstrap GCC on AIX?
> > > > > >
> > > > > > The CompileFarm page in the GCC wiki has instructions under "build tips":
> > > > > >
> > > > > > https://gcc.gnu.org/wiki/CompileFarm#Services_and_software_installed_on_farm_machines
> > > > > >
> > > > > > The error that you show might be due to not having /opt/freeware/bin
> > > > > > first in your path and the bootstrap used the AIX version of lex or
> > > > > > sed or some other command.
> > > > > >
> > > > >
> > > > > Hi David,
> > > > >
> > > > > I made some progress.  I am trying to verify my libffi sync branch:
> > > > >
> > > > > https://gitlab.com/x86-gcc/gcc/-/tree/users/hjl/libffi/master
> > > > >
> > > > > on powerpc64-aix.  Go is the only user of libffi.  But compiler farm
> > > > > suggestion is
> > > > >
> > > > > .../src/configure --disable-werror --enable-languages=c,c++
> > > > > --with-gmp=/opt/cfarm --with-libiconv-prefix=/opt/cfarm
> > > > > --disable-libstdcxx-pch --with-included-gettext
> > > > >
> > > > > When I added Go with --enable-languages=c,c++,go, I got
> > > > >
> > > > > /opt/freeware/bin/bash: missing-objcopy: command not found
> > > > > /opt/freeware/bin/bash: missing-objcopy: command not found
> > > > > /opt/freeware/bin/bash /home/hjl/work/git/gitlab/x86-gcc/libgo/mvifdiff.sh build
> > > > > cfg.go.tmp buildcfg.go
> > > > > mv: 0653-401 Cannot rename internal/unsafeheader.s-gox.tmp to internal/unsafehea
> > > > > der.gox:
> > > > >              A file or directory in the path name does not exist.
> > > > > mv: 0653-401 Cannot rename runtime/internal/atomic.s-gox.tmp to runtime/internal
> > > > > /atomic.gox:
> > > > >              A file or directory in the path name does not exist.
> > > > > mv: 0653-401 Cannot rename internal/race.s-gox.tmp to internal/race.gox:
> > > > >              A file or directory in the path name does not exist.
> > > > > Makefile:3019: recipe for target 'runtime/internal/atomic.s-gox' failed
> > > > > gmake[10]: *** [runtime/internal/atomic.s-gox] Error 1
> > > > >
> > > > > Is Go supported on AIX? If yes, what did I do wrong?
> > > >
> > > > I believe that GCC Go works on AIX, but I have not been testing it
> > > > regularly.  There was more focus on GCC Go for AIX prior to the Golang
> > > > port to AIX.
> > > >
> > > > It looks like the problem is that the objcopy command is not
> > > > installed.  There have been various issues with Binutils on AIX.  I'm
> > > > not certain if GCC Go had relied on objcopy or not.
> > > >
> > >
> > > Hi Richard,
> > >
> > > Go on master branch isn't buildable on AIX.  What should I do
> > > next to get my libffi sync patches into master branch?
> >
> > H.J.,
> >
> > You have mischaracterized and misrepresented the issue.  No one stated
> > that GCC Go is not buildable, only that GCC Go requires objcopy, which
> > is not currently installed on gcc119.
> >
> > Thanks, David
>
> I installed GNU binutils and got
>
>
> /home/hjl/work/git/gitlab/x86-gcc/libgo/go/hash/maphash/maphash.go:17:30:
> error: ./internal/unsafeheader.gox exists but does not contain any Go
> export data
>    17 |         "internal/unsafeheader"
>       |                              ^
> /home/hjl/work/git/gitlab/x86-gcc/libgo/go/hash/maphash/maphash.go:17:30:
> error: internal/unsafeheader.gox exists but does not contain any Go
> export data
> /home/hjl/work/git/gitlab/x86-gcc/libgo/go/hash/maphash/maphash.go:17:30:
> error: import file 'internal/unsafeheader' not found
> /home/hjl/work/git/gitlab/x86-gcc/libgo/go/hash/maphash/maphash.go:146:42:
> error: reference to undefined name 'unsafeheader'
>   146 |                         ptr :=
> (*byte)((*unsafeheader.String)(unsafe.Pointer(&s)).Data)
>       |                                          ^
> /home/hjl/work/git/gitlab/x86-gcc/libgo/go/hash/maphash/maphash.go:146:41:
> error: expected pointer
>   146 |                         ptr :=
> (*byte)((*unsafeheader.String)(unsafe.Pointer(&s)).Data)
>       |                                         ^
> gmake[8]: Entering directory
> '/home/hjl/work/build/gnu/tools-build/gcc-gitlab/build-powerpc64-aix7.2/powerpc-ibm-aix7.2.4.0/pthread/libgo'
> f="internal/itoa.o"; if test ! -f $f; then f="internal/.libs/itoa.o";
> fi; objcopy -j .go_export $f internal/itoa.s-gox.tmp;
> /opt/freeware/bin/bash
> /home/hjl/work/git/gitlab/x86-gcc/libgo/mvifdiff.sh
> internal/itoa.s-gox.tmp `echo internal/itoa.s-gox | sed -e
> 's/s-gox/gox/'`
> echo timestamp > internal/itoa.s-gox
> Makefile:3019: recipe for target 'hash/maphash.lo' failed
> gmake[6]: *** [hash/maphash.lo] Error 1
>
> and there were more errors like this.
>
> --
> H.J.