Re: [PATCH v3 0/2] Replace intl/ with out-of-tree GNU gettext

[David Edelsohn <dje.gcc@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 10361 bytes --]

On Thu, Nov 16, 2023 at 5:52 PM Arsen Arsenović <arsen@aarsen.me> wrote:

>
> David Edelsohn <dje.gcc@gmail.com> writes:
>
> > On Thu, Nov 16, 2023 at 5:22 PM Arsen Arsenović <arsen@aarsen.me> wrote:
> >
> >>
> >> David Edelsohn <dje.gcc@gmail.com> writes:
> >>
> >> > Don't build with the dependent libraries in tree.  Don't build the
> >> > dependent libraries as shared libraries. The libraries are already
> built
> >> > and in /opt/cfarm, as mentioned in the Compile Farm wiki.
> >> >
> >> > AIX is not Solaris and not Linux.  It doesn't use ELF.  AIX shared
> >> > libraries *ARE* shared object files in archives.  Shared object
> >> versioning
> >> > is handled by multiple objects in the same archive.
> >>
> >> Hmm, I see.  I removed all the deps but gettext from the tree.
> >>
> >> This leaves gettext-runtime fulfilling the previous role of intl/.
> >>
> >> However, I'm confused about how this worked before, in that case, since,
> >> IIRC, intl also produced libraries and was also put into host exports.
> >>
> >> Leaving gettext in tree produces:
> >>
> >> Could not load program gawk:
> >>         Dependent module
> >> /home/arsen/build/./gettext/intl/.libs/libintl.a(libintl.so.8) could
> not be
> >> loaded.
> >>         Member libintl.so.8 is not found in archive
> >>
> >> I'll try to see why intl/ didn't cause the same issue soon.
> >>
> >> Thanks, have a lovely evening.
> >>
> >
> > The previous version of "intl" was built as a static library.  Configure
> in
> > the older package had the option --enable-host-shared,
> > which I did not use.  Based on the failure message, the in-tree gettext
> > seems to be built as a shared library.  If you explicitly
> > pass --disable-shared to the in-tree configure, you may get farther.  I'm
> > currently using --disable-shared --disable-threads.
> > As we have discussed, the current gettext will retain some references to
> > pthreads despite the configure option.
>
> Sure, but my patch does insert --disable-shared:
>
> --8<---------------cut here---------------start------------->8---
> host_modules= { module= gettext; bootstrap=true; no_install=true;
>                 module_srcdir= "gettext/gettext-runtime";
>                 // We always build gettext with pic, because some packages
> (e.g. gdbserver)
>                 // need it in some configuratons, which is determined via
> nontrivial tests.
>                 // Always enabling pic seems to make sense for something
> tied to
>                 // user-facing output.
>                 extra_configure_flags='--disable-shared --disable-java
> --disable-csharp --with-pic';
>                 lib_path=intl/.libs; };
> --8<---------------cut here---------------end--------------->8---
>
> ... and it is applied:
>
> --8<---------------cut here---------------start------------->8---
> -bash-5.1$ ./config.status --config
> --srcdir=../../gcc/gettext/gettext-runtime --cache-file=./config.cache
>   --disable-werror --with-gmp=/opt/cfarm
>   --with-libiconv-prefix=/opt/cfarm --disable-libstdcxx-pch
>   --with-included-gettext --program-transform-name=s,y,y,
>   --disable-option-checking --build=powerpc-ibm-aix7.3.1.0
>   --host=powerpc-ibm-aix7.3.1.0 --target=powerpc-ibm-aix7.3.1.0
>   --disable-intermodule --enable-checking=yes,types,extra
>   --disable-coverage --enable-languages=c,c++
>   --disable-build-format-warnings --disable-shared --disable-java
>   --disable-csharp --with-pic build_alias=powerpc-ibm-aix7.3.1.0
>   host_alias=powerpc-ibm-aix7.3.1.0 target_alias=powerpc-ibm-aix7.3.1.0
>   CC=gcc CFLAGS=-g 'LDFLAGS=-static-libstdc++ -static-libgcc
>   -Wl,-bbigtoc' 'CXX=g++ -std=c++11' CXXFLAGS=-g
> --8<---------------cut here---------------end--------------->8---
>
> I'm unsure how to tell what the produced binaries are w.r.t static or
> shared, but I only see .o files inside intl/.libs/libintl.a, while I see
> a .so.1 in (e.g.) /lib/libz.a, hinting at it not being shared (?)
>

An AIX shared library created by libtool will look like
libfoo.a[libfoo.so.N], where N is the package major version number.
Normally with one file.

An AIX static library will look like libfoo.a[a.o, b.o, c.o]
with multiple object files.

An AIX archive can contain a combination of shared objects and
normal object files.

AIX normally uses the convention shr.o or shr_64.o for the name
of the shared object file.  Hint, hint, an AIX archive can contain
both 32 bit and 64 bit object files or shared objects.

I don't know why the gettext build system would create
/home/arsen/build/./gettext/intl/.libs/libintl.a(libintl.so.8)
if --disable-shared was requested.  That clearly is using the
naming of a libtool AIX shared object and failing due to
the missing shared object.  Although in this case, the problem
seems to be the shared library load path.  AIX uses LIBPATH,
not LD_LIBRARY_PATH.

Also, for me, the out of tree path was

gettext/gettext-runtime/intl/.libs

Is your search path missing a level?

Thanks, David


>
> I do see that the build system adds intl to the LD_LIBRARY_PATH.
>
> I will be testing dropping lib_path from the module definition above.
> It might be superflous (I think it is only used for LD_LIBRARY_PATH, for
> when the libs built by the build system are shared - which they never
> are for in-tree gettext).  I'll take the shot to add --disable-threads,
> too, for this test.
>
>
> > Thanks, David
> >
> >
> >>
> >> > Thanks, David
> >> >
> >> >
> >> >
> >> > On Thu, Nov 16, 2023 at 4:15 PM Arsen Arsenović <arsen@aarsen.me>
> wrote:
> >> >
> >> >>
> >> >> Arsen Arsenović <arsen@aarsen.me> writes:
> >> >>
> >> >> > [[PGP Signed Part:Good signature from 52C294301EA2C493 Arsen
> Arsenović
> >> >> (Gentoo Developer UID) <arsen@gentoo.org> (trust ultimate) created
> at
> >> >> 2023-11-16T19:47:16+0100 using EDDSA]]
> >> >> >
> >> >> > David Edelsohn <dje.gcc@gmail.com> writes:
> >> >> >
> >> >> >> On Wed, Nov 15, 2023 at 9:22 AM Arsen Arsenović <arsen@aarsen.me>
> >> >> wrote:
> >> >> >>
> >> >> >>>
> >> >> >>> David Edelsohn <dje.gcc@gmail.com> writes:
> >> >> >>>
> >> >> >>> > GCC had been working on AIX with NLS, using
> >> >> "--with-included-gettext".
> >> >> >>> > --disable-nls gets past the breakage, but GCC does not build
> for
> >> me
> >> >> on
> >> >> >>> AIX
> >> >> >>> > with NLS enabled.
> >> >> >>>
> >> >> >>> That should still work with gettext 0.22+ extracted in-tree (it
> >> should
> >> >> >>> be fetched by download_prerequisites).
> >> >> >>>
> >> >> >>> > A change in dependencies for GCC should have been announced and
> >> more
> >> >> >>> widely
> >> >> >>> > socialized in the GCC development mailing list, not just GCC
> >> patches
> >> >> >>> > mailing list.
> >> >> >>> >
> >> >> >>> > I have tried both the AIX Open Source libiconv and libgettext
> >> >> package,
> >> >> >>> and
> >> >> >>> > the ones that I previously built.  Both fail because GCC
> configure
> >> >> >>> decides
> >> >> >>> > to disable NLS, despite being requested, while libcpp is
> >> satisfied,
> >> >> so
> >> >> >>> > tools in the gcc subdirectory don't link against libiconv and
> the
> >> >> build
> >> >> >>> > fails.  With the included gettext, I was able to rely on a
> >> >> >>> self-consistent
> >> >> >>> > solution.
> >> >> >>>
> >> >> >>> That is interesting.  They should be using the same checks.  I've
> >> >> >>> checked trunk and regenerated files on it, and saw no significant
> >> diff
> >> >> >>> (some whitespace changes only).  Could you post the config.log of
> >> both?
> >> >> >>>
> >> >> >>> I've never used AIX.  Can I reproduce this on one of the cfarm
> >> machines
> >> >> >>> to poke around?  I've tried cfarm119, but that one lacked git,
> and I
> >> >> >>> haven't poked around much further due to time constraints.
> >> >> >>>
> >> >> >>
> >> >> >> The AIX system in the Compile Farm has a complete complement of
> Open
> >> >> Source
> >> >> >> software installed.
> >> >> >>
> >> >> >> Please ensure that /opt/freeware/bin is in your path.  Also, the
> GCC
> >> >> Wiki
> >> >> >> Compile Farm page has build tips that include AIX
> >> >> >>
> >> >> >>
> >> >>
> >>
> https://gcc.gnu.org/wiki/CompileFarm#Services_and_software_installed_on_farm_machines
> >> >> >
> >> >> > Thanks, that got me further.
> >> >> >
> >> >> >> that recommended --with-included-gettext configuration option.
> >> >> >
> >> >> > This flag should still exist and operate the same if gettext is
> >> present
> >> >> > in tree.  I've cloned gcc and downloaded prerequisites (via
> >> >> > contrib/download_prerequisites) and I am trying to configure it
> now.
> >> >>
> >> >> The build failed.  After gettext/gmp/... (in-tree hostlibs) get built
> >> >> and added to library paths, further GCC processes fail to run:
> >> >>
> >> >> configure:3305: gcc -g  -static-libstdc++ -static-libgcc -Wl,-bbigtoc
> >> >> conftest.c  >&5
> >> >> Could not load program
> >> >> /opt/freeware/libexec/gcc/powerpc-ibm-aix7.3.0.0/10/cc1:
> >> >>         Dependent module
> >> >> /home/arsen/build/./gmp/.libs/libgmp.a(libgmp.so.10) could not be
> >> loaded.
> >> >>         Member libgmp.so.10 is not found in archive
> >> >>
> >> >> This seems odd.  I am not sure what compels the RTDL (?) to look up
> .sos
> >> >> in archives, or how it knows about these archives..  I suspect it's
> >> >> getting tripped by something in HOST_EXPORTS.
> >> >>
> >> >> >> Thanks, David
> >> >> >>
> >> >> >>
> >> >> >>>
> >> >> >>> TIA, sorry about the inconvenience.  Have a lovely day.
> >> >> >>>
> >> >> >>> > The current gettext-0.22.3 fails to build for me on AIX.
> >> >> >>> >
> >> >> >>> > libcpp configure believes that NLS functions on AIX, but gcc
> >> >> configure
> >> >> >>> > fails in its tests of gettext functionality, which leads to an
> >> >> >>> inconsistent
> >> >> >>> > configuration and build breakage.
> >> >> >>> >
> >> >> >>> > Thanks, David
> >> >> >>>
> >> >> >>>
> >> >> >>> --
> >> >> >>> Arsen Arsenović
> >> >> >>>
> >> >>
> >> >>
> >> >> --
> >> >> Arsen Arsenović
> >> >>
> >>
> >>
> >> --
> >> Arsen Arsenović
> >>
>
>
> --
> Arsen Arsenović
>