[Bug bootstrap/98318] New: libcody breaks DragonFly bootstrap

[Bug bootstrap/98318] New: libcody breaks DragonFly bootstrap

[rimvydas.jas at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

            Bug ID: 98318
           Summary: libcody breaks DragonFly bootstrap
           Product: gcc
           Version: 11.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: bootstrap
          Assignee: unassigned at gcc dot gnu.org
          Reporter: rimvydas.jas at gmail dot com
  Target Milestone: ---

Created attachment 49776
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=49776&action=edit
possible fix

Since the introduction of libcody, gcc bootstrap is broken on DragonFly BSD:

gmake[3]: Leaving directory '/build/trunk/libcpp'
Configuring stage 1 in ./libcody
configure: creating cache ./config.cache
checking build system type... x86_64-unknown-dragonfly5.9
checking host system type... x86_64-unknown-dragonfly5.9
checking number of CPUs... unknown
checking maintainer-mode...
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ -std=c++11 accepts -g... yes
checking whether g++ -std=c++11 is for C++11... yes
checking adding -Wl,--no-undefined to linker... ok
/data/gg/libcody/configure: CONFIG_FILES+= ./Makesub: not found
/data/gg/libcody/configure: CONFIG_FILES+= ./tests/Makesub: not found
checking bugurl... github.com/urnathan/libcody
checking checking... yes
checking exceptions... no
...
ar rc libcommon.a diagnostic.o diagnostic-color.o diagnostic-show-locus.o
diagnostic-format-json.o json.o edit-context.o pretty-print.o intl.o sbitmap.o
vec.o input.o version.o hash-table.o ggc-none.o memory-block.o selftest.o
selftest-diagnostic.o sort.o
ranlib  libcommon.a
gmake[3]: *** No rule to make target '../libcody/libcody.a', needed by
'cc1-checksum.c'.  Stop.
gmake[3]: Leaving directory '/build/trunk/gcc'
gmake[2]: *** [Makefile:4781: all-stage1-gcc] Error 2
gmake[2]: Leaving directory '/build/trunk'
gmake[1]: *** [Makefile:27488: stage1-bubble] Error 2
gmake[1]: Leaving directory '/build/trunk'
gmake: *** [Makefile:1004: all] Error 2

The /bin/sh is almquist shell and do not take VAR+= style appends.

Another issue missing <netinet/in.h> for sockaddr_in6 in netclient.cc and
netserver.cc.

Attached diff allows to bootstrap gcc11 once again.

[Bug bootstrap/98318] libcody breaks DragonFly bootstrap

[rimvydas.jas at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

--- Comment #1 from Rimvydas (RJ) <rimvydas.jas at gmail dot com> ---
Could there be added configure option to disable use of libcody functionality
globally like "./configure --disable-cody" or --disable-libstdcxx-modules?

[Bug bootstrap/98318] libcody breaks DragonFly bootstrap

[rimvydas.jas at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

--- Comment #2 from Rimvydas (RJ) <rimvydas.jas at gmail dot com> ---
With configure fixed in g:6d972f5183d8d476cfb008b85e224aa9b90e628d
only missing <netinet/in.h> header issue remains in netclient.cc and
netserver.cc:
g++ -std=c++11 -g -fno-enforce-eh-specs -fno-stack-protector
-fno-threadsafe-statics -fno-exceptions -fno-rtti
-fdebug-prefix-map=/data/gg/libcody/= -W -Wall -include config.h
-I/data/gg/libcody \
  -MMD -MP -MF netclient.d -c -o netclient.o /data/gg/libcody/netclient.cc
/data/gg/libcody/netclient.cc: In function 'int Cody::OpenInet6(const char**,
const char*, int)':
/data/gg/libcody/netclient.cc:114:3: error: 'sockaddr_in6' was not declared in
this scope
   sockaddr_in6 addr;
   ^~~~~~~~~~~~
/data/gg/libcody/netclient.cc:114:3: note: suggested alternative: 'sockaddr_un'
   sockaddr_in6 addr;
   ^~~~~~~~~~~~
   sockaddr_un
/data/gg/libcody/netclient.cc:115:12: error: 'addr' was not declared in this
scope
   memset (&addr, 0, sizeof (addr));

[Bug bootstrap/98318] libcody breaks DragonFly bootstrap

[hliu at amperecomputing dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

Hao Liu <hliu at amperecomputing dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hliu at amperecomputing dot com

--- Comment #3 from Hao Liu <hliu at amperecomputing dot com> ---
We can reproduce the failure on CentOS. But Ubuntu can pass.

This failure is related to following files and code:

---
1. gcc/Makefile.in

CODYLIB = ../libcody/libcody.a
BACKEND = libbackend.a main.o libcommon-target.a libcommon.a \
        $(CPPLIB) $(CODYLIB) $(LIBDECNUMBER)


2. gcc/gcc/c/Make-lang.in

cc1-checksum.c : build/genchecksum$(build_exeext) checksum-options \
        $(C_OBJS) $(BACKEND) $(LIBDEPS) 
---

It should have some dependence problems, as "libcody.a" must be ready before
building cc1-checksum.c. But don't know how to fix this problem, as I'm not
farmiliar with Makefile :(

[Bug bootstrap/98318] libcody breaks DragonFly bootstrap

[nathan at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

Nathan Sidwell <nathan at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING
   Last reconfirmed|                            |2020-12-21
     Ever confirmed|0                           |1

--- Comment #4 from Nathan Sidwell <nathan at gcc dot gnu.org> ---
/data/gg/libcody/configure: CONFIG_FILES+= ./Makesub: not found
/data/gg/libcody/configure: CONFIG_FILES+= ./tests/Makesub: not found

I think your shell is not understanding += -- what shell are you using?

Might be fixed by:

6d972f5183d 2020-12-16 | libcody: More dashism [Nathan Sidwell]

can you update and try again?

[Bug bootstrap/98318] libcody breaks DragonFly bootstrap

[hliu at amperecomputing dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

--- Comment #5 from Hao Liu <hliu at amperecomputing dot com> ---
Hi Nanthan,

We can still reprodcue this problem on CentOS 7 (X86) and CentOS 8.2 (AArch64).
We use last  GCC version of yesterday:108beb75da

The configure and build commands are (Bash is used):
$ ../gcc/configure --disable-bootstrap --disable-multilib
--enable-checking=release
$ make -j32
...
make[2]: *** No rule to make target '../libcody/libcody.a', needed by
'cc1-checksum.c'.  Stop.
make[2]: *** Waiting for unfinished jobs....

Do you have any idea about how to fix this?

[Bug bootstrap/98318] libcody breaks DragonFly bootstrap

[nathan at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

--- Comment #6 from Nathan Sidwell <nathan at gcc dot gnu.org> ---
FWIW my build server is CentOS Stream release 8 (x86_64), so it seems
something's different in your setup.  The toplevel make knows that libcody must
be built before gcc.  Perhaps the libcody build is failing, but not being
reported?

What happens if you cd into the libcody obj directory and try a 'make' there? 
(after you've hit the failure).  What does that dir's config.log look like?

[Bug bootstrap/98318] libcody breaks DragonFly bootstrap

[hliu at amperecomputing dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

--- Comment #7 from Hao Liu <hliu at amperecomputing dot com> ---
I found that:
  1. "make -j1" can pass, but "make -j8" always fails. It seems something wrong
with parallel build
  2. When "make -j8" failed, if I try "make -j8" again, it can pass.

> What happens if you cd into the libcody obj directory and try a 'make' there?  (after you've hit the failure).
I tried "make -j8" and libcody.a can be built successfully. This is why the 2nd
"make -j8" try can pass.

> What does that dir's config.log look like?
The config.log look OK, as "make -j1" can always work well. I compared
config.log of Ubuntu vs Centos, they are similar. The tail lines of config.log:
---
/* confdefs.h */
#define PACKAGE_NAME "codylib"
#define PACKAGE_TARNAME "codylib"
#define PACKAGE_VERSION "0.0"
#define PACKAGE_STRING "codylib 0.0"
#define PACKAGE_BUGREPORT "github.com/urnathan/libcody"
#define PACKAGE_URL ""
#define BUGURL "github.com/urnathan/libcody"
#define NMS_CHECKING 0

configure: exit 0
---

> The toplevel make knows that libcody must be built before gcc.
So the problem seems to be why libcody.a is not built. The build log of "make
-j8" on CentOS is strange, as it enters build/libcody/ and then leave the dir
without doing anything. The log is as following:
---
$ grep "libcody" out-j8.log
checking for memchr... mkdir -p -- ./libcody
checking for unistd.h... Configuring in ./libcody
checking bugurl... github.com/urnathan/libcody
checking for strtol... make[2]: Entering directory '.../build/libcody'
checking whether gcc hidden aliases work... make[2]: Leaving directory
'.../build/libcody'
make[2]: *** No rule to make target '../libcody/libcody.a', needed by
'cc1-checksum.c'.  Stop.
---

It seems nothing happenend afer entering build/libcody, no building is
triggered in build/libcody (If it is triggered, it should success just as
manually run "make -j8" in build/libcody). The log of success job is:
---
$ grep "libcody" out-j1.log
mkdir -p -- ./libcody
Configuring in ./libcody
checking bugurl... github.com/urnathan/libcody
make[2]: Entering directory '/home/ec2-user/gcc_tmp/build/libcody'
g++ -g -O2 -fno-enforce-eh-specs -fno-stack-protector -fno-threadsafe-statics
-fno-exceptions -fno-rtti -fdebug-prefix-map=../../gcc/libcody/= -W -Wall
-include config.h -I../../gcc/libcody \
  -MMD -MP -MF buffer.d -c -o buffer.o ../../gcc/libcody/buffer.cc
...
ar -cr libcody.a buffer.o client.o fatal.o netclient.o netserver.o resolver.o
packet.o server.o
ranlib libcody.a
make[2]: Leaving directory '.../build/libcody'
---


Nathan, do you have any idea why libcody.a is not built with "make -j8"?  It
seems configure is OK but something wrong with parallel build. Other libraries
(e.g. gmp, libdecnumber) don't have such problem. Thanks very much.

[Bug bootstrap/98318] libcody breaks DragonFly bootstrap

[hliu at amperecomputing dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

--- Comment #8 from Hao Liu <hliu at amperecomputing dot com> ---
Hi Nathan,

The problem is related to use another make binary, which is 4.2.0 and built by
ourselves. Maybe there is a strange bug.

Anyway, after using the system installed make (which is 4.2.1 and under
/usr/bin/), the problem is solved.

Thanks for your help!

[Bug bootstrap/98318] libcody breaks DragonFly bootstrap

[nathan at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

Nathan Sidwell <nathan at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|---                         |INVALID

--- Comment #9 from Nathan Sidwell <nathan at gcc dot gnu.org> ---
thanks for investigating.  glad you found the cause.  (I regularly build with
-j24 and -j8, so it did surprise me that you found the parallelism to be
signficant).  I'll close this report.

[Bug bootstrap/98318] libcody breaks DragonFly bootstrap

[rimvydas.jas at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

Rimvydas (RJ) <rimvydas.jas at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|INVALID                     |FIXED

--- Comment #10 from Rimvydas (RJ) <rimvydas.jas at gmail dot com> ---
With commits:
g:6d972f5183d8d476cfb008b85e224aa9b90e628d
g:31705b068fa5d6cbd04aa4ac5f5275bad37d2222
g:cf22f78ff6ead2b1f022d7e7367daedf459aa355
gcc11 now can successfully bootstrap on DragonFly BSD.
Marking as fixed.

[Bug bootstrap/98318] libcody breaks DragonFly bootstrap

[rimvydas.jas at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

--- Comment #11 from Rimvydas (RJ) <rimvydas.jas at gmail dot com> ---
Nathan,

there seem to be another issue for 'make check' invoke in top level dir:
configure --enable-bootstrap ...
gmake -j128 && gmake -j1 -k check

gmake[2]: Leaving directory '/zzz/build/trunk/libbacktrace'
gmake[2]: Entering directory '/zzz/build/trunk/libcpp'
gmake[2]: Nothing to be done for 'check'.
gmake[2]: Leaving directory '/zzz/build/trunk/libcpp'
gmake[2]: Entering directory '/zzz/build/trunk/libcody'
No ZSH, maybe flakey
echo '# Automatically generated by Make' >tests/cody.defs
echo 'testdir=/data/gg/libcody/tests' >>tests/cody.defs
echo 'timelimit=60' >>tests/cody.defs
echo 'memlimit=1' >>tests/cody.defs
echo 'cpulimit=60' >>tests/cody.defs
echo 'filelimit=1' >>tests/cody.defs
echo 'SHELL=/bin/sh' >>tests/cody.defs
/zzz/build/trunk/./prev-gcc/xg++ -B/zzz/build/trunk/./prev-gcc/
-B/opt/gcc11f/x86_64-unknown-dragonfly5.9/bin/ -nostdinc++
-B/zzz/build/trunk/prev-x86_64-unknown-dragonfly5.9/libstdc++-v3/src/.libs
-B/zzz/build/trunk/prev-x86_64-unknown-dragonfly5.9/libstdc++-v3/libsupc++/.libs

-I/zzz/build/trunk/prev-x86_64-unknown-dragonfly5.9/libstdc++-v3/include/x86_64-unknown-dragonfly5.9
 -I/zzz/build/trunk/prev-x86_64-unknown-dragonfly5.9/libstdc++-v3/include 
-I/data/gg/libstdc++-v3/libsupc++
-L/zzz/build/trunk/prev-x86_64-unknown-dragonfly5.9/libstdc++-v3/src/.libs
-L/zzz/build/trunk/prev-x86_64-unknown-dragonfly5.9/libstdc++-v3/libsupc++/.libs
-g -O2 -include config.h -I/data/gg/libcody
-I/zzz/build/trunk/prev-x86_64-unknown-dragonfly5.9/libstdc++-v3/include/x86_64-unknown-dragonfly5.9
-I/zzz/build/trunk/prev-x86_64-unknown-dragonfly5.9/libstdc++-v3/include
-I/data/gg/libstdc++-v3/libsupc++ \
  -MMD -MP -MF tests/01-serialize/connect.d -c -o tests/01-serialize/connect.o
/data/gg/libcody/tests/01-serialize/connect.cc
/zzz/build/trunk/./prev-gcc/xg++ -B/zzz/build/trunk/./prev-gcc/
-B/opt/gcc11f/x86_64-unknown-dragonfly5.9/bin/ -nostdinc++
-B/zzz/build/trunk/prev-x86_64-unknown-dragonfly5.9/libstdc++-v3/src/.libs
-B/zzz/build/trunk/prev-x86_64-unknown-dragonfly5.9/libstdc++-v3/libsupc++/.libs

-I/zzz/build/trunk/prev-x86_64-unknown-dragonfly5.9/libstdc++-v3/include/x86_64-unknown-dragonfly5.9
 -I/zzz/build/trunk/prev-x86_64-unknown-dragonfly5.9/libstdc++-v3/include 
-I/data/gg/libstdc++-v3/libsupc++
-L/zzz/build/trunk/prev-x86_64-unknown-dragonfly5.9/libstdc++-v3/src/.libs
-L/zzz/build/trunk/prev-x86_64-unknown-dragonfly5.9/libstdc++-v3/libsupc++/.libs
-static-libstdc++ -static-libgcc  tests/01-serialize/connect.o -lcody  -o
tests/01-serialize/connect
/usr/libexec/binutils234/elf/ld.gold: error: cannot find -lcody
/data/gg/libcody/cody.hh:288: error: undefined reference to
'Cody::Packet::Destroy()'
/data/gg/libcody/tests/01-serialize/connect.cc:24: error: undefined reference
to 'Cody::Client::~Client()

The make -k check issue in libcody directory is reproducible on Linux box too.

[Bug bootstrap/98318] libcody breaks DragonFly bootstrap

[nathan at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

Nathan Sidwell <nathan at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|FIXED                       |---
             Status|RESOLVED                    |ASSIGNED

[Bug bootstrap/98318] libcody breaks DragonFly bootstrap

[nathan at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

Nathan Sidwell <nathan at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |WAITING

--- Comment #12 from Nathan Sidwell <nathan at gcc dot gnu.org> ---
Deleting testsuite:
16929214fd1 2020-12-23 | libcody:  Remove testsuite [PR 98318]

[Bug bootstrap/98318] [11 Regression] libcody breaks DragonFly bootstrap

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |11.0
            Summary|libcody breaks DragonFly    |[11 Regression] libcody
                   |bootstrap                   |breaks DragonFly bootstrap
             Target|                            |x86_64-*-dragonfly

[Bug bootstrap/98318] [11 Regression] libcody breaks DragonFly bootstrap

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #13 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
So anything left to do here?
This seemed to be marked as fixed, then reopened for the testsuite, but the
testsuite has been removed.

[Bug bootstrap/98318] [11 Regression] libcody breaks DragonFly bootstrap

[rimvydas.jas at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

--- Comment #14 from Rimvydas (RJ) <rimvydas.jas at gmail dot com> ---
Nathan,

It has come to our attention that some of c++ modules tests are failing if the
kernel has IPV6 support disabled as per bootstrap tools policies.  Are there
guarantees that local two stage bootstrap will remain possible without need for
a functional (non blocked/filtered) network stack?

Also, two possible typos in:
gcc/cp/mapper-client.cc#L252 should be "= -1"? This avoids timeouts in the
testsuite.
                        int fd = 01;
#if CODY_NETWORKING
                        fd = Cody::OpenInet6 (&errmsg, name.c_str (), port);
#endif

gcc/cp/module.cc:#if NETWORKING vs
c++tools/server.cc:#ifdef NETWORKING

[Bug bootstrap/98318] [11 Regression] libcody breaks DragonFly bootstrap

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

--- Comment #15 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Nathan Sidwell <nathan@gcc.gnu.org>:

https://gcc.gnu.org/g:2bf9e68a9589f9d3ace3663a17ff923191144c3b

commit r11-7389-g2bf9e68a9589f9d3ace3663a17ff923191144c3b
Author: Nathan Sidwell <nathan@acm.org>
Date:   Thu Feb 25 04:57:09 2021 -0800

    c++:  Fix typo in module-mapper [PR 98318]

    User reported this typo: '0' and '-' are right next to each other, and
    as it happened I always had networking, so it went unnoticed.

            PR c++/98318
            gcc/cp/
            * mapper-client.cc (module_client::open_module_client): Fix typo
            of fd init.

[Bug bootstrap/98318] [11 Regression] libcody breaks DragonFly bootstrap

[nathan at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

--- Comment #16 from Nathan Sidwell <nathan at gcc dot gnu.org> ---
Which tests are failing and how?

[Bug bootstrap/98318] [11 Regression] libcody breaks DragonFly bootstrap

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98318

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|---                         |FIXED

--- Comment #17 from Richard Biener <rguenth at gcc dot gnu.org> ---
The bootstrap issue is fixed.  Please move unrelated issues to new bugs.