[Bug modula2/110284] New: [14 Regression] Bootstrap failures with m2

[Bug modula2/110284] New: [14 Regression] Bootstrap failures with m2

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

            Bug ID: 110284
           Summary: [14 Regression] Bootstrap failures with m2
           Product: gcc
           Version: 14.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: modula2
          Assignee: gaius at gcc dot gnu.org
          Reporter: jakub at gcc dot gnu.org
  Target Milestone: ---

As mentioned in
https://gcc.gnu.org/pipermail/gcc-patches/2023-June/620819.html
and later in
https://gcc.gnu.org/pipermail/gcc-patches/2023-June/621576.html
since the inclusion of insn-opinit.h (generated header) in internal-fn.h
parallel bootstraps with modula2 enabled randomly fail with errors like:
In file included from ../../gcc/m2/gm2-gcc/gcc-consolidation.h:74,
                 from ../../gcc/m2/gm2-lang.cc:24:
../../gcc/internal-fn.h:24:10: fatal error: insn-opinit.h: No such file or
directory
   24 | #include "insn-opinit.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.
(not always, just sometimes with bad luck).
My r14-1603-g00bfc503cc3b3e8f3 commit for this was actually incorrect and
should be reverted - the problem is that gcc/m2/Make-lang.in is included before
generated_files
variable is set in gcc/Makefile.in and the order only rule
$(GM2_C_OBJS) : | $(generated_files)
is then expanded at that point as
$(GM2_C_OBJS) : | 
and so doesn't actually do anything.

As I said, the right fix is mentioned in the second mail,
-$(GM2_C_OBJS) : | $(generated_files)
+m2_OBJS = $(GM2_C_OBJS)
should work correctly, because then the
$(ALL_HOST_OBJS) : | $(generated_files)
rule in gcc/Makefile.in will include the m2 FE files and will DTRT.

The problem is that this needs further work on the m2 FE, which I can't easily
do because I don't know much about the FE.  Inclusion in m2_OBJS means they
will be
compiled with -DIN_GCC_FRONTEND and e.g. checking in system.h tries to make
sure
that rtl.h, except.h etc. headers aren't included in the FEs, but m2 FE does
include them (not sure why, most of them probably uselessly).  There is a
further problem that
m2 compiles its own variant of stor-layout.cc which is clearly a backend file.

If this can't be made to work correctly, I guess as last resort perhaps
-$(GM2_C_OBJS) : | $(generated_files)
+ALL_HOST_OBJS += $(GM2_C_OBJS)
could be used, which would be an ugly hack, but I think putting it in m2_OBJS
is the right thing.

Gaius, could you please have a look?
Thanks.

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |14.0
           Priority|P3                          |P1

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[gaius at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

Gaius Mulley <gaius at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2023-06-17
     Ever confirmed|0                           |1
             Status|UNCONFIRMED                 |ASSIGNED

--- Comment #1 from Gaius Mulley <gaius at gcc dot gnu.org> ---
Certainly - will investigate - thanks for the diagnosis.

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[gaius at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

Gaius Mulley <gaius at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |gaius at gcc dot gnu.org

--- Comment #2 from Gaius Mulley <gaius at gcc dot gnu.org> ---
Created attachment 55359
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=55359&action=edit
Proposed fix

Thanks for the hints and diagnosis - very useful - especially as it should fail
reliably.  Here is a proposed patch based on your 2nd email using:

m2_OBJS = $(GM2_C_OBJS)

apologies for not seeing the 2nd email on Monday.  I've seen the patch
bootstrap successfully on lp64 amd64 using make -j 30.

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

--- Comment #3 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Comment on attachment 55359
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=55359
Proposed fix

Except for the stor-layout changes that is what I was trying, so if it works,
great.

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[gaius at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

--- Comment #4 from Gaius Mulley <gaius at gcc dot gnu.org> ---
Also now bootstrapped successfully on ppc64le - so I'll git commit the changes.

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

--- Comment #5 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Gaius Mulley <gaius@gcc.gnu.org>:

https://gcc.gnu.org/g:24f75498ea59be94ae3eb1b82aa0b9de6ebfa232

commit r14-1911-g24f75498ea59be94ae3eb1b82aa0b9de6ebfa232
Author: Gaius Mulley <gaiusmod2@gmail.com>
Date:   Sun Jun 18 21:07:59 2023 +0100

    PR modula2/110284 Remove stor-layout.o and backend header files

    This patch removes stor-layout.o from the front end and also removes
    back end header files from gcc-consolidation.h.

    gcc/m2/ChangeLog:

            PR modula2/110284
            * Make-lang.in (m2_OBJS): Assign $(GM2_C_OBJS).
            (GM2_C_OBJS): Remove m2/stor-layout.o.
            (m2/stor-layout.o): Remove rule.
            * gm2-gcc/gcc-consolidation.h (rtl.h): Remove include.
            (df.h): Remove include.
            (except.h): Remove include.

    Signed-off-by: Gaius Mulley <gaiusmod2@gmail.com>

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[gaius at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

--- Comment #6 from Gaius Mulley <gaius at gcc dot gnu.org> ---
ok to close?

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #7 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Sure.  It will take a week or two to be sure it worked, but I can always reopen
if it didn't.

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[gaius at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

Gaius Mulley <gaius at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #8 from Gaius Mulley <gaius at gcc dot gnu.org> ---
optimistically closing - thanks!

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|CLOSED                      |REOPENED
         Resolution|FIXED                       |---

--- Comment #9 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Arsen/Iain reported on IRC
is this known/fixable? (I think I have seen other folks comment on it)
In file included from /src-local/arsen/gcc/m2/gm2-gcc/gcc-consolidation.h:71,
         from /src-local/arsen/gcc/m2/m2.flex:23:
#include "insn-opinit.h"

Neither m2/gm2-compiler/m2flex.o nor m2/gm2-compiler-boot/m2flex.o are included
in m2_OBJS (they surely don't need to be in GM2_C_OBJS, that is m2 internal
variable).
Question is what other objects are suffering from that.

I guess best test would be to add sleep 1h into the s-opinit rule in
gcc/Makefile.in,
before the $(RUN_GEN) build/genopinit$(build_exeext) $(md_file) \ line or so
and then make -j128 -k (or some other very high number, such that it prints all
the problems).

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

--- Comment #10 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
gcc-consolidation.h is included by
m2/m2.flex:#include "gm2-gcc/gcc-consolidation.h"
m2/gm2-lang.cc:#include "gm2-gcc/gcc-consolidation.h"
m2/m2pp.cc:#include "gm2-gcc/gcc-consolidation.h"
m2/m2pp.cc:#include "gm2-gcc/gcc-consolidation.h"
m2/gm2-gcc/m2tree.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2expr.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2linemap.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2builtins.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2misc.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2convert.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2treelib.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2statement.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2top.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2assert.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2decl.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2type.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2color.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2except.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2block.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/m2configure.cc:#include "gcc-consolidation.h"
m2/gm2-gcc/rtegraph.cc:#include "gcc-consolidation.h"
so perhaps
-m2_OBJS = $(GM2_C_OBJS)
+m2_OBJS = $(GM2_C_OBJS) m2/gm2-gcc/rtegraph.o \
+          m2/gm2-compiler-boot/m2flex.o m2/gm2-compiler/m2flex.o
could be enough, all the other files are in $(GM2_C_OBJS).
Oh, and there is a weird:
c-family/m2pp.o : $(srcdir)/m2/m2pp.cc $(GCC_HEADER_DEPENDENCIES_FOR_M2)
        $(COMPILER) -c -g $(ALL_COMPILERFLAGS) \
            $(ALL_CPPFLAGS) $(INCLUDES) $< $(OUTPUT_OPTION)
rule.

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[iains at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

Iain Sandoe <iains at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |iains at gcc dot gnu.org

--- Comment #11 from Iain Sandoe <iains at gcc dot gnu.org> ---
I have seen (but this is random, I guess depending on the order in which builds
occur):

In file included from /src-local/arsen/gcc/m2/gm2-gcc/gcc-consolidation.h:71,
                from /src-local/arsen/gcc/m2/m2.flex:23:
24 | #include "insn-opinit.h"

If I can repeat will add more here.

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[gaius at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

--- Comment #12 from Gaius Mulley <gaius at gcc dot gnu.org> ---
Created attachment 55391
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=55391&action=edit
Proposed fix (addendum)

I wonder whether this modification to Jakub's patch might help?

m2/gm2-compiler-boot/m2flex.o is needed by cc1gm2  whereas
m2/gm2-compiler/m2flex.o is required by m2/stage2/cc1gm2 (only built in
maintainer to allow debugging via m2 sources rather than the translated to C++
sources.

Also moved the unused (debugging rule) c-family/m2pp.o into Make-maintainer.o
(currently being bootstrapped on x86_64 make -j 12) and (ppc64le make -j 128)

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

--- Comment #13 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
My bootstrap today on i686-linux failed again
In file included from ../../gcc/m2/gm2-gcc/gcc-consolidation.h:71,
                 from ../../gcc/m2/gm2-gcc/rtegraph.cc:22:
../../gcc/internal-fn.h:24:10: fatal error: insn-opinit.h: No such file or
directory
   24 | #include "insn-opinit.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [../../gcc/m2/Make-lang.in:586: m2/gm2-gcc/rtegraph.o] Error 1
What is the state of this patch?

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[gaius at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

--- Comment #14 from Gaius Mulley <gaius at gcc dot gnu.org> ---
The first patch was pushed on 2023-06-18 and the second one hasn't been pushed
yet.  It has been tested though - I'll re-test it against master and push.

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

--- Comment #15 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Gaius Mulley <gaius@gcc.gnu.org>:

https://gcc.gnu.org/g:01f6e8b013237041adeac370f8d229aea6304591

commit r14-2654-g01f6e8b013237041adeac370f8d229aea6304591
Author: Gaius Mulley <gaiusmod2@gmail.com>
Date:   Wed Jul 19 21:01:53 2023 +0100

    PR modula2/110284 Make-lang-in m2flex.o and m2pp.o

    This patch moves the rule c-family/m2pp.o from Make-lang.in into
    Make-maintainer.in.  It also adds m2/gm2-gcc/rtegraph.o and
    m2/gm2-compiler-boot/m2flex.o to m2_OBJS.  The object
    m2/gm2-compiler-boot/m2flex.o is needed by cc1gm2 whereas
    m2/gm2-compiler/m2flex.o is required by m2/stage2/cc1gm2
    (which is only built in maintainer to allow debugging via m2
    sources rather than the translated to C++ sources).

            PR modula2/110284
            * Make-lang.in (m2_OBJS): Add m2/gm2-gcc/rtegraph.o and
            m2/gm2-compiler-boot/m2flex.o.
            (c-family/m2pp.o): Remove.
            * Make-maintainer.in (c-family/m2pp.o): Add.

    Signed-off-by: Gaius Mulley <gaiusmod2@gmail.com>

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[gaius at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

Gaius Mulley <gaius at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #16 from Gaius Mulley <gaius at gcc dot gnu.org> ---
optimistically closing again - please re-open if it fails on large builds -
thanks!

[Bug modula2/110284] [14 Regression] Bootstrap failures with m2

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110284

--- Comment #17 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-13 branch has been updated by Gaius Mulley
<gaius@gcc.gnu.org>:

https://gcc.gnu.org/g:1f0933298c4aa76646b4ea964d6fbc07504526c8

commit r13-7675-g1f0933298c4aa76646b4ea964d6fbc07504526c8
Author: Gaius Mulley <gaiusmod2@gmail.com>
Date:   Mon Jul 31 19:43:02 2023 +0100

    modula2: Fix bootstrap

    Combining 3 patches from master for Make-lang.in and header
    file changes to ensure that this sequence does not break git bisect.

    gcc/m2/ChangeLog:

            PR modula2/110284
            * Make-lang.in: Build $(generated_files) before building
            all $(GM2_C_OBJS).
            (m2_OBJS): Assign $(GM2_C_OBJS).  Add m2/gm2-gcc/rtegraph.o and
            m2/gm2-compiler-boot/m2flex.o.
            (GM2_C_OBJS): Remove m2/stor-layout.o.
            (m2/stor-layout.o): Remove rule.
            * gm2-gcc/gcc-consolidation.h (rtl.h): Remove include.
            (df.h): Remove include.
            (except.h): Remove include.
            (c-family/m2pp.o): Remove.
            * Make-maintainer.in (c-family/m2pp.o): Add.

    Signed-off-by: Gaius Mulley <gaiusmod2@gmail.com>