contributor guidelines

contributor guidelines

[Bruno Haible]

Hi,

I was asked to post a patch for a bugzilla PR to gcc-patches@. Two questions
regarding https://gcc.gnu.org/contribute.html#testing :

1) This web page says:
   "For a normal native configuration, running

      make bootstrap
      make -k check

    from the top level of the GCC tree (not the gcc subdirectory) will
    accomplish this."

But what I get, from a git checkout of the GCC tree:

  $ make bootstrap
  make: *** No rule to make target 'bootstrap'.  Stop.

I think the instructions should be amended to read:

      ./configure [options]
      make bootstrap
      make -k check

Right?

2) That's what I did:

$ ./configure --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu --prefix=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --enable-shared --enable-nls --enable-threads=posix --enable-__cxa_atexit --disable-multilib --with-as=/arch/x86_64-linux-gnu/gnu-inst-binutils/2.38/bin/as --with-ld=/arch/x86_64-linux-gnu/gnu-inst-binutils/2.38/bin/ld --with-gmp=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --with-mpfr=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --with-mpc=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --with-isl=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --enable-languages=c,c++,objc,obj-c++,lto,jit,fortran,go,d,m2 --enable-host-shared 2>&1 | tee log0
$ make bootstrap 2>&1 | tee log1

and it fails like this:

/GCC/gcc/host-x86_64-pc-linux-gnu/gcc/xgcc -B/GCC/gcc/host-x86_64-pc-linux-gnu/gcc/ -B/arch/x86_64-linux-gnu/gnu-inst-gcc/testing/x86_64-pc-linux-gnu/bin/ -B/arch/x86_64-linux-gnu/gnu-inst-gcc/testing/x86_64-pc-linux-gnu/lib/ -isystem /arch/x86_64-linux-gnu/gnu-inst-gcc/testing/x86_64-pc-linux-gnu/include -isystem /arch/x86_64-linux-gnu/gnu-inst-gcc/testing/x86_64-pc-linux-gnu/sys-include   -fno-checking -g -O2 -O2  -g -O2 -DIN_GCC   -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wno-format -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition  -isystem ./include  -fpic -mlong-double-80 -DUSE_ELF_SYMVER -fcf-protection -mshstk -g -DIN_LIBGCC2 -fbuilding-libgcc -fno-stack-protector  -fpic -mlong-double-80 -DUSE_ELF_SYMVER -fcf-protection -mshstk -I. -I. -I../../host-x86_64-pc-linux-gnu/gcc -I../.././libgcc -I../.././libgcc/. -I../.././libgcc/../gcc -I../.././libgcc/../include -I../.././libgcc/config/libbid -DENABLE_DECIMAL_BID_FORMAT -DHAVE_CC_TLS  -DUSE_TLS  -o bid128_add.o -MT bid128_add.o -MD -MP -MF bid128_add.dep -c ../.././libgcc/config/libbid/bid128_add.c
during RTL pass: expand
../.././libgcc/config/libbid/bid128_add.c: In function ‘__bid128_add’:
../.././libgcc/config/libbid/bid128_add.c:637:42: internal compiler error: Aborted
  637 |       else if (rnd_mode == ROUNDING_DOWN && x_sign != y_sign)
0x178a410 crash_signal
        ../.././gcc/toplev.cc:316
0x7fd115f1c51f ???
        ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x7fd115f709fc __pthread_kill_implementation
        ./nptl/pthread_kill.c:44
0x7fd115f709fc __pthread_kill_internal
        ./nptl/pthread_kill.c:78
0x7fd115f709fc __GI___pthread_kill
        ./nptl/pthread_kill.c:89
0x7fd115f1c475 __GI_raise
        ../sysdeps/posix/raise.c:26
0x7fd115f027f2 __GI_abort
        ./stdlib/abort.c:79
0x110f4ff choose_mult_variant(machine_mode, long, algorithm*, mult_variant*, int)
        ../.././gcc/expmed.cc:3282
0x1149c12 expand_expr_real_2(separate_ops*, rtx_def*, machine_mode, expand_modifier)
        ../.././gcc/expr.cc:10225
0x114e729 expand_expr_real_1(tree_node*, rtx_def*, machine_mode, expand_modifier, rtx_def**, bool)
        ../.././gcc/expr.cc:11216
0x1145ae1 expand_expr_real(tree_node*, rtx_def*, machine_mode, expand_modifier, rtx_def**, bool)
        ../.././gcc/expr.cc:9408
0xf2ed74 expand_expr(tree_node*, rtx_def*, machine_mode, expand_modifier)
        ../.././gcc/expr.h:313
0x11441c7 expand_operands(tree_node*, tree_node*, rtx_def*, rtx_def**, rtx_def**, expand_modifier)
        ../.././gcc/expr.cc:8986
0x114d793 expand_expr_real_2(separate_ops*, rtx_def*, machine_mode, expand_modifier)
        ../.././gcc/expr.cc:10987
0xf69453 expand_gimple_stmt_1
        ../.././gcc/cfgexpand.cc:3984
0xf6971f expand_gimple_stmt
        ../.././gcc/cfgexpand.cc:4045
0xf72801 expand_gimple_basic_block
        ../.././gcc/cfgexpand.cc:6101
0xf752b2 execute
        ../.././gcc/cfgexpand.cc:6836
Please submit a full bug report, with preprocessed source (by using -freport-bug).
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.
make[3]: *** [Makefile:650: bid128_add.o] Error 1
make[3]: Leaving directory '/GCC/gcc/x86_64-pc-linux-gnu/libgcc'
make[2]: *** [Makefile:20611: all-stage1-target-libgcc] Error 2
make[2]: Leaving directory '/GCC/gcc'
make[1]: *** [Makefile:30945: stage1-bubble] Error 2
make[1]: Leaving directory '/GCC/gcc'
make: *** [Makefile:31282: bootstrap] Error 2

I am surprised to see such a failure, since x86_64-linux-gnu is the most
common platform.

What am I supposed to do?
  - Use the latest release tag (13.2.0) instead of the tip of 'master',
    and test my contribution there?
  - Wait a few days, do a 'git pull', and retry?
  - Something else?

Bruno

Re: contributor guidelines

[Jonathan Wakely]

On Mon, 4 Dec 2023 at 12:19, Bruno Haible wrote:
>
> Hi,
>
> I was asked to post a patch for a bugzilla PR to gcc-patches@. Two questions
> regarding https://gcc.gnu.org/contribute.html#testing :
>
> 1) This web page says:
>    "For a normal native configuration, running
>
>       make bootstrap
>       make -k check
>
>     from the top level of the GCC tree (not the gcc subdirectory) will
>     accomplish this."
>
> But what I get, from a git checkout of the GCC tree:
>
>   $ make bootstrap
>   make: *** No rule to make target 'bootstrap'.  Stop.
>
> I think the instructions should be amended to read:
>
>       ./configure [options]
>       make bootstrap
>       make -k check
>
> Right?

No, don't configure in the source directory:
https://gcc.gnu.org/wiki/FAQ#configure

But otherwise yes, the fact you need to configure before running make
is so obvious that apparently we took it for granted :-)

And it hasn't been necessary to run 'make bootstrap' for years, just
running 'make' does exactly the same thing now.

>
> 2) That's what I did:
>
> $ ./configure --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu --prefix=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --enable-shared --enable-nls --enable-threads=posix --enable-__cxa_atexit --disable-multilib --with-as=/arch/x86_64-linux-gnu/gnu-inst-binutils/2.38/bin/as --with-ld=/arch/x86_64-linux-gnu/gnu-inst-binutils/2.38/bin/ld --with-gmp=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --with-mpfr=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --with-mpc=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --with-isl=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --enable-languages=c,c++,objc,obj-c++,lto,jit,fortran,go,d,m2 --enable-host-shared 2>&1 | tee log0

This seems unnecessarily overcomplicated. Most of those options are
just restating the defaults.


> $ make bootstrap 2>&1 | tee log1
>
> and it fails like this:
>
> /GCC/gcc/host-x86_64-pc-linux-gnu/gcc/xgcc -B/GCC/gcc/host-x86_64-pc-linux-gnu/gcc/ -B/arch/x86_64-linux-gnu/gnu-inst-gcc/testing/x86_64-pc-linux-gnu/bin/ -B/arch/x86_64-linux-gnu/gnu-inst-gcc/testing/x86_64-pc-linux-gnu/lib/ -isystem /arch/x86_64-linux-gnu/gnu-inst-gcc/testing/x86_64-pc-linux-gnu/include -isystem /arch/x86_64-linux-gnu/gnu-inst-gcc/testing/x86_64-pc-linux-gnu/sys-include   -fno-checking -g -O2 -O2  -g -O2 -DIN_GCC   -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wno-format -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition  -isystem ./include  -fpic -mlong-double-80 -DUSE_ELF_SYMVER -fcf-protection -mshstk -g -DIN_LIBGCC2 -fbuilding-libgcc -fno-stack-protector  -fpic -mlong-double-80 -DUSE_ELF_SYMVER -fcf-protection -mshstk -I. -I. -I../../host-x86_64-pc-linux-gnu/gcc -I../.././libgcc -I../.././libgcc/. -I../.././libgcc/../gcc -I../.././libgcc/../include -I../.././libgcc/config/libbid -DENABLE_DECIMAL_BID_FORMAT -DHAVE_CC_TLS  -DUSE_TLS  -o bid128_add.o -MT bid128_add.o -MD -MP -MF bid128_add.dep -c ../.././libgcc/config/libbid/bid128_add.c
> during RTL pass: expand
> ../.././libgcc/config/libbid/bid128_add.c: In function ‘__bid128_add’:
> ../.././libgcc/config/libbid/bid128_add.c:637:42: internal compiler error: Aborted
>   637 |       else if (rnd_mode == ROUNDING_DOWN && x_sign != y_sign)
> 0x178a410 crash_signal
>         ../.././gcc/toplev.cc:316

I don't see this error when building master on x86_64-pc-linux-gnu.
This might be a local problem, caused by your host compiler, or the
versions of GMP, MPFR, or MPC, or something else.


> I am surprised to see such a failure, since x86_64-linux-gnu is the most
> common platform.
>
> What am I supposed to do?
>   - Use the latest release tag (13.2.0) instead of the tip of 'master',
>     and test my contribution there?
>   - Wait a few days, do a 'git pull', and retry?
>   - Something else?

Re: contributor guidelines

[waffl3x]

On Monday, December 4th, 2023 at 5:37 AM, Jonathan Wakely via Gcc <gcc@gcc.gnu.org> wrote:


> 
> 
> On Mon, 4 Dec 2023 at 12:19, Bruno Haible wrote:
> 
> > Hi,
> > 
> > I was asked to post a patch for a bugzilla PR to gcc-patches@. Two questions
> > regarding https://gcc.gnu.org/contribute.html#testing :
> > 
> > 1) This web page says:
> > "For a normal native configuration, running
> > 
> > make bootstrap
> > make -k check
> > 
> > from the top level of the GCC tree (not the gcc subdirectory) will
> > accomplish this."
> > 
> > But what I get, from a git checkout of the GCC tree:
> > 
> > $ make bootstrap
> > make: *** No rule to make target 'bootstrap'. Stop.
> > 
> > I think the instructions should be amended to read:
> > 
> > ./configure [options]
> > make bootstrap
> > make -k check
> > 
> > Right?
> 
> 
> No, don't configure in the source directory:
> https://gcc.gnu.org/wiki/FAQ#configure
> 
> But otherwise yes, the fact you need to configure before running make
> is so obvious that apparently we took it for granted :-)
> 
> And it hasn't been necessary to run 'make bootstrap' for years, just
> running 'make' does exactly the same thing now.

It falls under the weird paradox region imo, the one where it's such
common knowledge that it doesn't get written down anywhere, so it makes
it really hard for a newbie to figure out. As long as you have a mentor
or are brave enough to ask questions it's rarely an issue but I love
this little paradox. So easy that it's arcane to the uninitiated.

I've been writing `make bootstrap` myself, I'm surprised to hear it's
unnecessary. Perhaps we should update some of these documents a bit? 

> > 2) That's what I did:
> > 
> > $ ./configure --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu --prefix=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --enable-shared --enable-nls --enable-threads=posix --enable-__cxa_atexit --disable-multilib --with-as=/arch/x86_64-linux-gnu/gnu-inst-binutils/2.38/bin/as --with-ld=/arch/x86_64-linux-gnu/gnu-inst-binutils/2.38/bin/ld --with-gmp=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --with-mpfr=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --with-mpc=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --with-isl=/arch/x86_64-linux-gnu/gnu-inst-gcc/testing --enable-languages=c,c++,objc,obj-c++,lto,jit,fortran,go,d,m2 --enable-host-shared 2>&1 | tee log0
> 
> 
> This seems unnecessarily overcomplicated. Most of those options are
> just restating the defaults.
> 
> > $ make bootstrap 2>&1 | tee log1
> > 
> > and it fails like this:
> > 
> > *snip*
> 
> 
> I don't see this error when building master on x86_64-pc-linux-gnu.
> This might be a local problem, caused by your host compiler, or the
> versions of GMP, MPFR, or MPC, or something else.
> 

I would suggest just trying again with a out of tree build, in tree
builds are not supported. I can definitely imagine an in tree build
causing some wild errors like you described.

So for example, if you had your git repo in a folder named workspace,
like so.

workspace/gcc

You want another folder where you build in, this is frequently referred
to as the object directory.

workspace/obj

When in the object directory, workspace/obj you configure in there.

[obj]$ ../gcc/configure
THEN you do the rest.
(NOTE: -j16 sets the amount of threads to use in case you don't know)
[obj]$ make -j16 bootstrap

As jwakely said, bootstrap isn't required anymore, but this is how I've
done it and I don't wanna change it for my example, just in case.

After it's done, while still in the obj directory you do

[obj]$ make -j16 -k check

You'll notice that there's a surprising amount of test failures, the
only real way I know of to check for regressions is to have a pristine
version without your patch applied that you compare against.

My layout looks something like this right now.
workspace/bootstrap/obj/pristine-gcc14
workspace/bootstrap/obj/my-patch-gcc14
workspace/bootstrap/src/pristine-gcc14
workspace/bootstrap/src/my-patch-gcc14

Once you've ran the testsuite on both your pristine bootstrapped build
and your patched bootstrapped build, you can compare them like so.

[workspace]$ gcc/contrib/compare_tests /bootstrap/obj/pristine-gcc14/
 /bootstrap/obj/my-patch-gcc14/ > results

This is just how I do it, there might be a better way, I'm still pretty
green so don't take it as gospel, but on the off chance it's helpful I
wanted to share my process that has worked for me. Besides, this is a
great excuse for me to get criticism of my workflow. :^)

Oh and one other thing to note, when using this workflow, you'll get
some bizarre results where some tests will be listed as both new and
having disappeared. This is because some of the tests either
incorrectly include the full patch, or include the line number. Here's
examples of each of these from my latest comparison.
```
New tests that FAIL (4 tests):

g++.dg/modules/xtreme-header-4_b.C -std=c++2b (internal compiler error:
tree check: expected class 'type', have 'declaration' (template_decl)
in get_originating_module_decl, at cp/module.cc:18661)

libitm.c++/dropref.C -B
/home/waffl3x/projects/gcc-dev/bootstrap/obj/xobj-next-2/
x86_64-pc-linux-gnu/./libitm/../libstdc++-v3/src/.libs
execution test

Old tests that failed, that have disappeared (4 tests): (Eeek!)

g++.dg/modules/xtreme-header-4_b.C -std=c++2b (internal compiler error:
tree check: expected class 'type', have 'declaration' (template_decl)
in get_originating_module_decl, at cp/module.cc:18659)

libitm.c++/dropref.C -B
/home/waffl3x/projects/gcc-dev/bootstrap/obj/pristine-gcc14-2/
x86_64-pc-linux-gnu/./libitm/../libstdc++-v3/src/.libs
execution test
```
Note the line number in the xtreme-header-4_b.C test, this is due to me
adding 2 whole lines in modules.cc.

In the libitm test you'll notice it includes the full path in the test
log, that's what causes that one.
```
New tests that PASS (18 tests):

libitm.c++/dropref.C -B /home/waffl3x/projects/gcc-dev/bootstrap/obj/
xobj-next-2/x86_64-pc-linux-gnu/./libitm/../libstdc++-v3/src/.libs
(test for excess errors)

Old tests that passed, that have disappeared (18 tests): (Eeek!)

libitm.c++/dropref.C -B /home/waffl3x/projects/gcc-dev/bootstrap/obj/
pristine-gcc14-2/x86_64-pc-linux-gnu/./libitm/../libstdc++-v3/src/.libs
(test for excess errors)
```
As you can see, this also happens for tests that pass too. So if you
encounter something like this, it's nothing to be concerned about. This
cost me some time when I encountered it so hopefully you can avoid it
with this warning.

Additionally, here is a list of options to reduce build times for when
you need to build frequently. It was passed down to me and now I'll
pass it down to you. However, do take note that you STILL NEED TO DO A
FULL BOOTSTRAP before submitting your patch, this is just to expedite
the initial "make change, test, repeat" phase.

--disable-analyzer
--disable-bootstrap
--disable-cet
--disable-default-pie
--disable-default-ssp
--disable-fixincludes
--disable-gcov
--disable-libada
--disable-libatomic
--disable-libgomp
--disable-libitm
--disable-libquadmath
--disable-libsanitizer
--disable-libssp
--disable-libstdcxx-pch
--disable-libvtv
--disable-lto
--disable-multilib
--disable-nls
--disable-objc-gc
--disable-systemtap
--disable-werror
--enable-languages=c,c++
--prefix= *REPLACE WITH YOUR INSTALL DIRECTORY*
--with-checking=yes,extra,rtl
--without-isl
--without-zstd
--with-system-zlib

In case it wasn't obvious, these are the settings I use for development
of the C++ front end. If you're doing work on a different part of GCC
you'll need different flags, I can't tell you exactly which ones you
need unfortunately but hopefully this is useful information for you.

I'm sorry if any of this came off as condescending, you seem like
you're more proficient with a terminal than I but I didn't want to make
any assumptions on what could be left out and what couldn't be.

Hopefully this answers all your questions! If it doesn't I doubt I'll
be able to answer them as this is about the limit of my experience so
far, but I'll be happy to try if you do have more.

Alex

Re: contributor guidelines

[Bruno Haible]

[-- Attachment #1: Type: text/plain, Size: 1845 bytes --]

Hi Jonathan,

Thank you for your answer.

> No, don't configure in the source directory:
> https://gcc.gnu.org/wiki/FAQ#configure

OK.

> And it hasn't been necessary to run 'make bootstrap' for years, just
> running 'make' does exactly the same thing now.

Actually, "make" works better than "make bootstrap": While "make bootstrap"
fails with the cited internal compiler error while compiling
libgcc/config/libbid/bid128_add.c, "make" runs to completion.
I started both in separated subdirectories and configured with the same
options.

So, if nobody is using "make bootstrap" any more, that explains why I hit
that internal compiler error.

> I don't see this error when building master on x86_64-pc-linux-gnu.
> This might be a local problem, caused by your host compiler, or the
> versions of GMP, MPFR, or MPC, or something else.

You were probably running "make", not "make bootstrap"?

Next, I ran "make -k check". Its output shows 182 failures in various
test suites. What am I supposed to do?
  - Apply common sense and ignore the failures if they don't have an
    apparent relation with my contribution?
  - Repeat the entire build without my patch, and see if I get the same
    182 failures?
  - Submit my patch to gcc-patches@ anyway, attaching the list of failures?

In summary
==========

I think it would greatly help contributors if the wiki page
https://gcc.gnu.org/contribute.html#testing
was modified:
  1) to mention that the build should be done in a subdirectory or sibling
     directory,
  2) to mention that '../configure [options]' should be run (even if, to
     some people, it is obvious),
  3) to mention to run 'make', not 'make bootstrap' — because 'make bootstrap'
     is too fragile,
  4) to mention how to deal with the result of "make -k check".

Bruno

[-- Attachment #2: make-check.log.gz --]
[-- Type: application/gzip, Size: 28968 bytes --]