[Bug gcov-profile/100788] New: Internal compiler error related to #line macros(?)

[Bug gcov-profile/100788] New: Internal compiler error related to #line macros(?)

[sebastian-gcc at sipsolutions dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

            Bug ID: 100788
           Summary: Internal compiler error related to #line macros(?)
           Product: gcc
           Version: 11.1.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: gcov-profile
          Assignee: unassigned at gcc dot gnu.org
          Reporter: sebastian-gcc at sipsolutions dot net
                CC: marxin at gcc dot gnu.org
  Target Milestone: ---

I first thought this might have been the identical issue as
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95847 as it seemed similar.  There
is also https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96353, but I do not think
this is the same issue.

We have a large file in NumPy which generates compile errors on gcc 9+ (gcc 8
seems fine as far as I can tell).  The issue on NumPy is:
https://github.com/numpy/numpy/issues/18529  (It is not a major issue, but I
fear it might get problematic for our coverage tools when they stop using GCC
8, which I think they are still using.)


I have attached the source file that generates the issue.  Note that it is
generated by a custom templating language, so I can't fully rule out that the
inserted `#line` directives are incorrect.

The one clear thing I know is that removing all `#line` directives makes the
coverage compilation successfully.

I expect you may need more information, but am not sure how to best provide it,
considering that the file is part of the larger project.

The error/log is (this one with gcc (Debian 10.2.1-6) 10.2.1 20210110, but
confirmed on a gcc 11:

...
x86_64-linux-gnu-gcc: numpy/core/src/common/python_xerbla.c
during IPA pass: profile
numpy/core/src/multiarray/einsum_sumprod.c.src: In function
‘longdouble_sum_of_products_contig_three’:
numpy/core/src/multiarray/einsum_sumprod.c.src:1264:1: internal compiler error:
in coverage_begin_function, at coverage.c:656
 1264 | }
      | ^
0x7ff60fe2cd09 __libc_start_main
        ../csu/libc-start.c:308
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <file:///usr/share/doc/gcc-10/README.Bugs> for instructions.
error: Command "x86_64-linux-gnu-gcc -pthread --coverage -Wno-unused-result
-Wsign-compare -O0 -ggdb -g
-ffile-prefix-map=/build/python3.9-RNBry6/python3.9-3.9.2=.
-fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2
-Werror=vla -Werror=nonnull -Werror=pointer-arith -Wlogical-op
-Werror=unused-function -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC
-DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64
-D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -DHAVE_CBLAS
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/common
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/umath -Inumpy/core/include
-Ibuild/src.linux-x86_64-3.9/numpy/core/include/numpy
-Ibuild/src.linux-x86_64-3.9/numpy/distutils/include -Inumpy/core/src/common
-Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath
-Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort
-Inumpy/core/src/_simd -I/usr/include/python3.9
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/common
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/npymath -c
build/src.linux-x86_64-3.9/numpy/core/src/multiarray/einsum_sumprod.c -o
build/temp.linux-x86_64-3.9/build/src.linux-x86_64-3.9/numpy/core/src/multiarray/einsum_sumprod.o
-MMD -MF
build/temp.linux-x86_64-3.9/build/src.linux-x86_64-3.9/numpy/core/src/multiarray/einsum_sumprod.o.d
-msse -msse2 -msse3" failed with exit status 1

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[sebastian-gcc at sipsolutions dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

--- Comment #1 from seberg <sebastian-gcc at sipsolutions dot net> ---
Created attachment 50875
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=50875&action=edit
preprocessed_file

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING
   Last reconfirmed|                            |2021-05-27
     Ever confirmed|0                           |1

--- Comment #2 from Martin Liška <marxin at gcc dot gnu.org> ---
It seems the pre-processed file is bad.
Please create one with -E option and show please full command line that ICEs.

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[sebastian-gcc at sipsolutions dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

--- Comment #3 from seberg <sebastian-gcc at sipsolutions dot net> ---
Created attachment 50878
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=50878&action=edit
actual_preprocessed_with_gcc

Sorry, I think this is what you are looking for now, adding -E to it.

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

--- Comment #4 from Martin Liška <marxin at gcc dot gnu.org> ---
> 
> Sorry, I think this is what you are looking for now, adding -E to it.

The file is correct now. Now I miss the command line.

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[sebastian-gcc at sipsolutions dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

--- Comment #5 from seberg <sebastian-gcc at sipsolutions dot net> ---
> Now I miss the command line.

Do you mean more than the failing call?:

x86_64-linux-gnu-gcc -pthread --coverage -O0 -ggdb -g
-ffile-prefix-map=/build/python3.9-RNBry6/python3.9-3.9.2=.
-fstack-protector-strong -g -fwrapv -O2  -D_FORTIFY_SOURCE=2 -fPIC
-DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64
-D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -DHAVE_CBLAS
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/common
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/umath -Inumpy/core/include
-Ibuild/src.linux-x86_64-3.9/numpy/core/include/numpy
-Ibuild/src.linux-x86_64-3.9/numpy/distutils/include -Inumpy/core/src/common
-Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath
-Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort
-Inumpy/core/src/_simd -I/usr/include/python3.9
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/common
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/npymath -c
build/src.linux-x86_64-3.9/numpy/core/src/multiarray/einsum_sumprod.c -o
build/temp.linux-x86_64-3.9/build/src.linux-x86_64-3.9/numpy/core/src/multiarray/einsum_sumprod.o
-msse2 -msse3

(with some warning flags removed compared to the initial report; I can try to
zip up the dependencies tomorrow. To make it reproducable)

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

--- Comment #6 from Martin Liška <marxin at gcc dot gnu.org> ---
(In reply to seberg from comment #5)
> > Now I miss the command line.
> 
> Do you mean more than the failing call?:
> 
> x86_64-linux-gnu-gcc -pthread --coverage -O0 -ggdb -g
> -ffile-prefix-map=/build/python3.9-RNBry6/python3.9-3.9.2=.
> -fstack-protector-strong -g -fwrapv -O2  -D_FORTIFY_SOURCE=2 -fPIC
> -DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64
> -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -DHAVE_CBLAS
> -Ibuild/src.linux-x86_64-3.9/numpy/core/src/common
> -Ibuild/src.linux-x86_64-3.9/numpy/core/src/umath -Inumpy/core/include
> -Ibuild/src.linux-x86_64-3.9/numpy/core/include/numpy
> -Ibuild/src.linux-x86_64-3.9/numpy/distutils/include -Inumpy/core/src/common
> -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath
> -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort
> -Inumpy/core/src/_simd -I/usr/include/python3.9
> -Ibuild/src.linux-x86_64-3.9/numpy/core/src/common
> -Ibuild/src.linux-x86_64-3.9/numpy/core/src/npymath -c
> build/src.linux-x86_64-3.9/numpy/core/src/multiarray/einsum_sumprod.c -o
> build/temp.linux-x86_64-3.9/build/src.linux-x86_64-3.9/numpy/core/src/
> multiarray/einsum_sumprod.o -msse2 -msse3

This works for me. Btw. you reproduce the issue when using the pre-processed
source file?

The crash is about a function that has wrong source start and end location from
C front-end.

> 
> (with some warning flags removed compared to the initial report; I can try
> to zip up the dependencies tomorrow. To make it reproducable)

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[sebastian-gcc at sipsolutions dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

--- Comment #7 from seberg <sebastian-gcc at sipsolutions dot net> ---
Oh, interesting.  You are right, if I run it on the preprocessed file it
succeeds (note the `tmp/preprocessed.c`:

x86_64-linux-gnu-gcc -pthread --coverage -O0 -ggdb -g
-ffile-prefix-map=/build/python3.9-RNBry6/python3.9-3.9.2=.
-fstack-protector-strong -g -fwrapv -O2  -D_FORTIFY_SOURCE=2 -fPIC
-DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64
-D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -DHAVE_CBLAS
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/common
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/umath -Inumpy/core/include
-Ibuild/src.linux-x86_64-3.9/numpy/core/include/numpy
-Ibuild/src.linux-x86_64-3.9/numpy/distutils/include -Inumpy/core/src/common
-Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath
-Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort
-Inumpy/core/src/_simd -I/usr/include/python3.9
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/common
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/npymath -c /tmp/preprocessed.c -o
build/temp.linux-x86_64-3.9/build/src.linux-x86_64-3.9/numpy/core/src/multiarray/einsum_sumprod.o
-msse2 -msse3

<no error>


Just to cover my ground the preprocessed file was generated with:

x86_64-linux-gnu-gcc -pthread --coverage -Wno-unused-result -Wsign-compare -O0
-ggdb -g -ffile-prefix-map=/build/python3.9-RNBry6/python3.9-3.9.2=.
-fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2
-Werror=vla -Werror=nonnull -Werror=pointer-arith -Wlogical-op
-Werror=unused-function -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC
-DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64
-D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -DHAVE_CBLAS
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/common
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/umath -Inumpy/core/include
-Ibuild/src.linux-x86_64-3.9/numpy/core/include/numpy
-Ibuild/src.linux-x86_64-3.9/numpy/distutils/include -Inumpy/core/src/common
-Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath
-Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort
-Inumpy/core/src/_simd -I/usr/include/python3.9
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/common
-Ibuild/src.linux-x86_64-3.9/numpy/core/src/npymath -c
build/src.linux-x86_64-3.9/numpy/core/src/multiarray/einsum_sumprod.c -E >
/tmp/preprocessed.c

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

--- Comment #8 from Martin Liška <marxin at gcc dot gnu.org> ---
(In reply to seberg from comment #7)
> Oh, interesting.  You are right, if I run it on the preprocessed file it
> succeeds (note the `tmp/preprocessed.c`:

That's what I was affraid. It's related to function location streaming for
pre-processed files.

Is it difficult to build NumPy from sources? Can you please provide steps?

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[sebastian-gcc at sipsolutions dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

--- Comment #9 from seberg <sebastian-gcc at sipsolutions dot net> ---
NumPy is pretty light weight (at least on linux systems, and if you do not care
about setting up BLAS manually).  You probably only need the Python dev
headers, that I expect you already have.

Hopefully, the following will just work for you:

git clone https://github.com/numpy/numpy
cd numpy
python runtests.py --show-build-log --gcov
# If you got problems due to blas/lapack, try with NPY_BLAS_ORDER=""

which will leave all the temporary files on error and print out the compiler
options.  (There is some spammy output at the very end to scroll past, and the
exact commands are the concatenation of the print out -- but the erroring one
is printed in full after the GCC error.)

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |marxin at gcc dot gnu.org

--- Comment #10 from Martin Liška <marxin at gcc dot gnu.org> ---
Thanks. I was able to reproduce that. It's related to the usage of '#line XYZ'
directives which generate a situation where the function ends before it begins.
I was able to reduce that to something like:

$ cat x.c
void
foo()
{
#line 1
}

$ gcc --coverage x.c
during IPA pass: profile
x.c: In function ‘foo’:
x.c:1:1: internal compiler error: in coverage_begin_function, at coverage.c:662
    1 | void
      | ^
0xa3168b coverage_begin_function(unsigned int, unsigned int)
        /home/marxin/Programming/gcc/gcc/coverage.c:662
0xde935f branch_prob(bool)
        /home/marxin/Programming/gcc/gcc/profile.c:1328
0xf7bd60 tree_profiling
        /home/marxin/Programming/gcc/gcc/tree-profile.c:782
0xf7bd60 execute
        /home/marxin/Programming/gcc/gcc/tree-profile.c:888
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.

We should not ICE, but rather a warning should be emitted.

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[sebastian-gcc at sipsolutions dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

--- Comment #11 from seberg <sebastian-gcc at sipsolutions dot net> ---
Does that mean that fixing the `#line` directives (or inserting additional
ones) should be able to fix the issue?  (Or work around it, if you consider it
a bug.)

I tried to figure out where the `#line` directives currently causes this
situation, and add new ones in some places in the templater.  But I don't
really see where it goes wrong.  That is, unless `#if 0` causes the directives
within the block to be ignored?

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

--- Comment #12 from Martin Liška <marxin at gcc dot gnu.org> ---
(In reply to seberg from comment #11)
> Does that mean that fixing the `#line` directives (or inserting additional
> ones) should be able to fix the issue?  (Or work around it, if you consider
> it a bug.)

Yes. I've got an approved patch for the current master:
https://gcc.gnu.org/pipermail/gcc-patches/2021-June/571593.html

For older releases, I consider relaxation of the assert.

> 
> I tried to figure out where the `#line` directives currently causes this
> situation, and add new ones in some places in the templater.  But I don't
> really see where it goes wrong.  That is, unless `#if 0` causes the
> directives within the block to be ignored?

Well, the problematic function is:

numpy/core/src/multiarray/einsum_sumprod.c.src: In function
‘longdouble_sum_of_products_contig_three’:

So one should see a line directives in the function body that cause it ends
before it begins.

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

--- Comment #13 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Martin Liska <marxin@gcc.gnu.org>:

https://gcc.gnu.org/g:8bf728aecc4fea46b4490e950b9ae229f90597b0

commit r12-1383-g8bf728aecc4fea46b4490e950b9ae229f90597b0
Author: Martin Liska <mliska@suse.cz>
Date:   Tue Jun 1 15:13:18 2021 +0200

    Introduce -Wcoverage-invalid-line-number

            PR gcov-profile/100788

    gcc/ChangeLog:

            * common.opt: Add new option.
            * coverage.c (coverage_begin_function): Emit warning instead on
            the internal compiler error.
            * doc/invoke.texi: Document the option.
            * toplev.c (process_options): Enable it by default.

    gcc/testsuite/ChangeLog:

            * gcc.dg/pr100788.c: New test.

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Known to work|                            |12.0

--- Comment #14 from Martin Liška <marxin at gcc dot gnu.org> ---
Fixed on master so far.

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

--- Comment #15 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Martin Liska
<marxin@gcc.gnu.org>:

https://gcc.gnu.org/g:5f00ef3bc724e22628fdfd81855013115d115ebe

commit r11-8863-g5f00ef3bc724e22628fdfd81855013115d115ebe
Author: Martin Liska <mliska@suse.cz>
Date:   Tue Jun 1 15:13:18 2021 +0200

    coverage: handle function beginning

            PR gcov-profile/100788

    gcc/ChangeLog:

            * coverage.c (coverage_begin_function): Update function
              beginning when #line macro is used.

    gcc/testsuite/ChangeLog:

            * gcc.dg/pr100788.c: New test.

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

--- Comment #16 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-10 branch has been updated by Martin Liska
<marxin@gcc.gnu.org>:

https://gcc.gnu.org/g:ff3978f5ea2672b208830a97d276cc1c856f6231

commit r10-10035-gff3978f5ea2672b208830a97d276cc1c856f6231
Author: Martin Liska <mliska@suse.cz>
Date:   Tue Jun 1 15:13:18 2021 +0200

    coverage: handle function beginning

            PR gcov-profile/100788

    gcc/ChangeLog:

            * coverage.c (coverage_begin_function): Update function
              beginning when #line macro is used.

    gcc/testsuite/ChangeLog:

            * gcc.dg/pr100788.c: New test.

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #17 from Martin Liška <marxin at gcc dot gnu.org> ---
Fixed in gcc-10 and gcc-11 branches. One can't easily apply the patch to
gcc-10, closing as fixed.

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |andre.schackier at gmail dot com

--- Comment #18 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
*** Bug 103651 has been marked as a duplicate of this bug. ***

[Bug gcov-profile/100788] Internal compiler error related to #line macros(?)

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100788

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |10.4
      Known to work|                            |10.3.1, 11.2.1