[Bug c/100547] New: ICE with "-g -O1": Segmentation fault signal terminated program cc1, during RTL pass: expand

[Bug c/100547] New: ICE with "-g -O1": Segmentation fault signal terminated program cc1, during RTL pass: expand

[cnsun at uwaterloo dot ca]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100547

            Bug ID: 100547
           Summary: ICE with "-g -O1": Segmentation fault signal
                    terminated program cc1, during RTL pass: expand
           Product: gcc
           Version: tree-ssa
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: cnsun at uwaterloo dot ca
  Target Milestone: ---

$ gcc-trunk -v
Using built-in specs.
COLLECT_GCC=gcc-trunk
COLLECT_LTO_WRAPPER=/scratch/software/gcc-trunk/libexec/gcc/x86_64-pc-linux-gnu/12.0.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /tmp/tmp.FUbEknNORK-gcc-builder/gcc/configure
--enable-languages=c,c++,lto --enable-checking-yes --enable-multiarch
--prefix=/scratch/software/gcc-trunk --disable-bootstrap
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 12.0.0 20210511 (experimental) [master revision
:7db32cac8:aa891c56f25baac94db004e309d1b6e40b770a95] (GCC)

$ cat mutant.c
typedef __attribute__((vector_size(
    ((((((((((((((((((((((((((((((8 * sizeof(short)) * sizeof(short)) *
                                sizeof(short)) *
                               sizeof(short)) *
                              sizeof(short)) *
                             sizeof(short)) *
                            sizeof(short)) *
                           sizeof(short)) *
                          sizeof(short)) *
                         sizeof(short)) *
                        sizeof(short)) *
                       sizeof(short)) *
                      sizeof(short)) *
                     sizeof(short)) *
                    sizeof(short)) *
                   sizeof(short)) *
                  sizeof(short)) *
                 sizeof(short)) *
                sizeof(short)) *
               sizeof(short)) *
              sizeof(short)) *
             sizeof(short)) *
            sizeof(short)) *
           sizeof(short)) *
          sizeof(short)) *
         sizeof(short)) *
        sizeof(short)) *
       sizeof(short)) *
      sizeof(short)) *
     sizeof(short)) *
    sizeof(short)))) V;
k() { V w = {}; }

$ gcc-trunk -O1 -g mutant.c
mutant.c:31:22: warning: type defaults to ‘int’ in declaration of ‘V’
[-Wimplicit-int]
   31 |     sizeof(short)))) V;
      |                      ^
mutant.c:32:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
   32 | k() { V w = {}; }
      | ^
during RTL pass: expand
gcc-trunk: internal compiler error: Segmentation fault signal terminated
program cc1
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://gcc.gnu.org/bugs/> for instructions.

[Bug c/100547] ICE with "-g -O1": stack overflow in expand with a very large vector_size attribute

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100547

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2021-05-12
           Keywords|                            |accepts-invalid
     Ever confirmed|0                           |1
          Component|middle-end                  |c

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> ---
The issue is that VECTOR_CST_NELTS does not fit 32bits but RTX vectors have int
sizes.

I suggest to limit the vector_size attribute argument accordingly.

Also a better failure mode might be to make the rtvec_alloc size argument
size_t and ICE on a size that doesn't fit an int rather than silently
truncating.

[Bug c/100547] ICE with "-g -O1": stack overflow in expand with a very large vector_size attribute

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100547

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |rguenth at gcc dot gnu.org
             Status|NEW                         |ASSIGNED

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
Testing patch.

[Bug c/100547] ICE with "-g -O1": stack overflow in expand with a very large vector_size attribute

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100547

--- Comment #3 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Richard Biener <rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:229a6dbd9ef1680f1ca02d6ce63e8abdffaaeeba

commit r12-737-g229a6dbd9ef1680f1ca02d6ce63e8abdffaaeeba
Author: Richard Biener <rguenther@suse.de>
Date:   Wed May 12 09:07:42 2021 +0200

    middle-end/100547 - check rtvec_alloc size

    This makes the rtvec_alloc argument size_t catching overflow and
    truncated arguments (from "invalid" testcases), verifying the
    argument against INT_MAX which is the limit set by the int
    typed rtvec_def.num_elem member.

    2021-05-12  Richard Biener  <rguenther@suse.de>

            PR middle-end/100547
            * rtl.h (rtvec_alloc): Make argument size_t.
            * rtl.c (rtvec_alloc): Verify the count is less than INT_MAX.

[Bug c/100547] ICE with "-g -O1": stack overflow in expand with a very large vector_size attribute

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100547

--- Comment #4 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Richard Biener <rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:4054472b3fa15e11ccd48190f5e3ecfc89d65af9

commit r12-859-g4054472b3fa15e11ccd48190f5e3ecfc89d65af9
Author: Richard Biener <rguenther@suse.de>
Date:   Wed May 12 09:20:17 2021 +0200

    c/100547 - reject overly large vector_size attributes

    This rejects a number of vector components that does not fit an 'int'
    which is an internal limitation of RTVEC.  This requires adjusting
    gcc.dg/attr-vector_size.c which checks for much larger
    supported vectors.  Note that the RTVEC limitation is a host specific
    limitation (unless we change this 'int' to int32_t), but should be
    32bits in practice everywhere.

    2021-05-12  Richard Biener  <rguenther@suse.de>

            PR c/100547
    gcc/c-family/
            * c-attribs.c (type_valid_for_vector_size): Reject too large
nunits.
            Reword existing nunit diagnostic.

    gcc/testsuite/
            * gcc.dg/pr100547.c: New testcase.
            * gcc.dg/attr-vector_size.c: Adjust.

[Bug c/100547] ICE with "-g -O1": stack overflow in expand with a very large vector_size attribute

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100547

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #5 from Richard Biener <rguenth at gcc dot gnu.org> ---
Fixed for GCC 12.