[Bug c++/102432] New: [11/12 Regression] ICE in _cpp_lex_direct, at libcpp/lex.c:2949

[Bug c++/102432] New: [11/12 Regression] ICE in _cpp_lex_direct, at libcpp/lex.c:2949

[gscfq@t-online.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102432

            Bug ID: 102432
           Summary: [11/12 Regression] ICE in _cpp_lex_direct, at
                    libcpp/lex.c:2949
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: gscfq@t-online.de
  Target Milestone: ---

Started between 20201018 and 20201108 :


$ cat z1.cc
#define loop(x)
#pragma omp loop


$ cat z2.cc
#define loop(x)
#pragma acc loop


$ g++-12-20210919 -c z1.cc -fopenmp
z1.cc:2:9: internal compiler error: in _cpp_lex_direct, at libcpp/lex.c:2949
    2 | #pragma omp loop
      |         ^~~
0x19c3cf6 _cpp_lex_direct
        ../../libcpp/lex.c:2949
0x19c407d _cpp_lex_token
        ../../libcpp/lex.c:2810
0x19cb04f cpp_get_token_1
        ../../libcpp/macro.c:2898
0x8c18ad c_lex_with_flags(tree_node**, unsigned int*, unsigned char*, int)
        ../../gcc/c-family/c-lex.c:483
0x7b4fb6 cp_lexer_get_preprocessor_token
        ../../gcc/cp/parser.c:864
0x7fa14d cp_lexer_new_main
        ../../gcc/cp/parser.c:663
0x7fa14d c_parse_file()
        ../../gcc/cp/parser.c:47690
0x8cb652 c_common_parse_file()
        ../../gcc/c-family/c-opts.c:1236

[Bug c++/102432] [11/12 Regression] ICE in _cpp_lex_direct, at libcpp/lex.c:2949

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102432

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
I just saw a related bug which is much older and dates from around 4.4 time
frame.

[Bug preprocessor/102432] [11/12 Regression] ICE in _cpp_lex_direct, at libcpp/lex.c:2949

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102432

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |11.3
           See Also|                            |https://gcc.gnu.org/bugzill
                   |                            |a/show_bug.cgi?id=97471

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Andrew Pinski from comment #1)
> I just saw a related bug which is much older and dates from around 4.4 time
> frame.

Actually I am wrong.  Anyways this is related to PR 97471 Which I suspect was
caused by the same patch.

[Bug preprocessor/102432] [11/12 Regression] ICE in _cpp_lex_direct with function like macro without arguments within "pragma omp" statement

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102432

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2021-09-21

--- Comment #3 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Andrew Pinski from comment #2)
> (In reply to Andrew Pinski from comment #1)
> > I just saw a related bug which is much older and dates from around 4.4 time
> > frame.
> 
> Actually I am wrong.  Anyways this is related to PR 97471 Which I suspect
> was caused by the same patch.

Or rather caused by the patch for that bug :).

Confirmed.

[Bug preprocessor/102432] [11/12 Regression] ICE in _cpp_lex_direct with function like macro without arguments within "pragma omp" statement

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102432

--- Comment #4 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Started with r11-4686-g8bd9a00f4349ebcd65223e3dcdfe83867e417287

[Bug preprocessor/102432] [11/12 Regression] ICE in _cpp_lex_direct with function like macro without arguments within "pragma omp" statement

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102432

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |jakub at gcc dot gnu.org

--- Comment #5 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Created attachment 51918
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51918&action=edit
gcc12-pr102432.patch

Untested fix.

[Bug preprocessor/102432] [11/12 Regression] ICE in _cpp_lex_direct with function like macro without arguments within "pragma omp" statement

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102432

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|ice-on-invalid-code         |ice-on-valid-code

--- Comment #6 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
While #c0 testcases were ice-on-invalid, the ones included in the patch are
ice-on-valid-code.

[Bug preprocessor/102432] [11/12 Regression] ICE in _cpp_lex_direct with function like macro without arguments within "pragma omp" statement

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102432

--- Comment #7 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jakub Jelinek <jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:55dfce4d5cb4a366ced7e1194a1c7f04389e3087

commit r12-5788-g55dfce4d5cb4a366ced7e1194a1c7f04389e3087
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Sat Dec 4 11:00:09 2021 +0100

    libcpp: Fix up handling of deferred pragmas [PR102432]

    The https://gcc.gnu.org/pipermail/gcc-patches/2020-November/557903.html
    change broke the following testcases.  The problem is when a pragma
    namespace allows expansion (i.e. p->is_nspace && p->allow_expansion),
    e.g. the omp or acc namespaces do, then when parsing the second pragma
    token we do it with pfile->state.in_directive set,
    pfile->state.prevent_expansion clear and pfile->state.in_deferred_pragma
    clear (the last one because we don't know yet if it will be a deferred
    pragma or not).  If the pragma line only contains a single name
    and newline after it, and there exists a function-like macro with the
    same name, the preprocessor needs to peek in funlike_invocation_p
    the next token whether it isn't ( but in this case it will see a newline.
    As pfile->state.in_directive is set, we don't read anything after the
    newline, pfile->buffer->need_line is set and CPP_EOF is lexed, which
    funlike_invocation_p doesn't push back.  Because name is a function-like
    macro and on the pragma line there is no ( after the name, it isn't
    expanded, and control flow returns to do_pragma.  If name is valid
    deferred pragma, we set pfile->state.in_deferred_pragma (and really
    need it set so that e.g. end_directive later on doesn't eat all the
    tokens from the pragma line).

    Before Nathan's change (which unfortunately didn't contain rationale
    on why it is better to do it like that), this wasn't a problem,
    next _cpp_lex_direct called when we want next token would return
    CPP_PRAGMA_EOF when it saw buffer->need_line, which would turn off
    pfile->state.in_deferred_pragma and following get token would already
    read the next line.  But Nathan's patch replaced it with an assertion
    failure that now triggers and CPP_PRAGMA_EOL is done only when lexing
    the '\n'.  Except for this special case that works fine, but in
    this case it doesn't because when peeking the token we still didn't know
    that it will be a deferred pragma.
    I've tried to fix that up in do_pragma by detecting this and pushing
    CPP_PRAGMA_EOL as lookahead, but that doesn't work because end_directive
    still needs to see pfile->state.in_deferred_pragma set.

    So, this patch affectively reverts part of Nathan's change, CPP_PRAGMA_EOL
    addition isn't done only when parsing the '\n', but is now done in both
    places, in the first one instead of the assertion failure.

    2021-12-04  Jakub Jelinek  <jakub@redhat.com>

            PR preprocessor/102432
            * lex.c (_cpp_lex_direct): If buffer->need_line while
            pfile->state.in_deferred_pragma, return CPP_PRAGMA_EOL token
instead
            of assertion failure.

            * c-c++-common/gomp/pr102432.c: New test.
            * c-c++-common/goacc/pr102432.c: New test.

[Bug preprocessor/102432] [11/12 Regression] ICE in _cpp_lex_direct with function like macro without arguments within "pragma omp" statement

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102432

--- Comment #8 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Jakub Jelinek
<jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:2fdef526a3a8cb4a6f89852979c7ca6437b994f3

commit r11-9354-g2fdef526a3a8cb4a6f89852979c7ca6437b994f3
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Sat Dec 4 11:00:09 2021 +0100

    libcpp: Fix up handling of deferred pragmas [PR102432]

    The https://gcc.gnu.org/pipermail/gcc-patches/2020-November/557903.html
    change broke the following testcases.  The problem is when a pragma
    namespace allows expansion (i.e. p->is_nspace && p->allow_expansion),
    e.g. the omp or acc namespaces do, then when parsing the second pragma
    token we do it with pfile->state.in_directive set,
    pfile->state.prevent_expansion clear and pfile->state.in_deferred_pragma
    clear (the last one because we don't know yet if it will be a deferred
    pragma or not).  If the pragma line only contains a single name
    and newline after it, and there exists a function-like macro with the
    same name, the preprocessor needs to peek in funlike_invocation_p
    the next token whether it isn't ( but in this case it will see a newline.
    As pfile->state.in_directive is set, we don't read anything after the
    newline, pfile->buffer->need_line is set and CPP_EOF is lexed, which
    funlike_invocation_p doesn't push back.  Because name is a function-like
    macro and on the pragma line there is no ( after the name, it isn't
    expanded, and control flow returns to do_pragma.  If name is valid
    deferred pragma, we set pfile->state.in_deferred_pragma (and really
    need it set so that e.g. end_directive later on doesn't eat all the
    tokens from the pragma line).

    Before Nathan's change (which unfortunately didn't contain rationale
    on why it is better to do it like that), this wasn't a problem,
    next _cpp_lex_direct called when we want next token would return
    CPP_PRAGMA_EOF when it saw buffer->need_line, which would turn off
    pfile->state.in_deferred_pragma and following get token would already
    read the next line.  But Nathan's patch replaced it with an assertion
    failure that now triggers and CPP_PRAGMA_EOL is done only when lexing
    the '\n'.  Except for this special case that works fine, but in
    this case it doesn't because when peeking the token we still didn't know
    that it will be a deferred pragma.
    I've tried to fix that up in do_pragma by detecting this and pushing
    CPP_PRAGMA_EOL as lookahead, but that doesn't work because end_directive
    still needs to see pfile->state.in_deferred_pragma set.

    So, this patch affectively reverts part of Nathan's change, CPP_PRAGMA_EOL
    addition isn't done only when parsing the '\n', but is now done in both
    places, in the first one instead of the assertion failure.

    2021-12-04  Jakub Jelinek  <jakub@redhat.com>

            PR preprocessor/102432
            * lex.c (_cpp_lex_direct): If buffer->need_line while
            pfile->state.in_deferred_pragma, return CPP_PRAGMA_EOL token
instead
            of assertion failure.

            * c-c++-common/gomp/pr102432.c: New test.
            * c-c++-common/goacc/pr102432.c: New test.

    (cherry picked from commit 55dfce4d5cb4a366ced7e1194a1c7f04389e3087)

[Bug preprocessor/102432] [11/12 Regression] ICE in _cpp_lex_direct with function like macro without arguments within "pragma omp" statement

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102432

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|ASSIGNED                    |RESOLVED

--- Comment #9 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Fixed.