[Bug c++/96328] New: Singe keyword "friend" makes GCC ICE in cp_lexer_previous_token, at cp/parser.c:769

[Bug c++/96328] New: Singe keyword "friend" makes GCC ICE in cp_lexer_previous_token, at cp/parser.c:769

[haoxintu at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96328

            Bug ID: 96328
           Summary: Singe keyword "friend" makes GCC ICE in
                    cp_lexer_previous_token, at cp/parser.c:769
           Product: gcc
           Version: 11.0
            Status: UNCONFIRMED
          Keywords: error-recovery, ice-on-invalid-code
          Severity: normal
          Priority: P3
         Component: c++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: haoxintu at gmail dot com
  Target Milestone: ---

Hi, all.

Weirdly, a single "friend" makes GCC crashed in the current trunk.

Input: 
//small.cc
friend

Command:
g++ small.cc

Output:
small.cc:1:1: error: ‘friend’ used outside of class
    1 | friend
      | ^~~~~~
      | ------
small.cc:2: internal compiler error: in cp_lexer_previous_token, at
cp/parser.c:769
0x66759d cp_lexer_previous_token
        ../../gcc/cp/parser.c:769
0x667c28 cp_lexer_previous_token
        ../../gcc/tree.h:3303
0x667c28 cp_parser_error_1
        ../../gcc/cp/parser.c:2943
0x9e04c0 cp_parser_error
        ../../gcc/cp/parser.c:3003
0x9e04c0 cp_parser_error
        ../../gcc/cp/parser.c:3000
0x9e04c0 cp_parser_unqualified_id
        ../../gcc/cp/parser.c:6332
0x9d7cf8 cp_parser_direct_declarator
        ../../gcc/cp/parser.c:22255
0x9d7cf8 cp_parser_declarator
        ../../gcc/cp/parser.c:21137
0x9ed1a6 cp_parser_init_declarator
        ../../gcc/cp/parser.c:20637
0x9cdacc cp_parser_simple_declaration
        ../../gcc/cp/parser.c:13799
0x9f8d2e cp_parser_declaration
        ../../gcc/cp/parser.c:13498
0x9f941b cp_parser_translation_unit
        ../../gcc/cp/parser.c:4781
0x9f941b c_parse_file()
        ../../gcc/cp/parser.c:44069
0xb13e2d c_common_parse_file()
        ../../gcc/c-family/c-opts.c:1188
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.

$g++ -v
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/home/tuhaoxin/compilers/gcc-0726/build/bin/../libexec/gcc/x86_64-pc-linux-gnu/11.0.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: ../configure CC=gcc-10 CXX=g++-10
--prefix=/home/hatuhaoxin/compilers/gcc-0726/build/ --enable-languages=c,c++
--disable-multilib
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 11.0.0 20200726 (experimental) (GCC) 

Interestingly, this case only occurred in the 20200726 trunk version, it does
not crash on 20200725.

Thanks,
Haoxin

[Bug c++/96328] [11 Regression] Single keyword "friend" makes GCC ICE in cp_lexer_previous_token, at cp/parser.c:769 since r11-891-g1dc83b460653c29f

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96328

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Single keyword "friend"     |[11 Regression] Single
                   |makes GCC ICE in            |keyword "friend" makes GCC
                   |cp_lexer_previous_token, at |ICE in
                   |cp/parser.c:769             |cp_lexer_previous_token, at
                   |                            |cp/parser.c:769 since
                   |                            |r11-891-g1dc83b460653c29f
                 CC|                            |mark at gcc dot gnu.org,
                   |                            |marxin at gcc dot gnu.org
   Last reconfirmed|                            |2020-07-27
             Status|UNCONFIRMED                 |NEW
     Ever confirmed|0                           |1
      Known to fail|                            |11.0
      Known to work|                            |10.1.0

--- Comment #1 from Martin Liška <marxin at gcc dot gnu.org> ---
Confirmed, started with r11-891-g1dc83b460653c29f.

[Bug c++/96328] [11 Regression] Single keyword "friend" makes GCC ICE in cp_lexer_previous_token, at cp/parser.c:769 since r11-891-g1dc83b460653c29f

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96328

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |11.0
           Priority|P3                          |P4

[Bug c++/96328] [11 Regression] Single keyword "friend" makes GCC ICE in cp_lexer_previous_token, at cp/parser.c:769 since r11-891-g1dc83b460653c29f

[mark at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96328

Mark Wielaard <mark at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED

--- Comment #2 from Mark Wielaard <mark at gcc dot gnu.org> ---
Only tested on the failing testcase, but this seems to fix it:

$ echo "friend" | /opt/local/install/gcc/bin/g++ -c -xc++ -
<stdin>:1:1: error: ‘friend’ used outside of class
<stdin>:2: error: expected unqualified-id at end of input

diff --git a/gcc/cp/parser.c b/gcc/cp/parser.c
index 0c77c20da862..9c4f6a50523c 100644
--- a/gcc/cp/parser.c
+++ b/gcc/cp/parser.c
@@ -781,9 +781,13 @@ static cp_token *
 cp_lexer_safe_previous_token (cp_lexer *lexer)
 {
   if (lexer->buffer)
-    if (lexer->next_token != lexer->buffer->address ())
-      return cp_lexer_previous_token (lexer);
-
+    {
+      cp_token_position tp = cp_lexer_previous_token_position (lexer);
+      while (tp->purged_p && tp != lexer->buffer->address ())
+       tp--;
+      if (tp != lexer->buffer->address ())
+       return tp;
+    }
   return NULL;
 }

@@ -2932,15 +2936,15 @@ cp_parser_error_1 (cp_parser* parser, const char*
gmsgid,
   gcc_rich_location richloc (input_location);

   bool added_matching_location = false;
+  cp_token *prev_token = cp_lexer_safe_previous_token (parser->lexer);

-  if (missing_token_desc != RT_NONE)
+  if (prev_token && missing_token_desc != RT_NONE)
     {
       /* Potentially supply a fix-it hint, suggesting to add the
         missing token immediately after the *previous* token.
         This may move the primary location within richloc.  */
       enum cpp_ttype ttype = get_required_cpp_ttype (missing_token_desc);
-      location_t prev_token_loc
-       = cp_lexer_previous_token (parser->lexer)->location;
+      location_t prev_token_loc = prev_token->location;
       maybe_suggest_missing_token_insertion (&richloc, ttype, prev_token_loc);

       /* If matching_location != UNKNOWN_LOCATION, highlight it.
@@ -2957,7 +2961,6 @@ cp_parser_error_1 (cp_parser* parser, const char* gmsgid,
      standard string literal constants defined in header files. If
      there is one, then add that as an hint to the error message. */
   name_hint h;
-  cp_token *prev_token = cp_lexer_safe_previous_token (parser->lexer);
   if (prev_token && cp_parser_is_string_literal (prev_token)
       && token->type == CPP_NAME)
     {

[Bug c++/96328] [11 Regression] Single keyword "friend" makes GCC ICE in cp_lexer_previous_token, at cp/parser.c:769 since r11-891-g1dc83b460653c29f

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96328

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #3 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Created attachment 48930
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=48930&action=edit
gcc11-pr96328.patch

I wrote this for it (the first hunk is similar).

[Bug c++/96328] [11 Regression] Single keyword "friend" makes GCC ICE in cp_lexer_previous_token, at cp/parser.c:769 since r11-891-g1dc83b460653c29f

[mark at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96328

--- Comment #4 from Mark Wielaard <mark at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #3)
> Created attachment 48930 [details]
> gcc11-pr96328.patch
> 
> I wrote this for it (the first hunk is similar).

Yours is nicer because it fixes just the specific part that fails (and it
includes an actual test case).

I tried to fix any use of prev_token in cp_parser_error_1. But I cannot tell if
the one under missing_token_desc != RT_NONE can ever trigger. It was just the
most safe fix I could come up with.

[Bug c++/96328] [11 Regression] Single keyword "friend" makes GCC ICE in cp_lexer_previous_token, at cp/parser.c:769 since r11-891-g1dc83b460653c29f

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96328

--- Comment #5 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Created attachment 48931
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=48931&action=edit
gcc11-pr96328-alt.patch

If you want, we could call the safe_previous_token also in the other spot,
while we don't have a known testcase for those cases, it is still just a hint
and thus not really required if something goes wrong and everything before it
is purged.

[Bug c++/96328] [11 Regression] Single keyword "friend" makes GCC ICE in cp_lexer_previous_token, at cp/parser.c:769 since r11-891-g1dc83b460653c29f

[mark at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96328

--- Comment #6 from Mark Wielaard <mark at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #5)
> Created attachment 48931 [details]
> gcc11-pr96328-alt.patch
> 
> If you want, we could call the safe_previous_token also in the other spot,
> while we don't have a known testcase for those cases, it is still just a
> hint and thus not really required if something goes wrong and everything
> before it is purged.

Yes. I think this alt.patch is perfect. Could you please submit it? Thanks.

[Bug c++/96328] [11 Regression] Single keyword "friend" makes GCC ICE in cp_lexer_previous_token, at cp/parser.c:769 since r11-891-g1dc83b460653c29f

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96328

--- Comment #7 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jakub Jelinek <jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:86cb35983f55d6039b99b82ace30d2730fcb1eb1

commit r11-2385-g86cb35983f55d6039b99b82ace30d2730fcb1eb1
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Tue Jul 28 15:41:30 2020 +0200

    c++: Fix up cp_lexer_safe_previous_token [PR96328]

    The following testcase ICEs, because cp_lexer_safe_previous_token calls
    cp_lexer_previous_token and that ICEs, because all tokens in the lexer
    buffer before the current one (CPP_EOF) have been purged.

    cp_lexer_safe_previous_token is used in the context where it is ok if it
    punts, so the patch changes the function so that it doesn't assert there is
    some previous token, but instead returns NULL like in other cases where it
    punts.

    In addition to this, in the last hunk it does a micro-optimization, don't
    call the potentially expensive function if it will not need the result,
    instead check the least expensive condition first.

    And the middle hunk is a similar change from Mark's version of the patch,
    to use the safe variant in there because it is again just about a hint
    and it is better not to provide the hint than to ICE, though we don't have
a
    testcase that would ICE.

    2020-07-28  Jakub Jelinek  <jakub@redhat.com>
                Mark Wielaard  <mark@klomp.org>

            PR c++/96328
            * parser.c (cp_lexer_safe_previous_token): Don't call
            cp_lexer_previous_token, instead inline it by hand and return NULL
            instead of failing assertion if all previous tokens until the first
            one are purged.
            (cp_parser_error_1): Optimize - only call
cp_lexer_safe_previous_token
            if token->type is CPP_NAME.  Use cp_lexer_safe_previous_token
instead
            of cp_lexer_previous_token for the missing_token_desc != RT_NONE
            case too.

            * g++.dg/diagnostic/pr96328.C: New test.

    Co-Authored-By: Mark Wielaard <mark@klomp.org>

[Bug c++/96328] [11 Regression] Single keyword "friend" makes GCC ICE in cp_lexer_previous_token, at cp/parser.c:769 since r11-891-g1dc83b460653c29f

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96328

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #8 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Fixed.