[pushed] c++: input_location and lookahead [PR104025]

[pushed] c++: input_location and lookahead [PR104025]

[Jason Merrill]

Debug information was getting confused because input_location was different
depending on whether we had looked ahead to see if the next tokens look like
a template argument list.

I tried resetting input_location in cp_lexer_rollback_tokens itself, but
that caused regressions, so let's just do it here for now.

Tested x86_64-pc-linux-gnu, applying to trunk.

	PR c++/104025

gcc/cp/ChangeLog:

	* parser.cc (saved_token_sentinel::rollback): Call
	cp_lexer_set_source_position.
	(~saved_token_sentinel): Call rollback.

gcc/testsuite/ChangeLog:

	* g++.dg/warn/pr104025.C: New test.

Co-authored-by: Jakub Jelinek  <jakub@redhat.com>
---
 gcc/cp/parser.cc                     |  3 ++-
 gcc/testsuite/g++.dg/warn/pr104025.C | 20 ++++++++++++++++++++
 2 files changed, 22 insertions(+), 1 deletion(-)
 create mode 100644 gcc/testsuite/g++.dg/warn/pr104025.C

diff --git a/gcc/cp/parser.cc b/gcc/cp/parser.cc
index b262b765a9a..60d9f7bb723 100644
--- a/gcc/cp/parser.cc
+++ b/gcc/cp/parser.cc
@@ -1308,6 +1308,7 @@ struct saved_token_sentinel
   void rollback ()
   {
     cp_lexer_rollback_tokens (lexer);
+    cp_lexer_set_source_position (lexer);
     mode = STS_DONOTHING;
   }
   ~saved_token_sentinel ()
@@ -1315,7 +1316,7 @@ struct saved_token_sentinel
     if (mode == STS_COMMIT)
       cp_lexer_commit_tokens (lexer);
     else if (mode == STS_ROLLBACK)
-      cp_lexer_rollback_tokens (lexer);
+      rollback ();
 
     gcc_assert (lexer->saved_tokens.length () == len);
   }
diff --git a/gcc/testsuite/g++.dg/warn/pr104025.C b/gcc/testsuite/g++.dg/warn/pr104025.C
new file mode 100644
index 00000000000..9457c8ef52c
--- /dev/null
+++ b/gcc/testsuite/g++.dg/warn/pr104025.C
@@ -0,0 +1,20 @@
+// PR c++/104025
+// { dg-do compile }
+// { dg-options "-Wmissing-template-keyword -fcompare-debug" }
+
+void bar (int);
+
+struct S { int i; };
+
+template <class C>
+struct T
+{
+  int m;
+  C c;
+  void foo ()
+  {
+    bar (c.i < m);
+  }
+};
+
+template void T<S>::foo ();

base-commit: 7ca21601704c4a637f3cefa7c8814920782354d8
-- 
2.27.0

Re: [pushed] c++: input_location and lookahead [PR104025]

[Jakub Jelinek]

On Tue, Jan 18, 2022 at 05:56:47PM -0500, Jason Merrill wrote:
> Debug information was getting confused because input_location was different
> depending on whether we had looked ahead to see if the next tokens look like
> a template argument list.
> 
> I tried resetting input_location in cp_lexer_rollback_tokens itself, but
> that caused regressions, so let's just do it here for now.
> 
> Tested x86_64-pc-linux-gnu, applying to trunk.
> 
> 	PR c++/104025
> 
> gcc/cp/ChangeLog:
> 
> 	* parser.cc (saved_token_sentinel::rollback): Call
> 	cp_lexer_set_source_position.
> 	(~saved_token_sentinel): Call rollback.
> 
> gcc/testsuite/ChangeLog:
> 
> 	* g++.dg/warn/pr104025.C: New test.

The testcase fails for me with this change.
I see:
Breakpoint 5, cp_parser_id_expression (parser=0x7fffea26e7b8, template_keyword_p=false, check_dependency_p=true, template_p=0x7fffffffaf5f, declarator_p=false, optional_p=false)
    at ../../gcc/cp/parser.cc:6257
6257	      saved_token_sentinel toks (parser->lexer, STS_ROLLBACK);
(gdb) p expand_location (input_location)
$6 = {file = 0x4162bb0 "pr104025.C", line = 16, column = 12, data = 0x0, sysp = false}
- input_location is when entering this code conditional on the warning
set to the last committed token, i.e. i in
    bar (c.i < m);
Next it commits the < and m tokens (so up to column 16) and then fails
and invokes rollback and cp_lexer_set_source_position.  But that sets
input_location from the peeked token, i.e. the < one with column 14.
So, with -w, we get 16:12, without -w we get 16:14.

> --- a/gcc/cp/parser.cc
> +++ b/gcc/cp/parser.cc
> @@ -1308,6 +1308,7 @@ struct saved_token_sentinel
>    void rollback ()
>    {
>      cp_lexer_rollback_tokens (lexer);
> +    cp_lexer_set_source_position (lexer);
>      mode = STS_DONOTHING;
>    }
>    ~saved_token_sentinel ()
> @@ -1315,7 +1316,7 @@ struct saved_token_sentinel
>      if (mode == STS_COMMIT)
>        cp_lexer_commit_tokens (lexer);
>      else if (mode == STS_ROLLBACK)
> -      cp_lexer_rollback_tokens (lexer);
> +      rollback ();
>  
>      gcc_assert (lexer->saved_tokens.length () == len);
>    }
> diff --git a/gcc/testsuite/g++.dg/warn/pr104025.C b/gcc/testsuite/g++.dg/warn/pr104025.C
> new file mode 100644
> index 00000000000..9457c8ef52c
> --- /dev/null
> +++ b/gcc/testsuite/g++.dg/warn/pr104025.C
> @@ -0,0 +1,20 @@
> +// PR c++/104025
> +// { dg-do compile }
> +// { dg-options "-Wmissing-template-keyword -fcompare-debug" }
> +
> +void bar (int);
> +
> +struct S { int i; };
> +
> +template <class C>
> +struct T
> +{
> +  int m;
> +  C c;
> +  void foo ()
> +  {
> +    bar (c.i < m);
> +  }
> +};
> +
> +template void T<S>::foo ();
> 
> base-commit: 7ca21601704c4a637f3cefa7c8814920782354d8

	Jakub

Re: [pushed] c++: input_location and lookahead [PR104025]

[Jason Merrill]

[-- Attachment #1: Type: text/plain, Size: 819 bytes --]

On 1/18/22 18:20, Jakub Jelinek wrote:
> On Tue, Jan 18, 2022 at 05:56:47PM -0500, Jason Merrill wrote:
>> Debug information was getting confused because input_location was different
>> depending on whether we had looked ahead to see if the next tokens look like
>> a template argument list.
>>
>> I tried resetting input_location in cp_lexer_rollback_tokens itself, but
>> that caused regressions, so let's just do it here for now.
>>
>> Tested x86_64-pc-linux-gnu, applying to trunk.
>>
>> 	PR c++/104025
>>
>> gcc/cp/ChangeLog:
>>
>> 	* parser.cc (saved_token_sentinel::rollback): Call
>> 	cp_lexer_set_source_position.
>> 	(~saved_token_sentinel): Call rollback.
>>
>> gcc/testsuite/ChangeLog:
>>
>> 	* g++.dg/warn/pr104025.C: New test.
> 
> The testcase fails for me with this change.

Oops, thanks.  Fixed thus:


[-- Attachment #2: 0001-c-fix-PR104025-change.patch --]
[-- Type: text/x-patch, Size: 991 bytes --]

From bd0ef3534816a1b8ffad544d9ed720690a97d7cc Mon Sep 17 00:00:00 2001
From: Jason Merrill <jason@redhat.com>
Date: Tue, 18 Jan 2022 22:11:56 -0500
Subject: [PATCH] c++: fix PR104025 change
To: gcc-patches@gcc.gnu.org

Somehow I pushed my earlier patch without it actually fixing the test; we
need input_location to be for the last consumed token, not the next one.

gcc/cp/ChangeLog:

	* parser.cc (saved_token_sentinel::rollback): Use
	cp_lexer_previous_token.
---
 gcc/cp/parser.cc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/gcc/cp/parser.cc b/gcc/cp/parser.cc
index 60d9f7bb723..00279c43404 100644
--- a/gcc/cp/parser.cc
+++ b/gcc/cp/parser.cc
@@ -1308,7 +1308,8 @@ struct saved_token_sentinel
   void rollback ()
   {
     cp_lexer_rollback_tokens (lexer);
-    cp_lexer_set_source_position (lexer);
+    cp_lexer_set_source_position_from_token
+      (cp_lexer_previous_token (lexer));
     mode = STS_DONOTHING;
   }
   ~saved_token_sentinel ()
-- 
2.27.0