* [PATCH] preprocessor: Fix tracking of system header state [PR60014,PR60723]
@ 2022-10-08 21:18 Lewis Hyatt
2022-10-12 17:09 ` [PATCH] preprocessor: Fix tracking of system header state [PR60014, PR60723] Jeff Law
0 siblings, 1 reply; 2+ messages in thread
From: Lewis Hyatt @ 2022-10-08 21:18 UTC (permalink / raw)
To: gcc-patches; +Cc: Lewis Hyatt
The token_streamer class (which implements gcc mode -E and
-save-temps/-no-integrated-cpp) needs to keep track whether the last tokens
output were in a system header, so that it can generate line marker
annotations as necessary for a downstream consumer to reconstruct the
state. The logic for tracking it, which was added by r5-1863 to resolve
PR60723, has some edge case issues as revealed by the three new test
cases. The first, coming from the original PR60014, was incidentally fixed by
r9-1926 for unrelated reasons. The other two were still failing on master
prior to this commit. Such code paths were not realizable prior to r13-1544,
which made it possible for the token streamer to see CPP_PRAGMA tokens in more
contexts.
The two main issues being corrected here are:
1) print.prev_was_system_token needs to indicate whether the previous token
output was in a system location. However, it was not being set on every token,
only on those that triggered the main code path; specifically it was not
triggered on a CPP_PRAGMA token. Testcase 2 covers this case.
2) The token_streamer uses a variable "line_marker_emitted" to remember
whether a line marker has been emitted while processing a given token, so that
it wouldn't be done more than once in case multiple conditions requiring a
line marker are true. There was no reason for this to be a member variable
that retains its value from token to token, since it is just needed for
tracking the state locally while processing a single given token. The fact
that it could retain its value for a subsequent token is rather difficult to
observe, but testcase 3 demonstrates incorrect behavior resulting from
that. Moving this to a local variable also simplifies understanding the
control flow going forward.
gcc/c-family/ChangeLog:
PR preprocessor/60014
PR preprocessor/60723
* c-ppoutput.cc (class token_streamer): Remove member
line_marker_emitted to...
(token_streamer::stream): ...a local variable here. Set
print.prev_was_system_token on all code paths.
gcc/testsuite/ChangeLog:
PR preprocessor/60014
PR preprocessor/60723
* gcc.dg/cpp/pr60014-1.c: New test.
* gcc.dg/cpp/pr60014-1.h: New test.
* gcc.dg/cpp/pr60014-2.c: New test.
* gcc.dg/cpp/pr60014-2.h: New test.
* gcc.dg/cpp/pr60014-3.c: New test.
* gcc.dg/cpp/pr60014-3.h: New test.
---
Notes:
Hello-
I tried to describe it all in the commit message, please see also
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60014#c8 for more
details. bootstrap+regtest all languages looks good on x86-64 Linux. Please
let me know if it looks OK? Thanks!
-Lewis
gcc/c-family/c-ppoutput.cc | 17 ++++++++++-------
gcc/testsuite/gcc.dg/cpp/pr60014-1.c | 9 +++++++++
gcc/testsuite/gcc.dg/cpp/pr60014-1.h | 5 +++++
gcc/testsuite/gcc.dg/cpp/pr60014-2.c | 5 +++++
gcc/testsuite/gcc.dg/cpp/pr60014-2.h | 5 +++++
gcc/testsuite/gcc.dg/cpp/pr60014-3.c | 16 ++++++++++++++++
gcc/testsuite/gcc.dg/cpp/pr60014-3.h | 2 ++
7 files changed, 52 insertions(+), 7 deletions(-)
create mode 100644 gcc/testsuite/gcc.dg/cpp/pr60014-1.c
create mode 100644 gcc/testsuite/gcc.dg/cpp/pr60014-1.h
create mode 100644 gcc/testsuite/gcc.dg/cpp/pr60014-2.c
create mode 100644 gcc/testsuite/gcc.dg/cpp/pr60014-2.h
create mode 100644 gcc/testsuite/gcc.dg/cpp/pr60014-3.c
create mode 100644 gcc/testsuite/gcc.dg/cpp/pr60014-3.h
diff --git a/gcc/c-family/c-ppoutput.cc b/gcc/c-family/c-ppoutput.cc
index 98081ccfbb0..a99d9e9c5ca 100644
--- a/gcc/c-family/c-ppoutput.cc
+++ b/gcc/c-family/c-ppoutput.cc
@@ -184,15 +184,13 @@ class token_streamer
bool avoid_paste;
bool do_line_adjustments;
bool in_pragma;
- bool line_marker_emitted;
public:
token_streamer (cpp_reader *pfile)
:avoid_paste (false),
do_line_adjustments (cpp_get_options (pfile)->lang != CLK_ASM
&& !flag_no_line_commands),
- in_pragma (false),
- line_marker_emitted (false)
+ in_pragma (false)
{
gcc_assert (!print.streamer);
print.streamer = this;
@@ -227,7 +225,14 @@ token_streamer::stream (cpp_reader *pfile, const cpp_token *token,
if (token->type == CPP_EOF)
return;
+ /* Keep track when we move into and out of system locations. */
+ const bool is_system_token = in_system_header_at (loc);
+ const bool system_state_changed
+ = (is_system_token != print.prev_was_system_token);
+ print.prev_was_system_token = is_system_token;
+
/* Subtle logic to output a space if and only if necessary. */
+ bool line_marker_emitted = false;
if (avoid_paste)
{
unsigned src_line = LOCATION_LINE (loc);
@@ -301,19 +306,17 @@ token_streamer::stream (cpp_reader *pfile, const cpp_token *token,
if (do_line_adjustments
&& !in_pragma
&& !line_marker_emitted
- && print.prev_was_system_token != !!in_system_header_at (loc)
+ && system_state_changed
&& !is_location_from_builtin_token (loc))
/* The system-ness of this token is different from the one of
the previous token. Let's emit a line change to mark the
new system-ness before we emit the token. */
{
- do_line_change (pfile, token, loc, false);
- print.prev_was_system_token = !!in_system_header_at (loc);
+ line_marker_emitted = do_line_change (pfile, token, loc, false);
}
if (!in_pragma || should_output_pragmas ())
{
cpp_output_token (token, print.outf);
- line_marker_emitted = false;
print.printed = true;
}
}
diff --git a/gcc/testsuite/gcc.dg/cpp/pr60014-1.c b/gcc/testsuite/gcc.dg/cpp/pr60014-1.c
new file mode 100644
index 00000000000..de52b30c161
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/cpp/pr60014-1.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* { dg-options "-save-temps -Wint-conversion" } */
+#include "pr60014-1.h"
+int main ()
+{
+ X(a,
+ b);
+ char *should_warn = 1; /* { dg-warning {-Wint-conversion} } */
+}
diff --git a/gcc/testsuite/gcc.dg/cpp/pr60014-1.h b/gcc/testsuite/gcc.dg/cpp/pr60014-1.h
new file mode 100644
index 00000000000..50c159c44ee
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/cpp/pr60014-1.h
@@ -0,0 +1,5 @@
+#pragma GCC system_header
+
+/* N.B. the semicolon in the macro definition is important, since it produces a
+ second token from this system header on the same line as the __LINE__ token. */
+#define X(a, b) __LINE__;
diff --git a/gcc/testsuite/gcc.dg/cpp/pr60014-2.c b/gcc/testsuite/gcc.dg/cpp/pr60014-2.c
new file mode 100644
index 00000000000..115c9858ec7
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/cpp/pr60014-2.c
@@ -0,0 +1,5 @@
+/* { dg-do compile } */
+/* { dg-options "-save-temps -Wint-conversion" } */
+#include "pr60014-2.h"
+X
+char *should_warn = 1; /* { dg-warning {-Wint-conversion} } */
diff --git a/gcc/testsuite/gcc.dg/cpp/pr60014-2.h b/gcc/testsuite/gcc.dg/cpp/pr60014-2.h
new file mode 100644
index 00000000000..455f1ed2e5b
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/cpp/pr60014-2.h
@@ -0,0 +1,5 @@
+#pragma GCC system_header
+
+/* N.B. the semicolon in the macro definition is important, since it produces a
+ second token from this system header on the same line as the _Pragma. */
+#define X _Pragma("GCC diagnostic push");
diff --git a/gcc/testsuite/gcc.dg/cpp/pr60014-3.c b/gcc/testsuite/gcc.dg/cpp/pr60014-3.c
new file mode 100644
index 00000000000..c4306035f05
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/cpp/pr60014-3.c
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* { dg-options "-save-temps -Wint-conversion" } */
+#include "pr60014-3.h"
+
+/* The line continuation on the next line is what triggers the problem here,
+ because it synchronizes the output line between the input source and the
+ preprocessed output (whereas without the line continuation, the
+ preprocessed output would be off by one line from having output a #pragma
+ on a line by itself). Therefore, the token streamer doesn't have a reason
+ to generate a line marker purely based on the line number. That gives it
+ the chance to consider whether instead it needs to generate a line marker
+ based on a change of the "in-system-header" state, allowing us to test that
+ it comes to the right conclusion, which it did not, prior to this commit to
+ resolve PR60014. */
+P(GCC diagnostic) \
+const char *should_warn = 1; /* { dg-warning {-Wint-conversion} } */
diff --git a/gcc/testsuite/gcc.dg/cpp/pr60014-3.h b/gcc/testsuite/gcc.dg/cpp/pr60014-3.h
new file mode 100644
index 00000000000..aedf038d95f
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/cpp/pr60014-3.h
@@ -0,0 +1,2 @@
+#pragma GCC system_header
+#define P(x) _Pragma(#x)
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [PATCH] preprocessor: Fix tracking of system header state [PR60014, PR60723]
2022-10-08 21:18 [PATCH] preprocessor: Fix tracking of system header state [PR60014,PR60723] Lewis Hyatt
@ 2022-10-12 17:09 ` Jeff Law
0 siblings, 0 replies; 2+ messages in thread
From: Jeff Law @ 2022-10-12 17:09 UTC (permalink / raw)
To: gcc-patches
On 10/8/22 15:18, Lewis Hyatt via Gcc-patches wrote:
> The token_streamer class (which implements gcc mode -E and
> -save-temps/-no-integrated-cpp) needs to keep track whether the last tokens
> output were in a system header, so that it can generate line marker
> annotations as necessary for a downstream consumer to reconstruct the
> state. The logic for tracking it, which was added by r5-1863 to resolve
> PR60723, has some edge case issues as revealed by the three new test
> cases. The first, coming from the original PR60014, was incidentally fixed by
> r9-1926 for unrelated reasons. The other two were still failing on master
> prior to this commit. Such code paths were not realizable prior to r13-1544,
> which made it possible for the token streamer to see CPP_PRAGMA tokens in more
> contexts.
>
> The two main issues being corrected here are:
>
> 1) print.prev_was_system_token needs to indicate whether the previous token
> output was in a system location. However, it was not being set on every token,
> only on those that triggered the main code path; specifically it was not
> triggered on a CPP_PRAGMA token. Testcase 2 covers this case.
>
> 2) The token_streamer uses a variable "line_marker_emitted" to remember
> whether a line marker has been emitted while processing a given token, so that
> it wouldn't be done more than once in case multiple conditions requiring a
> line marker are true. There was no reason for this to be a member variable
> that retains its value from token to token, since it is just needed for
> tracking the state locally while processing a single given token. The fact
> that it could retain its value for a subsequent token is rather difficult to
> observe, but testcase 3 demonstrates incorrect behavior resulting from
> that. Moving this to a local variable also simplifies understanding the
> control flow going forward.
>
> gcc/c-family/ChangeLog:
>
> PR preprocessor/60014
> PR preprocessor/60723
> * c-ppoutput.cc (class token_streamer): Remove member
> line_marker_emitted to...
> (token_streamer::stream): ...a local variable here. Set
> print.prev_was_system_token on all code paths.
>
> gcc/testsuite/ChangeLog:
>
> PR preprocessor/60014
> PR preprocessor/60723
> * gcc.dg/cpp/pr60014-1.c: New test.
> * gcc.dg/cpp/pr60014-1.h: New test.
> * gcc.dg/cpp/pr60014-2.c: New test.
> * gcc.dg/cpp/pr60014-2.h: New test.
> * gcc.dg/cpp/pr60014-3.c: New test.
> * gcc.dg/cpp/pr60014-3.h: New test.
OK
jeff
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2022-10-12 17:09 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-10-08 21:18 [PATCH] preprocessor: Fix tracking of system header state [PR60014,PR60723] Lewis Hyatt
2022-10-12 17:09 ` [PATCH] preprocessor: Fix tracking of system header state [PR60014, PR60723] Jeff Law
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).