[PATCH] libcpp: Ignore CPP_PADDING tokens in _cpp_parse_expr [PR105732]

[PATCH] libcpp: Ignore CPP_PADDING tokens in _cpp_parse_expr [PR105732]

[Jakub Jelinek]

Hi!

The first part of the following testcase (m1-m3 macros and its use)
regressed with my PR89971 fix, but as the m1,m4-m5 and its use part shows,
the problem isn't new, we can emit a CPP_PADDING token to avoid it from
being adjacent to whatever comes after the __VA_OPT__ (in this case there
is nothing afterwards, true).

In most cases these CPP_PADDING tokens don't matter, all other
callers of cpp_get_token_with_location either ignore CPP_PADDING tokens
completely (e.g. c_lex_with_flags) or they just remember them and
take them into account when printing stuff whether there should be
added whitespace or not (scan_translation_unit + token_streamer::stream).
So, I think we should just ignore CPP_PADDING tokens the same way in
_cpp_parse_expr.

Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk?

2022-05-27  Jakub Jelinek  <jakub@redhat.com>

	PR preprocessor/105732
	* expr.cc (_cpp_parse_expr): Handle CPP_PADDING by just another
	token.

	* c-c++-common/cpp/va-opt-10.c: New test.

--- libcpp/expr.cc.jj	2022-01-18 11:59:00.258972399 +0100
+++ libcpp/expr.cc	2022-05-26 15:39:54.348780446 +0200
@@ -1366,6 +1366,10 @@ _cpp_parse_expr (cpp_reader *pfile, bool
 	    op.op = CPP_UMINUS;
 	  break;
 
+	case CPP_PADDING:
+	  lex_count--;
+	  continue;
+
 	default:
 	  if ((int) op.op <= (int) CPP_EQ || (int) op.op >= (int) CPP_PLUS_EQ)
 	    SYNTAX_ERROR2_AT (op.loc,
--- gcc/testsuite/c-c++-common/cpp/va-opt-10.c.jj	2022-05-26 15:54:40.279766330 +0200
+++ gcc/testsuite/c-c++-common/cpp/va-opt-10.c	2022-05-26 15:54:24.028928687 +0200
@@ -0,0 +1,18 @@
+/* PR preprocessor/105732 */
+/* { dg-do compile } */
+/* { dg-options "-std=gnu99" { target c } } */
+/* { dg-options "-std=c++20" { target c++ } } */
+
+#define m1(p1, p2, p3) p3
+#define m2(p1, ...) 1##__VA_OPT__(foo)
+#define m3(...) m1(1, 2, m2)
+#define m4(p1, ...) 1 __VA_OPT__()
+#define m5(...) m1(1, 2, m4)
+#if m3(,)(,)
+#else
+#error
+#endif
+#if m5(,)(,)
+#else
+#error
+#endif

	Jakub

Re: [PATCH] libcpp: Ignore CPP_PADDING tokens in _cpp_parse_expr [PR105732]

[Jason Merrill]

On 5/27/22 04:16, Jakub Jelinek wrote:
> Hi!
> 
> The first part of the following testcase (m1-m3 macros and its use)
> regressed with my PR89971 fix, but as the m1,m4-m5 and its use part shows,
> the problem isn't new, we can emit a CPP_PADDING token to avoid it from
> being adjacent to whatever comes after the __VA_OPT__ (in this case there
> is nothing afterwards, true).
> 
> In most cases these CPP_PADDING tokens don't matter, all other
> callers of cpp_get_token_with_location either ignore CPP_PADDING tokens
> completely (e.g. c_lex_with_flags) or they just remember them and
> take them into account when printing stuff whether there should be
> added whitespace or not (scan_translation_unit + token_streamer::stream).
> So, I think we should just ignore CPP_PADDING tokens the same way in
> _cpp_parse_expr.
> 
> Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk?

OK.

> 2022-05-27  Jakub Jelinek  <jakub@redhat.com>
> 
> 	PR preprocessor/105732
> 	* expr.cc (_cpp_parse_expr): Handle CPP_PADDING by just another
> 	token.
> 
> 	* c-c++-common/cpp/va-opt-10.c: New test.
> 
> --- libcpp/expr.cc.jj	2022-01-18 11:59:00.258972399 +0100
> +++ libcpp/expr.cc	2022-05-26 15:39:54.348780446 +0200
> @@ -1366,6 +1366,10 @@ _cpp_parse_expr (cpp_reader *pfile, bool
>   	    op.op = CPP_UMINUS;
>   	  break;
>   
> +	case CPP_PADDING:
> +	  lex_count--;
> +	  continue;
> +
>   	default:
>   	  if ((int) op.op <= (int) CPP_EQ || (int) op.op >= (int) CPP_PLUS_EQ)
>   	    SYNTAX_ERROR2_AT (op.loc,
> --- gcc/testsuite/c-c++-common/cpp/va-opt-10.c.jj	2022-05-26 15:54:40.279766330 +0200
> +++ gcc/testsuite/c-c++-common/cpp/va-opt-10.c	2022-05-26 15:54:24.028928687 +0200
> @@ -0,0 +1,18 @@
> +/* PR preprocessor/105732 */
> +/* { dg-do compile } */
> +/* { dg-options "-std=gnu99" { target c } } */
> +/* { dg-options "-std=c++20" { target c++ } } */
> +
> +#define m1(p1, p2, p3) p3
> +#define m2(p1, ...) 1##__VA_OPT__(foo)
> +#define m3(...) m1(1, 2, m2)
> +#define m4(p1, ...) 1 __VA_OPT__()
> +#define m5(...) m1(1, 2, m4)
> +#if m3(,)(,)
> +#else
> +#error
> +#endif
> +#if m5(,)(,)
> +#else
> +#error
> +#endif
> 
> 	Jakub
>