[gcc r9-10121] libcpp: Fix up padding handling in funlike_invocation_p [PR104147]

[gcc r9-10121] libcpp: Fix up padding handling in funlike_invocation_p [PR104147]

[Jakub Jelinek]

https://gcc.gnu.org/g:7157e074472435489963cf6961cd4c2d06804f4d

commit r9-10121-g7157e074472435489963cf6961cd4c2d06804f4d
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Tue Feb 1 20:48:03 2022 +0100

    libcpp: Fix up padding handling in funlike_invocation_p [PR104147]
    
    As mentioned in the PR, in some cases we preprocess incorrectly when we
    encounter an identifier which is defined as function-like macro, followed
    by at least 2 CPP_PADDING tokens and then some other identifier.
    On the following testcase, the problem is in the 3rd funlike_invocation_p,
    the tokens are CPP_NAME Y, CPP_PADDING (the pfile->avoid_paste shared token),
    CPP_PADDING (one created with padding_token, val.source is non-NULL and
    val.source->flags & PREV_WHITE is non-zero) and then another CPP_NAME.
    funlike_invocation_p remembers there was a padding token, but remembers the
    first one because of its condition, then the next token is the CPP_NAME,
    which is not CPP_OPEN_PAREN, so the CPP_NAME token is backed up, but as we
    can't easily backup more tokens, it pushes into a new context the padding
    token (the pfile->avoid_paste one).  The net effect is that when Y is not
    defined as fun-like macro, we read Y, avoid_paste, padding_token, Y,
    while if Y is fun-like macro, we read Y, avoid_paste, avoid_paste, Y
    (the second avoid_paste is because that is how we handle end of a context).
    Now, for stringify_arg that is unfortunately a significant difference,
    which handles CPP_PADDING tokens with:
          if (token->type == CPP_PADDING)
            {
              if (source == NULL
                  || (!(source->flags & PREV_WHITE)
                      && token->val.source == NULL))
                source = token->val.source;
              continue;
            }
    and later on
          /* Leading white space?  */
          if (dest - 1 != BUFF_FRONT (pfile->u_buff))
            {
              if (source == NULL)
                source = token;
              if (source->flags & PREV_WHITE)
                *dest++ = ' ';
            }
          source = NULL;
    (and c-ppoutput.cc has similar code).
    So, when Y is not fun-like macro, ' ' is added because padding_token's
    val.source->flags & PREV_WHITE is non-zero, while when it is fun-like
    macro, we don't add ' ' in between, because source is NULL and so
    used from the next token (CPP_NAME Y), which doesn't have PREV_WHITE set.
    
    Now, the funlike_invocation_p condition
           if (padding == NULL
               || (!(padding->flags & PREV_WHITE) && token->val.source == NULL))
            padding = token;
    looks very similar to that in stringify_arg/c-ppoutput.cc, so I assume
    the intent was to prefer do the same thing and pick the right padding.
    But there are significant differences.  Both stringify_arg and c-ppoutput.cc
    don't remember the CPP_PADDING token, but its val.source instead, while
    in funlike_invocation_p we want to remember the padding token that has the
    significant information for stringify_arg/c-ppoutput.cc.
    So, IMHO we want to overwrite padding if:
    1) padding == NULL (remember that there was any padding at all)
    2) padding->val.source == NULL (this matches the source == NULL
       case in stringify_arg)
    3) !(padding->val.source->flags & PREV_WHITE) && token->val.source == NULL
       (this matches the !(source->flags & PREV_WHITE) && token->val.source == NULL
       case in stringify_arg)
    
    2022-02-01  Jakub Jelinek  <jakub@redhat.com>
    
            PR preprocessor/104147
            * macro.c (funlike_invocation_p): For padding prefer a token
            with val.source non-NULL especially if it has PREV_WHITE set
            on val.source->flags.  Add gcc_assert that CPP_PADDING tokens
            don't have PREV_WHITE set in flags.
    
            * c-c++-common/cpp/pr104147.c: New test.
    
    (cherry picked from commit 95ac5635409606386259d2ff21fb61738858ca4a)

Diff:
---
 gcc/testsuite/c-c++-common/cpp/pr104147.c | 27 +++++++++++++++++++++++++++
 libcpp/macro.c                            |  5 ++++-
 2 files changed, 31 insertions(+), 1 deletion(-)

diff --git a/gcc/testsuite/c-c++-common/cpp/pr104147.c b/gcc/testsuite/c-c++-common/cpp/pr104147.c
new file mode 100644
index 00000000000..4ee237ed6ba
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cpp/pr104147.c
@@ -0,0 +1,27 @@
+/* PR preprocessor/104147 */
+/* { dg-do run } */
+
+#define X(x,y) 	x y
+#define STR_(x) #x
+#define STR(x) 	STR_(x)
+const char *str =
+STR(X(Y,Y))
+#define Y()
+STR(X(Y,Y))
+#undef Y
+STR(X(Y,Y))
+#define Y()
+STR(X(Y,Y))
+STR(X(Y,
+Y))
+STR(X(Y
+,Y))
+;
+
+int
+main ()
+{
+  if (__builtin_strcmp (str, "Y YY YY YY YY YY Y") != 0)
+    __builtin_abort ();
+  return 0;
+}
diff --git a/libcpp/macro.c b/libcpp/macro.c
index 8dcc21de10a..5f50438cf4f 100644
--- a/libcpp/macro.c
+++ b/libcpp/macro.c
@@ -1209,8 +1209,11 @@ funlike_invocation_p (cpp_reader *pfile, cpp_hashnode *node,
       token = cpp_get_token (pfile);
       if (token->type != CPP_PADDING)
 	break;
+      gcc_assert ((token->flags & PREV_WHITE) == 0);
       if (padding == NULL
-	  || (!(padding->flags & PREV_WHITE) && token->val.source == NULL))
+	  || padding->val.source == NULL
+	  || (!(padding->val.source->flags & PREV_WHITE)
+	      && token->val.source == NULL))
 	padding = token;
     }