From mboxrd@z Thu Jan 1 00:00:00 1970 From: Neil Booth To: Byron Stanoszek Cc: geoffk@cygnus.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org, jakub@redhat.com Subject: Re: warning: pasting would not give a valid preprocessing token Date: Thu, 14 Sep 2000 15:53:00 -0000 Message-id: <20000914235302.A9969@daikokuya.demon.co.uk> References: <20000914185040.A12722@daikokuya.demon.co.uk> X-SW-Source: 2000-09/msg00323.html [There's no point cc-ing Zack; his mail address is no more] Byron, I hope this fixes it for you. I'm bootstrapping now, and will then run the testsuite. This area is a bit ugly and in bad need of a cleanup. I need to extract more flexibility from the lexer first, though. The problem was in the way we handle macro arguments: each argument is a list of pointers to the tokens making up the argument. These tend to point to the original tokens (in this case, the comma in the macro definition in the hash table). That token has a PASTE_LEFT flag to say it's on the left hand side of a ##. The problem is that after processing the paste, and correctly not emitting a warning about invalid pastes, we maintained a pointer to the same comma for the nested macro replacements (since it is part of an argument itself). Unforunately, of course, that comma still has its paste flag set, so the paste mechanism gets re-invoked in later unintended places; hence the (repeated) warnings. This is a nice testcase for quite a deep nested macro bug. I simplified your case to the one below, which I'll commit along with the fix. If you have a chance, please give it a whirl. It also fixes a bug in ON_REST_ARG - the macro was testing the flags of the wrong thing (easy to do since #defines are typeless). Any relief the original fix brought might therefore be undone; I'd be interested in knowing if the warnings it "cured" come back with this patch. OK to commit? Neil. * cpplex.c (ON_REST_ARG): Correct the test. (maybe_paste_with_next): Duplicate a token that fail pasting, and clear its PASTE_LEFT flag, so that nested pasting attempts do not occur. * gcc.dg/cpp/paste10.c: Testcase. Index: cpplex.c =================================================================== RCS file: /cvs/gcc/egcs/gcc/cpplex.c,v retrieving revision 1.95 diff -u -p -r1.95 cpplex.c --- cpplex.c 2000/09/12 03:42:29 1.95 +++ cpplex.c 2000/09/14 22:28:32 @@ -201,9 +201,9 @@ TOKEN_LEN (token) #define IS_ARG_CONTEXT(c) ((c)->flags & CONTEXT_ARG) #define CURRENT_CONTEXT(pfile) ((pfile)->contexts + (pfile)->cur_context) #define ON_REST_ARG(c) \ - (((c)->flags & VAR_ARGS) \ - && ((c)-1)->u.list->tokens[((c)-1)->posn - 1].val.aux \ - == (unsigned int) (((c)-1)->u.list->paramc - 1)) + (((c)->u.list->flags & VAR_ARGS) \ + && (c)->u.list->tokens[(c)->posn - 1].val.aux \ + == (unsigned int) ((c)->u.list->paramc - 1)) #define ASSIGN_FLAGS_AND_POS(d, s) \ do {(d)->flags = (s)->flags & (PREV_WHITE | BOL | PASTE_LEFT); \ @@ -2787,14 +2787,17 @@ maybe_paste_with_next (pfile, token) the special extended semantics (see above). */ if (token->type == CPP_COMMA && IS_ARG_CONTEXT (CURRENT_CONTEXT (pfile)) - && ON_REST_ARG (CURRENT_CONTEXT (pfile))) + && ON_REST_ARG (CURRENT_CONTEXT (pfile) - 1)) /* no warning */; else cpp_warning (pfile, "pasting would not give a valid preprocessing token"); } _cpp_push_token (pfile, second); - return token; + /* A short term hack to safely clear the PASTE_LEFT flag. */ + pasted = duplicate_token (pfile, token); + pasted->flags &= ~PASTE_LEFT; + return pasted; } if (type == CPP_NAME || type == CPP_NUMBER) Index: cmdlne-C.c =================================================================== RCS file: /cvs/gcc/egcs/gcc/testsuite/gcc.dg/cpp/cmdlne-C.c,v retrieving revision 1.2 diff -u -p -r1.2 cmdlne-C.c --- cmdlne-C.c 2000/07/11 13:51:39 1.2 +++ cmdlne-C.c 2000/09/14 22:47:29 @@ -10,7 +10,7 @@ the beginning of a directive turns it into a non-directive. */ #define simple no comments -#/**/define bad_directive /* { dg-error "invalid" } */ + #define/**/obj_like/**/(some)/**/thing/**/ #define fun_like(/**/x/**/,/**/y/**/)/**/ /**/#define not_a_macro Index: paste10.c =================================================================== RCS file: paste10.c diff -N paste10.c --- /dev/null Tue May 5 13:32:27 1998 +++ paste10.c Thu Sep 14 15:47:29 2000 @@ -0,0 +1,14 @@ +/* Copyright (C) 2000 Free Software Foundation, Inc. */ + +/* { dg-do preprocess } */ + +/* This testcase used to produce a bogus "invalid paste" warning, owing + to not clearing a PASTE_LEFT flag. */ + +#define strcpy(src) __strcpy_small (src) + +#define __strcpy_small(src) src + +#define tprintf(format, args...) sprintf(format, ## args) + +strcpy(tprintf("<%s>", test))