[PATCH] Optional escaped commas in macros

[PATCH] Optional escaped commas in macros

[Andres Tiraboschi]

This patch is for gcc-5.2.0 adds the XXX option to the preprocessor
that enables to escape
commas when passing macro arguments. This feature is useful in C++ when the
macro argument is a template with more than one argument, and adding extra
( ) is not possible, as shown in this example:

    template <class X, class Y> struct S;

    #define Macro(arg) S arg

    Macro(<int, int>);  // error: passing 2 arguments

Adding extra ( ) as a workaround doesn't work either:

    Macro((<int, int>));  // expands to struct S (<int, int>)

Despite there are workarounds using variadic macros (as already discussed
in the C++' std-proposals mailing list) and using indirect macros (such as
defining a COMMA macro or alike), this patch enables to escape the
comma to prevent it behave as an argument separator, allowing to write
the example above as:

    Macro(<int\, int>);   // only one argument

Other people in the same mailing list proposed to teach the preprocessor to do
tokens balancing, but this approach is both more complicated and fails in the
following examples:

   #define IF(condition1, condition2) if(lower condition1 && condition2 upper)
   IF(<b, c>) return 1;

   #define BRACKET(x)  x] = 1
   BRACKET(a[1);


While we start a discussion with the C committee regarding the standarization
of this feature, we think that this is a useful nonstandard addition to be
early adopted.

I ran all the test and passed.


Thanks,
Andrés.

diff --git a/gcc/c-family/c.opt b/gcc/c-family/c.opt
index 983f4a8..41fa9dd 100644
--- a/gcc/c-family/c.opt
+++ b/gcc/c-family/c.opt
@@ -181,6 +181,10 @@ A
 C ObjC C++ ObjC++ Joined Separate MissingArgError(assertion missing after %qs)
 -A<question>=<answer>    Assert the <answer> to <question>.  Putting
'-' before <question> disables the <answer> to <question>

+fmacro-escaped-commas
+C ObjC C++ ObjC++ CPP(macro_escaped_commas)
Var(cpp_macro_escaped_commas) Init(0)
+Allows using escaped commas in macros arguments
+
 C
 C ObjC C++ ObjC++
 Do not discard comments
diff --git a/gcc/testsuite/gcc.dg/cpp/macro-escaped-commas.c
b/gcc/testsuite/gcc.dg/cpp/macro-escaped-commas.c
new file mode 100644
index 0000000..e56df04
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/cpp/macro-escaped-commas.c
@@ -0,0 +1,11 @@
+/* { dg-do run } */
+/* { dg-options "-fmacro-escaped-commas" } */
+
+#define MACRO_DEFINITION0(a, b) a + b
+#define MACRO_DEFINITION1(a) 0
+int main(void)
+{
+    int a = MACRO_DEFINITION1({1\,2\,3});
+    int b = MACRO_DEFINITION0(MACRO_DEFINITION1({1\,2\,3}), 1);
+    return 0;
+}
diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
index 5e08014..c3d404b 100644
--- a/libcpp/include/cpplib.h
+++ b/libcpp/include/cpplib.h
@@ -481,6 +481,9 @@ struct cpp_options
   /* True if dependencies should be restored from a precompiled header.  */
   bool restore_pch_deps;

+  /* True to use escaped commas in macros*/
+  bool macro_escaped_commas;
+
   /* True if warn about differences between C90 and C99.  */
   signed char cpp_warn_c90_c99_compat;

diff --git a/libcpp/macro.c b/libcpp/macro.c
index 1e0a0b5..f9eba81 100644
--- a/libcpp/macro.c
+++ b/libcpp/macro.c
@@ -811,6 +811,7 @@ collect_args (cpp_reader *pfile, const cpp_hashnode *node,
   source_location virt_loc;
   bool track_macro_expansion_p = CPP_OPTION (pfile, track_macro_expansion);
   unsigned num_args_alloced = 0;
+  const bool escaped_commas_p = CPP_OPTION(pfile, macro_escaped_commas);

   macro = node->value.macro;
   if (macro->paramc)
@@ -835,6 +836,8 @@ collect_args (cpp_reader *pfile, const cpp_hashnode *node,
      few.  Hence the slightly bizarre usage of "argc" and "arg".  */
   do
     {
+      bool prev_backslash_p = false;
+      bool token_taken_p = false;
       unsigned int paren_depth = 0;
       unsigned int ntokens = 0;
       unsigned virt_locs_capacity = DEFAULT_NUM_TOKENS_PER_MACRO_ARG;
@@ -867,8 +870,9 @@ collect_args (cpp_reader *pfile, const cpp_hashnode *node,
                        arg->virt_locs,
                        virt_locs_capacity);
         }
-
-      token = cpp_get_token_1 (pfile, &virt_loc);
+      //If a token wasn't already taken, get a token
+      if (!token_taken_p)
+        token = cpp_get_token_1 (pfile, &virt_loc);

       if (token->type == CPP_PADDING)
         {
@@ -886,9 +890,12 @@ collect_args (cpp_reader *pfile, const cpp_hashnode *node,
       else if (token->type == CPP_COMMA)
         {
           /* A comma does not terminate an argument within
-         parentheses or as part of a variable argument.  */
+         parentheses, as part of a variable argument or
+                 if the macro_escaped_commas flag is on and
+                 if its preceded by a backslash.  */
           if (paren_depth == 0
-          && ! (macro->variadic && argc == macro->paramc))
+          && ! (macro->variadic && argc == macro->paramc)
+                  && ! (prev_backslash_p && escaped_commas_p))
         break;
         }
       else if (token->type == CPP_EOF
@@ -939,10 +946,32 @@ collect_args (cpp_reader *pfile, const cpp_hashnode *node,
           else
         continue;
         }
-      set_arg_token (arg, token, virt_loc,
-             ntokens, MACRO_ARG_TOKEN_NORMAL,
-             CPP_OPTION (pfile, track_macro_expansion));
-      ntokens++;
+      prev_backslash_p = (token->type == CPP_OTHER &&
token->val.str.text[0] == '\\');
+      token_taken_p = escaped_commas_p && prev_backslash_p && paren_depth == 0;
+      /* If we have the macros-escaped-commas flag on, the current
token is a backslash
+      and we are not between parenthesis, get a new token and check
if it's a comma.
+      In that case we are going to ignore the backslash.
+      */
+      if (token_taken_p)
+      {
+        const cpp_token *peek_tok = cpp_get_token_1 (pfile, &virt_loc);
+        //If the next token is a comma, ignore the backslash.
+        if (peek_tok->type != CPP_COMMA)
+        {
+          set_arg_token (arg, token, virt_loc,
+                    ntokens, MACRO_ARG_TOKEN_NORMAL,
+                    CPP_OPTION (pfile, track_macro_expansion));
+          ntokens++;
+        }
+        token = peek_tok;
+      }
+      else
+      {
+        set_arg_token (arg, token, virt_loc,
+                 ntokens, MACRO_ARG_TOKEN_NORMAL,
+                 CPP_OPTION (pfile, track_macro_expansion));
+        ntokens++;
+      }
     }

       /* Drop trailing padding.  */

Re: [PATCH] Optional escaped commas in macros

[Andres Tiraboschi]

2015-10-27 16:35 GMT-03:00 Andres Tiraboschi
<andres.tiraboschi@tallertechnologies.com>:
> This patch is for gcc-5.2.0 adds the XXX option to the preprocessor
Sorry I forgot to clarify that fmacro-escaped-commas is the option name.

Thanks,
Andrés.

Re: [PATCH] Optional escaped commas in macros

[Joseph Myers]

On Tue, 27 Oct 2015, Andres Tiraboschi wrote:

> While we start a discussion with the C committee regarding the standarization
> of this feature, we think that this is a useful nonstandard addition to be
> early adopted.

Are you raising the issue with WG14 in Kona this week?  I'd be wary of 
this sort of extension without positive views from WG14 / WG21.

This patch is, in any case, missing sufficient testcases and 
documentation.  Testcases need to include lots of variations that make 
clear whether the \, must appear literally like that without whitespace, 
or whether there might be space between them, or they might come from 
expansions of separate macros, etc.; formatting of the patch also appears 
messed up.  Testcases also need to include cases that are valid with and 
without the option but with different semantics, to make clear that the 
option and its absence do the right thing in such cases.  Documentation 
should include precise proposed standard wording to make such semantic 
issues clear; that's the level at which this would need to be defined.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [PATCH] Optional escaped commas in macros

[Daniel Gutson]

On Tue, Oct 27, 2015 at 7:30 PM, Joseph Myers <joseph@codesourcery.com> wrote:

Hello Joseph.

> On Tue, 27 Oct 2015, Andres Tiraboschi wrote:
>
>> While we start a discussion with the C committee regarding the standarization
>> of this feature, we think that this is a useful nonstandard addition to be
>> early adopted.
>
> Are you raising the issue with WG14 in Kona this week?  I'd be wary of
> this sort of extension without positive views from WG14 / WG21.

I presented the issue in the WG21 std-proposal mailing list and the general
consensus was that I should ask WG14 first. The problem is that this issue
is C++ only so I'm kind of stuck. Anyway I already asked the convener about
this (David Keaton) and I'm waiting for his answer. I don't know another way
to raise the issue in WG14 since I'm not attending the Kona meeting. Are you a
member of the WG14 committee?
Considering that this is an opt-in feature controlled by a command line flag,
couldn't be just another GNU extension meanwhile? The issue is relevant in C++.

>
> This patch is, in any case, missing sufficient testcases and
> documentation.  Testcases need to include lots of variations that make

OK, agreed. We'll create more test cases and documentation.

> clear whether the \, must appear literally like that without whitespace,

Yes, there should not be whitespace between the \ and the comma.

> or whether there might be space between them, or they might come from
> expansions of separate macros, etc.; formatting of the patch also appears

Something worth to mention is that the \, is only escaped at top-level in terms
of parenthesis, so
     mymacro(function(x \, y))
will not be escaped with this patch, so behavior is not altered in such case.


> messed up.  Testcases also need to include cases that are valid with and
> without the option but with different semantics, to make clear that the
> option and its absence do the right thing in such cases.  Documentation
> should include precise proposed standard wording to make such semantic
> issues clear; that's the level at which this would need to be defined.

Thanks,

   Daniel.

>
> --
> Joseph S. Myers
> joseph@codesourcery.com



-- 

Daniel F. Gutson
Chief Engineering Officer, SPD

San Lorenzo 47, 3rd Floor, Office 5
Córdoba, Argentina

Phone:   +54 351 4217888 / +54 351 4218211
Skype:    dgutson
LinkedIn: http://ar.linkedin.com/in/danielgutson

Re: [PATCH] Optional escaped commas in macros

[Joseph Myers]

On Wed, 28 Oct 2015, Daniel Gutson wrote:

> I presented the issue in the WG21 std-proposal mailing list and the general
> consensus was that I should ask WG14 first. The problem is that this issue
> is C++ only so I'm kind of stuck. Anyway I already asked the convener about
> this (David Keaton) and I'm waiting for his answer. I don't know another way
> to raise the issue in WG14 since I'm not attending the Kona meeting. Are you a
> member of the WG14 committee?

* Write a document explaining the issue, including proposed C standard 
text to define the semantics unambiguously and examples that would be 
added to the standard.

* Obtain a document number from Dan Plakosh, insert it in the document and 
send it to him.  See <http://www.open-std.org/jtc1/sc22/wg14/13410> for 
detailed instructions on submitting documents to WG14.

* It may be helpful to attend the following WG14 meeting where the 
document is discussed.

* Repeat as needed taking account of feedback.  If the initial submission 
is while there isn't an active C standard revision underway, you'll need 
to resubmit when the C standard is open for proposed changes for the next 
revision.

> Considering that this is an opt-in feature controlled by a command line flag,
> couldn't be just another GNU extension meanwhile? The issue is relevant in C++.

We'd still need to be confident we have good semantics that are worth 
supporting long-term.  I don't have that confidence at present.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [PATCH] Optional escaped commas in macros

[Daniel Gutson]

On Wed, Oct 28, 2015 at 11:58 AM, Joseph Myers <joseph@codesourcery.com> wrote:
> On Wed, 28 Oct 2015, Daniel Gutson wrote:
>
>> I presented the issue in the WG21 std-proposal mailing list and the general
>> consensus was that I should ask WG14 first. The problem is that this issue
>> is C++ only so I'm kind of stuck. Anyway I already asked the convener about
>> this (David Keaton) and I'm waiting for his answer. I don't know another way
>> to raise the issue in WG14 since I'm not attending the Kona meeting. Are you a
>> member of the WG14 committee?
>
> * Write a document explaining the issue, including proposed C standard
> text to define the semantics unambiguously and examples that would be
> added to the standard.
>
> * Obtain a document number from Dan Plakosh, insert it in the document and
> send it to him.  See <http://www.open-std.org/jtc1/sc22/wg14/13410> for
> detailed instructions on submitting documents to WG14.
>
> * It may be helpful to attend the following WG14 meeting where the
> document is discussed.
>
> * Repeat as needed taking account of feedback.  If the initial submission
> is while there isn't an active C standard revision underway, you'll need
> to resubmit when the C standard is open for proposed changes for the next
> revision.

OK, thanks for all the information.

>
>> Considering that this is an opt-in feature controlled by a command line flag,
>> couldn't be just another GNU extension meanwhile? The issue is relevant in C++.
>
> We'd still need to be confident we have good semantics that are worth
> supporting long-term.  I don't have that confidence at present.
>
> --
> Joseph S. Myers
> joseph@codesourcery.com



-- 

Daniel F. Gutson
Chief Engineering Officer, SPD

San Lorenzo 47, 3rd Floor, Office 5
Córdoba, Argentina

Phone:   +54 351 4217888 / +54 351 4218211
Skype:    dgutson
LinkedIn: http://ar.linkedin.com/in/danielgutson