[Bug preprocessor/56825] Preprocessor does not expand macro correctly if it is an argument and argument of a macro contains ##

[Bug preprocessor/56825] Preprocessor does not expand macro correctly if it is an argument and argument of a macro contains ##

[stelek at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56825

--- Comment #1 from Evgeny Televitckiy <stelek at gmail dot com> 2013-04-03 09:05:47 UTC ---
Created attachment 29787
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=29787
Preprocessor result

[Bug preprocessor/56825] New: Preprocessor does not expand macro correctly if it is an argument and argument of a macro contains ##

[stelek at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56825

             Bug #: 56825
           Summary: Preprocessor does not expand macro correctly if it is
                    an argument and argument of a macro contains ##
    Classification: Unclassified
           Product: gcc
           Version: 4.7.2
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: preprocessor
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: stelek@gmail.com


Created attachment 29786
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=29786
Test file

Ok. That is a little bit hard to explain. Consider the following macros:
#define E3N(n, ar...) E3NI(_, ##ar, n, n, n)
#define E3NI(a0, a1, a2, a3, ...) a1, a2, a3

#define USE_E3_ARG_R(n, ar...) (E3N(n, ar))
#define USE_E3_ARG_W(n, ar...) (E3N(n, ##ar))

If we write something like:
 USE_E3_ARG_W(3, 4);
It expands correctly to:
 (4, 3, 3);

But if we use more complicate:
 USE_E3_ARG_R(5, USE_E3_ARG_R(6, 7));
It expands to:
 (USE_E3_ARG_W(9, 0), 8, 8);

While I would expect:
 ((0, 9, 9), 8, 8);

Mind that the following expands correctly:
 USE_E3_ARG_R(5, USE_E3_ARG_R(6, 7));
To:
  ((7, 6, 6), 5, 5);

The problem is probably related to the following paragraph in standart:
"... After the arguments for the invocation of a function-like macro have been
identified, argument substitution takes place. A parameter in the replacement
list, unless preceded by a # or ## preprocessing token or followed by a ##
preprocessing token (see below), is replaced by the corresponding argument
after all macros contained therein have been expanded..."

But IMHO this statement is relevant to the macro itself and should not apply to
the argument of the macro containing ## to remove comma (##ar in our case).

I know that it will not make any point but Eclipse expands all these macros
correctly with internal macro expansion.

Output of the test compilation:
gcc-mp-4.7 test2.c -v --save-temps
Using built-in specs.
COLLECT_GCC=gcc-mp-4.7
COLLECT_LTO_WRAPPER=/opt/local/libexec/gcc/x86_64-apple-darwin12/4.7.2/lto-wrapper
Target: x86_64-apple-darwin12
Configured with: ../gcc-4.7.2/configure --prefix=/opt/local
--build=x86_64-apple-darwin12
--enable-languages=c,c++,objc,obj-c++,lto,fortran,java
--libdir=/opt/local/lib/gcc47 --includedir=/opt/local/include/gcc47
--infodir=/opt/local/share/info --mandir=/opt/local/share/man
--datarootdir=/opt/local/share/gcc-4.7 --with-libiconv-prefix=/opt/local
--with-local-prefix=/opt/local --with-system-zlib --disable-nls
--program-suffix=-mp-4.7 --with-gxx-include-dir=/opt/local/include/gcc47/c++/
--with-gmp=/opt/local --with-mpfr=/opt/local --with-mpc=/opt/local
--with-ppl=/opt/local --with-cloog=/opt/local --enable-cloog-backend=isl
--disable-cloog-version-check --enable-stage1-checking --disable-multilib
--enable-lto --enable-libstdcxx-time --with-as=/opt/local/bin/as
--with-ld=/opt/local/bin/ld --with-ar=/opt/local/bin/ar
--with-bugurl=https://trac.macports.org/newticket --disable-ppl-version-check
--with-pkgversion='MacPorts gcc47 4.7.2_2'
Thread model: posix
gcc version 4.7.2 (MacPorts gcc47 4.7.2_2)
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.8.3' '-v' '-save-temps'
'-mtune=core2'
 /opt/local/libexec/gcc/x86_64-apple-darwin12/4.7.2/cc1 -E -quiet -v
-D__DYNAMIC__ test2.c -fPIC -mmacosx-version-min=10.8.3 -mtune=core2
-fpch-preprocess -o test2.i
ignoring nonexistent directory
"/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/../../../../../x86_64-apple-darwin12/include"
#include "..." search starts here:
#include <...> search starts here:
 /opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/include
 /opt/local/include
 /opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/include-fixed
 /usr/include
 /System/Library/Frameworks
 /Library/Frameworks
End of search list.
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.8.3' '-v' '-save-temps'
'-mtune=core2'
 /opt/local/libexec/gcc/x86_64-apple-darwin12/4.7.2/cc1 -fpreprocessed test2.i
-fPIC -quiet -dumpbase test2.c -mmacosx-version-min=10.8.3 -mtune=core2
-auxbase test2 -version -o test2.s
GNU C (MacPorts gcc47 4.7.2_2) version 4.7.2 (x86_64-apple-darwin12)
    compiled by GNU C version 4.7.2, GMP version 5.0.5, MPFR version 3.1.1-p2,
MPC version 1.0.1
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
GNU C (MacPorts gcc47 4.7.2_2) version 4.7.2 (x86_64-apple-darwin12)
    compiled by GNU C version 4.7.2, GMP version 5.0.5, MPFR version 3.1.1-p2,
MPC version 1.0.1
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 7de89c8f14784e88078e79248a999e9e
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.8.3' '-v' '-save-temps'
'-mtune=core2'
 /opt/local/bin/as -v -arch x86_64 -force_cpusubtype_ALL -o test2.o test2.s
Apple Inc version cctools-836, GNU assembler version 1.38
COMPILER_PATH=/opt/local/libexec/gcc/x86_64-apple-darwin12/4.7.2/:/opt/local/libexec/gcc/x86_64-apple-darwin12/4.7.2/:/opt/local/libexec/gcc/x86_64-apple-darwin12/:/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/:/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/
LIBRARY_PATH=/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/:/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/../../../:/usr/lib/
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.8.3' '-v' '-save-temps'
'-mtune=core2'
 /opt/local/libexec/gcc/x86_64-apple-darwin12/4.7.2/collect2 -dynamic -arch
x86_64 -macosx_version_min 10.8.3 -weak_reference_mismatches non-weak -o a.out
-L/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2
-L/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/../../.. test2.o
-no_compact_unwind -lSystem -lgcc_ext.10.5 -lgcc -lSystem -v
collect2 version 4.7.2
/opt/local/bin/ld -dynamic -arch x86_64 -macosx_version_min 10.8.3
-weak_reference_mismatches non-weak -o a.out
-L/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2
-L/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/../../.. test2.o
-no_compact_unwind -lSystem -lgcc_ext.10.5 -lgcc -lSystem -v
@(#)PROGRAM:ld  PROJECT:ld64-134.9
configured to support archs: i386 x86_64
Library search paths:
    /opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2
    /opt/local/lib/gcc47
    /usr/lib
    /usr/local/lib
Framework search paths:
    /Library/Frameworks/
    /System/Library/Frameworks/
Undefined symbols for architecture x86_64:
  "_USE_E3_ARG_W", referenced from:
      _main in test2.o
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status

Thanks,
Evgeny

[Bug preprocessor/56825] Preprocessor does not expand macro correctly if it is an argument and argument of a macro contains ##

[stelek at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56825

--- Comment #2 from Evgeny Televitckiy <stelek at gmail dot com> 2013-04-03 09:09:22 UTC ---
A small correction.
Lines:
But if we use more complicate:
 USE_E3_ARG_R(5, USE_E3_ARG_R(6, 7));
It expands to:
 (USE_E3_ARG_W(9, 0), 8, 8);

Should be changed to:
But if we use more complicate:
 USE_E3_ARG_W(8, USE_E3_ARG_W(9, 0));
It expands to:
 (USE_E3_ARG_W(9, 0), 8, 8);

[Bug preprocessor/56825] Preprocessor does not expand macro correctly if it is an argument and argument of a macro contains ##

[harald at gigawatt dot nl]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56825

Harald van Dijk <harald at gigawatt dot nl> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |harald at gigawatt dot nl

--- Comment #3 from Harald van Dijk <harald at gigawatt dot nl> 2013-04-03 17:31:26 UTC ---
For the related
  #define foo(x, y) x ## y
both foo(f,oo(,)) and foo(,foo(,)) expand similarly to foo(,) and in that case,
that is the only valid expansion in standard C (C99, anyway). The macro
argument foo(,) is not expanded during argument substitution because it is an
operand of a ## operator. The macro argument is not expanded after argument
substitution because that happens in the context of another expansion of the
same foo macro.

> But IMHO this statement is relevant to the macro itself and should not apply to the argument of the macro

That is exactly what it is meant to apply to.

#define A 1
#define B 2
#define AB 3
#define C(a, b) a ## b
C(A, B)

must expand to 3. Neither A nor B is allowed to be expanded here before
concatenation takes place. AB must be expanded after that, as long as no other
expansion of AB is already taking place.

That said,
  , ## x
is a GNU extension that never concatenates (except for the rare case where x is
an empty macro argument), so the standard cannot and does not require it to
behave exactly the same way as the concatenation operator.

[Bug preprocessor/56825] Preprocessor does not expand macro correctly if it is an argument and argument of a macro contains ##

[stelek at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56825

--- Comment #4 from Evgeny Televitckiy <stelek at gmail dot com> 2013-04-04 06:41:38 UTC ---
Ye, I thought it was something along those lines. Thanks for the clarification,
now I understand the reason why this paragraph appear in standard.
Yet, appealing to the end of your comment. I would expect ,##x construct to be
an exception, especially that it has a totally different meaning then
concatenating # or stringifying ##.
Clearly one would expect:
#define E3N(n, ar...) E3NI(_, ##ar, n, n, n)
#define E3NI(a0, a1, a2, a3, ...) a1, a2, a3

#define USE_E3_ARG_R(n, ar...) (E3N(n, ar))
#define USE_E3_ARG_W(n, ar...) (E3N(n, ##ar))

and used:
 USE_E3_ARG_R(5, USE_E3_ARG_R(6, 7));
 USE_E3_ARG_W(5, USE_E3_ARG_W(6, 7));

To produce the same result...

In addition I wanted to add that it is not some virtual problem. I actually
spent 3 hours of my life in that macro hell, trying to understand what went
wrong with that macro,