Re: preprocessor/4923: Concatenation appears to handle whitespace incorrectly

Re: preprocessor/4923: Concatenation appears to handle whitespace incorrectly

[rodrigc]

The following reply was made to PR preprocessor/4923; it has been noted by GNATS.

From: rodrigc@gcc.gnu.org
To: gcc-bugs@gcc.gnu.org, gcc-gnats@gcc.gnu.org, gcc-prs@gcc.gnu.org,
  nobody@gcc.gnu.org, s0009525@chelt.ac.uk
Cc:  
Subject: Re: preprocessor/4923: Concatenation appears to handle whitespace incorrectly
Date: 22 Nov 2001 03:00:00 -0000

 Synopsis: Concatenation appears to handle whitespace incorrectly
 
 State-Changed-From-To: open->closed
 State-Changed-By: rodrigc
 State-Changed-When: Wed Nov 21 18:59:59 2001
 State-Changed-Why:
     As Zack pointed out, this is correct behavior, and
     gcc 3.0 issues a warning about your code example.
 
 http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&pr=4923&database=gcc

Re: preprocessor/4923: Concatenation appears to handle whitespace incorrectly

['Zack Weinberg']

The following reply was made to PR preprocessor/4923; it has been noted by GNATS.

From: 'Zack Weinberg' <zack@codesourcery.com>
To: "Baum, Nathan I" <s0009525@chelt.ac.uk>
Cc: "'gcc-gnats@gcc.gnu.org'" <gcc-gnats@gcc.gnu.org>
Subject: Re: preprocessor/4923: Concatenation appears to handle whitespace incorrectly
Date: Fri, 23 Nov 2001 15:33:56 -0800

 On Fri, Nov 23, 2001 at 07:32:29PM -0000, Baum, Nathan I wrote:
 > 
 > >This would be a lot of work and frankly I don't see the point.  Show
 > >me real code that desperately needs this functionality and I'll show
 > >you how to fix it so it's standard conforming C.
 > 
 > As far as I tell, I have no use for this feature, myself. Somebody wanted to
 > know how they could do something like:
 > 
 >   #define  CONCAT(a,b) a ## b
 >   #define  STRCAT(a,b) #CONCAT(a,b)
 >   #include STRCAT(PREFIX, foobar.h)
 > 
 > Naturally, this didn't work because of the macros' order of evaluation. On
 > the other hand:
 > 
 >   #define STRCAT(a,b) #a #b
 >   #include STRCAT(PREFIX, foobar.h)
 
 Given the presence of "foobar.h", I suspect someone is trying to glue
 together an #include directive.  And, indeed,
 
 #include "prefix" "foobar.h"
 
 is invalid - C99 6.10.3 contemplates only #include directives which
 (after being completely macro expanded) take the forms
 
 # include "pseudo-string-constant"
 # include <pseudo-string-constant>
 
 (pseudo-string-constant because they don't obey quite the same rules
 as string constants; in particular, escape sequences are not honored.)
 
 There really isn't a way around this - I'd support an extension which
 permitted string constant concatenation in #include but the semantics
 would have to be carefully hammered out.
 
 zw

RE: preprocessor/4923: Concatenation appears to handle whitespace incorrectly

[Baum, Nathan I]

The following reply was made to PR preprocessor/4923; it has been noted by GNATS.

From: "Baum, Nathan I" <s0009525@chelt.ac.uk>
To: 'Zack Weinberg' <zack@codesourcery.com>
Cc: "'gcc-gnats@gcc.gnu.org'" <gcc-gnats@gcc.gnu.org>
Subject: RE: preprocessor/4923: Concatenation appears to handle whitespace
	 incorrectly
Date: Fri, 23 Nov 2001 19:32:29 -0000

 >From: 'Zack Weinberg' [mailto:zack@codesourcery.com]
 >Sent: 23 November 2001 18:51
 >To: Baum, Nathan I
 
 >> Would it break existing programs if ## were to concatenate the two tokens
 >> regardless (in some later version of gcc)? I'd imagine that no sensible
 >> program would rely upon undefined behaviour, so it shouldn't.
 Presumeably,
 >> it'd have to throw the two tokens away and rescan the newly created
 string,
 >> but that doesn't _seem_ like a major problem (I don't know how CPP
 handles
 >> these things internally, so it might be).
 >
 >It does, in fact, concatenate "the two tokens" in the textual output -
 >but the two tokens which get concatenated are ")" and "def".
 >Contrast
 >
 >#define a(x) FOO(abc) ## x
 >#define b(x) FOO(abc) x
 >
 >a(def)	-> FOO(abc)def
 >b(def)	-> FOO(abc) def
 
 Ah-ha. I grok that now. The problem isn't anything to do with ##, it's that
 the macro substitution puts in the whitespace, _after_ ## has done its
 stuff.
 
 >To do what you apparently want, when the ")" was the last character of
 >a macro invocation, it would have to remember that it had been pasted
 >after and apply the paste to the last token of the macro expansion.
 
 It looks like you could elect not to add any whitespace if the last
 character was a ")" (or some other single-character token ("(*/?:%^!")).
 Presumeably, that would never break a program. But, I've just checked the
 C++ specification for the preprocessor, and it appears that would be against
 the rules.
 
 >This would be a lot of work and frankly I don't see the point.  Show
 >me real code that desperately needs this functionality and I'll show
 >you how to fix it so it's standard conforming C.
 
 As far as I tell, I have no use for this feature, myself. Somebody wanted to
 know how they could do something like:
 
   #define  CONCAT(a,b) a ## b
   #define  STRCAT(a,b) #CONCAT(a,b)
   #include STRCAT(PREFIX, foobar.h)
 
 Naturally, this didn't work because of the macros' order of evaluation. On
 the other hand:
 
   #define STRCAT(a,b) #a #b
   #include STRCAT(PREFIX, foobar.h)
 
 I just tested that on gcc, to prove to myself that it wouldn't work, and it
 did. I assume, therefore, that he was using a non-gcc compiler (probably
 VC++, thinking about it), which wasn't happy about the multiple strings.
 
 The Standard says that the behaviour in this instance is undefined (oddly,
 it goes so far as to point out that by this time, strings wouldn't have been
 concatenated, but doesn't bother to specify that they should be in this
 situation.)

Re: preprocessor/4923: Concatenation appears to handle whitespace incorrectly

['Zack Weinberg']

The following reply was made to PR preprocessor/4923; it has been noted by GNATS.

From: 'Zack Weinberg' <zack@codesourcery.com>
To: "Baum, Nathan I" <s0009525@chelt.ac.uk>
Cc: gcc-gnats@gcc.gnu.org
Subject: Re: preprocessor/4923: Concatenation appears to handle whitespace incorrectly
Date: Fri, 23 Nov 2001 10:51:01 -0800

 On Thu, Nov 22, 2001 at 01:47:51PM -0000, Baum, Nathan I wrote:
 > 
 > > At that point the tokens on either side of ## are ")" and "def".  
 > > Pasting them together produces an invalid token,
 > > ")def", which triggers undefined behavior - we choose to pretend the
 > > ## never happened.  Then "FOO ( abc )" gets expanded to produce "abc".
 > > Since "abc" and "def" were not concatenated, cpp has to put a space
 > > between them so they are interpreted sa separate tokens.
 > 
 > Would it break existing programs if ## were to concatenate the two tokens
 > regardless (in some later version of gcc)? I'd imagine that no sensible
 > program would rely upon undefined behaviour, so it shouldn't. Presumeably,
 > it'd have to throw the two tokens away and rescan the newly created string,
 > but that doesn't _seem_ like a major problem (I don't know how CPP handles
 > these things internally, so it might be).
 
 It does, in fact, concatenate "the two tokens" in the textual output -
 but the two tokens which get concatenated are ")" and "def".
 Contrast
 
 #define a(x) FOO(abc) ## x
 #define b(x) FOO(abc) x
 
 a(def)	-> FOO(abc)def
 b(def)	-> FOO(abc) def
 
 To do what you apparently want, when the ")" was the last character of
 a macro invocation, it would have to remember that it had been pasted
 after and apply the paste to the last token of the macro expansion.
 This would be a lot of work and frankly I don't see the point.  Show
 me real code that desperately needs this functionality and I'll show
 you how to fix it so it's standard conforming C.
 
 > > gcc 3.x will warn you when this happens:
 > > test.c:5:1: warning: pasting ")" and "def" does not give a valid
 > preprocessing token
 > 
 > Hmm. Shouldn't the message say 'concatenating' rather than 'pasting'?
 
 The manual uses the two terms interchangeably, and I think the error
 message is already long enough.
 
 zw

RE: preprocessor/4923: Concatenation appears to handle whitespace incorrectly

[Baum, Nathan I]

The following reply was made to PR preprocessor/4923; it has been noted by GNATS.

From: "Baum, Nathan I" <s0009525@chelt.ac.uk>
To: 'Zack Weinberg' <zack@codesourcery.com>
Cc: gcc-gnats@gcc.gnu.org
Subject: RE: preprocessor/4923: Concatenation appears to handle whitespace
	 incorrectly
Date: Thu, 22 Nov 2001 13:47:51 -0000

 > From: Zack Weinberg [mailto:zack@codesourcery.com]
 > This behavior is correct.  The concatenation operator acts before the
 > macro FOO is expanded.  
 
 Ah yes, that makes sense, now that you point it out.
 
 > At that point the tokens on either side of ## are ")" and "def".  
 > Pasting them together produces an invalid token,
 > ")def", which triggers undefined behavior - we choose to pretend the
 > ## never happened.  Then "FOO ( abc )" gets expanded to produce "abc".
 > Since "abc" and "def" were not concatenated, cpp has to put a space
 > between them so they are interpreted sa separate tokens.
 
 Would it break existing programs if ## were to concatenate the two tokens
 regardless (in some later version of gcc)? I'd imagine that no sensible
 program would rely upon undefined behaviour, so it shouldn't. Presumeably,
 it'd have to throw the two tokens away and rescan the newly created string,
 but that doesn't _seem_ like a major problem (I don't know how CPP handles
 these things internally, so it might be). 
 
 > gcc 3.x will warn you when this happens:
 > test.c:5:1: warning: pasting ")" and "def" does not give a valid
 preprocessing token
 
 Hmm. Shouldn't the message say 'concatenating' rather than 'pasting'?

Re: preprocessor/4923: Concatenation appears to handle whitespace incorrectly

[rodrigc]

Synopsis: Concatenation appears to handle whitespace incorrectly

State-Changed-From-To: open->closed
State-Changed-By: rodrigc
State-Changed-When: Wed Nov 21 18:59:59 2001
State-Changed-Why:
    As Zack pointed out, this is correct behavior, and
    gcc 3.0 issues a warning about your code example.

http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&pr=4923&database=gcc

Re: preprocessor/4923: Concatenation appears to handle whitespace incorrectly

[Zack Weinberg]

The following reply was made to PR preprocessor/4923; it has been noted by GNATS.

From: Zack Weinberg <zack@codesourcery.com>
To: s0009525@chelt.ac.uk
Cc: gcc-gnats@gcc.gnu.org
Subject: Re: preprocessor/4923: Concatenation appears to handle whitespace incorrectly
Date: Wed, 21 Nov 2001 09:24:54 -0800

 On Wed, Nov 21, 2001 at 04:30:06PM -0000, s0009525@chelt.ac.uk wrote:
 > #define FOO(x) x
 > #define BAR    FOO(abc) ## def
 > BAR
 > 
 > /* cpp < test.h */
 > 
 > # 1 ""
 >  abc  def
 
 This behavior is correct.  The concatenation operator acts before the
 macro FOO is expanded.  At that point the tokens on either side of ##
 are ")" and "def".  Pasting them together produces an invalid token,
 ")def", which triggers undefined behavior - we choose to pretend the
 ## never happened.  Then "FOO ( abc )" gets expanded to produce "abc".
 Since "abc" and "def" were not concatenated, cpp has to put a space
 between them so they are interpreted sa separate tokens.
 
 gcc 3.x will warn you when this happens:
 
 $ cpp-3.0 test.c
 test.c:5:1: warning: pasting ")" and "def" does not give a valid 
 preprocessing token
 # 5 "test.c"
 abc def
 
 zw

preprocessor/4923: Concatenation appears to handle whitespace incorrectly

[s0009525]

>Number:         4923
>Category:       preprocessor
>Synopsis:       Concatenation appears to handle whitespace incorrectly
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    unassigned
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Nov 21 08:36:01 PST 2001
>Closed-Date:
>Last-Modified:
>Originator:     s0009525@chelt.ac.uk
>Release:        gcc version 2.95.2 20000220
>Organization:
>Environment:
(Debian) Linux 2.4.12-shells1 i686
>Description:
With the ## operator, if one of the arguments is a macro function, the whitespace around the arguments is not eaten, which is contrary to the GNU cpp documentation.
>How-To-Repeat:
/* test.h */

#define FOO(x) x
#define BAR    FOO(abc) ## def
BAR

/* cpp < test.h */

# 1 ""
 abc  def
>Fix:
None Found.
>Release-Note:
>Audit-Trail:
>Unformatted: