Warning for trigraphs in comment?

Warning for trigraphs in comment?

[Gabriel Dos Reis]

[-- Attachment #1: Type: text/plain, Size: 1552 bytes --]


This 

<quote>

   Second, consider the comment line.  Did you notice that it ends oddly,
   with a "/"?

       // What will the next line do? Increment???????????/
                                                          ^

   Nikolai Smirnov writes:

      "Probably, what's happened in the program is obvious for you but I
      lost a couple of days debugging a big program where I made a
      similar error.  I put a comment line ending with a lot of question
      marks accidentally releasing the 'Shift' key at the end.  The
      result is unexpected trigraph sequence '??/' which was converted to
      '\' (phase 1) which was annihilated with the following '\n' (phase
      2)."  [1]

   The "??/" sequence is converted to '\' which, at the end of a line, is
   a line-splicing directive (surprise!).  In this case, it splices the
   following line "++x;" to the end of the comment line and thus makes
   the increment part of the comment.  The increment is never executed.

   Interestingly, if you look at the Gnu g++ documentation for the
   -Wtrigraphs command-line switch, you will encounter the following
   statement:

      "Warnings are not given for trigraphs within comments, as they do
      not affect the meaning of the program."  [2]

   That may be true most of the time, but here we have a case in point --
   from real-world code, no less -- where this expectation does not hold.

</quote>

is extracted from Herb's GTW #86 -- full message appended below.
I would suggest we warn for trigraphs in comments.

-- Gaby



[-- Attachment #2: Type: message/rfc822, Size: 6408 bytes --]

From: Herb Sutter <hsutter@gotw.ca>
Subject: Guru of the Week #86: Solution
Date: 17 May 2003 17:04:31 -0400
Message-ID: <a8mccvcai5tpq5pogb6hluh0cp970ukrgj@4ax.com>


 -------------------------------------------------------------------
   Guru of the Week problems and solutions are posted regularly on
    news:comp.lang.c++.moderated. For past problems and solutions
      see the GotW archive at www.GotW.ca. (c) 2003 H.P.Sutter
            News archives may keep copies of this article.
 -------------------------------------------------------------------

______________________________________________________________________

GotW #86:   Slight Typos? Graphic Language and Other Curiosities

Difficulty: 5 / 10
______________________________________________________________________



>Answer the following questions without using a compiler.
>
>1. What is the output of the following program on a
>   standards-conforming C++ compiler?
>
>    #include <iostream>
>
>    int main()
>    {
>      int x = 1;
>      for( int i = 0; i < 100; ++i );
>        // What will the next line do? Increment???????????/
>        ++x;
>      std::cout << x;
>    }

Assuming that there is no invisible whitespace at the end of the
comment line, the output is "1".

There are two tricks here, one obvious and one less so.

First, consider the for loop line:

  for( int i = 0; i < 100; ++i );
                                ^

There's a semicolon at the end, a "curiously recurring typo pattern"
that (usually accidentally) makes the body of the for loop just the
empty statement.  Even though the following lines may be indented, and
may even have braces around them, they are not part of the body of the
for loop.  This was a deliberate red herring -- in this case, because
of the next point, it doesn't matter that the for loop never repeats
any statements because there's no increment statement to be repeated
at all (even though there appears to be one).  This brings us to the
second point:

Second, consider the comment line.  Did you notice that it ends oddly,
with a "/"?

    // What will the next line do? Increment???????????/
                                                       ^

Nikolai Smirnov writes:

   "Probably, what's happened in the program is obvious for you but I
   lost a couple of days debugging a big program where I made a
   similar error.  I put a comment line ending with a lot of question
   marks accidentally releasing the 'Shift' key at the end.  The
   result is unexpected trigraph sequence '??/' which was converted to
   '\' (phase 1) which was annihilated with the following '\n' (phase
   2)."  [1]

The "??/" sequence is converted to '\' which, at the end of a line, is
a line-splicing directive (surprise!).  In this case, it splices the
following line "++x;" to the end of the comment line and thus makes
the increment part of the comment.  The increment is never executed.

Interestingly, if you look at the Gnu g++ documentation for the
-Wtrigraphs command-line switch, you will encounter the following
statement:

   "Warnings are not given for trigraphs within comments, as they do
   not affect the meaning of the program."  [2]

That may be true most of the time, but here we have a case in point --
from real-world code, no less -- where this expectation does not hold.


>2. How many distinct errors should be reported when compiling the
>   following code on a conforming C++ compiler?
>
>    struct X {
>      static bool f( int* p )
>      {
>        return p && 0[p] and not p[1:>>p[2];
>      };
>    };
>


The short answer is:  Zero.  This code is perfectly legal and
standards-conforming (whether the author might have wanted it to be or
not).

Let's consider in turn each of the expressions that might be
questionable, and see why they're really okay:

 - 0[p] is legal and is defined to have the same meaning as "p[0]".
   In C (and C++), an expression of the form x[y], where one of x and
   y is a pointer type and the other is an integer value, always means
   *(x+y).  In this case, 0[p] and p[0] have the same meaning because
   they mean *(0+p) and *(p+0), respectively, which comes out to the
   same thing.  For more details, see clause 6.5.2.1 in the C99
   standard [3].

 - and and not are valid keywords that are alternative spellings of &&
   and !, respectively.

 - :> is legal.  It is a digraph for the "]" character, not a smiley
   (smileys are unsupported in the C++ language outside comment
   blocks, which is rather a shame).  This turns the final part of the
   expression into "p[1]>p[2]".

 - The "extra" semicolon is allowed at the end of a function
   declaration.

Of course, it could well be that the colon ":"  was a typo and the
author really meant "p[1]>>p[2]", but even if it was a typo it's still
(unfortunately, in that case) perfectly legal code.


Acknowledgements
----------------

Thanks to Nikolai Smirnov for contributing part of the Example 1 code;
I added the for loop line.


References
----------

[1] N. Smirnov, private communication.

[2] A Google search for "trigraphs within comments" yields this and
several other interesting and/or amusing hits.

[3] ISO/IEC 9899:1999 (E), International Standard, Programming
Languages -- C.


---
Herb Sutter (www.gotw.ca)

Convener, ISO WG21 (C++ standards committee)     (www.gotw.ca/iso)
Contributing editor, C/C++ Users Journal         (www.gotw.ca/cuj)
Visual C++ program manager, Microsoft      (www.gotw.ca/microsoft)

      [ Send an empty e-mail to c++-help@netlab.cs.rpi.edu for info ]
      [ about comp.lang.c++.moderated. First time posters: do this! ]

[-- Attachment #3: Type: text/plain, Size: 53 bytes --]



-- 
Gabriel Dos Reis,	gdr@integrable-solutions.net

Re: Warning for trigraphs in comment?

[Zack Weinberg]

Gabriel Dos Reis <gdr@integrable-solutions.net> writes:

>    Second, consider the comment line.  Did you notice that it ends oddly,
>    with a "/"?
>
>        // What will the next line do? Increment???????????/
>                                                           ^
...
>
> is extracted from Herb's GTW #86 -- full message appended below.
> I would suggest we warn for trigraphs in comments.

This was discussed a couple weeks back.  3.4 now warns about ??/\n in
a comment.  The change is self-contained and could be put into 3.3.1
if people think that's a good idea.

zw

Re: Warning for trigraphs in comment?

[Neil Booth]

Gabriel Dos Reis wrote:-

> is extracted from Herb's GTW #86 -- full message appended below.
> I would suggest we warn for trigraphs in comments.

I changed this particular case a few weeks ago.  In general I think
warning about trigraphs in comments is a bad idea.  This particular
case I was of course always aware of, but considered it obscure.
3.4 documentation and behaviour is already good enough for you I
think.  The patch is not appropriate for 3.3.

The real fix is to remove trigraphs from the standard.

Neil.

Re: Warning for trigraphs in comment?

[Neil Booth]

Zack Weinberg wrote:-

> > is extracted from Herb's GTW #86 -- full message appended below.
> > I would suggest we warn for trigraphs in comments.
> 
> This was discussed a couple weeks back.  3.4 now warns about ??/\n in
> a comment.  The change is self-contained and could be put into 3.3.1
> if people think that's a good idea.

It's part of the complete reorganization of line lexing, so I don't
think it's an easy patch for 3.3.  I'd be happy to be proved wrong
though.

Neil.

Re: Warning for trigraphs in comment?

[Zack Weinberg]

Neil Booth <neil@daikokuya.co.uk> writes:

> Zack Weinberg wrote:-
>
>> > is extracted from Herb's GTW #86 -- full message appended below.
>> > I would suggest we warn for trigraphs in comments.
>> 
>> This was discussed a couple weeks back.  3.4 now warns about ??/\n in
>> a comment.  The change is self-contained and could be put into 3.3.1
>> if people think that's a good idea.
>
> It's part of the complete reorganization of line lexing, so I don't
> think it's an easy patch for 3.3.  I'd be happy to be proved wrong
> though.

My bad, I was thinking only about the patch itself, not its
dependencies.

I don't have time to try to reimplement it over 3.3.1's codebase, but
if someone else wants to take a stab at it I don't think it'd be all
that bad.

zw

Re: Warning for trigraphs in comment?

[Gabriel Dos Reis]

Neil Booth <neil@daikokuya.co.uk> writes:

| Gabriel Dos Reis wrote:-
| 
| > is extracted from Herb's GTW #86 -- full message appended below.
| > I would suggest we warn for trigraphs in comments.
| 
| I changed this particular case a few weeks ago.

Thanks!

Since the discussion was about compilers released for production used,
I just content myself with checking 3.2.3 and 3.3 and found that the
comment and the behaviour were indeed there.

| In general I think warning about trigraphs in comments is a bad idea.

I see you stated a personal opinion but I think we need to weight that
against the fact that we got real world experience and the fact that
the comment is not accurate. 

|  This particular
| case I was of course always aware of, but considered it obscure.
| 3.4 documentation and behaviour is already good enough for you I
| think. 

It is not particularly me.  I was reporting input from discussions
which at first sight looked to me as just an academic exercise and it
turned out that it happened in reality.

| The patch is not appropriate for 3.3.
| 
| The real fix is to remove trigraphs from the standard.

Oh, I'm all for removing trigraphs, digraphs, quadrigraphs and
whatevergraphs -- I've also thought they were mistakes.  But, in fact,
I'm skeptical that  would ever happen. 

Anyway, I'm glad to hear that 3.4 will behave differently.

-- Gaby

Re: Warning for trigraphs in comment?

[Gabriel Dos Reis]

Zack Weinberg <zack@codesourcery.com> writes:

| Gabriel Dos Reis <gdr@integrable-solutions.net> writes:
| 
| >    Second, consider the comment line.  Did you notice that it ends oddly,
| >    with a "/"?
| >
| >        // What will the next line do? Increment???????????/
| >                                                           ^
| ...
| >
| > is extracted from Herb's GTW #86 -- full message appended below.
| > I would suggest we warn for trigraphs in comments.
| 
| This was discussed a couple weeks back.

Sorry, I missed that.

|  3.4 now warns about ??/\n in a comment.

I'm glad to learn that.

Thanks,

-- Gaby

Re: Warning for trigraphs in comment?

[Paul Koning]

>>>>> "Gabriel" == Gabriel Dos Reis <gdr@integrable-solutions.net> writes:
 >> The patch is not appropriate for 3.3.  
 >> The real fix is to remove trigraphs from the standard.

 Gabriel> Oh, I'm all for removing trigraphs, digraphs, quadrigraphs
 Gabriel> and whatevergraphs -- I've also thought they were mistakes.
 Gabriel> But, in fact, I'm skeptical that would ever happen.

I like the notion too.  Trigraphs are an utter misfeature.  

Would it make sense to have the lexer ignore trigraphs (i.e., just
treat ??/ as three characters) unless specifically enabled with
-fenable-trigraphs?   In the unlikely case that anyone actually uses
trigraphs, this would enable them to continue doing so.

   paul

Re: Warning for trigraphs in comment?

[Zack Weinberg]

Paul Koning <pkoning@equallogic.com> writes:

>>>>>> "Gabriel" == Gabriel Dos Reis <gdr@integrable-solutions.net> writes:
>  >> The patch is not appropriate for 3.3.  
>  >> The real fix is to remove trigraphs from the standard.
>
>  Gabriel> Oh, I'm all for removing trigraphs, digraphs, quadrigraphs
>  Gabriel> and whatevergraphs -- I've also thought they were mistakes.
>  Gabriel> But, in fact, I'm skeptical that would ever happen.
>
> I like the notion too.  Trigraphs are an utter misfeature.  
>
> Would it make sense to have the lexer ignore trigraphs (i.e., just
> treat ??/ as three characters) unless specifically enabled with
> -fenable-trigraphs?   In the unlikely case that anyone actually uses
> trigraphs, this would enable them to continue doing so.

GCC has done this for decades.  See the documentation of the
-trigraphs option.

zw

Re: Warning for trigraphs in comment?

[Paul Koning]

>>>>> "Zack" == Zack Weinberg <zack@codesourcery.com> writes:

 Zack> Paul Koning <pkoning@equallogic.com> writes:
 >>>>>>> "Gabriel" == Gabriel Dos Reis <gdr@integrable-solutions.net>
 >>>>>>> writes:
 >> >> The patch is not appropriate for 3.3.  >> The real fix is to
 >> remove trigraphs from the standard.
 >> 
 Gabriel> Oh, I'm all for removing trigraphs, digraphs, quadrigraphs
 Gabriel> and whatevergraphs -- I've also thought they were mistakes.
 Gabriel> But, in fact, I'm skeptical that would ever happen.
 >> I like the notion too.  Trigraphs are an utter misfeature.
 >> 
 >> Would it make sense to have the lexer ignore trigraphs (i.e., just
 >> treat ??/ as three characters) unless specifically enabled with
 >> -fenable-trigraphs?  In the unlikely case that anyone actually
 >> uses trigraphs, this would enable them to continue doing so.

 Zack> GCC has done this for decades.  See the documentation of the
 Zack> -trigraphs option.

Oh.  So it looks like what's needed is to have -ansi NOT imply
-trigraphs.

	paul

Re: Warning for trigraphs in comment?

[Gabriel Dos Reis]

Paul Koning <pkoning@equallogic.com> writes:

| >>>>> "Zack" == Zack Weinberg <zack@codesourcery.com> writes:
| 
|  Zack> Paul Koning <pkoning@equallogic.com> writes:
|  >>>>>>> "Gabriel" == Gabriel Dos Reis <gdr@integrable-solutions.net>
|  >>>>>>> writes:
|  >> >> The patch is not appropriate for 3.3.  >> The real fix is to
|  >> remove trigraphs from the standard.
|  >> 
|  Gabriel> Oh, I'm all for removing trigraphs, digraphs, quadrigraphs
|  Gabriel> and whatevergraphs -- I've also thought they were mistakes.
|  Gabriel> But, in fact, I'm skeptical that would ever happen.
|  >> I like the notion too.  Trigraphs are an utter misfeature.
|  >> 
|  >> Would it make sense to have the lexer ignore trigraphs (i.e., just
|  >> treat ??/ as three characters) unless specifically enabled with
|  >> -fenable-trigraphs?  In the unlikely case that anyone actually
|  >> uses trigraphs, this would enable them to continue doing so.
| 
|  Zack> GCC has done this for decades.  See the documentation of the
|  Zack> -trigraphs option.
| 
| Oh.  So it looks like what's needed is to have -ansi NOT imply
| -trigraphs.

I'm not sure we are talking about the same thing. 

What is happening with current GCC in production use is that GCC
choosed NOT to warn for trigraphs in comment even if -Wtrigraph. 

-- Gaby

Re: Warning for trigraphs in comment?

[Paul Koning]

>>>>> "Gabriel" == Gabriel Dos Reis <gdr@integrable-solutions.net> writes:

 Gabriel> Paul Koning <pkoning@equallogic.com> writes:

 >> Oh.  So it looks like what's needed is to have -ansi NOT imply
 >> -trigraphs.

 Gabriel> I'm not sure we are talking about the same thing.

I was responding to your comment that "the real fix is to remove
trigraphs from the standard".  Indeed it is, but until then, making it
maximally hard to turn them on would be a start.

 Gabriel> What is happening with current GCC in production use is that
 Gabriel> GCC choosed NOT to warn for trigraphs in comment even if
 Gabriel> -Wtrigraph.

I'm guessing that you didn't say -trigraph, but rather that you got it
implicitly because you said -ansi.  That's where my suggestion came
from.

	paul

Re: Warning for trigraphs in comment?

[Zack Weinberg]

Paul Koning <pkoning@equallogic.com> writes:

>  >> Would it make sense to have the lexer ignore trigraphs (i.e., just
>  >> treat ??/ as three characters) unless specifically enabled with
>  >> -fenable-trigraphs?  In the unlikely case that anyone actually
>  >> uses trigraphs, this would enable them to continue doing so.
>
>  Zack> GCC has done this for decades.  See the documentation of the
>  Zack> -trigraphs option.
>
> Oh.  So it looks like what's needed is to have -ansi NOT imply
> -trigraphs.

We can't do that.  It would violate user expectations that -ansi
puts GCC in fully conforming mode.  I should point out here that
GCC's default ignoring of trigraphs is controversial in some circles;
suggest you read through some back issues of comp.std.c.


Gabriel Dos Reis <gdr@integrable-solutions.net> writes:

> What is happening with current GCC in production use is that GCC
> choosed NOT to warn for trigraphs in comment even if -Wtrigraph. 

Correct, because the common case of such trigraphs is

  /* some text (???) */

which obviously cannot affect the meaning of the program.  You're
right that ??/\n can change a comment boundary; Neil and I knew that
this was a potential issue but we decided not to worry about it until
it came up, which has now happened, and we've addressed it.

zw

Re: Warning for trigraphs in comment?

[Joe Buck]

On Mon, May 19, 2003 at 04:44:26PM +0200, Gabriel Dos Reis wrote:
> What is happening with current GCC in production use is that GCC
> choosed NOT to warn for trigraphs in comment even if -Wtrigraph. 

Ideally GCC should not warn about trigraphs in comments, ever, unless that
trigraph can change the meaning of the program.
 

Re: Warning for trigraphs in comment?

[Neil Booth]

Joe Buck wrote:-

> On Mon, May 19, 2003 at 04:44:26PM +0200, Gabriel Dos Reis wrote:
> > What is happening with current GCC in production use is that GCC
> > choosed NOT to warn for trigraphs in comment even if -Wtrigraph. 
> 
> Ideally GCC should not warn about trigraphs in comments, ever, unless that
> trigraph can change the meaning of the program.

That is what 3.4 does.  It's not as easy as you might think to do this
efficiently, trigraphs being a PITA efficiency-wise anyway, never mind
doing long-winded checks to see if something constitutes a change in
the meaning of the program.

Basically 3.4 warns about trigraph escaped newlines in comments, even
in the middle of a big block comment, but nothing else.

Neil.