[Bug preprocessor/8270] [4.3/4.4/4.5/4.6/4.7 Regression] back-slash white space newline with comments, no warning

[Bug preprocessor/8270] [4.3/4.4/4.5/4.6/4.7 Regression] back-slash white space newline with comments, no warning

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Richard Guenther <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.3.6                       |4.4.7

--- Comment #44 from Richard Guenther <rguenth at gcc dot gnu.org> 2011-06-27 12:11:52 UTC ---
4.3 branch is being closed, moving to 4.4.7 target.

[Bug preprocessor/8270] [4.5/4.6/4.7/4.8 Regression] back-slash white space newline with comments, no warning

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.4.7                       |4.5.4

--- Comment #45 from Jakub Jelinek <jakub at gcc dot gnu.org> 2012-03-13 12:44:39 UTC ---
4.4 branch is being closed, moving to 4.5.4 target.

[Bug preprocessor/8270] [4.6/4.7/4.8 Regression] back-slash white space newline with comments, no warning

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Richard Guenther <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.5.4                       |4.6.4

[Bug preprocessor/8270] [4.7/4.8/4.9 Regression] back-slash white space newline with comments, no warning

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.6.4                       |4.7.4

--- Comment #46 from Jakub Jelinek <jakub at gcc dot gnu.org> 2013-04-12 15:15:22 UTC ---
GCC 4.6.4 has been released and the branch has been closed.

[Bug preprocessor/8270] [4.7/4.8/4.9 Regression] back-slash white space newline with comments, no warning

[GoWhoopee at yahoo dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

GoWhoopee at yahoo dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |GoWhoopee at yahoo dot com

--- Comment #47 from GoWhoopee at yahoo dot com ---
There is a rule: single line comments are extended by backslash newline.
Ludicrous as it is, this rule is not optional.

Failure to observe this rule by a compiler is criminal: the result is that
lines of code which a correctly written syntax highlighter shows as code that
will be compiled, are not.
This has cost this company several days in time. We write code for the
military: the cost could have been far worse had the unusual behaviour not been
noticed.
Compilers MUST NOT randomly ignore lines of code!

Programmers frequently illustrate code with ASCII art and backslash is a
valuable ASCII art character. To ensure the single-line comment is not extended
and the clarity of the ASCII art remains, a space would be inserted after any
trailing backslash.
ASCII art (or anything attempt at explanation of the code) in comment is NOT
bad coding style (as stated by Atmel technical support).
Any editor or compiler operation that trims spaces from lines MUST NOT trim
spaces all the way back to a backslash because that will change the programmers
intent by creating backslash newline where there was none!
That should be obvious!

GCC isn't alone in missing this crucial point.
Atmel's version of MS Visual Studio offers a
[Tools][Options][Environment][Custom Settings][Remove whitespaces trailing end
of line, while saving the document] which does what it says, silently breaking
code as it does so.
Their tech support said their environment did what it said on the tin and they
couldn't impose themselves on the GCC community, but they could write to the
syntax highlight software company and ask them to break their code too!

Oh, how we laughed...

The solution should be simple: when trimming white-space from the right of a
line of code, don't create backslash newline where it didn't exist before.

Please reconsider and stop gcc from changing our code without our permission.

[Bug preprocessor/8270] [4.7/4.8/4.9 Regression] back-slash white space newline with comments, no warning

[pinskia at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

--- Comment #48 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to GoWhoopee from comment #47)
> Please reconsider and stop gcc from changing our code without our permission.

It is not changing your code at all.  Read comment #39 to understand this issue
at full understanding of the standard.

[Bug preprocessor/8270] [4.7/4.8/4.9 Regression] back-slash white space newline with comments, no warning

[GoWhoopee at yahoo dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

--- Comment #49 from GoWhoopee at yahoo dot com ---
I've read all the comments and all those on linked forums and I have no idea
how you struggle with this!

If a compiler changes backslash space into backslash newline and consequently
deletes the newline it is changing the meaning of the code!

All other development environments people here have used don't do this and gcc
shouldn't!

Here's an example of code your compiler changes:

#define HIGH_SPEED_TURRET   // \ 
#define SAFETY_LOCKED_ON    //  >------------- Critical Configuration
#define NEVER_PRIME_MISSILE // /

The programmer put backslash space and the syntax highlighter correctly showed
the safety was locked on.

Sleep well.

[Bug preprocessor/8270] [4.7/4.8/4.9 Regression] back-slash white space newline with comments, no warning

[mikpelinux at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Mikael Pettersson <mikpelinux at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mikpelinux at gmail dot com

--- Comment #50 from Mikael Pettersson <mikpelinux at gmail dot com> ---
(In reply to Andrew Pinski from comment #48)
> (In reply to GoWhoopee from comment #47)
> > Please reconsider and stop gcc from changing our code without our permission.
> 
> It is not changing your code at all.  Read comment #39 to understand this
> issue at full understanding of the standard.

I'm looking at N1570 section 5.1.1.2 "Translation phases".  Phase 1 only maps
multibyte characters and trigraphs,  Backslash-space-newline is neither so
should be preserved as-is to phase 2.  The splicing in phase 2 then shouldn't
occur because of the space.  Or am I missing something?

[Bug preprocessor/8270] [4.7/4.8/4.9 Regression] back-slash white space newline with comments, no warning

[GoWhoopee at yahoo dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

--- Comment #51 from GoWhoopee at yahoo dot com ---
http://web.cs.dal.ca/~vlado/pl/C_Standard_2011-n1570.pdf

That's the principle, but not what happens with gcc...

Phase 2 says, "Each instance of a backslash character (\) immediately followed
by a newline character is deleted, splicing physical source lines to form
logical source lines.", and explicitly states that, "Only the last backslash on
any physical source line shall be eligible for such a splice.".

I wonder if all trailing white-space is being trimmed from each source line
before or during the first Translation Phase?

[Bug preprocessor/8270] [4.7/4.8/4.9 Regression] back-slash white space newline with comments, no warning

[GoWhoopee at yahoo dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

--- Comment #52 from GoWhoopee at yahoo dot com ---
Whitespace is required by Translation Phase 3, consequently Translation Phase 1
should not be changing whitespace at all, only mapping multibyte characters and
trigraphs.

Comment #39: Indicates that gcc is known to work incorrectly, "This (removal of
such spaces) is part of how GCC defines the implementation-defined mapping in
translation phase 1.": the removal of white-space is not mapping multibyte
characters or trigraphs, it is removing critical information from Translation
Phases 2 and 3 resulting in misinterpretation of the source code.

Looking at the 4.8.2 source, libcpp\lex.c line 1427, there is a fix when
parsing raw strings, after the event:
______________________________________________
static void
lex_raw_string (cpp_reader *pfile, cpp_token *token, const uchar *base,
        const uchar *cur)
{
[...]
      switch (note->type)
        {
        case '\\':
        case ' ':
          /* Restore backslash followed by newline.  */
          BUF_APPEND (base, cur - base);
          base = cur;
          BUF_APPEND ("\\", 1);
        after_backslash:
          if (note->type == ' ')
        {
          /* GNU backslash whitespace newline extension.  FIXME
             could be any sequence of non-vertical space.  When we
             can properly restore any such sequence, we should mark
             this note as handled so _cpp_process_line_notes
             doesn't warn.  */
          BUF_APPEND (" ", 1);
        }

          BUF_APPEND ("\n", 1);
          break;
______________________________________________

but fixing all the varieties of broken things after the event wouldn't be
necessary if Translation Phase 1 didn't trim whitespace.

If Translation Phase 1 is required to trim whitespace for some reason
(performance, perhaps) then it should trim multiple consecutive spaces down to
exactly one space; which wouldn't break Translation Phase 2 and 3.

Does that sound like a sensible compromise?

[Bug preprocessor/8270] [4.7/4.8/4.9/4.10 Regression] back-slash white space newline with comments, no warning

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.7.4                       |4.8.4

--- Comment #53 from Richard Biener <rguenth at gcc dot gnu.org> ---
The 4.7 branch is being closed, moving target milestone to 4.8.4.

[Bug preprocessor/8270] [4.8/4.9/5 Regression] back-slash white space newline with comments, no warning

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.8.4                       |4.8.5

--- Comment #54 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
GCC 4.8.4 has been released.

[Bug preprocessor/8270] [4.8/4.9/5 Regression] back-slash white space newline with comments, no warning

[ktietz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Kai Tietz <ktietz at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ktietz at gcc dot gnu.org

--- Comment #55 from Kai Tietz <ktietz at gcc dot gnu.org> ---
Well, by looking into the standard ISO/IEC 9899:TC3 I found the following
statement:

5.1.12 Translation phase
"2. Each instance of a backslash character (\) immediately followed by a
new-line
character is deleted, splicing physical source lines to form logical source
lines.
Only the last backslash on any physical source line shall be eligible for being
part of such a splice. A source file that is not empty shall end in a new-line
character, which shall not be immediately preceded by a backslash character
before any such splicing takes place."

For ISO/IEC 14882:2003 we see at topic "2 Lexical Convention"

"2 Each instance of a new-line character and an immediately preceding backslash
character is deleted, splicing physical source lines to form logical source
lines. If, as a result, a character sequence that matches the syntax of a
universal-character-name is produced, the behavior is undefined. If a source
file that is not empty does not end in a new-line character, or ends in a
new-line character immediately preceded by a backslash character, the behavior
is undefined."

So the handling of backslash whitespace newline is clearly a gnu-extension and
not part of the standard.

I suggest something like this patch for fixing standard-requirement. 
Additionally we could check here for cpp_option lang being gnu-style for
allowing 'backslash,whitespaces,newling' too.

Index: lex.c
===================================================================
--- lex.c       (Revision 221514)
+++ lex.c       (Arbeitskopie)
@@ -896,6 +896,11 @@ _cpp_clean_line (cpp_reader *pfile)
        p--;
       if (p - 1 != pbackslash)
        goto done;
+      if (p != d)
+       {
+         ++p;
+         goto done;
+       }

       /* Have an escaped newline; process it and proceed to
         the slow path.  */
@@ -917,13 +922,19 @@ _cpp_clean_line (cpp_reader *pfile)
              if (s == buffer->rlimit)
                break;

-             /* Escaped?  */
+             /* Escaped?
+                But make sure it isn't a backslash followed by a
+                whitespace.  */
              p = d;
              while (p != buffer->next_line && is_nvspace (p[-1]))
                p--;
              if (p == buffer->next_line || p[-1] != '\\')
                break;
-
+             if (p != d)
+               {
+                 ++p;
+                 break;
+               }
              add_line_note (buffer, p - 1, p != d ? ' ': '\\');
              d = p - 2;
              buffer->next_line = p - 1;

[Bug preprocessor/8270] [4.8/4.9/5 Regression] back-slash white space newline with comments, no warning

[doug at cs dot dartmouth.edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

--- Comment #56 from doug mcilroy <doug at cs dot dartmouth.edu> ---
(In reply to Kai Tietz from comment #55)
Comment #55 overlooks the Standard's translation phase 1, which replaces an
implementation-defined end-of-line indicator with a new-line character. GCC's
convention of including in the end-of-line indicator any white space that is
preceded by a backslash conforms, though it may be a surprise.

The surprise is perversely out of sympathy with the raison d'etre of the
standard--maximal portability. It is incompatible with the most direct (and
historically prior) implementations, wherein the end-of-line indicator is
simply a new-line character.

A suitable fix is to warn when white space occurs in an end-of-line indicator.
This will break no code that GCC currently compiles, yet draw attention to the
nonportable construct.

Here is what the C11 standard says about the end-of-line indicator:

5.1.1.2
Physical source file multibyte characters are mapped, in an implementation
defined manner, to the source character set (introducing new-line characters
for end-of-line indicators) if necessary.

5.2.1 paragraph 3
In source files, there shall be some way of indicating the end of each line of
text; this International Standard treats such an end-of-line indicator as if it
were a single new-line character.

[Bug preprocessor/8270] [4.8/4.9/5 Regression] back-slash white space newline with comments, no warning

[ktietz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

--- Comment #57 from Kai Tietz <ktietz at gcc dot gnu.org> ---
(In reply to doug mcilroy from comment #56)
> (In reply to Kai Tietz from comment #55)
> Comment #55 overlooks the Standard's translation phase 1, which replaces an
> implementation-defined end-of-line indicator with a new-line character.
> GCC's convention of including in the end-of-line indicator any white space
> that is preceded by a backslash conforms, though it may be a surprise.

Sure, sorry for omitting that.  Common understanding of "multibyte" (this term
is indeed misleading here) newline characters are in common the combination of
'\r' and '\n'.  So by interpreting any whitespace + new-line being seen as a
single-character is valid, but has indeed semantic differences.

> The surprise is perversely out of sympathy with the raison d'etre of the
> standard--maximal portability. It is incompatible with the most direct (and
> historically prior) implementations, wherein the end-of-line indicator is
> simply a new-line character.

Agreed, and we should at least consider to provide an option - beside the
necessary warning - to not strip whitespaces from right-handside of lines
containing a backslash at line's end.
Should we use an existing option (like -ansi), or introduce new option for
this?

> A suitable fix is to warn when white space occurs in an end-of-line
> indicator. This will break no code that GCC currently compiles, yet draw
> attention to the nonportable construct.

Well, in general we are warning, but within comments.  For C-style comments
there is indeed not much reason to warn, as there is no semantic difference. 
But for C++-style comments we should, as here indeed a semantic difference can
occure for gnu-style end-of-line treating

[Bug preprocessor/8270] [4.8/4.9/5/6 Regression] back-slash white space newline with comments, no warning

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.8.5                       |4.9.3

--- Comment #58 from Richard Biener <rguenth at gcc dot gnu.org> ---
The gcc-4_8-branch is being closed, re-targeting regressions to 4.9.3.

[Bug preprocessor/8270] [4.9/5/6 Regression] back-slash white space newline with comments, no warning

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

--- Comment #59 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
GCC 4.9.3 has been released.

[Bug preprocessor/8270] [4.9/5/6 Regression] back-slash white space newline with comments, no warning

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.9.3                       |4.9.4

[Bug preprocessor/8270] [9/10/11/12 Regression] back-slash white space newline with comments, no warning

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|8.5                         |9.4

--- Comment #65 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
GCC 8 branch is being closed.

[Bug preprocessor/8270] [9/10/11/12 Regression] back-slash white space newline with comments, no warning

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|9.4                         |9.5

--- Comment #66 from Richard Biener <rguenth at gcc dot gnu.org> ---
GCC 9.4 is being released, retargeting bugs to GCC 9.5.

[Bug preprocessor/8270] [10/11/12/13 Regression] back-slash white space newline with comments, no warning

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|9.5                         |10.4

--- Comment #67 from Richard Biener <rguenth at gcc dot gnu.org> ---
GCC 9 branch is being closed

[Bug preprocessor/8270] [10/11/12/13 Regression] back-slash white space newline with comments, no warning

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|10.4                        |10.5

--- Comment #68 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
GCC 10.4 is being released, retargeting bugs to GCC 10.5.

[Bug preprocessor/8270] [11/12/13/14 Regression] back-slash white space newline with comments, no warning

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|10.5                        |11.5

--- Comment #69 from Richard Biener <rguenth at gcc dot gnu.org> ---
GCC 10 branch is being closed.

[Bug preprocessor/8270] [11/12/13/14 Regression] back-slash white space newline with comments, no warning

[verodeving at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8270

veronica alphonso <verodeving at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |verodeving at gmail dot com

--- Comment #70 from veronica alphonso <verodeving at gmail dot com> ---
Any update on this bug?