[Bug preprocessor/65852] New: Incorrect escaping in depfiles

[Bug preprocessor/65852] New: Incorrect escaping in depfiles

[paul_robinson at playstation dot sony.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65852

            Bug ID: 65852
           Summary: Incorrect escaping in depfiles
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: preprocessor
          Assignee: unassigned at gcc dot gnu.org
          Reporter: paul_robinson at playstation dot sony.com

When gcc creates a depfile, and a target or dependency has a space or # in it,
the filespec in the depfile is escaped with a backslash so GNU Make won't
interpret it as a delimiter or comment character.
GNU Make also expects backslashes to be escaped, which can lead to some
misinterpretation if a backslash in the original filename is immediately
followed by a space or # character.

gcc will take care to double the backslashes immediately preceding a space,
but does not do the same for #, which results in a bogus depfile.


$ cat tspace.c
#include "foo\ bar.h"
#include "bar\#foo.h"
int i;
$ ls
bar\#foo.h foo\ bar.h tspace.c
$ gcc -c -MMD tspace.c
$ cat tspace.d
tspace.o: tspace.c foo\\\ bar.h bar\\#foo.h
$ make -f tspace.d tspace.o
make: *** No rule to make target `bar\', needed by `tspace.o'. Stop.
$

[Bug preprocessor/65852] Incorrect escaping in depfiles

[paul_robinson at playstation dot sony.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65852

Paul Robinson <paul_robinson at playstation dot sony.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|unknown                     |4.8.2

--- Comment #1 from Paul Robinson <paul_robinson at playstation dot sony.com> ---
Using GNU Make 3.8.1, which is what came with Ubuntu 14.04.

[Bug preprocessor/65852] Incorrect escaping in depfiles

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65852

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
This was bug 35458 which says it was fixed in 4.4.0.

[Bug preprocessor/65852] Incorrect escaping in depfiles

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65852

--- Comment #3 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Here is what the comment says in the source:
/* Given a filename, quote characters in that filename which are
   significant to Make.  Note that it's not possible to quote all such
   characters - e.g. \n, %, *, ?, [, \ (in some contexts), and ~ are
   not properly handled.  It isn't possible to get this right in any
   current version of Make.  (??? Still true?  Old comment referred to
   3.76.1.)  */

So maybe there was an issue with make at one point.  Basically \ is not mungled
to be what it should be done.

[Bug preprocessor/65852] Incorrect escaping in depfiles

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65852

--- Comment #4 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Really for portability using \ and/or # should be avoided.

[Bug preprocessor/65852] Incorrect escaping in depfiles

[paul_robinson at playstation dot sony.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65852

--- Comment #5 from Paul Robinson <paul_robinson at playstation dot sony.com> ---
Compare how space and # are handled, to see the problem.
They need to be handled the same way.

[Bug preprocessor/65852] Incorrect escaping in depfiles

[paul_robinson at playstation dot sony.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65852

--- Comment #6 from Paul Robinson <paul_robinson at playstation dot sony.com> ---
(In reply to Andrew Pinski from comment #4)
> Really for portability using \ and/or # should be avoided.

I accept that, however we do have Windows clients trying to use Make,
so \ is a fact of life.
These are admittedly obscure corner cases, however gcc took the trouble
to get space right, it would be nice to get # right also.

[Bug preprocessor/65852] Incorrect escaping in depfiles

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65852

--- Comment #7 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Hmm, I thought \ would be converted into / by the time we reach here for
windows.  I wonder if that is the better fix.

[Bug preprocessor/65852] Incorrect escaping in depfiles

[paul_robinson at playstation dot sony.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65852

--- Comment #8 from Paul Robinson <paul_robinson at playstation dot sony.com> ---
(In reply to Andrew Pinski from comment #7)
> Hmm, I thought \ would be converted into / by the time we reach here for
> windows.  I wonder if that is the better fix.

Admittedly the gcc I have on Windows is *very* old, but it does not
normalize \ to / before writing the depfile.

[Bug preprocessor/65852] Incorrect escaping in depfiles

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65852

--- Comment #9 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Paul Robinson from comment #8)
> Admittedly the gcc I have on Windows is *very* old, but it does not
> normalize \ to / before writing the depfile.

I know it is very old, I worked on it before leaving Sony :).  I did a 4.3 port
and it seemed to be working better for windows but Sony want to use this port.

[Bug preprocessor/65852] Incorrect escaping in depfiles

[paul_robinson at playstation dot sony.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65852

--- Comment #10 from Paul Robinson <paul_robinson at playstation dot sony.com> ---
I thought I had heard your name before!  Howdy...

Granted this is an obscure corner case, but they are legal filenames,
using a leading # is less horrifying than a leading space, and the
depfile that gcc produces is demonstrably wrong.

I just downloaded Cygwin with gcc 4.9.2 to my Windows, and the behavior
is exactly the same (using \ instead of / as the separator, seeing \\\ 
before the space and only \\ before the #).

Anyway, can we agree that the inconsistent handling of space and # is
a bug?  You can set the priority as low as you like.

FYI, you do _not_ want to escape all backslashes; that will break
BSD Make, which doesn't do any quoting at all.

[Bug preprocessor/65852] Incorrect escaping in depfiles

[ccoutant at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65852

Cary Coutant <ccoutant at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ccoutant at gmail dot com

--- Comment #11 from Cary Coutant <ccoutant at gmail dot com> ---
According to the comments in mkdeps.c, the "bar\\#foo.h" is actually
correct:

          /* GNU make uses a weird quoting scheme for white space.
             A space or tab preceded by 2N+1 backslashes represents
             N backslashes followed by space; a space or tab
             preceded by 2N backslashes represents N backslashes at
             the end of a file name; and backslashes in other
             contexts should not be doubled.  */

In this case, the '\' is not doubled, and the '#' is escaped as '\#',
giving you the '\\#' sequence.

But experimentation with make demonstrates that the scheme described
in that comment also works for '#', so I guess that GCC ought to
escape it the same way.

Then again, the Gnu make manual, in section 4.4.2, says this about
Windows filenames:

"""
Microsoft operating systems (MS-DOS and MS-Windows) use backslashes to
separate directories in pathnames, like so:

  c:\foo\bar\baz.c

This is equivalent to the Unix-style c:/foo/bar/baz.c (the c: part is
the so-called drive letter). When make runs on these systems, it
supports backslashes as well as the Unix-style forward slashes in
pathnames. However, this support does not include the wildcard
expansion, where backslash is a quote character. Therefore, you must
use Unix-style slashes in these cases.
"""

It does not work the same way for escaping a wildcard character:
writing "foo\\\*.h" gives this error:

   make: *** No rule to make target `foo\\\*.h', needed by `foo.o'.  Stop.

It works if you have a file literally named "foo\\\*.h", but it's not
a wildcard, while writing "foo\\*.h" will match a file named
"foo\\bar.h" (so an odd number of backslashes escape the wildcard, but
don't escape themselves, while an even number of backslashes remain as
is).

The whole thing seems horribly messed up in several respects. Still, it
seems reasonable to expect GCC to escape a filename containing '#' such
that GNU make can un-escape it properly.