[Bug driver/71850] @file should be used to cc1/cc1plus when @file is used

[Bug driver/71850] @file should be used to cc1/cc1plus when @file is used

[costas.argyris at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71850

Costas Argyris <costas.argyris at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |costas.argyris at gmail dot com

--- Comment #6 from Costas Argyris <costas.argyris at gmail dot com> ---
A few things have changed since the last update:

cc1 is no longer the problem, and it is not the subprocess that breaks for >
32k worth of command line arguments, but the bug has not gone away completely
because another subprocess still breaks after cc1, and that is the assembler
(as) this time.

At the time the bug was reported (version 6.1.0), the specs language did not
have the %@{...} feature, and include paths were handled in cpp_unique_options
as %{I*&F*}

https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/gcc.c;h=7460f6af148969734c6cffeef870388355ef76cf;hb=HEAD#l1105

This was causing the include path arguments to get passed to cc1 verbatim, i.e.
without being stored in a temporary @file and that file being passed to cc1
instead (assuming a @file was given to gcc in the first place).    Note that
there existed ways to create temporary @files back then (as pointed out by
Andrew), but they were not backed by the specs language itself through %@{...}.
   Same for the assembler with %{I*}

https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/gcc.c;h=7460f6af148969734c6cffeef870388355ef76cf;hb=HEAD#l1151

but it never got to that point because it was crashing earlier in cc1 when the
include paths were more than 32k bytes.

The %@{...} syntax didn't exist back then because it doesn't show up neither in
the source code above nor in the 6.1.0 doc

https://gcc.gnu.org/onlinedocs/gcc-6.1.0/gcc/Spec-Files.html    and search for
%@, no hits

Some time between then and now %@{...} was added in the specs language

https://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/Spec-Files.html   %@ has a hit
now

and is now being used for the linker -L flags

https://github.com/gcc-mirror/gcc/blob/releases/gcc-12/gcc/gcc.cc#L1162

and preprocessor include paths -I

https://github.com/gcc-mirror/gcc/blob/releases/gcc-12/gcc/gcc.cc#L1238

The latter silently fixed the problem with cc1 because it now takes a temporary
file instead of the raw include paths (assuming a @file was given to gcc to
begin with).    But now we just hit the same problem with the assembler,
because that one did not get updated to use %@{I*} and still uses %{I*}

https://github.com/gcc-mirror/gcc/blob/releases/gcc-12/gcc/gcc.cc#L1294

So right now it is 'as' that is breaking due to the 32k limit on Windows, not
'cc1'.    Simply adding the @ symbol for the include args in asm_options fixes
the issue.    See the experiments below for more details.




$ gcc --version
gcc.exe (Rev4, Built by MSYS2 project) 12.2.0

sample response file with > 32k worth of include paths

$ head -n3 rsp
-I/akjdhaskjdhfskjdfhakjsdhfkjlsahdfkjashdfasdkjfsadfasdkfjhsdakfjhds/asdkljfhsadkjfhaskjdfhkjasdhfkjashdfkjashfsdklafhdskjfhjkfhskjfhsakdjfh/LJHFDJKHSDKJFHASKDJdkjfghskdjfhgksdjfg/fgsdklfjghskdjfhgdksjfghdskjfgh
-I/akjdhaskjdhfskjdfhakjsdhfkjlsahdfkjashdfasdkjfsadfasdkfjhsdakfjhds/asdkljfhsadkjfhaskjdfhkjasdhfkjashdfkjashfsdklafhdskjfhjkfhskjfhsakdjfh/LJHFDJKHSDKJFHASKDJdkjfghskdjfhgksdjfg/fgsdklfjghskdjfhgdksjfghdskjfgh
-I/akjdhaskjdhfskjdfhakjsdhfkjlsahdfkjashdfasdkjfsadfasdkfjhsdakfjhds/asdkljfhsadkjfhaskjdfhkjasdhfkjashdfkjashfsdklafhdskjfhjkfhskjfhsakdjfh/LJHFDJKHSDKJFHASKDJdkjfghskdjfhgksdjfg/fgsdklfjghskdjfhgdksjfghdskjfgh

$ wc --chars rsp
37914 rsp


$ gcc -v @rsp src.c
...
...
...
 C:/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/12.2.0/cc1.exe -quiet -v
@C:\msys64\tmp\ccHiOua0 -D_REENTRANT src.c -quiet -dumpdir a- -dumpbase src.c
-dumpbase-ext .c -mtune=generic -march=x86-64 -version -o
C:\msys64\tmp\ccsGuieQ.s
...
...
...

C:/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/as.exe
-v -I
/akjdhaskjdhfskjdfhakjsdhfkjlsahdfkjashdfasdkjfsadfasdkfjhsdakfjhds/asdkljfhsadkjfhaskjdfhkjasdhfkjashdfkjashfsdklafhdskjfhjkfhskjfhsakdjfh/LJHFDJKHSDKJFHASKDJdkjfghskdjfhgksdjfg/fgsdklfjghskdjfhgdksjfghdskjfgh
-I
/akjdhaskjdhfskjdfhakjsdhfkjlsahdfkjashdfasdkjfsadfasdkfjhsdakfjhds/asdkljfhsadkjfhaskjdfhkjasdhfkjashdfkjashfsdklafhdskjfhjkfhskjfhsakdjfh/LJHFDJKHSDKJFHASKDJdkjfghskdjfhgksdjfg/fgsdklfjghskdjfhgdksjfghdskjfgh
-I<more than 32k> -o C:\msys64\tmp\ccy7bVd1.o C:\msys64\tmp\ccsGuieQ.s
gcc.exe: fatal error: cannot execute
'C:/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/as.exe':
CreateProcess: No such file or directory
compilation terminated.

cc1.exe is OK because it takes a temporary @file, but as.exe takes the include
paths so it breaks.    Confirm this by observing how include args are handled
in the two subprocesses.

$ gcc -dumpspecs | grep "{I" -C1
*asm_options:
%{-target-help:%:print-asm-header()} %{v} %{w:-W} %{I*} %(asm_debug_option)
%{gz|gz=zlib-gnu:--compress-debug-sections}
%{gz=none:--nocompress-debug-sections} %{gz=zlib:%e-gz=zlib is not supported in
this configuration} %a %Y %{c:%W{o*}%{!o*:-o %w%b%O}}%{!c:-o %d%w%u%O}

--
*cpp_unique_options:
%{!Q:-quiet} %{nostdinc*} %{C} %{CC} %{v} %@{I*&F*} %{P} %I %{MD:-MD
%{!o:%b.d}%{o*:%.d%*}} %{MMD:-MMD %{!o:%b.d}%{o*:%.d%*}} %{M} %{MM} %{MF*}
%{MG} %{MP} %{MQ*} %{MT*} %{Mmodules} %{Mno-modules}
%{!E:%{!M:%{!MM:%{!MT:%{!MQ:%{MD|MMD:%{o*:-MQ %*}}}}}}} %{remap}
%{%:debug-level-gt(2):-dD} %{!iplugindir*:%{fplugin*:%:find-plugindir()}} %{H}
%C %{D*&U*&A*} %{i*} %Z %i %{E|M|MM:%W{o*}}


create a custom specs_file that just adds @ for the assembler include paths

$ cat specs_file
*asm_options:
%{-target-help:%:print-asm-header()} %{v} %{w:-W} %@{I*} %(asm_debug_option)
%{gz|gz=zlib-gnu:--compress-debug-sections}
%{gz=none:--nocompress-debug-sections} %{gz=zlib:%e-gz=zlib is not supported in
this configuration} %a %Y %{c:%W{o*}%{!o*:-o %w%b%O}}%{!c:-o %d%w%u%O}

use it to override the built-in asm_options and the build succeeds

$ gcc -v @rsp src.c -specs=specs_file
...
...
...
 C:/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/12.2.0/cc1.exe -quiet -v
@C:\msys64\tmp\cczj5Mkl -D_REENTRANT src.c -quiet -dumpdir a- -dumpbase src.c
-dumpbase-ext .c -mtune=generic -march=x86-64 -version -o
C:\msys64\tmp\cckwORzw.s
...
...
...

C:/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/as.exe
-v @C:\msys64\tmp\ccDFfiec -o C:\msys64\tmp\ccQzY8VR.o C:\msys64\tmp\cckwORzw.s
...
...
...
no errors

Proposed fix: In the driver gcc.cc, asm_options string, change %{I*} to %@{I*}
and then we no longer have to pass a custom specs file to avoid the error.   
After making the change in gcc.cc and rebuilding gcc:

$ gcc -v @rsp src.c

...
...
...
 C:/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/12.2.0/cc1.exe -quiet -v
@C:\msys64\tmp\ccUzV01R -D_REENTRANT src.c -quiet -dumpdir a- -dumpbase src.c
-dumpbase-ext .c -mtune=generic -march=x86-64 -version -o
C:\msys64\tmp\cckn0YWz.s
...
...
...

C:/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/as.exe
-v @C:\msys64\tmp\cce7PpKJ -o C:\msys64\tmp\ccGWfEIT.o C:\msys64\tmp\cckn0YWz.s
...
...
...
no errors

$ gcc -dumpspecs | grep "{I" -C1
*asm_options:
%{-target-help:%:print-asm-header()} %{v} %{w:-W} %@{I*} %(asm_debug_option)
%{gz|gz=zlib-gnu:--compress-debug-sections}
%{gz=none:--nocompress-debug-sections} %{gz=zlib:%e-gz=zlib is not supported in
this configuration} %a %Y %{c:%W{o*}%{!o*:-o %w%b%O}}%{!c:-o %d%w%u%O}

--
*cpp_unique_options:
%{!Q:-quiet} %{nostdinc*} %{C} %{CC} %{v} %@{I*&F*} %{P} %I %{MD:-MD
%{!o:%b.d}%{o*:%.d%*}} %{MMD:-MMD %{!o:%b.d}%{o*:%.d%*}} %{M} %{MM} %{MF*}
%{MG} %{MP} %{MQ*} %{MT*} %{Mmodules} %{Mno-modules}
%{!E:%{!M:%{!MM:%{!MT:%{!MQ:%{MD|MMD:%{o*:-MQ %*}}}}}}} %{remap}
%{%:debug-level-gt(2):-dD} %{!iplugindir*:%{fplugin*:%:find-plugindir()}} %{H}
%C %{D*&U*&A*} %{i*} %Z %i %{E|M|MM:%W{o*}}



This fix doesn't prevent all cases of exceeding the 32k limit on Windows by
using a @file.    It only handles the include paths passed with -I.   
Therefore, it is still possible to exceed it using other arguments than -I.   
However, I think -I is by far the most likely candidate to exceed the limit,
and the one mentioned by the original poster.

[Bug driver/71850] @file should be used to cc1/cc1plus when @file is used

[costas.argyris at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71850

--- Comment #7 from Costas Argyris <costas.argyris at gmail dot com> ---
Created attachment 54575
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=54575&action=edit
Treat include path args the same way between cpp_unique_options and asm_options

The proposed patch extends the logic already used for passing include paths
down to the compiler (when a @file was given to gcc), also to the assembler.   
This unifies the logic and also avoids overflowing the Windows command line
when many include paths have been passed to gcc through a @file.

[Bug driver/71850] @file should be used to cc1/cc1plus when @file is used

[costas.argyris at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71850

--- Comment #8 from Costas Argyris <costas.argyris at gmail dot com> ---
Not likely to be fixed, so if someone stumbles upon this and needs a
workaround, you need to override the default specs as in the example above.

[Bug driver/71850] @file should be used to cc1/cc1plus when @file is used

[costas.argyris at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71850

--- Comment #9 from Costas Argyris <costas.argyris at gmail dot com> ---
Fixed by
https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=180ebb8a24d24fc5b105f2257d6216f6dfde62df

Also this is windows-host-specific (Host field in the PR is empty).

[Bug driver/71850] @file should be used to cc1/cc1plus when @file is used

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71850

Segher Boessenkool <segher at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |segher at gcc dot gnu.org
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #10 from Segher Boessenkool <segher at gcc dot gnu.org> ---
Costas says this is fixed by g:180ebb8a24d2 .  Marking as such.  Thanks :-)