public inbox for gcc-help@gcc.gnu.org
 help / color / mirror / Atom feed
* CreateProcess No such file or directory
@ 2022-09-20 16:02 mizo 91
  2022-09-20 16:18 ` Richard Earnshaw
  2022-09-22  6:44 ` LIU Hao
  0 siblings, 2 replies; 26+ messages in thread
From: mizo 91 @ 2022-09-20 16:02 UTC (permalink / raw)
  To: gcc-help

[-- Attachment #1: Type: text/plain, Size: 819 bytes --]

Hello,

I'm having trouble compiling simple test program on windows 10 with long
list of includes provided via '@response_file' argument

The full error is:

gcc.exe: fatal error: cannot execute
'C:/Tools/x86_64-12.2.0-release-win32-seh-rt_v10-rev0/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/as.exe':
CreateProcess: No such file or directory

compilation terminated.

Above path to the assembler program is valid therefore I don't understand
what is the issue here?

You can try to recreate the problem by compiling simple main.c file with
nothing in it but main function using command:
gcc @includes.txt main.c

I assume it has something to do with windows 32k CreateProcess command
length limitation? But isn't that the reason why respone files were
invented?

Regards,
Filip

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-20 16:02 CreateProcess No such file or directory mizo 91
@ 2022-09-20 16:18 ` Richard Earnshaw
  2022-09-20 17:17   ` mizo 91
  2022-09-22  6:44 ` LIU Hao
  1 sibling, 1 reply; 26+ messages in thread
From: Richard Earnshaw @ 2022-09-20 16:18 UTC (permalink / raw)
  To: mizo 91, gcc-help



On 20/09/2022 17:02, mizo 91 via Gcc-help wrote:
> Hello,
> 
> I'm having trouble compiling simple test program on windows 10 with long
> list of includes provided via '@response_file' argument
> 
> The full error is:
> 
> gcc.exe: fatal error: cannot execute
> 'C:/Tools/x86_64-12.2.0-release-win32-seh-rt_v10-rev0/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/as.exe':
> CreateProcess: No such file or directory
> 
> compilation terminated.
> 
> Above path to the assembler program is valid therefore I don't understand
> what is the issue here?
> 
> You can try to recreate the problem by compiling simple main.c file with
> nothing in it but main function using command:
> gcc @includes.txt main.c
> 
> I assume it has something to do with windows 32k CreateProcess command
> length limitation? But isn't that the reason why respone files were
> invented?
> 
> Regards,
> Filip

I'm by no means a windows expert, but I wonder if this might be related 
to the length of the path to the assembler and that it's somehow getting 
truncated.

R.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-20 16:18 ` Richard Earnshaw
@ 2022-09-20 17:17   ` mizo 91
  2022-09-20 22:27     ` Tamar Christina
  0 siblings, 1 reply; 26+ messages in thread
From: mizo 91 @ 2022-09-20 17:17 UTC (permalink / raw)
  To: Richard Earnshaw; +Cc: gcc-help

[-- Attachment #1: Type: text/plain, Size: 1876 bytes --]

 According to Microsoft docs path length limitation for windows is 256. In
my case its 143 chars. I think the problem might be related to how gcc
driver is passing along unmarshalled arguments to its subprograms like
as.exe. Suprisingly this works correctly if I enforce gcc driver to use
preprocessor only by adding -E flag

btw. I failed to previously mention that I'm using Mingw-W64 version
provided kindly by niXman
Binaries taken directly from official MinGW-W64 builds:
https://www.mingw-w64.org/downloads/#mingw-builds
https://github.com/niXman/mingw-builds-binaries/releases

R.

wt., 20 wrz 2022 o 18:18 Richard Earnshaw <Richard.Earnshaw@foss.arm.com>
napisał(a):

>
>
> On 20/09/2022 17:02, mizo 91 via Gcc-help wrote:
> > Hello,
> >
> > I'm having trouble compiling simple test program on windows 10 with long
> > list of includes provided via '@response_file' argument
> >
> > The full error is:
> >
> > gcc.exe: fatal error: cannot execute
> >
> 'C:/Tools/x86_64-12.2.0-release-win32-seh-rt_v10-rev0/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/as.exe':
> > CreateProcess: No such file or directory
> >
> > compilation terminated.
> >
> > Above path to the assembler program is valid therefore I don't understand
> > what is the issue here?
> >
> > You can try to recreate the problem by compiling simple main.c file with
> > nothing in it but main function using command:
> > gcc @includes.txt main.c
> >
> > I assume it has something to do with windows 32k CreateProcess command
> > length limitation? But isn't that the reason why respone files were
> > invented?
> >
> > Regards,
> > Filip
>
> I'm by no means a windows expert, but I wonder if this might be related
> to the length of the path to the assembler and that it's somehow getting
> truncated.
>
> R.
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* RE: CreateProcess No such file or directory
  2022-09-20 17:17   ` mizo 91
@ 2022-09-20 22:27     ` Tamar Christina
  2022-09-21  2:42       ` fedor_qd
  0 siblings, 1 reply; 26+ messages in thread
From: Tamar Christina @ 2022-09-20 22:27 UTC (permalink / raw)
  To: mizo 91, Richard Earnshaw; +Cc: gcc-help

We'd mostly be guessing here, use strace or https://learn.microsoft.com/en-us/sysinternals/downloads/procmon to see which path is actually being accessed and why it failed.

Also check that you can run `as` correctly. This error can also happen if a dependency of `as.exe` isn't found during loading.

Cheers,
Tamar

> -----Original Message-----
> From: Gcc-help <gcc-help-bounces+tamar.christina=arm.com@gcc.gnu.org>
> On Behalf Of mizo 91 via Gcc-help
> Sent: Tuesday, September 20, 2022 1:17 PM
> To: Richard Earnshaw <Richard.Earnshaw@foss.arm.com>
> Cc: gcc-help@gcc.gnu.org
> Subject: Re: CreateProcess No such file or directory
> 
>  According to Microsoft docs path length limitation for windows is 256. In my
> case its 143 chars. I think the problem might be related to how gcc driver is
> passing along unmarshalled arguments to its subprograms like as.exe.
> Suprisingly this works correctly if I enforce gcc driver to use preprocessor only
> by adding -E flag
> 
> btw. I failed to previously mention that I'm using Mingw-W64 version
> provided kindly by niXman Binaries taken directly from official MinGW-W64
> builds:
> https://www.mingw-w64.org/downloads/#mingw-builds
> https://github.com/niXman/mingw-builds-binaries/releases
> 
> R.
> 
> wt., 20 wrz 2022 o 18:18 Richard Earnshaw
> <Richard.Earnshaw@foss.arm.com>
> napisał(a):
> 
> >
> >
> > On 20/09/2022 17:02, mizo 91 via Gcc-help wrote:
> > > Hello,
> > >
> > > I'm having trouble compiling simple test program on windows 10 with
> > > long list of includes provided via '@response_file' argument
> > >
> > > The full error is:
> > >
> > > gcc.exe: fatal error: cannot execute
> > >
> > 'C:/Tools/x86_64-12.2.0-release-win32-seh-rt_v10-
> rev0/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-
> w64-mingw32/bin/as.exe':
> > > CreateProcess: No such file or directory
> > >
> > > compilation terminated.
> > >
> > > Above path to the assembler program is valid therefore I don't
> > > understand what is the issue here?
> > >
> > > You can try to recreate the problem by compiling simple main.c file
> > > with nothing in it but main function using command:
> > > gcc @includes.txt main.c
> > >
> > > I assume it has something to do with windows 32k CreateProcess
> > > command length limitation? But isn't that the reason why respone
> > > files were invented?
> > >
> > > Regards,
> > > Filip
> >
> > I'm by no means a windows expert, but I wonder if this might be
> > related to the length of the path to the assembler and that it's
> > somehow getting truncated.
> >
> > R.
> >

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: RE: CreateProcess No such file or directory
  2022-09-20 22:27     ` Tamar Christina
@ 2022-09-21  2:42       ` fedor_qd
  2022-09-21 15:27         ` mizo 91
  0 siblings, 1 reply; 26+ messages in thread
From: fedor_qd @ 2022-09-21  2:42 UTC (permalink / raw)
  To: Tamar Christina; +Cc: gcc-help, mizo 91, Richard Earnshaw

[-- Attachment #1: Type: text/plain, Size: 3022 bytes --]


I saw such error in my build gcc 12.1.0 for Symbian. GCC 11.2.0 works fine.
I encounter such error while building ScummVM for Symbian. This error happened randomly. I thought my environment broken and switched to GCC 11.2.0 and errors gone.
GCC 11.2.0 comes with binutils 2.35, GCC 12.1.0 comes with binutils 2.29.1
What binutils version you have?
--
Sent from Outlook Email App for Android серада, 21 верасня 2022, 01:29AM +03:00 from Tamar Christina via Gcc-help  gcc-help@gcc.gnu.org :

>We'd mostly be guessing here, use strace or  https://learn.microsoft.com/en-us/sysinternals/downloads/procmon to see which path is actually being accessed and why it failed.
>
>Also check that you can run `as` correctly. This error can also happen if a dependency of `as.exe` isn't found during loading.
>
>Cheers,
>Tamar
>
> -----Original Message-----
> From: Gcc-help < gcc-help-bounces+tamar.christina=arm.com@gcc.gnu.org>
> On Behalf Of mizo 91 via Gcc-help
> Sent: Tuesday, September 20, 2022 1:17 PM
> To: Richard Earnshaw < Richard.Earnshaw@foss.arm.com>
> Cc:  gcc-help@gcc.gnu.org
> Subject: Re: CreateProcess No such file or directory
>
> According to Microsoft docs path length limitation for windows is 256. In my
> case its 143 chars. I think the problem might be related to how gcc driver is
> passing along unmarshalled arguments to its subprograms like as.exe.
> Suprisingly this works correctly if I enforce gcc driver to use preprocessor only
> by adding -E flag
>
> btw. I failed to previously mention that I'm using Mingw-W64 version
> provided kindly by niXman Binaries taken directly from official MinGW-W64
> builds:
> https://www.mingw-w64.org/downloads/#mingw-builds
> https://github.com/niXman/mingw-builds-binaries/releases
>
> R.
>
> wt., 20 wrz 2022 o 18:18 Richard Earnshaw
>< Richard.Earnshaw@foss.arm.com>
> napisał(a):
>
>>
>>
>> On 20/09/2022 17:02, mizo 91 via Gcc-help wrote:
>>> Hello,
>>>
>>> I'm having trouble compiling simple test program on windows 10 with
>>> long list of includes provided via '@response_file' argument
>>>
>>> The full error is:
>>>
>>> gcc.exe: fatal error: cannot execute
>>>
>> 'C:/Tools/x86_64-12.2.0-release-win32-seh-rt_v10-
> rev0/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-
> w64-mingw32/bin/as.exe':
>>> CreateProcess: No such file or directory
>>>
>>> compilation terminated.
>>>
>>> Above path to the assembler program is valid therefore I don't
>>> understand what is the issue here?
>>>
>>> You can try to recreate the problem by compiling simple main.c file
>>> with nothing in it but main function using command:
>>> gcc @includes.txt main.c
>>>
>>> I assume it has something to do with windows 32k CreateProcess
>>> command length limitation? But isn't that the reason why respone
>>> files were invented?
>>>
>>> Regards,
>>> Filip
>>
>> I'm by no means a windows expert, but I wonder if this might be
>> related to the length of the path to the assembler and that it's
>> somehow getting truncated.
>>
>> R.
>>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: RE: CreateProcess No such file or directory
  2022-09-21  2:42       ` fedor_qd
@ 2022-09-21 15:27         ` mizo 91
  2022-09-21 15:41           ` Tom Kacvinsky
  2022-09-26  6:58           ` Re[2]: " fedor_qd
  0 siblings, 2 replies; 26+ messages in thread
From: mizo 91 @ 2022-09-21 15:27 UTC (permalink / raw)
  To: fedor_qd; +Cc: Tamar Christina, gcc-help, Richard Earnshaw

[-- Attachment #1: Type: text/plain, Size: 4530 bytes --]

@Tamar.Christina

I have created public repository on github and placed 'procmon' log file
there:

https://github.com/fskoras/CreateProcess-gcc-issue/blob/master/Logfile.CSV

You can also clone the repository if you want to try to recreate the issue:

https://github.com/fskoras/CreateProcess-gcc-issue

I have executed the as.exe directly and it worked without problems:

D:/Tools/x86_64-12.2.0-release-win32-seh-rt_v10-rev0/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/as.exe
--version

GNU assembler (GNU Binutils) 2.38

Copyright (C) 2022 Free Software Foundation, Inc.

This program is free software; you may redistribute it under the terms of

the GNU General Public License version 3 or later.

This program has absolutely no warranty.

This assembler was configured for a target of `x86_64-w64-mingw32'.

@fedor_qd

I have checked version 11.2.0 and the same problem occurs


binutils version used for both versions:

gcc x86_64-11.2.0: GNU assembler (GNU Binutils) 2.37

gcc x86_64-12.2.0: GNU assembler (GNU Binutils) 2.38

R.


śr., 21 wrz 2022 o 04:42 <fedor_qd@mail.ru> napisał(a):

> I saw such error in my build gcc 12.1.0 for Symbian. GCC 11.2.0 works fine.
>
> I encounter such error while building ScummVM for Symbian. This error
> happened randomly. I thought my environment broken and switched to GCC
> 11.2.0 and errors gone.
>
> GCC 11.2.0 comes with binutils 2.35, GCC 12.1.0 comes with binutils 2.29.1
> What binutils version you have?
>
> --
> Sent from Outlook Email App for Android
> серада, 21 верасня 2022, 01:29AM +03:00 from Tamar Christina via Gcc-help
> gcc-help@gcc.gnu.org:
>
> We'd mostly be guessing here, use strace or
> https://learn.microsoft.com/en-us/sysinternals/downloads/procmon to see
> which path is actually being accessed and why it failed.
>
> Also check that you can run `as` correctly. This error can also happen if
> a dependency of `as.exe` isn't found during loading.
>
> Cheers,
> Tamar
>
> > -----Original Message-----
> > From: Gcc-help <gcc-help-bounces+tamar.christina=arm.com@gcc.gnu.org>
> > On Behalf Of mizo 91 via Gcc-help
> > Sent: Tuesday, September 20, 2022 1:17 PM
> > To: Richard Earnshaw <Richard.Earnshaw@foss.arm.com>
> > Cc: gcc-help@gcc.gnu.org
> > Subject: Re: CreateProcess No such file or directory
> >
> > According to Microsoft docs path length limitation for windows is 256.
> In my
> > case its 143 chars. I think the problem might be related to how gcc
> driver is
> > passing along unmarshalled arguments to its subprograms like as.exe.
> > Suprisingly this works correctly if I enforce gcc driver to use
> preprocessor only
> > by adding -E flag
> >
> > btw. I failed to previously mention that I'm using Mingw-W64 version
> > provided kindly by niXman Binaries taken directly from official MinGW-W64
> > builds:
> > https://www.mingw-w64.org/downloads/#mingw-builds
> > https://github.com/niXman/mingw-builds-binaries/releases
> >
> > R.
> >
> > wt., 20 wrz 2022 o 18:18 Richard Earnshaw
> > <Richard.Earnshaw@foss.arm.com>
> > napisał(a):
> >
> > >
> > >
> > > On 20/09/2022 17:02, mizo 91 via Gcc-help wrote:
> > > > Hello,
> > > >
> > > > I'm having trouble compiling simple test program on windows 10 with
> > > > long list of includes provided via '@response_file' argument
> > > >
> > > > The full error is:
> > > >
> > > > gcc.exe: fatal error: cannot execute
> > > >
> > > 'C:/Tools/x86_64-12.2.0-release-win32-seh-rt_v10-
> > rev0/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-
> > w64-mingw32/bin/as.exe':
> > > > CreateProcess: No such file or directory
> > > >
> > > > compilation terminated.
> > > >
> > > > Above path to the assembler program is valid therefore I don't
> > > > understand what is the issue here?
> > > >
> > > > You can try to recreate the problem by compiling simple main.c file
> > > > with nothing in it but main function using command:
> > > > gcc @includes.txt main.c
> > > >
> > > > I assume it has something to do with windows 32k CreateProcess
> > > > command length limitation? But isn't that the reason why respone
> > > > files were invented?
> > > >
> > > > Regards,
> > > > Filip
> > >
> > > I'm by no means a windows expert, but I wonder if this might be
> > > related to the length of the path to the assembler and that it's
> > > somehow getting truncated.
> > >
> > > R.
> > >
>
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: RE: CreateProcess No such file or directory
  2022-09-21 15:27         ` mizo 91
@ 2022-09-21 15:41           ` Tom Kacvinsky
  2022-09-26  6:58           ` Re[2]: " fedor_qd
  1 sibling, 0 replies; 26+ messages in thread
From: Tom Kacvinsky @ 2022-09-21 15:41 UTC (permalink / raw)
  To: gcc-help

[-- Attachment #1: Type: text/plain, Size: 5486 bytes --]

My input way below...

On Wed, Sep 21, 2022 at 11:28 AM mizo 91 via Gcc-help <gcc-help@gcc.gnu.org>
wrote:

> @Tamar.Christina
>
> I have created public repository on github and placed 'procmon' log file
> there:
>
> https://github.com/fskoras/CreateProcess-gcc-issue/blob/master/Logfile.CSV
>
> You can also clone the repository if you want to try to recreate the issue:
>
> https://github.com/fskoras/CreateProcess-gcc-issue
>
> I have executed the as.exe directly and it worked without problems:
>
>
> D:/Tools/x86_64-12.2.0-release-win32-seh-rt_v10-rev0/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/as.exe
> --version
>
> GNU assembler (GNU Binutils) 2.38
>
> Copyright (C) 2022 Free Software Foundation, Inc.
>
> This program is free software; you may redistribute it under the terms of
>
> the GNU General Public License version 3 or later.
>
> This program has absolutely no warranty.
>
> This assembler was configured for a target of `x86_64-w64-mingw32'.
>
> @fedor_qd
>
> I have checked version 11.2.0 and the same problem occurs
>
>
> binutils version used for both versions:
>
> gcc x86_64-11.2.0: GNU assembler (GNU Binutils) 2.37
>
> gcc x86_64-12.2.0: GNU assembler (GNU Binutils) 2.38
>
> R.
>
>
> śr., 21 wrz 2022 o 04:42 <fedor_qd@mail.ru> napisał(a):
>
> > I saw such error in my build gcc 12.1.0 for Symbian. GCC 11.2.0 works
> fine.
> >
> > I encounter such error while building ScummVM for Symbian. This error
> > happened randomly. I thought my environment broken and switched to GCC
> > 11.2.0 and errors gone.
> >
> > GCC 11.2.0 comes with binutils 2.35, GCC 12.1.0 comes with binutils
> 2.29.1
> > What binutils version you have?
> >
> > --
> > Sent from Outlook Email App for Android
> > серада, 21 верасня 2022, 01:29AM +03:00 from Tamar Christina via Gcc-help
> > gcc-help@gcc.gnu.org:
> >
> > We'd mostly be guessing here, use strace or
> > https://learn.microsoft.com/en-us/sysinternals/downloads/procmon to see
> > which path is actually being accessed and why it failed.
> >
> > Also check that you can run `as` correctly. This error can also happen if
> > a dependency of `as.exe` isn't found during loading.
> >
> > Cheers,
> > Tamar
>

I have built GCC for MinGW-w64  and have had exactly this same issue.  I was
able to get around it by following what the MSYS2/MinGW-w64 folks do (using
a
MSYS2 shell).  Look at this repo

https://github.com/msys2/MINGW-packages

and search for the GCC subsdirectry.  In there you will find all sorts of
good stuff
and patches to apply.  Follow that build script setup as closely as
possible and the
problems with cc1/as/etc... not being found should be cleared up.

Also of note - the MinGW-w64 people run configure with --disable-bootstrap.

Tom

> >
> > > -----Original Message-----
> > > From: Gcc-help <gcc-help-bounces+tamar.christina=arm.com@gcc.gnu.org>
> > > On Behalf Of mizo 91 via Gcc-help
> > > Sent: Tuesday, September 20, 2022 1:17 PM
> > > To: Richard Earnshaw <Richard.Earnshaw@foss.arm.com>
> > > Cc: gcc-help@gcc.gnu.org
> > > Subject: Re: CreateProcess No such file or directory
> > >
> > > According to Microsoft docs path length limitation for windows is 256.
> > In my
> > > case its 143 chars. I think the problem might be related to how gcc
> > driver is
> > > passing along unmarshalled arguments to its subprograms like as.exe.
> > > Suprisingly this works correctly if I enforce gcc driver to use
> > preprocessor only
> > > by adding -E flag
> > >
> > > btw. I failed to previously mention that I'm using Mingw-W64 version
> > > provided kindly by niXman Binaries taken directly from official
> MinGW-W64
> > > builds:
> > > https://www.mingw-w64.org/downloads/#mingw-builds
> > > https://github.com/niXman/mingw-builds-binaries/releases
> > >
> > > R.
> > >
> > > wt., 20 wrz 2022 o 18:18 Richard Earnshaw
> > > <Richard.Earnshaw@foss.arm.com>
> > > napisał(a):
> > >
> > > >
> > > >
> > > > On 20/09/2022 17:02, mizo 91 via Gcc-help wrote:
> > > > > Hello,
> > > > >
> > > > > I'm having trouble compiling simple test program on windows 10 with
> > > > > long list of includes provided via '@response_file' argument
> > > > >
> > > > > The full error is:
> > > > >
> > > > > gcc.exe: fatal error: cannot execute
> > > > >
> > > > 'C:/Tools/x86_64-12.2.0-release-win32-seh-rt_v10-
> > >
> rev0/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-
> > > w64-mingw32/bin/as.exe':
> > > > > CreateProcess: No such file or directory
> > > > >
> > > > > compilation terminated.
> > > > >
> > > > > Above path to the assembler program is valid therefore I don't
> > > > > understand what is the issue here?
> > > > >
> > > > > You can try to recreate the problem by compiling simple main.c file
> > > > > with nothing in it but main function using command:
> > > > > gcc @includes.txt main.c
> > > > >
> > > > > I assume it has something to do with windows 32k CreateProcess
> > > > > command length limitation? But isn't that the reason why respone
> > > > > files were invented?
> > > > >
> > > > > Regards,
> > > > > Filip
> > > >
> > > > I'm by no means a windows expert, but I wonder if this might be
> > > > related to the length of the path to the assembler and that it's
> > > > somehow getting truncated.
> > > >
> > > > R.
> > > >
> >
> >
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-20 16:02 CreateProcess No such file or directory mizo 91
  2022-09-20 16:18 ` Richard Earnshaw
@ 2022-09-22  6:44 ` LIU Hao
  2022-09-22  7:35   ` mizo 91
  2022-09-22  8:19   ` Jonathan Wakely
  1 sibling, 2 replies; 26+ messages in thread
From: LIU Hao @ 2022-09-22  6:44 UTC (permalink / raw)
  To: mizo 91, gcc-help


[-- Attachment #1.1: Type: text/plain, Size: 2029 bytes --]

在 2022/9/21 00:02, mizo 91 via Gcc-help 写道:
> Hello,
> 
> I'm having trouble compiling simple test program on windows 10 with long
> list of includes provided via '@response_file' argument
> 
> 

Greetings. mingw-w64 developer speaking.

As far as I can see, there are at least two issues about your report:


The first, obvious issue is that the error message is incorrect. The reason for that is, if we take 
a look at 'libiberty/pex-win32.c' we see the following:

   853   /* Create the child process.  */
   854   pid = win32_spawn (executable, (flags & PEX_SEARCH) != 0,
   855                      argv, env, dwCreationFlags, &si, &pi);
   856   if (pid == (pid_t) -1)
   857     pid = spawn_script (executable, argv, env, dwCreationFlags,
   858                         &si, &pi);
   859   if (pid == (pid_t) -1)
   860     {
   861       *err = ENOENT;
   862       *errmsg = "CreateProcess";
   863     }

We also notice this is the only place where `"CreateProcess"` appears as a sole part of an error 
message.

The cause of this issue is apparent: libiberty tries `win32_spawn`, and if for whatever reason it 
fails, it makes another attempt with `spawn_script`, and if it fails again, `*err` is always set to 
`ENOENT` i.e. `No such file or directory`, no matter why.


Since we are invoking 'as.exe' here, which is never a script, the first attempt must have failed. We 
can start 'gcc.exe' with a debugger. There are actually two calls to `CreateProcessA()`: one to 
'cc1.exe' and the other to 'as.exe'; the latter fails with `ERROR_INVALID_PARAMETER`.

So, the error happens, not because anything can't be found, but because the command line is too 
long. With your example, it contains a lot of `-include` arguments and takes ~44K characters, and is 
not valid. Windows only allows a single command line up to 32,767 characters, because the NT syscall 
uses a 16-bit length for a UTF-16 string which includes a null terminator.


-- 
Best regards,
LIU Hao

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-22  6:44 ` LIU Hao
@ 2022-09-22  7:35   ` mizo 91
  2022-09-22  9:46     ` LIU Hao
  2022-09-22  8:19   ` Jonathan Wakely
  1 sibling, 1 reply; 26+ messages in thread
From: mizo 91 @ 2022-09-22  7:35 UTC (permalink / raw)
  To: gcc-help, LIU Hao

[-- Attachment #1: Type: text/plain, Size: 2952 bytes --]

Hello LIU Hao,

Thank you for your insight. In my understanding main purpouse of response
file is to overcome command too long issue(which can also happen on linux
but is by magnitude less likely). I imagine if my command is too long gcc
driver should ensure that subprograms like assembler should have recieved
command arguments in the same way gcc did (which is through response file).
Insted the assembler process is spawned directly By CreateProcessA whith
unwrapped arguments by the driver which is like you said the reason for the
error. Additionaly isnt -I flags required for preprocessor only? Why gcc is
trying to pass -I arguments to assembler program? Wouldnt you consider this
gcc driver bug than?

Kind Regards,
Filip

czw., 22 wrz 2022 o 08:44 LIU Hao <lh_mouse@126.com> napisał(a):

> 在 2022/9/21 00:02, mizo 91 via Gcc-help 写道:
> > Hello,
> >
> > I'm having trouble compiling simple test program on windows 10 with long
> > list of includes provided via '@response_file' argument
> >
> >
>
> Greetings. mingw-w64 developer speaking.
>
> As far as I can see, there are at least two issues about your report:
>
>
> The first, obvious issue is that the error message is incorrect. The
> reason for that is, if we take
> a look at 'libiberty/pex-win32.c' we see the following:
>
>    853   /* Create the child process.  */
>    854   pid = win32_spawn (executable, (flags & PEX_SEARCH) != 0,
>    855                      argv, env, dwCreationFlags, &si, &pi);
>    856   if (pid == (pid_t) -1)
>    857     pid = spawn_script (executable, argv, env, dwCreationFlags,
>    858                         &si, &pi);
>    859   if (pid == (pid_t) -1)
>    860     {
>    861       *err = ENOENT;
>    862       *errmsg = "CreateProcess";
>    863     }
>
> We also notice this is the only place where `"CreateProcess"` appears as a
> sole part of an error
> message.
>
> The cause of this issue is apparent: libiberty tries `win32_spawn`, and if
> for whatever reason it
> fails, it makes another attempt with `spawn_script`, and if it fails
> again, `*err` is always set to
> `ENOENT` i.e. `No such file or directory`, no matter why.
>
>
> Since we are invoking 'as.exe' here, which is never a script, the first
> attempt must have failed. We
> can start 'gcc.exe' with a debugger. There are actually two calls to
> `CreateProcessA()`: one to
> 'cc1.exe' and the other to 'as.exe'; the latter fails with
> `ERROR_INVALID_PARAMETER`.
>
> So, the error happens, not because anything can't be found, but because
> the command line is too
> long. With your example, it contains a lot of `-include` arguments and
> takes ~44K characters, and is
> not valid. Windows only allows a single command line up to 32,767
> characters, because the NT syscall
> uses a 16-bit length for a UTF-16 string which includes a null terminator.
>
>
> --
> Best regards,
> LIU Hao
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-22  6:44 ` LIU Hao
  2022-09-22  7:35   ` mizo 91
@ 2022-09-22  8:19   ` Jonathan Wakely
  2022-09-22  8:42     ` Jonathan Wakely
  1 sibling, 1 reply; 26+ messages in thread
From: Jonathan Wakely @ 2022-09-22  8:19 UTC (permalink / raw)
  To: LIU Hao; +Cc: mizo 91, gcc-help

On Thu, 22 Sept 2022 at 07:45, LIU Hao via Gcc-help
<gcc-help@gcc.gnu.org> wrote:
>
> 在 2022/9/21 00:02, mizo 91 via Gcc-help 写道:
> > Hello,
> >
> > I'm having trouble compiling simple test program on windows 10 with long
> > list of includes provided via '@response_file' argument
> >
> >
>
> Greetings. mingw-w64 developer speaking.
>
> As far as I can see, there are at least two issues about your report:
>
>
> The first, obvious issue is that the error message is incorrect. The reason for that is, if we take
> a look at 'libiberty/pex-win32.c' we see the following:
>
>    853   /* Create the child process.  */
>    854   pid = win32_spawn (executable, (flags & PEX_SEARCH) != 0,
>    855                      argv, env, dwCreationFlags, &si, &pi);
>    856   if (pid == (pid_t) -1)
>    857     pid = spawn_script (executable, argv, env, dwCreationFlags,
>    858                         &si, &pi);
>    859   if (pid == (pid_t) -1)
>    860     {
>    861       *err = ENOENT;
>    862       *errmsg = "CreateProcess";
>    863     }
>
> We also notice this is the only place where `"CreateProcess"` appears as a sole part of an error
> message.
>
> The cause of this issue is apparent: libiberty tries `win32_spawn`, and if for whatever reason it
> fails, it makes another attempt with `spawn_script`, and if it fails again, `*err` is always set to
> `ENOENT` i.e. `No such file or directory`, no matter why.

It seems to me that pex-win32.c should use GetLastError() (and maybe
FormatMessage) to get a Windows error code, rather than assuming
ENOENT.

Using C++ that would look something like:

std::string errmsg = "CreateProcess:" +
std::system_category().message(GetLastError());

Or maybe win32_spawn should call GetLastError() and convert that to an
errno value and store it in errno:

std::error_condition ec =
std::system_category().default_error_condition(GetLastError());
if (ec.category() == std::generic_category())
  errno = ec.value();
else
  errno = ENOENT; // or EINVAL might be better?

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-22  8:19   ` Jonathan Wakely
@ 2022-09-22  8:42     ` Jonathan Wakely
  2022-09-22  9:48       ` LIU Hao
  2022-09-22  9:50       ` mizo 91
  0 siblings, 2 replies; 26+ messages in thread
From: Jonathan Wakely @ 2022-09-22  8:42 UTC (permalink / raw)
  To: LIU Hao; +Cc: mizo 91, gcc-help

On Thu, 22 Sept 2022 at 09:19, Jonathan Wakely wrote:
>
> On Thu, 22 Sept 2022 at 07:45, LIU Hao via Gcc-help
> <gcc-help@gcc.gnu.org> wrote:
> >
> > 在 2022/9/21 00:02, mizo 91 via Gcc-help 写道:
> > > Hello,
> > >
> > > I'm having trouble compiling simple test program on windows 10 with long
> > > list of includes provided via '@response_file' argument
> > >
> > >
> >
> > Greetings. mingw-w64 developer speaking.
> >
> > As far as I can see, there are at least two issues about your report:
> >
> >
> > The first, obvious issue is that the error message is incorrect. The reason for that is, if we take
> > a look at 'libiberty/pex-win32.c' we see the following:
> >
> >    853   /* Create the child process.  */
> >    854   pid = win32_spawn (executable, (flags & PEX_SEARCH) != 0,
> >    855                      argv, env, dwCreationFlags, &si, &pi);
> >    856   if (pid == (pid_t) -1)
> >    857     pid = spawn_script (executable, argv, env, dwCreationFlags,
> >    858                         &si, &pi);
> >    859   if (pid == (pid_t) -1)
> >    860     {
> >    861       *err = ENOENT;
> >    862       *errmsg = "CreateProcess";
> >    863     }
> >
> > We also notice this is the only place where `"CreateProcess"` appears as a sole part of an error
> > message.
> >
> > The cause of this issue is apparent: libiberty tries `win32_spawn`, and if for whatever reason it
> > fails, it makes another attempt with `spawn_script`, and if it fails again, `*err` is always set to
> > `ENOENT` i.e. `No such file or directory`, no matter why.
>
> It seems to me that pex-win32.c should use GetLastError() (and maybe
> FormatMessage) to get a Windows error code, rather than assuming
> ENOENT.
>
> Using C++ that would look something like:
>
> std::string errmsg = "CreateProcess:" +
> std::system_category().message(GetLastError());
>
> Or maybe win32_spawn should call GetLastError() and convert that to an
> errno value and store it in errno:
>
> std::error_condition ec =
> std::system_category().default_error_condition(GetLastError());
> if (ec.category() == std::generic_category())
>   errno = ec.value();
> else
>   errno = ENOENT; // or EINVAL might be better?

I reported https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107007

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-22  7:35   ` mizo 91
@ 2022-09-22  9:46     ` LIU Hao
  2022-09-22 14:55       ` mizo 91
  0 siblings, 1 reply; 26+ messages in thread
From: LIU Hao @ 2022-09-22  9:46 UTC (permalink / raw)
  To: mizo 91, gcc-help


[-- Attachment #1.1: Type: text/plain, Size: 1204 bytes --]

在 2022/9/22 15:35, mizo 91 写道:
> Hello LIU Hao,
> 
> Thank you for your insight. In my understanding main purpouse of response file is to overcome 
> command too long issue(which can also happen on linux but is by magnitude less likely). I imagine if 
> my command is too long gcc driver should ensure that subprograms like assembler should have recieved 
> command arguments in the same way gcc did (which is through response file). Insted the assembler 
> process is spawned directly By CreateProcessA whith unwrapped arguments by the driver which is like 
> you said the reason for the error. Additionaly isnt -I flags required for preprocessor only? Why gcc 
> is trying to pass -I arguments to assembler program? Wouldnt you consider this gcc driver bug than?
> 
>

I doubt whether there is necessity to fix this 'bug', if it has to be a bug, after all. Normally a 
user is not expected to compose a command that comprises tens of thousands of characters. If they do 
then they are likely doing something wrong; they should have split such a source into multiple files 
to reduce dependencies and passed shorter command lines to build them.



-- 
Best regards,
LIU Hao

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-22  8:42     ` Jonathan Wakely
@ 2022-09-22  9:48       ` LIU Hao
  2022-09-22  9:50       ` mizo 91
  1 sibling, 0 replies; 26+ messages in thread
From: LIU Hao @ 2022-09-22  9:48 UTC (permalink / raw)
  To: Jonathan Wakely; +Cc: mizo 91, gcc-help


[-- Attachment #1.1: Type: text/plain, Size: 327 bytes --]

在 2022/9/22 16:42, Jonathan Wakely 写道:
> 
> I reported https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107007

Thank you. It's much better than what we have now. I recall that I did encounter such cryptic errors 
some times before, due to misconfiguration. It would have helped a lot.


-- 
Best regards,
LIU Hao

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-22  8:42     ` Jonathan Wakely
  2022-09-22  9:48       ` LIU Hao
@ 2022-09-22  9:50       ` mizo 91
  1 sibling, 0 replies; 26+ messages in thread
From: mizo 91 @ 2022-09-22  9:50 UTC (permalink / raw)
  To: Jonathan Wakely; +Cc: LIU Hao, gcc-help

[-- Attachment #1: Type: text/plain, Size: 2848 bytes --]

Thank you both,

I did a little research on gcc/driver bugzilla and it seepms my problem
might be ralated to those bug reports:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86030
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=45749 <- this one goes way
back to 2010 but it is still pending

Kind Regards,
Filip

czw., 22 wrz 2022 o 10:42 Jonathan Wakely <jwakely.gcc@gmail.com>
napisał(a):

> On Thu, 22 Sept 2022 at 09:19, Jonathan Wakely wrote:
> >
> > On Thu, 22 Sept 2022 at 07:45, LIU Hao via Gcc-help
> > <gcc-help@gcc.gnu.org> wrote:
> > >
> > > 在 2022/9/21 00:02, mizo 91 via Gcc-help 写道:
> > > > Hello,
> > > >
> > > > I'm having trouble compiling simple test program on windows 10 with
> long
> > > > list of includes provided via '@response_file' argument
> > > >
> > > >
> > >
> > > Greetings. mingw-w64 developer speaking.
> > >
> > > As far as I can see, there are at least two issues about your report:
> > >
> > >
> > > The first, obvious issue is that the error message is incorrect. The
> reason for that is, if we take
> > > a look at 'libiberty/pex-win32.c' we see the following:
> > >
> > >    853   /* Create the child process.  */
> > >    854   pid = win32_spawn (executable, (flags & PEX_SEARCH) != 0,
> > >    855                      argv, env, dwCreationFlags, &si, &pi);
> > >    856   if (pid == (pid_t) -1)
> > >    857     pid = spawn_script (executable, argv, env, dwCreationFlags,
> > >    858                         &si, &pi);
> > >    859   if (pid == (pid_t) -1)
> > >    860     {
> > >    861       *err = ENOENT;
> > >    862       *errmsg = "CreateProcess";
> > >    863     }
> > >
> > > We also notice this is the only place where `"CreateProcess"` appears
> as a sole part of an error
> > > message.
> > >
> > > The cause of this issue is apparent: libiberty tries `win32_spawn`,
> and if for whatever reason it
> > > fails, it makes another attempt with `spawn_script`, and if it fails
> again, `*err` is always set to
> > > `ENOENT` i.e. `No such file or directory`, no matter why.
> >
> > It seems to me that pex-win32.c should use GetLastError() (and maybe
> > FormatMessage) to get a Windows error code, rather than assuming
> > ENOENT.
> >
> > Using C++ that would look something like:
> >
> > std::string errmsg = "CreateProcess:" +
> > std::system_category().message(GetLastError());
> >
> > Or maybe win32_spawn should call GetLastError() and convert that to an
> > errno value and store it in errno:
> >
> > std::error_condition ec =
> > std::system_category().default_error_condition(GetLastError());
> > if (ec.category() == std::generic_category())
> >   errno = ec.value();
> > else
> >   errno = ENOENT; // or EINVAL might be better?
>
> I reported https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107007
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-22  9:46     ` LIU Hao
@ 2022-09-22 14:55       ` mizo 91
  2022-09-22 16:04         ` LIU Hao
  2022-09-23 17:14         ` David Brown
  0 siblings, 2 replies; 26+ messages in thread
From: mizo 91 @ 2022-09-22 14:55 UTC (permalink / raw)
  To: LIU Hao; +Cc: gcc-help

[-- Attachment #1: Type: text/plain, Size: 2745 bytes --]

Hi LIU Hao,

Following that logic, let's just get rid of response file feature while we
are at it. Why encourage bad programming practices? Why maintain something
that doesn't even work properly?

I think this topic is very underrated and many people working with large
codebases have problems because of it.

For example, let's say I have a project structure of over 100+ modules.
Each module uses internal headers from other modules. It's much easier to
maintain a single global "include directories" list than to do it on a
per-module basis. Because if something changes in one module I would have
to rewrite configuraiton for all 100 modules as well. If I'm not mistaken
Eclise CDT is using this kind of project configuration approach.

Additionaly projects may rely on macro definitions to provide configuration
values through the command line. And for large codebases, there could be
hundreds of such macros. This alone can easily bring the length of compile
command closer to this limit. Of course, you can define these macros in the
header and add them as another dependency, but this is just a workaround,
not a solution.

I'm sure there are many different kinds of codebase configurations that
will rely on this feature. And the people adopting these codebases would
hit a brick wall.

So yeah this is definitely a 'BUG'. Big one in my opinion.

Kind Regards,
Filip

czw., 22 wrz 2022 o 11:47 LIU Hao <lh_mouse@126.com> napisał(a):

> 在 2022/9/22 15:35, mizo 91 写道:
> > Hello LIU Hao,
> >
> > Thank you for your insight. In my understanding main purpouse of
> response file is to overcome
> > command too long issue(which can also happen on linux but is by
> magnitude less likely). I imagine if
> > my command is too long gcc driver should ensure that subprograms like
> assembler should have recieved
> > command arguments in the same way gcc did (which is through response
> file). Insted the assembler
> > process is spawned directly By CreateProcessA whith unwrapped arguments
> by the driver which is like
> > you said the reason for the error. Additionaly isnt -I flags required
> for preprocessor only? Why gcc
> > is trying to pass -I arguments to assembler program? Wouldnt you
> consider this gcc driver bug than?
> >
> >
>
> I doubt whether there is necessity to fix this 'bug', if it has to be a
> bug, after all. Normally a
> user is not expected to compose a command that comprises tens of thousands
> of characters. If they do
> then they are likely doing something wrong; they should have split such a
> source into multiple files
> to reduce dependencies and passed shorter command lines to build them.
>
>
>
> --
> Best regards,
> LIU Hao
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-22 14:55       ` mizo 91
@ 2022-09-22 16:04         ` LIU Hao
  2022-09-22 16:50           ` mizo 91
  2022-09-23 17:14         ` David Brown
  1 sibling, 1 reply; 26+ messages in thread
From: LIU Hao @ 2022-09-22 16:04 UTC (permalink / raw)
  To: mizo 91; +Cc: gcc-help


[-- Attachment #1.1: Type: text/plain, Size: 2778 bytes --]

在 2022-09-22 22:55, mizo 91 写道:
> Hi LIU Hao,
> 
> Following that logic, let's just get rid of response file feature while we are at it. Why encourage 
> bad programming practices? Why maintain something that doesn't even work properly?
> 
> I think this topic is very underrated and many people working with large codebases have problems 
> because of it.
> 

Such limits exist because the Windows NT syscall passes command lines via the `UNICODE_STRING` 
struct, which stores the string length as the number of bytes as an `unsigned short`. The maximum 
number of UTF-16 code units is consequently 0xFFFF / 2 = 32767.

There isn't "something that doesn't even work properly". It's just the system limit. On Linux we 
have a much larger limit (usually a few MiBs) but there is still one. Given an arbitrary repository, 
checked out at an arbitrary directory, with an arbitrary number of submodules, then it's likely that 
the limit will be hit sooner or later.


> For example, let's say I have a project structure of over 100+ modules. Each module uses internal 
> headers from other modules. It's much easier to maintain a single global "include directories" list 
> than to do it on a per-module basis. Because if something changes in one module I would have to 
> rewrite configuraiton for all 100 modules as well. If I'm not mistaken Eclise CDT is using this kind 
> of project configuration approach.
> 
> Additionaly projects may rely on macro definitions to provide configuration values through the 
> command line. And for large codebases, there could be hundreds of such macros. This alone can easily 
> bring the length of compile command closer to this limit. Of course, you can define these macros in 
> the header and add them as another dependency, but this is just a workaround, not a solution.
> 

I think I have to disagree here. Response files are there to work around the 8-KiB limit of command 
lines in CMD, but it cannot work around system limits. Using a 'config.h' of macros is widely known, 
accepted and preferred, rather than passing a lot of macros via command line. Similarly, if there 
are too many object files to link, a convenience library or incremental linking with `ld -r` will 
usually solve the issue. If you don't want to do these by tampering with build systems, xargs may 
help. There are a lot of mature solutions for keeping away from system limits, but generally we 
assume our users know what they're doing.


> I'm sure there are many different kinds of codebase configurations that will rely on this feature. 
> And the people adopting these codebases would hit a brick wall.
> 
> So yeah this is definitely a 'BUG'. Big one in my opinion.
> 
>
-- 
Best regards,
LIU Hao


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-22 16:04         ` LIU Hao
@ 2022-09-22 16:50           ` mizo 91
  2022-09-23 17:40             ` Xi Ruoyao
  0 siblings, 1 reply; 26+ messages in thread
From: mizo 91 @ 2022-09-22 16:50 UTC (permalink / raw)
  To: LIU Hao; +Cc: gcc-help

[-- Attachment #1: Type: text/plain, Size: 3758 bytes --]

Ok, if indeed this is a system limitation that cannot be avoided then why
is the clang compiler not having this problem? It uses exactly the same
argument interface and I was able to compile a test program with much more
than 32k characters given by the response file.

I understand why the limit exists and that it is a Windows specific process
creation limitation. And I agree with your stand that project should be
structure in a way that this limit never occures. However sometimes it's
just not feasible to do because of reasons beyond your controll. I'm not
going too deep into that statement :)

I'm just trying to say that this border can be overcome. There is
definitely room for improvement here. And if possible, why not just admit
it? :)

czw., 22 wrz 2022, 18:04 użytkownik LIU Hao <lh_mouse@126.com> napisał:

> 在 2022-09-22 22:55, mizo 91 写道:
> > Hi LIU Hao,
> >
> > Following that logic, let's just get rid of response file feature while
> we are at it. Why encourage
> > bad programming practices? Why maintain something that doesn't even work
> properly?
> >
> > I think this topic is very underrated and many people working with large
> codebases have problems
> > because of it.
> >
>
> Such limits exist because the Windows NT syscall passes command lines via
> the `UNICODE_STRING`
> struct, which stores the string length as the number of bytes as an
> `unsigned short`. The maximum
> number of UTF-16 code units is consequently 0xFFFF / 2 = 32767.
>
> There isn't "something that doesn't even work properly". It's just the
> system limit. On Linux we
> have a much larger limit (usually a few MiBs) but there is still one.
> Given an arbitrary repository,
> checked out at an arbitrary directory, with an arbitrary number of
> submodules, then it's likely that
> the limit will be hit sooner or later.
>
>
> > For example, let's say I have a project structure of over 100+ modules.
> Each module uses internal
> > headers from other modules. It's much easier to maintain a single global
> "include directories" list
> > than to do it on a per-module basis. Because if something changes in one
> module I would have to
> > rewrite configuraiton for all 100 modules as well. If I'm not mistaken
> Eclise CDT is using this kind
> > of project configuration approach.
> >
> > Additionaly projects may rely on macro definitions to provide
> configuration values through the
> > command line. And for large codebases, there could be hundreds of such
> macros. This alone can easily
> > bring the length of compile command closer to this limit. Of course, you
> can define these macros in
> > the header and add them as another dependency, but this is just a
> workaround, not a solution.
> >
>
> I think I have to disagree here. Response files are there to work around
> the 8-KiB limit of command
> lines in CMD, but it cannot work around system limits. Using a 'config.h'
> of macros is widely known,
> accepted and preferred, rather than passing a lot of macros via command
> line. Similarly, if there
> are too many object files to link, a convenience library or incremental
> linking with `ld -r` will
> usually solve the issue. If you don't want to do these by tampering with
> build systems, xargs may
> help. There are a lot of mature solutions for keeping away from system
> limits, but generally we
> assume our users know what they're doing.
>
>
> > I'm sure there are many different kinds of codebase configurations that
> will rely on this feature.
> > And the people adopting these codebases would hit a brick wall.
> >
> > So yeah this is definitely a 'BUG'. Big one in my opinion.
> >
> >
> --
> Best regards,
> LIU Hao
>
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-22 14:55       ` mizo 91
  2022-09-22 16:04         ` LIU Hao
@ 2022-09-23 17:14         ` David Brown
  2022-09-23 20:55           ` mizo 91
  1 sibling, 1 reply; 26+ messages in thread
From: David Brown @ 2022-09-23 17:14 UTC (permalink / raw)
  To: mizo 91, LIU Hao; +Cc: gcc-help

On 22/09/2022 16:55, mizo 91 via Gcc-help wrote:
> Hi LIU Hao,
> 
> Following that logic, let's just get rid of response file feature while we
> are at it. Why encourage bad programming practices? Why maintain something
> that doesn't even work properly?
> 
> I think this topic is very underrated and many people working with large
> codebases have problems because of it.
> 
> For example, let's say I have a project structure of over 100+ modules.
> Each module uses internal headers from other modules. It's much easier to
> maintain a single global "include directories" list than to do it on a
> per-module basis. Because if something changes in one module I would have
> to rewrite configuraiton for all 100 modules as well. If I'm not mistaken
> Eclise CDT is using this kind of project configuration approach.
> 
> Additionaly projects may rely on macro definitions to provide configuration
> values through the command line. And for large codebases, there could be
> hundreds of such macros. This alone can easily bring the length of compile
> command closer to this limit. Of course, you can define these macros in the
> header and add them as another dependency, but this is just a workaround,
> not a solution.
> 
> I'm sure there are many different kinds of codebase configurations that
> will rely on this feature. And the people adopting these codebases would
> hit a brick wall.
> 
> So yeah this is definitely a 'BUG'. Big one in my opinion.
> 
> Kind Regards,
> Filip
> 

This all sounds like a very questionable way to organise your project. 
(I hesitate to say "wrong", because people have good reasons for 
organising projects in different ways, and sometimes it's just down to 
personal preferences.  But if I were given charge of this project as you 
describe it, I'd say it's wrong and must be changed.)

I would say that if you are listing a large number of include 
directories in the include search path, you are doing things wrong. 
This is especially true when you have code from different places and 
different modules - you are just asking for trouble when there are 
filename clashes between parts.  It is much better to put the directory 
explicitly in the #include statements.  Alternatively, you can have 
indirect inclusions - have a single directory "modules_includes" that 
has an include file for each module.  Those include files have nothing 
but an include directive with the real module public include file, 
including the directory.  This "modules_include" might then go on your 
compiler include path, but I'd not do that either.

That way you can easily include the files you want, and not accidentally 
include private headers that are internal to modules.  It is much easier 
to find what you want, and much easier to read the source code.  When 
you see "#include "modules_include/foo.h" ", or "foo/public.h" 
(depending on which solution you use) in the source code, you know it is 
the public interface for the module "foo".  You don't have to wonder 
which of the hundred directories it might be in or what module it is.

Yes, Eclipse CDT likes to make huge include paths.  (At least, that is 
my experience using many embedded toolchain packages based on Eclipse.) 
  It is a PITA.  It also has a painfully inefficient build system by 
default.  It's okay for tiny test projects - for anything serious, you 
want a different project organisation and a different build system 
(make, or whatever you like).  This has the extra advantage of making 
Eclipse CDT much faster and more convenient when you use it as an editor 
and IDE.


Regarding the #define macros, put them all in a single header.  If it 
makes life easier, use the gcc option "-imacros <file>" to include it 
for all compilations.  A file of defines is simpler to read, edit, 
comment, change and maintain than a list of "-D..." options passed to 
the compiler.  It also gives you more flexibility, such as using 
conditional compilation to allow some macros to depend on others.


Of course I don't know your project at all, and don't know how realistic 
these suggestions are, but hopefully you'll find them helpful.

David

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-22 16:50           ` mizo 91
@ 2022-09-23 17:40             ` Xi Ruoyao
  2022-09-23 21:11               ` mizo 91
  0 siblings, 1 reply; 26+ messages in thread
From: Xi Ruoyao @ 2022-09-23 17:40 UTC (permalink / raw)
  To: mizo 91, LIU Hao; +Cc: gcc-help

On Thu, 2022-09-22 at 18:50 +0200, mizo 91 via Gcc-help wrote:

> Additionaly isnt -I flags required for preprocessor only? Why gcc is
> trying to pass -I arguments to assembler program? Wouldnt you consider this
> gcc driver bug than?

From GNU assembler manual:

    -I dir
      Add directory dir to the search list for ".include" directives.

So -I flags may be for preprocessor, or for assembler.

> Ok, if indeed this is a system limitation that cannot be avoided then why
> is the clang compiler not having this problem?

Because clang emits an object file without invoking an assembler.  This
is a difference in the fundamental design of the two compilers, and you
can't change it just for some looooooong command line.

-- 
Xi Ruoyao <xry111@xry111.site>
School of Aerospace Science and Technology, Xidian University

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-23 17:14         ` David Brown
@ 2022-09-23 20:55           ` mizo 91
  0 siblings, 0 replies; 26+ messages in thread
From: mizo 91 @ 2022-09-23 20:55 UTC (permalink / raw)
  To: David Brown; +Cc: LIU Hao, gcc-help

[-- Attachment #1: Type: text/plain, Size: 3604 bytes --]

Hi David,

At the begining let me thank you for seriously addressing my concern. I
aggre that approach with global list of includes is not perfect. But it's
been in use for so long by so many people in my company that switching to
something else at this point is simply impossible. They are using Tasking
compiler which comes integrated with some derivative form of Eclipse IDE.
What I have to work with is preconfigured, There is a lot of auto generated
code and changing the approach would require extrem amount of work which I
simply cant handle on my own. I didnt invented it. I'm just trying to make
it work in my Software in the loop solution for testing purpouses and not
having a compiler that could handle this amount of load is a big no no for
me. I will simply switch to Clang and never think about this problem
again.Thank you all for you valuable sugestions.

Kind Regards,
Filip

This all sounds like a very questionable way to organise your project.
> (I hesitate to say "wrong", because people have good reasons for
> organising projects in different ways, and sometimes it's just down to
> personal preferences.  But if I were given charge of this project as you
> describe it, I'd say it's wrong and must be changed.)
>

> I would say that if you are listing a large number of include
> directories in the include search path, you are doing things wrong.
> This is especially true when you have code from different places and
> different modules - you are just asking for trouble when there are
> filename clashes between parts.  It is much better to put the directory
> explicitly in the #include statements.  Alternatively, you can have
> indirect inclusions - have a single directory "modules_includes" that
> has an include file for each module.  Those include files have nothing
> but an include directive with the real module public include file,
> including the directory.  This "modules_include" might then go on your
> compiler include path, but I'd not do that either.
>
> That way you can easily include the files you want, and not accidentally
> include private headers that are internal to modules.  It is much easier
> to find what you want, and much easier to read the source code.  When
> you see "#include "modules_include/foo.h" ", or "foo/public.h"
> (depending on which solution you use) in the source code, you know it is
> the public interface for the module "foo".  You don't have to wonder
> which of the hundred directories it might be in or what module it is.
>
> Yes, Eclipse CDT likes to make huge include paths.  (At least, that is
> my experience using many embedded toolchain packages based on Eclipse.)
>   It is a PITA.  It also has a painfully inefficient build system by
> default.  It's okay for tiny test projects - for anything serious, you
> want a different project organisation and a different build system
> (make, or whatever you like).  This has the extra advantage of making
> Eclipse CDT much faster and more convenient when you use it as an editor
> and IDE.
>
>
> Regarding the #define macros, put them all in a single header.  If it
> makes life easier, use the gcc option "-imacros <file>" to include it
> for all compilations.  A file of defines is simpler to read, edit,
> comment, change and maintain than a list of "-D..." options passed to
> the compiler.  It also gives you more flexibility, such as using
> conditional compilation to allow some macros to depend on others.
>
>
> Of course I don't know your project at all, and don't know how realistic
> these suggestions are, but hopefully you'll find them helpful.
>
> David
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-23 17:40             ` Xi Ruoyao
@ 2022-09-23 21:11               ` mizo 91
  2022-09-24  5:13                 ` Xi Ruoyao
  0 siblings, 1 reply; 26+ messages in thread
From: mizo 91 @ 2022-09-23 21:11 UTC (permalink / raw)
  To: Xi Ruoyao; +Cc: LIU Hao, gcc-help

[-- Attachment #1: Type: text/plain, Size: 1141 bytes --]

pt., 23 wrz 2022 o 19:40 Xi Ruoyao <xry111@xry111.site> napisał(a):

> On Thu, 2022-09-22 at 18:50 +0200, mizo 91 via Gcc-help wrote:
>
> > Additionaly isnt -I flags required for preprocessor only? Why gcc is
> > trying to pass -I arguments to assembler program? Wouldnt you consider
> this
> > gcc driver bug than?
>
> From GNU assembler manual:
>
>     -I dir
>       Add directory dir to the search list for ".include" directives.
>
> So -I flags may be for preprocessor, or for assembler.
>

I'm not calling the assembler. Im calling gcc. Check what gcc manual have
to say about it.


> > Ok, if indeed this is a system limitation that cannot be avoided then why
> > is the clang compiler not having this problem?
>
> Because clang emits an object file without invoking an assembler.  This
> is a difference in the fundamental design of the two compilers,


Make sense


> and you
> can't change it just for some looooooong command line.
>

I'm not going to comment that.You do You :0


>
> --
> Xi Ruoyao <xry111@xry111.site>
> School of Aerospace Science and Technology, Xidian University
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-23 21:11               ` mizo 91
@ 2022-09-24  5:13                 ` Xi Ruoyao
  2022-09-24  9:28                   ` mizo 91
  0 siblings, 1 reply; 26+ messages in thread
From: Xi Ruoyao @ 2022-09-24  5:13 UTC (permalink / raw)
  To: mizo 91; +Cc: LIU Hao, gcc-help

On Fri, 2022-09-23 at 23:11 +0200, mizo 91 wrote:
> > From GNU assembler manual:
> > 
> >     -I dir
> >       Add directory dir to the search list for ".include"
> > directives.
> > 
> > So -I flags may be for preprocessor, or for assembler.
>  
> I'm not calling the assembler. Im calling gcc. Check what gcc manual
> have to say about it.

The problem is it's common to use "gcc -c foo.s" instead of "as foo.s -o
foo.o" in the building system of the packages.  It can ensure the
assembler used for both C files and assembly files the same one.  If you
suddenly stop to pass -I options to the assembler, you can break the
building process of existing packages.
-- 
Xi Ruoyao <xry111@xry111.site>
School of Aerospace Science and Technology, Xidian University

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-24  5:13                 ` Xi Ruoyao
@ 2022-09-24  9:28                   ` mizo 91
  2022-09-24  9:51                     ` Xi Ruoyao
  0 siblings, 1 reply; 26+ messages in thread
From: mizo 91 @ 2022-09-24  9:28 UTC (permalink / raw)
  To: Xi Ruoyao; +Cc: LIU Hao, gcc-help

[-- Attachment #1: Type: text/plain, Size: 1407 bytes --]

You are right that changing this behavior would have serious consequences.
But wouldn't you agree that gcc could somehow distinguish whether the input
files is of *.c or *.s source kind, and then decide if it should pass -I to
the assembler or the preprocessor or  both? That would be more in line with
what I would expect after reading the documentation. Passing -I to the
assembler when invoking the command 'gcc -c source.c' seems unnecessary. Of
course I admit there could be something I'm missing here.

sob., 24 wrz 2022 o 07:13 Xi Ruoyao <xry111@xry111.site> napisał(a):

> On Fri, 2022-09-23 at 23:11 +0200, mizo 91 wrote:
> > > From GNU assembler manual:
> > >
> > >     -I dir
> > >       Add directory dir to the search list for ".include"
> > > directives.
> > >
> > > So -I flags may be for preprocessor, or for assembler.
> >
> > I'm not calling the assembler. Im calling gcc. Check what gcc manual
> > have to say about it.
>
> The problem is it's common to use "gcc -c foo.s" instead of "as foo.s -o
> foo.o" in the building system of the packages.  It can ensure the
> assembler used for both C files and assembly files the same one.  If you
> suddenly stop to pass -I options to the assembler, you can break the
> building process of existing packages.
> --
> Xi Ruoyao <xry111@xry111.site>
> School of Aerospace Science and Technology, Xidian University
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-24  9:28                   ` mizo 91
@ 2022-09-24  9:51                     ` Xi Ruoyao
  2022-09-24 10:20                       ` Jonathan Wakely
  0 siblings, 1 reply; 26+ messages in thread
From: Xi Ruoyao @ 2022-09-24  9:51 UTC (permalink / raw)
  To: mizo 91; +Cc: LIU Hao, gcc-help

On Sat, 2022-09-24 at 11:28 +0200, mizo 91 wrote:
> You are right that changing this behavior would have serious
> consequences. But wouldn't you agree that gcc could somehow
> distinguish whether the input files is of *.c or *.s source kind
> , and then decide if it should pass -I to the assembler or the
> preprocessor or  both? That would be more in line with what I would
> expect after reading the documentation. Passing -I to the assembler
> when invoking the command 'gcc -c source.c' seems unnecessary.

Even with .c files it's still legal to use ".include" directives in
inline assembly.

The only rational thing to do is creating a file containing the options
for the assembler, then pass it to the assembler with @FILE.  As @FILE
is not supported by non-GNU (and, maybe, old GNU) assemblers, the
feature cannot be the default and we need to add a new option "-use-
cmdfile-for-as" (or some better name).

If someone really cares about this he/she can make a patch and send it
to gcc-patches@gcc.gnu.org.
-- 
Xi Ruoyao <xry111@xry111.site>
School of Aerospace Science and Technology, Xidian University

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: CreateProcess No such file or directory
  2022-09-24  9:51                     ` Xi Ruoyao
@ 2022-09-24 10:20                       ` Jonathan Wakely
  0 siblings, 0 replies; 26+ messages in thread
From: Jonathan Wakely @ 2022-09-24 10:20 UTC (permalink / raw)
  To: Xi Ruoyao; +Cc: mizo 91, gcc-help

On Sat, 24 Sept 2022 at 10:52, Xi Ruoyao wrote:
> The only rational thing to do is creating a file containing the options
> for the assembler, then pass it to the assembler with @FILE.  As @FILE
> is not supported by non-GNU (and, maybe, old GNU) assemblers, the
> feature cannot be the default and we need to add a new option "-use-
> cmdfile-for-as" (or some better name).

The configure process can check whether it's supported, or just assume
it's supported when --with-gnu-as is in effect.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re[2]: RE: CreateProcess No such file or directory
  2022-09-21 15:27         ` mizo 91
  2022-09-21 15:41           ` Tom Kacvinsky
@ 2022-09-26  6:58           ` fedor_qd
  1 sibling, 0 replies; 26+ messages in thread
From: fedor_qd @ 2022-09-26  6:58 UTC (permalink / raw)
  To: mizo 91; +Cc: Tamar Christina, gcc-help, Richard Earnshaw

[-- Attachment #1: Type: text/plain, Size: 4401 bytes --]


I was very lucky and never see that error before. 11.2.0 has same error too.
--
Sent from Outlook Email App for Android серада, 21 верасня 2022, 06:27PM +03:00 from mizo 91  mizo91@gmail.com :

>@Tamar.Christina
>I have created public repository on github and placed 'procmon' log file there:
>https://github.com/fskoras/CreateProcess-gcc-issue/blob/master/Logfile.CSV
>You can also clone the repository if you want to try to recreate the issue:
>https://github.com/fskoras/CreateProcess-gcc-issue
>I have executed the as.exe directly and it worked without problems:
>D:/Tools/x86_64-12.2.0-release-win32-seh-rt_v10-rev0/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/as.exe --version
>GNU assembler (GNU Binutils) 2.38
>Copyright (C) 2022 Free Software Foundation, Inc.
>This program is free software; you may redistribute it under the terms of
>the GNU General Public License version 3 or later.
>This program has absolutely no warranty.
>This assembler was configured for a target of `x86_64-w64-mingw32'.
>@fedor_qd
>I have checked version 11.2.0 and the same problem occurs
>
>binutils version used for both versions:
>gcc x86_64-11.2.0: GNU assembler (GNU Binutils) 2.37
>gcc x86_64-12.2.0: GNU assembler (GNU Binutils) 2.38
>
>R.
>
>
>śr., 21 wrz 2022 o 04:42 < fedor_qd@mail.ru> napisał(a):
>>I saw such error in my build gcc 12.1.0 for Symbian. GCC 11.2.0 works fine.
>>I encounter such error while building ScummVM for Symbian. This error happened randomly. I thought my environment broken and switched to GCC 11.2.0 and errors gone.
>>GCC 11.2.0 comes with binutils 2.35, GCC 12.1.0 comes with binutils 2.29.1
>>What binutils version you have?
>>--
>>Sent from Outlook Email App for Android серада, 21 верасня 2022, 01:29AM +03:00 from Tamar Christina via Gcc-help  gcc-help@gcc.gnu.org :
>>
>>>We'd mostly be guessing here, use strace or  https://learn.microsoft.com/en-us/sysinternals/downloads/procmon to see which path is actually being accessed and why it failed.
>>>
>>>Also check that you can run `as` correctly. This error can also happen if a dependency of `as.exe` isn't found during loading.
>>>
>>>Cheers,
>>>Tamar
>>>
> -----Original Message-----
> From: Gcc-help < gcc-help-bounces+tamar.christina=arm.com@gcc.gnu.org>
> On Behalf Of mizo 91 via Gcc-help
> Sent: Tuesday, September 20, 2022 1:17 PM
> To: Richard Earnshaw < Richard.Earnshaw@foss.arm.com>
> Cc:  gcc-help@gcc.gnu.org
> Subject: Re: CreateProcess No such file or directory
>
> According to Microsoft docs path length limitation for windows is 256. In my
> case its 143 chars. I think the problem might be related to how gcc driver is
> passing along unmarshalled arguments to its subprograms like as.exe.
> Suprisingly this works correctly if I enforce gcc driver to use preprocessor only
> by adding -E flag
>
> btw. I failed to previously mention that I'm using Mingw-W64 version
> provided kindly by niXman Binaries taken directly from official MinGW-W64
> builds:
> https://www.mingw-w64.org/downloads/#mingw-builds
> https://github.com/niXman/mingw-builds-binaries/releases
>
> R.
>
> wt., 20 wrz 2022 o 18:18 Richard Earnshaw
>< Richard.Earnshaw@foss.arm.com>
> napisał(a):
>
>>
>>
>> On 20/09/2022 17:02, mizo 91 via Gcc-help wrote:
>>> Hello,
>>>
>>> I'm having trouble compiling simple test program on windows 10 with
>>> long list of includes provided via '@response_file' argument
>>>
>>> The full error is:
>>>
>>> gcc.exe: fatal error: cannot execute
>>>
>> 'C:/Tools/x86_64-12.2.0-release-win32-seh-rt_v10-
> rev0/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-
> w64-mingw32/bin/as.exe':
>>> CreateProcess: No such file or directory
>>>
>>> compilation terminated.
>>>
>>> Above path to the assembler program is valid therefore I don't
>>> understand what is the issue here?
>>>
>>> You can try to recreate the problem by compiling simple main.c file
>>> with nothing in it but main function using command:
>>> gcc @includes.txt main.c
>>>
>>> I assume it has something to do with windows 32k CreateProcess
>>> command length limitation? But isn't that the reason why respone
>>> files were invented?
>>>
>>> Regards,
>>> Filip
>>
>> I'm by no means a windows expert, but I wonder if this might be
>> related to the length of the path to the assembler and that it's
>> somehow getting truncated.
>>
>> R.
>>

^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2022-09-26  6:58 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-20 16:02 CreateProcess No such file or directory mizo 91
2022-09-20 16:18 ` Richard Earnshaw
2022-09-20 17:17   ` mizo 91
2022-09-20 22:27     ` Tamar Christina
2022-09-21  2:42       ` fedor_qd
2022-09-21 15:27         ` mizo 91
2022-09-21 15:41           ` Tom Kacvinsky
2022-09-26  6:58           ` Re[2]: " fedor_qd
2022-09-22  6:44 ` LIU Hao
2022-09-22  7:35   ` mizo 91
2022-09-22  9:46     ` LIU Hao
2022-09-22 14:55       ` mizo 91
2022-09-22 16:04         ` LIU Hao
2022-09-22 16:50           ` mizo 91
2022-09-23 17:40             ` Xi Ruoyao
2022-09-23 21:11               ` mizo 91
2022-09-24  5:13                 ` Xi Ruoyao
2022-09-24  9:28                   ` mizo 91
2022-09-24  9:51                     ` Xi Ruoyao
2022-09-24 10:20                       ` Jonathan Wakely
2022-09-23 17:14         ` David Brown
2022-09-23 20:55           ` mizo 91
2022-09-22  8:19   ` Jonathan Wakely
2022-09-22  8:42     ` Jonathan Wakely
2022-09-22  9:48       ` LIU Hao
2022-09-22  9:50       ` mizo 91

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).