Native posix_spawn() in Cygwin?

Native posix_spawn() in Cygwin?

[Dan Shelton]

Hello!

Does Cygwin implement a native, i.e. without form(),exec(),
implementation of posix_spawn()?

Dan
-- 
Dan Shelton - Cluster Specialist Win/Lin/Bsd

Re: Native posix_spawn() in Cygwin?

[Brian Inglis]

On 2024-02-26 20:23, Dan Shelton via Cygwin wrote:
> Does Cygwin implement a native, i.e. without form(),exec(), implementation of posix_spawn()?

Check the API compatibility docs online:

	https://cygwin.com/cygwin-api/compatibility.html#std-susv4

or optional locally installed package cygwin-doc:

	/usr/share/doc/cygwin-doc/html/cygwin-api/compatibility.html#std-susv4

-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer     but when there is no more to cut
                                 -- Antoine de Saint-Exupéry

Re: Native posix_spawn() in Cygwin?

[Dan Shelton]

On Tue, 27 Feb 2024 at 06:47, Brian Inglis via Cygwin <cygwin@cygwin.com> wrote:
>
> On 2024-02-26 20:23, Dan Shelton via Cygwin wrote:
> > Does Cygwin implement a native, i.e. without form(),exec(), implementation of posix_spawn()?
>
> Check the API compatibility docs online:
>
>         https://cygwin.com/cygwin-api/compatibility.html#std-susv4
>
> or optional locally installed package cygwin-doc:
>
>         /usr/share/doc/cygwin-doc/html/cygwin-api/compatibility.html#std-susv4

That document does not answer my question.

I know posix_spawn() is there. But the question is: Does it use just
Cygwin fork(),exec(), or the native Win32 spawn() api?

Dan
-- 
Dan Shelton - Cluster Specialist Win/Lin/Bsd

Re: Native posix_spawn() in Cygwin?

[gs-cygwin.com]

On Tue, Feb 27, 2024 at 06:54:42AM +0100, Dan Shelton via Cygwin wrote:
> On Tue, 27 Feb 2024 at 06:47, Brian Inglis via Cygwin <cygwin@cygwin.com> wrote:
> >
> > On 2024-02-26 20:23, Dan Shelton via Cygwin wrote:
> > > Does Cygwin implement a native, i.e. without form(),exec(), implementation of posix_spawn()?
> >
> > Check the API compatibility docs online:
> >
> >         https://cygwin.com/cygwin-api/compatibility.html#std-susv4
> >
> > or optional locally installed package cygwin-doc:
> >
> >         /usr/share/doc/cygwin-doc/html/cygwin-api/compatibility.html#std-susv4
> 
> That document does not answer my question.
> 
> I know posix_spawn() is there. But the question is: Does it use just
> Cygwin fork(),exec(), or the native Win32 spawn() api?
> 
> Dan
> -- 
> Dan Shelton - Cluster Specialist Win/Lin/Bsd

If you were going to make a small effort to answer the question
yourself, you could use strace, you could step through a debugger, or
you could check the source code.  Have you tried any of these?  What did
you find?  If you are unable to take any of those steps, why does
posix_spawn() matter to you?

Cheers, Glenn

Re: Native posix_spawn() in Cygwin?

[Dan Shelton]

On Tue, 27 Feb 2024 at 07:34, <gs-cygwin.com@gluelogic.com> wrote:
>
> On Tue, Feb 27, 2024 at 06:54:42AM +0100, Dan Shelton via Cygwin wrote:
> > On Tue, 27 Feb 2024 at 06:47, Brian Inglis via Cygwin <cygwin@cygwin.com> wrote:
> > >
> > > On 2024-02-26 20:23, Dan Shelton via Cygwin wrote:
> > > > Does Cygwin implement a native, i.e. without form(),exec(), implementation of posix_spawn()?
> > >
> > > Check the API compatibility docs online:
> > >
> > >         https://cygwin.com/cygwin-api/compatibility.html#std-susv4
> > >
> > > or optional locally installed package cygwin-doc:
> > >
> > >         /usr/share/doc/cygwin-doc/html/cygwin-api/compatibility.html#std-susv4
> >
> > That document does not answer my question.
> >
> > I know posix_spawn() is there. But the question is: Does it use just
> > Cygwin fork(),exec(), or the native Win32 spawn() api?
> >
> > Dan
> > --
> > Dan Shelton - Cluster Specialist Win/Lin/Bsd
>
> If you were going to make a small effort to answer the question
> yourself, you could use strace, you could step through a debugger, or
> you could check the source code.  Have you tried any of these?  What did
> you find?  If you are unable to take any of those steps, why does
> posix_spawn() matter to you?

strace does not help, as I need the Win32 calls BELOW posix_spawn(),
to see the implementation details.

Dan
-- 
Dan Shelton - Cluster Specialist Win/Lin/Bsd

Re: Native posix_spawn() in Cygwin?

[Mark Geisert]

On 3/3/2024 7:27 PM, Dan Shelton via Cygwin wrote:
> On Tue, 27 Feb 2024 at 07:34, <gs-cygwin.com@gluelogic.com> wrote:
>>
>> On Tue, Feb 27, 2024 at 06:54:42AM +0100, Dan Shelton via Cygwin wrote:
>>> On Tue, 27 Feb 2024 at 06:47, Brian Inglis via Cygwin <cygwin@cygwin.com> wrote:
>>>>
>>>> On 2024-02-26 20:23, Dan Shelton via Cygwin wrote:
>>>>> Does Cygwin implement a native, i.e. without form(),exec(), implementation of posix_spawn()?
>>>>
>>>> Check the API compatibility docs online:
>>>>
>>>>          https://cygwin.com/cygwin-api/compatibility.html#std-susv4
>>>>
>>>> or optional locally installed package cygwin-doc:
>>>>
>>>>          /usr/share/doc/cygwin-doc/html/cygwin-api/compatibility.html#std-susv4
>>>
>>> That document does not answer my question.
>>>
>>> I know posix_spawn() is there. But the question is: Does it use just
>>> Cygwin fork(),exec(), or the native Win32 spawn() api?
>>>
>>> Dan
>>> --
>>> Dan Shelton - Cluster Specialist Win/Lin/Bsd
>>
>> If you were going to make a small effort to answer the question
>> yourself, you could use strace, you could step through a debugger, or
>> you could check the source code.  Have you tried any of these?  What did
>> you find?  If you are unable to take any of those steps, why does
>> posix_spawn() matter to you?
> 
> strace does not help, as I need the Win32 calls BELOW posix_spawn(),
> to see the implementation details.

Check the source code, then. It's at:
     https://cygwin.com/cgit/newlib-cygwin/tree/winsup/cygwin/fork.cc

Look at line 587; there's the static function dofork(). Look at the
thirty or so lines above that; there's both fork() and 
__posix_spawn_fork() calling dofork(). So both those user-level
functions call into the exact same internals. (BTW __posix_spawn_fork() 
is called from posix_spawn(); the latter is in newlib and not Cygwin.)

You can even see the reason it's done this way by reading the comment.

..mark

Re: Native posix_spawn() in Cygwin?

[Dan Shelton]

On Mon, 4 Mar 2024 at 07:45, Mark Geisert via Cygwin <cygwin@cygwin.com> wrote:
>
> On 3/3/2024 7:27 PM, Dan Shelton via Cygwin wrote:
> > On Tue, 27 Feb 2024 at 07:34, <gs-cygwin.com@gluelogic.com> wrote:
> >>
> >> On Tue, Feb 27, 2024 at 06:54:42AM +0100, Dan Shelton via Cygwin wrote:
> >>> On Tue, 27 Feb 2024 at 06:47, Brian Inglis via Cygwin <cygwin@cygwin.com> wrote:
> >>>>
> >>>> On 2024-02-26 20:23, Dan Shelton via Cygwin wrote:
> >>>>> Does Cygwin implement a native, i.e. without form(),exec(), implementation of posix_spawn()?
> >>>>
> >>>> Check the API compatibility docs online:
> >>>>
> >>>>          https://cygwin.com/cygwin-api/compatibility.html#std-susv4
> >>>>
> >>>> or optional locally installed package cygwin-doc:
> >>>>
> >>>>          /usr/share/doc/cygwin-doc/html/cygwin-api/compatibility.html#std-susv4
> >>>
> >>> That document does not answer my question.
> >>>
> >>> I know posix_spawn() is there. But the question is: Does it use just
> >>> Cygwin fork(),exec(), or the native Win32 spawn() api?
> >>>
> >>> Dan
> >>> --
> >>> Dan Shelton - Cluster Specialist Win/Lin/Bsd
> >>
> >> If you were going to make a small effort to answer the question
> >> yourself, you could use strace, you could step through a debugger, or
> >> you could check the source code.  Have you tried any of these?  What did
> >> you find?  If you are unable to take any of those steps, why does
> >> posix_spawn() matter to you?
> >
> > strace does not help, as I need the Win32 calls BELOW posix_spawn(),
> > to see the implementation details.
>
> Check the source code, then. It's at:
>      https://cygwin.com/cgit/newlib-cygwin/tree/winsup/cygwin/fork.cc
>
> Look at line 587; there's the static function dofork(). Look at the
> thirty or so lines above that; there's both fork() and
> __posix_spawn_fork() calling dofork(). So both those user-level
> functions call into the exact same internals. (BTW __posix_spawn_fork()
> is called from posix_spawn(); the latter is in newlib and not Cygwin.)
>
> You can even see the reason it's done this way by reading the comment.

Yes, but it is as I feared, Cygwin posix_spawn() does not use Win32
spawn() at all, and instead uses a rather inefficient vfork()
solution.

posix_spawn() was added to POSIX so a Win32 implementation can use Win32 spawn()

Dan
-- 
Dan Shelton - Cluster Specialist Win/Lin/Bsd

Re: Native posix_spawn() in Cygwin?

[Mark Geisert]

On 3/5/2024 2:42 PM, Dan Shelton via Cygwin wrote:
> On Mon, 4 Mar 2024 at 07:45, Mark Geisert via Cygwin <cygwin@cygwin.com> wrote:
>>
>> On 3/3/2024 7:27 PM, Dan Shelton via Cygwin wrote:
[...]
>>> strace does not help, as I need the Win32 calls BELOW posix_spawn(),
>>> to see the implementation details.
>>
>> Check the source code, then. It's at:
>>       https://cygwin.com/cgit/newlib-cygwin/tree/winsup/cygwin/fork.cc
>>
>> Look at line 587; there's the static function dofork(). Look at the
>> thirty or so lines above that; there's both fork() and
>> __posix_spawn_fork() calling dofork(). So both those user-level
>> functions call into the exact same internals. (BTW __posix_spawn_fork()
>> is called from posix_spawn(); the latter is in newlib and not Cygwin.)
>>
>> You can even see the reason it's done this way by reading the comment.
> 
> Yes, but it is as I feared, Cygwin posix_spawn() does not use Win32
> spawn() at all, and instead uses a rather inefficient vfork()
> solution.

Cygwin's vfork() is just a wrapper around fork(), so yes. But anyway...

> posix_spawn() was added to POSIX so a Win32 implementation can use Win32 spawn()

...now I see what you're getting at:

If posix_spawn() is intended to launch truly unrelated processes, with 
minimal or no coordination with the launching process, why can't it just 
use Windows' CreateProcess? I assume here that's what Win32 spawn() does.

That's an interesting research question for somebody. If somebody steps 
up for that, great, otherwise as usual PTC.
Regards,

..mark

Re: Native posix_spawn() in Cygwin?

[Roland Mainz]

On Wed, Mar 6, 2024 at 1:08 AM Mark Geisert via Cygwin
<cygwin@cygwin.com> wrote:
> On 3/5/2024 2:42 PM, Dan Shelton via Cygwin wrote:
> > On Mon, 4 Mar 2024 at 07:45, Mark Geisert via Cygwin <cygwin@cygwin.com> wrote:
> >> On 3/3/2024 7:27 PM, Dan Shelton via Cygwin wrote:
> [...]
> >>> strace does not help, as I need the Win32 calls BELOW posix_spawn(),
> >>> to see the implementation details.
> >>
> >> Check the source code, then. It's at:
> >>       https://cygwin.com/cgit/newlib-cygwin/tree/winsup/cygwin/fork.cc
> >>
> >> Look at line 587; there's the static function dofork(). Look at the
> >> thirty or so lines above that; there's both fork() and
> >> __posix_spawn_fork() calling dofork(). So both those user-level
> >> functions call into the exact same internals. (BTW __posix_spawn_fork()
> >> is called from posix_spawn(); the latter is in newlib and not Cygwin.)
> >>
> >> You can even see the reason it's done this way by reading the comment.
> >
> > Yes, but it is as I feared, Cygwin posix_spawn() does not use Win32
> > spawn() at all, and instead uses a rather inefficient vfork()
> > solution.
>
> Cygwin's vfork() is just a wrapper around fork(), so yes. But anyway...
>
> > posix_spawn() was added to POSIX so a Win32 implementation can use Win32 spawn()
>
> ...now I see what you're getting at:
>
> If posix_spawn() is intended to launch truly unrelated processes, with
> minimal or no coordination with the launching process, why can't it just
> use Windows' CreateProcess? I assume here that's what Win32 spawn() does.
>
> That's an interesting research question for somebody. If somebody steps
> up for that, great, otherwise as usual PTC.
> Regards,

Just one note (which applies to UNIX/Linux):
When I was at SUN we had severe performance problems with large
JAVA&&database processes launching little helper apps. It turned out
to be a |fork()|+|exec()| problem - the |fork()| was basically
harmless, but the |exec()| syscall required to tear down all address
space, which involved cross-calls to all other CPUs. The performance
penalty on a 64 CPU Enterprise 10000 or 72 CPU (up to 144 threads)
SF25k was *DEVASTATING* ( <--- understatement).

This was fixed for Solaris 11 in several ways:
1. Solaris 11 got a native |posix_spawn()| syscall, and which avoids
both |fork()| and the (in this context) dreaded |exec()|, and just
sets up a new process with copying only the bare minimum of data from
the parent process.
2. Solaris's ksh93 (which is used as /sbin/sh+/bin/sh) got support for
|posix_spawn()| (thanks for David Korn and Glenn Fowler for doing
that)
3. JAVA was modified to use |posix_spawn()|
4. Oracle was asked to use |posix_spawn()| too

For Cygwin I think it would be good to implement a |posix_spawn()|
using the native Win32 API too...

----

Bye,
Roland
-- 
  __ .  . __
 (o.\ \/ /.o) roland.mainz@nrubsig.org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)