From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 2155) id 7F8E4385843A; Thu, 20 Apr 2023 14:21:17 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 7F8E4385843A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com; s=default; t=1682000477; bh=YEdHVpO/Pe9saTyKWrqk1q21MMa3zETu3J8OWxYx8ic=; h=Date:From:To:Cc:Subject:Reply-To:References:In-Reply-To:From; b=luZBcKyYNbiHnOwX1Amylb3jHal9v8oDSFZ8CTIJDNefx6rAnfxtei+lOZ4S8WKR1 3R+o7ipUjXhVSJGHVo8cAz2OeUXF5AvA1XF24WQlgUon5tEQvXx98oz/E+udSvAy46 ssz2qHwP/5qxcP64RFKMgZVwxZ+XgvT9PDuHMgm0= Received: by calimero.vinschen.de (Postfix, from userid 500) id 5DE91A80B9C; Thu, 20 Apr 2023 16:21:15 +0200 (CEST) Date: Thu, 20 Apr 2023 16:21:15 +0200 From: Corinna Vinschen To: Bruno Haible Cc: cygwin@cygwin.com Subject: Re: posix_spawn facility Message-ID: Reply-To: cygwin@cygwin.com Mail-Followup-To: Bruno Haible , cygwin@cygwin.com References: <1752276.7aRn1RRit1@nimes> <1853259.CQOukoFCf9@nimes> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <1853259.CQOukoFCf9@nimes> List-Id: On Apr 20 12:18, Bruno Haible via Cygwin wrote: > Corinna Vinschen wrote: > > Unfortunately you can't expect any noticable difference on Cygwin by > > using posix_spawn. While Cygwin has a spawn() family of functions, we > > don't (and can't... yet) use them. > > > > The problem is that we don't have a safe way to perform the spawn > > attributes and file actions which are supposed to be performed between > > the fork() and the exec() step when using the spawn() functions. This > > would have to be implemented first. > > So, you are saying that on Cygwin, spawnvpe() and friends are fast, > because child_info_spawn::worker ends up calling CreateProcessW rather > immediately? But posix_spawn is slow, because it goes another path, > through fork()? Basically, it's still fork/exec, just with some additional code to handle process synchronization so we can propagate the error from a failing execve to the parent process. > If that's the case, it might help to look at how I created Gnulib's > posix_spawn for native Windows: Great, thanks for that. I'm a bit busy ATM, but I'll take a look, this might be very helpful. > 1) Extend the spawnvpe function to a spawnvpech function. It takes > additional arguments > - const char *currdir > - int currdirfd > - HANDLE stdin_handle, HANDLE stdout_handle, HANDLE stderr_handle > (The latter is because the first 3 file descriptors are passed > specially on Windows.) > > 2) Refactor the guts of spawnvpech, moving as much as possible into > helper functions. In my case, spawnvpech shrunk to only 150 lines > of code. > > The helper functions, in the native Windows case, were: > - prepare_spawn > - compose_command > - compose_envblock > - init_inheritable_handles, compose_handles_block, > free_inheritable_handles > - convert_CreateProcess_error > See https://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blob;f=lib/windows-spawn.h . > > I would expect that similar helper functions can be created in > Cygwin. The additional problem in Cygwin is that we also have to care for the POSIX stuff, like signals, effective vs. real UID/GID, etc. > The "inheritable handles" is a data structure that allows for the > arbitrary reshuffling of file descriptors required by posix_spawn > (the addopen, adddup2, addclose actions), i.e. which does the book- > keeping which HANDLE must in the end be open in the parent and which > must in the end be open in the child, and at which position. Hmm. Your code uses lpReserved2 for that, but the functionality is one implemented in MSVCRT. For obvious reasons, Cygwin executables are not linked against msvcrt.dll and we're using lpReserved2 for our own purposes. While we probably could implement the same mechanism, we would have to implement the target process side as well, and the result would be incompatible with MSVCRT. We also have virtual files with handles pointing to \Device\Null (just so an inheritable handle is available) Additionally, we already propagate complete fhandlers (the datastructures the descriptors in Cygwin are pointing to) via the Cygwin heap. The annoying thing here is that we probably need tow different mechanisms, one for Cygwin POSIX child processes, and one for native Windows child processes. > During this refactoring, it is essential to have a good test suite. > Because when you make a mistake and leave a fd / HANDLE accidentally > open (in the parent or in the child), applications that invoke a pipe > start to hang. > > 3) With these helper functions, write a new core for posix_spawn[p]. > In my case, it was less than 250 lines of code, which is not > straightforward, but also not really hard either. > See https://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blob;f=lib/spawni.c > lines 610..852. > > At this step, you can use Gnulib's test suite for verifying that the new > code works properly. > > Hope this helps. I think so, yes, thanks a lot! Corinna