Pipes bug when spawning non-cygwin processes

Pipes bug when spawning non-cygwin processes

[Edward Lam]

Hi Folks,

I'm getting a problem where cygwin parent processes spawning non-cygwin
child processes no longer detect when stdin has been closed. Please see the
sample python code at the end where I've isolated the problem. I've got
cygwin's python2 running spawn_bar.py that popen's a native non-cygwin
python2 running bar.py. The steps to reproduce are to run this command
using the two files detailed at the end of this email:

$ python2 spawn_bar.py

On Windows 10, this command quits right away as expected when using cygwin
3.0.7 but hangs when using cygwin 3.1.2 (version number as reported from
running `cygcheck -c cygwin`).

Looking through the release logs, nothing sticks out except for the change
in 3.1.0 that mentions PTY changes. However, when I originally ran into
this, it wasn't even being done through a terminal but some background
script.

Any ideas?

Thanks,
-Edward

# spawn_bar.py
import os
import subprocess
import sys
python = "python2" # this works
python = "/path/to/native/python2" # this hangs
p = subprocess.Popen([python, 'bar.py'], stdin=subprocess.PIPE)
p.stdin.close()
pid, status = os.waitpid(p.pid, 0)

# bar.py
import sys
for line in sys.stdin:
    print 'got: ' + line

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Pipes bug when spawning non-cygwin processes

[Takashi Yano]

On Thu, 30 Jan 2020 14:03:50 -0500
Edward Lam wrote:
> I'm getting a problem where cygwin parent processes spawning non-cygwin
> child processes no longer detect when stdin has been closed. Please see the
> sample python code at the end where I've isolated the problem. I've got
> cygwin's python2 running spawn_bar.py that popen's a native non-cygwin
> python2 running bar.py. The steps to reproduce are to run this command
> using the two files detailed at the end of this email:
> 
> $ python2 spawn_bar.py
> 
> On Windows 10, this command quits right away as expected when using cygwin
> 3.0.7 but hangs when using cygwin 3.1.2 (version number as reported from
> running `cygcheck -c cygwin`).
> 
> Looking through the release logs, nothing sticks out except for the change
> in 3.1.0 that mentions PTY changes. However, when I originally ran into
> this, it wasn't even being done through a terminal but some background
> script.
> 
> Any ideas?

Probably, this is the same issue as
https://www.cygwin.com/ml/cygwin/2020-01/msg00093.html.

Please try latest cygwin snapshot.
https://cygwin.com/snapshots/

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Pipes bug when spawning non-cygwin processes

[Edward Lam]

On Thu, Jan 30, 2020 at 5:25 PM Takashi Yano <takashi.yano@nifty.ne.jp>
wrote:

> Probably, this is the same issue as
> https://www.cygwin.com/ml/cygwin/2020-01/msg00093.html.
>
> Please try latest cygwin snapshot.
> https://cygwin.com/snapshots/
>
>
Indeed, this works again after  20202401 snapshot!

Thanks,
-Edward

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Pipes bug when spawning non-cygwin processes

[Edward Lam]

Hi Folks,

I'm noticing a different problem with pipes that isn't fixed even in the
latest 2020-02-12 cygwin1.dll snapshot. When a native graphical Windows
process is launched (in the foreground without & in the command line) from
mintty/bash, the STARTUPINFO structure for the child Windows process
correctly has the STARTF_USESTDHANDLES bit set into info.dwFlags BUT both
info.hStdInput and info.hStdOutput handles are 0, only info.hStdError is
assigned to a handle. So in the older cygwin versions, this application's
printf() calls go out to the shell as desired but not anymore.

Any ideas?

-Edward

On Fri, Jan 31, 2020 at 1:25 AM Edward Lam <triplequadsupport@gmail.com>
wrote:

> On Thu, Jan 30, 2020 at 5:25 PM Takashi Yano <takashi.yano@nifty.ne.jp>
> wrote:
>
>> Probably, this is the same issue as
>> https://www.cygwin.com/ml/cygwin/2020-01/msg00093.html.
>>
>> Please try latest cygwin snapshot.
>> https://cygwin.com/snapshots/
>>
>>
> Indeed, this works again after  20202401 snapshot!
>
> Thanks,
> -Edward
>


-- 
Support GOToronto by rating us in the App Store!

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Pipes bug when spawning non-cygwin processes

[Takashi Yano]

On Tue, 18 Feb 2020 14:24:38 -0500
Edward Lam wrote:
> I'm noticing a different problem with pipes that isn't fixed even in the
> latest 2020-02-12 cygwin1.dll snapshot. When a native graphical Windows
> process is launched (in the foreground without & in the command line) from
> mintty/bash, the STARTUPINFO structure for the child Windows process
> correctly has the STARTF_USESTDHANDLES bit set into info.dwFlags BUT both
> info.hStdInput and info.hStdOutput handles are 0, only info.hStdError is
> assigned to a handle. So in the older cygwin versions, this application's
> printf() calls go out to the shell as desired but not anymore.

Could you please provide a simple test case?

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Pipes bug when spawning non-cygwin processes

[Edward Lam]

Hi Takashi,

On Tue, Feb 18, 2020 at 2:43 PM Takashi Yano wrote:

> Could you please provide a simple test case?


Here you go:

// pipes.cpp
//
// Compile in a  Visual Studio x64 Native Tools Command Prompt:
//     cl pipes.cpp /link /subsystem:windows
//
// Run from inside a Cygwin shell, the produced pipes.exe
//

#include <stdio.h>
#include <windows.h>

INT WinMain(HINSTANCE, HINSTANCE, PSTR, INT)
{
    printf("This message used to show up in mintty cygwin v.2.11.2 shell!
or from ssh session\n");
    return 0;
}
// end of pipes.cpp

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Pipes bug when spawning non-cygwin processes

[Takashi Yano]

On Thu, 20 Feb 2020 14:33:27 -0500
Edward Lam wrote:
> On Tue, Feb 18, 2020 at 2:43 PM Takashi Yano wrote:
> > Could you please provide a simple test case?
> 
> Here you go:
> 
> // pipes.cpp
> //
> // Compile in a  Visual Studio x64 Native Tools Command Prompt:
> //     cl pipes.cpp /link /subsystem:windows
> //
> // Run from inside a Cygwin shell, the produced pipes.exe
> //
> 
> #include <stdio.h>
> #include <windows.h>
> 
> INT WinMain(HINSTANCE, HINSTANCE, PSTR, INT)
> {
>     printf("This message used to show up in mintty cygwin v.2.11.2 shell!
> or from ssh session\n");
>     return 0;
> }
> // end of pipes.cpp

Thanks for the test case. Indeed, this works upto cygwin 3.0.7,
and does not work in cygwin 3.1.0 or later.

However, I wonder what platform is your program for. This test
case does not work also in native windows command prompt.
Your test case works only in old cygwin pty.

If you want to make a program which works in cygwin pty, you
can use cygwin g++ like:
g++ -mwindows pipes.cpp -o pipes
The binary built by above command works in cygwin pty, but does
not work in cygwin console (cygwin in command prompt) even with
cygwin 3.0.7.

If you want to make a program which works with windows console,
you should change the code like:

INT WinMain(HINSTANCE, HINSTANCE, PSTR, INT)
{
    if (!AttachConsole(ATTACH_PARENT_PROCESS)) AllocConsole();
    freopen("CONOUT$", "w", stdout);
    printf("This message used to show up in mintty cygwin v.2.11.2 shell! or from ssh session\n");
    return 0;
}

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Pipes bug when spawning non-cygwin processes

[Takashi Yano]

On Fri, 21 Feb 2020 10:01:21 +0900
Takashi Yano wrote:
> On Thu, 20 Feb 2020 14:33:27 -0500
> Edward Lam wrote:
> > On Tue, Feb 18, 2020 at 2:43 PM Takashi Yano wrote:
> > > Could you please provide a simple test case?
> > 
> > Here you go:
> > 
> > // pipes.cpp
> > //
> > // Compile in a  Visual Studio x64 Native Tools Command Prompt:
> > //     cl pipes.cpp /link /subsystem:windows
> > //
> > // Run from inside a Cygwin shell, the produced pipes.exe
> > //
> > 
> > #include <stdio.h>
> > #include <windows.h>
> > 
> > INT WinMain(HINSTANCE, HINSTANCE, PSTR, INT)
> > {
> >     printf("This message used to show up in mintty cygwin v.2.11.2 shell!
> > or from ssh session\n");
> >     return 0;
> > }
> > // end of pipes.cpp
> 
> Thanks for the test case. Indeed, this works upto cygwin 3.0.7,
> and does not work in cygwin 3.1.0 or later.
> 
> However, I wonder what platform is your program for. This test
> case does not work also in native windows command prompt.
> Your test case works only in old cygwin pty.
> 
> If you want to make a program which works in cygwin pty, you
> can use cygwin g++ like:
> g++ -mwindows pipes.cpp -o pipes
> The binary built by above command works in cygwin pty, but does
> not work in cygwin console (cygwin in command prompt) even with
> cygwin 3.0.7.
> 
> If you want to make a program which works with windows console,
> you should change the code like:
> 
> INT WinMain(HINSTANCE, HINSTANCE, PSTR, INT)
> {
>     if (!AttachConsole(ATTACH_PARENT_PROCESS)) AllocConsole();
>     freopen("CONOUT$", "w", stdout);
>     printf("This message used to show up in mintty cygwin v.2.11.2 shell! or from ssh session\n");
>     return 0;
> }

Or just compile with:
cl pipes.cpp /link /subsystem:console

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Pipes bug when spawning non-cygwin processes

[Takashi Yano]

On Fri, 21 Feb 2020 10:07:59 +0900
Takashi Yano wrote:
> Or just compile with:
> cl pipes.cpp /link /subsystem:console

Sorry, you need main() for this.

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Pipes bug when spawning non-cygwin processes

[Edward Lam]

Hi,

Thank you for your reply.

On Thu, Feb 20, 2020 at 8:01 PM Takashi Yano <takashi.yano@nifty.ne.jp>
wrote:

> Thanks for the test case. Indeed, this works upto cygwin 3.0.7,
> and does not work in cygwin 3.1.0 or later.
>
> However, I wonder what platform is your program for. This test
> case does not work also in native windows command prompt.
> Your test case works only in old cygwin pty.
>

I have a native graphical Windows application that relied on cygwin/mintty
providing stdout so that my printf's work.

I would contend that there's no reason for cygwin/mintty to work the same
as the native windows command prompt because the behaviour there is for it
to detach the child process. This is not the case when we run a graphical
process in a cygwin/mintty shell. The old behaviour was much welcome
because it allowed graphical applications that did not detach from the
console work just like on Linux.

I should note that there is also the odd behaviour that the child process
is created with an inherited stderr handle but when I try to write to it,
it doesn't show up anywhere. On the native windows command prompt, the
child process isn't even created with the inherit handles flag enabled.

If you want to make a program which works in cygwin pty, you
> can use cygwin g++ like:
> g++ -mwindows pipes.cpp -o pipes
> The binary built by above command works in cygwin pty, but does
> not work in cygwin console (cygwin in command prompt) even with
> cygwin 3.0.7.
>

This might not have worked in cygwin 3.0.07 because it's missing a
freopen("CONOUT$", "w", stdout). I had omitted this in my minimal test case
because I found that this was unnecessary when I built with Visual Studio.

If you want to make a program which works with windows console,
> you should change the code like:
>
> INT WinMain(HINSTANCE, HINSTANCE, PSTR, INT)
> {
>     if (!AttachConsole(ATTACH_PARENT_PROCESS)) AllocConsole();
>     freopen("CONOUT$", "w", stdout);
>     printf("This message used to show up in mintty cygwin v.2.11.2 shell!
> or from ssh session\n");
>     return 0;
> }
>

Ah, I did not now about AttachConsole(ATTACH_PARENT_PROCESS).

The application that I ship out to my users cannot force the requirement of
having cygwin, nor is compiling with cygwin g++ an option. Is there some
way to have a non-cygwin app detect the presence of cygwin pty so that it
can behave differently? When the native command prompt detaches the child
process, the child process should not really write to the parent's console
and should allocate its own instead.

Thanks again,
-Edward

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple