* Why does scp leave ssh running? @ 2000-11-27 12:26 Patrick Doyle 2000-11-28 2:12 ` Graham Bloice 2000-11-29 19:21 ` Why does scp leave ssh running? -- select() never returns Patrick Doyle 0 siblings, 2 replies; 20+ messages in thread From: Patrick Doyle @ 2000-11-27 12:26 UTC (permalink / raw) To: cygwin When I run scp somefile host:. I am left with a copy of SSH running. At first, I thought this was because "scp" negotiated a connection with the remote host using ssh and then left that connection open in the background in case I wanted to perform more remote copies. But experimentation has led me to believe that each time I perform a copy, another ssh gets left running. Also interesting is that if I copy _from_ the remote computer to my local computer, no copy of ssh is left running. The problem is, on my W98 boxes, I cannot quit out of my bash prompt until I kill all of the pesky ssh's left lying around. I am running cygwin-1.1.6-1 and openssh-2.3.0p1-1. 1) Is this known behavior? 2) Is this expected behavior? 3) Does anybody care to see the output of cygcheck -s to consider this problem? I will track down and see if I can diagnose and correct the root of this problem myself if nobody else has heard of it and/or nobody is working on solving it. --wpd -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: Why does scp leave ssh running? 2000-11-27 12:26 Why does scp leave ssh running? Patrick Doyle @ 2000-11-28 2:12 ` Graham Bloice 2000-11-29 19:21 ` Why does scp leave ssh running? -- select() never returns Patrick Doyle 1 sibling, 0 replies; 20+ messages in thread From: Graham Bloice @ 2000-11-28 2:12 UTC (permalink / raw) To: cygwin > When I run > > scp somefile host:. > > I am left with a copy of SSH running. At first, I thought this was > because "scp" negotiated a connection with the remote host using ssh and > then left that connection open in the background in case I wanted to > perform more remote copies. But experimentation has led me to believe > that each time I perform a copy, another ssh gets left running. Also > interesting is that if I copy _from_ the remote computer to my local > computer, no copy of ssh is left running. The problem is, on my W98 > boxes, I cannot quit out of my bash prompt until I kill all of the pesky > ssh's left lying around. > > I am running cygwin-1.1.6-1 and openssh-2.3.0p1-1. > > 1) Is this known behavior? > 2) Is this expected behavior? > 3) Does anybody care to see the output of cygcheck -s to consider this > problem? > > I will track down and see if I can diagnose and correct the root of this > problem myself if nobody else has heard of it and/or nobody is working > on solving it. > This is very similar to the problem I and others have reported with using CVS over ssh on win98. When a cvs command such as 'cvs update' is executed, the command runs successfully to completion with all it's normal output, but control isn't returned to the bash prompt. Using task manager reveals a copy of ssh still running, and killing that returns control to the prompt. he trick is knowing when the cvs update has completed and it's safe to kill ssh. I've done some limited debugging that seemed to indicate that cvs was attempting to shutdown the ssh connection, and was checking to see if there were any characters left in the pipe. At this point cvs hangs in a getc(). I haven't had time to take it further, but I have my suspicions about pipes and win98. I attempted to extract the essence of what cvs was doing with ssh to a separate program which of course ran perfectly :-( Graham -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-11-27 12:26 Why does scp leave ssh running? Patrick Doyle 2000-11-28 2:12 ` Graham Bloice @ 2000-11-29 19:21 ` Patrick Doyle 2000-11-29 19:41 ` Christopher Faylor 1 sibling, 1 reply; 20+ messages in thread From: Patrick Doyle @ 2000-11-29 19:21 UTC (permalink / raw) To: cygwin Patrick Doyle wrote: > > When I run > > scp somefile host:. > > I am left with a copy of SSH running. ... I believe that have traced this problem down to the call to select() in ssh. Basically, it appears that ssh calls select() on a pipe. If the writer of that pipe closes it, select() never returns. I have attached code to the end of this email demonstrating this phenomenon. I believe that I have traced this down to the call to PeekNamedPipe() in "select.cc". For some reason, on my W98 box, PeekNamedPipe() does not return an error when the other end of the pipe is closed. So, a few questions... 1) Is this behavior (of PeekNamedPipe()) a W9x bug? (That is, does it work correctly on NT/2K?) 2) What is the expected behavior of select() on a pipe when the writer closes the pipe? The code in ssh seems to imply that it should return, probably with the "read" bit set for the closed pipe. 3) Does anybody have any ideas of how I might work around this problem, preferably in cygwin? (Otherwise, I could change the call to select() "clientloop.c" to include a timeout). 4) Would folks mind trying this code and confirming if I am on the right track or not? Thanks for any help and/or advice... --wpd code follows... sorry about the leftover #ifdef debug stuff. /************************************************************************ * Simple program that demonstrates the "problem" with select(). Basically, * when a process is blocked on a select() call to read from a pipe, and the * writer for the pipe closes it, the select call never returns (unless, * presumably, the timeout is set. * * This code demonstrats this phenomenon by creating a pipe, forking * a subprocess, closing the write side of the pipe (in the parent process) * and performing a select() on the read side of the pipe. Meanwhile, the * child process closes the pipe, either after a timeout, or upon receipt * if a SIGHUP. ************************************************************************/ #include <stdlib.h> #include <stdio.h> #include <unistd.h> #include <sys/types.h> #include <sys/select.h> #include <signal.h> void sighup(int x) { /* do nothing */ } int main(int argc, char *argv[]) { int pid; int timeout; int pipe_fd[2]; char buf[256]; int bytes_read; int input_fd; fd_set readset; if (argc < 2) { timeout = 5; } else { if (sscanf(argv[1], "%d", &timeout) != 1 || timeout < 0) { fprintf(stderr, "usage: test1 [timeout]\n"); exit(1); } } if (pipe(pipe_fd) < 0) { perror("pipe"); exit(1); } if ((pid = fork()) == 0) { /* We are the child -- wait the specified timeout, write something * to the pipe, wait again, then close the pipe. */ if (timeout == 0) { signal(SIGHUP, sighup); printf("Waiting for signal\n"); sigpause(0); } else { sleep(timeout); } #if 0 printf("child: about to write to pipe\n"); if (write(pipe_fd[1], "hello", sizeof("hello")) != sizeof("hello")) { perror("child: write"); exit(1); } if (timeout == 0) { signal(SIGHUP, sighup); printf("Waiting for signal\n"); sigpause(0); } else { sleep(timeout); } #endif if (close(pipe_fd[1]) < 0) { perror("child: close: pipe"); exit(1); } printf("child: pipe closed, exiting\n"); /* Exit successfully */ exit(0); } else if (pid < 0) { /* The fork failed for some reason */ perror("fork"); exit(1); } /* We are the parent */ close(pipe_fd[1]); #if 1 input_fd = pipe_fd[0]; #else input_fd = 0; /* Read from stdin */ #endif printf("parent: about to select on fd %d for reading\n", input_fd); FD_ZERO(&readset); FD_SET(input_fd, &readset); if (select(pipe_fd[0] + 1, &readset, 0, 0, 0) < 0) { perror("select"); exit(1); } printf("parent: About to read from fd %d\n", input_fd); if ((bytes_read = read(input_fd, buf, sizeof(buf))) < 0) { perror("parent: read"); exit(1); } buf[bytes_read] = 0; printf("read buffer of length %d: %s\n", bytes_read, buf); close(pipe_fd[0]); exit(0); } -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-11-29 19:21 ` Why does scp leave ssh running? -- select() never returns Patrick Doyle @ 2000-11-29 19:41 ` Christopher Faylor 2000-11-30 6:15 ` Patrick Doyle 0 siblings, 1 reply; 20+ messages in thread From: Christopher Faylor @ 2000-11-29 19:41 UTC (permalink / raw) To: cygwin On Wed, Nov 29, 2000 at 10:22:03PM -0500, Patrick Doyle wrote: >So, a few questions... >1) Is this behavior (of PeekNamedPipe()) a W9x bug? (That is, does it >work correctly on NT/2K?) I think you are correct. It is a w9x bug. I vaguely recall seeing this before. I can't think of any way to work around this, unfortunately. I remember trying things like zero byte reads on the pipe but they didn't do anything. The only way to trigger the EOF is to actually read from the pipe. cgf -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-11-29 19:41 ` Christopher Faylor @ 2000-11-30 6:15 ` Patrick Doyle 2000-11-30 6:35 ` Corinna Vinschen 2000-11-30 19:40 ` Patrick Doyle 0 siblings, 2 replies; 20+ messages in thread From: Patrick Doyle @ 2000-11-30 6:15 UTC (permalink / raw) To: cygwin Christopher Faylor wrote: > > On Wed, Nov 29, 2000 at 10:22:03PM -0500, Patrick Doyle wrote: > >So, a few questions... > >1) Is this behavior (of PeekNamedPipe()) a W9x bug? (That is, does it > >work correctly on NT/2K?) > > I think you are correct. It is a w9x bug. I vaguely recall seeing this > before. > > I can't think of any way to work around this, unfortunately. I remember trying > things like zero byte reads on the pipe but they didn't do anything. > > The only way to trigger the EOF is to actually read from the pipe. > > cgf If that is the case, would anybody mind if I posted a (simple) patch to openSSH code which added a timeout to the call(s) to select()? If I do that, will the patch be incorporated into the cygwin release of openSSH? Is there some way to feed that patch back to the openSSH development team? --wpd (Note, I haven't tried this patch yet -- I just thought of it when I was posting my original email last night -- so the patch may not be as "simple" as I envisioned). -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-11-30 6:15 ` Patrick Doyle @ 2000-11-30 6:35 ` Corinna Vinschen 2000-11-30 7:05 ` Patrick Doyle 2000-11-30 19:40 ` Patrick Doyle 1 sibling, 1 reply; 20+ messages in thread From: Corinna Vinschen @ 2000-11-30 6:35 UTC (permalink / raw) To: cygwin Patrick Doyle wrote: > If that is the case, would anybody mind if I posted a (simple) patch to > openSSH code which added a timeout to the call(s) to select()? If I do > that, will the patch be incorporated into the cygwin release of > openSSH? Is there some way to feed that patch back to the openSSH > development team? I am that way. However, your patch should be clean in terms of OS handling. I recommend using the already used construct: #ifdef HAVE_CYGWIN if (! is_winnt) { ...your w9x specific code... } #endif The definition of is_winnt is used eg. in `cygwin_utils.c': #define is_winnt (GetVersion() < 0x80000000) Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Developer mailto:cygwin@cygwin.com Red Hat, Inc. mailto:vinschen@redhat.com -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-11-30 6:35 ` Corinna Vinschen @ 2000-11-30 7:05 ` Patrick Doyle 0 siblings, 0 replies; 20+ messages in thread From: Patrick Doyle @ 2000-11-30 7:05 UTC (permalink / raw) To: cygwin Great...I'll give it a try sometime tonight or over the weekend & hopefully post a patch soon. --wpd Corinna Vinschen wrote: > > Patrick Doyle wrote: > > If that is the case, would anybody mind if I posted a (simple) patch to > > openSSH code which added a timeout to the call(s) to select()? If I do > > that, will the patch be incorporated into the cygwin release of > > openSSH? Is there some way to feed that patch back to the openSSH > > development team? > > I am that way. > > However, your patch should be clean in terms of OS handling. > > I recommend using the already used construct: > > #ifdef HAVE_CYGWIN > if (! is_winnt) > { > ...your w9x specific code... > } > #endif > > The definition of is_winnt is used eg. in `cygwin_utils.c': > > #define is_winnt (GetVersion() < 0x80000000) > > Corinna > > -- > Corinna Vinschen Please, send mails regarding Cygwin to > Cygwin Developer mailto:cygwin@cygwin.com > Red Hat, Inc. > mailto:vinschen@redhat.com > > -- > Want to unsubscribe from this list? > Send a message to cygwin-unsubscribe@sourceware.cygnus.com -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-11-30 6:15 ` Patrick Doyle 2000-11-30 6:35 ` Corinna Vinschen @ 2000-11-30 19:40 ` Patrick Doyle 2000-12-02 17:08 ` Christopher Faylor 1 sibling, 1 reply; 20+ messages in thread From: Patrick Doyle @ 2000-11-30 19:40 UTC (permalink / raw) To: cygwin > Christopher Faylor wrote: > > > > On Wed, Nov 29, 2000 at 10:22:03PM -0500, Patrick Doyle wrote: > > >1) Is this behavior (of PeekNamedPipe()) a W9x bug? (That is, does it > > >work correctly on NT/2K?) > > > > I think you are correct. It is a w9x bug. I vaguely recall seeing this > > before. > > > > I can't think of any way to work around this, unfortunately. I remember trying > > things like zero byte reads on the pipe but they didn't do anything. > > > > The only way to trigger the EOF is to actually read from the pipe. > > > > cgf > If that is the case, would anybody mind if I posted a (simple) patch to > openSSH code which added a timeout to the call(s) to select()? > > (Note, I haven't tried this patch yet -- I just thought of it ... Status update... Adding the timeout to the call to select() did not solve the problem (which is consistent with Christopher's assertion that "The only way to trigger the EOF is to actually read from the pipe". Actually, I wasn't too keen about solving the problem by hacking openSSH; it would just mean that the problem had to be solved again and again for other packages. So now what? Does anybody have any suggestions for other things that I could try? I would prefer to solve this problem in cygwin1.dll, but I am, by no means, an expert windows programmer, so right now, I find myself just flailing. The two ideas I have so far are: 1) Switch from using anonymous pipes to using named pipes with unique names. According to the documentation on the MSDN web site, this is how anonymous pipes are implemented in W2K and NT. Maybe this will solve the problem, maybe it won't. Maybe it will allow me to call a function which will let me know if the other end has closed the pipe. (Most of the functions I have found on the MSDN web site are specific to named pipes). Maybe it won't. 2) Change the thread_pipe to call ReadFile() (reading one byte) instead of PeekNamedPipe() and save the result for the next call to read. This has the advantage that it would probably work (based on Christopher's comment and my own experimentation with read() vs. select()), but it looks like a heck-of-a-lot of work (read: prone to bugs) to solve such a stupid problem. Actually, suggestion #1 suffers from this con as well. Any suggestions, comments, retorts, complaints about me cluttering up your email box with trivialities that you don't care about, would be welcome, er... probably not the last one. --wpd -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-11-30 19:40 ` Patrick Doyle @ 2000-12-02 17:08 ` Christopher Faylor 2000-12-02 19:47 ` Patrick Doyle 0 siblings, 1 reply; 20+ messages in thread From: Christopher Faylor @ 2000-12-02 17:08 UTC (permalink / raw) To: cygwin Sorry for the delay in responding. On Thu, Nov 30, 2000 at 10:41:01PM -0500, Patrick Doyle wrote: >> If that is the case, would anybody mind if I posted a (simple) patch to >> openSSH code which added a timeout to the call(s) to select()? >> >> (Note, I haven't tried this patch yet -- I just thought of it ... > >Status update... > >Adding the timeout to the call to select() did not solve the problem >(which is consistent with Christopher's assertion that "The only way to >trigger the EOF is to actually read from the pipe". Actually, I wasn't >too keen about solving the problem by hacking openSSH; it would just >mean that the problem had to be solved again and again for other >packages. > >So now what? Does anybody have any suggestions for other things that I >could try? I would prefer to solve this problem in cygwin1.dll, but I >am, by no means, an expert windows programmer, so right now, I find >myself just flailing. > >The two ideas I have so far are: >1) Switch from using anonymous pipes to using named pipes with unique >names. According to the documentation on the MSDN web site, this is how >anonymous pipes are implemented in W2K and NT. Maybe this will solve >the problem, maybe it won't. Maybe it will allow me to call a function >which will let me know if the other end has closed the pipe. (Most of >the functions I have found on the MSDN web site are specific to named >pipes). Maybe it won't. You can't create named pipes on Windows 95, unfortunately. >2) Change the thread_pipe to call ReadFile() (reading one byte) instead >of PeekNamedPipe() and save the result for the next call to read. This >has the advantage that it would probably work (based on Christopher's >comment and my own experimentation with read() vs. select()), but it >looks like a heck-of-a-lot of work (read: prone to bugs) to solve such a >stupid problem. Actually, suggestion #1 suffers from this con as well. Your analysis is correct, IMO. I've been mulling this over for a couple of years and have never come up with a satisfactory solution. Even the thread method is prone to difficulties. If you are blocked on a read of a pipe, terminating the thread cleanly becomes problematic. It's been some time since I mentioned that "I hate Windows 9x." This seems like a good place to bring this up. >Any suggestions, comments, retorts, complaints about me cluttering up >your email box with trivialities that you don't care about, would be >welcome, er... probably not the last one. I would be really happy if someone had a creative solution to this problem. It would be nice to fix this in the Cygwin DLL. cgf -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-12-02 17:08 ` Christopher Faylor @ 2000-12-02 19:47 ` Patrick Doyle 2000-12-02 20:39 ` Christopher Faylor 0 siblings, 1 reply; 20+ messages in thread From: Patrick Doyle @ 2000-12-02 19:47 UTC (permalink / raw) To: cygwin Christopher Faylor wrote: > Sorry for the delay in responding. No problem... I haven't had a chance to work on this Friday or today anyway... I did start to get worried when nobody responded 37 minutes after I first posted my message. Isn't that the way this list is supposed to work? ;-) > >The two ideas I have so far are: > >1) Switch from using anonymous pipes to using named pipes with unique > >names. According to the documentation on the MSDN web site, this is how > >... > > You can't create named pipes on Windows 95, unfortunately. > Oh > >2) Change the thread_pipe to call ReadFile() (reading one byte) instead > >of PeekNamedPipe() and save the result for the next call to read. This > >has the advantage that it would probably work (based on Christopher's > >comment and my own experimentation with read() vs. select()), but it > >looks like a heck-of-a-lot of work (read: prone to bugs) to solve such a > >stupid problem. Actually, suggestion #1 suffers from this con as well. > > Your analysis is correct, IMO. I've been mulling this over for a couple > of years and have never come up with a satisfactory solution. Even the > thread method is prone to difficulties. If you are blocked on a read of > a pipe, terminating the thread cleanly becomes problematic. > Oh again Does "problematic" mean "impossible", or just "messy". If the first, then I'll forget about this approach. If the second, then I'll start thinking about writing some test code to explore how to do this. Am I correct in my interpretation that this is only a W9x problem? Does PeekNamedPipe() do the right thing when the writer closes the pipe on NT/2K? If so, then I would be more inclined to bracket a bunch of messy hack code with: if (w9x) { do_the_slow_hack_thing() } > It's been some time since I mentioned that "I hate Windows 9x." This seems > like a good place to bring this up. > Yeah, but it was a significant improvement over Windows 3.1. For the first time (when W95 came out), I was able to run Emacs on the same machine that I ran my word processor and DSP development tools on; and more recently, I am able to run all of my favorite GNU utilities, thanks to the magic of CYGWIN. So far, I haven't seen such a quantum improvement in W98, W98SE, or W98ME. Someday I'll probably switch to NT, but I worry that it won't work on my notebook. Sigh... > I would be really happy if someone had a creative solution to this problem. > It would be nice to fix this in the Cygwin DLL. > I'm still thinking about it. I'm not sure I'll come up with a "creative" solution, most likely a hack solution. Some other ideas I've had: What about implementing pipes using the Windows networking stack (i.e. implement the pipe by creating a socket from localhost to localhost)? Is there any way for the writer to a pipe to know anything about the reader of the pipe? If that is the case, then the writer could send a (special) signal to the reader when it closed the pipe. What to do when somebody dup()'s one end or the other. Gee, this problem gets uglier and uglier the more I think about it. What about using some other IPC mechanism to implement pipes? It looks to me that the way fhandler's are implemented, this should be possible. Is there a requirement that pipes must work with non-cygwin applications? Hmmm... doesn't bash use pipes to implement, er, pipes? If so, then I guess a non-cygwin application would expect to be able to call ReadFile() in order to get at stdin. Hopefully, somebody who actually knows how to program in Windows will also start thinking about this. Gee, I thought I would start with a nice, simple problem for my first attempt at contributing to Cygwin. Oh well... --wpd -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-12-02 19:47 ` Patrick Doyle @ 2000-12-02 20:39 ` Christopher Faylor 2000-12-03 1:47 ` Corinna Vinschen 0 siblings, 1 reply; 20+ messages in thread From: Christopher Faylor @ 2000-12-02 20:39 UTC (permalink / raw) To: cygwin On Sat, Dec 02, 2000 at 10:47:44PM -0500, Patrick Doyle wrote: >> >2) Change the thread_pipe to call ReadFile() (reading one byte) instead >> >of PeekNamedPipe() and save the result for the next call to read. This >> >has the advantage that it would probably work (based on Christopher's >> >comment and my own experimentation with read() vs. select()), but it >> >looks like a heck-of-a-lot of work (read: prone to bugs) to solve such a >> >stupid problem. Actually, suggestion #1 suffers from this con as well. >> >> Your analysis is correct, IMO. I've been mulling this over for a couple >> of years and have never come up with a satisfactory solution. Even the >> thread method is prone to difficulties. If you are blocked on a read of >> a pipe, terminating the thread cleanly becomes problematic. >> >Oh again >Does "problematic" mean "impossible", or just "messy". If the first, >then I'll forget about this approach. If the second, then I'll start >thinking about writing some test code to explore how to do this. The problem is that you can terminate a thread using "TerminateThread" but if the thread is in the middle of doing I/O (i.e. reading from a pipe) you could leave things in an unstable state. Microsoft seems to not want you to use terminate thread: REMARKS TerminateThread is used to cause a thread to exit. When this occurs, the target thread has no chance to execute any user-mode code and its initial stack is not deallo- cated. DLLs attached to the thread are not notified that the thread is terminating. TerminateThread is a dangerous function that should only be used in the most extreme cases. You should call Termi- nateThread only if you know exactly what the target thread is doing, and you control all of the code that the target thread could possibly be running at the time of the termi- nation. For example, TerminateThread can result in the following problems: [more problems deleted] >Am I correct in my interpretation that this is only a W9x problem? Does >PeekNamedPipe() do the right thing when the writer closes the pipe on >NT/2K? Yes. >> It's been some time since I mentioned that "I hate Windows 9x." This seems >> like a good place to bring this up. >> >Yeah, but it was a significant improvement over Windows 3.1. For the >first time (when W95 came out), I was able to run Emacs on the same >machine that I ran my word processor and DSP development tools on; and >more recently, I am able to run all of my favorite GNU utilities, thanks >to the magic of CYGWIN. So far, I haven't seen such a quantum >improvement in W98, W98SE, or W98ME. Someday I'll probably switch to >NT, but I worry that it won't work on my notebook. Sigh... > >> I would be really happy if someone had a creative solution to this problem. >> It would be nice to fix this in the Cygwin DLL. >> >I'm still thinking about it. I'm not sure I'll come up with a >"creative" solution, most likely a hack solution. > >Some other ideas I've had: >What about implementing pipes using the Windows networking stack (i.e. >implement the pipe by creating a socket from localhost to localhost)? I'm sure that that would have its problems too. I don't like the idea of using something other than a pipe for a pipe. For one thing, I think it will really screw up non-cygwin I/O. >Is there any way for the writer to a pipe to know anything about the >reader of the pipe? If that is the case, then the writer could send a >(special) signal to the reader when it closed the pipe. What to do when >somebody dup()'s one end or the other. Gee, this problem gets uglier >and uglier the more I think about it. Cygwin's tty handling has stuff like this. It is ugly. It also suffers from not working with non-cygwin apps. cgf -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-12-02 20:39 ` Christopher Faylor @ 2000-12-03 1:47 ` Corinna Vinschen 2000-12-11 10:19 ` Patrick Doyle 0 siblings, 1 reply; 20+ messages in thread From: Corinna Vinschen @ 2000-12-03 1:47 UTC (permalink / raw) To: cygwin Christopher Faylor wrote: > On Sat, Dec 02, 2000 at 10:47:44PM -0500, Patrick Doyle wrote: > >improvement in W98, W98SE, or W98ME. Someday I'll probably switch to > >NT, but I worry that it won't work on my notebook. Sigh... W2K really works fine on my Vaio 505 and an HP 4150. The latter only suffers from not being able to access the USB port for some reason. > Cygwin's tty handling has stuff like this. It is ugly. It also suffers > from not working with non-cygwin apps. But it's the better mode for telnet/ssh connections to the box. Corinna -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-12-03 1:47 ` Corinna Vinschen @ 2000-12-11 10:19 ` Patrick Doyle 2000-12-11 10:36 ` Christopher Faylor 2000-12-12 5:46 ` David Starks-Browning 0 siblings, 2 replies; 20+ messages in thread From: Patrick Doyle @ 2000-12-11 10:19 UTC (permalink / raw) To: cygwin Status update... First, to refresh the memory of those who are interested in following this thread: 1) cygwin implements pipes (i.e. the 'pipe()' system call) using Windows "anonymous pipes". 2) cygwin implements the 'select()' system call by creating a thread which wakes up every 10 ms and calls 'PeekNamedPipe()' to see if any data are available in the pipe. 3) On Windows 9x machines, 'PeekNamedPipe()' gives no indication when the writer of a pipe closes the pipe. It simply indicates that there are 0 bytes available to be read from the pipe. The only way to know that the writer has closed the pipe is to call 'ReadFile()'. On Windows NT/2K, 'PeekNamedPipe()' returns an indication that the pipe has been closed. 4) SSH uses 'select()' to determine when it is time to read from stdin. If stdin is a pipe and the writer closes the pipe, SSH does not detect this and is left waiting for more input. This happens when using "scp" and when using SSH with "cvs". 5) I have been racking my brain trying to figure out a way to solve this problem. (I posted a simple program to the list a couple of weeks ago that demonstrates this problem). So far, the most elegant solution I have come up with is to reimplement 'select()' so that it calls 'ReadFileEx()' with a suitable 'OVERLAPPED' structure, thus eliminating the separate thread that wakes up every 10 ms. Of course, if I read a byte from the pipe, I would have to buffer it somewhere and fix 'read()' so that it checks for the "readahead" byte first, but that is basically solvable. If the call to 'select()' returned for some other reason (i.e. another fd was made ready), then call 'CancelIo()' to terminate the asynchronous read. The problem with this approach is that the documentation on the MSDN web site states that, for Windows 95/98, "You cannot perform asynchronous read operations on mailslots, named pipes, or disk files", leaving the possibility that it might work on anonymous pipes. Before I expend a lot of effort on this, does anybody know if asynchronous reads of anonymous pipes are supported on 9x? If there is documentation that says that it is not supported, but experimentation indicates that it works anyway, would such an approach be incorporated into cygwin? (In other words, is cygwin willing to depend on code behavior that is unsupported by MS, specifically, behavior of 9x code -- if the answer is no, then I won't waste any effort trying to figure out if it would work). The only other viable alternative I have come up with is to open a second anonymous pipe in parallel with the first and use it to send OOB messages such as "dup occured" or "closing pipe" to the reader. I would bracket all of this code with: if (win9x) { anonymous_pipe_hack(); } so that it would not impact NT/2K users. It would not work with non-cygwin apps (on 9x machines) and could suffer from being fooled somehow. I am reminded that I meant to look into how the TTY code works, since Christopher said that it has some similar issues (and has problems working with non-cygwin apps). I have come to appreciate Christopher's statement "I hate Windows 9x." more and more as each idea I have come up with to address this problem has been shot down when I check the MSDN website and find words to the effect of "this is unsupported on Windows 95/98" in the middle of the function that I think might solve the problem. As a final note, I wondered why RSH didn't suffer from this problem. It's because it forks a separate process just to read from stdin, and doesn't call 'select()'. I keep toying with the idea of modifying SSH to do something similar, but it seems like too much work, and too much modification to SSH, to be a "good" solution. (I would fork a process that just read from stdin and wrote to stdout and then use SIGCHILD to notify the parent process that it should no longer attempt to read from stdin. If we were willing to tolerate an additional process-per-pipe on 9x machines, I could look into doing something like this automatically in cygwin, but I think I'll try my "second anonymous pipe" approach first). If anybody has bothered to continue reading this far, I just want to say that I truly appreciate the work that has gone into cygwin and that I especially like the fact that I can compile, test, and debug modifications to cygwin1.dll myself (i.e. on my W98 box). I remember the days when cygwin1.dll had to be compiled on a Linux box and then copied to the Windows box, and I am very grateful for all the work that has gone into making the tools self-hosting. Thanks again folks! --wpd -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-12-11 10:19 ` Patrick Doyle @ 2000-12-11 10:36 ` Christopher Faylor 2000-12-11 10:56 ` Patrick Doyle 2000-12-12 5:46 ` David Starks-Browning 1 sibling, 1 reply; 20+ messages in thread From: Christopher Faylor @ 2000-12-11 10:36 UTC (permalink / raw) To: cygwin On Mon, Dec 11, 2000 at 01:20:14PM -0500, Patrick Doyle wrote: >So far, the most elegant solution I have come up with is to reimplement >'select()' so that it calls 'ReadFileEx()' with a suitable 'OVERLAPPED' >structure, thus eliminating the separate thread that wakes up every 10 >ms. Of course, if I read a byte from the pipe, I would have to buffer >it somewhere and fix 'read()' so that it checks for the "readahead" byte >first, but that is basically solvable. If the call to 'select()' >returned for some other reason (i.e. another fd was made ready), then >call 'CancelIo()' to terminate the asynchronous read. I'm pretty sure that overlapped I/O does not work with any flavor of pipe. It should be possible to make a simple test case, to verify though. cgf -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-12-11 10:36 ` Christopher Faylor @ 2000-12-11 10:56 ` Patrick Doyle 2000-12-11 11:08 ` Christopher Faylor 2001-01-06 19:21 ` Patrick Doyle 0 siblings, 2 replies; 20+ messages in thread From: Patrick Doyle @ 2000-12-11 10:56 UTC (permalink / raw) To: cygwin Christopher Faylor wrote: > > On Mon, Dec 11, 2000 at 01:20:14PM -0500, Patrick Doyle wrote: > >So far, the most elegant solution I have come up with is to reimplement > >'select()' so that it calls 'ReadFileEx()' with a suitable 'OVERLAPPED' > >structure, thus eliminating the separate thread that wakes up every 10 > >ms. Of course, if I read a byte from the pipe, I would have to buffer > >it somewhere and fix 'read()' so that it checks for the "readahead" byte > >first, but that is basically solvable. If the call to 'select()' > >returned for some other reason (i.e. another fd was made ready), then > >call 'CancelIo()' to terminate the asynchronous read. > > I'm pretty sure that overlapped I/O does not work with any flavor of pipe. > It should be possible to make a simple test case, to verify though. > My question is more of a philosophical one. If I try it, and it happens to work on my W98 box, but it is listed as "unsupported" in the official Microsoft documentation, would you still want to incorporate it into cygwin? On the one hand, I can understand the SQA argument that you would not want the behavior of cygwin to depend upon undocumeneted or unsupported features of Windows. On the other hand, I expect that the behavior of Windows 9x is what it is, and it will never change from here, so if we find a loophole, we might as well use it -- for Windows 9x _only_. But, based on your comment, if I try it, and it doesn't work, my first inclination will be that "it doesn't work because it is unsupported", rather than my default inclination of "it doesn't work because I did something wrong". When that happens, I think I will start down the path of implementing the parallel pipe for OOB data. What an ugly mess this has turned out to be! Thanks for your comments. --wpd -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-12-11 10:56 ` Patrick Doyle @ 2000-12-11 11:08 ` Christopher Faylor 2001-01-06 19:21 ` Patrick Doyle 1 sibling, 0 replies; 20+ messages in thread From: Christopher Faylor @ 2000-12-11 11:08 UTC (permalink / raw) To: cygwin On Mon, Dec 11, 2000 at 01:57:23PM -0500, Patrick Doyle wrote: >Christopher Faylor wrote: >> >> On Mon, Dec 11, 2000 at 01:20:14PM -0500, Patrick Doyle wrote: >> >So far, the most elegant solution I have come up with is to reimplement >> >'select()' so that it calls 'ReadFileEx()' with a suitable 'OVERLAPPED' >> >structure, thus eliminating the separate thread that wakes up every 10 >> >ms. Of course, if I read a byte from the pipe, I would have to buffer >> >it somewhere and fix 'read()' so that it checks for the "readahead" byte >> >first, but that is basically solvable. If the call to 'select()' >> >returned for some other reason (i.e. another fd was made ready), then >> >call 'CancelIo()' to terminate the asynchronous read. >> >> I'm pretty sure that overlapped I/O does not work with any flavor of pipe. >> It should be possible to make a simple test case, to verify though. >> >My question is more of a philosophical one. If I try it, and it happens >to work on my W98 box, but it is listed as "unsupported" in the official >Microsoft documentation, would you still want to incorporate it into >cygwin? I wouldn't suggest a test case if I wasn't willing to try it. cgf -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-12-11 10:56 ` Patrick Doyle 2000-12-11 11:08 ` Christopher Faylor @ 2001-01-06 19:21 ` Patrick Doyle 2001-01-08 3:21 ` Ehud Karni 1 sibling, 1 reply; 20+ messages in thread From: Patrick Doyle @ 2001-01-06 19:21 UTC (permalink / raw) To: cygwin I have been working on a hack workaround for this problem. (Short summary -- select() fails to detect when all of the writers of a pipe have closed the pipe because PeekNamedPipe() on W9x boxen fails to report when all of the writers of a pipe have closed the pipe. I am told that this all works correctly on NT/2K boxen). Basically, I have modified 'pipe()' to create two windows pipes (on W9x boxen). The first pipe is the same one that has always been created. The second one is used simply to pass EOF information to the reader of the (main) pipe. So, I modified (or rather, created) fhandler_pipe::close(), which posts an "EOF" message to the 2nd pipe and I created fhandler_pipe::dup() and fhandler_pipe::fixup_after_fork() to post a "DUP" message to the 2nd pipe. Finally, I modified the processing for select() on an input pipe to check the 2nd pipe, count the number of DUP and EOF messages, and return "READY" when all of the writers have closed the pipe. As I said, it's a hack, but it is one that only activates on W9x boxen, so it shouldn't be too horrible. My current problem is, "How do I handle exec()"? When the new process is started, I see that handles to both pipes will be inherited by the exec'd process, but I have no idea how to re-associate the two of them with each other. In my search for an answer, I have gotten more and more confused. How do fhandlers get recreated for all of the open fd's in the newly exec'd process? I see stdio_init(), which creates fhandlers for fd's 0, 1, and 2, but what about the rest of the open fd's. I don't really need an answer to that, but it will keep me puzzled for weeks. What I really need an answer to is, "How can an exec'd process know things about the state of the orignal process, specifically, what handles were associated with which fhandler objects?" Am I off on the totally wrong track? From what I see in spawn.cc, I think that the exec'd process will start at ground zero, with regard to the cygwin library. Somehow information about the fdtab needs to be communicated to the new process. I see pointers from the parent process being passed to the exec'd process via the STARTUPINFO structure, but I don't see anything like what is in fork.cc, wherein memory regions are physically copied from one process to the other. What am I missing. (And, BTW, the MSDN documentation for STARTUPINFO says that lpReserved2 must be set to NULL -- you set it to &ciresrv -- how can you get away with doing something that the MSDN docs say you shouldn't?) I appreciate any an all pointers that folks would care to give me regarding this. TIA --wpd -- Want to unsubscribe from this list? Check out: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2001-01-06 19:21 ` Patrick Doyle @ 2001-01-08 3:21 ` Ehud Karni 2001-01-08 6:24 ` Patrick Doyle 0 siblings, 1 reply; 20+ messages in thread From: Ehud Karni @ 2001-01-08 3:21 UTC (permalink / raw) To: wpd; +Cc: cygwin On Sat, 06 Jan 2001 22:22:24 -0500, Patrick Doyle <wpd@delcomsys.com> wrote: > > I have been working on a hack workaround for this problem. (Short > summary -- select() fails to detect when all of the writers of a pipe > have closed the pipe because PeekNamedPipe() on W9x boxen fails to > report when all of the writers of a pipe have closed the pipe. I am > told that this all works correctly on NT/2K boxen). My work around for Win95/98 is a bash script, executed directly from the windows command (I associated sh with /bin/bash): #! /bin/sh -ex # -------------------------------------------------- set +e scp -p -v -r -C "$1" ehud@ekc-1:$2 set +x GRP=`ps | grep "[/]PS"` # find this PS line GRP=`echo $GRP | cut "-d " -f3` # compress spaces PID=`ps | grep $GRP | grep "[/]bin[/]SSH"` # My ssh line PID=`echo $PID | cut "-d " -f1` # compress spaces ps # show running processes set -x if [ "$PID" != "" ] ; then # is there "My SSH" kill $PID # yes kill it else : " Error in copy to linux " # no, error message fi sleep 30 # keep console open ############################## scpt.sh ############################## > Basically, I have modified 'pipe()' to create two windows pipes (on W9x > boxen). The first pipe is the same one that has always been created. > [ snip ] Well, it seems a very complicated work around for the W9x bug. I would have taken a much simpler and direct approach like this: 1) Save the child (ssh) process id in a global variable, (i.e. child=fork() ;) 2) Just before exiting, send kill signal to the child with no checks, (i.e. kill(child, SIGTERM) ;) This might look like brute force, but it works. It has the advantage that it works with ssh unmodified and with any other program that may be given as argument to scp. Ehud. -- @@@@@@ @@@ @@@@@@ @ @ Ehud Karni Simon & Wiesel Insurance agency @ @ @ @@ @ Tel: +972-3-6212-757 Fax: +972-3-6292-544 @ @ @ @ @ @@ (USA) Fax and voice mail: 1-815-5509341 @ @ @ @ @ @ Better Safe Than Sorry http://www.simonwiesel.co.il mailto:ehud@unix.simonwiesel.co.il -- Want to unsubscribe from this list? Check out: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2001-01-08 3:21 ` Ehud Karni @ 2001-01-08 6:24 ` Patrick Doyle 0 siblings, 0 replies; 20+ messages in thread From: Patrick Doyle @ 2001-01-08 6:24 UTC (permalink / raw) To: ehud; +Cc: cygwin Ehud Karni wrote: > > On Sat, 06 Jan 2001 22:22:24 -0500, Patrick Doyle <wpd@delcomsys.com> wrote: > > > > I have been working on a hack workaround for this problem. (Short > > summary -- select() fails to detect when all of the writers of a pipe > > have closed the pipe because PeekNamedPipe() on W9x boxen fails to > > report when all of the writers of a pipe have closed the pipe. I am > > told that this all works correctly on NT/2K boxen). > > My work around for Win95/98 is a bash script, executed directly from > the windows command (I associated sh with /bin/bash): > Thanks for the excellent script. It looks like it solves the specific problem that I was complaining about (namely that scp leaves an ssh running), but unfortunately it doesn't solve other related problems that share the same root cause. For example, using SSH with CVS appears to hang at the end of the CVS operation. (CVS is waiting for SSH to complete. SSH is waiting to detect an EOF on its input pipe). I am hoping to solve this problem in the cygwil DLL, so that it is solved once, rather than for each application that encounters it. > > Basically, I have modified 'pipe()' to create two windows pipes (on W9x > > boxen). The first pipe is the same one that has always been created. > > [ snip ] > > Well, it seems a very complicated work around for the W9x bug. I would > have taken a much simpler and direct approach like this: > 1) Save the child (ssh) process id in a global variable, > (i.e. child=fork() ;) > 2) Just before exiting, send kill signal to the child with no > checks, (i.e. kill(child, SIGTERM) ;) > > This might look like brute force, but it works. It has the advantage > that it works with ssh unmodified and with any other program that may > be given as argument to scp. > I think that you are saying that your approach would have been to modify scp (and later cvs) to work around this problem. Again, I would rather fix it in the library, and not have to modify each application that runs into this. I admit, my solution is complicated, it is a hack, in the end I may never be able to make it work right, etc..., but, so far, it is the only viable approach that I have been able to envision. Every other approach I have thought of runs into the problem that it depends on capabilities that are in NT/2K, but not W9x. Sigh... I really appreciate Christoper's statement that he hates Window 9x programming. --wpd -- Want to unsubscribe from this list? Check out: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Why does scp leave ssh running? -- select() never returns 2000-12-11 10:19 ` Patrick Doyle 2000-12-11 10:36 ` Christopher Faylor @ 2000-12-12 5:46 ` David Starks-Browning 1 sibling, 0 replies; 20+ messages in thread From: David Starks-Browning @ 2000-12-12 5:46 UTC (permalink / raw) To: cygwin On Monday 11 Dec 00, Patrick Doyle writes: > Status update... > > First, to refresh the memory of those who are interested in following > this thread: > > ... I added an entry about this in the 'Known Problems' section of the FAQ, and linked to this excellent description of the problem. Thanks Patrick. David -- Want to unsubscribe from this list? Send a message to cygwin-unsubscribe@sourceware.cygnus.com ^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2001-01-08 6:24 UTC | newest] Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2000-11-27 12:26 Why does scp leave ssh running? Patrick Doyle 2000-11-28 2:12 ` Graham Bloice 2000-11-29 19:21 ` Why does scp leave ssh running? -- select() never returns Patrick Doyle 2000-11-29 19:41 ` Christopher Faylor 2000-11-30 6:15 ` Patrick Doyle 2000-11-30 6:35 ` Corinna Vinschen 2000-11-30 7:05 ` Patrick Doyle 2000-11-30 19:40 ` Patrick Doyle 2000-12-02 17:08 ` Christopher Faylor 2000-12-02 19:47 ` Patrick Doyle 2000-12-02 20:39 ` Christopher Faylor 2000-12-03 1:47 ` Corinna Vinschen 2000-12-11 10:19 ` Patrick Doyle 2000-12-11 10:36 ` Christopher Faylor 2000-12-11 10:56 ` Patrick Doyle 2000-12-11 11:08 ` Christopher Faylor 2001-01-06 19:21 ` Patrick Doyle 2001-01-08 3:21 ` Ehud Karni 2001-01-08 6:24 ` Patrick Doyle 2000-12-12 5:46 ` David Starks-Browning
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).