Why does scp leave ssh running?

Why does scp leave ssh running?

[Patrick Doyle]

When I run

scp somefile host:.

I am left with a copy of SSH running.  At first, I thought this was
because "scp" negotiated a connection with the remote host using ssh and
then left that connection open in the background in case I wanted to
perform more remote copies.  But experimentation has led me to believe
that each time I perform a copy, another ssh gets left running.  Also
interesting is that if I copy _from_ the remote computer to my local
computer, no copy of ssh is left running.  The problem is, on my W98
boxes, I cannot quit out of my bash prompt until I kill all of the pesky
ssh's left lying around.

I am running cygwin-1.1.6-1 and openssh-2.3.0p1-1.

1) Is this known behavior?
2) Is this expected behavior?
3) Does anybody care to see the output of cygcheck -s to consider this
problem?

I will track down and see if I can diagnose and correct the root of this
problem myself if nobody else has heard of it and/or nobody is working
on solving it.

--wpd

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

RE: Why does scp leave ssh running?

[Graham Bloice]

> When I run
>
> scp somefile host:.
>
> I am left with a copy of SSH running.  At first, I thought this was
> because "scp" negotiated a connection with the remote host using ssh and
> then left that connection open in the background in case I wanted to
> perform more remote copies.  But experimentation has led me to believe
> that each time I perform a copy, another ssh gets left running.  Also
> interesting is that if I copy _from_ the remote computer to my local
> computer, no copy of ssh is left running.  The problem is, on my W98
> boxes, I cannot quit out of my bash prompt until I kill all of the pesky
> ssh's left lying around.
>
> I am running cygwin-1.1.6-1 and openssh-2.3.0p1-1.
>
> 1) Is this known behavior?
> 2) Is this expected behavior?
> 3) Does anybody care to see the output of cygcheck -s to consider this
> problem?
>
> I will track down and see if I can diagnose and correct the root of this
> problem myself if nobody else has heard of it and/or nobody is working
> on solving it.
>

This is very similar to the problem I and others have reported with using
CVS over ssh on win98.  When a cvs command such as 'cvs update' is executed,
the command runs successfully to completion with all it's normal output, but
control isn't returned to the bash prompt.  Using task manager reveals a
copy of ssh still running, and killing that returns control to the prompt.
he trick is knowing when the cvs update has completed and it's safe to kill
ssh.

I've done some limited debugging that seemed to indicate that cvs was
attempting to shutdown the ssh connection, and was checking to see if there
were any characters left in the pipe.  At this point cvs hangs in a getc().
I haven't had time to take it further, but I have my suspicions about pipes
and win98.

I attempted to extract the essence of what cvs was doing with ssh to a
separate program which of course ran perfectly :-(

Graham


--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Patrick Doyle]

Patrick Doyle wrote:
> 
> When I run
> 
> scp somefile host:.
> 
> I am left with a copy of SSH running.  ...

I believe that have traced this problem down to the call to select() in
ssh.  Basically, it appears that ssh calls select() on a pipe.  If the
writer of that pipe closes it, select() never returns.  I have attached
code to the end of this email demonstrating this phenomenon.

I believe that I have traced this down to the call to PeekNamedPipe() in
"select.cc".  For some reason, on my W98 box, PeekNamedPipe() does not
return an error when the other end of the pipe is closed.

So, a few questions...
1) Is this behavior (of PeekNamedPipe()) a W9x bug?  (That is, does it
work correctly on NT/2K?)

2) What is the expected behavior of select() on a pipe when the writer
closes the pipe?  The code in ssh seems to imply that it should return,
probably with the "read" bit set for the closed pipe.

3) Does anybody have any ideas of how I might work around this problem,
preferably in cygwin?  (Otherwise, I could change the call to select()
"clientloop.c" to include a timeout).

4) Would folks mind trying this code and confirming if I am on the right
track or not?

Thanks for any help and/or advice...

--wpd

code follows... sorry about the leftover #ifdef debug stuff.

/************************************************************************
 * Simple program that demonstrates the "problem" with select(). 
Basically,
 * when a process is blocked on a select() call to read from a pipe, and
the
 * writer for the pipe closes it, the select call never returns (unless,
 * presumably, the timeout is set.
 *
 * This code demonstrats this phenomenon by creating a pipe, forking
 * a subprocess, closing the write side of the pipe (in the parent
process)
 * and performing a select() on the read side of the pipe.  Meanwhile,
the
 * child process closes the pipe, either after a timeout, or upon
receipt
 * if a SIGHUP.

************************************************************************/
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/select.h>
#include <signal.h>

void
sighup(int x)
{
  /* do nothing */
}

int
main(int argc, char *argv[])
{
  int pid;
  int timeout;
  int pipe_fd[2];
  char buf[256];
  int bytes_read;
  int input_fd;
  fd_set readset;

  if (argc < 2) {
    timeout = 5;
  } else {
    if (sscanf(argv[1], "%d", &timeout) != 1 || timeout < 0) {
      fprintf(stderr, "usage: test1 [timeout]\n");
      exit(1);
    }
  }

  if (pipe(pipe_fd) < 0) {
    perror("pipe");
    exit(1);
  }
  if ((pid = fork()) == 0) {
    /* We are the child -- wait the specified timeout, write something
     * to the pipe, wait again, then close the pipe.
     */
    if (timeout == 0) {
      signal(SIGHUP, sighup);
      printf("Waiting for signal\n");
      sigpause(0);
    } else {
      sleep(timeout);
    }
#if 0
    printf("child: about to write to pipe\n");
    if (write(pipe_fd[1], "hello", sizeof("hello")) != sizeof("hello"))
{
      perror("child: write");
      exit(1);
    }
    if (timeout == 0) {
      signal(SIGHUP, sighup);
      printf("Waiting for signal\n");
      sigpause(0);
    } else {
      sleep(timeout);
    }
#endif
    if (close(pipe_fd[1]) < 0) {
      perror("child: close: pipe");
      exit(1);
    }
    printf("child: pipe closed, exiting\n");
    /* Exit successfully */
    exit(0);
  } else if (pid < 0) {
    /* The fork failed for some reason */
    perror("fork");
    exit(1);
  }
  /* We are the parent */
  close(pipe_fd[1]);
#if 1
  input_fd = pipe_fd[0];
#else
  input_fd = 0; /* Read from stdin */
#endif

  printf("parent: about to select on fd %d for reading\n", input_fd);
  FD_ZERO(&readset);
  FD_SET(input_fd, &readset);
  if (select(pipe_fd[0] + 1, &readset, 0, 0, 0) < 0) {
    perror("select");
    exit(1);
  }

  printf("parent: About to read from fd %d\n", input_fd);
  if ((bytes_read = read(input_fd, buf, sizeof(buf))) < 0) {
    perror("parent: read");
    exit(1);
  }
  buf[bytes_read] = 0;

  printf("read buffer of length %d: %s\n", bytes_read, buf);
  close(pipe_fd[0]);
  exit(0);
}

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Christopher Faylor]

On Wed, Nov 29, 2000 at 10:22:03PM -0500, Patrick Doyle wrote:
>So, a few questions...
>1) Is this behavior (of PeekNamedPipe()) a W9x bug?  (That is, does it
>work correctly on NT/2K?)

I think you are correct.  It is a w9x bug.  I vaguely recall seeing this
before.

I can't think of any way to work around this, unfortunately.  I remember trying
things like zero byte reads on the pipe but they didn't do anything.

The only way to trigger the EOF is to actually read from the pipe.

cgf

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Patrick Doyle]

Christopher Faylor wrote:
> 
> On Wed, Nov 29, 2000 at 10:22:03PM -0500, Patrick Doyle wrote:
> >So, a few questions...
> >1) Is this behavior (of PeekNamedPipe()) a W9x bug?  (That is, does it
> >work correctly on NT/2K?)
> 
> I think you are correct.  It is a w9x bug.  I vaguely recall seeing this
> before.
> 
> I can't think of any way to work around this, unfortunately.  I remember trying
> things like zero byte reads on the pipe but they didn't do anything.
> 
> The only way to trigger the EOF is to actually read from the pipe.
> 
> cgf
If that is the case, would anybody mind if I posted a (simple) patch to
openSSH code which added a timeout to the call(s) to select()?  If I do
that, will the patch be incorporated into the cygwin release of
openSSH?  Is there some way to feed that patch back to the openSSH
development team?

--wpd
(Note, I haven't tried this patch yet -- I just thought of it when I was
posting my original email last night -- so the patch may not be as
"simple" as I envisioned).

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Corinna Vinschen]

Patrick Doyle wrote:
> If that is the case, would anybody mind if I posted a (simple) patch to
> openSSH code which added a timeout to the call(s) to select()?  If I do
> that, will the patch be incorporated into the cygwin release of
> openSSH?  Is there some way to feed that patch back to the openSSH
> development team?

I am that way.

However, your patch should be clean in terms of OS handling.

I recommend using the already used construct:

#ifdef HAVE_CYGWIN
  if (! is_winnt)
    {
      ...your w9x specific code...
    }
#endif


The definition of is_winnt is used eg. in `cygwin_utils.c':

#define is_winnt       (GetVersion() < 0x80000000)

Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Developer                                mailto:cygwin@cygwin.com
Red Hat, Inc.
mailto:vinschen@redhat.com

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Patrick Doyle]

Great...I'll give it a try sometime tonight or over the weekend &
hopefully post a patch soon.

--wpd

Corinna Vinschen wrote:
> 
> Patrick Doyle wrote:
> > If that is the case, would anybody mind if I posted a (simple) patch to
> > openSSH code which added a timeout to the call(s) to select()?  If I do
> > that, will the patch be incorporated into the cygwin release of
> > openSSH?  Is there some way to feed that patch back to the openSSH
> > development team?
> 
> I am that way.
> 
> However, your patch should be clean in terms of OS handling.
> 
> I recommend using the already used construct:
> 
> #ifdef HAVE_CYGWIN
>   if (! is_winnt)
>     {
>       ...your w9x specific code...
>     }
> #endif
> 
> The definition of is_winnt is used eg. in `cygwin_utils.c':
> 
> #define is_winnt       (GetVersion() < 0x80000000)
> 
> Corinna
> 
> --
> Corinna Vinschen                  Please, send mails regarding Cygwin to
> Cygwin Developer                                mailto:cygwin@cygwin.com
> Red Hat, Inc.
> mailto:vinschen@redhat.com
> 
> --
> Want to unsubscribe from this list?
> Send a message to cygwin-unsubscribe@sourceware.cygnus.com

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Patrick Doyle]

> Christopher Faylor wrote:
> >
> > On Wed, Nov 29, 2000 at 10:22:03PM -0500, Patrick Doyle wrote:
> > >1) Is this behavior (of PeekNamedPipe()) a W9x bug?  (That is, does it
> > >work correctly on NT/2K?)
> >
> > I think you are correct.  It is a w9x bug.  I vaguely recall seeing this
> > before.
> >
> > I can't think of any way to work around this, unfortunately.  I remember trying
> > things like zero byte reads on the pipe but they didn't do anything.
> >
> > The only way to trigger the EOF is to actually read from the pipe.
> >
> > cgf
> If that is the case, would anybody mind if I posted a (simple) patch to
> openSSH code which added a timeout to the call(s) to select()?
>
> (Note, I haven't tried this patch yet -- I just thought of it ...

Status update... 

Adding the timeout to the call to select() did not solve the problem
(which is consistent with Christopher's assertion that "The only way to
trigger the EOF is to actually read from the pipe".  Actually, I wasn't
too keen about solving the problem by hacking openSSH; it would just
mean that the problem had to be solved again and again for other
packages.

So now what?  Does anybody have any suggestions for other things that I
could try?  I would prefer to solve this problem in cygwin1.dll, but I
am, by no means, an expert windows programmer, so right now, I find
myself just flailing.

The two ideas I have so far are:
1) Switch from using anonymous pipes to using named pipes with unique
names.  According to the documentation on the MSDN web site, this is how
anonymous pipes are implemented in W2K and NT.  Maybe this will solve
the problem, maybe it won't.  Maybe it will allow me to call a function
which will let me know if the other end has closed the pipe.  (Most of
the functions I have found on the MSDN web site are specific to named
pipes).  Maybe it won't.

2) Change the thread_pipe to call ReadFile() (reading one byte) instead
of PeekNamedPipe() and save the result for the next call to read.  This
has the advantage that it would probably work (based on Christopher's
comment and my own experimentation with read() vs. select()), but it
looks like a heck-of-a-lot of work (read: prone to bugs) to solve such a
stupid problem.  Actually, suggestion #1 suffers from this con as well.

Any suggestions, comments, retorts, complaints about me cluttering up
your email box with trivialities that you don't care about, would be
welcome, er... probably not the last one.

--wpd

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Christopher Faylor]

Sorry for the delay in responding.

On Thu, Nov 30, 2000 at 10:41:01PM -0500, Patrick Doyle wrote:
>> If that is the case, would anybody mind if I posted a (simple) patch to
>> openSSH code which added a timeout to the call(s) to select()?
>>
>> (Note, I haven't tried this patch yet -- I just thought of it ...
>
>Status update... 
>
>Adding the timeout to the call to select() did not solve the problem
>(which is consistent with Christopher's assertion that "The only way to
>trigger the EOF is to actually read from the pipe".  Actually, I wasn't
>too keen about solving the problem by hacking openSSH; it would just
>mean that the problem had to be solved again and again for other
>packages.
>
>So now what?  Does anybody have any suggestions for other things that I
>could try?  I would prefer to solve this problem in cygwin1.dll, but I
>am, by no means, an expert windows programmer, so right now, I find
>myself just flailing.
>
>The two ideas I have so far are:
>1) Switch from using anonymous pipes to using named pipes with unique
>names.  According to the documentation on the MSDN web site, this is how
>anonymous pipes are implemented in W2K and NT.  Maybe this will solve
>the problem, maybe it won't.  Maybe it will allow me to call a function
>which will let me know if the other end has closed the pipe.  (Most of
>the functions I have found on the MSDN web site are specific to named
>pipes).  Maybe it won't.

You can't create named pipes on Windows 95, unfortunately.

>2) Change the thread_pipe to call ReadFile() (reading one byte) instead
>of PeekNamedPipe() and save the result for the next call to read.  This
>has the advantage that it would probably work (based on Christopher's
>comment and my own experimentation with read() vs. select()), but it
>looks like a heck-of-a-lot of work (read: prone to bugs) to solve such a
>stupid problem.  Actually, suggestion #1 suffers from this con as well.

Your analysis is correct, IMO.  I've been mulling this over for a couple
of years and have never come up with a satisfactory solution.  Even the
thread method is prone to difficulties.  If you are blocked on a read of
a pipe, terminating the thread cleanly becomes problematic.

It's been some time since I mentioned that "I hate Windows 9x."  This seems
like a good place to bring this up.

>Any suggestions, comments, retorts, complaints about me cluttering up
>your email box with trivialities that you don't care about, would be
>welcome, er... probably not the last one.

I would be really happy if someone had a creative solution to this problem.
It would be nice to fix this in the Cygwin DLL.

cgf

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Patrick Doyle]

Christopher Faylor wrote:
> Sorry for the delay in responding.
No problem... I haven't had a chance to work on this Friday or today
anyway...  I did start to get worried when nobody responded 37 minutes
after I first posted my message.  Isn't that the way this list is
supposed to work? ;-)

> >The two ideas I have so far are:
> >1) Switch from using anonymous pipes to using named pipes with unique
> >names.  According to the documentation on the MSDN web site, this is how
> >...
> 
> You can't create named pipes on Windows 95, unfortunately.
> 
Oh

> >2) Change the thread_pipe to call ReadFile() (reading one byte) instead
> >of PeekNamedPipe() and save the result for the next call to read.  This
> >has the advantage that it would probably work (based on Christopher's
> >comment and my own experimentation with read() vs. select()), but it
> >looks like a heck-of-a-lot of work (read: prone to bugs) to solve such a
> >stupid problem.  Actually, suggestion #1 suffers from this con as well.
> 
> Your analysis is correct, IMO.  I've been mulling this over for a couple
> of years and have never come up with a satisfactory solution.  Even the
> thread method is prone to difficulties.  If you are blocked on a read of
> a pipe, terminating the thread cleanly becomes problematic.
> 
Oh again
Does "problematic" mean "impossible", or just "messy".  If the first,
then I'll forget about this approach.  If the second, then I'll start
thinking about writing some test code to explore how to do this.

Am I correct in my interpretation that this is only a W9x problem?  Does
PeekNamedPipe() do the right thing when the writer closes the pipe on
NT/2K?  If so, then I would be more inclined to bracket a bunch of messy
hack code with:

if (w9x) {
  do_the_slow_hack_thing()
}

> It's been some time since I mentioned that "I hate Windows 9x."  This seems
> like a good place to bring this up.
> 
Yeah, but it was a significant improvement over Windows 3.1.  For the
first time (when W95 came out), I was able to run Emacs on the same
machine that I ran my word processor and DSP development tools on; and
more recently, I am able to run all of my favorite GNU utilities, thanks
to the magic of CYGWIN.  So far, I haven't seen such a quantum
improvement in W98, W98SE, or W98ME.  Someday I'll probably switch to
NT, but I worry that it won't work on my notebook.  Sigh...

> I would be really happy if someone had a creative solution to this problem.
> It would be nice to fix this in the Cygwin DLL.
> 
I'm still thinking about it.  I'm not sure I'll come up with a
"creative" solution, most likely a hack solution.  

Some other ideas I've had:
What about implementing pipes using the Windows networking stack (i.e.
implement the pipe by creating a socket from localhost to localhost)?

Is there any way for the writer to a pipe to know anything about the
reader of the pipe?  If that is the case, then the writer could send a
(special) signal to the reader when it closed the pipe.  What to do when
somebody dup()'s one end or the other.  Gee, this problem gets uglier
and uglier the more I think about it.

What about using some other IPC mechanism to implement pipes?  It looks
to me that the way fhandler's are implemented, this should be possible. 
Is there a requirement that pipes must work with non-cygwin
applications?  Hmmm... doesn't bash use pipes to implement, er, pipes? 
If so, then I guess a non-cygwin application would expect to be able to
call ReadFile() in order to get at stdin.

Hopefully, somebody who actually knows how to program in Windows will
also start thinking about this.

Gee, I thought I would start with a nice, simple problem for my first
attempt at contributing to Cygwin.  Oh well...

--wpd

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Christopher Faylor]

On Sat, Dec 02, 2000 at 10:47:44PM -0500, Patrick Doyle wrote:
>> >2) Change the thread_pipe to call ReadFile() (reading one byte) instead
>> >of PeekNamedPipe() and save the result for the next call to read.  This
>> >has the advantage that it would probably work (based on Christopher's
>> >comment and my own experimentation with read() vs. select()), but it
>> >looks like a heck-of-a-lot of work (read: prone to bugs) to solve such a
>> >stupid problem.  Actually, suggestion #1 suffers from this con as well.
>> 
>> Your analysis is correct, IMO.  I've been mulling this over for a couple
>> of years and have never come up with a satisfactory solution.  Even the
>> thread method is prone to difficulties.  If you are blocked on a read of
>> a pipe, terminating the thread cleanly becomes problematic.
>> 
>Oh again
>Does "problematic" mean "impossible", or just "messy".  If the first,
>then I'll forget about this approach.  If the second, then I'll start
>thinking about writing some test code to explore how to do this.

The problem is that you can terminate a thread using "TerminateThread"
but if the thread is in the middle of doing I/O (i.e. reading from a
pipe) you could leave things in an unstable state.  Microsoft seems to
not want you to use terminate thread:

REMARKS
       TerminateThread  is  used to cause a thread to exit.  When
       this occurs, the target thread has no  chance  to  execute
       any  user-mode  code  and its initial stack is not deallo-
       cated.  DLLs attached to the thread are not notified  that
       the thread is terminating.

       TerminateThread  is  a dangerous function that should only
       be used in the most extreme cases.  You should call Termi-
       nateThread only if you know exactly what the target thread
       is doing, and you control all of the code that the  target
       thread could possibly be running at the time of the termi-
       nation.  For example, TerminateThread can  result  in  the
       following problems:

[more problems deleted]

>Am I correct in my interpretation that this is only a W9x problem?  Does
>PeekNamedPipe() do the right thing when the writer closes the pipe on
>NT/2K?

Yes.

>> It's been some time since I mentioned that "I hate Windows 9x."  This seems
>> like a good place to bring this up.
>> 
>Yeah, but it was a significant improvement over Windows 3.1.  For the
>first time (when W95 came out), I was able to run Emacs on the same
>machine that I ran my word processor and DSP development tools on; and
>more recently, I am able to run all of my favorite GNU utilities, thanks
>to the magic of CYGWIN.  So far, I haven't seen such a quantum
>improvement in W98, W98SE, or W98ME.  Someday I'll probably switch to
>NT, but I worry that it won't work on my notebook.  Sigh...
>
>> I would be really happy if someone had a creative solution to this problem.
>> It would be nice to fix this in the Cygwin DLL.
>> 
>I'm still thinking about it.  I'm not sure I'll come up with a
>"creative" solution, most likely a hack solution.  
>
>Some other ideas I've had:
>What about implementing pipes using the Windows networking stack (i.e.
>implement the pipe by creating a socket from localhost to localhost)?

I'm sure that that would have its problems too.  I don't like the idea
of using something other than a pipe for a pipe.  For one thing, I think
it will really screw up non-cygwin I/O.

>Is there any way for the writer to a pipe to know anything about the
>reader of the pipe?  If that is the case, then the writer could send a
>(special) signal to the reader when it closed the pipe.  What to do when
>somebody dup()'s one end or the other.  Gee, this problem gets uglier
>and uglier the more I think about it.

Cygwin's tty handling has stuff like this.  It is ugly.  It also suffers
from not working with non-cygwin apps.

cgf

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Corinna Vinschen]

Christopher Faylor wrote:
> On Sat, Dec 02, 2000 at 10:47:44PM -0500, Patrick Doyle wrote:
> >improvement in W98, W98SE, or W98ME.  Someday I'll probably switch to
> >NT, but I worry that it won't work on my notebook.  Sigh...

W2K really works fine on my Vaio 505 and an HP 4150. The latter
only suffers from not being able to access the USB port for some reason.

> Cygwin's tty handling has stuff like this.  It is ugly.  It also suffers
> from not working with non-cygwin apps.

But it's the better mode for telnet/ssh connections to the box.

Corinna

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Patrick Doyle]

Status update...

First, to refresh the memory of those who are interested in following
this thread:

1) cygwin implements pipes (i.e. the 'pipe()' system call) using Windows
"anonymous pipes".

2) cygwin implements the 'select()' system call by creating a thread
which wakes up every 10 ms and calls 'PeekNamedPipe()' to see if any
data are available in the pipe.

3) On Windows 9x machines, 'PeekNamedPipe()' gives no indication when
the writer of a pipe closes the pipe.  It simply indicates that there
are 0 bytes available to be read from the pipe.  The only way to know
that the writer has closed the pipe is to call 'ReadFile()'.  On Windows
NT/2K, 'PeekNamedPipe()' returns an indication that the pipe has been
closed.

4) SSH uses 'select()' to determine when it is time to read from stdin. 
If stdin is a pipe and the writer closes the pipe, SSH does not detect
this and is left waiting for more input.  This happens when using "scp"
and when using SSH with "cvs".

5) I have been racking my brain trying to figure out a way to solve this
problem.  (I posted a simple program to the list a couple of weeks ago
that demonstrates this problem).

So far, the most elegant solution I have come up with is to reimplement
'select()' so that it calls 'ReadFileEx()' with a suitable 'OVERLAPPED'
structure, thus eliminating the separate thread that wakes up every 10
ms.  Of course, if I read a byte from the pipe, I would have to buffer
it somewhere and fix 'read()' so that it checks for the "readahead" byte
first, but that is basically solvable.  If the call to 'select()'
returned for some other reason (i.e. another fd was made ready), then
call 'CancelIo()' to terminate the asynchronous read.

The problem with this approach is that the documentation on the MSDN web
site states that, for Windows 95/98, "You cannot perform asynchronous
read operations on mailslots, named pipes, or disk files", leaving the
possibility that it might work on anonymous pipes.

Before I expend a lot of effort on this, does anybody know if
asynchronous reads of anonymous pipes are supported on 9x?  If there is
documentation that says that it is not supported, but experimentation
indicates that it works anyway, would such an approach be incorporated
into cygwin?  (In other words, is cygwin willing to depend on code
behavior that is unsupported by MS, specifically, behavior of 9x code --
if the answer is no, then I won't waste any effort trying to figure out
if it would work).

The only other viable alternative I have come up with is to open a
second anonymous pipe in parallel with the first and use it to send OOB
messages such as "dup occured" or "closing pipe" to the reader.  I would
bracket all of this code with:

if (win9x) {
  anonymous_pipe_hack();
}

so that it would not impact NT/2K users.  It would not work with
non-cygwin apps (on 9x machines) and could suffer from being fooled
somehow.

I am reminded that I meant to look into how the TTY code works, since
Christopher said that it has some similar issues (and has problems
working with non-cygwin apps).

I have come to appreciate Christopher's statement "I hate Windows 9x."
more and more as each idea I have come up with to address this problem
has been shot down when I check the MSDN website and find words to the
effect of "this is unsupported on Windows 95/98" in the middle of the
function that I think might solve the problem.

As a final note, I wondered why RSH didn't suffer from this problem. 
It's because it forks a separate process just to read from stdin, and
doesn't call 'select()'.  I keep toying with the idea of modifying SSH
to do something similar, but it seems like too much work, and too much
modification to SSH, to be a "good" solution.  (I would fork a process
that just read from stdin and wrote to stdout and then use SIGCHILD to
notify the parent process that it should no longer attempt to read from
stdin.  If we were willing to tolerate an additional process-per-pipe on
9x machines, I could look into doing something like this automatically
in cygwin, but I think I'll try my "second anonymous pipe" approach
first).

If anybody has bothered to continue reading this far, I just want to say
that I truly appreciate the work that has gone into cygwin and that I
especially like the fact that I can compile, test, and debug
modifications to cygwin1.dll myself (i.e. on my W98 box).  I remember
the days when cygwin1.dll had to be compiled on a Linux box and then
copied to the Windows box, and I am very grateful for all the work that
has gone into making the tools self-hosting.  Thanks again folks!

--wpd

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Christopher Faylor]

On Mon, Dec 11, 2000 at 01:20:14PM -0500, Patrick Doyle wrote:
>So far, the most elegant solution I have come up with is to reimplement
>'select()' so that it calls 'ReadFileEx()' with a suitable 'OVERLAPPED'
>structure, thus eliminating the separate thread that wakes up every 10
>ms.  Of course, if I read a byte from the pipe, I would have to buffer
>it somewhere and fix 'read()' so that it checks for the "readahead" byte
>first, but that is basically solvable.  If the call to 'select()'
>returned for some other reason (i.e. another fd was made ready), then
>call 'CancelIo()' to terminate the asynchronous read.

I'm pretty sure that overlapped I/O does not work with any flavor of pipe.
It should be possible to make a simple test case, to verify though.

cgf

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Patrick Doyle]

Christopher Faylor wrote:
> 
> On Mon, Dec 11, 2000 at 01:20:14PM -0500, Patrick Doyle wrote:
> >So far, the most elegant solution I have come up with is to reimplement
> >'select()' so that it calls 'ReadFileEx()' with a suitable 'OVERLAPPED'
> >structure, thus eliminating the separate thread that wakes up every 10
> >ms.  Of course, if I read a byte from the pipe, I would have to buffer
> >it somewhere and fix 'read()' so that it checks for the "readahead" byte
> >first, but that is basically solvable.  If the call to 'select()'
> >returned for some other reason (i.e. another fd was made ready), then
> >call 'CancelIo()' to terminate the asynchronous read.
> 
> I'm pretty sure that overlapped I/O does not work with any flavor of pipe.
> It should be possible to make a simple test case, to verify though.
> 
My question is more of a philosophical one.  If I try it, and it happens
to work on my W98 box, but it is listed as "unsupported" in the official
Microsoft documentation, would you still want to incorporate it into
cygwin?  On the one hand, I can understand the SQA argument that you
would not want the behavior of cygwin to depend upon undocumeneted or
unsupported features of Windows.  On the other hand, I expect that the
behavior of Windows 9x is what it is, and it will never change from
here, so if we find a loophole, we might as well use it -- for Windows
9x _only_.

But, based on your comment, if I try it, and it doesn't work, my first
inclination will be that "it doesn't work because it is unsupported",
rather than my default inclination of "it doesn't work because I did
something wrong".  When that happens, I think I will start down the path
of implementing the parallel pipe for OOB data.

What an ugly mess this has turned out to be!  Thanks for your comments.

--wpd

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Christopher Faylor]

On Mon, Dec 11, 2000 at 01:57:23PM -0500, Patrick Doyle wrote:
>Christopher Faylor wrote:
>> 
>> On Mon, Dec 11, 2000 at 01:20:14PM -0500, Patrick Doyle wrote:
>> >So far, the most elegant solution I have come up with is to reimplement
>> >'select()' so that it calls 'ReadFileEx()' with a suitable 'OVERLAPPED'
>> >structure, thus eliminating the separate thread that wakes up every 10
>> >ms.  Of course, if I read a byte from the pipe, I would have to buffer
>> >it somewhere and fix 'read()' so that it checks for the "readahead" byte
>> >first, but that is basically solvable.  If the call to 'select()'
>> >returned for some other reason (i.e. another fd was made ready), then
>> >call 'CancelIo()' to terminate the asynchronous read.
>> 
>> I'm pretty sure that overlapped I/O does not work with any flavor of pipe.
>> It should be possible to make a simple test case, to verify though.
>> 
>My question is more of a philosophical one.  If I try it, and it happens
>to work on my W98 box, but it is listed as "unsupported" in the official
>Microsoft documentation, would you still want to incorporate it into
>cygwin?

I wouldn't suggest a test case if I wasn't willing to try it.

cgf

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[David Starks-Browning]

On Monday 11 Dec 00, Patrick Doyle writes:
> Status update...
> 
> First, to refresh the memory of those who are interested in following
> this thread:
> 
> ...

I added an entry about this in the 'Known Problems' section of the
FAQ, and linked to this excellent description of the problem.  Thanks
Patrick.

David


--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

Re: Why does scp leave ssh running? -- select() never returns

[Patrick Doyle]

I have been working on a hack workaround for this problem.  (Short
summary -- select() fails to detect when all of the writers of a pipe
have closed the pipe because PeekNamedPipe() on W9x boxen fails to
report when all of the writers of a pipe have closed the pipe.  I am
told that this all works correctly on NT/2K boxen).

Basically, I have modified 'pipe()' to create two windows pipes (on W9x
boxen).  The first pipe is the same one that has always been created. 
The second one is used simply to pass EOF information to the reader of
the (main) pipe.  So, I modified (or rather, created)
fhandler_pipe::close(), which posts an "EOF" message to the 2nd pipe and
I created fhandler_pipe::dup() and fhandler_pipe::fixup_after_fork() to
post a "DUP" message to the 2nd pipe.  Finally, I modified the
processing for select() on an input pipe to check the 2nd pipe, count
the number of DUP and EOF messages, and return "READY" when all of the
writers have closed the pipe.  As I said, it's a hack, but it is one
that only activates on W9x boxen, so it shouldn't be too horrible.

My current problem is, "How do I handle exec()"?  When the new process
is started, I see that handles to both pipes will be inherited by the
exec'd process, but I have no idea how to re-associate the two of them
with each other.  In my search for an answer, I have gotten more and
more confused.  How do fhandlers get recreated for all of the open fd's
in the newly exec'd process?  I see stdio_init(), which creates
fhandlers for fd's 0, 1, and 2, but what about the rest of the open
fd's.  I don't really need an answer to that, but it will keep me
puzzled for weeks.  What I really need an answer to is, "How can an
exec'd process know things about the state of the orignal process,
specifically, what handles were associated with which fhandler objects?"

Am I off on the totally wrong track?  From what I see in spawn.cc, I
think that the exec'd process will start at ground zero, with regard to
the cygwin library.  Somehow information about the fdtab needs to be
communicated to the new process.  I see pointers from the parent process
being passed to the exec'd process via the STARTUPINFO structure, but I
don't see anything like what is in fork.cc, wherein memory regions are
physically copied from one process to the other.  What am I missing. 
(And, BTW, the MSDN documentation for STARTUPINFO says that lpReserved2
must be set to NULL -- you set it to &ciresrv -- how can you get away
with doing something that the MSDN docs say you shouldn't?)

I appreciate any an all pointers that folks would care to give me
regarding this.

TIA

--wpd

--
Want to unsubscribe from this list?
Check out: http://cygwin.com/ml/#unsubscribe-simple

Re: Why does scp leave ssh running? -- select() never returns

[Ehud Karni]

On Sat, 06 Jan 2001 22:22:24 -0500, Patrick Doyle <wpd@delcomsys.com> wrote:
> 
> I have been working on a hack workaround for this problem.  (Short
> summary -- select() fails to detect when all of the writers of a pipe
> have closed the pipe because PeekNamedPipe() on W9x boxen fails to
> report when all of the writers of a pipe have closed the pipe.  I am
> told that this all works correctly on NT/2K boxen).

My work around for Win95/98 is a bash script, executed directly from
the windows command (I associated sh with /bin/bash):


#! /bin/sh -ex
# --------------------------------------------------

set +e
scp -p -v -r -C "$1" ehud@ekc-1:$2

set +x
GRP=`ps | grep "[/]PS"`                    # find this PS line      
GRP=`echo $GRP | cut "-d " -f3`            # compress spaces        
PID=`ps | grep $GRP | grep "[/]bin[/]SSH"` # My ssh line            
PID=`echo $PID | cut "-d " -f1`            # compress spaces        
                                                                    
ps                                         # show running processes 
set -x                                                              
                                                                    
if [ "$PID" != "" ] ; then                 # is there "My SSH"      
    kill $PID                              # yes kill it            
else                                                                
    : " Error in copy to linux "           # no, error message      
fi                                                                  
sleep 30                                   # keep console open      
                                           
############################## scpt.sh ##############################


> Basically, I have modified 'pipe()' to create two windows pipes (on W9x
> boxen).  The first pipe is the same one that has always been created. 
> [ snip ] 

Well, it seems a very complicated work around for the W9x bug. I would
have taken a much simpler and direct approach like this:
   1) Save the child (ssh) process id in a global variable,
           (i.e. child=fork() ;)
   2) Just before exiting, send kill signal to the child with no
     checks,  (i.e. kill(child, SIGTERM) ;)

This might look like brute force, but it works. It has the advantage
that it works with ssh unmodified and with any other program that may
be given as argument to scp.

Ehud.
       

-- 
 @@@@@@ @@@ @@@@@@ @    @   Ehud Karni  Simon & Wiesel  Insurance agency
     @    @      @  @@  @   Tel: +972-3-6212-757    Fax: +972-3-6292-544
     @    @ @    @ @  @@    (USA)  Fax  and  voice  mail:  1-815-5509341
     @    @ @    @ @    @        Better     Safe     Than     Sorry
 http://www.simonwiesel.co.il    mailto:ehud@unix.simonwiesel.co.il

--
Want to unsubscribe from this list?
Check out: http://cygwin.com/ml/#unsubscribe-simple

Re: Why does scp leave ssh running? -- select() never returns

[Patrick Doyle]

Ehud Karni wrote:
> 
> On Sat, 06 Jan 2001 22:22:24 -0500, Patrick Doyle <wpd@delcomsys.com> wrote:
> >
> > I have been working on a hack workaround for this problem.  (Short
> > summary -- select() fails to detect when all of the writers of a pipe
> > have closed the pipe because PeekNamedPipe() on W9x boxen fails to
> > report when all of the writers of a pipe have closed the pipe.  I am
> > told that this all works correctly on NT/2K boxen).
> 
> My work around for Win95/98 is a bash script, executed directly from
> the windows command (I associated sh with /bin/bash):
> 
Thanks for the excellent script.  It looks like it solves the specific
problem that I was complaining about (namely that scp leaves an ssh
running), but unfortunately it doesn't solve other related problems that
share the same root cause.  For example, using SSH with CVS appears to
hang at the end of the CVS operation.  (CVS is waiting for SSH to
complete.  SSH is waiting to detect an EOF on its input pipe).  I am
hoping to solve this problem in the cygwil DLL, so that it is solved
once, rather than for each application that encounters it.

> > Basically, I have modified 'pipe()' to create two windows pipes (on W9x
> > boxen).  The first pipe is the same one that has always been created.
> > [ snip ]
> 
> Well, it seems a very complicated work around for the W9x bug. I would
> have taken a much simpler and direct approach like this:
>    1) Save the child (ssh) process id in a global variable,
>            (i.e. child=fork() ;)
>    2) Just before exiting, send kill signal to the child with no
>      checks,  (i.e. kill(child, SIGTERM) ;)
> 
> This might look like brute force, but it works. It has the advantage
> that it works with ssh unmodified and with any other program that may
> be given as argument to scp.
> 
I think that you are saying that your approach would have been to modify
scp (and later cvs) to work around this problem.  Again, I would rather
fix it in the library, and not have to modify each application that runs
into this.  I admit, my solution is complicated, it is a hack, in the
end I may never be able to make it work right, etc..., but, so far, it
is the only viable approach that I have been able to envision.  Every
other approach I have thought of runs into the problem that it depends
on capabilities that are in NT/2K, but not W9x.  Sigh...  I really
appreciate Christoper's statement that he hates Window 9x programming.

--wpd

--
Want to unsubscribe from this list?
Check out: http://cygwin.com/ml/#unsubscribe-simple