sshd issues on Windows 10 version 1803

sshd issues on Windows 10 version 1803

[Hiya Z]

Hello,

Now that Microsoft has bundled OpenSSH for Windows in update 1803 (see
https://blogs.msdn.microsoft.com/commandline/2018/03/07/windows10v1803/),
here are some observations and the resulting unfortunate behavior:

-  On a clean vanilla 1803 install, even with Windows Subsystem for Linux
(WSL) not installed, there exists a folder %SystemRoot%\system32\openssh
that has native Windows ssh and sshd executables.

-  %SystemRoot%\system32\openssh is added to the system PATH.

-  a (manual start) service "sshd", displayname="OpenSSH SSH Server",
imagepath=%SystemRoot%\system32\openssh\sshd.exe, gets created.

All of the above have the unfortunate consequence that Cygwin sshd no
longer works reliably on a 1803 install. First, the service name "sshd"
conflicts. This can be worked around by something like "ssh-host-config -N
cygwinsshd". Even then, the Cygwin sshd does not always automatically start
upon reboot. The Windows sshd is not configured/running, so there should
not be a port conflict. SCM reports timeouts (waiting for cygwinsshd to
respond) in the event viewer. A manual "sc start cygwinsshd" works, but
again, upon the next reboot, the problem is back.

There are other permutations depending on the sequence of install. If I
were to upgrade a Windows 10 1709 node with a working Cygwin sshd, it seems
that the 1803 update detects an existing sshd registry key (which really is
Cygwin), does not clobber the registry keys, nor does it install sshd.exe
under %SystemRoot%\system32\openssh (though ssh.exe and family get
installed). Even on this node, the Cygwin sshd shows the same issue of not
autostarting reliably upon reboots.

I am not sure what Cygwin can do... but reporting this issue to see if
Cygwin sshd can be fixed to autostart reliably upon reboots. As a minimum,
the default service name in ssh-host-config should be changed to not
collide with Windows "sshd".

Thanks.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: sshd issues on Windows 10 version 1803

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2381 bytes --]

On Jun 22 11:27, Hiya Z wrote:
> Hello,
> 
> Now that Microsoft has bundled OpenSSH for Windows in update 1803 (see
> https://blogs.msdn.microsoft.com/commandline/2018/03/07/windows10v1803/),
> here are some observations and the resulting unfortunate behavior:
> 
> -  On a clean vanilla 1803 install, even with Windows Subsystem for Linux
> (WSL) not installed, there exists a folder %SystemRoot%\system32\openssh
> that has native Windows ssh and sshd executables.
> 
> -  %SystemRoot%\system32\openssh is added to the system PATH.
> 
> -  a (manual start) service "sshd", displayname="OpenSSH SSH Server",
> imagepath=%SystemRoot%\system32\openssh\sshd.exe, gets created.
> 
> All of the above have the unfortunate consequence that Cygwin sshd no
> longer works reliably on a 1803 install. First, the service name "sshd"
> conflicts. This can be worked around by something like "ssh-host-config -N
> cygwinsshd". Even then, the Cygwin sshd does not always automatically start
> upon reboot. The Windows sshd is not configured/running, so there should
> not be a port conflict. SCM reports timeouts (waiting for cygwinsshd to
> respond) in the event viewer. A manual "sc start cygwinsshd" works, but
> again, upon the next reboot, the problem is back.
> 
> There are other permutations depending on the sequence of install. If I
> were to upgrade a Windows 10 1709 node with a working Cygwin sshd, it seems
> that the 1803 update detects an existing sshd registry key (which really is
> Cygwin), does not clobber the registry keys, nor does it install sshd.exe
> under %SystemRoot%\system32\openssh (though ssh.exe and family get
> installed). Even on this node, the Cygwin sshd shows the same issue of not
> autostarting reliably upon reboots.
> 
> I am not sure what Cygwin can do... but reporting this issue to see if
> Cygwin sshd can be fixed to autostart reliably upon reboots. As a minimum,
> the default service name in ssh-host-config should be changed to not
> collide with Windows "sshd".

There's also the problem that %SystemRoot%\System32\OpenSSH is now in
%Path% by default.  Very nice of MIcrosoft to just overload the service
name used by Cygwin for ages :(


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: sshd issues on Windows 10 version 1803

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2988 bytes --]

On Jun 25 14:10, Corinna Vinschen wrote:
> On Jun 22 11:27, Hiya Z wrote:
> > Hello,
> > 
> > Now that Microsoft has bundled OpenSSH for Windows in update 1803 (see
> > https://blogs.msdn.microsoft.com/commandline/2018/03/07/windows10v1803/),
> > here are some observations and the resulting unfortunate behavior:
> > 
> > -  On a clean vanilla 1803 install, even with Windows Subsystem for Linux
> > (WSL) not installed, there exists a folder %SystemRoot%\system32\openssh
> > that has native Windows ssh and sshd executables.
> > 
> > -  %SystemRoot%\system32\openssh is added to the system PATH.
> > 
> > -  a (manual start) service "sshd", displayname="OpenSSH SSH Server",
> > imagepath=%SystemRoot%\system32\openssh\sshd.exe, gets created.
> > 
> > All of the above have the unfortunate consequence that Cygwin sshd no
> > longer works reliably on a 1803 install. First, the service name "sshd"
> > conflicts. This can be worked around by something like "ssh-host-config -N
> > cygwinsshd". Even then, the Cygwin sshd does not always automatically start
> > upon reboot. The Windows sshd is not configured/running, so there should
> > not be a port conflict. SCM reports timeouts (waiting for cygwinsshd to
> > respond) in the event viewer. A manual "sc start cygwinsshd" works, but
> > again, upon the next reboot, the problem is back.
> > 
> > There are other permutations depending on the sequence of install. If I
> > were to upgrade a Windows 10 1709 node with a working Cygwin sshd, it seems
> > that the 1803 update detects an existing sshd registry key (which really is
> > Cygwin), does not clobber the registry keys, nor does it install sshd.exe
> > under %SystemRoot%\system32\openssh (though ssh.exe and family get
> > installed). Even on this node, the Cygwin sshd shows the same issue of not
> > autostarting reliably upon reboots.
> > 
> > I am not sure what Cygwin can do... but reporting this issue to see if
> > Cygwin sshd can be fixed to autostart reliably upon reboots. As a minimum,
> > the default service name in ssh-host-config should be changed to not
> > collide with Windows "sshd".
> 
> There's also the problem that %SystemRoot%\System32\OpenSSH is now in
> %Path% by default.  Very nice of MIcrosoft to just overload the service
> name used by Cygwin for ages :(

However, the longer I think about it, the less I see the problem.  You
can run only one sshd service on port 22 anyway.  If you install 1803 on
a machine with Cygwin sshd service installed, the Microsoft service will
not override (which is rather courteous).  If you install a new machine,
Microsoft's sshd gets installed by default (nice), and if you want to
install Cygwin's sshd instead, you can simply remove the MSFT service
beforehand with `cygrunsrv -R sshd' (easy).


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: sshd issues on Windows 10 version 1803

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2336 bytes --]

On Jun 25 16:08, Corinna Vinschen wrote:
> On Jun 25 14:10, Corinna Vinschen wrote:
> > On Jun 22 11:27, Hiya Z wrote:
> > > Hello,
> > > 
> > > Now that Microsoft has bundled OpenSSH for Windows in update 1803 (see
> > > https://blogs.msdn.microsoft.com/commandline/2018/03/07/windows10v1803/),
> > > here are some observations and the resulting unfortunate behavior:
> > > 
> > > -  On a clean vanilla 1803 install, even with Windows Subsystem for Linux
> > > (WSL) not installed, there exists a folder %SystemRoot%\system32\openssh
> > > that has native Windows ssh and sshd executables.
> > > 
> > > -  %SystemRoot%\system32\openssh is added to the system PATH.
> > > 
> > > -  a (manual start) service "sshd", displayname="OpenSSH SSH Server",
> > > imagepath=%SystemRoot%\system32\openssh\sshd.exe, gets created.
> > > 
> > > All of the above have the unfortunate consequence that Cygwin sshd no
> > > longer works reliably on a 1803 install. First, the service name "sshd"
> > > conflicts. This can be worked around by something like "ssh-host-config -N
> > > cygwinsshd". Even then, the Cygwin sshd does not always automatically start
> > > upon reboot. The Windows sshd is not configured/running, so there should
> > > not be a port conflict. SCM reports timeouts (waiting for cygwinsshd to
> > > respond) in the event viewer. A manual "sc start cygwinsshd" works, but
> > > again, upon the next reboot, the problem is back.
> > > 
> > > There are other permutations depending on the sequence of install. If I
> > > were to upgrade a Windows 10 1709 node with a working Cygwin sshd, it seems
> > > that the 1803 update detects an existing sshd registry key (which really is
> > > Cygwin), does not clobber the registry keys, nor does it install sshd.exe
> > > under %SystemRoot%\system32\openssh (though ssh.exe and family get
> > > installed). Even on this node, the Cygwin sshd shows the same issue of not
> > > autostarting reliably upon reboots.

Btw., I can't reproduce this problem.  Cygwin's sshd service always
starts for me at boottime as expected.

Did you check if the log file in /var/log contains any more info?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: sshd issues on Windows 10 version 1803

[Hiya Z]

Upon upgrading another 1709 node (with working Cygwin sshd), the 1803
update clobbered the sshd service registry tree. The older Cygwin sshd key
was deleted and a new sshd service got created pointing to
%SystemRoot%\System32\OpenSSH\sshd.exe.
This is different from what I observed on the 1st node that I upgraded a
few weeks ago (wherein the Cygwin sshd registry tree was left intact). I
deleted the sshd service, rerun ssh-host-config, and am back to random
failed auto-restarts of sshd.

I think there is a myriad of issues here, the least of which is MS
hijacking the "sshd" service name. As a minimum, the default 'service_name'
in ssh-host-config should be changed to not conflict with "sshd". Maybe
ssh-host-config should also consider deleting any pre-existing "sshd"
service and then recreate a new one. The FAQ may require some mention as
well.

I got another hit of this conflict on the web at
https://gist.github.com/roxlu/5038729  (last few comments).

Thanks.

On Tue, Jun 26, 2018 at 1:16 AM, Corinna Vinschen <corinna-cygwin@cygwin.com
> wrote:

> On Jun 25 11:59, Hiya Z wrote:
> > >Btw., I can't reproduce this problem.  Cygwin's sshd service always
> > >starts for me at boottime as expected.
> >
> > >Did you check if the log file in /var/log contains any more info?
> >
> > I see nothing in /var/log/sshd.log. Windows event log shows following
> > errors:
> >
> > --
> > Error      6/20/2018 3:45:19 PM   Service Control Manager
> > 7009      None
> > A timeout was reached (30000 milliseconds) while waiting for the sshd
> > service to connect.
> >
> > Error      6/20/2018 3:45:19 PM   Service Control Manager
> > 7000      None
> > The sshd service failed to start due to the following error:
> > The service did not respond to the start or control request in a timely
> > fashion.
> > --
> >
> > Another odd behavior that I observe is that after a reboot, starting a
> > Cygwin64 terminal (first time) takes a long time. Launching subsequent
> > mintty terminals is quick as it should be. Could it be that sshd startup
> > upon reboot is hitting the same delay as the 1st mintty?
> >
> > Will try removing %SystemRoot%\System32\OpenSSH from the PATH to see if
> it
> > improves things.
>
> I didn't remove it and it works fine for me.  The OpenSSH path is
> pretty late in $PATH and should not affect starting with cygrunsrv
> because it prepends /bin to $PATH before running the service...
> unless you used the -e option to cygrunsrv to infuse your own $PATH.
>
>
> Corinna
>
> --
> Corinna Vinschen                  Please, send mails regarding Cygwin to
> Cygwin Maintainer                 cygwin AT cygwin DOT com
> Red Hat
>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: sshd issues on Windows 10 version 1803

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1559 bytes --]

On Jun 25 11:59, Hiya Z wrote:
> >Btw., I can't reproduce this problem.  Cygwin's sshd service always
> >starts for me at boottime as expected.
> 
> >Did you check if the log file in /var/log contains any more info?
> 
> I see nothing in /var/log/sshd.log. Windows event log shows following
> errors:
> 
> --
> Error      6/20/2018 3:45:19 PM   Service Control Manager
> 7009      None
> A timeout was reached (30000 milliseconds) while waiting for the sshd
> service to connect.
> 
> Error      6/20/2018 3:45:19 PM   Service Control Manager
> 7000      None
> The sshd service failed to start due to the following error:
> The service did not respond to the start or control request in a timely
> fashion.
> --
> 
> Another odd behavior that I observe is that after a reboot, starting a
> Cygwin64 terminal (first time) takes a long time. Launching subsequent
> mintty terminals is quick as it should be. Could it be that sshd startup
> upon reboot is hitting the same delay as the 1st mintty?
> 
> Will try removing %SystemRoot%\System32\OpenSSH from the PATH to see if it
> improves things.

I didn't remove it and it works fine for me.  The OpenSSH path is
pretty late in $PATH and should not affect starting with cygrunsrv
because it prepends /bin to $PATH before running the service...
unless you used the -e option to cygrunsrv to infuse your own $PATH.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: sshd issues on Windows 10 version 1803

[Hiya Z]

>Btw., I can't reproduce this problem.  Cygwin's sshd service always
>starts for me at boottime as expected.

>Did you check if the log file in /var/log contains any more info?

I see nothing in /var/log/sshd.log. Windows event log shows following
errors:

--
Error      6/20/2018 3:45:19 PM   Service Control Manager
7009      None
A timeout was reached (30000 milliseconds) while waiting for the sshd
service to connect.

Error      6/20/2018 3:45:19 PM   Service Control Manager
7000      None
The sshd service failed to start due to the following error:
The service did not respond to the start or control request in a timely
fashion.
--

Another odd behavior that I observe is that after a reboot, starting a
Cygwin64 terminal (first time) takes a long time. Launching subsequent
mintty terminals is quick as it should be. Could it be that sshd startup
upon reboot is hitting the same delay as the 1st mintty?

Will try removing %SystemRoot%\System32\OpenSSH from the PATH to see if it
improves things.

Thanks.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple