Re: cygcheck output sometimes gets truncated when piped (cygwin-1.7)

Re: cygcheck output sometimes gets truncated when piped (cygwin-1.7)

[Herb Maeder]

On Sun, Oct 05, 2008 at 12:51:59 -0400, Christopher Faylor wrote:
> >On a fresh install of cygwin-1.7 (1.7.0-30) on Vista, the output of
> >"cygcheck -s" sometimes gets truncated.  This seems to be more
> >reproducible when the output is piped to another command.
> 
> This has been reported before and I was able to duplicate it in a VM.
> I thought I'd fixed it when I released 1.7-30 but obviously it's still
> there.
> 
> The good news is that I can reproduce the problem 100% of the time in
> gdb so I finally was able to track down the root cause.
> 
> I'm generating a snapshot with a fix now.  The fix is in cygwin1.dll,
> not cygcheck.exe, so you'll need to download and install that.
> 
> If you can verify that the snapshot works, I'll generate a new 1.7
> release.

The snapshot looks good to me.  I can no longer reproduce the problem with
the cygwin1-20081005.dll.  Thanks for the quick fix.

Herb.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: cygwin-1.7 sshd/ssh-host-config install issues on Vista

[Herb Maeder]

On 17 Oct 2008 20:06:35 EDT, Christopher Faylor wrote:
> On Fri, Oct 17, 2008 at 12:50:48PM -0700, Herb Maeder wrote:
> >2. (Vista/XP) The tcp_wrappers dependency is missing in openssh/setup.hint
> >
> >Sincd sshd.exe depends on cygwrap-0.dll, the tcp_wrappers package must be
> >installed in order to avoid this error when starting up the sshd service:
> >
> >   $ cygrunsrv --start sshd
> >   cygrunsrv: Error starting a service: QueryServiceStatus:  Win32 error
> >1062: The service has not been started.
> >
> >But the tcp_wrappers package is not listed as a dependency in the openssh
> >setup.hint file.
>
> libwrap0 is listed as an openssh dependency and that's where
> cygwrap-0.dll comes from.  Relying on tcp_wrappers would not be correct.

Correct you are, I failed to notice that cygwrap-0.dll was in a library of
its own.  Thanks for the correction.

> AFAICT, setup.ini is correct.

setup.ini is correct.  setup-2.ini is not.

Herb.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: cygwin-1.7 sshd/ssh-host-config install issues on Vista

[Corinna Vinschen]

On Oct 17 17:38, Herb Maeder wrote:
> On 17 Oct 2008 20:06:35 EDT, Christopher Faylor wrote:
> > On Fri, Oct 17, 2008 at 12:50:48PM -0700, Herb Maeder wrote:
> > >But the tcp_wrappers package is not listed as a dependency in the openssh
> > >setup.hint file.
> >
> > libwrap0 is listed as an openssh dependency and that's where
> > cygwrap-0.dll comes from.  Relying on tcp_wrappers would not be correct.
> 
> Correct you are, I failed to notice that cygwrap-0.dll was in a library of
> its own.  Thanks for the correction.
> 
> > AFAICT, setup.ini is correct.
> 
> setup.ini is correct.  setup-2.ini is not.

Thanks for the hint.  I fixed that.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Apologies for multiple messages (Please Help!)

[Herb Maeder]

On 28 Oct 2008 10:38:21 EDT, Christopher Faylor wrote:
> >>Please correct me if I am wrong, but this type of thing has happened to
> >>the list before, right?  I seem to remember it happening to someone
> >>else a while back, but I cannot find it in the archives at the moment.
> >
> >Sure.  Not allot but it has happened before.  I think Chris usually
> >puts a block on things like this to stop it, if nothing else is
> >possible.
> 
> I do but I was on vacation (in Italy!) last week.  Someone eventually
> notified the overseers mailing list about this.  Someone there *should*
> have been able to block the messsages from arriving.  I don't know if
> that is what happened or if things were fixed on the sending end but
> sending email to either postmaster or overseers will contact the right
> people.

Neither postmaster nor overseers ever responded, AFAIK.

I think the sending end eventually gave up.  I got the bounce message
included at the end of this message, after which the duplicated messages
stopped.  I'm still not sure what was happening under the hood (on either
side) though.

One thing we suspect is that the lack of a Message-ID field in the header
reduces the ability of the receiving side to automatically stop posting
these duplicated messages to the list.

As such, making the Message-ID field a requirement for posting to the
cygwin mailing lists might not be a bad idea.  

FYI, over the last month, the only message that came in without the
Message-ID field (besides my own) was spam.

Herb.


From MAILER-DAEMON Fri Oct 24 21:44:33 2008
Return-Path: <>
Delivered-To: maeder-cygml AT maeder DOT org
Received: (qmail 71304 invoked for bounce); 24 Oct 2008 21:44:33 -0000
Date: 24 Oct 2008 21:44:33 -0000
From: MAILER-DAEMON AT bouncehost
To: maeder-cygml AT maeder DOT org
Subject: failure notice
Status: RO
X-Status: 
X-Keywords:                 
X-UID: 4208

Hi. This is the qmail-send program.
I'm afraid I wasn't able to deliver your message to the following addresses.
This is a permanent error; I've given up. Sorry it didn't work out.

<cygwin AT cygwin DOT com>:
Connected to 209.132.176.174 but connection died. Possible duplicate! (#4.4.2)
I'm not going to try again; this message has been in the queue too long.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [ANNOUNCEMENT] Updated: OpenSSH-5.1p1-6 (-7)

[Herb Maeder]

On 07 Nov 2008 17:10:02 EST, Christopher Faylor wrote:
> On Fri, Nov 07, 2008 at 01:37:44PM -0800, Herb Maeder wrote:
> >On 07 Nov 2008 12:00:56 +0100, Corinna Vinschen wrote:   
> >> This is a bugfix release which fixes a bug in the ssh-host-config script
> >> which stumbles over user names with a substring of "ssh" in them and
> >> thinks that ssh processes are still running.
> >
> >Is the intent now to catch only processes named 'sshd'?  If so, the
> >current "grep -q 'sshd*$'" may still be a little too loose.  For example, 
> >it could match stuff like "/home/user/flosshdd".  Ok, maybe not likely, 
> >but still it would cause the script to end in an error.
> >
> >Assuming we can depend on "ps -ef" always printing full path names without 
> >any arguments, then "grep -q '/sshd$'" might do the trick.  Is there any
> >reason to catch multiple trailing d's?
> 
> It's possible that Corinna was looking for zero or more d's.
> 
> So, something like grep -qP '/sshd?' would accommodate that.

Yes, that makes sense.  Zero or one was probably the intent.  Thanks for
pointing it out.

Though the trailing $ probably still makes sense to restrict it to
matching only ssh or sshd processes.  grep -qP '/sshd?$'

Herb.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: rsync 3.0.4 over ssh hanging on cygwin 1.7

[Herb Maeder]

On 19 Nov 2008 09:54:41 EST, Christopher Faylor wrote:
> On Wed, Nov 19, 2008 at 07:24:33AM -0500, Brett Serkez wrote:
> >I spent considerable time on this and reported were the problem is
> >occurring to no avail, don't waste your time.  In a nutshell the issue
> >is with Cygwin's bi-directional pipe emulation, this is a fundamental
> >feature of all UNIXies.  Secure Shell "forks and execs" rsync,
> >connecting standard out and in so that data flows over the internet
> >to/from SSH and then locally to/from rsync.  The problem is that
> >eventually a "signal" is missed and SSH and rsync deadlock, the local
> >pipe emulation is imperfect, and the rsync protocol has no provision to
> >recover from this dead lock.
> >
> >A fix would require a change to this fundamental feature of Cygwin, it
> >is not clear to me that Windows has the necessary functionality to
> >properly implement, such a fix would require extensive retesting.
> 
> Your analysis of the problem is likely incorrect.  The problem has been
> reported to be due to the fact that there is no foolproof way in Windows
> to tell when a pipe can be written to in a non-blocking fashion.  Since
> that fact hasn't changed, I doubt that this has anything to do with
> missed "signal"s and it certainly doesn't have anything to do with
> bidirectional pipes.
> 
> Nevertheless, the problem is still there and as always PGA.  I have
> spent a considerable amount of time staring at the code and googling for
> a solution but to no avail.

Sounds like solving the root of problem may be beyond our control for now.
But, as an alternative, do you think that a cygwin specific workaround to
this problem in rsync (and/or sshd) might be feasible?  If so, would you
have any suggestions on how to approach that task?

Obviously the ideal solution would be to get the underlying problem fixed
in windows, especially since there's no reason other programs won't run
into the same issue.  But since the cygwin+sshd+rsync combo is quite
useful, it shows this problem regularly, and the alternatives aren't
great, having a specific workaround would be nice.  Even if it means
trading off performance and/or convenience for correct functionality.

Herb.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: rsync 3.0.4 over ssh hanging on cygwin 1.7

[Fred Kemp]

Thanks to all who have replied so far - as Herb says, the cygwin+sshd 
+rsync combo is extremely useful, not least in that it allows my  
client PC's to simply run sshd as a very efficient (ie resource  
unintensive) background service, requiring no further user  
intervention (A Good Thing™).  Workarounds such as running OpenVPN or  
VirtualBox seem like the proverbial  sledgehammer to crack a nut and  
additionally add a further layer to configure/go wrong.

 From my own point of view, I'm going to try playing around with a few  
options such as creating an ssh tunnel as described here: http://backuppc.wiki.sourceforge.net/Workaround+BackupPC+Windows+2003+Hang

to see if that helps and beyond that see if I can come up with some  
combination of rsync options and shell scripting that might allow me  
to atomise the transfer somewhat, possibly using --write-batch -- 
timeout and some batch vs logfile crunching to restart transfer at  
last file transferred at timeout point. It won't be as quick or as  
simple, but if it works and is robust, that'll do for me for now (I'm  
at least a week late on rolling this out as it is!) I won't however  
hold my breath as a test of recursively running rsync with --timeout  
on one user directory is showing only about another 5 files each time...

As ever all tips and pointers gratefully received, and if I have any  
blinding glimpses of the obvious, I'll certainly share with the  
list. :-)

Cheers,

Fred.



On Nov 20, 2008, at 1:48 AM, Herb Maeder wrote:

> On 19 Nov 2008 09:54:41 EST, Christopher Faylor wrote:
>> On Wed, Nov 19, 2008 at 07:24:33AM -0500, Brett Serkez wrote:
>>> I spent considerable time on this and reported were the problem is
>>> occurring to no avail, don't waste your time.  In a nutshell the  
>>> issue
>>> is with Cygwin's bi-directional pipe emulation, this is a  
>>> fundamental
>>> feature of all UNIXies.  Secure Shell "forks and execs" rsync,
>>> connecting standard out and in so that data flows over the internet
>>> to/from SSH and then locally to/from rsync.  The problem is that
>>> eventually a "signal" is missed and SSH and rsync deadlock, the  
>>> local
>>> pipe emulation is imperfect, and the rsync protocol has no  
>>> provision to
>>> recover from this dead lock.
>>>
>>> A fix would require a change to this fundamental feature of  
>>> Cygwin, it
>>> is not clear to me that Windows has the necessary functionality to
>>> properly implement, such a fix would require extensive retesting.
>>
>> Your analysis of the problem is likely incorrect.  The problem has  
>> been
>> reported to be due to the fact that there is no foolproof way in  
>> Windows
>> to tell when a pipe can be written to in a non-blocking fashion.   
>> Since
>> that fact hasn't changed, I doubt that this has anything to do with
>> missed "signal"s and it certainly doesn't have anything to do with
>> bidirectional pipes.
>>
>> Nevertheless, the problem is still there and as always PGA.  I have
>> spent a considerable amount of time staring at the code and  
>> googling for
>> a solution but to no avail.
>
> Sounds like solving the root of problem may be beyond our control  
> for now.
> But, as an alternative, do you think that a cygwin specific  
> workaround to
> this problem in rsync (and/or sshd) might be feasible?  If so, would  
> you
> have any suggestions on how to approach that task?
>
> Obviously the ideal solution would be to get the underlying problem  
> fixed
> in windows, especially since there's no reason other programs won't  
> run
> into the same issue.  But since the cygwin+sshd+rsync combo is quite
> useful, it shows this problem regularly, and the alternatives aren't
> great, having a specific workaround would be nice.  Even if it means
> trading off performance and/or convenience for correct functionality.
>
> Herb.
>
> --
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> Problem reports:       http://cygwin.com/problems.html
> Documentation:         http://cygwin.com/docs.html
> FAQ:                   http://cygwin.com/faq/
>


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: rsync 3.0.4 over ssh hanging on cygwin 1.7

[Brett Serkez]

On Thu, Nov 20, 2008 at 11:32 AM, Fred Kemp <c.f.kemp@reading.ac.uk> wrote:
> Thanks to all who have replied so far - as Herb says, the cygwin+sshd+rsync
> combo is extremely useful, not least in that it allows my client PC's to
> simply run sshd as a very efficient (ie resource unintensive) background
> service, requiring no further user intervention (A Good Thing™).
>  Workarounds such as running OpenVPN or VirtualBox seem like the proverbial
>  sledgehammer to crack a nut and additionally add a further layer to
> configure/go wrong.

You may want to check out WinSCP:  http://winscp.net/eng/index.php

For adhoc work I use the graphical user interface, which supports
directory synchronizing, it does support scripting from the command
line for automation, it may do what you want.

Good luck,

Brett

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: cygwin-1.7 sshd/ssh-host-config install issues on Vista

[Christopher Faylor]

On Fri, Oct 17, 2008 at 12:50:48PM -0700, Herb Maeder wrote:
>2. (Vista/XP) The tcp_wrappers dependency is missing in openssh/setup.hint
>
>Sincd sshd.exe depends on cygwrap-0.dll, the tcp_wrappers package must be
>installed in order to avoid this error when starting up the sshd service:
>
>   $ cygrunsrv --start sshd
>   cygrunsrv: Error starting a service: QueryServiceStatus:  Win32 error
>1062: The service has not been started.
>
>But the tcp_wrappers package is not listed as a dependency in the openssh
>setup.hint file.

libwrap0 is listed as an openssh dependency and that's where
cygwrap-0.dll comes from.  Relying on tcp_wrappers would not be correct.
AFAICT, setup.ini is correct.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

cygwin-1.7 sshd/ssh-host-config install issues on Vista

[Herb Maeder]

On a fresh install of the cygwin-1.7 base package + openssh, I believe we
should expect the following to work for installing and testing sshd:

    ssh-host-config -y
    cygrunsrv --start sshd
    ssh localhost pwd    

If sshd had been previously installed on the system, the following
cleanup should be performed before invoking ssh-host-config:
    
    # Remove sshd service
    cygrunsrv --stop sshd
    cygrunsrv --remove sshd
    # Delete any sshd or related users (such as cyg_server) from /etc/passwd
    #   (use your favorite editor)
    # Delete any sshd or relaged users (such as cyg_server) from the system
    net user sshd /delete
    net user cyg_server /delete
    
But in trying to run the test case, I ran into a number of issues when
running on Vista (and some on XP).  I've been able to workaround all but
the last one.


1. (Vista) ssh-host-config needs to run with elevated permissions

This is not really a problem since we expected elevated permissions to be
required, but there are some issues surrounding it.  I believe the current
recommendation is to run ssh-host-config in a bash shell started with
"RightClick->Run As Administrator". 

But given that this requirement is specific to Vista, it might be worth a
check at the start of the script to make sure that run permissions are
good enough avoid the more obscure errors later on.  An error statement 
indicating the preferred way to invoke ssh-host-config will hopefully 
cut down on noise to the list from people switching to Vista.  

BTW, is there a simple command to unobtrusively detect if the runtime
permissions are correct?.

Also, running a bash shell as administrator is less than ideal.  It may
encourage always run as administrators even when not necessary, plus
administrator shells are not easily distinguishable from normal shells.  

I'm curious... is there a way to elevate permissions from a bash command
line (kind of like a poor man's sudo)?  The point would not be to avoid
the UAC prompt, but be able to invoke it when needed from the command line
rather than just getting permission denied errors.

I have come up with a couple of solutions to do this, but they have too
many drawbacks to be really useful (e.g. output ends up in a new cmd
window, UAC prompt lists wrong program,...)


2. (Vista/XP) The tcp_wrappers dependency is missing in openssh/setup.hint

Sincd sshd.exe depends on cygwrap-0.dll, the tcp_wrappers package must be
installed in order to avoid this error when starting up the sshd service:

   $ cygrunsrv --start sshd
   cygrunsrv: Error starting a service: QueryServiceStatus:  Win32 error
1062: The service has not been started.

But the tcp_wrappers package is not listed as a dependency in the openssh
setup.hint file.

It seems others have hit this problem already, but it hasn't been fixed at
the root of the problem yet:

     http://www.cygwin.com/ml/cygwin/2008-08/msg00746.html


3. (Vista) "ssh-host-config -y" still prompts for user input

The -y option to ssh-host-config should set up sshd with a usable default 
configuration without any further user input.  

But since the default for the "Do you want to use a different name?"
question is "yes", the user will be queried for the privileged user name
(and may not end up with the defauilt configuration):

  $ ssh-host-config -y
  <snip>
  *** Info: Note that creating a new user requires that the current account have
  *** Info: Administrator privileges itself.

  *** Info: No privileged account could be found.

  *** Info: This script plans to use 'cyg_server'.
  *** Info: 'cyg_server' will only be used by registered services.
  *** Query: Do you want to use a different name? (yes/no) yes
  *** Query: Enter the new user name:

The question should probably rephrased so that yes will keep the stock
name by default, for example, "Do you want to use this name? (yes/no)".

Or perhaps the "different name" question should come after the "Create new
privileged user account 'cyg_server'?" question (if it is answered 'no').

This seems to be in the csih package, in the csih_select_privileged_username()
function.


4. (Vista) Missing warning if cyg_server exists in /etc/passwd but not in SAM

If the cyg_server account is deleted from the local machine, but its entry
is still left in /etc/passwd, the next run of ssh-host-config will not
issue a warning.  Instead it will just result in a "Win32 error 1057":

   *** Info: The following privileged accounts were found: 'cyg_server' .

   *** Info: This script plans to use 'cyg_server'.
   *** Info: 'cyg_server' will only be used by registered services.
   *** Query: Do you want to use a different name? (yes/no) no
   *** Query: Please enter the password for user 'cyg_server':
   *** Query: Reenter:

   cygrunsrv: Error installing a service: CreateService:  Win32 error 1057: The account name is invalid or does not exist, or the password is invalid for the account name specified.

   *** Warning: Something went wrong installing the sshd service.

Unfortunately, this does not indicate the real root of the problem, so it
makes it a bit difficult for users to debug.  Perhaps a similar warning
for the sshd case should be issued:

   *** Warning: sshd is in /etc/passwd, but the
   *** Warning: local machine's SAM does not know about sshd.
   *** Warning: Perhaps sshd is a pre-existing domain account.
   *** Warning: Continuing, but check if this is ok.


5. (Vista) "ssh localhost pwd" gives 'ssh_exchange_identification' error

After running ssh-host-config and starting the server on Vista, the ssh
test gives the following error:

      $ ssh localhost pwd
      ssh_exchange_identification: Connection closed by remote host

This error is specific to using 'localhost' or a loopback ip address.
Using a real hostname does not generate this error.  I have the firewall
turned off.  Curious that it does not show up on an equivalent XP setup.

On the server side, "sshd -d" shows that the 'Connection refused by tcp
wrapper'.  

My /etc/hosts.allow looks like this, which appears to be the default
configuration: 

    ALL : PARANOID : deny
    sshd: ALL

I can work around the problem by putting a "sshd: ALL" or "sshd: PARANOID"
line first, but I don't think those are the right solution.  

There's something else going on in tcp_wrapper with the address/name
matching for localhost, but I can't quite figure out what.  I couldn't get
it to work by putting other lines first, like "ALL: localhost", "sshd:
KNOWN", "sshd:  UNKNOWN" or any other variant I could think of.

Can others reproduce this problem?


6. (Vista) error in setting cyg_server passwd expiry

When ssh-host-config tries to set the expiry on the cyg_server group, I
get the following error:

   passwd: unknown user herb
   *** Warning: Setting password expiry for user 'cyg_server' failed!
   *** Warning: Please check that password never expires or set it to your needs.

The command that generates the unknown user error is "passwd -e
cyg_server" from the csih script.  Note that it complains about the login
user, not the cyg_server user.

From my read of passwd.c, I'm not sure that 'passwd -e' can really be used
to set the expiry on a local user if the login user is a domain user.


7. (Vista) sshd responds to connection with "initgroups: Permission denied"

This one is the showstopper.  It is preventing me from being able to ssh
into a Vista machine at all.  I haven't found a workaround it or determine 
the root of problem.  

Any attempt to ssh results in this error:

        % ssh localhost pwd
        herb@localhost's password:
        initgroups: Permission denied

I think that this should be easily reproducible on a fresh install of
cygwin-1.7 base + openssh.  But if not, I can provide more information 
about my specific situation.

As near as I can tell (using "strace /usr/sbin/sshd -dd") the problem
appears to come from the call to NetUserGetGroups() in
sec_auth.cc:get_user_groups(), which returns an error.  But I have not
been able to determine the root of the problem yet.

Herb.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/