slow startup after upgrade

slow startup after upgrade

[xian]

Recently I updated my Cygwin packages and since then starting bash or
mintty takes much longer then before.

I tried with a fresh install too but with the same effect. I do have
(corporate) antivirus but the old version still starts pretty quickly.
How can I debug why do I have to wait long seconds for a prompt? I
still have the old install restored from backup, so I can make
comparisons if necessary.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[xian]

Sorry for the double post, just noticed a very similar thread. I do
have domain account. What I did now to speed up startup:
created local db with mkpasswd and mkgroup
created nsswitch.conf allowing only files to be read.

Now everything back to normal.

I read the new DB method is better, but I can access the domain only
when I make a VPN connection to my company, so for me an offline
password file works better.

2015-02-09 22:58 GMT+01:00 xian <xian@mailbox.hu>:
> Recently I updated my Cygwin packages and since then starting bash or
> mintty takes much longer then before.
>
> I tried with a fresh install too but with the same effect. I do have
> (corporate) antivirus but the old version still starts pretty quickly.
> How can I debug why do I have to wait long seconds for a prompt? I
> still have the old install restored from backup, so I can make
> comparisons if necessary.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 969 bytes --]

On Feb  9 23:26, xian wrote:
> Sorry for the double post, just noticed a very similar thread. I do
> have domain account. What I did now to speed up startup:
> created local db with mkpasswd and mkgroup
> created nsswitch.conf allowing only files to be read.
> 
> Now everything back to normal.
> 
> I read the new DB method is better, but I can access the domain only
> when I make a VPN connection to my company, so for me an offline
> password file works better.

Yes, that's expected.  I'm just wondering if the new documentation at
https://cygwin.com/cygwin-ug-net/ntsec.html is sufficient, or if that
qualifies for another FAQ entry along the lines of

  "Starting a Cygwin shell is slow, what's going on?"

I'd not be overly grudgy if somebody wants to come up with a FAQ text...


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: slow startup after upgrade

[Warren Young]

[-- Attachment #1: Type: text/plain, Size: 734 bytes --]

> On Feb 10, 2015, at 2:38 AM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote:
> 
>  "Starting a Cygwin shell is slow, what's going on?"
> 
> I'd not be overly grudgy if somebody wants to come up with a FAQ text…

I’m not entirely sure I understand the problem, but I’ve attached a starting point.

I’m pretty sure the issue brought up in this thread isn’t the only possible cause of slow launches.  DNS can do it, too, and I suspect if I wracked my brain some more on it, I could come up with other causes.

The DocBook isn’t validated.  It may need to be adjusted before it will let the FAQ document build correctly.  It might also be good to include olinks or ulinks to other parts of the Cygwin docs.


[-- Attachment #2: slowtty.dbx --]
[-- Type: application/octet-stream, Size: 2853 bytes --]

<title>Starting a new terminal window is slow. What's going on?</title>

<para>There are many possible causes for this. This answer is more a list of things to look into than a set of solutions.</para>

<orderedlist>
	<listitem>
		<para>If your terminal windows suddenly began starting slowly after a Cygwin upgrade, the most likely cause is that you have an outdated authentication setup.</para>

		<para>For almost all its lifetime, Cygwin has used Unix-like <filename>/etc/passwd</filename> and <filename>/etc/group</filename> files to mirror the contents of the Windows SAM and AD databases. Although these files can still be used, since Cygwin 1.7.34, new installations now use the SAM/AD databases directly.</para>

		<para>To switch to the new method, move these two files out of the way and restart the Cygwin terminal. That runs Cygwin in its new default mode. If you are on a system that isn't using AD domain logins, this makes Cygwin use the native Windows SAM database directly, which may be faster than the old method involving <filename>/etc/passwd</filename> and such. At worst, it will only be a bit slower. (Which situation applies depends on the benchmark you run.)</para>

		<para>If you are on an AD system, a hybrid approach you might consider is to re-run <programname>mkpasswd</programname> and <programname>mkgroup</programname>, then put the following into <filename>/etc/nsswitch.conf</filename> to make Cygwin treat these files as read-only local caches of your AD database:</para>

		<screen>
			passwd: files
			group:  files
		</screen>

		<para>By leaving out the "db" option, we are telling the Cygwin DLL not to even <emphasis>try</emphasis> to do AD lookups. If your AD servers are slow, this local cache will speed things up. The downside is the old stale cache problem: any time the AD databases change, your local cache will go out of date until you update the files manually.</para>
	</listitem>

	<listitem>
		<para>Another common cause of slow Cygwin Terminal starts is a bad DNS setup. Many things that occur during a Cygwin Terminal startup require fast DNS lookups.</para>
	</listitem>

	<listitem>
		<para>More...?</para>
	</listitem>
</orderedlist>

<para>If none of the above helps, the best troubleshooting method is to run your startup scripts in debug mode. Right-click your Cygwin Terminal icon, go to Properties, and edit the command. It should be something like <command>C:\cygwin\bin\mintty.exe -i /Cygwin-Terminal.ico -</command>. Assuming you are using Bash for your login shell, change it to <command>C:\cygwin64\bin\mintty /bin/bash -lx</command> That will cause it to write out a line for every command it runs. A slow Cygwin Terminal launch usually means one or more of the many commands Cygwin runs when starting up will take a long time. That will be your clue as to what's going on.</para>

[-- Attachment #3: Type: text/plain, Size: 218 bytes --]

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1145 bytes --]

On Feb 10 15:52, Warren Young wrote:
> > On Feb 10, 2015, at 2:38 AM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote:
> > 
> >  "Starting a Cygwin shell is slow, what's going on?"
> > 
> > I'd not be overly grudgy if somebody wants to come up with a FAQ text…
> 
> I’m not entirely sure I understand the problem, but I’ve attached a
> starting point.
> 
> I’m pretty sure the issue brought up in this thread isn’t the only
> possible cause of slow launches.  DNS can do it, too, and I suspect if
> I wracked my brain some more on it, I could come up with other causes.
> 
> The DocBook isn’t validated.  It may need to be adjusted before it
> will let the FAQ document build correctly.  It might also be good to
> include olinks or ulinks to other parts of the Cygwin docs.

Thanks a lot.  I added this to the FAQ in CVS with just the changes
necessary to make it a FAQ entry.  If somebody has to add something,
feel free to send text.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: slow startup after upgrade

[Roger Orr]

I am also hit by the slow AD issue; so thanks for the solution.

mkpasswd takes an hour and mkgroup takes longer -- is there anything I can
suggest to our administrators that would help make this time less?

We have not had previous problems reported with our AD being slow.

Regards,
Roger.

-----Original Message-----
From: Corinna Vinschen [mailto:corinna-cygwin@cygwin.com] 
Sent: 11 February 2015 08:53
To: cygwin@cygwin.com
Subject: Re: slow startup after upgrade

On Feb 10 15:52, Warren Young wrote:
> > On Feb 10, 2015, at 2:38 AM, Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
> > 
> >  "Starting a Cygwin shell is slow, what's going on?"
> > 
> > I'd not be overly grudgy if somebody wants to come up with a FAQ text.
> 
> I'm not entirely sure I understand the problem, but I've attached a
> starting point.
> 
> I'm pretty sure the issue brought up in this thread isn't the only
> possible cause of slow launches.  DNS can do it, too, and I suspect if
> I wracked my brain some more on it, I could come up with other causes.
> 
> The DocBook isn't validated.  It may need to be adjusted before it
> will let the FAQ document build correctly.  It might also be good to
> include olinks or ulinks to other parts of the Cygwin docs.

Thanks a lot.  I added this to the FAQ in CVS with just the changes
necessary to make it a FAQ entry.  If somebody has to add something,
feel free to send text.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 894 bytes --]

On Feb 13 08:35, Roger Orr wrote:
> I am also hit by the slow AD issue; so thanks for the solution.
> 
> mkpasswd takes an hour and mkgroup takes longer -- is there anything I can
> suggest to our administrators that would help make this time less?
> 
> We have not had previous problems reported with our AD being slow.

The problem with 1.7.34 is that it opens an LDAP connection and requests
passwd-entry related account information for your account and all your
groups in your user token.  The later, fetching the info for all your
groups, is rather useless.  I think that's the major problem.  So I'd
think the best way forward is to update to the 1.7.35-0.1 test release
and report further from there.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: slow startup after upgrade

[Roger Orr]

Corinna Vinschen wrote:
> So I'd think the best way forward is to update to the
> 1.7.35-0.1 test release and report further from there.     

Thanks, this does help a little. However I will still be using the 'files'
setting.

Here are some results in case they're of interest. (Windows 7/64 with
cygwin/64.)

1) Running cygwin "echo.exe" from a cmd.exe command shell

a) With passwd: files and group: files in /etc/nsswitch.conf
  0.03 - 0.4 s

b) With 1.7.34 and default /etc/nsswitch.conf

  around 120s

C) With 1.7.35 and default /etc/nsswitch.conf

  4.4 - 4.6s

2) mkgroup

1.7.34 took 46m 4s
1.7.35 took 39s

    output is 53kb, 681 lines

3) mkpasswd

1.7.34 took 1h 14m 6s
1.7.35 took 59m 0s

    output is 132kb, 1081 lines

Regards,
Roger.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1614 bytes --]

On Feb 16 20:02, Roger Orr wrote:
> Corinna Vinschen wrote:
> > So I'd think the best way forward is to update to the
> > 1.7.35-0.1 test release and report further from there.     
> 
> Thanks, this does help a little. However I will still be using the 'files'
> setting.

The idea was to test this stuff to find a better solution which is
acceptable.  If you simply revert to "files" without helping to test
we'll probably never find the culprit.

> Here are some results in case they're of interest. (Windows 7/64 with
> cygwin/64.)
> 
> 1) Running cygwin "echo.exe" from a cmd.exe command shell
> 
> a) With passwd: files and group: files in /etc/nsswitch.conf
>   0.03 - 0.4 s
> 
> b) With 1.7.34 and default /etc/nsswitch.conf
> 
>   around 120s
> 
> C) With 1.7.35 and default /etc/nsswitch.conf
> 
>   4.4 - 4.6s

Much better, but...

...what's going on with your AD?!?  There are only a couple of calls to
LookupAccountSid (one for your user account, one per group in your user
token) and a single LDAP call fetching a handful of attributes for your
user account.  This shouldn't really require 4 secs.

It would be nice to know what part of the code is so slow.  The
LookupAccountSid calls shouldn't be so slow because they only fetch
information already cached on the local machine.  So it's probably
the LDAP call.  Why does an LDAP call take 4 secs?!?

Are you remote from your DC, by any chance?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: slow startup after upgrade

[Roger Orr]

Corinna Vinschen wrote:
> On Feb 16 20:02, Roger Orr wrote:
>> Corinna Vinschen wrote:
>>> So I'd think the best way forward is to update to the
>>> 1.7.35-0.1 test release and report further from there.
>> 
>> Thanks, this does help a little. However I will still be using the
>> 'files' setting.
> 
> The idea was to test this stuff to find a better solution which is
> acceptable.  If you simply revert to "files" without helping to test
> we'll probably never find the culprit.  

I'm very happy to continue testing; I was merely meaning the 1.7.35
performance is still not adequate in my environment.

> It would be nice to know what part of the code is so slow.  The
> LookupAccountSid calls shouldn't be so slow because they only fetch
> information already cached on the local machine.  So it's probably
> the LDAP call.  Why does an LDAP call take 4 secs?!?   
> 
> Are you remote from your DC, by any chance?

I have made some progress with analysis (slightly handicapped as I'm a
novice with ldap and am not an admin)

According to nltest /dclist:
Our environment has 6 London based DCs 

According to ldp.exe Live Enterprise Tree we have a tree structure for LDAP.

6 leaf nodes at the top matching ther 6 DCs
4 leaf nodes under an "AUS" (Australia) node
3 leaf nodes under a "CHI" (Chicago) node
and a few more similar to this in other regions.

When running mkpasswd I see active sessions to all the nodes in the tree on
port 389 (ldap)

I have tried using Sysinternals ADInsight (with a 32bit cygwin) to see what
requests are made with 'echo.exe'

There are two searches shown:

A) RootDSE:LDAP_SCOPE_BASE:(objectclass=*)  (1.113ms)
B) <London DNS>:LDAP_SCOPE_SUBTREE:((objectClass=trustedDomain) AND
(name=<Australian DNS>))     (4.426s)

I don't know why the second query is being made with the Australian DNS name
but I suspect this is the problem.
(Perhaps it as simple as A coming first in the alphabet ...)

Happy to investigate further if someone can suggest some useful avenues.

Regards,
Roger.





--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3251 bytes --]

Hi Roger,

On Feb 17 19:13, Roger Orr wrote:
> Corinna Vinschen wrote:
> > It would be nice to know what part of the code is so slow.  The
> > LookupAccountSid calls shouldn't be so slow because they only fetch
> > information already cached on the local machine.  So it's probably
> > the LDAP call.  Why does an LDAP call take 4 secs?!?   
> > 
> > Are you remote from your DC, by any chance?
> 
> I have made some progress with analysis (slightly handicapped as I'm a
> novice with ldap and am not an admin)
> 
> According to nltest /dclist:
> Our environment has 6 London based DCs 
> 
> According to ldp.exe Live Enterprise Tree we have a tree structure for LDAP.
> 
> 6 leaf nodes at the top matching ther 6 DCs
> 4 leaf nodes under an "AUS" (Australia) node
> 3 leaf nodes under a "CHI" (Chicago) node
> and a few more similar to this in other regions.
> 
> When running mkpasswd I see active sessions to all the nodes in the tree on
> port 389 (ldap)
> 
> I have tried using Sysinternals ADInsight (with a 32bit cygwin) to see what
> requests are made with 'echo.exe'
> 
> There are two searches shown:
> 
> A) RootDSE:LDAP_SCOPE_BASE:(objectclass=*)  (1.113ms)
> B) <London DNS>:LDAP_SCOPE_SUBTREE:((objectClass=trustedDomain) AND
> (name=<Australian DNS>))     (4.426s)
> 
> I don't know why the second query is being made with the Australian DNS name
> but I suspect this is the problem.

Thanks for doing that!  It's really cool to get this info since it seems
to point to the culprit.

It's not the problem that the Australian DNS is mentioned here.  This is
perfectly valid.  The LDAP query is going to the London DNS DC
(apparently, I hope that's right in your case) and the query is for
information on a trusted domain.  It looks like you have a group from
the australian domain in your user token.  To compute the gid of the
group, cygwin asks *your* DC for a value called "posixOffset" for *that*
trusted domain.

The bottom line is, this is not going to Australia, because all DCs have
this info for their trusted domains in their own DB so it's a planly
local query.

However, that mean this local LDAP query is *extremly* slow.  I changed
the query now to limit the scope of the database search.  This should speed
up the request a lot.

I've just built a new developer snapshot and uploaded it to
https://cygwin.com/snapshots/  The latest one is it.  Just replacing
the Cygwin DLL is sufficient for this.  Can you please run the above
timing test again with the developer snapshot DLL, please?

On second thought, there's another screw I could use to speed up this
specific LDAP query even more, but I won't be able to come up with the
change today anymore.  I'm going to provide another developer snapshot
tomorrow for another test, ok?  It would be very helpful probably if we
can get this trustedDomain query into the millisecond area as well.


Idle musing:  It's apparently quite a difference between a real-world
AD and the funny little AD I'm using for testing at home...


Thanks a lot,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2429 bytes --]

Hi Roger,

On Feb 17 22:32, Corinna Vinschen wrote:
> On Feb 17 19:13, Roger Orr wrote:
> > According to nltest /dclist:
> > Our environment has 6 London based DCs 
> > 
> > According to ldp.exe Live Enterprise Tree we have a tree structure for LDAP.
> > 
> > 6 leaf nodes at the top matching ther 6 DCs
> > 4 leaf nodes under an "AUS" (Australia) node
> > 3 leaf nodes under a "CHI" (Chicago) node
> > and a few more similar to this in other regions.
> > 
> > When running mkpasswd I see active sessions to all the nodes in the tree on
> > port 389 (ldap)
> > 
> > I have tried using Sysinternals ADInsight (with a 32bit cygwin) to see what
> > requests are made with 'echo.exe'
> > 
> > There are two searches shown:
> > 
> > A) RootDSE:LDAP_SCOPE_BASE:(objectclass=*)  (1.113ms)
> > B) <London DNS>:LDAP_SCOPE_SUBTREE:((objectClass=trustedDomain) AND
> > (name=<Australian DNS>))     (4.426s)
> > 
> > I don't know why the second query is being made with the Australian DNS name
> > but I suspect this is the problem.
> 
> Thanks for doing that!  It's really cool to get this info since it seems
> to point to the culprit.
> 
> It's not the problem that the Australian DNS is mentioned here.  This is
> perfectly valid.  The LDAP query is going to the London DNS DC
> (apparently, I hope that's right in your case) and the query is for
> information on a trusted domain.  It looks like you have a group from
> the australian domain in your user token.  To compute the gid of the
> group, cygwin asks *your* DC for a value called "posixOffset" for *that*
> trusted domain.
> 
> The bottom line is, this is not going to Australia, because all DCs have
> this info for their trusted domains in their own DB so it's a planly
> local query.
> 
> However, that mean this local LDAP query is *extremly* slow.  I changed
> the query now to limit the scope of the database search.  This should speed
> up the request a lot.
> [...etc...]

I just release a new test release, 1.7.35-0.3, see
https://cygwin.com/ml/cygwin-announce/2015-02/msg00133.html

This should speed up the search for the trustedDomain info a lot.

Can you please give it a try and perform your fantastic timing test as
above?


Thanks in advance,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: slow startup after upgrade

[Roger Orr]

Hello Corinna,

I've just been trying out both the 2015-02-18 10:30:19/44 UTC and 2015-02-17 21:27:23/48 UTC patches.

Both are now down to the same timings as with a 'files' entry in /etc/nsswitch.cfg, (and there's no detectable speed difference between them.)

The scope restriction in the second query to \System reduces the query time to 1.1 - 1.3 ms (was 4 seconds) and also it no longer opens 14 TCP/IP sessions to various ldap servers around the planet (!)

I note that mkpasswd and mkgroup do still open many sessions to the ldap servers, but that may be inevitable. It's not an issue directly, of course, since I'll no longer need to make use of these, but it perhaps might indicate another place where the ldap queries are sub-optimal.

Thanks for your rapid response on this issue!
Regards,
Roger.
________________________________________
From: Corinna Vinschen [corinna-cygwin@cygwin.com]
Sent: 18 February 2015 11:18
To: cygwin@cygwin.com
Cc: Roger Orr
Subject: Re: slow startup after upgrade

Hi Roger,

On Feb 17 22:32, Corinna Vinschen wrote:
> On Feb 17 19:13, Roger Orr wrote:
> > According to nltest /dclist:
> > Our environment has 6 London based DCs
> >
> > According to ldp.exe Live Enterprise Tree we have a tree structure for LDAP.
> >
> > 6 leaf nodes at the top matching ther 6 DCs
> > 4 leaf nodes under an "AUS" (Australia) node
> > 3 leaf nodes under a "CHI" (Chicago) node
> > and a few more similar to this in other regions.
> >
> > When running mkpasswd I see active sessions to all the nodes in the tree on
> > port 389 (ldap)
> >
> > I have tried using Sysinternals ADInsight (with a 32bit cygwin) to see what
> > requests are made with 'echo.exe'
> >
> > There are two searches shown:
> >
> > A) RootDSE:LDAP_SCOPE_BASE:(objectclass=*)  (1.113ms)
> > B) <London DNS>:LDAP_SCOPE_SUBTREE:((objectClass=trustedDomain) AND
> > (name=<Australian DNS>))     (4.426s)
> >
> > I don't know why the second query is being made with the Australian DNS name
> > but I suspect this is the problem.
>
> Thanks for doing that!  It's really cool to get this info since it seems
> to point to the culprit.
>
> It's not the problem that the Australian DNS is mentioned here.  This is
> perfectly valid.  The LDAP query is going to the London DNS DC
> (apparently, I hope that's right in your case) and the query is for
> information on a trusted domain.  It looks like you have a group from
> the australian domain in your user token.  To compute the gid of the
> group, cygwin asks *your* DC for a value called "posixOffset" for *that*
> trusted domain.
>
> The bottom line is, this is not going to Australia, because all DCs have
> this info for their trusted domains in their own DB so it's a planly
> local query.
>
> However, that mean this local LDAP query is *extremly* slow.  I changed
> the query now to limit the scope of the database search.  This should speed
> up the request a lot.
> [...etc...]

I just release a new test release, 1.7.35-0.3, see
https://cygwin.com/ml/cygwin-announce/2015-02/msg00133.html

This should speed up the search for the trustedDomain info a lot.

Can you please give it a try and perform your fantastic timing test as
above?


Thanks in advance,
Corinna

--
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: slow startup after upgrade

[Roger Orr]

Hello Corinna,

I've just been trying out both the 2015-02-18 10:30:19/44 UTC and 2015-02-17 21:27:23/48 UTC patches.

Both are now down to the same timings as with a 'files' entry in /etc/nsswitch.cfg, (and there's no detectable speed difference between them.)

The scope restriction in the second query to \System reduces the query time to 1.1 - 1.3 ms (was 4 seconds) and also it no longer opens 14 TCP/IP sessions to various ldap servers around the planet (!)

I note that mkpasswd and mkgroup do still open many sessions to the ldap servers, but that may be inevitable. It's not an issue directly, of course, since I'll no longer need to make use of these, but it perhaps might indicate another place where the ldap queries are sub-optimal.

Thanks for your rapid response on this issue!
Regards,
Roger.
________________________________________
From: Corinna Vinschen [corinna-cygwin@cygwin.com]
Sent: 18 February 2015 11:18
To: cygwin@cygwin.com
Cc: Roger Orr
Subject: Re: slow startup after upgrade

Hi Roger,

On Feb 17 22:32, Corinna Vinschen wrote:
> On Feb 17 19:13, Roger Orr wrote:
> > According to nltest /dclist:
> > Our environment has 6 London based DCs
> >
> > According to ldp.exe Live Enterprise Tree we have a tree structure for LDAP.
> >
> > 6 leaf nodes at the top matching ther 6 DCs
> > 4 leaf nodes under an "AUS" (Australia) node
> > 3 leaf nodes under a "CHI" (Chicago) node
> > and a few more similar to this in other regions.
> >
> > When running mkpasswd I see active sessions to all the nodes in the tree on
> > port 389 (ldap)
> >
> > I have tried using Sysinternals ADInsight (with a 32bit cygwin) to see what
> > requests are made with 'echo.exe'
> >
> > There are two searches shown:
> >
> > A) RootDSE:LDAP_SCOPE_BASE:(objectclass=*)  (1.113ms)
> > B) <London DNS>:LDAP_SCOPE_SUBTREE:((objectClass=trustedDomain) AND
> > (name=<Australian DNS>))     (4.426s)
> >
> > I don't know why the second query is being made with the Australian DNS name
> > but I suspect this is the problem.
>
> Thanks for doing that!  It's really cool to get this info since it seems
> to point to the culprit.
>
> It's not the problem that the Australian DNS is mentioned here.  This is
> perfectly valid.  The LDAP query is going to the London DNS DC
> (apparently, I hope that's right in your case) and the query is for
> information on a trusted domain.  It looks like you have a group from
> the australian domain in your user token.  To compute the gid of the
> group, cygwin asks *your* DC for a value called "posixOffset" for *that*
> trusted domain.
>
> The bottom line is, this is not going to Australia, because all DCs have
> this info for their trusted domains in their own DB so it's a planly
> local query.
>
> However, that mean this local LDAP query is *extremly* slow.  I changed
> the query now to limit the scope of the database search.  This should speed
> up the request a lot.
> [...etc...]

I just release a new test release, 1.7.35-0.3, see
https://cygwin.com/ml/cygwin-announce/2015-02/msg00133.html

This should speed up the search for the trustedDomain info a lot.

Can you please give it a try and perform your fantastic timing test as
above?


Thanks in advance,
Corinna

--
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2119 bytes --]

Hi Roger,

On Feb 18 11:26, Roger Orr wrote:
> Hello Corinna,
> 
> I've just been trying out both the 2015-02-18 10:30:19/44 UTC and
> 2015-02-17 21:27:23/48 UTC patches.
> 
> Both are now down to the same timings as with a 'files' entry in
> /etc/nsswitch.cfg, (and there's no detectable speed difference between
> them.)
> 
> The scope restriction in the second query to \System reduces the query
> time to 1.1 - 1.3 ms (was 4 seconds)

Wow!  That's what I had hoped for but it's really incredible to read that.

> and also it no longer opens 14
> TCP/IP sessions to various ldap servers around the planet (!)

Uh, that might be the result of the other changes which don't open an
LDAP connection to fetch group info.  14 connections probably means,
you're in 14 groups in other domains than your login domain.

> I note that mkpasswd and mkgroup do still open many sessions to the
> ldap servers, but that may be inevitable.

Cygwin is using a bulk LDAP request, fetching 100 entries in each call.
I'm not quite sure how all that works under the LDAP library hood, but
one one hand mkpasswd/mkgroup need to make a lot of requests if you
enumerate your entire organization, and every domain needs its own
connection.

> It's not an issue directly,
> of course, since I'll no longer need to make use of these, but it
> perhaps might indicate another place where the ldap queries are
> sub-optimal.

The enumeration queries got the same treatment, so I assume this is
really just a result of having to enumerate all the domains.

> Thanks for your rapid response on this issue!

Same to you!  I'm glad to get testers in such big environments since, as
I said, it's kinda hard to test big stuff in a tiny domain like mine at
home :}

Final question:  Given the above improvement, are you going to run in
"db" only setting for the time being?  That would make some good long
time test... :)


Thanks a bunch,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: slow startup after upgrade

[Roger Orr]

> > and also it no longer opens 14
> > TCP/IP sessions to various ldap servers around the planet (!)
>
> Uh, that might be the result of the other changes which don't open an
> LDAP connection to fetch group info.  14 connections probably means,
> you're in 14 groups in other domains than your login domain.

There weren't any other LDAP requests logged (I was testing with the first
patch 1.7.35-0.1 that reduced the time for running "echo.exe" from 120s to
4.5s) so I don't think it was related to another query.  I may be able to
test this out again tomorrow and get the call stacks of the connect calss.

Incidentally how can you tell the patch level of cygwin1.dll -- the DLL
versions all seem to be 1007.35.0.0 ?

Regards,
Roger.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Marco Atzeri]

On 2/18/2015 11:01 PM, Roger Orr wrote:

>
> Incidentally how can you tell the patch level of cygwin1.dll -- the DLL
> versions all seem to be 1007.35.0.0 ?

uname -a


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1860 bytes --]

On Feb 18 22:01, Roger Orr wrote:
> > > and also it no longer opens 14
> > > TCP/IP sessions to various ldap servers around the planet (!)
> >
> > Uh, that might be the result of the other changes which don't open an
> > LDAP connection to fetch group info.  14 connections probably means,
> > you're in 14 groups in other domains than your login domain.
> 
> There weren't any other LDAP requests logged (I was testing with the first
> patch 1.7.35-0.1 that reduced the time for running "echo.exe" from 120s to
> 4.5s) so I don't think it was related to another query.

When you start a Cygwin process from a non-Cygwin process, Cygwin will
try to generate passwd entries for your account, as well as group
entries for all groups in your token.  For the group entries it only
needs LDAP calls if a group is in another domain than your login domain,
and the request only goes to your own DC.  This entire set of LDAP
calls are supposed to share the same LDAP connection.

I don't know what Windows is doing under the hood, but in theory there
should only be a single socket open for this.

> I may be able to
> test this out again tomorrow and get the call stacks of the connect calss.
> 
> Incidentally how can you tell the patch level of cygwin1.dll -- the DLL
> versions all seem to be 1007.35.0.0 ?

The versioning system isn't able to handle subversions.  As marco wrote,
uname -a (the build date or uname -v) can be used to distinguish the
versions.

I'm planning to change that at one point.  It shouldn't be too hard
to add the subversion to the DLL properties.  uname -r is a problem,
though, since the info already takes 18 of 19 allowed characters...


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: slow startup after upgrade

[Roger Orr]

I've tested again with the first patched cygwin1.dll:
CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35(0.286/5/3) 2015-02-16 13:18 i686 Cygwin

I can confirm the connections are occurring within the ldap_search_s call - here is one of the call stacks:

00ebc31c 76e5c451 00000278 0045dd28 00000010 WS2_32!connect (FPO: [Non-Fpo])
00ebc87c 76e5cad5 00462758 00ebc8a4 00000185 wldap32!LdapParallelConnect+0x2e3 (FPO: [Non-Fpo])
00ebca70 76e597a2 00462758 004568a0 00000000 wldap32!ConnectToSRVrecs+0x21b (FPO: [Non-Fpo])
00ebcac8 76e59688 00000000 00000000 00462758 wldap32!OpenLdapServer+0x612 (FPO: [Non-Fpo])
00ebcae8 76e62ca9 00462758 00000000 00000000 wldap32!LdapConnect+0x2cf (FPO: [Non-Fpo])
00ebcb58 76e63021 00432670 00000000 76e8e158 wldap32!LdapChaseReferral+0xb27 (FPO: [Non-Fpo])
00ebcb98 76e62e1d 0042ca00 004328b0 00432670 wldap32!HandleReferral+0x7ac (FPO: [Non-Fpo])
00ebcbd4 76e553e2 0042cab8 00ebcc04 00000000 wldap32!HandleReferrals+0x151 (FPO: [Non-Fpo])
00ebcc08 76e5b0f8 0042cab8 00000005 00000001 wldap32!ldap_result_with_error+0x30e (FPO: [Non-Fpo])
00ebcc3c 76e5e584 0042cce4 80039620 00000002 wldap32!ldap_search_ext_sW+0x87 (FPO: [Non-Fpo])
00ebcc80 76e5e783 0042cce4 80039620 00000002 wldap32!ldap_search_stW+0x45 (FPO: [Non-Fpo])
00ebcca8 610818e2 0042cce4 80039620 00000002 wldap32!ldap_search_sW+0x21 (FPO: [Non-Fpo])

I can see this occurring with "ldp.exe" (from Windows Server 2003 support tools; also works on newer version of windows) and "netstat.exe"

Connection->Connect (default server, default port 389)
(1 TCP/IP session to DC1:389)

Connection->Bind (enter username and password)
(no new sessions)

Browse->Search

Base Dn - first naming context
Filter: (&(objectClass=trustedDomain)(name=wibble))
Gets 0 entries, quickly, no extra sessions visible


Click 'Options'
Select 'Chase Referrals'
Click Ok

Search again.
Gets 0 entries, takes some seconds, and establishes nuermous TCP/IP connections.

(1) LDAP_OPT_REFERRALS is on by default
(2) The fix added CN=System to the Base Dn - this seems to turn off referrals

--
*aside*
Sysinternals "ADInsight" is a 32bit only tool and, in order to work on a 64bit Windows you seem to have to manually inject the DLL ADInsightDll.dll (which is extracted into %TEMP%) into the target (32-bit!) process.

Regards,
Roger.

________________________________________
From: Roger Orr
Sent: 18 February 2015 11:26
To: Corinna Vinschen
Cc: cygwin@cygwin.com
Subject: RE: slow startup after upgrade

Hello Corinna,

I've just been trying out both the 2015-02-18 10:30:19/44 UTC and 2015-02-17 21:27:23/48 UTC patches.

Both are now down to the same timings as with a 'files' entry in /etc/nsswitch.cfg, (and there's no detectable speed difference between them.)

The scope restriction in the second query to \System reduces the query time to 1.1 - 1.3 ms (was 4 seconds) and also it no longer opens 14 TCP/IP sessions to various ldap servers around the planet (!)

I note that mkpasswd and mkgroup do still open many sessions to the ldap servers, but that may be inevitable. It's not an issue directly, of course, since I'll no longer need to make use of these, but it perhaps might indicate another place where the ldap queries are sub-optimal.

Thanks for your rapid response on this issue!
Regards,
Roger.
________________________________________
From: Corinna Vinschen [corinna-cygwin@cygwin.com]
Sent: 18 February 2015 11:18
To: cygwin@cygwin.com
Cc: Roger Orr
Subject: Re: slow startup after upgrade

Hi Roger,

On Feb 17 22:32, Corinna Vinschen wrote:
> On Feb 17 19:13, Roger Orr wrote:
> > According to nltest /dclist:
> > Our environment has 6 London based DCs
> >
> > According to ldp.exe Live Enterprise Tree we have a tree structure for LDAP.
> >
> > 6 leaf nodes at the top matching ther 6 DCs
> > 4 leaf nodes under an "AUS" (Australia) node
> > 3 leaf nodes under a "CHI" (Chicago) node
> > and a few more similar to this in other regions.
> >
> > When running mkpasswd I see active sessions to all the nodes in the tree on
> > port 389 (ldap)
> >
> > I have tried using Sysinternals ADInsight (with a 32bit cygwin) to see what
> > requests are made with 'echo.exe'
> >
> > There are two searches shown:
> >
> > A) RootDSE:LDAP_SCOPE_BASE:(objectclass=*)  (1.113ms)
> > B) <London DNS>:LDAP_SCOPE_SUBTREE:((objectClass=trustedDomain) AND
> > (name=<Australian DNS>))     (4.426s)
> >
> > I don't know why the second query is being made with the Australian DNS name
> > but I suspect this is the problem.
>
> Thanks for doing that!  It's really cool to get this info since it seems
> to point to the culprit.
>
> It's not the problem that the Australian DNS is mentioned here.  This is
> perfectly valid.  The LDAP query is going to the London DNS DC
> (apparently, I hope that's right in your case) and the query is for
> information on a trusted domain.  It looks like you have a group from
> the australian domain in your user token.  To compute the gid of the
> group, cygwin asks *your* DC for a value called "posixOffset" for *that*
> trusted domain.
>
> The bottom line is, this is not going to Australia, because all DCs have
> this info for their trusted domains in their own DB so it's a planly
> local query.
>
> However, that mean this local LDAP query is *extremly* slow.  I changed
> the query now to limit the scope of the database search.  This should speed
> up the request a lot.
> [...etc...]

I just release a new test release, 1.7.35-0.3, see
https://cygwin.com/ml/cygwin-announce/2015-02/msg00133.html

This should speed up the search for the trustedDomain info a lot.

Can you please give it a try and perform your fantastic timing test as
above?


Thanks in advance,
Corinna

--
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1543 bytes --]

On Feb 19 11:53, Roger Orr wrote:
> I've tested again with the first patched cygwin1.dll:
> CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35(0.286/5/3) 2015-02-16 13:18 i686 Cygwin
> 
> I can confirm the connections are occurring within the ldap_search_s
> call - here is one of the call stacks:
> [...]

AFAICS that's expected when enumerating accounts.

> Connection->Bind (enter username and password)
> (no new sessions)
> 
> Browse->Search
> 
> Base Dn - first naming context
> Filter: (&(objectClass=trustedDomain)(name=wibble))
> Gets 0 entries, quickly, no extra sessions visible
> 
> 
> Click 'Options'
> Select 'Chase Referrals'
> Click Ok
> 
> Search again.
> Gets 0 entries, takes some seconds, and establishes nuermous TCP/IP connections.

Hunting down the other DCs I assume.

> (1) LDAP_OPT_REFERRALS is on by default
> (2) The fix added CN=System to the Base Dn - this seems to turn off referrals

I don't think it does.  The domain info is available on the DC you're
contacting so it doesn't have to ask others.  I assume here that
"wibble" is no valid domain name ;)

> --
> *aside*
> Sysinternals "ADInsight" is a 32bit only tool and, in order to work on
> a 64bit Windows you seem to have to manually inject the DLL
> ADInsightDll.dll (which is extracted into %TEMP%) into the target
> (32-bit!) process.

Uh, too bad.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1403 bytes --]

Hi Roger,

On Feb 16 20:02, Roger Orr wrote:
> Corinna Vinschen wrote:
> > So I'd think the best way forward is to update to the
> > 1.7.35-0.1 test release and report further from there.     
> 
> Thanks, this does help a little. However I will still be using the 'files'
> setting.
> 
> Here are some results in case they're of interest. (Windows 7/64 with
> cygwin/64.)
> 
> 1) Running cygwin "echo.exe" from a cmd.exe command shell
> 
> a) With passwd: files and group: files in /etc/nsswitch.conf
>   0.03 - 0.4 s
> 
> b) With 1.7.34 and default /etc/nsswitch.conf
> 
>   around 120s
> 
> C) With 1.7.35 and default /etc/nsswitch.conf
> 
>   4.4 - 4.6s

I rewrote the function responsible for the slow startup, the one
fetching all the group information for the groups in your user token.

As I just wrote in my mail to Dennis Hagarty, the performance improvement
should be noticable, even with /etc/nsswitch.conf only using the "db"
setting for groups.  The group info for a user token with 150 groups was
fetched in ~50 ms rather than the ~300ms of the former code.

Can you test this again with the Cygwin DLL from the latest developer
snaphshot at https://cygwin.com/snapshots/ please?


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: slow startup after upgrade

[Roger Orr]

Hi Corinna,
sounds good. I'll give this a go, all being well, tomorrow.

In other news it appears that the ADInsightDll uses some shared data to
communicate, so in order to get ADInsight to work with injection one has to
ensure that the DLL injected is the same actual *file* as the one loaded by
the ADInsight program (in the (windows) %TEMP% directory) - a copy of the
DLL doesn't seem to be effective.

Regards,
Roger.

-----Original Message-----
From: Corinna Vinschen [mailto:corinna-cygwin@cygwin.com] 
Sent: 23 February 2015 21:16
To: cygwin@cygwin.com
Cc: Roger Orr
Subject: Re: slow startup after upgrade

Hi Roger,

On Feb 16 20:02, Roger Orr wrote:
> Corinna Vinschen wrote:
> > So I'd think the best way forward is to update to the
> > 1.7.35-0.1 test release and report further from there.     
> 
> Thanks, this does help a little. However I will still be using the 'files'
> setting.
> 
> Here are some results in case they're of interest. (Windows 7/64 with
> cygwin/64.)
> 
> 1) Running cygwin "echo.exe" from a cmd.exe command shell
> 
> a) With passwd: files and group: files in /etc/nsswitch.conf
>   0.03 - 0.4 s
> 
> b) With 1.7.34 and default /etc/nsswitch.conf
> 
>   around 120s
> 
> C) With 1.7.35 and default /etc/nsswitch.conf
> 
>   4.4 - 4.6s

I rewrote the function responsible for the slow startup, the one
fetching all the group information for the groups in your user token.

As I just wrote in my mail to Dennis Hagarty, the performance improvement
should be noticable, even with /etc/nsswitch.conf only using the "db"
setting for groups.  The group info for a user token with 150 groups was
fetched in ~50 ms rather than the ~300ms of the former code.

Can you test this again with the Cygwin DLL from the latest developer
snaphshot at https://cygwin.com/snapshots/ please?


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1952 bytes --]

On Feb 26 16:18, Eric Blake wrote:
> On 02/26/2015 03:31 PM, Corinna Vinschen wrote:
> > On Feb 26 14:19, Eric Blake wrote:
> >> On 02/26/2015 01:24 PM, Corinna Vinschen wrote:
> >>>>> Fixes are in snapshots, as well as test releases (currently 1.7.35-0.4)
> >>>>> which can be installed with setup*.exe.
> >>>>
> >>>> Hmmm... https://cygwin.com/faq-nochunks.html#faq.setup.snapshots says:
> >>>>
> >>>> You cannot use Cygwin Setup to install a snapshot.
> >>>
> >>> Hmmm... maybe you should try to read Yaakov's entire sentence?
> >>
> >> Or maybe his point was that we need to update the FAQ wording now that
> >> it is out of date and no longer matching reality.
> > 
> > I don't understand.  It's still not possible to install a snapshot with
> > setup.  What's wrong with the FAQ entry?
> 
> Oh, I was equating test releases (1.7.35-0.4) with snapshots (since each
> test release has matched a particular snapshot),

Only because I did it so.  There's no automatism.

> but I guess you are
> distinguishing between them (since we have a lot more snapshots than
> what gets promoted to test release).  So maybe all we need is something
> along the lines of:
> 
> In general, snapshots cannot be installed via setup; however, there have
> been cases of a snapshot promoted to a test release that can be
> installed using the 'Experimental' know within setup.

I don't want them to be mixed into the same pot.  Test releases are
probably not bug-free, but in general I expect them to be a bit more
stable than a snapshot.  A snapshot is allowed to screw up terribly, a
test release isn't (or shouldn't).  That I created snapshots together
with the test releases was just to make sure that snapshot were not
in a worse shape than the test releases.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1970 bytes --]

On Feb 26 16:00, Andrew DeFaria wrote:
> On 2/26/2015 2:34 PM, Corinna Vinschen wrote:
> >On Feb 26 14:21, Andrew DeFaria wrote:
> >>On 2/26/2015 12:24 PM, Corinna Vinschen wrote:
> >>>On Feb 26 12:04, Andrew DeFaria wrote:
> >>>>On 2/25/2015 4:12 PM, Yaakov Selkowitz wrote:
> >>>>>On Wed, 2015-02-25 at 15:52 -0800, Andrew DeFaria wrote:
> >>>>>>Can somebody summarize where we're at here. I've been noticing all this
> >>>>>>email about slow startup and I'm excited by the inclusion of domain
> >>>>>>accounts using /etc/nsswitch.conf, etc. I downloaded a newer version of
> >>>>>>Cygwin a few weeks ago (I'm seeing 1.7.34) and I experienced the
> >>>>>>slowdown myself. Unfortunately I'm too busy to actively participate but
> >>>>>>it looks like some good progress has been made. Should I update or is
> >>>>>>everything still in snapshots?
> >>>>>
> >>>>>Fixes are in snapshots, as well as test releases (currently 1.7.35-0.4)
> >>>>>which can be installed with setup*.exe.
> >>>>
> >>>>Hmmm... https://cygwin.com/faq-nochunks.html#faq.setup.snapshots says:
> >>>>
> >>>>You cannot use Cygwin Setup to install a snapshot.
> >>>
> >>>Hmmm... maybe you should try to read Yaakov's entire sentence?
> >>>
> >>>
> >>>Corinna
> >>>
> >>
> >>It is not clear from Yaakov's sentence that the phrase "which can be
> >>installed with setup*.exe" applied to test releases, snapshots or both.
> >
> >So why didn't you just look for the test release in setup then?  I don't
> >see how complaining about the exact wording of a mail trying to be
> >helpful(!) does improve your situation.
> 
> Chill man! First off I wasn't complaining - I was simply pointing out

Shall I point out the error in your expression, or would you think this
is totally unnecessary?


Just saying...
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: slow startup after upgrade

[Andrew DeFaria]

On 2/26/2015 2:34 PM, Corinna Vinschen wrote:
> On Feb 26 14:21, Andrew DeFaria wrote:
>> On 2/26/2015 12:24 PM, Corinna Vinschen wrote:
>>> On Feb 26 12:04, Andrew DeFaria wrote:
>>>> On 2/25/2015 4:12 PM, Yaakov Selkowitz wrote:
>>>>> On Wed, 2015-02-25 at 15:52 -0800, Andrew DeFaria wrote:
>>>>>> Can somebody summarize where we're at here. I've been noticing all this
>>>>>> email about slow startup and I'm excited by the inclusion of domain
>>>>>> accounts using /etc/nsswitch.conf, etc. I downloaded a newer version of
>>>>>> Cygwin a few weeks ago (I'm seeing 1.7.34) and I experienced the
>>>>>> slowdown myself. Unfortunately I'm too busy to actively participate but
>>>>>> it looks like some good progress has been made. Should I update or is
>>>>>> everything still in snapshots?
>>>>>
>>>>> Fixes are in snapshots, as well as test releases (currently 1.7.35-0.4)
>>>>> which can be installed with setup*.exe.
>>>>
>>>> Hmmm... https://cygwin.com/faq-nochunks.html#faq.setup.snapshots says:
>>>>
>>>> You cannot use Cygwin Setup to install a snapshot.
>>>
>>> Hmmm... maybe you should try to read Yaakov's entire sentence?
>>>
>>>
>>> Corinna
>>>
>>
>> It is not clear from Yaakov's sentence that the phrase "which can be
>> installed with setup*.exe" applied to test releases, snapshots or both.
>
> So why didn't you just look for the test release in setup then?  I don't
> see how complaining about the exact wording of a mail trying to be
> helpful(!) does improve your situation.

Chill man! First off I wasn't complaining - I was simply pointing out

Secondly, why didn't I just install from a test release? Because like 
most of us here I have a real job and occasionally fires happen which I 
need to drop everything else and help put out. I've never installed a 
test release and though I was about to look into it, emergencies cropped 
up, alright?
-- 
Andrew DeFaria
http://defaria.com


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1357 bytes --]

On 02/26/2015 03:31 PM, Corinna Vinschen wrote:
> On Feb 26 14:19, Eric Blake wrote:
>> On 02/26/2015 01:24 PM, Corinna Vinschen wrote:
>>>>> Fixes are in snapshots, as well as test releases (currently 1.7.35-0.4)
>>>>> which can be installed with setup*.exe.
>>>>
>>>> Hmmm... https://cygwin.com/faq-nochunks.html#faq.setup.snapshots says:
>>>>
>>>> You cannot use Cygwin Setup to install a snapshot.
>>>
>>> Hmmm... maybe you should try to read Yaakov's entire sentence?
>>
>> Or maybe his point was that we need to update the FAQ wording now that
>> it is out of date and no longer matching reality.
> 
> I don't understand.  It's still not possible to install a snapshot with
> setup.  What's wrong with the FAQ entry?

Oh, I was equating test releases (1.7.35-0.4) with snapshots (since each
test release has matched a particular snapshot), but I guess you are
distinguishing between them (since we have a lot more snapshots than
what gets promoted to test release).  So maybe all we need is something
along the lines of:

In general, snapshots cannot be installed via setup; however, there have
been cases of a snapshot promoted to a test release that can be
installed using the 'Experimental' know within setup.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1619 bytes --]

On Feb 26 14:21, Andrew DeFaria wrote:
> On 2/26/2015 12:24 PM, Corinna Vinschen wrote:
> >On Feb 26 12:04, Andrew DeFaria wrote:
> >>On 2/25/2015 4:12 PM, Yaakov Selkowitz wrote:
> >>>On Wed, 2015-02-25 at 15:52 -0800, Andrew DeFaria wrote:
> >>>>Can somebody summarize where we're at here. I've been noticing all this
> >>>>email about slow startup and I'm excited by the inclusion of domain
> >>>>accounts using /etc/nsswitch.conf, etc. I downloaded a newer version of
> >>>>Cygwin a few weeks ago (I'm seeing 1.7.34) and I experienced the
> >>>>slowdown myself. Unfortunately I'm too busy to actively participate but
> >>>>it looks like some good progress has been made. Should I update or is
> >>>>everything still in snapshots?
> >>>
> >>>Fixes are in snapshots, as well as test releases (currently 1.7.35-0.4)
> >>>which can be installed with setup*.exe.
> >>
> >>Hmmm... https://cygwin.com/faq-nochunks.html#faq.setup.snapshots says:
> >>
> >>You cannot use Cygwin Setup to install a snapshot.
> >
> >Hmmm... maybe you should try to read Yaakov's entire sentence?
> >
> >
> >Corinna
> >
> 
> It is not clear from Yaakov's sentence that the phrase "which can be
> installed with setup*.exe" applied to test releases, snapshots or both.

So why didn't you just look for the test release in setup then?  I don't
see how complaining about the exact wording of a mail trying to be
helpful(!) does improve your situation.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 844 bytes --]

On Feb 26 14:19, Eric Blake wrote:
> On 02/26/2015 01:24 PM, Corinna Vinschen wrote:
> >>> Fixes are in snapshots, as well as test releases (currently 1.7.35-0.4)
> >>> which can be installed with setup*.exe.
> >>
> >> Hmmm... https://cygwin.com/faq-nochunks.html#faq.setup.snapshots says:
> >>
> >> You cannot use Cygwin Setup to install a snapshot.
> > 
> > Hmmm... maybe you should try to read Yaakov's entire sentence?
> 
> Or maybe his point was that we need to update the FAQ wording now that
> it is out of date and no longer matching reality.

I don't understand.  It's still not possible to install a snapshot with
setup.  What's wrong with the FAQ entry?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: slow startup after upgrade

[Andrew DeFaria]

On 2/26/2015 12:24 PM, Corinna Vinschen wrote:
> On Feb 26 12:04, Andrew DeFaria wrote:
>> On 2/25/2015 4:12 PM, Yaakov Selkowitz wrote:
>>> On Wed, 2015-02-25 at 15:52 -0800, Andrew DeFaria wrote:
>>>> Can somebody summarize where we're at here. I've been noticing all this
>>>> email about slow startup and I'm excited by the inclusion of domain
>>>> accounts using /etc/nsswitch.conf, etc. I downloaded a newer version of
>>>> Cygwin a few weeks ago (I'm seeing 1.7.34) and I experienced the
>>>> slowdown myself. Unfortunately I'm too busy to actively participate but
>>>> it looks like some good progress has been made. Should I update or is
>>>> everything still in snapshots?
>>>
>>> Fixes are in snapshots, as well as test releases (currently 1.7.35-0.4)
>>> which can be installed with setup*.exe.
>>
>> Hmmm... https://cygwin.com/faq-nochunks.html#faq.setup.snapshots says:
>>
>> You cannot use Cygwin Setup to install a snapshot.
>
> Hmmm... maybe you should try to read Yaakov's entire sentence?
>
>
> Corinna
>

It is not clear from Yaakov's sentence that the phrase "which can be 
installed with setup*.exe" applied to test releases, snapshots or both.
-- 
Andrew DeFaria
http://defaria.com


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 615 bytes --]

On 02/26/2015 01:24 PM, Corinna Vinschen wrote:
>>> Fixes are in snapshots, as well as test releases (currently 1.7.35-0.4)
>>> which can be installed with setup*.exe.
>>
>> Hmmm... https://cygwin.com/faq-nochunks.html#faq.setup.snapshots says:
>>
>> You cannot use Cygwin Setup to install a snapshot.
> 
> Hmmm... maybe you should try to read Yaakov's entire sentence?

Or maybe his point was that we need to update the FAQ wording now that
it is out of date and no longer matching reality.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1125 bytes --]

On Feb 26 12:04, Andrew DeFaria wrote:
> On 2/25/2015 4:12 PM, Yaakov Selkowitz wrote:
> >On Wed, 2015-02-25 at 15:52 -0800, Andrew DeFaria wrote:
> >>Can somebody summarize where we're at here. I've been noticing all this
> >>email about slow startup and I'm excited by the inclusion of domain
> >>accounts using /etc/nsswitch.conf, etc. I downloaded a newer version of
> >>Cygwin a few weeks ago (I'm seeing 1.7.34) and I experienced the
> >>slowdown myself. Unfortunately I'm too busy to actively participate but
> >>it looks like some good progress has been made. Should I update or is
> >>everything still in snapshots?
> >
> >Fixes are in snapshots, as well as test releases (currently 1.7.35-0.4)
> >which can be installed with setup*.exe.
> 
> Hmmm... https://cygwin.com/faq-nochunks.html#faq.setup.snapshots says:
> 
> You cannot use Cygwin Setup to install a snapshot.

Hmmm... maybe you should try to read Yaakov's entire sentence?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: slow startup after upgrade

[Andrew DeFaria]

On 2/25/2015 4:12 PM, Yaakov Selkowitz wrote:
> On Wed, 2015-02-25 at 15:52 -0800, Andrew DeFaria wrote:
>> Can somebody summarize where we're at here. I've been noticing all this
>> email about slow startup and I'm excited by the inclusion of domain
>> accounts using /etc/nsswitch.conf, etc. I downloaded a newer version of
>> Cygwin a few weeks ago (I'm seeing 1.7.34) and I experienced the
>> slowdown myself. Unfortunately I'm too busy to actively participate but
>> it looks like some good progress has been made. Should I update or is
>> everything still in snapshots?
>
> Fixes are in snapshots, as well as test releases (currently 1.7.35-0.4)
> which can be installed with setup*.exe.

Hmmm... https://cygwin.com/faq-nochunks.html#faq.setup.snapshots says:

You cannot use Cygwin Setup to install a snapshot.

???
-- 
Andrew DeFaria
http://defaria.com


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 533 bytes --]

On Feb 25 22:20, Roger Orr wrote:
> Good work -- at least in my environment ;-)
> 
> 20150225 DLL:
> mkgroup 0.63s
> mkpasswd 0.289s
> 
> compared to
> 
> 20150220 DLL:
> mkgroup 45.8s
> mkpasswd: 4572.7s
> 
> Output is
>  mkgroup: 53kb, 681 lines
>  mkpasswd: 132kb, 1081 lines.
> 
> And the output *is* the same :-)

Excellent, thanks for testing!


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: slow startup after upgrade

[John Hein]

Andrey Repin wrote at 00:42 +0300 on Feb 26, 2015:
 > >> On problematic winxp box:
 > >>     [passwd, group, db_shell: 'files db', 'files db' & /bin/dash]
 > >>       16.83, 17.03, 24.80, 28.79, 27.99, 27.39 sec
 > >>
 > >>     [passwd, group, db_shell: db, db & /bin/dash]
 > >>       23.51, 27.91, 28.78 sec
 >
 > > Doesn't sound good at all.  Terrible in fact.  That a 32 or 64 bit XP?
 > > I didn't fire up my XP test machines for ages and I'm not overly enjoying
 > > it, but I guess I have to.  Oh well, hang on...
 >
 > > [...time passes...]
 > > [...starting up...]
 > > [...XP...]
 > > [...32 bit...]
 > > [...updating...]
 > > [...testing...]
 >
 > > Hmm, no.  In my case there's no visible delay starting mintty, and tcsh
 > > is my login shell, too.  mkpasswd and mkgroup are running comparably
 > > fast as on my W8.1 64 bit machine I'm using for testing in the first
 > > place.
 >
 > > Something strange is going on on your XP machine, it seems.
 >
 > Although I've migrated to Win7, I'm still using XP systems extensively, many
 > of them have Cygwin, and I didn't see anything even remotely close to his
 > results, ever.
 > His messages basically left me speechless, I was afraid to guess.
 > The half a minute startup of a shell, I could only attribute to environmental
 > issues.

I would not be surprised if some of the malware... err software that
our IT folks load onto windows installations is causing some problem
(BLODA-like effects in cygwin parlance).  If other XP users in AD
domains are not having similar trouble, I'm content to chalk this up
to a local issue (that is perhaps tickled by some interaction with
cygwin 1.7.34+).  For now cygserver is an acceptible workaround and
some day this XP box will be retired (wiped and replaced with
something other than windows).


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Yaakov Selkowitz]

On Wed, 2015-02-25 at 15:52 -0800, Andrew DeFaria wrote:
> Can somebody summarize where we're at here. I've been noticing all this 
> email about slow startup and I'm excited by the inclusion of domain 
> accounts using /etc/nsswitch.conf, etc. I downloaded a newer version of 
> Cygwin a few weeks ago (I'm seeing 1.7.34) and I experienced the 
> slowdown myself. Unfortunately I'm too busy to actively participate but 
> it looks like some good progress has been made. Should I update or is 
> everything still in snapshots?

Fixes are in snapshots, as well as test releases (currently 1.7.35-0.4)
which can be installed with setup*.exe.

--
Yaakov



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Andrew DeFaria]

On 2/25/2015 2:20 PM, Roger Orr wrote:
> Good work -- at least in my environment ;-)
>
> 20150225 DLL:
> mkgroup 0.63s
> mkpasswd 0.289s
>
> compared to
>
> 20150220 DLL:
> mkgroup 45.8s
> mkpasswd: 4572.7s
>
> Output is
>   mkgroup: 53kb, 681 lines
>   mkpasswd: 132kb, 1081 lines.
>
> And the output *is* the same :-)

Can somebody summarize where we're at here. I've been noticing all this 
email about slow startup and I'm excited by the inclusion of domain 
accounts using /etc/nsswitch.conf, etc. I downloaded a newer version of 
Cygwin a few weeks ago (I'm seeing 1.7.34) and I experienced the 
slowdown myself. Unfortunately I'm too busy to actively participate but 
it looks like some good progress has been made. Should I update or is 
everything still in snapshots?

Thanks in advance.
-- 
Andrew DeFaria
http://defaria.com


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: slow startup after upgrade

[Roger Orr]

Good work -- at least in my environment ;-)

20150225 DLL:
mkgroup 0.63s
mkpasswd 0.289s

compared to

20150220 DLL:
mkgroup 45.8s
mkpasswd: 4572.7s

Output is
 mkgroup: 53kb, 681 lines
 mkpasswd: 132kb, 1081 lines.

And the output *is* the same :-)

Roger.

-----Original Message-----
From: Corinna Vinschen [mailto:corinna-cygwin@cygwin.com] 
Sent: 25 February 2015 12:52
To: cygwin@cygwin.com
Subject: Re: slow startup after upgrade

On Feb 25 12:29, Achim Gratz wrote:
> Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes:
> > > > While you're at it, does the new snapshot still stop after 3.5K
> > > > accounts even though you think there are 8K accounts?
> 
> $ mkpasswd -d | wc
> 
> I get ~30k AD accounts back in 75s instead of 315s and the network traffic
> is only half of the former value as well comparing the 20150224 vs. the
> latest 20150223 snapshot.

Cool!  Thanks for your feedback.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Andrey Repin]

Greetings, Corinna Vinschen!

>>  > Works very quickly for me.  Problem is, I don't know anything about
>>  > your environment.  Are you working remote or in an office?  How many
>>  > groups are in your user token?  What is the result of this funny
>>  > little statement Dennis created:
>>  >
>>  >   cmd /v:on /c "echo !TIME! & C:\cygwin64\bin\mintty.exe /bin/echo "test" & echo !TIME!"
>> 
>> On win7: that command takes .17, .44, .48, .53 sec between 1st & 2nd time stamp
>> 
>> That had 'db' only for passwd/group in nsswitch.conf, no cygwin procs
>> running including cygserver.

> Sounds good to me.

>> On problematic winxp box:
>>     [passwd, group, db_shell: 'files db', 'files db' & /bin/dash]
>>       16.83, 17.03, 24.80, 28.79, 27.99, 27.39 sec
>> 
>>     [passwd, group, db_shell: db, db & /bin/dash]
>>       23.51, 27.91, 28.78 sec

> Doesn't sound good at all.  Terrible in fact.  That a 32 or 64 bit XP?
> I didn't fire up my XP test machines for ages and I'm not overly enjoying
> it, but I guess I have to.  Oh well, hang on...

> [...time passes...]
> [...starting up...]
> [...XP...]
> [...32 bit...]
> [...updating...]
> [...testing...]

> Hmm, no.  In my case there's no visible delay starting mintty, and tcsh
> is my login shell, too.  mkpasswd and mkgroup are running comparably
> fast as on my W8.1 64 bit machine I'm using for testing in the first
> place.

> Something strange is going on on your XP machine, it seems.

Although I've migrated to Win7, I'm still using XP systems extensively, many
of them have Cygwin, and I didn't see anything even remotely close to his
results, ever.
His messages basically left me speechless, I was afraid to guess.
The half a minute startup of a shell, I could only attribute to environmental
issues.


--
WBR,
Andrey Repin (anrdaemon@yandex.ru) 26.02.2015, <00:40>

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3064 bytes --]

On Feb 25 11:15, John Hein wrote:
> Corinna Vinschen wrote at 09:56 +0100 on Feb 25, 2015:
>  > Really?  This is the first time I see your email address on this ML, and
>  > there's no name connected to it, so I don't even know if you're usually
>  > writing under another email address.  At least a forename would be nice...
> 
> Bah... sorry, you should get the name in this response.

Uh, it's you! :)

>  > > Running the 20150224 (and *23) snapshot produces the following on Win
>  > > XP (yes, I know) if cygserver is running:
>  >
>  > Sidenote: I'm contemplating to stop supporting XP end of this year.
> 
> I'm thinking it's either related to XP or _something_ with the
> particular XP box I'm using (when logged in on the domain, I can't
> disable many group policy things or temporarily inhibit AV, so it's
> harder to find/eliminate something to pin blame on - certainly may not
> be something the latest cygwin is doing when running on XP in a
> "large" AD domain).
> 
> I went to a win 7 box and the 2/24 snap makes mkpasswd -d run in 33
> seconds and enumerates all 8016 entries.  37 seconds with the 2/25
> snap.

The timing should be roughly the same, but fluctuations in the low
percentage range are normal.

> And win 7 doesn't get the 'wrong arg.type' error with the same 2/24
> cygwin1.dll for whatever reason.

Interesting.  I could reproduce this and it was very certainly a bug
in the code, fixed in -25 snapshot and 1.7.35-0.4.

>  > Works very quickly for me.  Problem is, I don't know anything about
>  > your environment.  Are you working remote or in an office?  How many
>  > groups are in your user token?  What is the result of this funny
>  > little statement Dennis created:
>  >
>  >   cmd /v:on /c "echo !TIME! & C:\cygwin64\bin\mintty.exe /bin/echo "test" & echo !TIME!"
> 
> On win7: that command takes .17, .44, .48, .53 sec between 1st & 2nd time stamp
> 
> That had 'db' only for passwd/group in nsswitch.conf, no cygwin procs
> running including cygserver.

Sounds good to me.

> On problematic winxp box:
>     [passwd, group, db_shell: 'files db', 'files db' & /bin/dash]
>       16.83, 17.03, 24.80, 28.79, 27.99, 27.39 sec
> 
>     [passwd, group, db_shell: db, db & /bin/dash]
>       23.51, 27.91, 28.78 sec

Doesn't sound good at all.  Terrible in fact.  That a 32 or 64 bit XP?
I didn't fire up my XP test machines for ages and I'm not overly enjoying
it, but I guess I have to.  Oh well, hang on...

[...time passes...]
[...starting up...]
[...XP...]
[...32 bit...]
[...updating...]
[...testing...]

Hmm, no.  In my case there's no visible delay starting mintty, and tcsh
is my login shell, too.  mkpasswd and mkgroup are running comparably
fast as on my W8.1 64 bit machine I'm using for testing in the first
place.

Something strange is going on on your XP machine, it seems.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: slow startup after upgrade

[John Hein]

John Hein wrote at 11:15 -0700 on Feb 25, 2015:
 > Corinna Vinschen wrote at 09:56 +0100 on Feb 25, 2015:
 >  > On Feb 24 16:03, John Hein wrote:
 >  > > Corinna Vinschen wrote at 22:13 +0100 on Feb 24, 2015:
 >  > >  > Hi Roger,
 >  > >  >
 >  > >  > On Feb 24 19:55, Roger Orr wrote:
 >  > >  > > Hello Corinna,
 >  > >  > > It seems slightly faster than the previous patch and I've not noticed a
 >  > >  > > downside yet.
 >  > >  > >
 >  > >  > >
 >  > >  > > CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35s(0.286/5/3) 20150220 15:47:55 i686
 >  > >  > > Cygwin:
 >  > >  > >
 >  > >  > > ~37ms to run echo.exe from Windows command prompt
 >  > >  > > ~60ms to run .\id.exe -a from Windows command prompt
 >  > >  > >
 >  > >  > > CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35s(0.286/5/3) 20150223 21:02:38 i686
 >  > >  > > Cygwin:
 >  > >  > >
 >  > >  > > ~35ms to run echo.exe from Windows command prompt
 >  > >  > > ~53ms to run .\id.exe -a from Windows command prompt
 >  > >  >
 >  > >  > That's a nice result.
 >  > >  >
 >  > >  > However, I don't quite understand this result for the older DLL.
 >  > >  > Weren't you reporting >4 secs as startup time from 1.7.35?!?
 >  > >  >
 >  > >  > On another note:
 >  > >  >
 >  > >  > I just uploaded a new developer snapshot (2015-02-24).  This snapshot
 >  > >  > should improve mkpasswd/mkgroup or, generally speaking, enumerating AD
 >  > >  > accounts, a lot.  Can you give it a try?
 >  > >  >
 >  > >  > While you're at it, does the new snapshot still stop after 3.5K accounts
 >  > >  > even though you think there are 8K accounts?  If so, I'd be interested
 >  > >  > to investigate this further.  The reason is, while testing my today's
 >  > >  > performance improvements, I stumbled over a bug in my code which also
 >  > >  > resulted in enumerating less accounts as desired.  So I'm not entirely
 >  > >  > sure your problem isn't related to a bug either.
 >  > >
 >  > > That mkpasswd symptom was reported by me
 >  >
 >  > Really?  This is the first time I see your email address on this ML, and
 >  > there's no name connected to it, so I don't even know if you're usually
 >  > writing under another email address.  At least a forename would be nice...
 >
 >
 > Bah... sorry, you should get the name in this response.
 >
 >
 >  > > (well, maybe others, too).
 >  > > I'm running it again, but it's still very slow, AFAICS.
 >  >
 >  > It shouldn't.  It performs only one LDAP call now per 100 accounts,
 >  > If that's still slow in your environment, I don't see any way to
 >  > speed this up any further.
 >  >
 >  > > I'll report
 >  > > how long it took when / if it finishes.
 >  > >
 >  > > New issue with recent snaps:
 >  > >
 >  > > Running the 20150224 (and *23) snapshot produces the following on Win
 >  > > XP (yes, I know) if cygserver is running:
 >  >
 >  > Sidenote: I'm contemplating to stop supporting XP end of this year.
 >
 > I'm thinking it's either related to XP or _something_ with the
 > particular XP box I'm using (when logged in on the domain, I can't
 > disable many group policy things or temporarily inhibit AV, so it's
 > harder to find/eliminate something to pin blame on - certainly may not
 > be something the latest cygwin is doing when running on XP in a
 > "large" AD domain).
 >
 > I went to a win 7 box and the 2/24 snap makes mkpasswd -d run in 33
 > seconds and enumerates all 8016 entries.  37 seconds with the 2/25
 > snap.
 >
 > And win 7 doesn't get the 'wrong arg.type' error with the same 2/24
 > cygwin1.dll for whatever reason.
 >
 >
 >  > > $ /usr/sbin/syslog-ng -F
 >  > >       1 [main] syslog-ng 5776 C:\cygwin\usr\sbin\syslog-ng.exe: *** fatal error - Fetching account info from cygserver with wrong arg.type 2
 >  >
 >  > Try the snapshot I just uploaded (2015-02-25).  I forgot a tiny, but
 >  > crucial statement in the code when I changed that code two days ago.
 >  > That should be fixed now.
 >  >
 >  > > tcsh still taking a long time here (also XP - 3+ minutes with
 >  > > cygserver running).  bash & dash much quicker.
 >  >
 >  > Works very quickly for me.  Problem is, I don't know anything about
 >  > your environment.  Are you working remote or in an office?  How many
 >  > groups are in your user token?  What is the result of this funny
 >  > little statement Dennis created:
 >  >
 >  >   cmd /v:on /c "echo !TIME! & C:\cygwin64\bin\mintty.exe /bin/echo "test" & echo !TIME!"
 >
 > On win7: that command takes .17, .44, .48, .53 sec between 1st & 2nd time stamp
 >
 > That had 'db' only for passwd/group in nsswitch.conf, no cygwin procs
 > running including cygserver.
 >
 >
 > On problematic winxp box:
 >     [passwd, group, db_shell: 'files db', 'files db' & /bin/dash]
 >       16.83, 17.03, 24.80, 28.79, 27.99, 27.39 sec
 >
 >     [passwd, group, db_shell: db, db & /bin/dash]
 >       23.51, 27.91, 28.78 sec
 >
 >     Any trials using /bin/tcsh without disabling complete.tcsh take "a
 >     really long time" (TM)... from minutes with 2/25 snap to 10s of
 >     minutes with 1.7.34.  I mentioned this in another thread (and
 >     strace sometimes showed 5 minute gaps around forks).


One further test with the 'cmd ... mintty.exe' command.
I started cygserver on the 'problematic winxp' box.

Then immediately ran the mintty.exe test after starting
the cygserver service - it took .08 sec - then .15 and .14.

Would be interesting if there are other people who could report on
experiences using 1.7.34 and later on XP boxes in an AD environment.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[John Hein]

Corinna Vinschen wrote at 09:56 +0100 on Feb 25, 2015:
 > On Feb 24 16:03, q2t2hzmwuk@snkmail.com wrote:
 > > Corinna Vinschen corinna-cygwin-at-cygwin.com |cygwin_ml_nodigest| wrote at 22:13 +0100 on Feb 24, 2015:
 > >  > Hi Roger,
 > >  >
 > >  > On Feb 24 19:55, Roger Orr wrote:
 > >  > > Hello Corinna,
 > >  > > It seems slightly faster than the previous patch and I've not noticed a
 > >  > > downside yet.
 > >  > >
 > >  > >
 > >  > > CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35s(0.286/5/3) 20150220 15:47:55 i686
 > >  > > Cygwin:
 > >  > >
 > >  > > ~37ms to run echo.exe from Windows command prompt
 > >  > > ~60ms to run .\id.exe -a from Windows command prompt
 > >  > >
 > >  > > CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35s(0.286/5/3) 20150223 21:02:38 i686
 > >  > > Cygwin:
 > >  > >
 > >  > > ~35ms to run echo.exe from Windows command prompt
 > >  > > ~53ms to run .\id.exe -a from Windows command prompt
 > >  >
 > >  > That's a nice result.
 > >  >
 > >  > However, I don't quite understand this result for the older DLL.
 > >  > Weren't you reporting >4 secs as startup time from 1.7.35?!?
 > >  >
 > >  > On another note:
 > >  >
 > >  > I just uploaded a new developer snapshot (2015-02-24).  This snapshot
 > >  > should improve mkpasswd/mkgroup or, generally speaking, enumerating AD
 > >  > accounts, a lot.  Can you give it a try?
 > >  >
 > >  > While you're at it, does the new snapshot still stop after 3.5K accounts
 > >  > even though you think there are 8K accounts?  If so, I'd be interested
 > >  > to investigate this further.  The reason is, while testing my today's
 > >  > performance improvements, I stumbled over a bug in my code which also
 > >  > resulted in enumerating less accounts as desired.  So I'm not entirely
 > >  > sure your problem isn't related to a bug either.
 > >
 > > That mkpasswd symptom was reported by me
 >
 > Really?  This is the first time I see your email address on this ML, and
 > there's no name connected to it, so I don't even know if you're usually
 > writing under another email address.  At least a forename would be nice...


Bah... sorry, you should get the name in this response.


 > > (well, maybe others, too).
 > > I'm running it again, but it's still very slow, AFAICS.
 >
 > It shouldn't.  It performs only one LDAP call now per 100 accounts,
 > If that's still slow in your environment, I don't see any way to
 > speed this up any further.
 >
 > > I'll report
 > > how long it took when / if it finishes.
 > >
 > > New issue with recent snaps:
 > >
 > > Running the 20150224 (and *23) snapshot produces the following on Win
 > > XP (yes, I know) if cygserver is running:
 >
 > Sidenote: I'm contemplating to stop supporting XP end of this year.

I'm thinking it's either related to XP or _something_ with the
particular XP box I'm using (when logged in on the domain, I can't
disable many group policy things or temporarily inhibit AV, so it's
harder to find/eliminate something to pin blame on - certainly may not
be something the latest cygwin is doing when running on XP in a
"large" AD domain).

I went to a win 7 box and the 2/24 snap makes mkpasswd -d run in 33
seconds and enumerates all 8016 entries.  37 seconds with the 2/25
snap.

And win 7 doesn't get the 'wrong arg.type' error with the same 2/24
cygwin1.dll for whatever reason.


 > > $ /usr/sbin/syslog-ng -F
 > >       1 [main] syslog-ng 5776 C:\cygwin\usr\sbin\syslog-ng.exe: *** fatal error - Fetching account info from cygserver with wrong arg.type 2
 >
 > Try the snapshot I just uploaded (2015-02-25).  I forgot a tiny, but
 > crucial statement in the code when I changed that code two days ago.
 > That should be fixed now.
 >
 > > tcsh still taking a long time here (also XP - 3+ minutes with
 > > cygserver running).  bash & dash much quicker.
 >
 > Works very quickly for me.  Problem is, I don't know anything about
 > your environment.  Are you working remote or in an office?  How many
 > groups are in your user token?  What is the result of this funny
 > little statement Dennis created:
 >
 >   cmd /v:on /c "echo !TIME! & C:\cygwin64\bin\mintty.exe /bin/echo "test" & echo !TIME!"

On win7: that command takes .17, .44, .48, .53 sec between 1st & 2nd time stamp

That had 'db' only for passwd/group in nsswitch.conf, no cygwin procs
running including cygserver.


On problematic winxp box:
    [passwd, group, db_shell: 'files db', 'files db' & /bin/dash]
      16.83, 17.03, 24.80, 28.79, 27.99, 27.39 sec

    [passwd, group, db_shell: db, db & /bin/dash]
      23.51, 27.91, 28.78 sec

    Any trials using /bin/tcsh without disabling complete.tcsh take "a
    really long time" (TM)... from minutes with 2/25 snap to 10s of
    minutes with 1.7.34.  I mentioned this in another thread (and
    strace sometimes showed 5 minute gaps around forks).

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 645 bytes --]

On Feb 25 12:29, Achim Gratz wrote:
> Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes:
> > > > While you're at it, does the new snapshot still stop after 3.5K
> > > > accounts even though you think there are 8K accounts?
> 
> $ mkpasswd -d | wc
> 
> I get ~30k AD accounts back in 75s instead of 315s and the network traffic
> is only half of the former value as well comparing the 20150224 vs. the
> latest 20150223 snapshot.

Cool!  Thanks for your feedback.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: slow startup after upgrade

[Achim Gratz]

Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes:
> > > While you're at it, does the new snapshot still stop after 3.5K
> > > accounts even though you think there are 8K accounts?

$ mkpasswd -d | wc

I get ~30k AD accounts back in 75s instead of 315s and the network traffic
is only half of the former value as well comparing the 20150224 vs. the
latest 20150223 snapshot.


Regards,
Achim.




--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 414 bytes --]

On Feb 24 23:57, Roger Orr wrote:
> Corinna Vinschen wrote:
> > While you're at it, does the new snapshot still stop after 3.5K
> > accounts even though you think there are 8K accounts?

At this point I think I mix you up with John Hein, sorry.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3446 bytes --]

On Feb 24 16:03, q2t2hzmwuk@snkmail.com wrote:
> Corinna Vinschen corinna-cygwin-at-cygwin.com |cygwin_ml_nodigest| wrote at 22:13 +0100 on Feb 24, 2015:
>  > Hi Roger,
>  >
>  > On Feb 24 19:55, Roger Orr wrote:
>  > > Hello Corinna,
>  > > It seems slightly faster than the previous patch and I've not noticed a
>  > > downside yet.
>  > >
>  > >
>  > > CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35s(0.286/5/3) 20150220 15:47:55 i686
>  > > Cygwin:
>  > >
>  > > ~37ms to run echo.exe from Windows command prompt
>  > > ~60ms to run .\id.exe -a from Windows command prompt
>  > >
>  > > CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35s(0.286/5/3) 20150223 21:02:38 i686
>  > > Cygwin:
>  > >
>  > > ~35ms to run echo.exe from Windows command prompt
>  > > ~53ms to run .\id.exe -a from Windows command prompt
>  >
>  > That's a nice result.
>  >
>  > However, I don't quite understand this result for the older DLL.
>  > Weren't you reporting >4 secs as startup time from 1.7.35?!?
>  >
>  > On another note:
>  >
>  > I just uploaded a new developer snapshot (2015-02-24).  This snapshot
>  > should improve mkpasswd/mkgroup or, generally speaking, enumerating AD
>  > accounts, a lot.  Can you give it a try?
>  >
>  > While you're at it, does the new snapshot still stop after 3.5K accounts
>  > even though you think there are 8K accounts?  If so, I'd be interested
>  > to investigate this further.  The reason is, while testing my today's
>  > performance improvements, I stumbled over a bug in my code which also
>  > resulted in enumerating less accounts as desired.  So I'm not entirely
>  > sure your problem isn't related to a bug either.
> 
> That mkpasswd symptom was reported by me

Really?  This is the first time I see your email address on this ML, and
there's no name connected to it, so I don't even know if you're usually
writing under another email address.  At least a forename would be nice...

> (well, maybe others, too).
> I'm running it again, but it's still very slow, AFAICS.

It shouldn't.  It performs only one LDAP call now per 100 accounts,
If that's still slow in your environment, I don't see any way to
speed this up any further.

> I'll report
> how long it took when / if it finishes.
> 
> New issue with recent snaps:
> 
> Running the 20150224 (and *23) snapshot produces the following on Win
> XP (yes, I know) if cygserver is running:

Sidenote: I'm contemplating to stop supporting XP end of this year.

> $ /usr/sbin/syslog-ng -F
>       1 [main] syslog-ng 5776 C:\cygwin\usr\sbin\syslog-ng.exe: *** fatal error - Fetching account info from cygserver with wrong arg.type 2

Try the snapshot I just uploaded (2015-02-25).  I forgot a tiny, but
crucial statement in the code when I changed that code two days ago.
That should be fixed now.

> tcsh still taking a long time here (also XP - 3+ minutes with
> cygserver running).  bash & dash much quicker.

Works very quickly for me.  Problem is, I don't know anything about
your environment.  Are you working remote or in an office?  How many
groups are in your user token?  What is the result of this funny
little statement Dennis created:

  cmd /v:on /c "echo !TIME! & C:\cygwin64\bin\mintty.exe /bin/echo "test" & echo !TIME!"


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: slow startup after upgrade

[Roger Orr]

Corinna Vinschen wrote:
> Hi Roger,
> 
> On Feb 24 19:55, Roger Orr wrote:
>> Hello Corinna,
>> It seems slightly faster than the previous patch and I've not
>> noticed a downside yet. 
>> 
>> 
>> CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35s(0.286/5/3) 20150220 15:47:55
>> i686 Cygwin:
>> 
>> ~37ms to run echo.exe from Windows command prompt
>> ~60ms to run .\id.exe -a from Windows command prompt
>> 
>> CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35s(0.286/5/3) 20150223 21:02:38
>> i686 Cygwin:
>> 
>> ~35ms to run echo.exe from Windows command prompt
>> ~53ms to run .\id.exe -a from Windows command prompt
> 
> That's a nice result.
> 
> However, I don't quite understand this result for the older DLL.
> Weren't you reporting >4 secs as startup time from 1.7.35?!? 

The slow (4s) startup was from an early 1.7.35 DLL before that of 20150220.

From an earlier message (2015-02-17):

1) Running cygwin "echo.exe" from a cmd.exe command shell

a) With passwd: files and group: files in /etc/nsswitch.conf
  0.03 - 0.4 s

b) With 1.7.34 and default /etc/nsswitch.conf

  around 120s

C) With 1.7.35 and default /etc/nsswitch.conf

  4.4 - 4.6s

> On another note:
> 
> I just uploaded a new developer snapshot (2015-02-24).  This snapshot
> should improve mkpasswd/mkgroup or, generally speaking, enumerating
> AD accounts, a lot.  Can you give it a try?  
> 
> While you're at it, does the new snapshot still stop after 3.5K
> accounts even though you think there are 8K accounts?  If so, I'd be
> interested to investigate this further.  The reason is, while testing
> my today's performance improvements, I stumbled over a bug in my code
> which also resulted in enumerating less accounts as desired.  So I'm
> not entirely sure your problem isn't related to a bug either.     


I'll try to find time to give this a go too.

> 
> 
> Thanks,
> Corinna



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[q2t2hzmwuk]

Corinna Vinschen corinna-cygwin-at-cygwin.com |cygwin_ml_nodigest| wrote at 22:13 +0100 on Feb 24, 2015:
 > Hi Roger,
 >
 > On Feb 24 19:55, Roger Orr wrote:
 > > Hello Corinna,
 > > It seems slightly faster than the previous patch and I've not noticed a
 > > downside yet.
 > >
 > >
 > > CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35s(0.286/5/3) 20150220 15:47:55 i686
 > > Cygwin:
 > >
 > > ~37ms to run echo.exe from Windows command prompt
 > > ~60ms to run .\id.exe -a from Windows command prompt
 > >
 > > CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35s(0.286/5/3) 20150223 21:02:38 i686
 > > Cygwin:
 > >
 > > ~35ms to run echo.exe from Windows command prompt
 > > ~53ms to run .\id.exe -a from Windows command prompt
 >
 > That's a nice result.
 >
 > However, I don't quite understand this result for the older DLL.
 > Weren't you reporting >4 secs as startup time from 1.7.35?!?
 >
 > On another note:
 >
 > I just uploaded a new developer snapshot (2015-02-24).  This snapshot
 > should improve mkpasswd/mkgroup or, generally speaking, enumerating AD
 > accounts, a lot.  Can you give it a try?
 >
 > While you're at it, does the new snapshot still stop after 3.5K accounts
 > even though you think there are 8K accounts?  If so, I'd be interested
 > to investigate this further.  The reason is, while testing my today's
 > performance improvements, I stumbled over a bug in my code which also
 > resulted in enumerating less accounts as desired.  So I'm not entirely
 > sure your problem isn't related to a bug either.

That mkpasswd symptom was reported by me (well, maybe others, too).
I'm running it again, but it's still very slow, AFAICS.  I'll report
how long it took when / if it finishes.

New issue with recent snaps:

Running the 20150224 (and *23) snapshot produces the following on Win
XP (yes, I know) if cygserver is running:

$ /usr/sbin/syslog-ng -F
      1 [main] syslog-ng 5776 C:\cygwin\usr\sbin\syslog-ng.exe: *** fatal error - Fetching account info from cygserver with wrong arg.type 2

.... and even if cygserver is not running:

$ getent group `id -G`
Domain Users:S-1-5-21-1643024071-179607362-792003330-513:10513:
      1 [main] getent 4120 C:\cygwin\bin\getent.exe: *** fatal error - Fetching account info from cygserver with wrong arg.type 2
Hangup

The 1.7.35-0.3 cygwin dll didn't trigger those symptoms.

tcsh still taking a long time here (also XP - 3+ minutes with
cygserver running).  bash & dash much quicker.  Maybe something with
the tcsh hash table (tcsh -f is fast)?  Doing 'env PATH=/bin tcsh'
helps some.  strace shows lots of mount_info::conv_to_posix_path
calls when PATH is not trimmed.

Other times, I've seen stalls with 'db' in nsswitch.conf and no
cygserver like so every time a new child tcsh.exe is forked:

00:00:00 [main] tcsh 4388 child_info::sync: n 2, waiting for subproc_ready(0x6BC) and child process(0x680)
00:05:00 [main] tcsh 4388 child_info::sync: wait failed, pid 2556, Win32 error 0

Why it's forking a number of child tcsh's is not clear even after
looking through the .csh files in /etc/profile.d Ahh... it must be
complete.tcsh.  Indeed - disabling that (by rename) helps a lot - lots
of back-tick commands in there.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: slow startup after upgrade

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1515 bytes --]

Hi Roger,

On Feb 24 19:55, Roger Orr wrote:
> Hello Corinna,
> It seems slightly faster than the previous patch and I've not noticed a
> downside yet.
> 
> 
> CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35s(0.286/5/3) 20150220 15:47:55 i686
> Cygwin:
> 
> ~37ms to run echo.exe from Windows command prompt
> ~60ms to run .\id.exe -a from Windows command prompt
> 
> CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35s(0.286/5/3) 20150223 21:02:38 i686
> Cygwin: 
> 
> ~35ms to run echo.exe from Windows command prompt
> ~53ms to run .\id.exe -a from Windows command prompt

That's a nice result.

However, I don't quite understand this result for the older DLL.
Weren't you reporting >4 secs as startup time from 1.7.35?!?

On another note:

I just uploaded a new developer snapshot (2015-02-24).  This snapshot
should improve mkpasswd/mkgroup or, generally speaking, enumerating AD
accounts, a lot.  Can you give it a try?

While you're at it, does the new snapshot still stop after 3.5K accounts
even though you think there are 8K accounts?  If so, I'd be interested
to investigate this further.  The reason is, while testing my today's
performance improvements, I stumbled over a bug in my code which also
resulted in enumerating less accounts as desired.  So I'm not entirely
sure your problem isn't related to a bug either.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: slow startup after upgrade

[Roger Orr]

Hello Corinna,
It seems slightly faster than the previous patch and I've not noticed a
downside yet.


CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35s(0.286/5/3) 20150220 15:47:55 i686
Cygwin:

~37ms to run echo.exe from Windows command prompt
~60ms to run .\id.exe -a from Windows command prompt

CYGWIN_NT-6.1-WOW LCLDN-DEV24 1.7.35s(0.286/5/3) 20150223 21:02:38 i686
Cygwin: 

~35ms to run echo.exe from Windows command prompt
~53ms to run .\id.exe -a from Windows command prompt

nsswitch.conf: passwd and group both set to 'db'

Regards,
Roger.

-----Original Message-----
From: Roger Orr [mailto:rogero@howzatt.demon.co.uk] 
Sent: 23 February 2015 23:33
To: 'cygwin@cygwin.com'
Subject: RE: slow startup after upgrade

Hi Corinna,
sounds good. I'll give this a go, all being well, tomorrow.

In other news it appears that the ADInsightDll uses some shared data to
communicate, so in order to get ADInsight to work with injection one has to
ensure that the DLL injected is the same actual *file* as the one loaded by
the ADInsight program (in the (windows) %TEMP% directory) - a copy of the
DLL doesn't seem to be effective.

Regards,
Roger.

-----Original Message-----
From: Corinna Vinschen [mailto:corinna-cygwin@cygwin.com] 
Sent: 23 February 2015 21:16
To: cygwin@cygwin.com
Cc: Roger Orr
Subject: Re: slow startup after upgrade

Hi Roger,

On Feb 16 20:02, Roger Orr wrote:
> Corinna Vinschen wrote:
> > So I'd think the best way forward is to update to the
> > 1.7.35-0.1 test release and report further from there.     
> 
> Thanks, this does help a little. However I will still be using the 'files'
> setting.
> 
> Here are some results in case they're of interest. (Windows 7/64 with
> cygwin/64.)
> 
> 1) Running cygwin "echo.exe" from a cmd.exe command shell
> 
> a) With passwd: files and group: files in /etc/nsswitch.conf
>   0.03 - 0.4 s
> 
> b) With 1.7.34 and default /etc/nsswitch.conf
> 
>   around 120s
> 
> C) With 1.7.35 and default /etc/nsswitch.conf
> 
>   4.4 - 4.6s

I rewrote the function responsible for the slow startup, the one
fetching all the group information for the groups in your user token.

As I just wrote in my mail to Dennis Hagarty, the performance improvement
should be noticable, even with /etc/nsswitch.conf only using the "db"
setting for groups.  The group info for a user token with 150 groups was
fetched in ~50 ms rather than the ~300ms of the former code.

Can you test this again with the Cygwin DLL from the latest developer
snaphshot at https://cygwin.com/snapshots/ please?


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple