Making Cygwin More Tolerant of Orphaned SIDs?

Making Cygwin More Tolerant of Orphaned SIDs?

[Bryan Berns]

Based on some rudimentary performance tests, it would appear that
Cygwin may repeatedly try to lookup information on a SID form an ACE
if cannot find a corresponding account.... which will undoubtedly
occur for orphaned SIDs.  If the volume being read is remote, this can
result in some massive slowdowns if there are many files in the
directory.  Is there any way to augment this behavior?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Achim Gratz]

Bryan Berns <bryan.berns <at> gmail.com> writes:
> Based on some rudimentary performance tests, it would appear that
> Cygwin may repeatedly try to lookup information on a SID form an ACE
> if cannot find a corresponding account.... which will undoubtedly
> occur for orphaned SIDs.  If the volume being read is remote, this can
> result in some massive slowdowns if there are many files in the
> directory.  Is there any way to augment this behavior?

Mounting the volume "noacl" doesn't help?


Regards,
Achim.





--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1032 bytes --]

On Apr 14 06:57, Achim Gratz wrote:
> Bryan Berns <bryan.berns <at> gmail.com> writes:
> > Based on some rudimentary performance tests, it would appear that
> > Cygwin may repeatedly try to lookup information on a SID form an ACE
> > if cannot find a corresponding account.... which will undoubtedly
> > occur for orphaned SIDs.  If the volume being read is remote, this can
> > result in some massive slowdowns if there are many files in the
> > directory.  Is there any way to augment this behavior?
> 
> Mounting the volume "noacl" doesn't help?

Orphaned SIDs shouldn't happen.  Disabling accounts, ok, but removing
them?  I don't know.  So the question is, if there's no account with
these SIDs anymore, why aren't these SIDs removed from the ACLs?
It's not only Cygwin.  These SIDs also unnecessarily slow down each
single access check of the OS.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Bryan Berns]

On Tue, Apr 14, 2015 at 4:00 AM, Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
>
> Orphaned SIDs shouldn't happen.  Disabling accounts, ok, but removing
> them?  I don't know.  So the question is, if there's no account with
> these SIDs anymore, why aren't these SIDs removed from the ACLs?
> It's not only Cygwin.  These SIDs also unnecessarily slow down each
> single access check of the OS.
>

In principal, I agree 100%.  Unfortunately, in some large enterprise
environments removal of orphaned SIDs rarely happens on a regular
basis.   The best way to manage this is typically to only delegate
access via groups and have those groups aligned to the file system
structure in some way (which tends to change less in practice than
company organizational structure).  Still, when you've got dozens of
people starting/leaving every week, per account permission are
occasionally established enumerating more a petabyte of data across
several sites to cleanup ACEs is certainly possible but not on the top
list of things to do (and mass alteration of ACLs carries some
liability to it).  Don't get me wrong, my anal retentive nature makes
me cringe when I see an orphaned SID; it's just the reality of the
situation.

That said, the origin of my question was actually not due to
unresolvable SIDs to due to removed accounts --- it was just the
easiest one to describe. The reason I noticed this is because we have
some NTFS assignments via local groups on a remote computers (and
those local groups then have nested Active Directory groups).  So the
ACE has REMOTECOMPUTER\Group vice DOMAIN\Group.  When Cygwin attempts
to retrieve information on these accounts, it seems to fail and causes
delays.  So with the newer versions of Cygwin, doing an 'ls -l' went
from 2 seconds to more than 30 seconds on some particular file
directories.

As Achim alluded, 'noacl' may be be the way to go for us, but I was
just asking the question in the even there was a configurable setting
or a feature enhancement that could be integrated to deal with these
scenarios.  Of course, 'noacl' seems to mark group / other masks as
readable so apps that do permissions checks on these files will return
inaccurate results :-(.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3265 bytes --]

On Apr 14 05:08, Bryan Berns wrote:
> On Tue, Apr 14, 2015 at 4:00 AM, Corinna Vinschen
> <corinna-cygwin@cygwin.com> wrote:
> >
> > Orphaned SIDs shouldn't happen.  Disabling accounts, ok, but removing
> > them?  I don't know.  So the question is, if there's no account with
> > these SIDs anymore, why aren't these SIDs removed from the ACLs?
> > It's not only Cygwin.  These SIDs also unnecessarily slow down each
> > single access check of the OS.
> >
> 
> In principal, I agree 100%.  Unfortunately, in some large enterprise
> environments removal of orphaned SIDs rarely happens on a regular
> basis.   The best way to manage this is typically to only delegate
> access via groups and have those groups aligned to the file system
> structure in some way (which tends to change less in practice than
> company organizational structure).  Still, when you've got dozens of
> people starting/leaving every week, per account permission are
> occasionally established enumerating more a petabyte of data across
> several sites to cleanup ACEs is certainly possible but not on the top
> list of things to do (and mass alteration of ACLs carries some
> liability to it).  Don't get me wrong, my anal retentive nature makes
> me cringe when I see an orphaned SID; it's just the reality of the
> situation.
> 
> That said, the origin of my question was actually not due to
> unresolvable SIDs to due to removed accounts --- it was just the
> easiest one to describe. The reason I noticed this is because we have
> some NTFS assignments via local groups on a remote computers (and
> those local groups then have nested Active Directory groups).  So the
> ACE has REMOTECOMPUTER\Group vice DOMAIN\Group.  When Cygwin attempts
> to retrieve information on these accounts, it seems to fail and causes
> delays.  So with the newer versions of Cygwin, doing an 'ls -l' went
> from 2 seconds to more than 30 seconds on some particular file
> directories.
> 
> As Achim alluded, 'noacl' may be be the way to go for us, but I was
> just asking the question in the even there was a configurable setting
> or a feature enhancement that could be integrated to deal with these
> scenarios.  Of course, 'noacl' seems to mark group / other masks as
> readable so apps that do permissions checks on these files will return
> inaccurate results :-(.

The problem is that Cygwin, or any other tool trying to resolve SIDs
doesn't know a SID won't resolve before it tried.  And then it's an
OS function which takes its time.  It's like checking for network
machines providing shares.  Sometimes this test takes ages, but in
this case, fortunately, you see that it takes ages in Explorer as
well.

As for ACLs, you can alleviate the problem somewhat by running cygserver
on the machine, which allows to cache SIDs for all processes.  So only
the first process trying the SID will take time, followup processes will
get the cached results from cygserver.

Other than that, except for ignoring ACLs entirely (noacl) I have
no idea how to solve this problem differently.  


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Bryan Berns]

On Tue, Apr 14, 2015 at 4:00 AM, Corinna Vinschen
>
> The problem is that Cygwin, or any other tool trying to resolve SIDs
> doesn't know a SID won't resolve before it tried.  And then it's an
> OS function which takes its time.  It's like checking for network
> machines providing shares.  Sometimes this test takes ages, but in
> this case, fortunately, you see that it takes ages in Explorer as
> well.
>
> As for ACLs, you can alleviate the problem somewhat by running cygserver
> on the machine, which allows to cache SIDs for all processes.  So only
> the first process trying the SID will take time, followup processes will
> get the cached results from cygserver.
>
> Other than that, except for ignoring ACLs entirely (noacl) I have
> no idea how to solve this problem differently.

Yes, I understand there's nothing Cygwin can do beforehand -- that
means sense.  I guess what I'm saying is that Cygwin doesn't appear to
be caching SIDs in certain scenarios.

For example, I create a whole bunch of files (like 5000),  I use
icacls to append a new ACE.  Then I do a 'time ls -l
/cygdrive/c/somedir/*'.  Takes four seconds.  In the same Cygwin
session, I remove the local group (net localgroup testgroup /delete).
 I do the same 'time ls -l /cygdrive/c/somedir/*'.  Takes 20 seconds.
Subsequent runs in the also take 20 seconds.  Since I'm able to
continue to see the slowdown in the same session, cygserver wouldn't
help right?

Is the above expected?  If not and it's something you're willing to
take a look at, I can certainly come up with a little script to
demonstrate the issue.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Noel Grandin]

On 2015-04-14 01:24 PM, Bryan Berns wrote:
>   I do the same 'time ls -l /cygdrive/c/somedir/*'.  Takes 20 seconds.
> Subsequent runs in the also take 20 seconds.  Since I'm able to

It sounds like Cygwin needs a negative-entry cache i.e. a cache of ACL's that are know to be orphans or otherwise bad to 
prevent it repeatedly trying them.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 772 bytes --]

On Apr 14 14:30, Noel Grandin wrote:
> 
> 
> On 2015-04-14 01:24 PM, Bryan Berns wrote:
> >  I do the same 'time ls -l /cygdrive/c/somedir/*'.  Takes 20 seconds.
> >Subsequent runs in the also take 20 seconds.  Since I'm able to
> 
> It sounds like Cygwin needs a negative-entry cache i.e. a cache of ACL's
> that are know to be orphans or otherwise bad to prevent it repeatedly trying
> them.

It doesn't if it tried it once since after that it's cached within the
limits of the caching algorithm, as explained in the documentation
nobody reads:

  https://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-files

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 945 bytes --]

On Apr 14 16:54, Corinna Vinschen wrote:
> On Apr 14 14:30, Noel Grandin wrote:
> > 
> > 
> > On 2015-04-14 01:24 PM, Bryan Berns wrote:
> > >  I do the same 'time ls -l /cygdrive/c/somedir/*'.  Takes 20 seconds.
> > >Subsequent runs in the also take 20 seconds.  Since I'm able to
> > 
> > It sounds like Cygwin needs a negative-entry cache i.e. a cache of ACL's
> > that are know to be orphans or otherwise bad to prevent it repeatedly trying
> > them.
> 
> It doesn't if it tried it once since after that it's cached within the
> limits of the caching algorithm, as explained in the documentation
> nobody reads:
> 
>   https://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-files

Actually:

    https://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-mapping-caching


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2003 bytes --]

On Apr 14 07:24, Bryan Berns wrote:
> On Tue, Apr 14, 2015 at 4:00 AM, Corinna Vinschen
> >
> > The problem is that Cygwin, or any other tool trying to resolve SIDs
> > doesn't know a SID won't resolve before it tried.  And then it's an
> > OS function which takes its time.  It's like checking for network
> > machines providing shares.  Sometimes this test takes ages, but in
> > this case, fortunately, you see that it takes ages in Explorer as
> > well.
> >
> > As for ACLs, you can alleviate the problem somewhat by running cygserver
> > on the machine, which allows to cache SIDs for all processes.  So only
> > the first process trying the SID will take time, followup processes will
> > get the cached results from cygserver.
> >
> > Other than that, except for ignoring ACLs entirely (noacl) I have
> > no idea how to solve this problem differently.
> 
> Yes, I understand there's nothing Cygwin can do beforehand -- that
> means sense.  I guess what I'm saying is that Cygwin doesn't appear to
> be caching SIDs in certain scenarios.
> 
> For example, I create a whole bunch of files (like 5000),  I use
> icacls to append a new ACE.  Then I do a 'time ls -l
> /cygdrive/c/somedir/*'.  Takes four seconds.  In the same Cygwin
> session, I remove the local group (net localgroup testgroup /delete).
>  I do the same 'time ls -l /cygdrive/c/somedir/*'.  Takes 20 seconds.
> Subsequent runs in the also take 20 seconds.  Since I'm able to
> continue to see the slowdown in the same session, cygserver wouldn't
> help right?
> 
> Is the above expected?

Yes.  Without cygserver, caching only works from parent to child process.
One run of ls can't cache data for a parallel run of ls in trhe same
session.  As, btw., explained in the documentation:

  https://cygwin.com/cygwin-ug-net/ntsec.html


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2378 bytes --]

On Apr 14 16:53, Corinna Vinschen wrote:
> On Apr 14 07:24, Bryan Berns wrote:
> > On Tue, Apr 14, 2015 at 4:00 AM, Corinna Vinschen
> > >
> > > The problem is that Cygwin, or any other tool trying to resolve SIDs
> > > doesn't know a SID won't resolve before it tried.  And then it's an
> > > OS function which takes its time.  It's like checking for network
> > > machines providing shares.  Sometimes this test takes ages, but in
> > > this case, fortunately, you see that it takes ages in Explorer as
> > > well.
> > >
> > > As for ACLs, you can alleviate the problem somewhat by running cygserver
> > > on the machine, which allows to cache SIDs for all processes.  So only
> > > the first process trying the SID will take time, followup processes will
> > > get the cached results from cygserver.
> > >
> > > Other than that, except for ignoring ACLs entirely (noacl) I have
> > > no idea how to solve this problem differently.
> > 
> > Yes, I understand there's nothing Cygwin can do beforehand -- that
> > means sense.  I guess what I'm saying is that Cygwin doesn't appear to
> > be caching SIDs in certain scenarios.
> > 
> > For example, I create a whole bunch of files (like 5000),  I use
> > icacls to append a new ACE.  Then I do a 'time ls -l
> > /cygdrive/c/somedir/*'.  Takes four seconds.  In the same Cygwin
> > session, I remove the local group (net localgroup testgroup /delete).
> >  I do the same 'time ls -l /cygdrive/c/somedir/*'.  Takes 20 seconds.
> > Subsequent runs in the also take 20 seconds.  Since I'm able to
> > continue to see the slowdown in the same session, cygserver wouldn't
> > help right?
> > 
> > Is the above expected?
> 
> Yes.  Without cygserver, caching only works from parent to child process.
> One run of ls can't cache data for a parallel run of ls in trhe same
> session.  As, btw., explained in the documentation:
> 
>   https://cygwin.com/cygwin-ug-net/ntsec.html

...and if my reply wasn't clear enough:

Cygserver will help in this scenario as outlined in the documentation.
It caches the account information system-wide, so the stuff the first ls
cached is available for the next ls.  Or the next shell session.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Bryan Berns]

On Tue, Apr 14, 2015 at 10:53 AM, Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
> On Apr 14 07:24, Bryan Berns wrote:
>> On Tue, Apr 14, 2015 at 4:00 AM, Corinna Vinschen
>> >
>> > The problem is that Cygwin, or any other tool trying to resolve SIDs
>> > doesn't know a SID won't resolve before it tried.  And then it's an
>> > OS function which takes its time.  It's like checking for network
>> > machines providing shares.  Sometimes this test takes ages, but in
>> > this case, fortunately, you see that it takes ages in Explorer as
>> > well.
>> >
>> > As for ACLs, you can alleviate the problem somewhat by running cygserver
>> > on the machine, which allows to cache SIDs for all processes.  So only
>> > the first process trying the SID will take time, followup processes will
>> > get the cached results from cygserver.
>> >
>> > Other than that, except for ignoring ACLs entirely (noacl) I have
>> > no idea how to solve this problem differently.
>>
>> Yes, I understand there's nothing Cygwin can do beforehand -- that
>> means sense.  I guess what I'm saying is that Cygwin doesn't appear to
>> be caching SIDs in certain scenarios.
>>
>> For example, I create a whole bunch of files (like 5000),  I use
>> icacls to append a new ACE.  Then I do a 'time ls -l
>> /cygdrive/c/somedir/*'.  Takes four seconds.  In the same Cygwin
>> session, I remove the local group (net localgroup testgroup /delete).
>>  I do the same 'time ls -l /cygdrive/c/somedir/*'.  Takes 20 seconds.
>> Subsequent runs in the also take 20 seconds.  Since I'm able to
>> continue to see the slowdown in the same session, cygserver wouldn't
>> help right?
>>
>> Is the above expected?
>
> Yes.  Without cygserver, caching only works from parent to child process.
> One run of ls can't cache data for a parallel run of ls in trhe same
> session.  As, btw., explained in the documentation:
>
>   https://cygwin.com/cygwin-ug-net/ntsec.html

Alright, I'll give it a shot when I get back to my lab.  I suspect it
shouldn't take an additional 16 seconds to attempt to lookup account
information (and fail) on my two node test network so I'm curious how
much this will cut the time by.
If I setup cygserver with all the --no options set (reference:
https://cygwin.com/cygwin-ug-net/using-cygserver.html) since I don't
want any accidental cross-user information sharing, will that
effectively only provide the SID caching functionality or is there
other functionality to be wary of?

Thanks for everything!

Bryan

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1829 bytes --]

On Apr 14 12:44, Bryan Berns wrote:
> On Tue, Apr 14, 2015 at 10:53 AM, Corinna Vinschen
> <corinna-cygwin@cygwin.com> wrote:
> > On Apr 14 07:24, Bryan Berns wrote:
> >> For example, I create a whole bunch of files (like 5000),  I use
> >> icacls to append a new ACE.  Then I do a 'time ls -l
> >> /cygdrive/c/somedir/*'.  Takes four seconds.  In the same Cygwin
> >> session, I remove the local group (net localgroup testgroup /delete).
> >>  I do the same 'time ls -l /cygdrive/c/somedir/*'.  Takes 20 seconds.
> >> Subsequent runs in the also take 20 seconds.  Since I'm able to
> >> continue to see the slowdown in the same session, cygserver wouldn't
> >> help right?
> >>
> >> Is the above expected?
> >
> > Yes.  Without cygserver, caching only works from parent to child process.
> > One run of ls can't cache data for a parallel run of ls in trhe same
> > session.  As, btw., explained in the documentation:
> >
> >   https://cygwin.com/cygwin-ug-net/ntsec.html
> 
> Alright, I'll give it a shot when I get back to my lab.  I suspect it
> shouldn't take an additional 16 seconds to attempt to lookup account
> information (and fail) on my two node test network so I'm curious how
> much this will cut the time by.
> If I setup cygserver with all the --no options set (reference:
> https://cygwin.com/cygwin-ug-net/using-cygserver.html) since I don't
> want any accidental cross-user information sharing, will that
> effectively only provide the SID caching functionality or is there
> other functionality to be wary of?

You don't have to disable anything.  Just don't set the debug option
to avoid logging passwd entries.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Bryan Berns]

On Tue, Apr 14, 2015 at 2:23 PM, Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
> On Apr 14 12:44, Bryan Berns wrote:
>> On Tue, Apr 14, 2015 at 10:53 AM, Corinna Vinschen
>> <corinna-cygwin@cygwin.com> wrote:
>> > On Apr 14 07:24, Bryan Berns wrote:
>> >> For example, I create a whole bunch of files (like 5000),  I use
>> >> icacls to append a new ACE.  Then I do a 'time ls -l
>> >> /cygdrive/c/somedir/*'.  Takes four seconds.  In the same Cygwin
>> >> session, I remove the local group (net localgroup testgroup /delete).
>> >>  I do the same 'time ls -l /cygdrive/c/somedir/*'.  Takes 20 seconds.
>> >> Subsequent runs in the also take 20 seconds.  Since I'm able to
>> >> continue to see the slowdown in the same session, cygserver wouldn't
>> >> help right?
>> >>
>> >> Is the above expected?
>> >
>> > Yes.  Without cygserver, caching only works from parent to child process.
>> > One run of ls can't cache data for a parallel run of ls in trhe same
>> > session.  As, btw., explained in the documentation:
>> >
>> >   https://cygwin.com/cygwin-ug-net/ntsec.html
>>
>> Alright, I'll give it a shot when I get back to my lab.  I suspect it
>> shouldn't take an additional 16 seconds to attempt to lookup account
>> information (and fail) on my two node test network so I'm curious how
>> much this will cut the time by.
>> If I setup cygserver with all the --no options set (reference:
>> https://cygwin.com/cygwin-ug-net/using-cygserver.html) since I don't
>> want any accidental cross-user information sharing, will that
>> effectively only provide the SID caching functionality or is there
>> other functionality to be wary of?
>
> You don't have to disable anything.  Just don't set the debug option
> to avoid logging passwd entries.
>

Finally tested with cygserver (temporarily with debug on so I can see
what's going on).  I can definitely see the one entry returned when I
run 'ls -l' over my whole collection of files while my test group
(LocalGroupTest) is still present.  Sample log as follows:

/home/corinna/src/cygwin/cygwin-2.0.0/prerelease/cygwin-2.0.0-0.4.i686/src/newlib-cygwin/winsup/cygserver/pwdgrp.cc,
line 167: Request account information returns
<BERNS-TEST+LocalGroupTest:S-1-5-21-1187188735-1394039937-4283913392-1011:197619:>
error 0

If I delete the group while cygserver is running, the results continue
to be speedy.   However, as soon as I delete the group and restart
cygserver, things go south.  Performance is even worse than without
cygserver and there are entries for EVERY file that 'ls' is hitting
even though they all have the same group in the ACL so it appears the
'Unknown' users/groups are not being cached.  Sample log as follows
(one of thousands of lines):

Request account information returns
<Unknown+User:*:4294967295:4294967295:U-Unknown\User,S-1-5-21-1187188735-1394039937-4283913392:/:/sbin/nologin>
error 0

Thoughts?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2846 bytes --]

On Apr 14 21:23, Bryan Berns wrote:
> >> On Tue, Apr 14, 2015 at 10:53 AM, Corinna Vinschen
> >> > On Apr 14 07:24, Bryan Berns wrote:
> >> >> For example, I create a whole bunch of files (like 5000),  I use
> >> >> icacls to append a new ACE.  Then I do a 'time ls -l
> >> >> /cygdrive/c/somedir/*'.  Takes four seconds.  In the same Cygwin
> >> >> session, I remove the local group (net localgroup testgroup /delete).
> >> >>  I do the same 'time ls -l /cygdrive/c/somedir/*'.  Takes 20 seconds.
> >> >> Subsequent runs in the also take 20 seconds.  Since I'm able to
> >> >> continue to see the slowdown in the same session, cygserver wouldn't
> >> >> help right?
> >> >>
> >> >> Is the above expected?
> >> >
> >> > Yes.  Without cygserver, caching only works from parent to child process.
> >> > One run of ls can't cache data for a parallel run of ls in trhe same
> >> > session.  As, btw., explained in the documentation:
> [...]
> Finally tested with cygserver (temporarily with debug on so I can see
> what's going on).  I can definitely see the one entry returned when I
> run 'ls -l' over my whole collection of files while my test group
> (LocalGroupTest) is still present.  Sample log as follows:
> 
> /home/corinna/src/cygwin/cygwin-2.0.0/prerelease/cygwin-2.0.0-0.4.i686/src/newlib-cygwin/winsup/cygserver/pwdgrp.cc,
> line 167: Request account information returns
> <BERNS-TEST+LocalGroupTest:S-1-5-21-1187188735-1394039937-4283913392-1011:197619:>
> error 0
> 
> If I delete the group while cygserver is running, the results continue
> to be speedy.   However, as soon as I delete the group and restart
> cygserver, things go south.  Performance is even worse than without
> cygserver and there are entries for EVERY file that 'ls' is hitting
> even though they all have the same group in the ACL so it appears the
> 'Unknown' users/groups are not being cached.  Sample log as follows
> (one of thousands of lines):
> 
> Request account information returns
> <Unknown+User:*:4294967295:4294967295:U-Unknown\User,S-1-5-21-1187188735-1394039937-4283913392:/:/sbin/nologin>
> error 0
> 
> Thoughts?

Not off the top of my head.  The mechanism doesn't check for the
content so it should cache the above line the same way as any other.
I'm puzzled about this behaviour myself.

That requires some debugging but I have other stuff on my plate atm
so it will take a while(*).  For the time being, use noacl mounts.


Corinna

(*) Of course, I'd appreciate any other person debugging what happens
    in Cygwin and Cygserver a lot.  We can discuss questions on serious
    debugging efforts on the cygwin-developers mailing list.


-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Bryan Berns]

On Wed, Apr 15, 2015 at 3:29 AM, Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
>
> Not off the top of my head.  The mechanism doesn't check for the
> content so it should cache the above line the same way as any other.
> I'm puzzled about this behaviour myself.
>
> That requires some debugging but I have other stuff on my plate atm
> so it will take a while(*).  For the time being, use noacl mounts.
>
>
> Corinna
>
> (*) Of course, I'd appreciate any other person debugging what happens
>     in Cygwin and Cygserver a lot.  We can discuss questions on serious
>     debugging efforts on the cygwin-developers mailing list.

I know C/C++ and IPC quite well so I'll try to take a look tomorrow
night.  Of course, I'm still a novice when it comes to the Cygwin
internals so it may take a bit.  Thanks Corinna!

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Making Cygwin More Tolerant of Orphaned SIDs?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1245 bytes --]

On Apr 15 05:44, Bryan Berns wrote:
> On Wed, Apr 15, 2015 at 3:29 AM, Corinna Vinschen
> <corinna-cygwin@cygwin.com> wrote:
> >
> > Not off the top of my head.  The mechanism doesn't check for the
> > content so it should cache the above line the same way as any other.
> > I'm puzzled about this behaviour myself.
> >
> > That requires some debugging but I have other stuff on my plate atm
> > so it will take a while(*).  For the time being, use noacl mounts.
> >
> >
> > Corinna
> >
> > (*) Of course, I'd appreciate any other person debugging what happens
> >     in Cygwin and Cygserver a lot.  We can discuss questions on serious
> >     debugging efforts on the cygwin-developers mailing list.
> 
> I know C/C++ and IPC quite well so I'll try to take a look tomorrow
> night.  Of course, I'm still a novice when it comes to the Cygwin
> internals so it may take a bit.  Thanks Corinna!

Thanks for the offer.  It's probably some weird thinko in one corner
of the code.  As I said, feel free to discuss gory source code details
on cygwin-developers.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]