* Making Cygwin More Tolerant of Orphaned SIDs? @ 2015-04-14 3:58 Bryan Berns 2015-04-14 6:57 ` Achim Gratz 0 siblings, 1 reply; 17+ messages in thread From: Bryan Berns @ 2015-04-14 3:58 UTC (permalink / raw) To: cygwin Based on some rudimentary performance tests, it would appear that Cygwin may repeatedly try to lookup information on a SID form an ACE if cannot find a corresponding account.... which will undoubtedly occur for orphaned SIDs. If the volume being read is remote, this can result in some massive slowdowns if there are many files in the directory. Is there any way to augment this behavior? -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-14 3:58 Making Cygwin More Tolerant of Orphaned SIDs? Bryan Berns @ 2015-04-14 6:57 ` Achim Gratz 2015-04-14 8:00 ` Corinna Vinschen 0 siblings, 1 reply; 17+ messages in thread From: Achim Gratz @ 2015-04-14 6:57 UTC (permalink / raw) To: cygwin Bryan Berns <bryan.berns <at> gmail.com> writes: > Based on some rudimentary performance tests, it would appear that > Cygwin may repeatedly try to lookup information on a SID form an ACE > if cannot find a corresponding account.... which will undoubtedly > occur for orphaned SIDs. If the volume being read is remote, this can > result in some massive slowdowns if there are many files in the > directory. Is there any way to augment this behavior? Mounting the volume "noacl" doesn't help? Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-14 6:57 ` Achim Gratz @ 2015-04-14 8:00 ` Corinna Vinschen 2015-04-14 9:08 ` Bryan Berns 0 siblings, 1 reply; 17+ messages in thread From: Corinna Vinschen @ 2015-04-14 8:00 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1032 bytes --] On Apr 14 06:57, Achim Gratz wrote: > Bryan Berns <bryan.berns <at> gmail.com> writes: > > Based on some rudimentary performance tests, it would appear that > > Cygwin may repeatedly try to lookup information on a SID form an ACE > > if cannot find a corresponding account.... which will undoubtedly > > occur for orphaned SIDs. If the volume being read is remote, this can > > result in some massive slowdowns if there are many files in the > > directory. Is there any way to augment this behavior? > > Mounting the volume "noacl" doesn't help? Orphaned SIDs shouldn't happen. Disabling accounts, ok, but removing them? I don't know. So the question is, if there's no account with these SIDs anymore, why aren't these SIDs removed from the ACLs? It's not only Cygwin. These SIDs also unnecessarily slow down each single access check of the OS. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-14 8:00 ` Corinna Vinschen @ 2015-04-14 9:08 ` Bryan Berns 2015-04-14 9:23 ` Corinna Vinschen 0 siblings, 1 reply; 17+ messages in thread From: Bryan Berns @ 2015-04-14 9:08 UTC (permalink / raw) To: cygwin On Tue, Apr 14, 2015 at 4:00 AM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote: > > Orphaned SIDs shouldn't happen. Disabling accounts, ok, but removing > them? I don't know. So the question is, if there's no account with > these SIDs anymore, why aren't these SIDs removed from the ACLs? > It's not only Cygwin. These SIDs also unnecessarily slow down each > single access check of the OS. > In principal, I agree 100%. Unfortunately, in some large enterprise environments removal of orphaned SIDs rarely happens on a regular basis. The best way to manage this is typically to only delegate access via groups and have those groups aligned to the file system structure in some way (which tends to change less in practice than company organizational structure). Still, when you've got dozens of people starting/leaving every week, per account permission are occasionally established enumerating more a petabyte of data across several sites to cleanup ACEs is certainly possible but not on the top list of things to do (and mass alteration of ACLs carries some liability to it). Don't get me wrong, my anal retentive nature makes me cringe when I see an orphaned SID; it's just the reality of the situation. That said, the origin of my question was actually not due to unresolvable SIDs to due to removed accounts --- it was just the easiest one to describe. The reason I noticed this is because we have some NTFS assignments via local groups on a remote computers (and those local groups then have nested Active Directory groups). So the ACE has REMOTECOMPUTER\Group vice DOMAIN\Group. When Cygwin attempts to retrieve information on these accounts, it seems to fail and causes delays. So with the newer versions of Cygwin, doing an 'ls -l' went from 2 seconds to more than 30 seconds on some particular file directories. As Achim alluded, 'noacl' may be be the way to go for us, but I was just asking the question in the even there was a configurable setting or a feature enhancement that could be integrated to deal with these scenarios. Of course, 'noacl' seems to mark group / other masks as readable so apps that do permissions checks on these files will return inaccurate results :-(. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-14 9:08 ` Bryan Berns @ 2015-04-14 9:23 ` Corinna Vinschen 2015-04-14 11:24 ` Bryan Berns 0 siblings, 1 reply; 17+ messages in thread From: Corinna Vinschen @ 2015-04-14 9:23 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 3265 bytes --] On Apr 14 05:08, Bryan Berns wrote: > On Tue, Apr 14, 2015 at 4:00 AM, Corinna Vinschen > <corinna-cygwin@cygwin.com> wrote: > > > > Orphaned SIDs shouldn't happen. Disabling accounts, ok, but removing > > them? I don't know. So the question is, if there's no account with > > these SIDs anymore, why aren't these SIDs removed from the ACLs? > > It's not only Cygwin. These SIDs also unnecessarily slow down each > > single access check of the OS. > > > > In principal, I agree 100%. Unfortunately, in some large enterprise > environments removal of orphaned SIDs rarely happens on a regular > basis. The best way to manage this is typically to only delegate > access via groups and have those groups aligned to the file system > structure in some way (which tends to change less in practice than > company organizational structure). Still, when you've got dozens of > people starting/leaving every week, per account permission are > occasionally established enumerating more a petabyte of data across > several sites to cleanup ACEs is certainly possible but not on the top > list of things to do (and mass alteration of ACLs carries some > liability to it). Don't get me wrong, my anal retentive nature makes > me cringe when I see an orphaned SID; it's just the reality of the > situation. > > That said, the origin of my question was actually not due to > unresolvable SIDs to due to removed accounts --- it was just the > easiest one to describe. The reason I noticed this is because we have > some NTFS assignments via local groups on a remote computers (and > those local groups then have nested Active Directory groups). So the > ACE has REMOTECOMPUTER\Group vice DOMAIN\Group. When Cygwin attempts > to retrieve information on these accounts, it seems to fail and causes > delays. So with the newer versions of Cygwin, doing an 'ls -l' went > from 2 seconds to more than 30 seconds on some particular file > directories. > > As Achim alluded, 'noacl' may be be the way to go for us, but I was > just asking the question in the even there was a configurable setting > or a feature enhancement that could be integrated to deal with these > scenarios. Of course, 'noacl' seems to mark group / other masks as > readable so apps that do permissions checks on these files will return > inaccurate results :-(. The problem is that Cygwin, or any other tool trying to resolve SIDs doesn't know a SID won't resolve before it tried. And then it's an OS function which takes its time. It's like checking for network machines providing shares. Sometimes this test takes ages, but in this case, fortunately, you see that it takes ages in Explorer as well. As for ACLs, you can alleviate the problem somewhat by running cygserver on the machine, which allows to cache SIDs for all processes. So only the first process trying the SID will take time, followup processes will get the cached results from cygserver. Other than that, except for ignoring ACLs entirely (noacl) I have no idea how to solve this problem differently. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-14 9:23 ` Corinna Vinschen @ 2015-04-14 11:24 ` Bryan Berns 2015-04-14 12:31 ` Noel Grandin 2015-04-14 14:53 ` Corinna Vinschen 0 siblings, 2 replies; 17+ messages in thread From: Bryan Berns @ 2015-04-14 11:24 UTC (permalink / raw) To: cygwin On Tue, Apr 14, 2015 at 4:00 AM, Corinna Vinschen > > The problem is that Cygwin, or any other tool trying to resolve SIDs > doesn't know a SID won't resolve before it tried. And then it's an > OS function which takes its time. It's like checking for network > machines providing shares. Sometimes this test takes ages, but in > this case, fortunately, you see that it takes ages in Explorer as > well. > > As for ACLs, you can alleviate the problem somewhat by running cygserver > on the machine, which allows to cache SIDs for all processes. So only > the first process trying the SID will take time, followup processes will > get the cached results from cygserver. > > Other than that, except for ignoring ACLs entirely (noacl) I have > no idea how to solve this problem differently. Yes, I understand there's nothing Cygwin can do beforehand -- that means sense. I guess what I'm saying is that Cygwin doesn't appear to be caching SIDs in certain scenarios. For example, I create a whole bunch of files (like 5000), I use icacls to append a new ACE. Then I do a 'time ls -l /cygdrive/c/somedir/*'. Takes four seconds. In the same Cygwin session, I remove the local group (net localgroup testgroup /delete). I do the same 'time ls -l /cygdrive/c/somedir/*'. Takes 20 seconds. Subsequent runs in the also take 20 seconds. Since I'm able to continue to see the slowdown in the same session, cygserver wouldn't help right? Is the above expected? If not and it's something you're willing to take a look at, I can certainly come up with a little script to demonstrate the issue. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-14 11:24 ` Bryan Berns @ 2015-04-14 12:31 ` Noel Grandin 2015-04-14 14:54 ` Corinna Vinschen 2015-04-14 14:53 ` Corinna Vinschen 1 sibling, 1 reply; 17+ messages in thread From: Noel Grandin @ 2015-04-14 12:31 UTC (permalink / raw) To: cygwin On 2015-04-14 01:24 PM, Bryan Berns wrote: > I do the same 'time ls -l /cygdrive/c/somedir/*'. Takes 20 seconds. > Subsequent runs in the also take 20 seconds. Since I'm able to It sounds like Cygwin needs a negative-entry cache i.e. a cache of ACL's that are know to be orphans or otherwise bad to prevent it repeatedly trying them. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-14 12:31 ` Noel Grandin @ 2015-04-14 14:54 ` Corinna Vinschen 2015-04-14 15:00 ` Corinna Vinschen 0 siblings, 1 reply; 17+ messages in thread From: Corinna Vinschen @ 2015-04-14 14:54 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 772 bytes --] On Apr 14 14:30, Noel Grandin wrote: > > > On 2015-04-14 01:24 PM, Bryan Berns wrote: > > I do the same 'time ls -l /cygdrive/c/somedir/*'. Takes 20 seconds. > >Subsequent runs in the also take 20 seconds. Since I'm able to > > It sounds like Cygwin needs a negative-entry cache i.e. a cache of ACL's > that are know to be orphans or otherwise bad to prevent it repeatedly trying > them. It doesn't if it tried it once since after that it's cached within the limits of the caching algorithm, as explained in the documentation nobody reads: https://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-files -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-14 14:54 ` Corinna Vinschen @ 2015-04-14 15:00 ` Corinna Vinschen 0 siblings, 0 replies; 17+ messages in thread From: Corinna Vinschen @ 2015-04-14 15:00 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 945 bytes --] On Apr 14 16:54, Corinna Vinschen wrote: > On Apr 14 14:30, Noel Grandin wrote: > > > > > > On 2015-04-14 01:24 PM, Bryan Berns wrote: > > > I do the same 'time ls -l /cygdrive/c/somedir/*'. Takes 20 seconds. > > >Subsequent runs in the also take 20 seconds. Since I'm able to > > > > It sounds like Cygwin needs a negative-entry cache i.e. a cache of ACL's > > that are know to be orphans or otherwise bad to prevent it repeatedly trying > > them. > > It doesn't if it tried it once since after that it's cached within the > limits of the caching algorithm, as explained in the documentation > nobody reads: > > https://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-files Actually: https://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-mapping-caching Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-14 11:24 ` Bryan Berns 2015-04-14 12:31 ` Noel Grandin @ 2015-04-14 14:53 ` Corinna Vinschen 2015-04-14 16:21 ` Corinna Vinschen 2015-04-14 16:45 ` Bryan Berns 1 sibling, 2 replies; 17+ messages in thread From: Corinna Vinschen @ 2015-04-14 14:53 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2003 bytes --] On Apr 14 07:24, Bryan Berns wrote: > On Tue, Apr 14, 2015 at 4:00 AM, Corinna Vinschen > > > > The problem is that Cygwin, or any other tool trying to resolve SIDs > > doesn't know a SID won't resolve before it tried. And then it's an > > OS function which takes its time. It's like checking for network > > machines providing shares. Sometimes this test takes ages, but in > > this case, fortunately, you see that it takes ages in Explorer as > > well. > > > > As for ACLs, you can alleviate the problem somewhat by running cygserver > > on the machine, which allows to cache SIDs for all processes. So only > > the first process trying the SID will take time, followup processes will > > get the cached results from cygserver. > > > > Other than that, except for ignoring ACLs entirely (noacl) I have > > no idea how to solve this problem differently. > > Yes, I understand there's nothing Cygwin can do beforehand -- that > means sense. I guess what I'm saying is that Cygwin doesn't appear to > be caching SIDs in certain scenarios. > > For example, I create a whole bunch of files (like 5000), I use > icacls to append a new ACE. Then I do a 'time ls -l > /cygdrive/c/somedir/*'. Takes four seconds. In the same Cygwin > session, I remove the local group (net localgroup testgroup /delete). > I do the same 'time ls -l /cygdrive/c/somedir/*'. Takes 20 seconds. > Subsequent runs in the also take 20 seconds. Since I'm able to > continue to see the slowdown in the same session, cygserver wouldn't > help right? > > Is the above expected? Yes. Without cygserver, caching only works from parent to child process. One run of ls can't cache data for a parallel run of ls in trhe same session. As, btw., explained in the documentation: https://cygwin.com/cygwin-ug-net/ntsec.html Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-14 14:53 ` Corinna Vinschen @ 2015-04-14 16:21 ` Corinna Vinschen 2015-04-14 16:45 ` Bryan Berns 1 sibling, 0 replies; 17+ messages in thread From: Corinna Vinschen @ 2015-04-14 16:21 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2378 bytes --] On Apr 14 16:53, Corinna Vinschen wrote: > On Apr 14 07:24, Bryan Berns wrote: > > On Tue, Apr 14, 2015 at 4:00 AM, Corinna Vinschen > > > > > > The problem is that Cygwin, or any other tool trying to resolve SIDs > > > doesn't know a SID won't resolve before it tried. And then it's an > > > OS function which takes its time. It's like checking for network > > > machines providing shares. Sometimes this test takes ages, but in > > > this case, fortunately, you see that it takes ages in Explorer as > > > well. > > > > > > As for ACLs, you can alleviate the problem somewhat by running cygserver > > > on the machine, which allows to cache SIDs for all processes. So only > > > the first process trying the SID will take time, followup processes will > > > get the cached results from cygserver. > > > > > > Other than that, except for ignoring ACLs entirely (noacl) I have > > > no idea how to solve this problem differently. > > > > Yes, I understand there's nothing Cygwin can do beforehand -- that > > means sense. I guess what I'm saying is that Cygwin doesn't appear to > > be caching SIDs in certain scenarios. > > > > For example, I create a whole bunch of files (like 5000), I use > > icacls to append a new ACE. Then I do a 'time ls -l > > /cygdrive/c/somedir/*'. Takes four seconds. In the same Cygwin > > session, I remove the local group (net localgroup testgroup /delete). > > I do the same 'time ls -l /cygdrive/c/somedir/*'. Takes 20 seconds. > > Subsequent runs in the also take 20 seconds. Since I'm able to > > continue to see the slowdown in the same session, cygserver wouldn't > > help right? > > > > Is the above expected? > > Yes. Without cygserver, caching only works from parent to child process. > One run of ls can't cache data for a parallel run of ls in trhe same > session. As, btw., explained in the documentation: > > https://cygwin.com/cygwin-ug-net/ntsec.html ...and if my reply wasn't clear enough: Cygserver will help in this scenario as outlined in the documentation. It caches the account information system-wide, so the stuff the first ls cached is available for the next ls. Or the next shell session. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-14 14:53 ` Corinna Vinschen 2015-04-14 16:21 ` Corinna Vinschen @ 2015-04-14 16:45 ` Bryan Berns 2015-04-14 18:23 ` Corinna Vinschen 1 sibling, 1 reply; 17+ messages in thread From: Bryan Berns @ 2015-04-14 16:45 UTC (permalink / raw) To: cygwin On Tue, Apr 14, 2015 at 10:53 AM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote: > On Apr 14 07:24, Bryan Berns wrote: >> On Tue, Apr 14, 2015 at 4:00 AM, Corinna Vinschen >> > >> > The problem is that Cygwin, or any other tool trying to resolve SIDs >> > doesn't know a SID won't resolve before it tried. And then it's an >> > OS function which takes its time. It's like checking for network >> > machines providing shares. Sometimes this test takes ages, but in >> > this case, fortunately, you see that it takes ages in Explorer as >> > well. >> > >> > As for ACLs, you can alleviate the problem somewhat by running cygserver >> > on the machine, which allows to cache SIDs for all processes. So only >> > the first process trying the SID will take time, followup processes will >> > get the cached results from cygserver. >> > >> > Other than that, except for ignoring ACLs entirely (noacl) I have >> > no idea how to solve this problem differently. >> >> Yes, I understand there's nothing Cygwin can do beforehand -- that >> means sense. I guess what I'm saying is that Cygwin doesn't appear to >> be caching SIDs in certain scenarios. >> >> For example, I create a whole bunch of files (like 5000), I use >> icacls to append a new ACE. Then I do a 'time ls -l >> /cygdrive/c/somedir/*'. Takes four seconds. In the same Cygwin >> session, I remove the local group (net localgroup testgroup /delete). >> I do the same 'time ls -l /cygdrive/c/somedir/*'. Takes 20 seconds. >> Subsequent runs in the also take 20 seconds. Since I'm able to >> continue to see the slowdown in the same session, cygserver wouldn't >> help right? >> >> Is the above expected? > > Yes. Without cygserver, caching only works from parent to child process. > One run of ls can't cache data for a parallel run of ls in trhe same > session. As, btw., explained in the documentation: > > https://cygwin.com/cygwin-ug-net/ntsec.html Alright, I'll give it a shot when I get back to my lab. I suspect it shouldn't take an additional 16 seconds to attempt to lookup account information (and fail) on my two node test network so I'm curious how much this will cut the time by. If I setup cygserver with all the --no options set (reference: https://cygwin.com/cygwin-ug-net/using-cygserver.html) since I don't want any accidental cross-user information sharing, will that effectively only provide the SID caching functionality or is there other functionality to be wary of? Thanks for everything! Bryan -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-14 16:45 ` Bryan Berns @ 2015-04-14 18:23 ` Corinna Vinschen 2015-04-15 1:23 ` Bryan Berns 0 siblings, 1 reply; 17+ messages in thread From: Corinna Vinschen @ 2015-04-14 18:23 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1829 bytes --] On Apr 14 12:44, Bryan Berns wrote: > On Tue, Apr 14, 2015 at 10:53 AM, Corinna Vinschen > <corinna-cygwin@cygwin.com> wrote: > > On Apr 14 07:24, Bryan Berns wrote: > >> For example, I create a whole bunch of files (like 5000), I use > >> icacls to append a new ACE. Then I do a 'time ls -l > >> /cygdrive/c/somedir/*'. Takes four seconds. In the same Cygwin > >> session, I remove the local group (net localgroup testgroup /delete). > >> I do the same 'time ls -l /cygdrive/c/somedir/*'. Takes 20 seconds. > >> Subsequent runs in the also take 20 seconds. Since I'm able to > >> continue to see the slowdown in the same session, cygserver wouldn't > >> help right? > >> > >> Is the above expected? > > > > Yes. Without cygserver, caching only works from parent to child process. > > One run of ls can't cache data for a parallel run of ls in trhe same > > session. As, btw., explained in the documentation: > > > > https://cygwin.com/cygwin-ug-net/ntsec.html > > Alright, I'll give it a shot when I get back to my lab. I suspect it > shouldn't take an additional 16 seconds to attempt to lookup account > information (and fail) on my two node test network so I'm curious how > much this will cut the time by. > If I setup cygserver with all the --no options set (reference: > https://cygwin.com/cygwin-ug-net/using-cygserver.html) since I don't > want any accidental cross-user information sharing, will that > effectively only provide the SID caching functionality or is there > other functionality to be wary of? You don't have to disable anything. Just don't set the debug option to avoid logging passwd entries. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-14 18:23 ` Corinna Vinschen @ 2015-04-15 1:23 ` Bryan Berns 2015-04-15 7:29 ` Corinna Vinschen 0 siblings, 1 reply; 17+ messages in thread From: Bryan Berns @ 2015-04-15 1:23 UTC (permalink / raw) To: cygwin On Tue, Apr 14, 2015 at 2:23 PM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote: > On Apr 14 12:44, Bryan Berns wrote: >> On Tue, Apr 14, 2015 at 10:53 AM, Corinna Vinschen >> <corinna-cygwin@cygwin.com> wrote: >> > On Apr 14 07:24, Bryan Berns wrote: >> >> For example, I create a whole bunch of files (like 5000), I use >> >> icacls to append a new ACE. Then I do a 'time ls -l >> >> /cygdrive/c/somedir/*'. Takes four seconds. In the same Cygwin >> >> session, I remove the local group (net localgroup testgroup /delete). >> >> I do the same 'time ls -l /cygdrive/c/somedir/*'. Takes 20 seconds. >> >> Subsequent runs in the also take 20 seconds. Since I'm able to >> >> continue to see the slowdown in the same session, cygserver wouldn't >> >> help right? >> >> >> >> Is the above expected? >> > >> > Yes. Without cygserver, caching only works from parent to child process. >> > One run of ls can't cache data for a parallel run of ls in trhe same >> > session. As, btw., explained in the documentation: >> > >> > https://cygwin.com/cygwin-ug-net/ntsec.html >> >> Alright, I'll give it a shot when I get back to my lab. I suspect it >> shouldn't take an additional 16 seconds to attempt to lookup account >> information (and fail) on my two node test network so I'm curious how >> much this will cut the time by. >> If I setup cygserver with all the --no options set (reference: >> https://cygwin.com/cygwin-ug-net/using-cygserver.html) since I don't >> want any accidental cross-user information sharing, will that >> effectively only provide the SID caching functionality or is there >> other functionality to be wary of? > > You don't have to disable anything. Just don't set the debug option > to avoid logging passwd entries. > Finally tested with cygserver (temporarily with debug on so I can see what's going on). I can definitely see the one entry returned when I run 'ls -l' over my whole collection of files while my test group (LocalGroupTest) is still present. Sample log as follows: /home/corinna/src/cygwin/cygwin-2.0.0/prerelease/cygwin-2.0.0-0.4.i686/src/newlib-cygwin/winsup/cygserver/pwdgrp.cc, line 167: Request account information returns <BERNS-TEST+LocalGroupTest:S-1-5-21-1187188735-1394039937-4283913392-1011:197619:> error 0 If I delete the group while cygserver is running, the results continue to be speedy. However, as soon as I delete the group and restart cygserver, things go south. Performance is even worse than without cygserver and there are entries for EVERY file that 'ls' is hitting even though they all have the same group in the ACL so it appears the 'Unknown' users/groups are not being cached. Sample log as follows (one of thousands of lines): Request account information returns <Unknown+User:*:4294967295:4294967295:U-Unknown\User,S-1-5-21-1187188735-1394039937-4283913392:/:/sbin/nologin> error 0 Thoughts? -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-15 1:23 ` Bryan Berns @ 2015-04-15 7:29 ` Corinna Vinschen 2015-04-15 9:44 ` Bryan Berns 0 siblings, 1 reply; 17+ messages in thread From: Corinna Vinschen @ 2015-04-15 7:29 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2846 bytes --] On Apr 14 21:23, Bryan Berns wrote: > >> On Tue, Apr 14, 2015 at 10:53 AM, Corinna Vinschen > >> > On Apr 14 07:24, Bryan Berns wrote: > >> >> For example, I create a whole bunch of files (like 5000), I use > >> >> icacls to append a new ACE. Then I do a 'time ls -l > >> >> /cygdrive/c/somedir/*'. Takes four seconds. In the same Cygwin > >> >> session, I remove the local group (net localgroup testgroup /delete). > >> >> I do the same 'time ls -l /cygdrive/c/somedir/*'. Takes 20 seconds. > >> >> Subsequent runs in the also take 20 seconds. Since I'm able to > >> >> continue to see the slowdown in the same session, cygserver wouldn't > >> >> help right? > >> >> > >> >> Is the above expected? > >> > > >> > Yes. Without cygserver, caching only works from parent to child process. > >> > One run of ls can't cache data for a parallel run of ls in trhe same > >> > session. As, btw., explained in the documentation: > [...] > Finally tested with cygserver (temporarily with debug on so I can see > what's going on). I can definitely see the one entry returned when I > run 'ls -l' over my whole collection of files while my test group > (LocalGroupTest) is still present. Sample log as follows: > > /home/corinna/src/cygwin/cygwin-2.0.0/prerelease/cygwin-2.0.0-0.4.i686/src/newlib-cygwin/winsup/cygserver/pwdgrp.cc, > line 167: Request account information returns > <BERNS-TEST+LocalGroupTest:S-1-5-21-1187188735-1394039937-4283913392-1011:197619:> > error 0 > > If I delete the group while cygserver is running, the results continue > to be speedy. However, as soon as I delete the group and restart > cygserver, things go south. Performance is even worse than without > cygserver and there are entries for EVERY file that 'ls' is hitting > even though they all have the same group in the ACL so it appears the > 'Unknown' users/groups are not being cached. Sample log as follows > (one of thousands of lines): > > Request account information returns > <Unknown+User:*:4294967295:4294967295:U-Unknown\User,S-1-5-21-1187188735-1394039937-4283913392:/:/sbin/nologin> > error 0 > > Thoughts? Not off the top of my head. The mechanism doesn't check for the content so it should cache the above line the same way as any other. I'm puzzled about this behaviour myself. That requires some debugging but I have other stuff on my plate atm so it will take a while(*). For the time being, use noacl mounts. Corinna (*) Of course, I'd appreciate any other person debugging what happens in Cygwin and Cygserver a lot. We can discuss questions on serious debugging efforts on the cygwin-developers mailing list. -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-15 7:29 ` Corinna Vinschen @ 2015-04-15 9:44 ` Bryan Berns 2015-04-15 15:44 ` Corinna Vinschen 0 siblings, 1 reply; 17+ messages in thread From: Bryan Berns @ 2015-04-15 9:44 UTC (permalink / raw) To: cygwin On Wed, Apr 15, 2015 at 3:29 AM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote: > > Not off the top of my head. The mechanism doesn't check for the > content so it should cache the above line the same way as any other. > I'm puzzled about this behaviour myself. > > That requires some debugging but I have other stuff on my plate atm > so it will take a while(*). For the time being, use noacl mounts. > > > Corinna > > (*) Of course, I'd appreciate any other person debugging what happens > in Cygwin and Cygserver a lot. We can discuss questions on serious > debugging efforts on the cygwin-developers mailing list. I know C/C++ and IPC quite well so I'll try to take a look tomorrow night. Of course, I'm still a novice when it comes to the Cygwin internals so it may take a bit. Thanks Corinna! -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Making Cygwin More Tolerant of Orphaned SIDs? 2015-04-15 9:44 ` Bryan Berns @ 2015-04-15 15:44 ` Corinna Vinschen 0 siblings, 0 replies; 17+ messages in thread From: Corinna Vinschen @ 2015-04-15 15:44 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1245 bytes --] On Apr 15 05:44, Bryan Berns wrote: > On Wed, Apr 15, 2015 at 3:29 AM, Corinna Vinschen > <corinna-cygwin@cygwin.com> wrote: > > > > Not off the top of my head. The mechanism doesn't check for the > > content so it should cache the above line the same way as any other. > > I'm puzzled about this behaviour myself. > > > > That requires some debugging but I have other stuff on my plate atm > > so it will take a while(*). For the time being, use noacl mounts. > > > > > > Corinna > > > > (*) Of course, I'd appreciate any other person debugging what happens > > in Cygwin and Cygserver a lot. We can discuss questions on serious > > debugging efforts on the cygwin-developers mailing list. > > I know C/C++ and IPC quite well so I'll try to take a look tomorrow > night. Of course, I'm still a novice when it comes to the Cygwin > internals so it may take a bit. Thanks Corinna! Thanks for the offer. It's probably some weird thinko in one corner of the code. As I said, feel free to discuss gory source code details on cygwin-developers. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2015-04-15 15:44 UTC | newest] Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-04-14 3:58 Making Cygwin More Tolerant of Orphaned SIDs? Bryan Berns 2015-04-14 6:57 ` Achim Gratz 2015-04-14 8:00 ` Corinna Vinschen 2015-04-14 9:08 ` Bryan Berns 2015-04-14 9:23 ` Corinna Vinschen 2015-04-14 11:24 ` Bryan Berns 2015-04-14 12:31 ` Noel Grandin 2015-04-14 14:54 ` Corinna Vinschen 2015-04-14 15:00 ` Corinna Vinschen 2015-04-14 14:53 ` Corinna Vinschen 2015-04-14 16:21 ` Corinna Vinschen 2015-04-14 16:45 ` Bryan Berns 2015-04-14 18:23 ` Corinna Vinschen 2015-04-15 1:23 ` Bryan Berns 2015-04-15 7:29 ` Corinna Vinschen 2015-04-15 9:44 ` Bryan Berns 2015-04-15 15:44 ` Corinna Vinschen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).