* Testers needed: New passwd/group handling in Cygwin @ 2014-02-13 14:44 Corinna Vinschen 2014-02-13 16:05 ` Steven Penny ` (8 more replies) 0 siblings, 9 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-13 14:44 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 27960 bytes --] Hi folks, this week I applied the first incarnation of the new passwd/group handling code to the Cygwin repository and after fixing a crash which manifested in Denis Excoffier's network, I think we're at a point which allows to push this forward. This is a pretty intrusive change, in need of some serious testing, so I'd like to ask for volunteers. The latest 2014-02-13 snapshot from http://cygwin.com/snapshots/ contains the changes, including the latest bugfix. If you're interested in helping, please read on. It's a long text, but I feel it's necessary to explain how this works, so you get a grasp of what to look out for. In the long run, this is the basis for the documentation to go into the User's Guide. ---------------------------------------------------------------------- Please note: If you have questions or comments, please do *NOT* full quote this mail! Just quote the important text snippet and ask away. Thanks for considering. ---------------------------------------------------------------------- Did I mention that this mail is *really* long? Sorry about that, but I don't see how to explain all the details otherwise. Fetch yourself some drink, preferredly some strong coffee or tea. Oh, and there be footnotes[0]. =========================== So, what is this all about? =========================== For as long as Cygwin has existed, it has stored user and group information in /etc/passwd and /etc/group files. Under the assumption that these files would never be too large, the first process in a process tree, as well as every execing process within the tree would parse them into structures in memory. Thus every Cygwin process would contain an expanded copy of the full information from /etc/passwd and /etc/group. This approach has a few downsides. One of them is that the idea to have always small files is flawed. Another one is that reading the entire file is most of the time entirely useless, since most processes only need information on their own user and the primary group. Last but not least, the passwd and group files have to be maintained separately from the already existing Windows user databases, the local SAM and Active Directory. On the other hand, we have to have a mapping between Windows SIDs and POSIX uid/gid values (for how this works in Cygwin so far, see [1]), so we rely on some mechanism to convert SIDs to uid/gid values and vice versa. Microsoft's "Services for UNIX" (SFU) (which are deprecated since Windows 8/Server 2012) never used passwd/group files. Rather, SFU used a fixed, computational mapping between SIDs and POSIX uid/gid. It allows to generate uid/gid values from SIDs and vice versa. The mechanism is documented, albeit in a confusing way and spread over multiple MSDN articles. For my changes, I took the liberty to clone the mapping, with a tiny difference for backward compatibility with existing Cygwin applications. =================================== Yes, yes, but how does it work now? =================================== You *are* comfortable with the concept of SIDs and RIDs, right? If not, please read [1] again. I'll wait... Finished? Ok, let's start. Cygwin's new mapping between SIDs and uid/gid values works in two ways. - Read /etc/passwd and /etc/group files, like before, mainly for backward compatibility. - If no files are present, or if an entry is missing in the files, ask Windows. At least, that's the default behaviour now. It will be configurable using a file /etc/nsswitch.conf, but I'm getting ahead of myself. Let's stick to the default for now. If files are present, they will be scanned on demand as soon as a mapping from SIDs to uid/gid or user names is required. The new mechanism will never read the entire file into memory, but only scan for the requested entry and cache this one in memory[2]. If no entry is found, or no passwd or group file was present, Cygwin will ask the OS. Note: If the first process in a Cygwin process tree determines that no passwd or group file is present, no other process in the entire process tree will try to read the files later on. This is done for self-preservation. It's kind of bad if the uid or gid of a user changes during the ride. So if we've drawn a blank reading the files, we're going to ask the OS. First thing, we ask the local machine for the SID or the user name. The OS functions (LookupAccountSid/LookupAccountName[3]) are kind of intelligent. They have all the stuff built in to ask for any account of the local machine, the Active Directory domain of the machine, the Global Catalog of the forest of the domain, as well as any trusted domain of our forest for the information. One OS call and we're practically done. Except... ...the calls only return the mapping between SID, account name and the account's domain. We don't have a mapping to POSIX uid/gid and we're missing information on the user's home dir and login shell. Let's discuss the SID<=>uid/gid mapping first. Here's how the SID<=>uid/gid mapping works. If you're fuzzy on the existing well-known SIDs, see [4]. - Well-known SIDs in the NT_AUTHORITY domain of the S-1-5-RID type, or aliases of the S-1-5-32-RID type are mapped to the uid/gid value RID[5]. Examples: "SYSTEM" S-1-5-18 <=> uid/gid: 18 "Users" S-1-5-32-545 <=> uid/gid: 545 - Other well-known SIDs in the NT_AUTHORITY domain (S-1-5-X-RID): S-1-5-X-RID <=> uid/gid: 0x1000 * X + RID Example: "NTLM Authentication" S-1-5-64-10 <=> uid/gid: 0x4000A == 262154 - Other well-known SIDs: S-1-X-Y <=> uid/gid: 0x10000 + 0x100 * X + Y Example: "LOCAL" S-1-2-0 <=> uid/gid: 0x10200 == 66048 "Creator Group" S-1-3-1 <=> uid/gid: 0x10301 == 66305 - Logon SIDs: The own LogonSid is converted to the fixed uid 0xfff == 4095 and named "CurrentSession". Any other LogonSid is converted to the fixed uid 0xffe == 4094 and named "OtherSession". - Mandatory Labels: S-1-16-RID <=> uid/gid: 0x60000 + RID Example: "Medium Mandatory Level" S-1-16-8192 <=> uid/gid: 0x62000 == 401408 - Accounts from the local machine's user DB (SAM): S-1-5-21-X-Y-Z-RID <=> 0x30000 + RID Example: "Administrator" S-1-5-X-Y-Z-500 <=> uid/gid: 0x301f4 == 197108 - Accounts from the machine's primary domain: S-1-5-21-X-Y-Z-RID <=> 0x100000 + RID Example: "Domain Users" S-1-5-X-Y-Z-513 <=> 0x100201 == 1049089 - Accounts from a trusted domain of the machine's primary domain: S-1-5-21-X-Y-Z-RID <=> trustPosixOffset(domain) + RID Hang on... "trustPosixOffset"? Uh, yes. This value exists in Windows domains already since before Active Directory days. What happens is this. If you create a domain trust between two domains, a trustedDomain entry will be added to both databases. It describes how *this* domain trusts the *other* domain. One attribute of a trust is a 32 bit value called "trustPosixOffset" For each new trust, trustPosixOffset will get some automatic value. In recent AD domain implementations, the first trusted domain will get trustPosixOffset set to 0x80000000. Following domains will get lower values. Also, the domain admins are allowed to set the trustPosixOffset value for each trusted domain to some arbitrary 32 bit value, thus allowing any kind of collisions between the trustPosixOffsets of domains. Nice, isn't it? Anyway, as the user of this value, we have to *trust* the domain admins to set trustPosixOffset to sensible values, or to leave it alone. So, for the first (or only) trusted domain of your domain, the automatic offset is 0x80000000. An example for a user of that trusted domain is: S-1-5-X-Y-Z-1234 <=> uid/gid 0x800004d2 == 2147484882 There's only one problem with this approach. Assuming you're running in the context of a local user on a domain member machine. Local users don't have the right to fetch this kind of domain information from the DC, they'll get permission denied. In this case Cygwin will fake a, mostly, sensible trustPosixOffset value for this session. - Local accounts from another machine in the network: There's no SID<=>uid/gid mapping implemented for this case. The problem is, there's no way to generate a bijective mapping. There's no central place which keeps an analogue value of the trustPosixOffset, and there's the additional problem that the LookupAccountSid and LookupAccountName functions cannnot resolve the SIDs, unless they know the name of the machine this SID comes from. And even then it will probably suffer a "Permission denied" when trying to ask the machine for its local account. SFU just prints the account RID in this case, Cygwin maps the account to the fake accounts "Unknown+User"/"Unknown+Group" with uid/gid -1. Ok, now we have a semi-bijective mapping between SIDs and POSIX uid/gid values, but, given that we have potentially users and groups in different domains having the same name, how do we uniquely differ between them by name? Well, we can do that by making their names unique in a per-machine way. Dependent on the domain membership of the account, and dependent of the machine being a domain member or not, the user and group names will be generated using a domain prefix and a separator character between domain and account name. The default separator character is the plus sign, '+', as in SFU. - Well-known SIDs will have the separator character prepended: "+SYSTEM", "+LOCAL", "+Medium Mandatory Level", ... - If the machine is no domain member machine, only local accounts can be resolved into names, so for ease of use, just the account names are used as Cygwin user/group names: "corinna", "bigfoot", "None", ... - If the machine is a domain member machine, all accounts from the primary domain of the machine are mapped to Cygwin names without domain prefix: "corinna", "bigfoot", "Domain Users", ... while accounts from other domains are prepended by their domain: "DOMAIN1+corinna", "DOMAIN2+bigfoot", "DOMAIN3+Domain Users", ... - Local machine accounts of a domain member machine get a Cygwin user name the same way as accounts from another domain: The local machine name gets prepended: "MYMACHINE+corinna", "MYMACHINE+bigfoot", "MYMACHINE+None", ... - If LookupAccountSid fails, Cygwin checks the accounts against the known trusted domains. If the account is from one of the trusted domains, an artificial account name is created. It consists of the domain name, and a special name created from the account RID: "MY_DOM+User(1234)", "MY_DOM+Group(5678)" Otherwise we know nothing about this SID, so it will be mapped to the fake accounts "Unknown+User"/"Unknown+Group" with uid/gid -1. Any questions? No? Ok, let's have a short break and then, next point. [...5 minutes commercials...] ========================================== Cygwin user names, home dirs, login shells ========================================== "That's all very interesting" you think (if you're still awake, that is), "but passwd entries consist of more than just uids. What's up with that?" Obviously, if you don't maintain passwd and group files, you need to have a way to maintain the other fields of a passwd entry as well. Three things come to mind: - You want to use a Cygwin user name different from your Windows user name. - You want a home dir different from the default /home/$USER. - You want to specify a different login shell than /bin/sh. How this is done depends on your account being a domain account or a local account. Let's start with the default. Assuming your Windows account name is "bigfoot" and your domain is "MY_DOM". Your default passwd entry in absence of anything I'll describe below looks like this: bigfoot:*:<uid>:<gid>:U-MY_DOM\bigfoot,S-1-5-....:/home/bigfoot:/bin/sh or, if your account is from a different domain than the primary domain of the machine: MY_DOM+bigfoot:*:<uid>:<gid>:U-MY_DOM\bigfoot,S-1-5-....:/home/bigfoot:/bin/sh Yes, the default homedir is still /home/bigfoot. If your account is a domain account: Either create an /etc/passwd and/or /etc/group file with entries for your account and use that, just as before. Or, Cygwin will utilize the posixAccount/posixGroup attributes per RFC 2307[6]. These attributes are by default available in Active Directory since Windows Server 2003 R2. They are "not set", unless utilized by the (deprecated since Server 2012 R2) Active Directory "Server for NIS" feature. The user attributes utilized by Cygwin are: uid If set, will be used as Cygwin user name. uidNumber See next section. gecos Content will be added to the pw_gecos field. unixHomeDirectory If set, will be used as Cygwin home directory. loginShell If set, will be used as Cygwin login shell. The group attributes utilized by Cygwin are: cn If set, will be used as Cygwin group name. gidNumber See next section. Apart from power shell scripting or inventing new CLI tools, these attributes can be changed using the "Attribute Editor" tab in the user properties dialog of the "Active Directory Users and Computers" MMC snap-in. Alternatively, if the "Server for NIS" administration feature has been installed, ther will be a "UNIX Attributes" tab which contains the required fields, except for the gecos field, which isn't really important anyway. Last resort is "ADSI Edit". The primary group of a user is always the Windows primary group set in Active Directory and can't be changed. If your machine is not a domain member machine or your account is a local account for some reason: Either create an /etc/passwd and/or /etc/group file with entries for your account and use that, just as before. Or enter the information into the "Comment" field of your local user entry. In the "Local Users and Groups" MMC snap-in it's called "Description". You can utilze this field even if you're running a "home edition" of Windows, using the command line. The "net user" command allows to set all values in the SAM, even if the GUI is crippled. A Cygwin SAM comment entry looks like this: <cygwin key="value" key="value" [...] /> The supported keys are name="value" Sets the Cygwin user name to value. home="value" Sets the Cygwin home dir to value. shell="value" Sets the Cygwin login shell to value. group="value" Sets the Cygwin primary group of the account to value, provided that the user *is* already a member of that group. This allows to override the default "None" primary group for local accounts. One nice idea here is, for instance group="Users". This is the *Windows* name of the group, not the Cygwin name, assuming they differ. unix="value" Sets the NFS/Samba uid of the user to the decimal value. See the next chapter. The <cygwin .../> string can start at any point in the comment, but you have to follow the rules: - It starts with "<cygwin " and ends with "/>". - The "cygwin" string and the key names have to be lowercase. - No spaces between key and "value", just the equal sign. - The value must be placed within double quotes and it must not contain a double quote itself. The double quotes are required for the decimal values as well! CMD example: net use corinna /comment:"<cygwin home=\"/home/foo\"/>" Bash example (use single quotes): net use corinna /comment:'<cygwin home="/home/foo"/>' For changing group comments, use the `net localgroup' command. The supported key/value pair for groups are name="value" Sets the Cygwin group name to value. unix="value" Sets the NFS/Samba gid of the group to the decimal value. See the next chapter. ============================= NFS and Samba account mapping ============================= If you're using Microsoft's NFS client to access UNIX shares, like me, you might have been bothered by each file on the share seemingly being yours. The reason for this is that Microsoft's NFS client does not map the uid/gid values on the NFS shares to SIDs. There's no such thing as a (fake) security descriptor returned to the application. Rather, via an undocumented API you can fetch RFC 1813 compatible NFSv3 stat information from the share[7]. This is what Cygwin is using to show stat information for files on NFS shares. The problem is this. While all other information in this stat record, like timestamps, or file size etc., can be used by Cygwin, Cygwin had no way to map the values of the st_uid and st_gid members to a Windows SID. So it just faked the file owner info and claimed that it's you. However, SFU has, over time, developed multiple methods to map UNIX uid/gid values on NFS shares to Windows SIDs. You'll find the full documentation of the mapping methods in [8]. Cygwin now utilizes the RFC 2307 mapping for this purpose. This is most of the time provided by an AD domain, but it could also be a standalone LDAP mapping server. Per RFC 2307, the uid is in the attribute uidNumber. For groups, the gid is in the gidNumber attribute. When Cygwin stat's files on an NFS share, it asks the mapping server via LDAP in two different ways, depending on the role of the mapping server. - If the server is an AD domain controller, it asks for an account with uidNumber attribute == st_uid field of the stat record returned by NFS. If an account matches, AD returns the Windows SID, so we have an immediate mapping from UNIX uid to a Windows SID, if the user account has a valid uidNumber attribute. For groups, the method is the same, just that Cygwin asks for a group with gidNumber attribute == st_gid field of the stat record. - If the server is a standalone LDAP mapping server Cygwin asks for the same uidNumber/gidNumber attributes, but it can't expect that the LDAP server knows anything about Windows SIDs. Rather, the mapping server returns the account name. Cygwin then asks the DC for an account with this name, and if that succeeds, we have a mapping between UNIX uid/gid and Windows SIDs. The mapping will be cached for the lifetime of the process, and inherited by child processes. And what about Samba? A fully set up Samba with domain integration is running winbindd to map Window SIDs to artificially created UNIX uids and gids, and this mapping is transparent within the domain, so Cygwin doesn't have to do anything special. However, setting up winbindd isn't for everybody, and it fails to map Windows accounts to already existing UNIX users or groups. In contrast to NFS, Samba returns security descriptors, but unmapped UNIX accounts get special SIDs: - A UNIX user account with uid X is mapped to the Windows SID S-1-22-1-X. - A UNIX group account with gid X is mapped to SID S-1-22-2-X. As you can see, even though we have SIDs, they just reflect the actual uid/gid values on the UNIX box in the RID value. It's only marginally different from the NFS method, so... why not just do the same rote as for NFS? That's what Cygwin will do. If it encounters a S-1-22-x-y SID, it will perform the same RFC 2307 mapping as for NFS shares. Hang on. What about home users without any Windows domain or LDAP server per RFC 2307, but with a Linux machine running Samba? Not all is lost. If you're a user on a standalone machine, you can add this information to your SAM account. Assuming the uid of your Linux user account is 505 and the gid of your primary group is, say, 100, just add the values to your SAM user and group accounts. The following example assumes you didn't already add something else to the comment field. To your user's SAM comment (remember: called "Description" in the GUI), add: <cygwin group="Users" unix="505"/> To the user's group SAM comment add: <cygwin unix="100"/> This should be sufficient to work on your Samba share and to see all files owned by your Linux user account as your files. ====================================================================== The /etc/nsswitch.conf file =========================== Last, but not least, let's talk about a way to configure how the stuff works on your machine. On Linux and some other UNIXy OSes, we have a file called /etc/nsswitch.conf[9]. One part of it is to specify how the passwd and group entries are generated. That's what Cygwin now provides as well. The /etc/nsswitch.conf file is optional. If you don't have one, Cygwin uses sensible defaults. Note: The /etc/nsswitch.conf file is read exactly once by the first process of a Cygwin process tree. If there was no /etc/nsswitch.conf file when this first process started, then no other process in the running Cygwin process trees will try to read the file. If you create or change /etc/nsswitch.conf, make sure to stop and restart all Cygwin processes to pick up the change. So, what mischief can we perform with /etc/nsswitch.conf? To explain, lets have a look into an /etc/nsswitch.conf file set up to all default values: # /etc/nsswitch.conf passwd: files db group: files db db_prefix: auto db_separator: + db_cache: yes The first line, starting with a hash '#' is a comment. The hash sign starts a comment, just as in shell scripts. Everything up to the end of the line is ignored. So this: foo: bar # baz means, for the entry foo, do bar, ignore everything after the hash sign, so baz is only a comment. The other lines define the available settings. The first word up to a colon is a keyword. Note that the colon *must* follow immediately after the keyword. This is a valid line: foo: bar This is not valid: foo : bar Apart from this restriction, the reminder of the line can have as may spaces and TABs as you like. This is a valid line: foo: bar baz Now let's have a look at the available keywords and settings. The two lines starting with the keywords "passwd" and "group" define where Cygwin gets its passwd and group information from. "files" means, fetch the information from the corresponding file in the /etc directory. "db" means, fetch the information from the Windows account databases, the SAM for local accounts, Active Directory for domain account. Examples: passwd: files Read passwd entries only from /etc/passwd. group: db Read group entries only from SAM/AD. group: files # db Read group entries only from /etc/group ("db" is ignored due to the preceding hash sign). passwd: files db Read passwd entries from /etc/passwd. If a user account isn't found, try to find it in SAM or AD. This is the default for both, passwd and group information. group: db files This is a valid entry, but the order will be ignored by Cygwin. If both, files and db are specified, Cygwin will always try the files first, then the db. The remaining entries define certain aspects of the Windows account database search. "db_prefix" determines how the Cygwin user or group name is created: db_prefix: auto This is the default. If your account is from the primary domain of your machine, or if your machine is a standalone machine (not a domain member), your Cygwin account name is just the Windows account name. If your account is from another domain, or if you're logged in as local user on a domain machine, the Cygwin username will be the combination of Windows domainname and username, with the separator char in between: MY_DOM+username (foreign domain) MACHINE+username (local account) Builtin accounts have just the separator char prepended: +LOCAL +Users Unknown accounts on NFS or Samba shares (that is, accounts which cannot be mapped to Windows user accounts via RFC 2307) get a Cygwin account name consisting of the artificial domains "UNIX_User" or "UNIX_Group" and the uid/gid value, for instance: UNIX_User+0 (root) UNIX_Group+10 (wheel) db_prefx: primary Like "auto", but primary domain accounts will be prepended by the domainname as well. db_prefix: always All accounts, even the builtin accounts, will have the domain name prepended: BUILTIN+Users As a special case, if the Cygwin account name differs from the Windows account name, it will be prepended by the artificial domain name "Posix_User" or "Posix_Group" if db_prefix is set to "always": Posix_User+cygwin_user_name Posix_Group+cygwin_group_name "db_separator" defines the spearator char used to prepend the domain name to the user or group name. The default is '+': MY_DOM+username With "db_separator", you can define any ASCII char except space, tab, carriage return, line feed, and the colon, as separator char. Example: db_separator: \ MY_DOM\username "db_cache" allows to specify if user and group data should be cached in the process at all or not. Default is "yes". "no" is an experimental feature. The idea is this. Assuming you're working in a volatile domain environment, and you have long-running processes like sshd. You might want sshd to recognize changes in the existing user entries as soon as they occur. Caching user data in sshd *might* not be helpful in such a scenario. However, the user data which gets changed a lot is typically *not* the data required for login purposes, except for the password, and that doesn't matter in Cygwin's case. So take this feature with a grain of salt. ========== Footnotes: ========== [0] Not dragons. Just a test. [1] http://cygwin.com/cygwin-ug-net/ntsec.html [2] This may change. Right now the file is read in 32K chunks, but we could easily read the file in 64K chunks and, if we find the file is < 64K anyway, just cache the entire bunch, like before. Not implemented yet, but something to keep in mind. [3] http://msdn.microsoft.com/en-us/library/windows/desktop/aa379166%28v=vs.85%29.aspx http://msdn.microsoft.com/en-us/library/windows/desktop/aa379159%28v=vs.85%29.aspx [4] http://support.microsoft.com/kb/243330 [5] This is where Cygwin differs from SFU. The reason is that we need the old uid/gid values for backward compatibility. There are Cygwin packages (cron, for instance) who rely on the fact that the uid of SYSTEM is 18. In SFU, these accounts get mapped like the other built in SIDs. [6] https://tools.ietf.org/html/rfc2307 [7] https://tools.ietf.org/html/rfc1813 [8] http://msdn.microsoft.com/en-us/library/cc980032.aspx [9] http://linux.die.net/man/5/nsswitch.conf -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 14:44 Testers needed: New passwd/group handling in Cygwin Corinna Vinschen @ 2014-02-13 16:05 ` Steven Penny 2014-02-13 16:17 ` Corinna Vinschen 2014-02-13 16:42 ` Andrey Repin 2014-02-13 19:33 ` Andrey Repin ` (7 subsequent siblings) 8 siblings, 2 replies; 145+ messages in thread From: Steven Penny @ 2014-02-13 16:05 UTC (permalink / raw) To: cygwin On Thu, Feb 13, 2014 at 8:38 AM, Corinna Vinschen wrote: > For as long as Cygwin has existed, it has stored user and group > information in /etc/passwd and /etc/group files. Under the assumption > that these files would never be too large, the first process in a > process tree, as well as every execing process within the tree would > parse them into structures in memory. Thus every Cygwin process would > contain an expanded copy of the full information from /etc/passwd and > /etc/group. Stellar writeup! I read the whole post. I am happy to help, but I have couple of questions - How will this affect "normal" users, that is to say one admin user on one computer with no domain or networking? Will it be better to use this new system or keep /etc/passwd? - Do you have any benchmarks available? Or instructions on how we could test the speed of the new system? -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 16:05 ` Steven Penny @ 2014-02-13 16:17 ` Corinna Vinschen 2014-02-13 16:20 ` Corinna Vinschen 2014-02-13 16:42 ` Andrey Repin 1 sibling, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-13 16:17 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1650 bytes --] On Feb 13 09:48, Steven Penny wrote: > On Thu, Feb 13, 2014 at 8:38 AM, Corinna Vinschen wrote: > > For as long as Cygwin has existed, it has stored user and group > > information in /etc/passwd and /etc/group files. Under the assumption > > that these files would never be too large, the first process in a > > process tree, as well as every execing process within the tree would > > parse them into structures in memory. Thus every Cygwin process would > > contain an expanded copy of the full information from /etc/passwd and > > /etc/group. > > Stellar writeup! I read the whole post. I am happy to help, but I have couple of > questions > > - How will this affect "normal" users, that is to say one admin user on one > computer with no domain or networking? Will it be better to use this new > system or keep /etc/passwd? That should have been clear from the writeup. Just continue to use /etc/passwd and /etc/group if you're not comfortable to change your local SAM. But my mail also contains examples how to change your SAM entry from the CMD or bach command line. > - Do you have any benchmarks available? Or instructions on how we could test the > speed of the new system? Nope. Try something time-consuming you're doing every day under time(1), I guess. Building some project or so. But first get comfortable with the new output of `id' and `ls -l' in some environments. I'm all for performance, but functionality first, please. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 16:17 ` Corinna Vinschen @ 2014-02-13 16:20 ` Corinna Vinschen 0 siblings, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-13 16:20 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 3432 bytes --] On Feb 13 17:09, Corinna Vinschen wrote: > On Feb 13 09:48, Steven Penny wrote: > > On Thu, Feb 13, 2014 at 8:38 AM, Corinna Vinschen wrote: > > > For as long as Cygwin has existed, it has stored user and group > > > information in /etc/passwd and /etc/group files. Under the assumption > > > that these files would never be too large, the first process in a > > > process tree, as well as every execing process within the tree would > > > parse them into structures in memory. Thus every Cygwin process would > > > contain an expanded copy of the full information from /etc/passwd and > > > /etc/group. > > > > Stellar writeup! I read the whole post. I am happy to help, but I have couple of > > questions > > > > - How will this affect "normal" users, that is to say one admin user on one > > computer with no domain or networking? Will it be better to use this new > > system or keep /etc/passwd? > > That should have been clear from the writeup. Just continue to use > /etc/passwd and /etc/group if you're not comfortable to change your > local SAM. But my mail also contains examples how to change your SAM > entry from the CMD or bach command line. > > > - Do you have any benchmarks available? Or instructions on how we could test the > > speed of the new system? > > Nope. Try something time-consuming you're doing every day under > time(1), I guess. Building some project or so. But first get > comfortable with the new output of `id' and `ls -l' in some > environments. I'm all for performance, but functionality first, please. Btw., for completeness, here's a /etc/nsswitch.conf file with comments, with everything set to the default values. Again, note that the file is optional and only needed if you want to diverge from the defaults: ========= SNIP ========== # /etc/nsswitch.conf # # This file is read once by the first process in a Cygwin process tree. # To pick up changes, restart all Cygwin processes. # # passwd: # group: # # "files" only use /etc/passwd or /etc/group file. # "db" only use SAM/AD retrieval. # "files db" both, files preferred. This is the default. # # "db files" does not make any sense # passwd: files db group: files db # # Configuration of "db" style passwd/group handling: # # db_prefix: # # "auto" If "auto", prepend domain to account name if the account # is not a member of the machine's primary domain. Prepend # just the separator char if the account is a well-known # or builtin group. # # "primary" "primary" is like "auto", but prepend domain to account name # as well, if the account is a member of the machine's primary # domain. # # "always" If set to "always", always prepend domain, even for # well-known and builtin accounts. # db_prefix: auto # # db_cache: # # "yes" If yes, cache once retrieved DB values in local process, # hand cache down to child processes. This is the default. # # "no" If no, fetch passwd or group entries anew, every time an # entry is requested. # db_cache: yes # # db_separator: # # Set separator character between domain and account name to # the ASCII char X. Default is '+'. # db_separator: + ========= SNAP ========== Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 16:05 ` Steven Penny 2014-02-13 16:17 ` Corinna Vinschen @ 2014-02-13 16:42 ` Andrey Repin 2014-02-13 18:41 ` Lord Laraby 2014-02-14 0:31 ` Warren Young 1 sibling, 2 replies; 145+ messages in thread From: Andrey Repin @ 2014-02-13 16:42 UTC (permalink / raw) To: Steven Penny, cygwin Greetings, Steven Penny! >> For as long as Cygwin has existed, it has stored user and group >> information in /etc/passwd and /etc/group files. Under the assumption >> that these files would never be too large, the first process in a >> process tree, as well as every execing process within the tree would >> parse them into structures in memory. Thus every Cygwin process would >> contain an expanded copy of the full information from /etc/passwd and >> /etc/group. > Stellar writeup! I read the whole post. I am happy to help, but I have couple of > questions > - How will this affect "normal" users, that is to say one admin user on one > computer with no domain or networking? Will it be better to use this new > system or keep /etc/passwd? Largely doesn't matter. Would probably simplify new installs once this change is tested and deemed stable enough to be the default way of going with it. > - Do you have any benchmarks available? Or instructions on how we could test the > speed of the new system? In the end of the day, this should affect program startup times. But in home environment, the difference will be negligible. The main point of testing is consistency and general usability of a new feature, as I see it. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 13.02.2014, <20:03> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 16:42 ` Andrey Repin @ 2014-02-13 18:41 ` Lord Laraby 2014-02-13 18:49 ` Larry Hall (Cygwin) 2014-02-13 19:52 ` Andrey Repin 2014-02-14 0:31 ` Warren Young 1 sibling, 2 replies; 145+ messages in thread From: Lord Laraby @ 2014-02-13 18:41 UTC (permalink / raw) To: Andrey Repin Subject: Re: get rid of getpwent? (Was: cygwin-1.7.28 getpwent header declaration changes ?) This would be a nice mechanism for typical windows home systems except for the format of the cygwin tag that goes in the comment field. Home users that utilize 'net user USERNAME "/comment:yadda yadda"' command cannot use equal signs on the command line, and the entire string has to be quoted to prevent angle brackets from messing with redirection. Equal signs are stripped by CMD.exe and the nexted quotes are stripped as well. Example: I entered the following at elevated cmd prompt, C:\WINDOWS\system32>net user TestUser "/comment:<cygwin check="maybe" then="okay>" The command completed successfullly. C:\WINDOWS\system32>net user TestUser User name TestUser Full name Test User Comment <cygwin check=maybe then=okay> ... The command completed successfullly. So, I do not know how to get the information into the field in the proper format. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 18:41 ` Lord Laraby @ 2014-02-13 18:49 ` Larry Hall (Cygwin) 2014-02-13 19:05 ` Lord Laraby 2014-02-13 19:37 ` Corinna Vinschen 2014-02-13 19:52 ` Andrey Repin 1 sibling, 2 replies; 145+ messages in thread From: Larry Hall (Cygwin) @ 2014-02-13 18:49 UTC (permalink / raw) To: cygwin On 2/13/2014 1:33 PM, Lord Laraby wrote: > Subject: Re: get rid of getpwent? (Was: cygwin-1.7.28 getpwent header > declaration changes ?) > > This would be a nice mechanism for typical windows home systems except for > the format of the cygwin tag that goes in the comment field. Home users > that utilize 'net user USERNAME "/comment:yadda yadda"' command cannot use > equal signs on the command line, and the entire string has to be quoted to > prevent angle brackets from messing with redirection. Equal signs are > stripped by CMD.exe and the nexted quotes are stripped as well. > > Example: I entered the following at elevated cmd prompt, > > C:\WINDOWS\system32>net user TestUser "/comment:<cygwin check="maybe" > then="okay>" > The command completed successfullly. > > C:\WINDOWS\system32>net user TestUser > User name TestUser > Full name Test User > Comment <cygwin check=maybe then=okay> > ... > The command completed successfullly. > Corinna's email with a description of this entire mechanism recommends this syntax: > CMD example: > > net use corinna /comment:"<cygwin home=\"/home/foo\"/>" Does that work for you? -- Larry _____________________________________________________________________ A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting annoying in email? -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 18:49 ` Larry Hall (Cygwin) @ 2014-02-13 19:05 ` Lord Laraby 2014-02-13 19:37 ` Corinna Vinschen 1 sibling, 0 replies; 145+ messages in thread From: Lord Laraby @ 2014-02-13 19:05 UTC (permalink / raw) To: Cygwin Mailing List Yes, thank you. That form of the syntax seems to work (at least on windows 8.1). Thanks again. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 18:49 ` Larry Hall (Cygwin) 2014-02-13 19:05 ` Lord Laraby @ 2014-02-13 19:37 ` Corinna Vinschen 1 sibling, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-13 19:37 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2055 bytes --] On Feb 13 13:40, Larry Hall (Cygwin) wrote: > On 2/13/2014 1:33 PM, Lord Laraby wrote: > >Subject: Re: get rid of getpwent? (Was: cygwin-1.7.28 getpwent header > >declaration changes ?) > > > >This would be a nice mechanism for typical windows home systems except for > >the format of the cygwin tag that goes in the comment field. Home users > >that utilize 'net user USERNAME "/comment:yadda yadda"' command cannot use > >equal signs on the command line, and the entire string has to be quoted to > >prevent angle brackets from messing with redirection. Equal signs are > >stripped by CMD.exe and the nexted quotes are stripped as well. > > > >Example: I entered the following at elevated cmd prompt, > > > >C:\WINDOWS\system32>net user TestUser "/comment:<cygwin check="maybe" > >then="okay>" > >The command completed successfullly. > > > >C:\WINDOWS\system32>net user TestUser > >User name TestUser > >Full name Test User > >Comment <cygwin check=maybe then=okay> > >... > >The command completed successfullly. > > > > Corinna's email with a description of this entire mechanism recommends this > syntax: > > > CMD example: > > > > net use corinna /comment:"<cygwin home=\"/home/foo\"/>" > > Does that work for you? Sigh. I read my mail at least 6 or 7 times, but never once it occured to me that I mistyped the commands. It's `net user', of course, not `net use'. Oh holy naïveté. So again, either use net user corinna /comment:"<cygwin home=\"/home/foo\"/>" from CMD (double quotes, escaping double quotes in the string) or net user corinna /comment:'<cygwin home="/home/foo"/>' from bash or tcsh (single quotes around the string, no escaping of double quotes inside the string). You are aware that you can start a Cygwin shell elevated as well, right? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 18:41 ` Lord Laraby 2014-02-13 18:49 ` Larry Hall (Cygwin) @ 2014-02-13 19:52 ` Andrey Repin 1 sibling, 0 replies; 145+ messages in thread From: Andrey Repin @ 2014-02-13 19:52 UTC (permalink / raw) To: Lord Laraby, cygwin Greetings, Lord Laraby! > This would be a nice mechanism for typical windows home systems except for > the format of the cygwin tag that goes in the comment field. ???... > Home users that utilize 'net user USERNAME "/comment:yadda yadda"' command > cannot use equal signs on the command line, That's just not true, unless someone holding their hands away from keyboard, mouse and every other input device. > and the entire string has to be quoted to prevent angle brackets from > messing with redirection. Well, see the statement above. Simply not true. > Equal signs are stripped by CMD.exe and the nexted quotes are stripped as well. > Example: I entered the following at elevated cmd prompt, > C:\WINDOWS\system32>net user TestUser "/comment:<cygwin check="maybe" > then="okay>" But Corinna did not said to enter this. She told you to enter (in your case): net user TestUser /comment:"<cygwin check=\"maybe\" then=\"okay\"/>" Or, using Cygwin shell (console or mintty): net user TestUser /comment:'<cygwin check="maybe" then="okay"/>' > The command completed successfullly. > C:\WINDOWS\system32>net user TestUser > User name TestUser > Full name Test User > Comment <cygwin check=maybe then=okay> > ... > The command completed successfullly. Of course, it did. Computers are very stupid. They do what you tell them to do, no more, no less. > So, I do not know how to get the information into the field in the proper > format. Exactly as Corinna told you to. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 13.02.2014, <23:01> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 16:42 ` Andrey Repin 2014-02-13 18:41 ` Lord Laraby @ 2014-02-14 0:31 ` Warren Young 2014-02-14 7:36 ` Andrey Repin 2014-02-14 8:00 ` Warren Young 1 sibling, 2 replies; 145+ messages in thread From: Warren Young @ 2014-02-14 0:31 UTC (permalink / raw) To: Cygwin-L On 2/13/2014 09:07, Andrey Repin wrote: > Greetings, Steven Penny! > >> - Do you have any benchmarks available? Or instructions on how we could test the >> speed of the new system? > > In the end of the day, this should affect program startup times. But in home > environment, the difference will be negligible. Whose home are we talking about here? :) In my home, we type commands like ./configure && make && make install That sort of thing may launch and kill hundreds of programs in a short period of time. If we save only 1/20 second (say 0.1 sec down to 0.05) per process launch, we can save tens of seconds on such a command sequence. I look forward to testing this. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 0:31 ` Warren Young @ 2014-02-14 7:36 ` Andrey Repin 2014-02-14 8:00 ` Warren Young 1 sibling, 0 replies; 145+ messages in thread From: Andrey Repin @ 2014-02-14 7:36 UTC (permalink / raw) To: Warren Young, cygwin Greetings, Warren Young! > On 2/13/2014 09:07, Andrey Repin wrote: >> Greetings, Steven Penny! >> >>> - Do you have any benchmarks available? Or instructions on how we could test the >>> speed of the new system? >> >> In the end of the day, this should affect program startup times. But in home >> environment, the difference will be negligible. > Whose home are we talking about here? :) <3 > In my home, we type commands like ./configure && make && make install > That sort of thing may launch and kill hundreds of programs in a short > period of time. > If we save only 1/20 second (say 0.1 sec down to 0.05) per process > launch, we can save tens of seconds on such a command sequence. > I look forward to testing this. I'm, already. It seems, I'm ought to retract my "negligible" statement. Even simple diff'ing is way faster now. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 14.02.2014, <10:39> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 0:31 ` Warren Young 2014-02-14 7:36 ` Andrey Repin @ 2014-02-14 8:00 ` Warren Young 2014-02-14 9:47 ` Corinna Vinschen 1 sibling, 1 reply; 145+ messages in thread From: Warren Young @ 2014-02-14 8:00 UTC (permalink / raw) To: cygwin On Feb 13, 2014, at 4:11 PM, Warren Young <warren@etr-usa.com> wrote: > If we save only 1/20 second (say 0.1 sec down to 0.05) per process launch, we can save tens of seconds on such a command sequence. > > I look forward to testing this. I decided to build R, since it’s big but fairly self-contained: With a 4-processor XP VM, I can’t tell the difference between 1.7.28-2 and 1.7.29s (20140213) in terms of build time. +/- a second or two over about 4.3 CPU-minutes of build time. Configure time doesn’t change much, either. My nsswitch.conf: passwd: db group: db I’ve tried it with /etc/files present and removed. Waah. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 8:00 ` Warren Young @ 2014-02-14 9:47 ` Corinna Vinschen 0 siblings, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-14 9:47 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1137 bytes --] On Feb 14 00:36, Warren Young wrote: > On Feb 13, 2014, at 4:11 PM, Warren Young <warren@etr-usa.com> wrote: > > > If we save only 1/20 second (say 0.1 sec down to 0.05) per process launch, we can save tens of seconds on such a command sequence. > > > > I look forward to testing this. > > I decided to build R, since it’s big but fairly self-contained: > > With a 4-processor XP VM, I can’t tell the difference between 1.7.28-2 and 1.7.29s (20140213) in terms of build time. +/- a second or two over about 4.3 CPU-minutes of build time. Configure time doesn’t change much, either. > > My nsswitch.conf: > > passwd: db > group: db > > I’ve tried it with /etc/files present and removed. > > Waah. Don't you worry. This may be a bad example. Most of the time the build spends with actual compiling and stuff, so the time spent in calling getpwXXX/getgrXXX functions is probably very, very minor. At least it didn't get slower :) Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 14:44 Testers needed: New passwd/group handling in Cygwin Corinna Vinschen 2014-02-13 16:05 ` Steven Penny @ 2014-02-13 19:33 ` Andrey Repin 2014-02-13 19:50 ` Corinna Vinschen 2014-02-13 19:41 ` m0viefreak ` (6 subsequent siblings) 8 siblings, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-02-13 19:33 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! > The group attributes utilized by Cygwin are: > cn If set, will be used as Cygwin group name. sambaPrimaryGroupSID ? (Yeah, I have Samba3/LDAP domain...) > The <cygwin .../> string can start at any point in the comment, but > you have to follow the rules: > - It starts with "<cygwin " and ends with "/>". > - The "cygwin" string and the key names have to be lowercase. > - No spaces between key and "value", just the equal sign. > - The value must be placed within double quotes and it must not > contain a double quote itself. The double quotes are required > for the decimal values as well! > CMD example: > net use corinna /comment:"<cygwin home=\"/home/foo\"/>" net user corinna /comment:"<cygwin home=\"/home/foo\"/>" ---------^ > Bash example (use single quotes): > net use corinna /comment:'<cygwin home="/home/foo"/>' Same as above. (I'm actually surprised, it works, though.) > However, setting up winbindd isn't for everybody, and it fails to map > Windows accounts to already existing UNIX users or groups. Oh?... it really depends on the curvature of admin's hands. Properly setup Samba accounting transparently map users through PAM. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 13.02.2014, <21:17> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 19:33 ` Andrey Repin @ 2014-02-13 19:50 ` Corinna Vinschen 0 siblings, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-13 19:50 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2030 bytes --] On Feb 13 22:59, Andrey Repin wrote: > Greetings, Corinna Vinschen! > > > The group attributes utilized by Cygwin are: > > > cn If set, will be used as Cygwin group name. > > sambaPrimaryGroupSID Apples/Oranges. "cn" is used as the Cygwin group name for an existing Windows group. It's just a name change. I don;'t know if anybody is interested in this at all, but it was possible with /etc/group. For instance, you just hate the group name "Domain Users", so you set the cn attribute to "HiveAnts", and Cygwin will use the name "HiveAnts" as Cygwin user name. > ? > (Yeah, I have Samba3/LDAP domain...) > > > > The <cygwin .../> string can start at any point in the comment, but > > you have to follow the rules: > > > - It starts with "<cygwin " and ends with "/>". > > - The "cygwin" string and the key names have to be lowercase. > > - No spaces between key and "value", just the equal sign. > > - The value must be placed within double quotes and it must not > > contain a double quote itself. The double quotes are required > > for the decimal values as well! > > > CMD example: > > > net use corinna /comment:"<cygwin home=\"/home/foo\"/>" > > net user corinna /comment:"<cygwin home=\"/home/foo\"/>" > ---------^ Yup, shame on me. > > Bash example (use single quotes): > > > net use corinna /comment:'<cygwin home="/home/foo"/>' > > Same as above. (I'm actually surprised, it works, though.) > > > However, setting up winbindd isn't for everybody, and it fails to map > > Windows accounts to already existing UNIX users or groups. > > Oh?... it really depends on the curvature of admin's hands. > Properly setup Samba accounting transparently map users through PAM. As I wrote. It's not for everybody. I never managed it :} Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 14:44 Testers needed: New passwd/group handling in Cygwin Corinna Vinschen 2014-02-13 16:05 ` Steven Penny 2014-02-13 19:33 ` Andrey Repin @ 2014-02-13 19:41 ` m0viefreak 2014-02-13 19:56 ` Corinna Vinschen 2014-02-13 22:20 ` David Stacey 2014-02-13 23:12 ` New passwd/group handling in Cygwin - test results and observations Andrey Repin ` (5 subsequent siblings) 8 siblings, 2 replies; 145+ messages in thread From: m0viefreak @ 2014-02-13 19:41 UTC (permalink / raw) To: Cygwin Mailing List Hello, I have a a question about how this change is going to affect third-party utilities. Especially in the case once chooses to use the PAM method and get rid of any /etc/{passwd,group} files completely. There seem to be programs (mostly scripts) that make use of /etc/passwd as a file directly. One of them is for example "ssh-host-config". The shell script: - works with the file directly: ... run_service_as=$(/usr/bin/grep -Fi "${accnt_name}" /etc/passwd | /usr/bin/awk -F: '{print $1;}') ... I assume this would have to be changed in the sshd package? - After the script is run it sets various entries in the /etc/passwd file (sshd shell to /bin/false and home to /var/empty). I assume I would have to migrate these changes into the <cygwin /> comment if I were to remove the passwd file now. But more importantly, if I were to run sshd-host-config with no passwd file present at all, would it correctly set up the <cygwin /> comment entry in the PAM? Grepping through /bin I found at least one other package that makes use of /etc/passwd as a file directly (cvsbug), but since I don't have everything installed I can only assume there are more cygwin-packages and other programs someone might build from source. An idea that might sound a bit far-fetched: In case the above is actually a problem, would it be possible to populate a virtual /etc/passwd file on-the-fly the same way /dev/ and /proc/ are? This way programs that rely on reading the passwd file direclty would just keep working (writing would be more complicated of course). -- m0vie -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 19:41 ` m0viefreak @ 2014-02-13 19:56 ` Corinna Vinschen 2014-02-13 22:20 ` David Stacey 1 sibling, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-13 19:56 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2592 bytes --] On Feb 13 20:36, m0viefreak wrote: > Hello, > > I have a a question about how this change is going to affect > third-party utilities. Especially in the case once chooses to > use the PAM method and get rid of any /etc/{passwd,group} > files completely. > > There seem to be programs (mostly scripts) that make use of > /etc/passwd as a file directly. > > One of them is for example "ssh-host-config". > > The shell script: > > - works with the file directly: > ... > run_service_as=$(/usr/bin/grep -Fi "${accnt_name}" /etc/passwd | /usr/bin/awk -F: '{print $1;}') > ... > > I assume this would have to be changed in the sshd package? Indeed. There's also the csih package which needs to adapt. > - After the script is run it sets various entries in the > /etc/passwd file (sshd shell to /bin/false and home to /var/empty). > I assume I would have to migrate these changes into the <cygwin /> > comment if I were to remove the passwd file now. Yes and no. In theory, yes, but in fact the settings for the sshd account don't really matter. It's the account used for privilege separation, and the part of privilege separation which actually needs the sshd account doesn't work on Cygwin due to missing sendmsg/recvmsg descriptor passing. > But more importantly, if I were to run sshd-host-config with > no passwd file present at all, would it correctly set up the > <cygwin /> comment entry in the PAM? Uh... there is no PAM here, just local SAM or AD :) But the anwser to your question is "no" at the moment. It's probably advisable to keep the necessary entries for services in /etc/passwd for now. I'll experiment with this, too, in the next couple of days. > Grepping through /bin I found at least one other package > that makes use of /etc/passwd as a file directly (cvsbug), but > since I don't have everything installed I can only assume there > are more cygwin-packages and other programs someone might build > from source. If the package isn't very explicitely a Cygwin-only package, it has to be treated as broken since direct access to /etc/passwd and /etc/group files is a no-no. Yes, the change is a big one, and we will encounter the occasional fallout in the next weeks or months. I'm pretty confident that moving to SAM/AD is the right way to go, but I'm also sure that a a few surprises are still waiting for us. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 19:41 ` m0viefreak 2014-02-13 19:56 ` Corinna Vinschen @ 2014-02-13 22:20 ` David Stacey 2014-02-14 9:05 ` Dave Kilroy 2014-02-14 9:49 ` Corinna Vinschen 1 sibling, 2 replies; 145+ messages in thread From: David Stacey @ 2014-02-13 22:20 UTC (permalink / raw) To: cygwin On 13/02/2014 19:36, m0viefreak wrote: > Grepping through /bin I found at least one other package > that makes use of /etc/passwd as a file directly (cvsbug), but > since I don't have everything installed I can only assume there > are more cygwin-packages and other programs someone might build > from source. My complete install is proving quite useful this week: chere cron_diagnose.sh cron-config cvsbug exim exim-config mailq mysqlbug newaliases nfs-server-config rcs2log runq ssh-host-config xhere All of these reference /etc/passwd; some of the above also use /etc/group. Dave. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 22:20 ` David Stacey @ 2014-02-14 9:05 ` Dave Kilroy 2014-02-14 10:16 ` Corinna Vinschen 2014-02-14 9:49 ` Corinna Vinschen 1 sibling, 1 reply; 145+ messages in thread From: Dave Kilroy @ 2014-02-14 9:05 UTC (permalink / raw) To: cygwin On 13/02/2014 21:48, David Stacey wrote: > On 13/02/2014 19:36, m0viefreak wrote: >> Grepping through /bin I found at least one other package >> that makes use of /etc/passwd as a file directly (cvsbug), but >> since I don't have everything installed I can only assume there >> are more cygwin-packages and other programs someone might build >> from source. > > My complete install is proving quite useful this week: > > chere > xhere > > All of these reference /etc/passwd; some of the above also use > /etc/group. chere (with the -s passwd) reads etc/passwd and starts the default shell of the current user from the explorer context menu. For now, I will expect users wanting this functionality to keep using/etc/passwd. If we grow a mechanism where the login shell can be queried easily, I'll update chere to use that. Regards, Dave. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 9:05 ` Dave Kilroy @ 2014-02-14 10:16 ` Corinna Vinschen 2014-02-14 11:00 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-14 10:16 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1846 bytes --] On Feb 14 10:02, Dave Kilroy wrote: > On 13/02/2014 21:48, David Stacey wrote: > >On 13/02/2014 19:36, m0viefreak wrote: > >>Grepping through /bin I found at least one other package > >>that makes use of /etc/passwd as a file directly (cvsbug), but > >>since I don't have everything installed I can only assume there > >>are more cygwin-packages and other programs someone might build > >>from source. > > > >My complete install is proving quite useful this week: > > > > chere > > xhere > > > >All of these reference /etc/passwd; some of the above also use > >/etc/group. > chere (with the -s passwd) reads etc/passwd and starts the default > shell of the current user from the explorer context menu. For now, I > will expect users wanting this functionality to keep > using/etc/passwd. If we grow a mechanism where the login shell can > be queried easily, I'll update chere to use that. Indeed. After the discussion yesterday I already realized that we will need some commandline tool to request passwd and group entries. There is a function called getpw on Linux which allows to fetch a passwd entry in the form it has in /etc/passwd: http://linux.die.net/man/3/getpw But I can't find any tool which actually utilizes the function. There's also no group equivalent. Does anybody know an existing CLI tool with the whole purpose of requesting user or group information? In the next few weeks I will change mkpasswd/mkgroup to create passwd and group entries which are equivalent to the entires created by Cygwin itself. So, if we don't find any tool, maybe we can utilize the `mkpasswd -u' or `mkgroup -g' options at least. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 10:16 ` Corinna Vinschen @ 2014-02-14 11:00 ` Corinna Vinschen 2014-02-14 13:51 ` Kurt Franke 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-14 11:00 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1599 bytes --] On Feb 14 11:10, Corinna Vinschen wrote: > On Feb 14 10:02, Dave Kilroy wrote: > > On 13/02/2014 21:48, David Stacey wrote: > > >On 13/02/2014 19:36, m0viefreak wrote: > > >>Grepping through /bin I found at least one other package > > >>that makes use of /etc/passwd as a file directly (cvsbug), but > > >>since I don't have everything installed I can only assume there > > >>are more cygwin-packages and other programs someone might build > > >>from source. > > > > > >My complete install is proving quite useful this week: > > > > > > chere > > > xhere > > > > > >All of these reference /etc/passwd; some of the above also use > > >/etc/group. > > chere (with the -s passwd) reads etc/passwd and starts the default > > shell of the current user from the explorer context menu. For now, I > > will expect users wanting this functionality to keep > > using/etc/passwd. If we grow a mechanism where the login shell can > > be queried easily, I'll update chere to use that. > > Indeed. After the discussion yesterday I already realized that we will > need some commandline tool to request passwd and group entries. > > There is a function called getpw on Linux which allows to fetch a passwd > entry in the form it has in /etc/passwd: > > http://linux.die.net/man/3/getpw Gosh, please ignore this function. It's old and dangerous. Treat this mail as if I never mentioned this function, ok? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 11:00 ` Corinna Vinschen @ 2014-02-14 13:51 ` Kurt Franke 2014-02-14 14:05 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Kurt Franke @ 2014-02-14 13:51 UTC (permalink / raw) To: cygwin Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes: > > Indeed. After the discussion yesterday I already realized that we will > > need some commandline tool to request passwd and group entries. > > > > There is a function called getpw on Linux which allows to fetch a passwd > > entry in the form it has in /etc/passwd: > > > > http://linux.die.net/man/3/getpw > > Gosh, please ignore this function. It's old and dangerous. Treat this > mail as if I never mentioned this function, ok? > > Corinna > Hi Corinna, I think the function family getpwent() getgrent() ... would be good for this if you add checking in SAM for asked information. And of course to small executables utilizing it for use in shell scripts Regards kf -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 13:51 ` Kurt Franke @ 2014-02-14 14:05 ` Corinna Vinschen 0 siblings, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-14 14:05 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 968 bytes --] On Feb 14 12:55, Kurt Franke wrote: > Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes: > > > > Indeed. After the discussion yesterday I already realized that we will > > > need some commandline tool to request passwd and group entries. > > > > > > There is a function called getpw on Linux which allows to fetch a passwd > > > entry in the form it has in /etc/passwd: > > > > > > http://linux.die.net/man/3/getpw > > > > Gosh, please ignore this function. It's old and dangerous. Treat this > > mail as if I never mentioned this function, ok? > > > > Corinna > > > > Hi Corinna, > > I think the function family > > getpwent() > getgrent() Sure, but the problem is not to have functions, the problem is to have a tool which can be used from shell scripts. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 22:20 ` David Stacey 2014-02-14 9:05 ` Dave Kilroy @ 2014-02-14 9:49 ` Corinna Vinschen 1 sibling, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-14 9:49 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1006 bytes --] On Feb 13 21:48, David Stacey wrote: > On 13/02/2014 19:36, m0viefreak wrote: > >Grepping through /bin I found at least one other package > >that makes use of /etc/passwd as a file directly (cvsbug), but > >since I don't have everything installed I can only assume there > >are more cygwin-packages and other programs someone might build > >from source. > > My complete install is proving quite useful this week: > > chere > cron_diagnose.sh > cron-config > cvsbug > exim > exim-config > mailq > mysqlbug > newaliases > nfs-server-config > rcs2log > runq > ssh-host-config > xhere > > All of these reference /etc/passwd; some of the above also use /etc/group. Wow, thank you. It's not as bad as I feared. Most of the stuff is actually Cygwin-specific. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* New passwd/group handling in Cygwin - test results and observations 2014-02-13 14:44 Testers needed: New passwd/group handling in Cygwin Corinna Vinschen ` (2 preceding siblings ...) 2014-02-13 19:41 ` m0viefreak @ 2014-02-13 23:12 ` Andrey Repin 2014-02-14 10:42 ` Corinna Vinschen 2014-02-14 6:40 ` Testers needed: New passwd/group handling in Cygwin Warren Young ` (4 subsequent siblings) 8 siblings, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-02-13 23:12 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! > This week I applied the first incarnation of the new passwd/group > handling code to the Cygwin repository and after fixing a crash which > manifested in Denis Excoffier's network, I think we're at a point > which allows to push this forward. Oookaaaay... What to say about it?... First impression is "oh my god, did I bought a new rig?" $ uname -a CYGWIN_NT-5.1 daemon2 1.7.29s(0.271/5/3) 20140213 14:06:26 i686 Cygwin mintty startup is almost instant. Same with diff in native console. I have no idea, what you did, I don't even have a good explanation of why it happened. My /etc/passwd is about 1.5kb, half that - /etc/group. I only dumped new cygwin1.dll into place and not changed anything else. I'm going to run unattended tests just to prove I'm not dreaming. HOWEVER, here's a first bug. Actually, it was a long standing issue, that I've been discarding as nonessential, but since you are here already, can we have a bit of attention? The issue can be observed when you have a user or group name containing characters outside basic ASCII character set. Even western diacritics will suffice. Add somewhere in your startup files an equivalent of the following block: (I have it in private .profile) ---->8-------->8-------->8-------->8-------->8-------->8-------->8---- case "$TERM" in xterm*) LANG=ru_RU.UTF-8 ;; *) LANG=ru_RU.CP866 ;; esac export PATH HISTCONTROL LANG ----8<--------8<--------8<--------8<--------8<--------8<--------8<---- restart your shell, and try to ls -l a directory, where you have files owned by abovementioned user/group. Try it in mintty(the encoding will be UTF-8 and names will show up readable) and in native console (with appropriate single-byte encoding, the names will still be printed in unicode, means, raw byte sequences will be dumped to terminal). I though it could be affected by the fact I'm changing LANG on the fly, but starting bash in a console that initially have correct LANG= variable doesn't change observed results. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 14.02.2014, <01:50> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: New passwd/group handling in Cygwin - test results and observations 2014-02-13 23:12 ` New passwd/group handling in Cygwin - test results and observations Andrey Repin @ 2014-02-14 10:42 ` Corinna Vinschen 2014-02-14 12:35 ` Andrey Repin 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-14 10:42 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 3560 bytes --] On Feb 14 02:11, Andrey Repin wrote: > Greetings, Corinna Vinschen! > > > This week I applied the first incarnation of the new passwd/group > > handling code to the Cygwin repository and after fixing a crash which > > manifested in Denis Excoffier's network, I think we're at a point > > which allows to push this forward. > > Oookaaaay... What to say about it?... > First impression is "oh my god, did I bought a new rig?" > > $ uname -a > CYGWIN_NT-5.1 daemon2 1.7.29s(0.271/5/3) 20140213 14:06:26 i686 Cygwin > > mintty startup is almost instant. Same with diff in native console. > I have no idea, what you did, I don't even have a good explanation of why it > happened. My /etc/passwd is about 1.5kb, half that - /etc/group. > > I only dumped new cygwin1.dll into place and not changed anything else. > I'm going to run unattended tests just to prove I'm not dreaming. > > HOWEVER, here's a first bug. Actually, it was a long standing issue, that > I've been discarding as nonessential, but since you are here already, can we > have a bit of attention? > > The issue can be observed when you have a user or group name containing > characters outside basic ASCII character set. Even western diacritics will > suffice. > > Add somewhere in your startup files an equivalent of the following block: > (I have it in private .profile) > > ---->8-------->8-------->8-------->8-------->8-------->8-------->8---- > case "$TERM" in > xterm*) > LANG=ru_RU.UTF-8 > ;; > *) > LANG=ru_RU.CP866 > ;; > esac > > export PATH HISTCONTROL LANG > ----8<--------8<--------8<--------8<--------8<--------8<--------8<---- > > restart your shell, and try to ls -l a directory, where you have files owned > by abovementioned user/group. > > Try it in mintty(the encoding will be UTF-8 and names will show up readable) > and in native console (with appropriate single-byte encoding, the names will > still be printed in unicode, means, raw byte sequences will be dumped to > terminal). > I though it could be affected by the fact I'm changing LANG on the fly, but > starting bash in a console that initially have correct LANG= variable doesn't > change observed results. Yes, this is a problem, and I'm not sure how to fix it, if at all. The problem is hopefully obvious. We have to initialize things in some order. For instance, to read /etc/fstab.d/$USER, we need the username. And since the Cygwin username can be different from the Windows username (I guess I should have never added this functionality in the first place), we have to read the user's passwd before we read the fstabs. Same for the initialization of $LANG and friends. That occurs pretty late in the process initialization. You know that Windows uses UTF-16 under the hood, so a lot of stuff gets read and converted to UTF-8 before we even care for the environment. And if you set the codeset in the application only, all the relevant information has already been read long ago, of course. But this is a problem not different from Linux. If you have a username with non-ASCII chars, it will use *some* encoding in the passwd DB, usually UTF-8 these days. If you then change the codeset in your application, you will still get your username in UTF-8. It won't be changed on the fly, just because your application calls setlocale. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: New passwd/group handling in Cygwin - test results and observations 2014-02-14 10:42 ` Corinna Vinschen @ 2014-02-14 12:35 ` Andrey Repin 2014-02-14 14:13 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-02-14 12:35 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! >> The issue can be observed when you have a user or group name containing >> characters outside basic ASCII character set. Even western diacritics will >> suffice. >> >> Add somewhere in your startup files an equivalent of the following block: >> (I have it in private .profile) >> >> ---->8-------->8-------->8-------->8-------->8-------->8-------->8---- >> case "$TERM" in >> xterm*) >> LANG=ru_RU.UTF-8 >> ;; >> *) >> LANG=ru_RU.CP866 >> ;; >> esac >> >> export PATH HISTCONTROL LANG >> ----8<--------8<--------8<--------8<--------8<--------8<--------8<---- >> >> restart your shell, and try to ls -l a directory, where you have files owned >> by abovementioned user/group. >> >> Try it in mintty(the encoding will be UTF-8 and names will show up readable) >> and in native console (with appropriate single-byte encoding, the names will >> still be printed in unicode, means, raw byte sequences will be dumped to >> terminal). >> I though it could be affected by the fact I'm changing LANG on the fly, but >> starting bash in a console that initially have correct LANG= variable doesn't >> change observed results. > Yes, this is a problem, and I'm not sure how to fix it, if at all. > The problem is hopefully obvious. We have to initialize things in some > order. For instance, to read /etc/fstab.d/$USER, we need the username. > And since the Cygwin username can be different from the Windows username > (I guess I should have never added this functionality in the first > place), I feel your pain... > we have to read the user's passwd before we read the fstabs. > Same for the initialization of $LANG and friends. That occurs pretty > late in the process initialization. You know that Windows uses UTF-16 > under the hood, so a lot of stuff gets read and converted to UTF-8 > before we even care for the environment. And if you set the codeset in > the application only, all the relevant information has already been read > long ago, of course. > But this is a problem not different from Linux. If you have a username > with non-ASCII chars, it will use *some* encoding in the passwd DB, > usually UTF-8 these days. If you then change the codeset in your > application, you will still get your username in UTF-8. It won't be > changed on the fly, just because your application calls setlocale. I understand it (mostly), but there's actually two issues, not one. One issue is the display part, where names are output for user consumption. Another can be observed in, i.e., rsync, and file access in general (remember the discussion about accessing long directory names in unicode). Changing LANG variable DO matter for the latter, and you may only hope that whatever is output in the former case is actually printable (thank God, most of the time it actually is, in case of UTF-8). It is getting even more complicated, when you consider the fact, that in Windows you have 2 different single-byte encodings, so-called ANSI (for GUI applications) and OEM (for console). And alot of stuff making assumptions without consulting with current status of things. As convoluted the problem is, I think, we need some sort of solution, or at the very least - documentation. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 14.02.2014, <15:15> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: New passwd/group handling in Cygwin - test results and observations 2014-02-14 12:35 ` Andrey Repin @ 2014-02-14 14:13 ` Corinna Vinschen 2014-02-16 14:34 ` Andrey Repin 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-14 14:13 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1553 bytes --] On Feb 14 15:56, Andrey Repin wrote: > > But this is a problem not different from Linux. If you have a username > > with non-ASCII chars, it will use *some* encoding in the passwd DB, > > usually UTF-8 these days. If you then change the codeset in your > > application, you will still get your username in UTF-8. It won't be > > changed on the fly, just because your application calls setlocale. > > I understand it (mostly), but there's actually two issues, not one. > One issue is the display part, where names are output for user consumption. > Another can be observed in, i.e., rsync, and file access in general (remember > the discussion about accessing long directory names in unicode). > Changing LANG variable DO matter for the latter, and you may only hope that > whatever is output in the former case is actually printable (thank God, most > of the time it actually is, in case of UTF-8). > It is getting even more complicated, when you consider the fact, that in > Windows you have 2 different single-byte encodings, so-called ANSI (for GUI > applications) and OEM (for console). And alot of stuff making assumptions > without consulting with current status of things. > As convoluted the problem is, I think, we need some sort of solution, or at > the very least - documentation. Sorry, I can't provide an easy solution, and afaik this is documented. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: New passwd/group handling in Cygwin - test results and observations 2014-02-14 14:13 ` Corinna Vinschen @ 2014-02-16 14:34 ` Andrey Repin 2014-02-16 14:48 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-02-16 14:34 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! >> > But this is a problem not different from Linux. If you have a username >> > with non-ASCII chars, it will use *some* encoding in the passwd DB, >> > usually UTF-8 these days. If you then change the codeset in your >> > application, you will still get your username in UTF-8. It won't be >> > changed on the fly, just because your application calls setlocale. >> >> I understand it (mostly), but there's actually two issues, not one. >> One issue is the display part, where names are output for user consumption. >> Another can be observed in, i.e., rsync, and file access in general (remember >> the discussion about accessing long directory names in unicode). >> Changing LANG variable DO matter for the latter, and you may only hope that >> whatever is output in the former case is actually printable (thank God, most >> of the time it actually is, in case of UTF-8). >> It is getting even more complicated, when you consider the fact, that in >> Windows you have 2 different single-byte encodings, so-called ANSI (for GUI >> applications) and OEM (for console). And alot of stuff making assumptions >> without consulting with current status of things. >> As convoluted the problem is, I think, we need some sort of solution, or at >> the very least - documentation. > Sorry, I can't provide an easy solution, and afaik this is documented. I don't know, how it did not hit my head earlier, but now that I think about it... The fact `ls' does not print correct group names, but correctly translate dates... eww, wait. There's something bad happens under the hood. Please check this image: http://img513.imageshack.us/img513/43/jn7n.png LANG is set (pre-set, before Cygwin tree start) to ru_RU.UTF-8 for mintty and ru_RU.CP866 for console. As you can see, existing group (i.e. - "Administrators") is properly translated to chosen locale, which is not the case for group "Отсутствует". The SID of the group "DAEMON2\Отсутствует" is S-1-5-21-1801674531-1644491937-1606980848-513 Cygwin show it as -rwxr-xr-x 1 197612 197121 45597 мар 22 2013 xargs.exe which matching your description for "Accounts from the local machine's user DB (SAM)" (0x30000+513 =197121) (and yes, the leading part of the SID is matching the one of the other users on this system), but the group is missing from `wmic GROUP LIST' output. I presume, it was deleted at one point, or it is remnants of the previous OS installation, transferred over with the disk to this system. Either case explains the name of the group - "Missing" or "Nonexistent". Means, it's name does not exist in current SAM database. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 14.02.2014, <21:19> Sorry for my terrible english... ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: New passwd/group handling in Cygwin - test results and observations 2014-02-16 14:34 ` Andrey Repin @ 2014-02-16 14:48 ` Corinna Vinschen 2014-02-16 15:44 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-16 14:48 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 3602 bytes --] On Feb 16 17:52, Andrey Repin wrote: > Greetings, Corinna Vinschen! > > >> > But this is a problem not different from Linux. If you have a username > >> > with non-ASCII chars, it will use *some* encoding in the passwd DB, > >> > usually UTF-8 these days. If you then change the codeset in your > >> > application, you will still get your username in UTF-8. It won't be > >> > changed on the fly, just because your application calls setlocale. > >> > >> I understand it (mostly), but there's actually two issues, not one. > >> One issue is the display part, where names are output for user consumption. > >> Another can be observed in, i.e., rsync, and file access in general (remember > >> the discussion about accessing long directory names in unicode). > >> Changing LANG variable DO matter for the latter, and you may only hope that > >> whatever is output in the former case is actually printable (thank God, most > >> of the time it actually is, in case of UTF-8). > >> It is getting even more complicated, when you consider the fact, that in > >> Windows you have 2 different single-byte encodings, so-called ANSI (for GUI > >> applications) and OEM (for console). And alot of stuff making assumptions > >> without consulting with current status of things. > >> As convoluted the problem is, I think, we need some sort of solution, or at > >> the very least - documentation. > > > Sorry, I can't provide an easy solution, and afaik this is documented. > > I don't know, how it did not hit my head earlier, but now that I think about > it... > The fact `ls' does not print correct group names, but correctly translate > dates... eww, wait. There's something bad happens under the hood. > > Please check this image: http://img513.imageshack.us/img513/43/jn7n.png > LANG is set (pre-set, before Cygwin tree start) to ru_RU.UTF-8 for mintty and > ru_RU.CP866 for console. > As you can see, existing group (i.e. - "Administrators") is properly > translated to chosen locale, which is not the case for group "Отсутствует". > > The SID of the group "DAEMON2\Отсутствует" is S-1-5-21-1801674531-1644491937-1606980848-513 > > Cygwin show it as > -rwxr-xr-x 1 197612 197121 45597 мар 22 2013 xargs.exe > > which matching your description for "Accounts from the local machine's user DB > (SAM)" (0x30000+513 =197121) (and yes, the leading part of the SID is matching > the one of the other users on this system), but the group is missing from > `wmic GROUP LIST' output. I presume, it was deleted at one point, or it is > remnants of the previous OS installation, transferred over with the disk to > this system. Either case explains the name of the group - "Missing" or > "Nonexistent". Means, it's name does not exist in current SAM database. No, it's the local group with the RID 513, which exists on *all* machines, but is invisible, unless you call NetUserEnum, which enumerates it, or you call NetUserGetInfo with the localized name. This group is called "None" in the english OS versions, "Kein" in the german version, "Aucun" in the french version, etc. All local users accounts have this group invariably set as their primary group. For some reason Microsoft decided ages ago, that this group does not show up in the GUI, and that it's not possible to change the primary group outside of domains. Why that is, I have no idea. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: New passwd/group handling in Cygwin - test results and observations 2014-02-16 14:48 ` Corinna Vinschen @ 2014-02-16 15:44 ` Corinna Vinschen 2014-02-16 15:52 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-16 15:44 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 3338 bytes --] On Feb 16 15:34, Corinna Vinschen wrote: > On Feb 16 17:52, Andrey Repin wrote: > > > Sorry, I can't provide an easy solution, and afaik this is documented. > > > > I don't know, how it did not hit my head earlier, but now that I think about > > it... > > The fact `ls' does not print correct group names, but correctly translate > > dates... eww, wait. There's something bad happens under the hood. > > > > Please check this image: http://img513.imageshack.us/img513/43/jn7n.png > > LANG is set (pre-set, before Cygwin tree start) to ru_RU.UTF-8 for mintty and > > ru_RU.CP866 for console. > > As you can see, existing group (i.e. - "Administrators") is properly > > translated to chosen locale, which is not the case for group "Отсутствует". > > > > The SID of the group "DAEMON2\Отсутствует" is S-1-5-21-1801674531-1644491937-1606980848-513 > > > > Cygwin show it as > > -rwxr-xr-x 1 197612 197121 45597 мар 22 2013 xargs.exe > > > > which matching your description for "Accounts from the local machine's user DB > > (SAM)" (0x30000+513 =197121) (and yes, the leading part of the SID is matching > > the one of the other users on this system), but the group is missing from > > `wmic GROUP LIST' output. I presume, it was deleted at one point, or it is > > remnants of the previous OS installation, transferred over with the disk to > > this system. Either case explains the name of the group - "Missing" or > > "Nonexistent". Means, it's name does not exist in current SAM database. > > No, it's the local group with the RID 513, which exists on *all* > machines, but is invisible, unless you call NetUserEnum, which > enumerates it, or you call NetUserGetInfo with the localized name. > > This group is called "None" in the english OS versions, "Kein" in > the german version, "Aucun" in the french version, etc. All local > users accounts have this group invariably set as their primary group. > > For some reason Microsoft decided ages ago, that this group does > not show up in the GUI, and that it's not possible to change the primary > group outside of domains. Why that is, I have no idea. Maybe I have a simple solution. From your other mails I gather you suceeded building your own Cygwin DLL. If you apply this patch: Index: dcrt0.cc =================================================================== RCS file: /cvs/src/src/winsup/cygwin/dcrt0.cc,v retrieving revision 1.453 diff -u -p -r1.453 dcrt0.cc --- dcrt0.cc 10 Feb 2014 10:45:50 -0000 1.453 +++ dcrt0.cc 16 Feb 2014 15:15:44 -0000 @@ -929,14 +929,14 @@ dll_crt0_1 (void *) /* Allocate cygheap->fdtab */ dtable_init (); + /* Set internal locale to the environment settings. */ + initial_setlocale (); + uinfo_init (); /* initialize user info */ /* Connect to tty. */ tty::init_session (); - /* Set internal locale to the environment settings. */ - initial_setlocale (); - if (!__argc) { PWCHAR wline = GetCommandLineW (); and rebuild your Cygwin DLL, does it make a difference? In theory it should work as expected now even with ru_RU.CP866. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: New passwd/group handling in Cygwin - test results and observations 2014-02-16 15:44 ` Corinna Vinschen @ 2014-02-16 15:52 ` Corinna Vinschen 2014-02-16 16:35 ` Andrey Repin 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-16 15:52 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1577 bytes --] On Feb 16 16:16, Corinna Vinschen wrote: > On Feb 16 15:34, Corinna Vinschen wrote: > > On Feb 16 17:52, Andrey Repin wrote: > > > > Sorry, I can't provide an easy solution, and afaik this is documented. > > > [...] > Maybe I have a simple solution. From your other mails I gather you > suceeded building your own Cygwin DLL. > > If you apply this patch: > > Index: dcrt0.cc > =================================================================== > RCS file: /cvs/src/src/winsup/cygwin/dcrt0.cc,v > retrieving revision 1.453 > diff -u -p -r1.453 dcrt0.cc > --- dcrt0.cc 10 Feb 2014 10:45:50 -0000 1.453 > +++ dcrt0.cc 16 Feb 2014 15:15:44 -0000 > @@ -929,14 +929,14 @@ dll_crt0_1 (void *) > /* Allocate cygheap->fdtab */ > dtable_init (); > > + /* Set internal locale to the environment settings. */ > + initial_setlocale (); > + > uinfo_init (); /* initialize user info */ > > /* Connect to tty. */ > tty::init_session (); > > - /* Set internal locale to the environment settings. */ > - initial_setlocale (); > - > if (!__argc) > { > PWCHAR wline = GetCommandLineW (); > > and rebuild your Cygwin DLL, does it make a difference? In theory > it should work as expected now even with ru_RU.CP866. It works for me when using german umlauts with LANG=de_DE.cp1252, while it didn't work before, so I applied this path to CVS. HTH, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: New passwd/group handling in Cygwin - test results and observations 2014-02-16 15:52 ` Corinna Vinschen @ 2014-02-16 16:35 ` Andrey Repin 0 siblings, 0 replies; 145+ messages in thread From: Andrey Repin @ 2014-02-16 16:35 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! >> > > > Sorry, I can't provide an easy solution, and afaik this is documented. >> > > [...] >> Maybe I have a simple solution. From your other mails I gather you >> suceeded building your own Cygwin DLL. I wasn't building cygwin1.dll specifically, but it appears it was actually doing so as a by-product... :) >> >> If you apply this patch: >> >> Index: dcrt0.cc >> =================================================================== >> RCS file: /cvs/src/src/winsup/cygwin/dcrt0.cc,v >> retrieving revision 1.453 >> diff -u -p -r1.453 dcrt0.cc >> --- dcrt0.cc 10 Feb 2014 10:45:50 -0000 1.453 >> +++ dcrt0.cc 16 Feb 2014 15:15:44 -0000 >> @@ -929,14 +929,14 @@ dll_crt0_1 (void *) >> /* Allocate cygheap->fdtab */ >> dtable_init (); >> >> + /* Set internal locale to the environment settings. */ >> + initial_setlocale (); >> + >> uinfo_init (); /* initialize user info */ >> >> /* Connect to tty. */ >> tty::init_session (); >> >> - /* Set internal locale to the environment settings. */ >> - initial_setlocale (); >> - >> if (!__argc) >> { >> PWCHAR wline = GetCommandLineW (); >> >> and rebuild your Cygwin DLL, does it make a difference? In theory >> it should work as expected now even with ru_RU.CP866. > It works for me when using german umlauts with LANG=de_DE.cp1252, while > it didn't work before, so I applied this path to CVS. IC. Picked the changes from CVS and rebuilding it right now. ... Done and installed new library. All seems to be in order. Thanks for support! And patch makes some sense to my untrained eye, too. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 16.02.2014, <20:11> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 14:44 Testers needed: New passwd/group handling in Cygwin Corinna Vinschen ` (3 preceding siblings ...) 2014-02-13 23:12 ` New passwd/group handling in Cygwin - test results and observations Andrey Repin @ 2014-02-14 6:40 ` Warren Young 2014-02-14 10:48 ` Corinna Vinschen 2014-02-14 10:20 ` Frank Fesevur ` (3 subsequent siblings) 8 siblings, 1 reply; 145+ messages in thread From: Warren Young @ 2014-02-14 6:40 UTC (permalink / raw) To: cygwin On 2/13/2014 07:38, Corinna Vinschen wrote: > > The new > mechanism will never read the entire file into memory, but only scan > for the requested entry and cache this one in memory[2]. Does this feature avoid the stale cache problem? For instance, do you check the mtime on /etc/{passwd,group} before checking the cache, then dump the whole cache if the file was changed since the last file scan? > Apart from power shell scripting or inventing new CLI tools, these > attributes can be changed using the "Attribute Editor" tab in the user > properties dialog of the "Active Directory Users and Computers" > MMC snap-in. A week ago, we were talking about possible Cygwin {user,group}{add,mod} programs, modeled on Linux's. Was that simply shelved once "net user" and MMC were found to be sufficient? If such programs existed, they could abstract away the difference between /etc/passwd, SAM and AD. Plus, net.exe is a hairball. If, magically, such programs were to appear from outside the Cygwin core dev group, would that be a good thing or a bad thing? > unix="value" Sets the NFS/Samba uid of the user to the decimal > value. See the next chapter. I know I'm bikeshedding, but "unix" seems like a pretty vague attribute name here. unix="good" ? unix="linux" ? unix="yesplease" ? I'd be happier with "uid" or similar. Write it in a sentence: a. My uid is 502; vs b. My unix is 502. > If you create or change /etc/nsswitch.conf, > make sure to stop and restart all Cygwin processes to pick up > the change. "All" processes? If I have cron running, then exit the one instance of MinTTY after vim'ing /etc/nsswitch.conf, will the file be read when I re-open MinTTY? cron.exe is running in a different process group, isn't it? If true, I realize cron.exe and any programs it runs will continue to believe there is no /etc/nsswitch.conf until *it* restarts. > the order [in nsswitch.conf] will be ignored by Cygwin. Hmmm...different from Linux. The inability to say "db files" as distinct from "files db" means you can't set up a SAM-only machine with SAM as a fast primary source of truth and "files" as a fallback. That seems like a sensible configuration to me, since SAM should always be more trustworthy than /etc/passwd. Personally, I'm going to try to get away with "db" only, but for systems that really require "files", I'd like the choice to make it secondary to "db". -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 6:40 ` Testers needed: New passwd/group handling in Cygwin Warren Young @ 2014-02-14 10:48 ` Corinna Vinschen 2014-02-14 12:56 ` Andrey Repin 2014-02-14 20:18 ` Warren Young 0 siblings, 2 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-14 10:48 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 3817 bytes --] On Feb 13 17:30, Warren Young wrote: > On 2/13/2014 07:38, Corinna Vinschen wrote: > > > >The new > >mechanism will never read the entire file into memory, but only scan > >for the requested entry and cache this one in memory[2]. > > Does this feature avoid the stale cache problem? > > For instance, do you check the mtime on /etc/{passwd,group} before > checking the cache, then dump the whole cache if the file was > changed since the last file scan? Yes. > > Apart from power shell scripting or inventing new CLI tools, these > > attributes can be changed using the "Attribute Editor" tab in the user > > properties dialog of the "Active Directory Users and Computers" > > MMC snap-in. > > A week ago, we were talking about possible Cygwin > {user,group}{add,mod} programs, modeled on Linux's. Was that simply > shelved once "net user" and MMC were found to be sufficient? Huh? "Apart from [...] or inventing new CLI tools, [...]" ^^^^^^^^^^^^^^^^^^^^^^^^^^ > If such programs existed, they could abstract away the difference > between /etc/passwd, SAM and AD. Plus, net.exe is a hairball. Well, "net user" works. New tools still have to catch up... > If, magically, such programs were to appear from outside the Cygwin > core dev group, would that be a good thing or a bad thing? It would be a really great thing! > > unix="value" Sets the NFS/Samba uid of the user to the decimal > > value. See the next chapter. > > I know I'm bikeshedding, but "unix" seems like a pretty vague > attribute name here. > > unix="good" ? > unix="linux" ? > unix="yesplease" ? > > I'd be happier with "uid" or similar. > > Write it in a sentence: > > a. My uid is 502; vs > b. My unix is 502. Would you believe me that I sweated about this, too? The important thing to consider is, the keyword should not give the impression, that the uid or gid set at this point is the Cygwin uid or gid. It's only the uid/gid of your NFS or Samba account on that weird Linux box. Really, I'm open to suggestions to have a better keyword, but it should make very clear that this is not your Cygwin uid/gid. > > If you create or change /etc/nsswitch.conf, > > make sure to stop and restart all Cygwin processes to pick up > > the change. > > "All" processes? > > If I have cron running, then exit the one instance of MinTTY after > vim'ing /etc/nsswitch.conf, will the file be read when I re-open > MinTTY? > > cron.exe is running in a different process group, isn't it? > > If true, I realize cron.exe and any programs it runs will continue > to believe there is no /etc/nsswitch.conf until *it* restarts. You are absolutely right, but, please, suggest a better wording. This is what I'm trying to explain in fact, but everything I came up with sounded like beating around the bush, more confusing than helping. > >the order [in nsswitch.conf] will be ignored by Cygwin. > > Hmmm...different from Linux. Yes, that's how it's implemented. It's just a bitmask and the order inside Cygwin is fixed. > The inability to say "db files" as distinct from "files db" means > you can't set up a SAM-only machine with SAM as a fast primary > source of truth and "files" as a fallback. What for? > That seems like a sensible configuration to me, since SAM should > always be more trustworthy than /etc/passwd. Again, what for? What entry would you find in passwd which you didn't already find in SAM or via the implemented automatisms for unknown SIDs? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 10:48 ` Corinna Vinschen @ 2014-02-14 12:56 ` Andrey Repin 2014-02-14 14:15 ` Corinna Vinschen 2014-02-14 20:18 ` Warren Young 1 sibling, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-02-14 12:56 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! >> > unix="value" Sets the NFS/Samba uid of the user to the decimal >> > value. See the next chapter. >> >> I know I'm bikeshedding, but "unix" seems like a pretty vague >> attribute name here. >> >> unix="good" ? >> unix="linux" ? >> unix="yesplease" ? >> >> I'd be happier with "uid" or similar. >> >> Write it in a sentence: >> >> a. My uid is 502; vs >> b. My unix is 502. > Would you believe me that I sweated about this, too? The important > thing to consider is, the keyword should not give the impression, that > the uid or gid set at this point is the Cygwin uid or gid. It's only > the uid/gid of your NFS or Samba account on that weird Linux box. > Really, I'm open to suggestions to have a better keyword, but it > should make very clear that this is not your Cygwin uid/gid. nfsUid/-Gid Since it is almost exclusively used to talk to NFS servers. >> > If you create or change /etc/nsswitch.conf, >> > make sure to stop and restart all Cygwin processes to pick up >> > the change. >> >> "All" processes? >> >> If I have cron running, then exit the one instance of MinTTY after >> vim'ing /etc/nsswitch.conf, will the file be read when I re-open >> MinTTY? >> >> cron.exe is running in a different process group, isn't it? >> >> If true, I realize cron.exe and any programs it runs will continue >> to believe there is no /etc/nsswitch.conf until *it* restarts. > You are absolutely right, but, please, suggest a better wording. > This is what I'm trying to explain in fact, but everything I came > up with sounded like beating around the bush, more confusing than > helping. "...each relevant Cygwin process tree needs to be restarted to catch up"? Something along these lines. Since these tokens maintained on per-tree basis, inherited from the very first Cygwin process started. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 14.02.2014, <16:19> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 12:56 ` Andrey Repin @ 2014-02-14 14:15 ` Corinna Vinschen 0 siblings, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-14 14:15 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2453 bytes --] On Feb 14 16:23, Andrey Repin wrote: > Greetings, Corinna Vinschen! > > >> > unix="value" Sets the NFS/Samba uid of the user to the decimal > >> > value. See the next chapter. > >> > >> I know I'm bikeshedding, but "unix" seems like a pretty vague > >> attribute name here. > >> > >> unix="good" ? > >> unix="linux" ? > >> unix="yesplease" ? > >> > >> I'd be happier with "uid" or similar. > >> > >> Write it in a sentence: > >> > >> a. My uid is 502; vs > >> b. My unix is 502. > > > Would you believe me that I sweated about this, too? The important > > thing to consider is, the keyword should not give the impression, that > > the uid or gid set at this point is the Cygwin uid or gid. It's only > > the uid/gid of your NFS or Samba account on that weird Linux box. > > > Really, I'm open to suggestions to have a better keyword, but it > > should make very clear that this is not your Cygwin uid/gid. > > nfsUid/-Gid > Since it is almost exclusively used to talk to NFS servers. Except when it's used for Samba shares, which is more likely than NFS, given that the NFS client is restricted to the Enterprise version of Windows since Windows 8. I asked twice to change that *at least* to the professional version, but Microsoft simply doesn't care, despite NFS being much faster than SMB. Grrrrrrrrrrrr. I would prefer a keyword in all lowercase, btw. I would also prefer a keyword without underscore. And I would prefer a keyword which can be used for user and group entries because that simplifies the code. And I would prefer golden or platinum framed letters... nfsuid/nfsgid nfsid unixuid/unixgid unixid unix_id [...to be continued...] > >> If true, I realize cron.exe and any programs it runs will continue > >> to believe there is no /etc/nsswitch.conf until *it* restarts. > > > You are absolutely right, but, please, suggest a better wording. > > This is what I'm trying to explain in fact, but everything I came > > up with sounded like beating around the bush, more confusing than > > helping. > > "...each relevant Cygwin process tree needs to be restarted to catch up"? > Something along these lines. Yes, something along these lines. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 10:48 ` Corinna Vinschen 2014-02-14 12:56 ` Andrey Repin @ 2014-02-14 20:18 ` Warren Young 2014-02-14 21:49 ` Warren Young 2014-02-15 12:52 ` Corinna Vinschen 1 sibling, 2 replies; 145+ messages in thread From: Warren Young @ 2014-02-14 20:18 UTC (permalink / raw) To: cygwin On 2/14/2014 03:42, Corinna Vinschen wrote: > On Feb 13 17:30, Warren Young wrote: >> On 2/13/2014 07:38, Corinna Vinschen wrote: > >>> Apart from power shell scripting or inventing new CLI tools, these >>> attributes can be changed using the "Attribute Editor" tab in the user >>> properties dialog of the "Active Directory Users and Computers" >>> MMC snap-in. >> >> A week ago, we were talking about possible Cygwin >> {user,group}{add,mod} programs, modeled on Linux's. Was that simply >> shelved once "net user" and MMC were found to be sufficient? > > Huh? "Apart from [...] or inventing new CLI tools, [...]" > ^^^^^^^^^^^^^^^^^^^^^^^^^^ I wasn't sure how to interpret that. It could be read as an unfulfilled possibility or as dismissing a ridiculous idea. i.e. "Apart from rewriting Cygwin in Erlang..." >> If, magically, such programs were to appear from outside the Cygwin >> core dev group, would that be a good thing or a bad thing? > > It would be a really great thing! Okay. I thought you might feel proprietary about such tools. "I know how it needs to be written, so I'm going to be the one to write it, right after the other 59 bazillion things on my wishlist." >> I know I'm bikeshedding, but "unix" seems like a pretty vague >> attribute name here. > > Really, I'm open to suggestions to have a better keyword, but it > should make very clear that this is not your Cygwin uid/gid. Okay, netfsuid, then. >> "All" processes? > > You are absolutely right, but, please, suggest a better wording. "If you create or change /etc/nsswitch.conf, you need to restart all Cygwin processes that need to see the change. If the process you want to see the change is a child of another process, you need to restart all of that process's parents, too." "For example, if you run Vim inside the default Cygwin Terminal, Vim is a child of your shell, which is a child of mintty.exe. If you edit /etc/nsswitch.conf in that Vim instance, your shell won't immediately see the change, nor will Vim if you restart it from that same shell instance. This is because both are getting their nsswitch information from their ancestor, mintty.exe. You need to start a fresh terminal window for the change to take effect." "By contrast, if you leave that Cygwin Terminal window open after making the change to /etc/nsswitch.conf, then restart a Cygwin service like cron, cron will see the change, because it is not a child of mintty.exe or any other Cygwin process. (Technically, it is a child of cygrunsrv, but that instance also restarts when you restart the service.)" "The reason we point all this out is that the requirements for restarting things are not quite as stringent as when you replace cygwin1.dll. If you have three process trees, you have three independent copies of the nsswitch information. You do not need to restart any of them for a new process tree to see the changes, but until each restarts, they will not see the change." Better? You could stop with the first paragraph, but I think the following exposition helps. > What entry would you find in passwd which you > didn't already find in SAM or via the implemented automatisms > for unknown SIDs? That makes sense. Is nsswitch.conf the right thing, then? Are we borrowing that mechanism just because it exists and looks close enough? It seems to me that we really only need a single Boolean setting: ignore_db=true If this is true, it uses files only. If false, DB is the sole source of truth if /etc/{passwd,group} are missing, or it is a fallback source of truth if those files are present. Does this help us get to a world where we configure this in nscd.conf, as cgf proposed? -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 20:18 ` Warren Young @ 2014-02-14 21:49 ` Warren Young 2014-02-15 12:52 ` Corinna Vinschen 1 sibling, 0 replies; 145+ messages in thread From: Warren Young @ 2014-02-14 21:49 UTC (permalink / raw) To: cygwin On 2/14/2014 13:11, Warren Young wrote: > "The reason we point all this out is that the requirements for > restarting things are not quite as stringent as when you replace > cygwin1.dll. If you have three process trees, you have three independent > copies of the nsswitch information. You do not need to restart any of > them for a new process tree to see the changes, but until each restarts, > they will not see the change." EDIT, replace last sentence: "If you start a fresh process tree, it will see the changes. As long as any process in an existing process tree remains running, all processes in that tree will continue to use the old information." -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 20:18 ` Warren Young 2014-02-14 21:49 ` Warren Young @ 2014-02-15 12:52 ` Corinna Vinschen 2014-02-16 11:48 ` Warren Young 1 sibling, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-15 12:52 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 3765 bytes --] On Feb 14 13:11, Warren Young wrote: > On 2/14/2014 03:42, Corinna Vinschen wrote: > >On Feb 13 17:30, Warren Young wrote: > >>On 2/13/2014 07:38, Corinna Vinschen wrote: > > > >>> Apart from power shell scripting or inventing new CLI tools, these > >>> attributes can be changed using the "Attribute Editor" tab in the user > >>> properties dialog of the "Active Directory Users and Computers" > >>> MMC snap-in. > >> > >>A week ago, we were talking about possible Cygwin > >>{user,group}{add,mod} programs, modeled on Linux's. Was that simply > >>shelved once "net user" and MMC were found to be sufficient? > > > >Huh? "Apart from [...] or inventing new CLI tools, [...]" > > ^^^^^^^^^^^^^^^^^^^^^^^^^^ > > I wasn't sure how to interpret that. It could be read as an > unfulfilled possibility or as dismissing a ridiculous idea. i.e. > "Apart from rewriting Cygwin in Erlang..." Nah, I'm not *always* cynical. > >>If, magically, such programs were to appear from outside the Cygwin > >>core dev group, would that be a good thing or a bad thing? > > > >It would be a really great thing! > > Okay. I thought you might feel proprietary about such tools. "I > know how it needs to be written, so I'm going to be the one to write > it, right after the other 59 bazillion things on my wishlist." Actually, high on my wishlist is more active maintainers for the distro. We have a couple of maintainers of very important packages which only show up very sporadically lately, which is pretty frustrating. > >>I know I'm bikeshedding, but "unix" seems like a pretty vague > >>attribute name here. > > > >Really, I'm open to suggestions to have a better keyword, but it > >should make very clear that this is not your Cygwin uid/gid. > > Okay, netfsuid, then. Hmm. > >>"All" processes? > > > >You are absolutely right, but, please, suggest a better wording. > > "If you create or change /etc/nsswitch.conf, you need to restart all > Cygwin processes that need to see the change. If the process you > want to see the change is a child of another process, you need to > restart all of that process's parents, too." > [...] > Better? Yes. Thanks a lot. I grabbed all of this including your followup change shamelessly and added it to the text. > >What entry would you find in passwd which you > >didn't already find in SAM or via the implemented automatisms > >for unknown SIDs? > > That makes sense. > > Is nsswitch.conf the right thing, then? Are we borrowing that > mechanism just because it exists and looks close enough? Perhaps. > It seems to me that we really only need a single Boolean setting: > > ignore_db=true No, that's not right. We have two mechanisms implemented you can choose three out of four possible combinations: files only db only files, then db > If this is true, it uses files only. If false, DB is the sole > source of truth if /etc/{passwd,group} are missing, or it is a > fallback source of truth if those files are present. The third combination is to prevent Cygwin from reading /etc/passwd and /etc/group at all. It drops any check for existence, too, which is one code point less which has to run for each getpwXXX/getgrXXX invocation. > Does this help us get to a world where we configure this in > nscd.conf, as cgf proposed? I'm open to discuss this. We can switch from nssswitch.conf to nscd.conf, but our settings will still not match the role-model, so it's kind of a name-reuse only, either way. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-15 12:52 ` Corinna Vinschen @ 2014-02-16 11:48 ` Warren Young 2014-02-16 12:00 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Warren Young @ 2014-02-16 11:48 UTC (permalink / raw) To: cygwin On Feb 15, 2014, at 5:50 AM, Corinna Vinschen wrote: > I grabbed all of this including your followup > change shamelessly and added it to the text. Happy to provide it. >> It seems to me that we really only need a single Boolean setting: >> >> ignore_db=true > > No, that's not right. We have two mechanisms implemented you can > choose three out of four possible combinations: > > files only > db only > files, then db Yes, I realize that. While composing the previous email, I considered a 2-Boolean design: ignore_db=false ignore_files=false I rejected that design when I realized that ignore_files is redundant with respect to /etc/foo file existence. If the files are present, use them. If not, you have a DB-only configuration. I consider it an advantage of this design that it is not possible for the files to be present but ignored. It avoids confusion, both human and software. A person modifying /etc/passwd expects the change to take immediate effect. (Obviously SAM or AD could mask it, but…) Software that scans /etc/passwd expects that the data found within have relevance to system operation. The nonsense configuration — ignore files *and* ignore DB — is still possible to achieve with the 1-Boolean design. Set ignore_db=true and remove the /etc files. Cygwin should simply treat this as a “DB-only” configuration, since that’s the only place it *can* get answers. > It drops any check for existence, too, which > is one code point less which has to run for each getpwXXX/getgrXXX > invocation. Cygwin should check for file existence at the process tree start, where it reads /etc/nsswitch.conf. If the files are missing at that time, set the “ignore_files” flag. Cygwin should use that flag instead of explicit file existence tests for the same reason it doesn’t keep re-reading nsswitch.conf. If the files are missing at process tree start, then later created, you have to create a new process tree for the new files to be used. This is no sacrifice. If you use a 2-Boolean design, you only end up in the code fork where file checking is done if ignore_files == false. That means there is no point to checking for file existence explicitly. Just blindly try to read the files; if the file is missing, the open will fail, so your check is implicit in the attempt. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-16 11:48 ` Warren Young @ 2014-02-16 12:00 ` Corinna Vinschen 2014-02-16 14:50 ` Corinna Vinschen 2014-02-16 16:35 ` Warren Young 0 siblings, 2 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-16 12:00 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 894 bytes --] On Feb 16 03:30, Warren Young wrote: > On Feb 15, 2014, at 5:50 AM, Corinna Vinschen wrote: > > No, that's not right. We have two mechanisms implemented you can > > choose three out of four possible combinations: > > > > files only > > db only > > files, then db > > Yes, I realize that. > > While composing the previous email, I considered a 2-Boolean design: > > ignore_db=false > ignore_files=false > > I rejected that design when I realized that ignore_files is redundant with respect to /etc/foo file existence. If the files are present, use them. If not, you have a DB-only configuration. What if an admin wants to avoid that files are read at all, even if a user manages to generate them? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-16 12:00 ` Corinna Vinschen @ 2014-02-16 14:50 ` Corinna Vinschen 2014-02-16 17:15 ` Christopher Faylor 2014-02-16 16:35 ` Warren Young 1 sibling, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-16 14:50 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2135 bytes --] On Feb 16 12:56, Corinna Vinschen wrote: > On Feb 16 03:30, Warren Young wrote: > > On Feb 15, 2014, at 5:50 AM, Corinna Vinschen wrote: > > > No, that's not right. We have two mechanisms implemented you can > > > choose three out of four possible combinations: > > > > > > files only > > > db only > > > files, then db > > > > Yes, I realize that. > > > > While composing the previous email, I considered a 2-Boolean design: > > > > ignore_db=false > > ignore_files=false > > > > I rejected that design when I realized that ignore_files is redundant with respect to /etc/foo file existence. If the files are present, use them. If not, you have a DB-only configuration. > > What if an admin wants to avoid that files are read at all, even if a > user manages to generate them? Anyway, I'm willing to switch from /etc/nsswitch.conf to something else. Let's assume we go with /etc/cygwin.conf instead. This wouldn't set a bad precedent in terms of syntax and keyword differences to configuration files existing on Linux, and it would allow to define more settings which have nothing to do with passwd/group stuff in the long run. So, how should this look like? Let's do some free floating, # /etc/cygwin.conf pwdgrp="files db" db_separator="+" db_cache="yes" db_prefix="auto" And now the new one. I have getpwent/getgrent practically ready to go. It's just missing the setting to restrict the enumeration. It occured to me that an admin or a user might want to restrict the enumeration to a specific set of domains. What about something like this? db_enum="local primary trusted_dom1 trusted_dom2" "local" and "primary" being fixed keywords for the local machine accounts and the accounts from the primary domain. Everything else is treated as a domain name of a trusted domain, and the name can be given as flat domain name as well as as dns domain name. Does that make sense? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-16 14:50 ` Corinna Vinschen @ 2014-02-16 17:15 ` Christopher Faylor 2014-02-16 18:30 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Christopher Faylor @ 2014-02-16 17:15 UTC (permalink / raw) To: cygwin On Sun, Feb 16, 2014 at 03:48:35PM +0100, Corinna Vinschen wrote: >On Feb 16 12:56, Corinna Vinschen wrote: >>On Feb 16 03:30, Warren Young wrote: >>>On Feb 15, 2014, at 5:50 AM, Corinna Vinschen wrote: >>>>No, that's not right. We have two mechanisms implemented you can >>>>choose three out of four possible combinations: >>>> >>>>files only db only files, then db >>> >>>Yes, I realize that. >>> >>>While composing the previous email, I considered a 2-Boolean design: >>> >>>ignore_db=false ignore_files=false >>> >>>I rejected that design when I realized that ignore_files is redundant >>>with respect to /etc/foo file existence. If the files are present, use >>>them. If not, you have a DB-only configuration. >> >>What if an admin wants to avoid that files are read at all, even if a >>user manages to generate them? > >Anyway, I'm willing to switch from /etc/nsswitch.conf to something >else. Can you summarize why this is necessary? I haven't really followed how we got to the point where documented nscd.conf/nsswitch.conf functionality wouldn't be sufficient. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-16 17:15 ` Christopher Faylor @ 2014-02-16 18:30 ` Corinna Vinschen 0 siblings, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-16 18:30 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1688 bytes --] On Feb 16 12:09, Christopher Faylor wrote: > On Sun, Feb 16, 2014 at 03:48:35PM +0100, Corinna Vinschen wrote: > >On Feb 16 12:56, Corinna Vinschen wrote: > >>On Feb 16 03:30, Warren Young wrote: > >>>On Feb 15, 2014, at 5:50 AM, Corinna Vinschen wrote: > >>>>No, that's not right. We have two mechanisms implemented you can > >>>>choose three out of four possible combinations: > >>>> > >>>>files only db only files, then db > >>> > >>>Yes, I realize that. > >>> > >>>While composing the previous email, I considered a 2-Boolean design: > >>> > >>>ignore_db=false ignore_files=false > >>> > >>>I rejected that design when I realized that ignore_files is redundant > >>>with respect to /etc/foo file existence. If the files are present, use > >>>them. If not, you have a DB-only configuration. > >> > >>What if an admin wants to avoid that files are read at all, even if a > >>user manages to generate them? > > > >Anyway, I'm willing to switch from /etc/nsswitch.conf to something > >else. > > Can you summarize why this is necessary? I haven't really followed how > we got to the point where documented nscd.conf/nsswitch.conf > functionality wouldn't be sufficient. Documented for Linux or documented for Cygwin? Most of the settings now done in nsswitch.conf have nothing to do with what is done in nsswitch.conf on other OSes. These settings have no equivalent to the settings done in nscd.conf either. Except for the passwd: and group: settings, they are plain Cygwin only. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-16 12:00 ` Corinna Vinschen 2014-02-16 14:50 ` Corinna Vinschen @ 2014-02-16 16:35 ` Warren Young 2014-02-16 16:53 ` Andrey Repin 1 sibling, 1 reply; 145+ messages in thread From: Warren Young @ 2014-02-16 16:35 UTC (permalink / raw) To: cygwin On Feb 16, 2014, at 4:56 AM, Corinna Vinschen wrote: > What if an admin wants to avoid that files are read at all, even if a > user manages to generate them? Such an admin should adjust the permissions on /etc so normal users can’t create files there, as on a “real” *ix. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-16 16:35 ` Warren Young @ 2014-02-16 16:53 ` Andrey Repin 0 siblings, 0 replies; 145+ messages in thread From: Andrey Repin @ 2014-02-16 16:53 UTC (permalink / raw) To: Warren Young, cygwin Greetings, Warren Young! >> What if an admin wants to avoid that files are read at all, even if a >> user manages to generate them? > Such an admin should adjust the permissions on /etc so normal users can’t > create files there, as on a “real” *ix. I approve of this product and/or service. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 16.02.2014, <20:24> Sorry for my terrible english... ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 14:44 Testers needed: New passwd/group handling in Cygwin Corinna Vinschen ` (4 preceding siblings ...) 2014-02-14 6:40 ` Testers needed: New passwd/group handling in Cygwin Warren Young @ 2014-02-14 10:20 ` Frank Fesevur 2014-02-14 10:50 ` Corinna Vinschen 2014-02-17 17:25 ` Corinna Vinschen ` (2 subsequent siblings) 8 siblings, 1 reply; 145+ messages in thread From: Frank Fesevur @ 2014-02-14 10:20 UTC (permalink / raw) To: cygwin 2014-02-13 15:38 GMT+01:00 Corinna Vinschen: > This is a pretty intrusive change, in need of some serious testing, so > I'd like to ask for volunteers. The latest 2014-02-13 snapshot from > http://cygwin.com/snapshots/ contains the changes, including the latest > bugfix. I've installed that snapshot on my work laptop, part of AD domain. I moved passwd and group out of the way and noticed my .bashrc is not executed. With 1.7.28: Frank@PC205 ~ $ uname -a CYGWIN_NT-6.1-WOW64 PC205 1.7.28(0.271/5/3) 2014-02-09 21:06 i686 Cygwin Frank@PC205 ~ $ ls -l total 86 -rw------- 1 Frank Domeingebruikers 6759 2014-02-14 09:30 .bash_history -rwxr-xr-x 1 Frank Domeingebruikers 1150 2010-11-17 18:58 .bash_profile* -rwxr-xr-x 1 Frank Domeingebruikers 3814 2013-10-18 12:02 .bashrc* <snip list of 26 files in total> With snapshot without passwd/group: Frank@PC205 ~ $ uname -a CYGWIN_NT-6.1-WOW64 PC205 1.7.29s(0.271/5/3) 20140213 14:06:26 i686 Cygwin Frank@PC205 ~ $ ls -l totaal 19 -rwxr-xr-x 1 Frank Domeingebruikers 98 16 mei 2012 mail-ek.sh -rw-r--r-- 1 Frank Domeingebruikers 811 16 mei 2012 mail-ek.txt -rwxr-xr-x 1 Frank Domeingebruikers 332 12 mei 2013 makeepub.sh drwxrwxrwt+ 1 +Administrators Domeingebruikers 0 24 okt 15:22 python drwxrwxrwt+ 1 +Administrators Domeingebruikers 0 13 feb 17:05 rsync drwxr-xr-x+ 1 Frank Domeingebruikers 0 13 feb 16:44 shutdown-package -rwxr-xr-x 1 Frank Domeingebruikers 38 13 feb 13:57 test.sh <this is the complete list of files> Most noticeable difference between the two are in the dates of the files (The months are in Dutch, and not in "long-iso") and not showing the dot-files. Among others, I have set an export LANG=en_US and an alias for "ls" in my bashrc for that. Reverting to 1.7.28 (with passwd/group) solved the problem. Regards, Frank -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 10:20 ` Frank Fesevur @ 2014-02-14 10:50 ` Corinna Vinschen 2014-02-14 12:05 ` Corinna Vinschen 2014-02-15 2:51 ` Frank Fesevur 0 siblings, 2 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-14 10:50 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2800 bytes --] On Feb 14 11:16, Frank Fesevur wrote: > 2014-02-13 15:38 GMT+01:00 Corinna Vinschen: > > This is a pretty intrusive change, in need of some serious testing, so > > I'd like to ask for volunteers. The latest 2014-02-13 snapshot from > > http://cygwin.com/snapshots/ contains the changes, including the latest > > bugfix. > > I've installed that snapshot on my work laptop, part of AD domain. I > moved passwd and group out of the way and noticed my .bashrc is not > executed. > > With 1.7.28: > > Frank@PC205 ~ > $ uname -a > CYGWIN_NT-6.1-WOW64 PC205 1.7.28(0.271/5/3) 2014-02-09 21:06 i686 Cygwin > > Frank@PC205 ~ > $ ls -l > total 86 > -rw------- 1 Frank Domeingebruikers 6759 2014-02-14 09:30 > .bash_history > -rwxr-xr-x 1 Frank Domeingebruikers 1150 2010-11-17 18:58 > .bash_profile* > -rwxr-xr-x 1 Frank Domeingebruikers 3814 2013-10-18 12:02 .bashrc* > <snip list of 26 files in total> > > With snapshot without passwd/group: > > Frank@PC205 ~ > $ uname -a > CYGWIN_NT-6.1-WOW64 PC205 1.7.29s(0.271/5/3) 20140213 14:06:26 i686 Cygwin > > Frank@PC205 ~ > $ ls -l > totaal 19 > -rwxr-xr-x 1 Frank Domeingebruikers 98 16 mei 2012 mail-ek.sh > -rw-r--r-- 1 Frank Domeingebruikers 811 16 mei 2012 mail-ek.txt > -rwxr-xr-x 1 Frank Domeingebruikers 332 12 mei 2013 makeepub.sh > drwxrwxrwt+ 1 +Administrators Domeingebruikers 0 24 okt 15:22 python > drwxrwxrwt+ 1 +Administrators Domeingebruikers 0 13 feb 17:05 rsync > drwxr-xr-x+ 1 Frank Domeingebruikers 0 13 feb 16:44 shutdown-package > -rwxr-xr-x 1 Frank Domeingebruikers 38 13 feb 13:57 test.sh > <this is the complete list of files> > > Most noticeable difference between the two are in the dates of the > files (The months are in Dutch, and not in "long-iso") and not showing > the dot-files. Among others, I have set an export LANG=en_US ...hopefully LANG='en_US.utf8"... > and an > alias for "ls" in my bashrc for that. Hmm, off the top of my head I can't explain this. Is that via mintty or a Windows console? What does `id' print? Is there a difference in upper/lower case of your name? [...time passes...] Oh, hang on. I'm a tcsh gal, so I set /bin/tcsh in my AD domain entry. However, the default is *not* /bin/bash, but /bin/sh at the moment. I'm a bit fuzzy on bash, but does bash read the .bachrc file only if it's called bash and not if it's called sh? What happens if you symlink your .bashrc to .profile (Not that I suggest this as the ultimate solution, this is just for testing)? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 10:50 ` Corinna Vinschen @ 2014-02-14 12:05 ` Corinna Vinschen 2014-02-14 14:04 ` Ken Brown 2014-02-15 2:51 ` Frank Fesevur 1 sibling, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-14 12:05 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1570 bytes --] On Feb 14 11:48, Corinna Vinschen wrote: > On Feb 14 11:16, Frank Fesevur wrote: > > I've installed that snapshot on my work laptop, part of AD domain. I > > moved passwd and group out of the way and noticed my .bashrc is not > > executed. > > [...] > > Most noticeable difference between the two are in the dates of the > > files (The months are in Dutch, and not in "long-iso") and not showing > > the dot-files. Among others, I have set an export LANG=en_US > > ...hopefully LANG='en_US.utf8"... > > > and an > > alias for "ls" in my bashrc for that. > > Hmm, off the top of my head I can't explain this. Is that > via mintty or a Windows console? What does `id' print? Is there a > difference in upper/lower case of your name? > > [...time passes...] > > Oh, hang on. I'm a tcsh gal, so I set /bin/tcsh in my AD domain entry. > However, the default is *not* /bin/bash, but /bin/sh at the moment. I'm > a bit fuzzy on bash, but does bash read the .bachrc file only if it's > called bash and not if it's called sh? What happens if you symlink > your .bashrc to .profile (Not that I suggest this as the ultimate > solution, this is just for testing)? Ultimately, the right thing to do is either: - We switch Cygwin to use /bin/bash as default, instead of /bin/sh, - or, you ask your admin to set the "loginShell" attribute of your AD entry to "/bin/bash". Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 12:05 ` Corinna Vinschen @ 2014-02-14 14:04 ` Ken Brown 2014-02-14 14:22 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Ken Brown @ 2014-02-14 14:04 UTC (permalink / raw) To: cygwin On 2/14/2014 6:00 AM, Corinna Vinschen wrote: > On Feb 14 11:48, Corinna Vinschen wrote: >> On Feb 14 11:16, Frank Fesevur wrote: >>> I've installed that snapshot on my work laptop, part of AD domain. I >>> moved passwd and group out of the way and noticed my .bashrc is not >>> executed. >>> [...] >>> Most noticeable difference between the two are in the dates of the >>> files (The months are in Dutch, and not in "long-iso") and not showing >>> the dot-files. Among others, I have set an export LANG=en_US >> >> ...hopefully LANG='en_US.utf8"... >> >>> and an >>> alias for "ls" in my bashrc for that. >> >> Hmm, off the top of my head I can't explain this. Is that >> via mintty or a Windows console? What does `id' print? Is there a >> difference in upper/lower case of your name? >> >> [...time passes...] >> >> Oh, hang on. I'm a tcsh gal, so I set /bin/tcsh in my AD domain entry. >> However, the default is *not* /bin/bash, but /bin/sh at the moment. I'm >> a bit fuzzy on bash, but does bash read the .bachrc file only if it's >> called bash and not if it's called sh? What happens if you symlink >> your .bashrc to .profile (Not that I suggest this as the ultimate >> solution, this is just for testing)? > > Ultimately, the right thing to do is either: > > - We switch Cygwin to use /bin/bash as default, instead of /bin/sh, Isn't /bin/bash already the default? The output of `mkpasswd -l' seems to confirm this. Ken -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 14:04 ` Ken Brown @ 2014-02-14 14:22 ` Corinna Vinschen 2014-02-14 15:23 ` Ken Brown 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-14 14:22 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1828 bytes --] On Feb 14 08:50, Ken Brown wrote: > On 2/14/2014 6:00 AM, Corinna Vinschen wrote: > >On Feb 14 11:48, Corinna Vinschen wrote: > >>On Feb 14 11:16, Frank Fesevur wrote: > >>>I've installed that snapshot on my work laptop, part of AD domain. I > >>>moved passwd and group out of the way and noticed my .bashrc is not > >>>executed. > >>>[...] > >>>Most noticeable difference between the two are in the dates of the > >>>files (The months are in Dutch, and not in "long-iso") and not showing > >>>the dot-files. Among others, I have set an export LANG=en_US > >> > >>...hopefully LANG='en_US.utf8"... > >> > >>>and an > >>>alias for "ls" in my bashrc for that. > >> > >>Hmm, off the top of my head I can't explain this. Is that > >>via mintty or a Windows console? What does `id' print? Is there a > >>difference in upper/lower case of your name? > >> > >>[...time passes...] > >> > >>Oh, hang on. I'm a tcsh gal, so I set /bin/tcsh in my AD domain entry. > >>However, the default is *not* /bin/bash, but /bin/sh at the moment. I'm > >>a bit fuzzy on bash, but does bash read the .bachrc file only if it's > >>called bash and not if it's called sh? What happens if you symlink > >>your .bashrc to .profile (Not that I suggest this as the ultimate > >>solution, this is just for testing)? > > > >Ultimately, the right thing to do is either: > > > >- We switch Cygwin to use /bin/bash as default, instead of /bin/sh, > > Isn't /bin/bash already the default? The output of `mkpasswd -l' > seems to confirm this. Oh, right. I didn't remember that. So I guess I should do the same for automatically created entries, right? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 14:22 ` Corinna Vinschen @ 2014-02-14 15:23 ` Ken Brown 0 siblings, 0 replies; 145+ messages in thread From: Ken Brown @ 2014-02-14 15:23 UTC (permalink / raw) To: cygwin On 2/14/2014 9:14 AM, Corinna Vinschen wrote: > On Feb 14 08:50, Ken Brown wrote: >> On 2/14/2014 6:00 AM, Corinna Vinschen wrote: >>> On Feb 14 11:48, Corinna Vinschen wrote: >>>> On Feb 14 11:16, Frank Fesevur wrote: >>>>> I've installed that snapshot on my work laptop, part of AD domain. I >>>>> moved passwd and group out of the way and noticed my .bashrc is not >>>>> executed. >>>>> [...] >>>>> Most noticeable difference between the two are in the dates of the >>>>> files (The months are in Dutch, and not in "long-iso") and not showing >>>>> the dot-files. Among others, I have set an export LANG=en_US >>>> >>>> ...hopefully LANG='en_US.utf8"... >>>> >>>>> and an >>>>> alias for "ls" in my bashrc for that. >>>> >>>> Hmm, off the top of my head I can't explain this. Is that >>>> via mintty or a Windows console? What does `id' print? Is there a >>>> difference in upper/lower case of your name? >>>> >>>> [...time passes...] >>>> >>>> Oh, hang on. I'm a tcsh gal, so I set /bin/tcsh in my AD domain entry. >>>> However, the default is *not* /bin/bash, but /bin/sh at the moment. I'm >>>> a bit fuzzy on bash, but does bash read the .bachrc file only if it's >>>> called bash and not if it's called sh? What happens if you symlink >>>> your .bashrc to .profile (Not that I suggest this as the ultimate >>>> solution, this is just for testing)? >>> >>> Ultimately, the right thing to do is either: >>> >>> - We switch Cygwin to use /bin/bash as default, instead of /bin/sh, >> >> Isn't /bin/bash already the default? The output of `mkpasswd -l' >> seems to confirm this. > > Oh, right. I didn't remember that. So I guess I should do the same > for automatically created entries, right? Right. Ken -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-14 10:50 ` Corinna Vinschen 2014-02-14 12:05 ` Corinna Vinschen @ 2014-02-15 2:51 ` Frank Fesevur 2014-02-15 12:56 ` Corinna Vinschen 1 sibling, 1 reply; 145+ messages in thread From: Frank Fesevur @ 2014-02-15 2:51 UTC (permalink / raw) To: cygwin 2014-02-14 11:48 GMT+01:00 Corinna Vinschen: > Oh, hang on. I'm a tcsh gal, so I set /bin/tcsh in my AD domain entry. > However, the default is *not* /bin/bash, but /bin/sh at the moment. I'm > a bit fuzzy on bash, but does bash read the .bachrc file only if it's > called bash and not if it's called sh? What happens if you symlink > your .bashrc to .profile (Not that I suggest this as the ultimate > solution, this is just for testing)? Just tested and copied .bashrc to .profile and that solved the problem. I vote[1] for /bin/bash being the default shell. Regards, Frank [1] not that there is any voting going on ;-) -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-15 2:51 ` Frank Fesevur @ 2014-02-15 12:56 ` Corinna Vinschen 2014-02-19 9:32 ` Frank Fesevur 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-15 12:56 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1041 bytes --] On Feb 14 22:48, Frank Fesevur wrote: > 2014-02-14 11:48 GMT+01:00 Corinna Vinschen: > > Oh, hang on. I'm a tcsh gal, so I set /bin/tcsh in my AD domain entry. > > However, the default is *not* /bin/bash, but /bin/sh at the moment. I'm > > a bit fuzzy on bash, but does bash read the .bachrc file only if it's > > called bash and not if it's called sh? What happens if you symlink > > your .bashrc to .profile (Not that I suggest this as the ultimate > > solution, this is just for testing)? > > Just tested and copied .bashrc to .profile and that solved the problem. Good, so it was really just a shell thingy. > I vote[1] for /bin/bash being the default shell. Done in CVS. > Regards, > Frank > > [1] not that there is any voting going on ;-) Not quite. This *is* a big change, so if I can smooth the ride for the users, I'm open to change. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-15 12:56 ` Corinna Vinschen @ 2014-02-19 9:32 ` Frank Fesevur 0 siblings, 0 replies; 145+ messages in thread From: Frank Fesevur @ 2014-02-19 9:32 UTC (permalink / raw) To: cygwin 2014-02-15 13:52 GMT+01:00 Corinna Vinschen: >> Just tested and copied .bashrc to .profile and that solved the problem. > > Good, so it was really just a shell thingy. > >> I vote[1] for /bin/bash being the default shell. > > Done in CVS. Just downloaded the 2014-02-18 snapshot, deleted that .profile and things works as expected! Regards, Frank -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 14:44 Testers needed: New passwd/group handling in Cygwin Corinna Vinschen ` (5 preceding siblings ...) 2014-02-14 10:20 ` Frank Fesevur @ 2014-02-17 17:25 ` Corinna Vinschen 2014-02-19 15:05 ` Andrey Repin 2014-02-21 15:33 ` Corinna Vinschen 2014-02-25 18:25 ` Achim Gratz 2014-02-27 19:08 ` Frank Fesevur 8 siblings, 2 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-17 17:25 UTC (permalink / raw) To: cygwin [-- Attachment #1.1: Type: text/plain, Size: 4052 bytes --] On Feb 13 15:38, Corinna Vinschen wrote: > Hi folks, > > > this week I applied the first incarnation of the new passwd/group > handling code to the Cygwin repository and after fixing a crash which > manifested in Denis Excoffier's network, I think we're at a point > which allows to push this forward. > [...] Ok guys, I just applied a patch implementing getpwent and getgrent, and the way to configure the output is probably more detailed than you ever wanted: The default output of both functions consists of the so far cached accounts, plus a fixed set of builtin accounts for backward compatibility with mkpasswd/mkgroup output. For instance, getpwent output looks like this: corinna:*:1049577:1049701:Corinna Vinschen,U-VINSCHEN\corinna,S-1-5-21-2913048732-1697188782-3448811101-1001:/home/corinna:/bin/tcsh +SYSTEM:*:18:18:U-NT AUTHORITY\SYSTEM,S-1-5-18:/home/SYSTEM:/bin/bash +LOCAL SERVICE:*:19:19:U-NT AUTHORITY\LOCAL SERVICE,S-1-5-19:/home/LOCAL SERVICE:/bin/bash +NETWORK SERVICE:*:20:20:U-NT AUTHORITY\NETWORK SERVICE,S-1-5-20:/home/NETWORK SERVICE:/bin/bash +Administrators:*:544:513:U-BUILTIN\Administrators,S-1-5-32-544:/home/Administrators:/bin/bash +TrustedInstaller:*:328384:328384:U-NT SERVICE\TrustedInstaller,S-1-5-80-956008885-3418522649-1831038044-1853292631-2271478464:/home/TrustedInstaller:/bin/bash and getgrent like this: vinschen:S-1-5-21-2913048732-1697188782-3448811101-1125:1049701: +Administrators:S-1-5-32-544:544: +SYSTEM:S-1-5-18:18: +TrustedInstaller:S-1-5-80-956008885-3418522649-1831038044-1853292631-2271478464:328384: As long as we didn't decide to do it differently, the configuration takes place in /etc/nsswitch.conf. The new keyword is "db_enum", and the value is a list of sources: db_enum: source... A source can be none No output from getpwent/getgrent at all. all The opposite. Output accounts from all known sources, including all trusted domains. cache List all accounts currently cached in memory. builtin List the predefined builtin accounts for backward compatibility. files List the accounts from /etc/passwd or /etc/group. local List all accounts from the local SAM. primary List all accounts from the primary domain. alltrusted List all accounts from all trusted domains. some.domain List all accounts from the trusted domain some.domain. The trusted domain can be given as Netbios flat name (MY_DOMAIN) or as dns domain name (my_domain.corp). In contrast to the aforementioned fixed source keywords, distinct domain names are caseinsensitive. Only domains which are actually trusted domains are enumerated. Unknown domains are simply ignored. Please note that this functionality does *not* test if an account was already listed from another source, so an account can easily show up twice or three times. Such a test would be rather tricky, nor does the Linux implementation perform such test. Here are a few examples for /etc/nsswitch.conf, which are hopefully self-explaining: db_enum: none db_enum: all db_enum: cache files db_enum: cache local primary db_enum: local primary alltrusted db_enum: primary domain1.corp sub.domain.corp domain2.net If those examples are *not* self-explaining, don't hesitate to ask. Please fetch the latest snapshot from http://cygwin.com/snapshots/ It contains this change. I attached two simple test applications to this mail. If it works nicely, I will rewrite mkpasswd and mkgroup to use the Cygwin implementation as well, rather than implementing their own enumerations. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #1.2: getpwent.c --] [-- Type: text/plain, Size: 321 bytes --] #include <stdio.h> #include <pwd.h> int main () { struct passwd *pwd; setpwent (); while ((pwd = getpwent ())) printf ("%s:%s:%u:%u:%s:%s:%s\n", pwd->pw_name, pwd->pw_passwd, pwd->pw_uid, pwd->pw_gid, pwd->pw_gecos, pwd->pw_dir, pwd->pw_shell); endpwent (); return 0; } [-- Attachment #1.3: getgrent.c --] [-- Type: text/plain, Size: 233 bytes --] #include <stdio.h> #include <grp.h> int main () { struct group *grp; setgrent (); while ((grp = getgrent ())) printf ("%s:%s:%u:\n", grp->gr_name, grp->gr_passwd, grp->gr_gid); endgrent (); return 0; } [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-17 17:25 ` Corinna Vinschen @ 2014-02-19 15:05 ` Andrey Repin 2014-02-19 18:50 ` Corinna Vinschen 2014-02-21 15:33 ` Corinna Vinschen 1 sibling, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-02-19 15:05 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! >> this week I applied the first incarnation of the new passwd/group >> handling code to the Cygwin repository and after fixing a crash which >> manifested in Denis Excoffier's network, I think we're at a point >> which allows to push this forward. >> [...] > Ok guys, I just applied a patch implementing getpwent and getgrent, > and the way to configure the output is probably more detailed than > you ever wanted: > db_enum: source... Setting db_enum: local seems to be really destructive. I'm not good at reading straces, so... Can I make a useful trace for your inspection? So far, I'm unable to start anything with freshly rebuilt Cygwin1.dll if db_enum set to local. Even uname -a crashes. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 19.02.2014, <18:42> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-19 15:05 ` Andrey Repin @ 2014-02-19 18:50 ` Corinna Vinschen 2014-02-19 19:32 ` Andrey Repin 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-19 18:50 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1237 bytes --] On Feb 19 18:56, Andrey Repin wrote: > Greetings, Corinna Vinschen! > > >> this week I applied the first incarnation of the new passwd/group > >> handling code to the Cygwin repository and after fixing a crash which > >> manifested in Denis Excoffier's network, I think we're at a point > >> which allows to push this forward. > >> [...] > > > Ok guys, I just applied a patch implementing getpwent and getgrent, > > and the way to configure the output is probably more detailed than > > you ever wanted: > > > db_enum: source... > > Setting > > db_enum: local > > seems to be really destructive. > I'm not good at reading straces, so... Can I make a useful trace for your > inspection? > So far, I'm unable to start anything with freshly rebuilt Cygwin1.dll > if db_enum set to local. Even uname -a crashes. The last one is weird since uname doesn't call getpwent. Anyway, with more help from Denis Excoffier I think I got it. I'm just building a new snapshot which should be up in half an hour or so. Please give it a try. Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-19 18:50 ` Corinna Vinschen @ 2014-02-19 19:32 ` Andrey Repin 2014-02-19 19:56 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-02-19 19:32 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! >> Setting >> >> db_enum: local >> >> seems to be really destructive. >> I'm not good at reading straces, so... Can I make a useful trace for your >> inspection? >> So far, I'm unable to start anything with freshly rebuilt Cygwin1.dll >> if db_enum set to local. Even uname -a crashes. > The last one is weird since uname doesn't call getpwent. That's what was my thought, too. Because (un)setting it back cures the crash. > Anyway, with more help from Denis Excoffier I think I got it. I'm > just building a new snapshot which should be up in half an hour or so. > Please give it a try. Looking into it now. Doesn't crash or anything so far. (With `db_enum: local' set.) Tried a few utilities, as well as your example get*ent code. BTW, can we have it as one "getent"[1] tool for the next release? Going to play with different settings, I will post if I find anything unexpected. [1] http://linux.die.net/man/1/getent -- WBR, Andrey Repin (anrdaemon@yandex.ru) 19.02.2014, <23:11> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-19 19:32 ` Andrey Repin @ 2014-02-19 19:56 ` Corinna Vinschen 2014-02-19 20:27 ` Andrey Repin 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-19 19:56 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1560 bytes --] On Feb 19 23:19, Andrey Repin wrote: > Greetings, Corinna Vinschen! > > >> Setting > >> > >> db_enum: local > >> > >> seems to be really destructive. > >> I'm not good at reading straces, so... Can I make a useful trace for your > >> inspection? > >> So far, I'm unable to start anything with freshly rebuilt Cygwin1.dll > >> if db_enum set to local. Even uname -a crashes. > > > The last one is weird since uname doesn't call getpwent. > > That's what was my thought, too. Because (un)setting it back cures the crash. > > > Anyway, with more help from Denis Excoffier I think I got it. I'm > > just building a new snapshot which should be up in half an hour or so. > > Please give it a try. > > Looking into it now. > Doesn't crash or anything so far. (With `db_enum: local' set.) > Tried a few utilities, as well as your example get*ent code. > BTW, can we have it as one "getent"[1] tool for the next release? > > Going to play with different settings, I will post if I find anything > unexpected. > > [1] http://linux.die.net/man/1/getent That sounds pretty much like the tool we need to be able to get rid of the `grep /etc/passwd' stuff we have in some of the service configuration helper scripts. Unfortunately it's a glibc tool so we probably have to create our own. For a start it would suffice to support only passwd and group databases. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-19 19:56 ` Corinna Vinschen @ 2014-02-19 20:27 ` Andrey Repin 2014-02-20 16:02 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-02-19 20:27 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! >> [1] http://linux.die.net/man/1/getent > That sounds pretty much like the tool we need to be able to get rid of > the `grep /etc/passwd' stuff we have in some of the service configuration > helper scripts. Unfortunately it's a glibc tool so we probably have to > create our own. > For a start it would suffice to support only passwd and group databases. Yup, was exactly what I thought. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 19.02.2014, <23:56> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-19 20:27 ` Andrey Repin @ 2014-02-20 16:02 ` Corinna Vinschen 2014-02-20 17:35 ` Andrey Repin 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-20 16:02 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 886 bytes --] On Feb 19 23:57, Andrey Repin wrote: > Greetings, Corinna Vinschen! > > >> [1] http://linux.die.net/man/1/getent > > > That sounds pretty much like the tool we need to be able to get rid of > > the `grep /etc/passwd' stuff we have in some of the service configuration > > helper scripts. Unfortunately it's a glibc tool so we probably have to > > create our own. > > > For a start it would suffice to support only passwd and group databases. > > Yup, was exactly what I thought. Just FYI, I succeeded in porting the Glibc getent. I will provide this as a standalone package ASAP, so we can start converting the service configuration helper scripts even before we release the new passwd/group code. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-20 16:02 ` Corinna Vinschen @ 2014-02-20 17:35 ` Andrey Repin 0 siblings, 0 replies; 145+ messages in thread From: Andrey Repin @ 2014-02-20 17:35 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! >> >> [1] http://linux.die.net/man/1/getent >> >> > That sounds pretty much like the tool we need to be able to get rid of >> > the `grep /etc/passwd' stuff we have in some of the service configuration >> > helper scripts. Unfortunately it's a glibc tool so we probably have to >> > create our own. >> >> > For a start it would suffice to support only passwd and group databases. >> >> Yup, was exactly what I thought. > Just FYI, I succeeded in porting the Glibc getent. I will provide this > as a standalone package ASAP, so we can start converting the service > configuration helper scripts even before we release the new passwd/group > code. O.o thursday of miracles. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 20.02.2014, <21:13> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-17 17:25 ` Corinna Vinschen 2014-02-19 15:05 ` Andrey Repin @ 2014-02-21 15:33 ` Corinna Vinschen 2014-02-21 16:29 ` Ken Brown 1 sibling, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-21 15:33 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 32654 bytes --] On Feb 17 17:21, Corinna Vinschen wrote: > On Feb 13 15:38, Corinna Vinschen wrote: > > Hi folks, > > > > > > this week I applied the first incarnation of the new passwd/group > > handling code to the Cygwin repository and after fixing a crash which > > manifested in Denis Excoffier's network, I think we're at a point > > which allows to push this forward. > > [...] > > Ok guys, I just applied a patch implementing getpwent and getgrent, > and the way to configure the output is probably more detailed than > you ever wanted > [...] I just added the latest changes to my preliminary documentation, see below. Please have a look. I'll rework this into our documetation at one point, and it would be nice if it's more or less bugfree and, hopefully, comprehensible at this point. Thanks, Corinna ======= History ======= For as long as Cygwin has existed, it has stored user and group information in /etc/passwd and /etc/group files. Under the assumption that these files would never be too large, the first process in a process tree, as well as every execing process within the tree would parse them into structures in memory. Thus every Cygwin process would contain an expanded copy of the full information from /etc/passwd and /etc/group. This approach has a few downsides. One of them is that the idea to have always small files is flawed. Another one is that reading the entire file is most of the time entirely useless, since most processes only need information on their own user and the primary group. Last but not least, the passwd and group files have to be maintained separately from the already existing Windows user databases, the local SAM and Active Directory. On the other hand, we have to have a mapping between Windows SIDs and POSIX uid/gid values (see [1]), so we rely on some mechanism to convert SIDs to uid/gid values and vice versa. Microsoft "Services for UNIX" (SFU) (which are unfortunately deprecated since Windows 8/Server 2012) never used passwd/group files. Rather, SFU used a fixed, computational mapping between SIDs and POSIX uid/gid. It allows to generate uid/gid values from SIDs and vice versa. The mechanism is documented, albeit in a confusing way and spread over multiple MSDN articles. The Cygwin approach clones the mapping, with just tiny differences for backward compatibility. ================= How does it work? ================= The following description assumes you're comfortable with the concept of Windows SIDs and RIDs. For a brief introduction, please read [1]. Cygwin's new mapping between SIDs and uid/gid values works in two ways. - Read /etc/passwd and /etc/group files, like before, mainly for backward compatibility. - If no files are present, or if an entry is missing in the files, ask Windows. At least, that's the default behaviour now. It will be configurable using a file /etc/nsswitch.conf, which is discussed in a later section. Let's explore the default for now. If files are present, they will be scanned on demand as soon as a mapping from SIDs to uid/gid or user names is required. The new mechanism will never read the entire file into memory, but only scan for the requested entry and cache this one in memory[2]. If no entry is found, or no passwd or group file was present, Cygwin will ask the OS. Note: If the first process in a Cygwin process tree determines that no /etc/passwd or /etc/group file is present, no other process in the entire process tree will try to read the files later on. This is done for self-preservation. It's rather bad if the uid or gid of a user changes during the lifetime of a process tree. For the same reason, if you delete the /etc/passwd or /etc/group file, this will be ignored. The passwd and group records read from the files will persist in memory until either a new /etc/passwd or /etc/group files is created, or you exit all processes in the current process tree. See the note in the section on /etc/nsswitch.conf for some comprehensive examples. So if we've drawn a blank reading the files, we're going to ask the OS. First thing, we ask the local machine for the SID or the user name. The OS functions LookupAccountSid and LookupAccountName[3] are pretty intelligent. They have all the stuff built in to ask for any account of the local machine, the Active Directory domain of the machine, the Global Catalog of the forest of the domain, as well as any trusted domain of our forest for the information. One OS call and we're practically done... Except, the calls only return the mapping between SID, account name and the account's domain. We don't have a mapping to POSIX uid/gid and we're missing information on the user's home dir and login shell. Let's discuss the SID<=>uid/gid mapping first. Here's how it works. - Well-known SIDs in the NT_AUTHORITY domain of the S-1-5-RID type, or aliases of the S-1-5-32-RID type are mapped to the uid/gid value RID[4]. For an overview of well-known SIDs, see [5]. Examples: "SYSTEM" S-1-5-18 <=> uid/gid: 18 "Users" S-1-5-32-545 <=> uid/gid: 545 - Other well-known SIDs in the NT_AUTHORITY domain (S-1-5-X-RID): S-1-5-X-RID <=> uid/gid: 0x1000 * X + RID Example: "NTLM Authentication" S-1-5-64-10 <=> uid/gid: 0x4000A == 262154 - Other well-known SIDs: S-1-X-Y <=> uid/gid: 0x10000 + 0x100 * X + Y Example: "LOCAL" S-1-2-0 <=> uid/gid: 0x10200 == 66048 "Creator Group" S-1-3-1 <=> uid/gid: 0x10301 == 66305 - Logon SIDs: The own LogonSid is converted to the fixed uid 0xfff == 4095 and named "CurrentSession". Any other LogonSid is converted to the fixed uid 0xffe == 4094 and named "OtherSession". - Mandatory Labels: S-1-16-RID <=> uid/gid: 0x60000 + RID Example: "Medium Mandatory Level" S-1-16-8192 <=> uid/gid: 0x62000 == 401408 - Accounts from the local machine's user DB (SAM): S-1-5-21-X-Y-Z-RID <=> 0x30000 + RID Example: "Administrator" S-1-5-X-Y-Z-500 <=> uid/gid: 0x301f4 == 197108 - Accounts from the machine's primary domain: S-1-5-21-X-Y-Z-RID <=> 0x100000 + RID Example: "Domain Users" S-1-5-X-Y-Z-513 <=> 0x100201 == 1049089 - Accounts from a trusted domain of the machine's primary domain: S-1-5-21-X-Y-Z-RID <=> trustPosixOffset(domain) + RID "trustPosixOffset"? This needs a bit of explanation. This value exists in Windows domains already since before Active Directory days. What happens is this. If you create a domain trust between two domains, a trustedDomain entry will be added to both databases. It describes how *this* domain trusts the *other* domain. One attribute of a trust is a 32 bit value called "trustPosixOffset" For each new trust, trustPosixOffset will get some automatic value. In recent AD domain implementations, the first trusted domain will get trustPosixOffset set to 0x80000000. Following domains will get lower values. Unfortunately the domain admins are allowed to set the trustPosixOffset value for each trusted domain to some arbitrary 32 bit value, no matter what the other trustPosixOffsets are seet to, thus allowing any kind of collisions between the trustPosixOffsets of domains. That's not exactly helpful, but as the user of this value, we have to *trust* the domain admins to set trustPosixOffset to sensible values, or to keep it at the system chosen values. So, for the first (or only) trusted domain of your domain, the automatic offset is 0x80000000. An example for a user of that trusted domain is: S-1-5-X-Y-Z-1234 <=> uid/gid 0x800004d2 == 2147484882 There's only one problem with this approach. Assuming you're running in the context of a local SAM user on a domain member machine. Local users don't have the right to fetch this kind of domain information from the DC, they'll get permission denied. In this case Cygwin will fake a, mostly, sensible trustPosixOffset value for this session. - Local accounts from another machine in the network: There's no SID<=>uid/gid mapping implemented for this case. The problem is, there's no way to generate a bijective mapping. There's no central place which keeps an analogue value of the trustPosixOffset, and there's the additional problem that the LookupAccountSid and LookupAccountName functions cannnot resolve the SIDs, unless they know the name of the machine this SID comes from. And even then it will probably suffer a "Permission denied" when trying to ask the machine for its local account. SFU just prints the account RID in this case, Cygwin maps the account to the fake accounts "Unknown+User"/"Unknown+Group" with uid/gid -1. Now we have a semi-bijective mapping between SIDs and POSIX uid/gid values, but, given that we have potentially users and groups in different domains having the same name, how do we uniquely differ between them by name? Well, we can do that by making their names unique in a per-machine way. Dependent on the domain membership of the account, and dependent of the machine being a domain member or not, the user and group names will be generated using a domain prefix and a separator character between domain and account name. The default separator character is the plus sign, '+', as in SFU. - Well-known SIDs will have the separator character prepended: "+SYSTEM", "+LOCAL", "+Medium Mandatory Level", ... - If the machine is no domain member machine, only local accounts can be resolved into names, so for ease of use, just the account names are used as Cygwin user/group names: "corinna", "bigfoot", "None", ... - If the machine is a domain member machine, all accounts from the primary domain of the machine are mapped to Cygwin names without domain prefix: "corinna", "bigfoot", "Domain Users", ... while accounts from other domains are prepended by their domain: "DOMAIN1+corinna", "DOMAIN2+bigfoot", "DOMAIN3+Domain Users", ... - Local machine accounts of a domain member machine get a Cygwin user name the same way as accounts from another domain: The local machine name gets prepended: "MYMACHINE+corinna", "MYMACHINE+bigfoot", "MYMACHINE+None", ... - If LookupAccountSid fails, Cygwin checks the accounts against the known trusted domains. If the account is from one of the trusted domains, an artificial account name is created. It consists of the domain name, and a special name created from the account RID: "MY_DOM+User(1234)", "MY_DOM+Group(5678)" Otherwise we know nothing about this SID, so it will be mapped to the fake accounts "Unknown+User"/"Unknown+Group" with uid/gid -1. ========================================== Cygwin user names, home dirs, login shells ========================================== Obviously, if you don't maintain passwd and group files, you need to have a way to maintain the other fields of a passwd entry as well. Three things come to mind: - You want to use a Cygwin user name different from your Windows user name. - You want a home dir different from the default /home/$USER. - You want to specify a different login shell than /bin/sh. How this is done depends on your account being a domain account or a local account. Let's start with the default. Assuming your Windows account name is "bigfoot" and your domain is "MY_DOM". Your default passwd entry in absence of anything I'll describe below looks like this: bigfoot:*:<uid>:<gid>:U-MY_DOM\bigfoot,S-1-5-....:/home/bigfoot:/bin/bash or, if your account is from a different domain than the primary domain of the machine: MY_DOM+bigfoot:*:<uid>:<gid>:U-MY_DOM\bigfoot,S-1-5-....:/home/bigfoot:/bin/bash Yes, the default homedir is still /home/bigfoot. If your account is a domain account: Either create an /etc/passwd and/or /etc/group file with entries for your account and use that, just as before. Or, Cygwin will utilize the posixAccount/posixGroup attributes per RFC 2307[6]. These attributes are by default available in Active Directory since Windows Server 2003 R2. They are "not set", unless utilized by the (deprecated since Server 2012 R2) Active Directory "Server for NIS" feature. The user attributes utilized by Cygwin are: uid If set, will be used as Cygwin user name. uidNumber See next section. gecos Content will be added to the pw_gecos field. unixHomeDirectory If set, will be used as Cygwin home directory. loginShell If set, will be used as Cygwin login shell. The group attributes utilized by Cygwin are: cn If set, will be used as Cygwin group name. gidNumber See next section. Apart from power shell scripting or inventing new CLI tools, these attributes can be changed using the "Attribute Editor" tab in the user properties dialog of the "Active Directory Users and Computers" MMC snap-in. Alternatively, if the "Server for NIS" administration feature has been installed, there will be a "UNIX Attributes" tab which contains the required fields, except for the gecos field, which isn't really important anyway. Last resort is "ADSI Edit". The primary group of a user is always the Windows primary group set in Active Directory and can't be changed. If your machine is not a domain member machine or your account is a local account for some reason: Either create an /etc/passwd and/or /etc/group file with entries for your account and use that, just as before. Or enter the information into the "Comment" field of your local user entry. In the "Local Users and Groups" MMC snap-in it's called "Description". You can utilze this field even if you're running a "home edition" of Windows, using the command line. The "net user" command allows to set all values in the SAM, even if the GUI is crippled. A Cygwin SAM comment entry looks like this: <cygwin key="value" key="value" [...] /> The supported keys are name="value" Sets the Cygwin user name to value. home="value" Sets the Cygwin home dir to value. shell="value" Sets the Cygwin login shell to value. group="value" Sets the Cygwin primary group of the account to value, provided that the user *is* already a member of that group. This allows to override the default "None" primary group for local accounts. One nice idea here is, for instance group="Users". This is the *Windows* name of the group, not the Cygwin name, assuming they differ. unix="value" Sets the NFS/Samba uid of the user to the decimal value. See the next chapter. The <cygwin .../> string can start at any point in the comment, but you have to follow the rules: - It starts with "<cygwin " and ends with "/>". - The "cygwin" string and the key names have to be lowercase. - No spaces between key and "value", just the equal sign. - The value must be placed within double quotes and it must not contain a double quote itself. The double quotes are required for the decimal values as well! CMD example: net user corinna /comment:"<cygwin home=\"/home/foo\"/>" Bash example (use single quotes): net user corinna /comment:'<cygwin home="/home/foo"/>' For changing group comments, use the `net localgroup' command. The supported key/value pair for groups are name="value" Sets the Cygwin group name to value. unix="value" Sets the NFS/Samba gid of the group to the decimal value. See the next chapter. =================== NFS account mapping =================== Microsoft's NFS client does not map the uid/gid values on the NFS shares to SIDs. There's no such thing as a (fake) security descriptor returned to the application. Rather, via an undocumented API an applications can fetch RFC 1813 compatible NFSv3 stat information from the share[7]. This is what Cygwin is using to show stat information for files on NFS shares. The problem is, while all other information in this stat record, like timestamps, file size etc., can be used by Cygwin, Cygwin had no way to map the values of the st_uid and st_gid members to a Windows SID for a long time. So it just faked the file owner info and claimed that it's you. However, SFU has, over time, developed multiple methods to map UNIX uid/gid values on NFS shares to Windows SIDs. You'll find the full documentation of the mapping methods in [8]. Cygwin now utilizes the RFC 2307 mapping for this purpose. This is most of the time provided by an AD domain, but it could also be a standalone LDAP mapping server. Per RFC 2307, the uid is in the attribute uidNumber. For groups, the gid is in the gidNumber attribute. When Cygwin stat's files on an NFS share, it asks the mapping server via LDAP in two different ways, depending on the role of the mapping server. - If the server is an AD domain controller, it asks for an account with uidNumber attribute == st_uid field of the stat record returned by NFS. If an account matches, AD returns the Windows SID, so we have an immediate mapping from UNIX uid to a Windows SID, if the user account has a valid uidNumber attribute. For groups, the method is the same, just that Cygwin asks for a group with gidNumber attribute == st_gid field of the stat record. - If the server is a standalone LDAP mapping server Cygwin asks for the same uidNumber/gidNumber attributes, but it can't expect that the LDAP server knows anything about Windows SIDs. Rather, the mapping server returns the account name. Cygwin then asks the DC for an account with this name, and if that succeeds, we have a mapping between UNIX uid/gid and Windows SIDs. The mapping will be cached for the lifetime of the process, and inherited by child processes. ===================== Samba account mapping ===================== A fully set up Samba with domain integration is running winbindd to map Window SIDs to artificially created UNIX uids and gids, and this mapping is transparent within the domain, so Cygwin doesn't have to do anything special. However, setting up winbindd isn't for everybody, and it fails to map Windows accounts to already existing UNIX users or groups. In contrast to NFS, Samba returns security descriptors, but unmapped UNIX accounts get special SIDs: - A UNIX user account with uid X is mapped to the Windows SID S-1-22-1-X. - A UNIX group account with gid X is mapped to SID S-1-22-2-X. As you can see, even though we have SIDs, they just reflect the actual uid/gid values on the UNIX box in the RID value. It's only marginally different from the NFS method, so why not just use the same method as for NFS? That's what Cygwin will do. If it encounters a S-1-22-x-y SID, it will perform the same RFC 2307 mapping as for NFS shares. For home users without any Windows domain or LDAP server per RFC 2307, but with a Linux machine running Samba, just add this information to your SAM account. Assuming the uid of your Linux user account is 505 and the gid of your primary group is, say, 100, just add the values to your SAM user and group accounts. The following example assumes you didn't already add something else to the comment field. To your user's SAM comment (remember: called "Description" in the GUI), add: <cygwin group="Users" unix="505"/> To the user's group SAM comment add: <cygwin unix="100"/> This should be sufficient to work on your Samba share and to see all files owned by your Linux user account as your files. =========================== The /etc/nsswitch.conf file =========================== Last, but not least, let's talk about the way to configure how the mapping works on your machine. On Linux and some other UNIXy OSes, we have a file called /etc/nsswitch.conf[9]. One part of it is to specify how the passwd and group entries are generated. That's what Cygwin now provides as well. The /etc/nsswitch.conf file is optional. If you don't have one, Cygwin uses sensible defaults. Note: The /etc/nsswitch.conf file is read exactly once by the first process of a Cygwin process tree. If there was no /etc/nsswitch.conf file when this first process started, then no other process in the running Cygwin process trees will try to read the file. If you create or change /etc/nsswitch.conf, you need to restart all Cygwin processes that need to see the change. If the process you want to see the change is a child of another process, you need to restart all of that process's parents, too. For example, if you run Vim inside the default Cygwin Terminal, Vim is a child of your shell, which is a child of mintty.exe. If you edit /etc/nsswitch.conf in that Vim instance, your shell won't immediately see the change, nor will Vim if you restart it from that same shell instance. This is because both are getting their nsswitch information from their ancestor, mintty.exe. You need to start a fresh terminal window for the change to take effect. By contrast, if you leave that Cygwin Terminal window open after making the change to /etc/nsswitch.conf, then restart a Cygwin service like cron, cron will see the change, because it is not a child of mintty.exe or any other Cygwin process. (Technically, it is a child of cygrunsrv, but that instance also restarts when you restart the service.) The reason we point all this out is that the requirements for restarting things are not quite as stringent as when you replace cygwin1.dll. If you have three process trees, you have three independent copies of the nsswitch information. If you start a fresh process tree, it will see the changes. As long as any process in an existing process tree remains running, all processes in that tree will continue to use the old information. So, what mischief can we perform with /etc/nsswitch.conf? To explain, lets have a look into an /etc/nsswitch.conf file set up to all default values: # /etc/nsswitch.conf passwd: files db group: files db db_prefix: auto db_separator: + db_enum: cache builtin db_cache: yes The first line, starting with a hash '#' is a comment. The hash character starts a comment, just as in shell scripts. Everything up to the end of the line is ignored. So this: foo: bar # baz means, for the entry "foo", do "bar", ignore everything after the hash sign. "baz" is only a comment. The other lines define the available settings. The first word up to a colon is a keyword. Note that the colon *must* follow immediately after the keyword. This is a valid line: foo: bar This is not valid: foo : bar Apart from this restriction, the reminder of the line can have as may spaces and TABs as you like. This is a valid line: foo: bar baz Now let's have a look at the available keywords and settings. The two lines starting with the keywords "passwd" and "group" define where Cygwin gets its passwd and group information from. "files" means, fetch the information from the corresponding file in the /etc directory. "db" means, fetch the information from the Windows account databases, the SAM for local accounts, Active Directory for domain account. Examples: passwd: files Read passwd entries only from /etc/passwd. group: db Read group entries only from SAM/AD. group: files # db Read group entries only from /etc/group ("db" is ignored due to the preceding hash sign). passwd: files db Read passwd entries from /etc/passwd. If a user account isn't found, try to find it in SAM or AD. This is the default for both, passwd and group information. group: db files This is a valid entry, but the order will be ignored by Cygwin. If both, files and db are specified, Cygwin will always try the files first, then the db. The remaining entries define certain aspects of the Windows account database search. "db_prefix" determines how the Cygwin user or group name is created: db_prefix: auto This is the default. If your account is from the primary domain of your machine, or if your machine is a standalone machine (not a domain member), your Cygwin account name is just the Windows account name. If your account is from another domain, or if you're logged in as local user on a domain machine, the Cygwin username will be the combination of Windows domainname and username, with the separator char in between: MY_DOM+username (foreign domain) MACHINE+username (local account) Builtin accounts have just the separator char prepended: +LOCAL +Users Unknown accounts on NFS or Samba shares (that is, accounts which cannot be mapped to Windows user accounts via RFC 2307) get a Cygwin account name consisting of the artificial domains "UNIX_User" or "UNIX_Group" and the uid/gid value, for instance: UNIX_User+0 (root) UNIX_Group+10 (wheel) db_prefx: primary Like "auto", but primary domain accounts will be prepended by the domainname as well. db_prefix: always All accounts, even the builtin accounts, will have the domain name prepended: BUILTIN+Users As a special case, if the Cygwin account name differs from the Windows account name, it will be prepended by the artificial domain name "Posix_User" or "Posix_Group" if db_prefix is set to "always": Posix_User+cygwin_user_name Posix_Group+cygwin_group_name "db_separator" defines the spearator char used to prepend the domain name to the user or group name. The default is '+': MY_DOM+username With "db_separator", you can define any ASCII char except space, tab, carriage return, line feed, and the colon, as separator char. Example: db_separator: \ MY_DOM\username "db_enum" defines the depth of a database search, if an application calls one of the enumeration functions getpwent[10] or getgrent[11]. The problem with these functions is, they neither allow to define how many entries will be enumerated when calling them in a loop, nor do they allow to add some filter criteria. They were designed back in the days, when only /etc/passwd and /etc/group files existed and the number of user accounts on a typical UNIX system was seldomly a three-digit number. These days, with user and group databases sometimes going in the six-digit range, they are a potential burden. For that reason, Cygwin does not enumerate all user or group accounts by default, but rather just a very small list, consisting only of the accounts cached in memory by the current process, as well as a handful of predefined builtin accounts. "db_enum" allows to specify the accounts to enumerate in a fine-grained way. It takes a list of sources as argument: db_enum: source1 source2 ... The recognized sources are the following: none No output from getpwent/getgrent at all. all The opposite. Enumerates accounts from all known sources, including all trusted domains. cache Enumerate all accounts currently cached in memory. builtin Enumerate the predefined builtin accounts for backward compatibility. These are five passwd accounts (SYSTEM, LocalService, NetworkService, Administrators, TrustedInstaller) and two group accounts (SYSTEM and TrustedInstaller). files Enumerate the accounts from /etc/passwd or /etc/group. local Enumerate all accounts from the local SAM. primary Enumerate all accounts from the primary domain. alltrusted Enumerate all accounts from all trusted domains. some.domain Enumerate all accounts from the trusted domain some.domain. The trusted domain can be given as Netbios flat name (MY_DOMAIN) or as dns domain name (my_domain.corp). In contrast to the aforementioned fixed source keywords, distinct domain names are caseinsensitive. Only domains which are actually trusted domains are enumerated. Unknown domains are simply ignored. Please note that getpwent/getgrent do *not* test if an account was already listed from another source, so an account can easily show up twice or three times. Such a test would be rather tricky, nor does the Linux implementation perform such test. Here are a few examples for /etc/nsswitch.conf: db_enum: none No output from getpwent/getgrent at all. The first call to the function immediately returns a NULL pointer. db_enum: cache files Enumerate all accounts cached by the current process, plus all entries from either the /etc/passwd or /etc/group file. db_enum: cache local primary Enumerate all accounts cached by the current process, all accounts from the SAM of the local machine, and all accounts from the primary domain of the machine. db_enum: local primary alltrusted Enumerate the accounts from the machine's SAM, from the primary domain of the machine, and from all trusted domains. db_enum: primary domain1.corp sub.domain.corp domain2.net Enumerate the accounts from the primary domain and from the domains domain1.corp, sub.domain.corp and domain2.net. db_enum: all Enumerate everything and the kitchen sink. "db_cache" allows to specify if user and group data should be cached in the process at all or not. Default is "yes". "no" is an experimental feature. The idea is this. Assuming you're working in a volatile domain environment, and you have long-running processes like sshd. You might want sshd to recognize changes in the existing user entries as soon as they occur. Caching user data in sshd *might* not be helpful in such a scenario. However, the user data which gets changed a lot is typically *not* the data required for login purposes, except for the password, and that doesn't matter in Cygwin's case. So take this feature with a grain of salt. ========== Footnotes: ========== [1] http://cygwin.com/cygwin-ug-net/ntsec.html [2] This may change. Right now the file is read in 32K chunks, but we could easily read the file in 64K chunks and, if we find the file is < 64K anyway, just cache the entire bunch, like before. Not implemented yet, but something to keep in mind. [3] http://msdn.microsoft.com/en-us/library/windows/desktop/aa379166%28v=vs.85%29.aspx http://msdn.microsoft.com/en-us/library/windows/desktop/aa379159%28v=vs.85%29.aspx [4] This is where Cygwin differs from SFU. The reason is that we need the old uid/gid values for backward compatibility. There are Cygwin packages (cron, for instance) who rely on the fact that the uid of SYSTEM is 18. In SFU, these accounts get mapped like the other built in SIDs. [5] http://support.microsoft.com/kb/243330 [6] https://tools.ietf.org/html/rfc2307 [7] https://tools.ietf.org/html/rfc1813 [8] http://msdn.microsoft.com/en-us/library/cc980032.aspx [9] http://linux.die.net/man/5/nsswitch.conf [10] http://linux.die.net/man/3/getpwent [11] http://linux.die.net/man/3/getgrent -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-21 15:33 ` Corinna Vinschen @ 2014-02-21 16:29 ` Ken Brown 2014-02-21 16:31 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Ken Brown @ 2014-02-21 16:29 UTC (permalink / raw) To: cygwin On 2/21/2014 9:55 AM, Corinna Vinschen wrote: > - You want to specify a different login shell than /bin/sh. Typo: You forgot to change /bin/sh to /bin/bash. Ken -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-21 16:29 ` Ken Brown @ 2014-02-21 16:31 ` Corinna Vinschen 0 siblings, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-21 16:31 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 399 bytes --] On Feb 21 11:11, Ken Brown wrote: > On 2/21/2014 9:55 AM, Corinna Vinschen wrote: > >- You want to specify a different login shell than /bin/sh. > > Typo: You forgot to change /bin/sh to /bin/bash. > > Ken Fixed, thank you. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 14:44 Testers needed: New passwd/group handling in Cygwin Corinna Vinschen ` (6 preceding siblings ...) 2014-02-17 17:25 ` Corinna Vinschen @ 2014-02-25 18:25 ` Achim Gratz 2014-02-25 19:16 ` Andrey Repin 2014-02-25 20:25 ` Corinna Vinschen 2014-02-27 19:08 ` Frank Fesevur 8 siblings, 2 replies; 145+ messages in thread From: Achim Gratz @ 2014-02-25 18:25 UTC (permalink / raw) To: cygwin Corinna Vinschen writes: > This is a pretty intrusive change, in need of some serious testing, so > I'd like to ask for volunteers. The latest 2014-02-13 snapshot from > http://cygwin.com/snapshots/ contains the changes, including the latest > bugfix. I've tested the 2014-02-19 snapshot at work. Two problems: 1. Running "id" takes 13 seconds to fetch my 440 group memberships (possibly there are some users that would be in ~10x as many groups). Caching doesn't seem to be effective for this since the next several invocations of "id" take the same time. During most of that time you actually can't ^C the process, lsass is growing a few dozen threads and seems to be talking to the DC. Falling back to use just the /etc files makes this work really fast (much faster than without the snapshot). 2. I use a few volumes on NetApp filers that have security set up so that you can't change attributes. That means POSIX permissions are always listed as "0000". I uausally mount these noacl, but when I access them via their UNC path (for instance when Windows runs a script from a CWD on that volume, then Perl reports false for file test operators (-x, -w) other than existence. Backing out the snapshot reverts to the previous behaviour of these test operators correctly determining that my effective rights (via normal and extended security attributes tied to a group memberships) are sufficient. The shell (bash, tcsh) test operators work correctly, but I don't know what Perl is doing differently. This is independent of whether I use just the /etc files or full AD lookup (switched via /etc/nsswitch.conf) and currently a showstopper for me. I've booby-trapped a few checks to always return true and the actual creation of directories and files then succeeds (as it has always done), but I can't just pull out every one of these from all scripts. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for KORG EX-800 and Poly-800MkII V0.9: http://Synth.Stromeko.net/Downloads.html#KorgSDada -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-25 18:25 ` Achim Gratz @ 2014-02-25 19:16 ` Andrey Repin 2014-02-25 20:04 ` Achim Gratz 2014-02-25 20:25 ` Corinna Vinschen 1 sibling, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-02-25 19:16 UTC (permalink / raw) To: Achim Gratz, cygwin Greetings, Achim Gratz! > 2. I use a few volumes on NetApp filers that have security set up so > that you can't change attributes. That means POSIX permissions are > always listed as "0000". That seems like a bug elsewhere. Being able to change permissions shouldn't restrict you from requesting them. > I uausally mount these noacl, but when I access them via their UNC path (for > instance when Windows runs a script from a CWD on that volume, then Perl > reports false for file test operators (-x, -w) other than existence. You can change that with cygdrive prefix options. Adjust your fstab line for it to include noacl, and operations on UNC paths should run better, according to your setup. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 25.02.2014, <22:45> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-25 19:16 ` Andrey Repin @ 2014-02-25 20:04 ` Achim Gratz 0 siblings, 0 replies; 145+ messages in thread From: Achim Gratz @ 2014-02-25 20:04 UTC (permalink / raw) To: cygwin Andrey Repin writes: > That seems like a bug elsewhere. Being able to change permissions shouldn't > restrict you from requesting them. There is no bug, not in Cygwin nor anywhere else. The standard file attributes are all cleared and since I can't set any of them (a policy which gets inherited) that stays for all files and directories that are created. Before you ask, I can't change any of that. > You can change that with cygdrive prefix options. That'd be a possible workaround, thanks. > Adjust your fstab line for it to include noacl, and operations on UNC paths > should run better, according to your setup. I'd rather want to know why the snapshot triggers such a change in behaviour and why the notion of the shells and Perl don't match. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-25 18:25 ` Achim Gratz 2014-02-25 19:16 ` Andrey Repin @ 2014-02-25 20:25 ` Corinna Vinschen 2014-02-25 20:28 ` Achim Gratz 2014-03-10 18:14 ` Achim Gratz 1 sibling, 2 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-25 20:25 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2877 bytes --] On Feb 25 19:14, Achim Gratz wrote: > Corinna Vinschen writes: > > This is a pretty intrusive change, in need of some serious testing, so > > I'd like to ask for volunteers. The latest 2014-02-13 snapshot from > > http://cygwin.com/snapshots/ contains the changes, including the latest > > bugfix. > > I've tested the 2014-02-19 snapshot at work. Two problems: > > 1. Running "id" takes 13 seconds to fetch my 440 group memberships > (possibly there are some users that would be in ~10x as many groups). > Caching doesn't seem to be effective for this since the next several The stuff in the `id' application is not cached at all. Caching is inherited from the parent process, but the parent never asked for all your groups so it hasn't cached this information. Every invocation of id has to request the group info anew. > invocations of "id" take the same time. During most of that time you > actually can't ^C the process, lsass is growing a few dozen threads and > seems to be talking to the DC. Falling back to use just the /etc files > makes this work really fast (much faster than without the snapshot). Do you have a very slow connection to your DC by any chance? I admit that I never tested with 440 groups, only with about 30 or so, but 13 seconds sounds *very* lame. OTOH, this isn't *quite* unexpected. Right now, the LDAP connection to the DC is opened and closed for every single account request. I wasn't sure yet if the ldap connection should be opened only once per process and then stay open for the rest of the process lifetime. This sounds so much like wasting sockets... > 2. I use a few volumes on NetApp filers that have security set up so > that you can't change attributes. That means POSIX permissions are > always listed as "0000". I uausally mount these noacl, but when I > access them via their UNC path (for instance when Windows runs a script > from a CWD on that volume, then Perl reports false for file test > operators (-x, -w) other than existence. Backing out the snapshot > reverts to the previous behaviour of these test operators correctly > determining that my effective rights (via normal and extended security > attributes tied to a group memberships) are sufficient. The shell > (bash, tcsh) test operators work correctly, but I don't know what Perl > is doing differently. The fact that the shells are doing it right seems to indicate that this isn't a generic problem. I can't debug this, though. Can you see if you can figure out what's going on under the hood? Does strace show anything of interest? Can we perhaps set up some joint debugging via private mail during the next couple of days? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-25 20:25 ` Corinna Vinschen @ 2014-02-25 20:28 ` Achim Gratz 2014-02-25 21:55 ` Corinna Vinschen 2014-03-10 18:14 ` Achim Gratz 1 sibling, 1 reply; 145+ messages in thread From: Achim Gratz @ 2014-02-25 20:28 UTC (permalink / raw) To: cygwin Corinna Vinschen writes: > The stuff in the `id' application is not cached at all. Caching is > inherited from the parent process, but the parent never asked for all > your groups so it hasn't cached this information. Every invocation of > id has to request the group info anew. OK, then I was misunderstanding what caching meant. I was operating under the assumption that there'd be something like a session cache that serves all Cygwin processes in the same process tree. > Do you have a very slow connection to your DC by any chance? I admit > that I never tested with 440 groups, only with about 30 or so, but 13 > seconds sounds *very* lame. That's around 33 lookups per second; not great, but also not bad (I'm far from the only user of that DC obviously). I'll have to check what our web application server is doing (albeit it uses a connection pool to amortize the connection overhead), but I don't think it'll do much more than about 200 peak. The answers can be insanely large depending on what and how you ask, too. In any case, based on the fact that I'm certainly not in as many groups as some other folks, I don't think I can drop file-based operation at the moment (which I really hoped I could). > OTOH, this isn't *quite* unexpected. Right now, the LDAP connection to > the DC is opened and closed for every single account request. I wasn't > sure yet if the ldap connection should be opened only once per process > and then stay open for the rest of the process lifetime. This sounds so > much like wasting sockets... See above, there's a timeout on each pooled connection, again I'll have to ask how long it is (I think somewhere in the single digits seconds range, we don't want to tie up the DC for too long). > The fact that the shells are doing it right seems to indicate that this > isn't a generic problem. I can't debug this, though. Can you see if > you can figure out what's going on under the hood? Does strace show > anything of interest? Can we perhaps set up some joint debugging via > private mail during the next couple of days? I can't dig further into this for the next two weeks. I need to get some work done, so I have rolled the snapshot back for now. I'll get back to you when this is out of the way, thanks for the offer. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-25 20:28 ` Achim Gratz @ 2014-02-25 21:55 ` Corinna Vinschen 2014-02-25 22:59 ` Andrey Repin ` (2 more replies) 0 siblings, 3 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-25 21:55 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 3706 bytes --] On Feb 25 21:25, Achim Gratz wrote: > Corinna Vinschen writes: > > The stuff in the `id' application is not cached at all. Caching is > > inherited from the parent process, but the parent never asked for all > > your groups so it hasn't cached this information. Every invocation of > > id has to request the group info anew. > > OK, then I was misunderstanding what caching meant. I was operating > under the assumption that there'd be something like a session cache that > serves all Cygwin processes in the same process tree. Right now the cache is in the cygheap, so it's per-process, with inheritance from the parent process. I am mulling over adding other caching methods, but I'm reluctant to add another shared memory region of variable and potentially big size to the 32 bit version of Cygwin. I'm not as reluctant to do this for the 64 bit version, but I *am* reluctant to add different code for 32 and 64 bit. > > Do you have a very slow connection to your DC by any chance? I admit > > that I never tested with 440 groups, only with about 30 or so, but 13 > > seconds sounds *very* lame. > > That's around 33 lookups per second; not great, but also not bad (I'm > far from the only user of that DC obviously). I'll have to check what > our web application server is doing (albeit it uses a connection pool to Sorry, I don't grok this. What has a web application server to do with asking a DC for user info? > than about 200 peak. The answers can be insanely large depending on > what and how you ask, too. In any case, based on the fact that I'm > certainly not in as many groups as some other folks, I don't think I can > drop file-based operation at the moment (which I really hoped I could). Erm... how often are you calling id, usually? Also, we're in the early stages of testing this change. The idea is not that you just switch, the idea is that we *test* this and I get enough feedback to be able to ease the biggest pains. Other than that, I just had an in-shower inspiration how to speed up `id' specificially. The getgroups(2) call is in the center of this and I could probably speed up the stuiff a lot by opening the LDAP connection in getgroups only once. Also, more radically, if we drop the functionality to define another group name for a group, we could drop the requirement to open an LDAP connection to fetch group information to the DC entirely(*). This would only affect domain groups, local groups could still have different names. But I'm already wondering for a couple of days if having a Cygwin group name different from the Windows group name is really necessary at all. I added this years ago for fun, but there's no serious reason I can think of which would require to keep up with this. (*) Assuming the group info is cached in the local LSA, which is pretty likely for the groups of the current user. > > The fact that the shells are doing it right seems to indicate that this > > isn't a generic problem. I can't debug this, though. Can you see if > > you can figure out what's going on under the hood? Does strace show > > anything of interest? Can we perhaps set up some joint debugging via > > private mail during the next couple of days? > > I can't dig further into this for the next two weeks. I need to get > some work done, so I have rolled the snapshot back for now. I'll get > back to you when this is out of the way, thanks for the offer. Sigh. Testing in this tempo will take ages. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-25 21:55 ` Corinna Vinschen @ 2014-02-25 22:59 ` Andrey Repin 2014-02-26 3:09 ` Andrey Repin 2014-02-26 9:06 ` Achim Gratz 2 siblings, 0 replies; 145+ messages in thread From: Andrey Repin @ 2014-02-25 22:59 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! > Also, more radically, if we drop the functionality to define another > group name for a group, we could drop the requirement to open an LDAP > connection to fetch group information to the DC entirely(*). This would > only affect domain groups, local groups could still have different > names. But I'm already wondering for a couple of days if having a > Cygwin group name different from the Windows group name is really > necessary at all. If you ask me, it not only unnecessary, but potentially confusing, if not dangerous. Given that you've presumable fixed the group name mangling on locale changes, I think all the necessity for this functionality is now gone. > I added this years ago for fun, but there's no > serious reason I can think of which would require to keep up with this. > (*) Assuming the group info is cached in the local LSA, which is > pretty likely for the groups of the current user. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 26.02.2014, <02:17> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-25 21:55 ` Corinna Vinschen 2014-02-25 22:59 ` Andrey Repin @ 2014-02-26 3:09 ` Andrey Repin 2014-02-26 10:02 ` Corinna Vinschen 2014-02-26 9:06 ` Achim Gratz 2 siblings, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-02-26 3:09 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! > Other than that, I just had an in-shower inspiration how to speed up > `id' specificially. The getgroups(2) call is in the center of this and > I could probably speed up the stuiff a lot by opening the LDAP > connection in getgroups only once. Isn't local SAM database augmented with domain entries, when machine is part of the domain? -- WBR, Andrey Repin (anrdaemon@yandex.ru) 26.02.2014, <06:16> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-26 3:09 ` Andrey Repin @ 2014-02-26 10:02 ` Corinna Vinschen 0 siblings, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-26 10:02 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 793 bytes --] On Feb 26 06:17, Andrey Repin wrote: > Greetings, Corinna Vinschen! > > > Other than that, I just had an in-shower inspiration how to speed up > > `id' specificially. The getgroups(2) call is in the center of this and > > I could probably speed up the stuiff a lot by opening the LDAP > > connection in getgroups only once. > > Isn't local SAM database augmented with domain entries, when machine is part > of the domain? No. The local SAM only ever contains local and well-known accounts. Side-effect: Even the supplementary well-known accounts defined in AD are not resolved by LookupAccountSid(NULL, ...). Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-25 21:55 ` Corinna Vinschen 2014-02-25 22:59 ` Andrey Repin 2014-02-26 3:09 ` Andrey Repin @ 2014-02-26 9:06 ` Achim Gratz 2014-02-26 10:07 ` Corinna Vinschen 2 siblings, 1 reply; 145+ messages in thread From: Achim Gratz @ 2014-02-26 9:06 UTC (permalink / raw) To: cygwin > Sorry, I don't grok this. What has a web application server to do with > asking a DC for user info? We have one of these that does a lot of DC lookups because it authenticates all users. It's also in a much faster network, so I can check there what the lookup rate could be reasonably expected to be. > Erm... how often are you calling id, usually? I'm currently doing this in the login process to figure out whether the prompt should show "root" powers. I'll have to figure out something else to do instead. > Also, we're in the early > stages of testing this change. The idea is not that you just switch, > the idea is that we *test* this and I get enough feedback to be able to > ease the biggest pains. Understood. Until now I had to generate passwd and group files and I was hoping that the need for doing that would go away (I'd also need to talk to our AD folks so they start populating the correct fields). > Other than that, I just had an in-shower inspiration how to speed up > `id' specificially. The getgroups(2) call is in the center of this and > I could probably speed up the stuiff a lot by opening the LDAP > connection in getgroups only once. Thursday? :-) > Also, more radically, if we drop the functionality to define another > group name for a group, we could drop the requirement to open an LDAP > connection to fetch group information to the DC entirely(*). This would > only affect domain groups, local groups could still have different > names. But I'm already wondering for a couple of days if having a > Cygwin group name different from the Windows group name is really > necessary at all. I added this years ago for fun, but there's no > serious reason I can think of which would require to keep up with this. > > (*) Assuming the group info is cached in the local LSA, which is > pretty likely for the groups of the current user. That would also work for me (I don't think I've ever used that feature consciously). > Sigh. Testing in this tempo will take ages. Sorry, but that's not my decision to make in this case. I'll see if I can sneak in something until the end of the week. Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-26 9:06 ` Achim Gratz @ 2014-02-26 10:07 ` Corinna Vinschen 2014-02-26 16:12 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-26 10:07 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2812 bytes --] On Feb 26 08:09, Achim Gratz wrote: > > Sorry, I don't grok this. What has a web application server to do with > > asking a DC for user info? > > We have one of these that does a lot of DC lookups because it authenticates > all users. It's also in a much faster network, so I can check there what > the lookup rate could be reasonably expected to be. > > > Erm... how often are you calling id, usually? > > I'm currently doing this in the login process to figure out whether the > prompt should show "root" powers. I'll have to figure out something else to > do instead. No, you don't. I'm actually doing the same. Let's keep up with this and try to make Cygwin faster in the first place. > > Also, we're in the early > > stages of testing this change. The idea is not that you just switch, > > the idea is that we *test* this and I get enough feedback to be able to > > ease the biggest pains. > > Understood. Until now I had to generate passwd and group files and I was > hoping that the need for doing that would go away (I'd also need to talk to > our AD folks so they start populating the correct fields). Yup. Feedback from AD admins is greatly appreciated. > > Other than that, I just had an in-shower inspiration how to speed up > > `id' specificially. The getgroups(2) call is in the center of this and > > I could probably speed up the stuiff a lot by opening the LDAP > > connection in getgroups only once. > > Thursday? :-) Hmm, probably. > > Also, more radically, if we drop the functionality to define another > > group name for a group, we could drop the requirement to open an LDAP > > connection to fetch group information to the DC entirely(*). This would > > only affect domain groups, local groups could still have different > > names. But I'm already wondering for a couple of days if having a > > Cygwin group name different from the Windows group name is really > > necessary at all. I added this years ago for fun, but there's no > > serious reason I can think of which would require to keep up with this. > > > > (*) Assuming the group info is cached in the local LSA, which is > > pretty likely for the groups of the current user. > > That would also work for me (I don't think I've ever used that feature > consciously). > > > Sigh. Testing in this tempo will take ages. > > Sorry, but that's not my decision to make in this case. I'll see if I can > sneak in something until the end of the week. That would be nice. Especially since your mail was the last straw which keeps me from having to do *shudder* more documentation. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-26 10:07 ` Corinna Vinschen @ 2014-02-26 16:12 ` Corinna Vinschen 2014-02-26 19:14 ` Achim Gratz 2014-02-27 9:08 ` Achim Gratz 0 siblings, 2 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-26 16:12 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 3605 bytes --] On Feb 26 11:02, Corinna Vinschen wrote: > On Feb 26 08:09, Achim Gratz wrote: > > > Sorry, I don't grok this. What has a web application server to do with > > > asking a DC for user info? > > > > We have one of these that does a lot of DC lookups because it authenticates > > all users. It's also in a much faster network, so I can check there what > > the lookup rate could be reasonably expected to be. > > > > > Erm... how often are you calling id, usually? > > > > I'm currently doing this in the login process to figure out whether the > > prompt should show "root" powers. I'll have to figure out something else to > > do instead. > > No, you don't. I'm actually doing the same. Let's keep up with this > and try to make Cygwin faster in the first place. I just created 400 groups in AD, and added myself as member. An `id' on a 32 bit Windows 7 domain member machine in my tiny network consisting only of a handful of Windows VMs and with me as the only real user takes about 3.6 secs with the latest code from CVS, using a non-optimzed Cygwin DLL. > > > Also, we're in the early > > > stages of testing this change. The idea is not that you just switch, > > > the idea is that we *test* this and I get enough feedback to be able to > > > ease the biggest pains. > > > > Understood. Until now I had to generate passwd and group files and I was > > hoping that the need for doing that would go away (I'd also need to talk to > > our AD folks so they start populating the correct fields). > > Yup. Feedback from AD admins is greatly appreciated. > > > > Other than that, I just had an in-shower inspiration how to speed up > > > `id' specificially. The getgroups(2) call is in the center of this and > > > I could probably speed up the stuiff a lot by opening the LDAP > > > connection in getgroups only once. > > > > Thursday? :-) > > Hmm, probably. With this patch applied, the aforementioned `id' now takes about 1.9 secs, in an otherwise identical scenario. > > > Also, more radically, if we drop the functionality to define another > > > group name for a group, we could drop the requirement to open an LDAP > > > connection to fetch group information to the DC entirely(*). This would > > > only affect domain groups, local groups could still have different > > > names. But I'm already wondering for a couple of days if having a > > > Cygwin group name different from the Windows group name is really > > > necessary at all. I added this years ago for fun, but there's no > > > serious reason I can think of which would require to keep up with this. > > > > > > (*) Assuming the group info is cached in the local LSA, which is > > > pretty likely for the groups of the current user. > > > > That would also work for me (I don't think I've ever used that feature > > consciously). With this patch applied as well, `id' now takes constantly 0.4 secs. Note that this speedup is only possible when fetching lots of group account information. For user accounts we still need the info from AD, but apart from the getpwent functionality, which can be restricted via the db_enum setting in /etc/nsswitch.conf, there's not very often a good reason to fetch information for hundreds of user accounts. Anyway, if I send you the link to two DLLs with these patches, would you mind to test their speed in your environment? Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-26 16:12 ` Corinna Vinschen @ 2014-02-26 19:14 ` Achim Gratz 2014-02-27 9:08 ` Achim Gratz 1 sibling, 0 replies; 145+ messages in thread From: Achim Gratz @ 2014-02-26 19:14 UTC (permalink / raw) To: cygwin Corinna Vinschen writes: > I just created 400 groups in AD, and added myself as member. An `id' on > a 32 bit Windows 7 domain member machine in my tiny network consisting > only of a handful of Windows VMs and with me as the only real user takes > about 3.6 secs with the latest code from CVS, using a non-optimzed > Cygwin DLL. OK, so you do less than 200 lu/s even in that favorable case. Our DC is hit by some four figure number of clients I suppose. I've asked my colleague to check the lookup rate on our test web server and he also gets around 30 lu/s with caching disabled, just like I did via Cygwin. So the network speed isn't the limiting factor. > With this patch applied, the aforementioned `id' now takes about 1.9 > secs, in an otherwise identical scenario. Sounds interesting. > With this patch applied as well, `id' now takes constantly 0.4 secs. Gets even better… :-) > Note that this speedup is only possible when fetching lots of group > account information. For user accounts we still need the info from AD, > but apart from the getpwent functionality, which can be restricted via > the db_enum setting in /etc/nsswitch.conf, there's not very often a good > reason to fetch information for hundreds of user accounts. > > Anyway, if I send you the link to two DLLs with these patches, would you > mind to test their speed in your environment? Bring it on. I'd need 32bit DLL since I have to keep the 64bit Cygwin running on my machine. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for KORG EX-800 and Poly-800MkII V0.9: http://Synth.Stromeko.net/Downloads.html#KorgSDada -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-26 16:12 ` Corinna Vinschen 2014-02-26 19:14 ` Achim Gratz @ 2014-02-27 9:08 ` Achim Gratz 2014-02-27 9:49 ` Achim Gratz 1 sibling, 1 reply; 145+ messages in thread From: Achim Gratz @ 2014-02-27 9:08 UTC (permalink / raw) To: cygwin Corinna Vinschen writes: [...] > With this patch applied, the aforementioned `id' now takes about 1.9 > secs, in an otherwise identical scenario. [...] > With this patch applied as well, `id' now takes constantly 0.4 secs. > > Note that this speedup is only possible when fetching lots of group > account information. For user accounts we still need the info from AD, > but apart from the getpwent functionality, which can be restricted via > the db_enum setting in /etc/nsswitch.conf, there's not very often a good > reason to fetch information for hundreds of user accounts. > > Anyway, if I send you the link to two DLLs with these patches, would you > mind to test their speed in your environment? Thanks. I'm basically seeing the same with your DLL snapshots, plus the login times (each with an invocation of "id") are following suit. It also seems that our DC is a bit less loaded today. Here are the results of doing this: foreach p ( `seq 1 10` ) time id >/dev/null end "stock" 0.561u 0.514s 0:08.93 11.9% 0+0k 0+0io 7690pf+0w 0.499u 0.499s 0:09.04 10.8% 0+0k 0+0io 7717pf+0w 0.592u 0.686s 0:09.03 14.0% 0+0k 0+0io 7690pf+0w 0.639u 0.576s 0:09.18 13.0% 0+0k 0+0io 7698pf+0w 0.608u 0.390s 0:09.11 10.8% 0+0k 0+0io 7691pf+0w 0.577u 0.514s 0:09.03 11.9% 0+0k 0+0io 7048pf+0w 0.499u 0.468s 0:08.91 10.6% 0+0k 0+0io 7724pf+0w 0.498u 0.561s 0:08.84 11.8% 0+0k 0+0io 7744pf+0w 0.561u 0.405s 0:08.73 10.9% 0+0k 0+0io 6664pf+0w 0.452u 0.545s 0:08.96 11.0% 0+0k 0+0io 6968pf+0w "getgroups" 0.249u 0.202s 0:03.33 13.2% 0+0k 0+0io 3506pf+0w 0.234u 0.296s 0:03.33 15.6% 0+0k 0+0io 3489pf+0w 0.171u 0.296s 0:03.26 14.1% 0+0k 0+0io 3493pf+0w 0.171u 0.265s 0:03.25 13.2% 0+0k 0+0io 3486pf+0w 0.156u 0.358s 0:03.33 15.0% 0+0k 0+0io 3486pf+0w 0.171u 0.265s 0:03.29 13.0% 0+0k 0+0io 3487pf+0w 0.311u 0.373s 0:03.46 19.6% 0+0k 0+0io 3506pf+0w 0.171u 0.217s 0:03.27 11.6% 0+0k 0+0io 3488pf+0w 0.234u 0.202s 0:03.36 12.7% 0+0k 0+0io 3491pf+0w 0.218u 0.202s 0:03.42 11.9% 0+0k 0+0io 3487pf+0w "noldap" 0.249u 0.202s 0:03.33 13.2% 0+0k 0+0io 3506pf+0w 0.234u 0.296s 0:03.33 15.6% 0+0k 0+0io 3489pf+0w 0.171u 0.296s 0:03.26 14.1% 0+0k 0+0io 3493pf+0w 0.171u 0.265s 0:03.25 13.2% 0+0k 0+0io 3486pf+0w 0.156u 0.358s 0:03.33 15.0% 0+0k 0+0io 3486pf+0w 0.171u 0.265s 0:03.29 13.0% 0+0k 0+0io 3487pf+0w 0.311u 0.373s 0:03.46 19.6% 0+0k 0+0io 3506pf+0w 0.171u 0.217s 0:03.27 11.6% 0+0k 0+0io 3488pf+0w 0.234u 0.202s 0:03.36 12.7% 0+0k 0+0io 3491pf+0w 0.218u 0.202s 0:03.42 11.9% 0+0k 0+0io 3487pf+0w Best regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-27 9:08 ` Achim Gratz @ 2014-02-27 9:49 ` Achim Gratz 2014-02-27 9:58 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Achim Gratz @ 2014-02-27 9:49 UTC (permalink / raw) To: cygwin Achim Gratz writes: > "noldap" Sorry, copied the "getgroups" data again, this is the data for "noldap" of course: 0.171u 0.015s 0:01.03 17.4% 0+0k 0+0io 3298pf+0w 0.093u 0.140s 0:00.97 23.7% 0+0k 0+0io 3292pf+0w 0.046u 0.093s 0:00.97 13.4% 0+0k 0+0io 3285pf+0w 0.062u 0.140s 0:00.97 20.6% 0+0k 0+0io 3283pf+0w 0.031u 0.124s 0:00.97 15.4% 0+0k 0+0io 3285pf+0w 0.093u 0.077s 0:00.99 16.1% 0+0k 0+0io 3291pf+0w 0.093u 0.077s 0:00.96 16.6% 0+0k 0+0io 3298pf+0w 0.015u 0.077s 0:00.97 8.2% 0+0k 0+0io 3291pf+0w 0.093u 0.108s 0:00.99 19.1% 0+0k 0+0io 3300pf+0w 0.062u 0.109s 0:00.97 16.4% 0+0k 0+0io 3294pf+0w Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-27 9:49 ` Achim Gratz @ 2014-02-27 9:58 ` Corinna Vinschen 2014-02-27 13:25 ` Achim Gratz 2014-02-28 15:49 ` Achim Gratz 0 siblings, 2 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-27 9:58 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1363 bytes --] Hi Achim, thanks for testing. On Feb 27 09:07, Achim Gratz wrote: > Achim Gratz writes: > > "noldap" > > Sorry, copied the "getgroups" data again, this is the data for "noldap" of > course: > > 0.171u 0.015s 0:01.03 17.4% 0+0k 0+0io 3298pf+0w > 0.093u 0.140s 0:00.97 23.7% 0+0k 0+0io 3292pf+0w > 0.046u 0.093s 0:00.97 13.4% 0+0k 0+0io 3285pf+0w > 0.062u 0.140s 0:00.97 20.6% 0+0k 0+0io 3283pf+0w > 0.031u 0.124s 0:00.97 15.4% 0+0k 0+0io 3285pf+0w > 0.093u 0.077s 0:00.99 16.1% 0+0k 0+0io 3291pf+0w > 0.093u 0.077s 0:00.96 16.6% 0+0k 0+0io 3298pf+0w > 0.015u 0.077s 0:00.97 8.2% 0+0k 0+0io 3291pf+0w > 0.093u 0.108s 0:00.99 19.1% 0+0k 0+0io 3300pf+0w > 0.062u 0.109s 0:00.97 16.4% 0+0k 0+0io 3294pf+0w 1 second? That sounds still a bit slow. Considering that I'm now member of 414 groups, and you are member of 440 groups, the extra number of groups cannot account for that. This sounds surprisingly as if the names of some of your groups are not cached on your machine. Or something. Or is this a rather slow machine?!? Still, it seems like the right thing to do to drop the group name configuration stuff entirely. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-27 9:58 ` Corinna Vinschen @ 2014-02-27 13:25 ` Achim Gratz 2014-02-27 15:09 ` Corinna Vinschen 2014-02-28 15:49 ` Achim Gratz 1 sibling, 1 reply; 145+ messages in thread From: Achim Gratz @ 2014-02-27 13:25 UTC (permalink / raw) To: cygwin Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes: > 1 second? That sounds still a bit slow. Considering that I'm now > member of 414 groups, and you are member of 440 groups, the extra number > of groups cannot account for that. > > This sounds surprisingly as if the > names of some of your groups are not cached on your machine. Or > something. Or is this a rather slow machine?!? It's not a slow machine by any means, but it certainly gets its fair share of security policies, so it may have something to do with that. I don't know. > Still, it seems like the right thing to do to drop the group name > configuration stuff entirely. Yes (unless you'd want to make it configurable like the getpwent stuff). Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-27 13:25 ` Achim Gratz @ 2014-02-27 15:09 ` Corinna Vinschen 2014-02-27 23:20 ` Andrey Repin 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-27 15:09 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1882 bytes --] On Feb 27 12:54, Achim Gratz wrote: > Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes: > > 1 second? That sounds still a bit slow. Considering that I'm now > > member of 414 groups, and you are member of 440 groups, the extra number > > of groups cannot account for that. > > > > This sounds surprisingly as if the > > names of some of your groups are not cached on your machine. Or > > something. Or is this a rather slow machine?!? > > It's not a slow machine by any means, but it certainly gets its fair share > of security policies, so it may have something to do with that. I don't know. > > > Still, it seems like the right thing to do to drop the group name > > configuration stuff entirely. > > Yes (unless you'd want to make it configurable like the getpwent stuff). Nah, not really. As I said, I'm questioning some of the old functionality anyway, and the less we have to ask AD the better for us. I applied my patch which removes this group name change facility from AD and uploaded a new snapshot to http://cygwin.com/snapshots/. While we're at it, I just had this weird idea. What if, as soon as the first Cygwin process in a process tree starts, this process not only caches the primary group info, but also caches all supplementary groups from the user's token? This would slow down startup of the first process slightly, but it would speed up any subsequent request for group information of a group in the user's token. An `id' call would be almost instant, and `ls' calls would probably be faster as well. As always, there's a trade-off: Users running cygwin processes from CMD a lot would encounter a slowdown. What do you think, guys? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-27 15:09 ` Corinna Vinschen @ 2014-02-27 23:20 ` Andrey Repin 2014-02-28 12:09 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-02-27 23:20 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! >> > 1 second? That sounds still a bit slow. Considering that I'm now >> > member of 414 groups, and you are member of 440 groups, the extra number >> > of groups cannot account for that. >> > >> > This sounds surprisingly as if the >> > names of some of your groups are not cached on your machine. Or >> > something. Or is this a rather slow machine?!? >> >> It's not a slow machine by any means, but it certainly gets its fair share >> of security policies, so it may have something to do with that. I don't know. >> >> > Still, it seems like the right thing to do to drop the group name >> > configuration stuff entirely. >> >> Yes (unless you'd want to make it configurable like the getpwent stuff). > Nah, not really. As I said, I'm questioning some of the old functionality > anyway, and the less we have to ask AD the better for us. > I applied my patch which removes this group name change facility from AD > and uploaded a new snapshot to http://cygwin.com/snapshots/. > While we're at it, I just had this weird idea. > What if, as soon as the first Cygwin process in a process tree starts, > this process not only caches the primary group info, but also caches all > supplementary groups from the user's token? This would slow down > startup of the first process slightly, but it would speed up any > subsequent request for group information of a group in the user's token. > An `id' call would be almost instant, and `ls' calls would probably be > faster as well. Umhm. I'd have to see, how slightly. Because my primary use of Cygwin tools does not expect to have a parent Cygwin process. (I.e. diff'ing between file manager panels.) > As always, there's a trade-off: Users running cygwin processes from CMD > a lot would encounter a slowdown. > What do you think, guys? Needs some tests. I would certainly appreciate faster initial startups, but if the delay is manageable, I can cope with more long-term friendly approach. I've got entangled in current work and lost track of snapshots for now. If there's some DEF's in library, that I can toggle and rebuild it with certain features enabled and disable for local test, I'd be glad to have a round at it over weekend. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 28.02.2014, <03:06> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-27 23:20 ` Andrey Repin @ 2014-02-28 12:09 ` Corinna Vinschen 2014-02-28 20:10 ` Achim Gratz 2014-02-28 22:35 ` Andrey Repin 0 siblings, 2 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-28 12:09 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2141 bytes --] On Feb 28 03:12, Andrey Repin wrote: > Greetings, Corinna Vinschen! > > While we're at it, I just had this weird idea. > > > What if, as soon as the first Cygwin process in a process tree starts, > > this process not only caches the primary group info, but also caches all > > supplementary groups from the user's token? This would slow down > > startup of the first process slightly, but it would speed up any > > subsequent request for group information of a group in the user's token. > > An `id' call would be almost instant, and `ls' calls would probably be > > faster as well. > > Umhm. I'd have to see, how slightly. Because my primary use of Cygwin tools > does not expect to have a parent Cygwin process. (I.e. diff'ing between > file manager panels.) > > > As always, there's a trade-off: Users running cygwin processes from CMD > > a lot would encounter a slowdown. > > > What do you think, guys? > > Needs some tests. I would certainly appreciate faster initial startups, but if > the delay is manageable, I can cope with more long-term friendly approach. > I've got entangled in current work and lost track of snapshots for now. > If there's some DEF's in library, that I can toggle and rebuild it with certain > features enabled and disable for local test, I'd be glad to have a round at it > over weekend. I've just uploaded a new snapshot to http://cygwin.com/snapshots/ It allows you to do the following in /etc/nsswitch.conf: db_cache: no No caching of passwd and group data at all db_cache: yes Caching of passwd and group data, no initial caching of the user's supplementary groups from the user token. db_cache: full This is the default now. Caching of passwd and group data *plus* initial caching of all groups from the user token. Please try it out. Average timings from your environment (process startup, and calling `id') would be nice. Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-28 12:09 ` Corinna Vinschen @ 2014-02-28 20:10 ` Achim Gratz 2014-02-28 20:29 ` Corinna Vinschen 2014-02-28 22:35 ` Andrey Repin 1 sibling, 1 reply; 145+ messages in thread From: Achim Gratz @ 2014-02-28 20:10 UTC (permalink / raw) To: cygwin Corinna Vinschen writes: > It allows you to do the following in /etc/nsswitch.conf: > > db_cache: no Using the 15:31 snapshot DLL again via VPN, id dumps core after about 2:30 minutes. > No caching of passwd and group data at all > > db_cache: yes Startup of a shell in mintty takes about 20 seconds, each id around 10...13s. > Caching of passwd and group data, no initial caching of the user's > supplementary groups from the user token. > > db_cache: full Startup in around 10seconds, each id takes a third of a second. > This is the default now. A good choice, I'd say. :-) Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf rackAttack: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-28 20:10 ` Achim Gratz @ 2014-02-28 20:29 ` Corinna Vinschen 2014-02-28 21:08 ` Frank Fesevur 2014-03-01 8:59 ` Achim Gratz 0 siblings, 2 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-28 20:29 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1633 bytes --] On Feb 28 20:49, Achim Gratz wrote: > Corinna Vinschen writes: > > It allows you to do the following in /etc/nsswitch.conf: > > > > db_cache: no > > Using the 15:31 snapshot DLL again via VPN, id dumps core after about > 2:30 minutes. How? Details? Stackdump? It works for me(TM). The timing only shows that it's not the right thing for you, or that in the long run the non-caching option should just go away. For the time being, though, I need *details*. > > No caching of passwd and group data at all > > > > db_cache: yes > > Startup of a shell in mintty takes about 20 seconds, each id around 10...13s. > > > Caching of passwd and group data, no initial caching of the user's > > supplementary groups from the user token. > > > > db_cache: full > > Startup in around 10seconds, each id takes a third of a second. Hang on. Are you telling me this snapshot takes longer than the snapshot from yesterday? And you're telling me that normal caching, as was the default all the time, is slower in startup than with the additional caching of groups at startup? Is your startup procedure calling `id' twice by any chance? The problem with your numbers is that I have nothing really to compare it with. It would be helpful to know what you're doing, the essential parts of your startup for instance. Please, please, please, if something doesn't work as expected, please add meat to your reports. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-28 20:29 ` Corinna Vinschen @ 2014-02-28 21:08 ` Frank Fesevur 2014-02-28 21:13 ` Corinna Vinschen 2014-03-01 8:59 ` Achim Gratz 1 sibling, 1 reply; 145+ messages in thread From: Frank Fesevur @ 2014-02-28 21:08 UTC (permalink / raw) To: cygwin 2014-02-28 18:14 GMT+01:00 Corinna Vinschen: > I made some tests myself today, while debugging Frank's problem. If I I updated to the snapshot of today (28th) and now the message is not shown anymore. Thanks for the fix. I have yet another observation. When I am not connected with VPN it takes a while (don't know how to time exactly, around 3 seconds) after I double click the shortcut for mintty to show up. When connect it shows up almost instantly. Tested 24th, 27th and 28th snapshots and see the problem with all of them. Regards, Frank -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-28 21:08 ` Frank Fesevur @ 2014-02-28 21:13 ` Corinna Vinschen 2014-02-28 21:50 ` Corinna Vinschen 2014-03-02 13:21 ` Frank Fesevur 0 siblings, 2 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-28 21:13 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1085 bytes --] On Feb 28 21:53, Frank Fesevur wrote: > 2014-02-28 18:14 GMT+01:00 Corinna Vinschen: > > I made some tests myself today, while debugging Frank's problem. If I > > I updated to the snapshot of today (28th) and now the message is not > shown anymore. > Thanks for the fix. Thanks for the report. > I have yet another observation. When I am not connected with VPN it > takes a while (don't know how to time exactly, around 3 seconds) after > I double click the shortcut for mintty to show up. When connect it > shows up almost instantly. Tested 24th, 27th and 28th snapshots and > see the problem with all of them. That's not really a problem but a case of "it is as it is". To get the user and group info, Cygwin has to contact the DC and/or GC and then runs into a timeout. Right now, the LDAP timeout is set to 3 seconds. I don't know yet if it's such a bright idea to lower this timeout value. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-28 21:13 ` Corinna Vinschen @ 2014-02-28 21:50 ` Corinna Vinschen 2014-03-02 13:21 ` Frank Fesevur 1 sibling, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-28 21:50 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1723 bytes --] On Feb 28 22:08, Corinna Vinschen wrote: > On Feb 28 21:53, Frank Fesevur wrote: > > 2014-02-28 18:14 GMT+01:00 Corinna Vinschen: > > > I made some tests myself today, while debugging Frank's problem. If I > > > > I updated to the snapshot of today (28th) and now the message is not > > shown anymore. > > Thanks for the fix. > > Thanks for the report. > > > I have yet another observation. When I am not connected with VPN it > > takes a while (don't know how to time exactly, around 3 seconds) after > > I double click the shortcut for mintty to show up. When connect it > > shows up almost instantly. Tested 24th, 27th and 28th snapshots and > > see the problem with all of them. > > That's not really a problem but a case of "it is as it is". To get the > user and group info, Cygwin has to contact the DC and/or GC and then > runs into a timeout. Right now, the LDAP timeout is set to 3 seconds. > I don't know yet if it's such a bright idea to lower this timeout value. Btw., if the timeout is unbearable, it might be an interesting idea to switch on reading passwd and group info from the files when the VPN is not available. The last snapshots contain a new version of the mkpasswd and mkgroup tools which produce passwd and group entries with the same values (especially uid and gid values!) as the "db" setting in Cygwin does. The idea is that files and db don't contradict each other by default, rather they provide the same values. Therefore it might be a nice addition to use the files in offline mode. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-28 21:13 ` Corinna Vinschen 2014-02-28 21:50 ` Corinna Vinschen @ 2014-03-02 13:21 ` Frank Fesevur 2014-03-03 9:21 ` Corinna Vinschen 1 sibling, 1 reply; 145+ messages in thread From: Frank Fesevur @ 2014-03-02 13:21 UTC (permalink / raw) To: cygwin 2014-02-28 22:08 GMT+01:00 Corinna Vinschen: > That's not really a problem but a case of "it is as it is". To get the > user and group info, Cygwin has to contact the DC and/or GC and then > runs into a timeout. Right now, the LDAP timeout is set to 3 seconds. > I don't know yet if it's such a bright idea to lower this timeout value. I understand why it happens. And three seconds are not the problem. It is just a bit confusing to activate the shortcut and for 3 seconds nothing seems to happen. First time I double-clicked again because I though it wasn't working at all. Obviously a bit later two terminal showed up. If mintty would should up empty and "wait" those three seconds before something happens it is more clear it just takes some startup time. But maybe that is more a mintty problem. Regards, Frank -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-02 13:21 ` Frank Fesevur @ 2014-03-03 9:21 ` Corinna Vinschen 2014-03-03 12:19 ` Frank Fesevur ` (2 more replies) 0 siblings, 3 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-03-03 9:21 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1841 bytes --] On Mar 2 14:20, Frank Fesevur wrote: > 2014-02-28 22:08 GMT+01:00 Corinna Vinschen: > > That's not really a problem but a case of "it is as it is". To get the > > user and group info, Cygwin has to contact the DC and/or GC and then > > runs into a timeout. Right now, the LDAP timeout is set to 3 seconds. > > I don't know yet if it's such a bright idea to lower this timeout value. > > I understand why it happens. And three seconds are not the problem. > > It is just a bit confusing to activate the shortcut and for 3 seconds > nothing seems to happen. First time I double-clicked again because I > though it wasn't working at all. Obviously a bit later two terminal > showed up. If mintty would should up empty and "wait" those three > seconds before something happens it is more clear it just takes some > startup time. But maybe that is more a mintty problem. No, this is not mintty's fault. You have to understand that the request for information from the DC is called before the actual application had any chance to initialize, This happens before the application has, in fact, been called from the Cygwin DLL. Therefore, if the three seconds are a burden, there are three choices going forward: - We reduce the timeout to 1 second in general, which could be a problem when utilizing a slow VPN conncetion to the DC, - We allow the user to specify a timeout value and set the default to 1 second. - We stick to the 3 seconds and users should switch to files (/etc/passwd /etc/group) when being offline from their DC. I'm really not sure what the best way is, I guess we as a community have to dice this out somehow. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-03 9:21 ` Corinna Vinschen @ 2014-03-03 12:19 ` Frank Fesevur 2014-03-03 12:29 ` Henry S. Thompson 2014-03-03 17:06 ` Andrey Repin 2014-03-03 18:24 ` Andy Hall 2 siblings, 1 reply; 145+ messages in thread From: Frank Fesevur @ 2014-03-03 12:19 UTC (permalink / raw) To: cygwin 2014-03-03 10:21 GMT+01:00 Corinna Vinschen: > No, this is not mintty's fault. You have to understand that the > request for information from the DC is called before the actual > application had any chance to initialize, This happens before > the application has, in fact, been called from the Cygwin DLL. Understood. > Therefore, if the three seconds are a burden, there are three choices > going forward: > > - We reduce the timeout to 1 second in general, which could be a problem > when utilizing a slow VPN conncetion to the DC, > > - We allow the user to specify a timeout value and set the default to > 1 second. > > - We stick to the 3 seconds and users should switch to files (/etc/passwd > /etc/group) when being offline from their DC. > > I'm really not sure what the best way is, I guess we as a community have > to dice this out somehow. I think the second one makes most sense. Maybe even keep the default timeout on 3 seconds. When the user can change the value, it is up to them to play with it and figure out what works best for them. I personally would try to set it to 1 second, since I almost never have a slow VPN. Regards, Frank -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-03 12:19 ` Frank Fesevur @ 2014-03-03 12:29 ` Henry S. Thompson 2014-03-03 13:19 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Henry S. Thompson @ 2014-03-03 12:29 UTC (permalink / raw) To: cygwin Frank Fesevur writes: > 2014-03-03 10:21 GMT+01:00 Corinna Vinschen: >> - We allow the user to specify a timeout value and set the default to >> 1 second. > I think the [above] makes most sense. Maybe even keep the default > timeout on 3 seconds. When the user can change the value, it is up to > them to play with it and figure out what works best for them. I > personally would try to set it to 1 second, since I almost never have > a slow VPN. As someone who _never_ cares about group membership at all (as far as I know), and runs on a network where no domain server will ever respond, can I request that the default setup be carefully considered, if the proposal above would mean me always waiting 1 second, to say nothing of 3 seconds, to launch anything. Should we be considering some form of configuration tool, similar to ssh-host-config, which will set appropriate configuration values for a range of common situations, including the "no domain controller" one? Not sure I can volunteer, as I haven't been following the details up to this point because the functionality is not one I use. ht -- Henry S. Thompson, School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB, SCOTLAND -- (44) 131 650-4440 Fax: (44) 131 650-4587, e-mail: ht@inf.ed.ac.uk URL: http://www.ltg.ed.ac.uk/~ht/ [mail from me _always_ has a .sig like this -- mail without it is forged spam] -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-03 12:29 ` Henry S. Thompson @ 2014-03-03 13:19 ` Corinna Vinschen 0 siblings, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-03-03 13:19 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 929 bytes --] On Mar 3 12:29, Henry S. Thompson wrote: > Frank Fesevur writes: > > > 2014-03-03 10:21 GMT+01:00 Corinna Vinschen: > >> - We allow the user to specify a timeout value and set the default to > >> 1 second. > > > I think the [above] makes most sense. Maybe even keep the default > > timeout on 3 seconds. When the user can change the value, it is up to > > them to play with it and figure out what works best for them. I > > personally would try to set it to 1 second, since I almost never have > > a slow VPN. > > As someone who _never_ cares about group membership at all (as far as > I know), and runs on a network where no domain server will ever > respond, If you don't have a domain member machine, you don't have this problem anyway. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-03 9:21 ` Corinna Vinschen 2014-03-03 12:19 ` Frank Fesevur @ 2014-03-03 17:06 ` Andrey Repin 2014-03-04 8:07 ` Warren Young 2014-03-03 18:24 ` Andy Hall 2 siblings, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-03-03 17:06 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! > No, this is not mintty's fault. You have to understand that the > request for information from the DC is called before the actual > application had any chance to initialize, This happens before > the application has, in fact, been called from the Cygwin DLL. > Therefore, if the three seconds are a burden, there are three choices > going forward: > - We reduce the timeout to 1 second in general, which could be a problem > when utilizing a slow VPN conncetion to the DC, > - We allow the user to specify a timeout value and set the default to > 1 second. > - We stick to the 3 seconds and users should switch to files (/etc/passwd > /etc/group) when being offline from their DC. > I'm really not sure what the best way is, I guess we as a community have > to dice this out somehow. I'd say it again, "sane defaults are better, than alot of options". 1 second to CONNECT to domain server is more than enough time under normal circumstances (for comparison, default DNS _roundtrip_ timeout is 2 seconds, that for a whole conversation sequence). Leave the connect timeout short, and if certain users are unable to cope with slow connections to their domain controllers, they can use less pre-caching, or switch to local files to speed up uid/gid resolution. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 03.03.2014, <18:40> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-03 17:06 ` Andrey Repin @ 2014-03-04 8:07 ` Warren Young 2014-03-04 8:08 ` Andrey Repin 0 siblings, 1 reply; 145+ messages in thread From: Warren Young @ 2014-03-04 8:07 UTC (permalink / raw) To: cygwin On 3/3/2014 08:52, Andrey Repin wrote: > > I'd say it again, "sane defaults are better, than alot of options". Agreed in principle. However, observe that all network stacks have a bunch of built-in timeout options. They're rarely exposed to the user level, but their defaults are typically quite high. (e.g. 60 seconds for connection timeout.) Over the past 3 decades of TCP/IP, we've discovered that networks are weird. > for comparison, default DNS _roundtrip_ timeout is 2 seconds, The typical DNS transaction is just 2 UDP packets, one each direction: query and response. I tested a simple, unencrypted LDAP login-and-drop-conn transaction here against a real production AD server, and it required 8 packets, 5 of which were TCP/IP connection establishment and shutdown. Add in the encryption, authentication, and authorization overheads of a "real" LDAP query, and it could go up to dozens of packets. That said, it only took about 1 ms to my simple test. The AD server was on the other side of a router, on a fast WAN. Someone testing this new cygwin1.dll in a domain environment[*] should do a packet capture of what Windows sends when the DLL does its new thing. The captured data isn't terribly interesting here. What we want to know is how many packets it takes, and what the timestamps are on the captured frames. Most especially, the delta between the first and last packets, but if there are any significant time gaps, that could be interesting, too. [*] Not me. The only reason we have any AD around here at all is for testing software that authenticates users against third party AD servers. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-04 8:07 ` Warren Young @ 2014-03-04 8:08 ` Andrey Repin 2014-03-04 8:10 ` Warren Young 0 siblings, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-03-04 8:08 UTC (permalink / raw) To: Warren Young, cygwin Greetings, Warren Young! >> I'd say it again, "sane defaults are better, than alot of options". > Agreed in principle. > However, observe that all network stacks have a bunch of built-in > timeout options. They're rarely exposed to the user level, but their > defaults are typically quite high. (e.g. 60 seconds for connection > timeout.) I've tried to rely on some obscure "system timeout"... it didn't worked well. Socket just doesn't die by itself after hours of waiting. I had to manually enforce some sane timeout to ensure consistent operation of the program. Even if it'd be something like "15 minutes", it was still better than "I don't know, may be next century..." > Over the past 3 decades of TCP/IP, we've discovered that > networks are weird. I concur. >> for comparison, default DNS _roundtrip_ timeout is 2 seconds, > The typical DNS transaction is just 2 UDP packets, one each direction: > query and response. > I tested a simple, unencrypted LDAP login-and-drop-conn transaction here > against a real production AD server, and it required 8 packets, 5 of > which were TCP/IP connection establishment and shutdown. > Add in the encryption, authentication, and authorization overheads of a > "real" LDAP query, and it could go up to dozens of packets. > That said, it only took about 1 ms to my simple test. The AD server was > on the other side of a router, on a fast WAN. Perhaps, you didn't understand, but I specifically mentioned connection time, which is clearly distinct in case of TCP connection. Once connection is established, control is returned to application, the connection timeout is a thing of past, and session can only be terminated by a request of calling application. > Someone testing this new cygwin1.dll in a domain environment[*] should > do a packet capture of what Windows sends when the DLL does its new thing. > The captured data isn't terribly interesting here. What we want to know > is how many packets it takes, and what the timestamps are on the > captured frames. Most especially, the delta between the first and last > packets, but if there are any significant time gaps, that could be > interesting, too. Amount of packets doesn't matter. Once TCP session is established, it remains, until closed or dropped on either end of the wire. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 04.03.2014, <05:25> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-04 8:08 ` Andrey Repin @ 2014-03-04 8:10 ` Warren Young 2014-03-04 8:19 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Warren Young @ 2014-03-04 8:10 UTC (permalink / raw) To: cygwin On 3/3/2014 18:36, Andrey Repin wrote: > Once TCP session is established, it remains, > until closed or dropped on either end of the wire. Have you done the packet capture to prove that Windows does in fact keep the LDAP connection to the AD server up continually, or are you making an assumption? > Amount of packets doesn't matter Even if the connection stays up all the time, an LDAP lookup is almost certain to take more than one packet each way. Packet round trip time (RTT) varies by orders of magnitude among networks, from sub-ms on a fast, quiet LAN to on the order of a full second for a slow WAN. Since the number of packets is a function of the number of round trips, a network with an RTT of 0.9s will take more than the proposed 3 seconds to process a query that requires 4 round trips, even if the processing delay on either end is essentially 0. You have to measure it to find out. And yes, I have personally used networks with an RTT > 1.0s. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-04 8:10 ` Warren Young @ 2014-03-04 8:19 ` Corinna Vinschen 2014-03-05 16:47 ` Warren Young 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-03-04 8:19 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1320 bytes --] On Mar 3 22:07, Warren Young wrote: > On 3/3/2014 18:36, Andrey Repin wrote: > >Once TCP session is established, it remains, > >until closed or dropped on either end of the wire. > > Have you done the packet capture to prove that Windows does in fact > keep the LDAP connection to the AD server up continually, or are you > making an assumption? > > >Amount of packets doesn't matter > > Even if the connection stays up all the time, an LDAP lookup is > almost certain to take more than one packet each way. > > Packet round trip time (RTT) varies by orders of magnitude among > networks, from sub-ms on a fast, quiet LAN to on the order of a full > second for a slow WAN. > > Since the number of packets is a function of the number of round > trips, a network with an RTT of 0.9s will take more than the > proposed 3 seconds to process a query that requires 4 round trips, > even if the processing delay on either end is essentially 0. > > You have to measure it to find out. > > And yes, I have personally used networks with an RTT > 1.0s. I was inclined to go with Andrey's suggestion for simplicity. What's yours? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-04 8:19 ` Corinna Vinschen @ 2014-03-05 16:47 ` Warren Young 2014-03-05 20:18 ` Andrey Repin 2014-03-05 23:09 ` Corinna Vinschen 0 siblings, 2 replies; 145+ messages in thread From: Warren Young @ 2014-03-05 16:47 UTC (permalink / raw) To: cygwin On 3/4/2014 01:07, Corinna Vinschen wrote: > On Mar 3 22:07, Warren Young wrote: >> >> You have to measure it to find out. > > I was inclined to go with Andrey's suggestion for simplicity. What's > yours? Science! Survey actual lookup times, and make a statistical determination from that. e.g. +3 sigma[1] above the normal to capture > 99% of the Cygwin user base, or +6 sigma to reduce outliers to the order of 2 ppm. To collect the data, instrument the DLL and report anonymous statistics back to cygwin.com if the user opts in via setup.exe. Rather than report back continually, just keep a model of the lookup time[2], and report the current lookup time average on each setup.exe run, such as in the HTTP request for the mirror list: GET /mirrors.lst?adluavg=123 That is, setup.exe is reporting that the typical AD lookup time at this site is currently 123 ms. Note that you don't have to write a CGI program to accomplish this. Apache will continue to serve static mirror.lst files for you with the GET parameter.[3] It will dutifully log each query to access_log, and you can dig the reported lookup time values out of the file when you want to compile your statistics. When calculating statistics, use only the latest value for each reporting IP, so that sites that update Cygwin more often don't skew the results. The default timeout should be baked into the DLL, but overridable in nsswitch.conf, with ms resolution. The default shouldn't be in the shipping nsswitch.conf file, because you want DLL updates to be able to provide a new default timeout if your initial guess turns out to be inappropriate. You want ms resolution, since my test above shows at least one case where the normal lookup time is on the order of 1 ms. If cygwin1.dll defaults to 1 s, I'd want to change my local timeout to 10 or 100 ms. [1] https://en.wikipedia.org/wiki/Standard_deviation [2] A decaying average technique should work. Save the last 10 or so lookup times to the registry on DLL shutdown. If there isn't that much lookup data on hand, start with repeats of the last known lookup time, or the default value. [3] Try it: http://cygwin.com/mirrors.lst?foo=bar -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-05 16:47 ` Warren Young @ 2014-03-05 20:18 ` Andrey Repin 2014-03-05 23:09 ` Corinna Vinschen 1 sibling, 0 replies; 145+ messages in thread From: Andrey Repin @ 2014-03-05 20:18 UTC (permalink / raw) To: Warren Young, cygwin Greetings, Warren Young! > On 3/4/2014 01:07, Corinna Vinschen wrote: >> On Mar 3 22:07, Warren Young wrote: >>> >>> You have to measure it to find out. >> >> I was inclined to go with Andrey's suggestion for simplicity. What's >> yours? > Science! > Survey actual lookup times, TCP is not UDP. Lookup time is not connection time. This is third time I say it. Should be enough. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 05.03.2014, <20:00> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-05 16:47 ` Warren Young 2014-03-05 20:18 ` Andrey Repin @ 2014-03-05 23:09 ` Corinna Vinschen 1 sibling, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-03-05 23:09 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1915 bytes --] On Mar 5 08:34, Warren Young wrote: > On 3/4/2014 01:07, Corinna Vinschen wrote: > >On Mar 3 22:07, Warren Young wrote: > >> > >>You have to measure it to find out. > > > >I was inclined to go with Andrey's suggestion for simplicity. What's > >yours? > > Science! > > Survey actual lookup times, and make a statistical determination > from that. e.g. +3 sigma[1] above the normal to capture > 99% of > the Cygwin user base, or +6 sigma to reduce outliers to the order of > 2 ppm. > > To collect the data, instrument the DLL and report anonymous > statistics back to cygwin.com if the user opts in via setup.exe. > > Rather than report back continually, just keep a model of the lookup > time[2], and report the current lookup time average on each > setup.exe run, such as in the HTTP request for the mirror list: > > GET /mirrors.lst?adluavg=123 > > That is, setup.exe is reporting that the typical AD lookup time at > this site is currently 123 ms. > > Note that you don't have to write a CGI program to accomplish this. > Apache will continue to serve static mirror.lst files for you with > the GET parameter.[3] It will dutifully log each query to > access_log, and you can dig the reported lookup time values out of > the file when you want to compile your statistics. > > When calculating statistics, use only the latest value for each > reporting IP, so that sites that update Cygwin more often don't skew > the results. > > The default timeout should be baked into the DLL, but overridable in > nsswitch.conf, with ms resolution. You're kidding, aren't you? Apart from that, ms are not useful since LDAP_OPT_TIMELIMIT used for ldap_bind_s only takes values in 1 second resolution. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* RE: Testers needed: New passwd/group handling in Cygwin 2014-03-03 9:21 ` Corinna Vinschen 2014-03-03 12:19 ` Frank Fesevur 2014-03-03 17:06 ` Andrey Repin @ 2014-03-03 18:24 ` Andy Hall 2 siblings, 0 replies; 145+ messages in thread From: Andy Hall @ 2014-03-03 18:24 UTC (permalink / raw) To: cygwin > On Mar 2 14:20, Frank Fesevur wrote: > > 2014-02-28 22:08 GMT+01:00 Corinna Vinschen: > > > That's not really a problem but a case of "it is as it is". To get the > > > user and group info, Cygwin has to contact the DC and/or GC and then > > > runs into a timeout. Right now, the LDAP timeout is set to 3 seconds. > > > I don't know yet if it's such a bright idea to lower this timeout value. > > > > I understand why it happens. And three seconds are not the problem. > > > > It is just a bit confusing to activate the shortcut and for 3 seconds > > nothing seems to happen. First time I double-clicked again because I > > though it wasn't working at all. Obviously a bit later two terminal > > showed up. If mintty would should up empty and "wait" those three > > seconds before something happens it is more clear it just takes some > > startup time. But maybe that is more a mintty problem. > > No, this is not mintty's fault. You have to understand that the > request for information from the DC is called before the actual > application had any chance to initialize, This happens before > the application has, in fact, been called from the Cygwin DLL. > > Therefore, if the three seconds are a burden, there are three choices > going forward: > > - We reduce the timeout to 1 second in general, which could be a problem > when utilizing a slow VPN conncetion to the DC, > > - We allow the user to specify a timeout value and set the default to > 1 second. > > - We stick to the 3 seconds and users should switch to files (/etc/passwd > /etc/group) when being offline from their DC. > > I'm really not sure what the best way is, I guess we as a community have > to dice this out somehow. > > > Corinna Why not have a command that allows you choose whether or not you are "on-line", that is, currently connected to a network with a DC or "off-line" where you are not. In the former you consult the DC in the latter you use the local files? That way you can leave the timeout at a reasonable value. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-28 20:29 ` Corinna Vinschen 2014-02-28 21:08 ` Frank Fesevur @ 2014-03-01 8:59 ` Achim Gratz 1 sibling, 0 replies; 145+ messages in thread From: Achim Gratz @ 2014-03-01 8:59 UTC (permalink / raw) To: cygwin Corinna Vinschen writes: > How? Details? Stackdump? It works for me(TM). The timing only shows > that it's not the right thing for you, or that in the long run the > non-caching option should just go away. For the time being, though, I > need *details*. Sorry, that's the best I can do at the moment. > Hang on. Are you telling me this snapshot takes longer than the > snapshot from yesterday? And you're telling me that normal caching, as > was the default all the time, is slower in startup than with the > additional caching of groups at startup? Is your startup procedure > calling `id' twice by any chance? I've been using the network via VPN, and measured the numbers with the previous DLL(s) before testing the snapshot, results are in a different reply. > The problem with your numbers is that I have nothing really to compare > it with. It would be helpful to know what you're doing, the essential > parts of your startup for instance. I'm on my way to catch a plane, sorry this will be in about a week. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Terratec KOMPLEXER: http://Synth.Stromeko.net/Downloads.html#KomplexerWaves -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-28 12:09 ` Corinna Vinschen 2014-02-28 20:10 ` Achim Gratz @ 2014-02-28 22:35 ` Andrey Repin 2014-02-28 23:03 ` Andrey Repin 2014-03-03 9:23 ` Corinna Vinschen 1 sibling, 2 replies; 145+ messages in thread From: Andrey Repin @ 2014-02-28 22:35 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! > I've just uploaded a new snapshot to http://cygwin.com/snapshots/ I've sent a reply with testing details, but it may have been caught by SPAM deflection for "excess use of foreign language". Please disregard this notification, if a message will finally come through. > Please try it out. Average timings from your environment (process > startup, and calling `id') would be nice. Oh, and I thought of a way to test only invocation times. I ran /bin/true alot... db_cache: no Timer 2 Elapsed: 0:00:00,75 Stable (+-0.2 regardless of current credentials.) db_cache: yes Timer 2 Elapsed: 0:00:00,85 (current user) Timer 2 Elapsed: 0:00:00,75 (different user) Unstable results. (I.e. vary by ~.10 in both cases.) Default caching settings. Timer 2 Elapsed: 0:00:01,15 Unstable results. (I.e. vary by ~.15 or more in all cases.) -- WBR, Andrey Repin (anrdaemon@yandex.ru) 01.03.2014, <01:51> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-28 22:35 ` Andrey Repin @ 2014-02-28 23:03 ` Andrey Repin 2014-03-03 9:23 ` Corinna Vinschen 1 sibling, 0 replies; 145+ messages in thread From: Andrey Repin @ 2014-02-28 23:03 UTC (permalink / raw) To: Andrey Repin Greetings, Andrey Repin! > db_cache: no > Timer 2 Elapsed: 0:00:00,75 > Stable (+-0.02 regardless of current credentials.) My bad. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 01.03.2014, <02:27> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-28 22:35 ` Andrey Repin 2014-02-28 23:03 ` Andrey Repin @ 2014-03-03 9:23 ` Corinna Vinschen 2014-03-03 14:50 ` Andrey Repin 1 sibling, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-03-03 9:23 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1341 bytes --] On Mar 1 02:08, Andrey Repin wrote: > Greetings, Corinna Vinschen! > > > I've just uploaded a new snapshot to http://cygwin.com/snapshots/ > > I've sent a reply with testing details, but it may have been caught by SPAM > deflection for "excess use of foreign language". > Please disregard this notification, if a message will finally come through. > > > Please try it out. Average timings from your environment (process > > startup, and calling `id') would be nice. > > Oh, and I thought of a way to test only invocation times. > I ran /bin/true alot... > > db_cache: no > Timer 2 Elapsed: 0:00:00,75 > Stable (+-0.2 regardless of current credentials.) > > db_cache: yes > Timer 2 Elapsed: 0:00:00,85 (current user) > Timer 2 Elapsed: 0:00:00,75 (different user) > Unstable results. (I.e. vary by ~.10 in both cases.) > > Default caching settings. > Timer 2 Elapsed: 0:00:01,15 > Unstable results. (I.e. vary by ~.15 or more in all cases.) So the process start takes 0.3 secs extra for you. Is that feasible? Would you rather keep the new "db_cache" setting as an option? What do you think is the best default, objectively spoken? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-03 9:23 ` Corinna Vinschen @ 2014-03-03 14:50 ` Andrey Repin 2014-03-03 15:07 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-03-03 14:50 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! >> > I've just uploaded a new snapshot to http://cygwin.com/snapshots/ >> >> I've sent a reply with testing details, but it may have been caught by SPAM >> deflection for "excess use of foreign language". >> Please disregard this notification, if a message will finally come through. >> >> > Please try it out. Average timings from your environment (process >> > startup, and calling `id') would be nice. >> >> Oh, and I thought of a way to test only invocation times. >> I ran /bin/true alot... >> >> db_cache: no >> Timer 2 Elapsed: 0:00:00,75 >> Stable (+-0.2 regardless of current credentials.) >> >> db_cache: yes >> Timer 2 Elapsed: 0:00:00,85 (current user) >> Timer 2 Elapsed: 0:00:00,75 (different user) >> Unstable results. (I.e. vary by ~.10 in both cases.) >> >> Default caching settings. >> Timer 2 Elapsed: 0:00:01,15 >> Unstable results. (I.e. vary by ~.15 or more in all cases.) > So the process start takes 0.3 secs extra for you. Is that feasible? > Would you rather keep the new "db_cache" setting as an option? What > do you think is the best default, objectively spoken? I could never notice it. And it's not 0.3 sec, it's closer to 0.005sec (the cycle was run about sixty times, so, divide approptely) I'd need to test it in domain environment, but I don't have access to one with Cygwin installed right now. Or, perhaps, I do?... let me check something. In the meantime, I see my previous mail was indeed caught in the middle. I'll repost it with a link to test results. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 03.03.2014, <18:41> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-03 14:50 ` Andrey Repin @ 2014-03-03 15:07 ` Corinna Vinschen 2014-03-03 15:35 ` Andrey Repin ` (2 more replies) 0 siblings, 3 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-03-03 15:07 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1947 bytes --] On Mar 3 18:47, Andrey Repin wrote: > Greetings, Corinna Vinschen! > > >> > I've just uploaded a new snapshot to http://cygwin.com/snapshots/ > >> > >> I've sent a reply with testing details, but it may have been caught by SPAM > >> deflection for "excess use of foreign language". > >> Please disregard this notification, if a message will finally come through. > >> > >> > Please try it out. Average timings from your environment (process > >> > startup, and calling `id') would be nice. > >> > >> Oh, and I thought of a way to test only invocation times. > >> I ran /bin/true alot... > >> > >> db_cache: no > >> Timer 2 Elapsed: 0:00:00,75 > >> Stable (+-0.2 regardless of current credentials.) > >> > >> db_cache: yes > >> Timer 2 Elapsed: 0:00:00,85 (current user) > >> Timer 2 Elapsed: 0:00:00,75 (different user) > >> Unstable results. (I.e. vary by ~.10 in both cases.) > >> > >> Default caching settings. > >> Timer 2 Elapsed: 0:00:01,15 > >> Unstable results. (I.e. vary by ~.15 or more in all cases.) > > > So the process start takes 0.3 secs extra for you. Is that feasible? > > Would you rather keep the new "db_cache" setting as an option? What > > do you think is the best default, objectively spoken? > > I could never notice it. And it's not 0.3 sec, it's closer to 0.005sec (the > cycle was run about sixty times, so, divide approptely) > I'd need to test it in domain environment, but I don't have access to one with > Cygwin installed right now. Or, perhaps, I do?... let me check something. Sure. > In the meantime, I see my previous mail was indeed caught in the middle. I'll > repost it with a link to test results. Nah, never mind. I'm more interested of your experiences than in dry numbers :) Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-03 15:07 ` Corinna Vinschen @ 2014-03-03 15:35 ` Andrey Repin 2014-03-03 15:41 ` Andrey Repin 2014-03-03 20:50 ` Andrey Repin 2014-03-03 23:09 ` Andrey Repin 2 siblings, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-03-03 15:35 UTC (permalink / raw) To: Corinna Vinschen, cygwin Greetings, Corinna Vinschen! >> In the meantime, I see my previous mail was indeed caught in the middle. I'll >> repost it with a link to test results. > Nah, never mind. I'm more interested of your experiences than in > dry numbers :) Oh, don't be so quick, lady... The results were quite interesting. http://www.rootdir.org/.offload/MissingMail.eml -- WBR, Andrey Repin (anrdaemon@yandex.ru) 03.03.2014, <19:20> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-03 15:35 ` Andrey Repin @ 2014-03-03 15:41 ` Andrey Repin 0 siblings, 0 replies; 145+ messages in thread From: Andrey Repin @ 2014-03-03 15:41 UTC (permalink / raw) To: Corinna Vinschen, cygwin Greetings, Corinna Vinschen! >> In the meantime, I see my previous mail was indeed caught in the middle. I'll >> repost it with a link to test results. > Nah, never mind. I'm more interested of your experiences than in > dry numbers :) Oh, don't be so quick, lady... The results were quite interesting. http://www.rootdir.org/.offload/MissingMail.eml -- WBR, Andrey Repin (anrdaemon@yandex.ru) 03.03.2014, <19:20> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-03 15:07 ` Corinna Vinschen 2014-03-03 15:35 ` Andrey Repin @ 2014-03-03 20:50 ` Andrey Repin 2014-03-03 22:28 ` Andrey Repin 2014-03-03 23:09 ` Andrey Repin 2 siblings, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-03-03 20:50 UTC (permalink / raw) To: Corinna Vinschen, cygwin Greetings, Corinna Vinschen! >> I'd need to test it in domain environment, but I don't have access to one with >> Cygwin installed right now. Or, perhaps, I do?... let me check something. > Sure. I just thought of a derelict pair of virtualbox containers, that were set up to simulate domain environment of my workplace. Glad they are still in a good shape... for tests. Here are the results: Round 1: Running `id' command 50 times: CYGWIN_NT-5.1 station1 1.7.29s(0.271/5/3) 20140228 15:31:43 i686 Cygwin db_cache: no Timer 1 Elapsed: 0:01:48,36 uid=1109576(domainuser) gid=1049089(Domain Users) groups=[some 200+ trash groups] CYGWIN_NT-5.1 station1 1.7.29s(0.271/5/3) 20140228 15:31:43 i686 Cygwin db_cache: yes Timer 1 Elapsed: 0:00:36,64 uid=1109576(domainuser) gid=1049089(Domain Users) groups=[some 200+ trash groups] CYGWIN_NT-5.1 station1 1.7.29s(0.271/5/3) 20140228 15:31:43 i686 Cygwin Default caching settings. Timer 1 Elapsed: 0:00:36,33 uid=1109576(domainuser) gid=1049089(Domain Users) groups=[some 200+ trash groups] Round 2: Running /bin/true same 50 times: CYGWIN_NT-5.1 station1 1.7.29s(0.271/5/3) 20140228 15:31:43 i686 Cygwin db_cache: no Timer 1 Elapsed: 0:00:01,64 uid=1109576(domainuser) gid=1049089(Domain Users) groups=[some 200+ trash groups] CYGWIN_NT-5.1 station1 1.7.29s(0.271/5/3) 20140228 15:31:43 i686 Cygwin db_cache: yes Timer 1 Elapsed: 0:00:01,58 uid=1109576(domainuser) gid=1049089(Domain Users) groups=[some 200+ trash groups] CYGWIN_NT-5.1 station1 1.7.29s(0.271/5/3) 20140228 15:31:43 i686 Cygwin db_cache: full Timer 1 Elapsed: 0:00:35,91 uid=1109576(domainuser) gid=1049089(Domain Users) groups=[some 200+ trash groups] I find it interesting, that running `id' with caching disabled takes about 2 times longer, than when any form of caching is enabled. But overall, when caching is set to minimum, having more than 200 supplementary groups attached to your profile doesn't impose alot of trouble on process startup times, as I can see. The Cygwin installation in this test was a fresh one, augmented with stuff from x86/cygwin-inst-20140228.tar.xz archive. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 03.03.2014, <23:10> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-03 20:50 ` Andrey Repin @ 2014-03-03 22:28 ` Andrey Repin 0 siblings, 0 replies; 145+ messages in thread From: Andrey Repin @ 2014-03-03 22:28 UTC (permalink / raw) To: Corinna Vinschen, cygwin Greetings, Corinna Vinschen! >> I'd need to test it in domain environment, but I don't have access to one with >> Cygwin installed right now. Or, perhaps, I do?... let me check something. > Sure. I just thought of a derelict pair of virtualbox containers, that were set up to simulate domain environment of my workplace. Glad they are still in a good shape... for tests. Here are the results: Round 1: Running `id' command 50 times: CYGWIN_NT-5.1 station1 1.7.29s(0.271/5/3) 20140228 15:31:43 i686 Cygwin db_cache: no Timer 1 Elapsed: 0:01:48,36 uid=1109576(domainuser) gid=1049089(Domain Users) groups=[some 200+ trash groups] CYGWIN_NT-5.1 station1 1.7.29s(0.271/5/3) 20140228 15:31:43 i686 Cygwin db_cache: yes Timer 1 Elapsed: 0:00:36,64 uid=1109576(domainuser) gid=1049089(Domain Users) groups=[some 200+ trash groups] CYGWIN_NT-5.1 station1 1.7.29s(0.271/5/3) 20140228 15:31:43 i686 Cygwin Default caching settings. Timer 1 Elapsed: 0:00:36,33 uid=1109576(domainuser) gid=1049089(Domain Users) groups=[some 200+ trash groups] Round 2: Running /bin/true same 50 times: CYGWIN_NT-5.1 station1 1.7.29s(0.271/5/3) 20140228 15:31:43 i686 Cygwin db_cache: no Timer 1 Elapsed: 0:00:01,64 uid=1109576(domainuser) gid=1049089(Domain Users) groups=[some 200+ trash groups] CYGWIN_NT-5.1 station1 1.7.29s(0.271/5/3) 20140228 15:31:43 i686 Cygwin db_cache: yes Timer 1 Elapsed: 0:00:01,58 uid=1109576(domainuser) gid=1049089(Domain Users) groups=[some 200+ trash groups] CYGWIN_NT-5.1 station1 1.7.29s(0.271/5/3) 20140228 15:31:43 i686 Cygwin db_cache: full Timer 1 Elapsed: 0:00:35,91 uid=1109576(domainuser) gid=1049089(Domain Users) groups=[some 200+ trash groups] I find it interesting, that running `id' with caching disabled takes about 2 times longer, than when any form of caching is enabled. But overall, when caching is set to minimum, having more than 200 supplementary groups attached to your profile doesn't impose alot of trouble on process startup times, as I can see. The Cygwin installation in this test was a fresh one, augmented with stuff from x86/cygwin-inst-20140228.tar.xz archive. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 03.03.2014, <23:10> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-03 15:07 ` Corinna Vinschen 2014-03-03 15:35 ` Andrey Repin 2014-03-03 20:50 ` Andrey Repin @ 2014-03-03 23:09 ` Andrey Repin 2014-03-04 0:37 ` Andrey Repin 2014-03-04 11:35 ` Corinna Vinschen 2 siblings, 2 replies; 145+ messages in thread From: Andrey Repin @ 2014-03-03 23:09 UTC (permalink / raw) To: Corinna Vinschen, cygwin Greetings, Corinna Vinschen! >> I'd need to test it in domain environment, And with sane number of groups (less than 20), the results are barely different between different caching strategies (less than 1%, which could easily be explained by measurement error), except (again) for the case of runing `id' under "db_cache: no", which takes approximately twice as long, as any other way. If you ask me, the choice is between "db_cahce: yes" and "db_cache: full", and I don't have solid reason to prefer one over another. Not running with remote connection to the domain here... -- WBR, Andrey Repin (anrdaemon@yandex.ru) 04.03.2014, <00:33> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-03 23:09 ` Andrey Repin @ 2014-03-04 0:37 ` Andrey Repin 2014-03-04 11:35 ` Corinna Vinschen 1 sibling, 0 replies; 145+ messages in thread From: Andrey Repin @ 2014-03-04 0:37 UTC (permalink / raw) To: Corinna Vinschen, cygwin Greetings, Corinna Vinschen! >> I'd need to test it in domain environment, And with sane number of groups (less than 20), the results are barely different between different caching strategies (less than 1%, which could easily be explained by measurement error), except (again) for the case of runing `id' under "db_cache: no", which takes approximately twice as long, as any other way. If you ask me, the choice is between "db_cahce: yes" and "db_cache: full", and I don't have solid reason to prefer one over another. Not running with remote connection to the domain here... -- WBR, Andrey Repin (anrdaemon@yandex.ru) 04.03.2014, <00:33> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-03 23:09 ` Andrey Repin 2014-03-04 0:37 ` Andrey Repin @ 2014-03-04 11:35 ` Corinna Vinschen 1 sibling, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-03-04 11:35 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 934 bytes --] On Mar 4 00:44, Andrey Repin wrote: > Greetings, Corinna Vinschen! > > >> I'd need to test it in domain environment, > > And with sane number of groups (less than 20), the results are barely > different between different caching strategies (less than 1%, which could > easily be explained by measurement error), except (again) for the case of > runing `id' under "db_cache: no", which takes approximately twice as long, as > any other way. > > If you ask me, the choice is between "db_cahce: yes" and "db_cache: full", and > I don't have solid reason to prefer one over another. Not running with remote > connection to the domain here... Ok, thanks. I guess I'll remove the "db_cache: no" option at one point. It just doesn't seem to make sense. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-27 9:58 ` Corinna Vinschen 2014-02-27 13:25 ` Achim Gratz @ 2014-02-28 15:49 ` Achim Gratz 2014-02-28 17:43 ` Corinna Vinschen 1 sibling, 1 reply; 145+ messages in thread From: Achim Gratz @ 2014-02-28 15:49 UTC (permalink / raw) To: cygwin Corinna Vinschen writes: > 1 second? That sounds still a bit slow. It appears that that there are multiple DC involved, either via delegation or redirection (as I've managed to get some partial group resolutions where groups from a particular domain were absent). So all this slowness probably has to do with roundtrip times. Based on that hypothesis I've done the same test again via DSL/VPN and got this: 1:49 stock-cvs 1:15 getgroups 0:13 noldap The times don't change all that much whether I've clogged the DSL connection or not, so the size of the response doesn't seem to be a major factor here. > Still, it seems like the right thing to do to drop the group name > configuration stuff entirely. Indeed. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Terratec KOMPLEXER: http://Synth.Stromeko.net/Downloads.html#KomplexerWaves -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-28 15:49 ` Achim Gratz @ 2014-02-28 17:43 ` Corinna Vinschen 0 siblings, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-28 17:43 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2041 bytes --] On Feb 28 16:45, Achim Gratz wrote: > Corinna Vinschen writes: > > 1 second? That sounds still a bit slow. > > It appears that that there are multiple DC involved, either via > delegation or redirection (as I've managed to get some partial group > resolutions where groups from a particular domain were absent). So all > this slowness probably has to do with roundtrip times. Based on that > hypothesis I've done the same test again via DSL/VPN and got this: > > 1:49 stock-cvs > 1:15 getgroups > 0:13 noldap > > The times don't change all that much whether I've clogged the DSL > connection or not, so the size of the response doesn't seem to be a > major factor here. I made some tests myself today, while debugging Frank's problem. If I had no network connection to my DC, the group names couldn't be resolved. This is using the stock LookupAccountSid function from advapi32.dll so that means, the names of the groups are not cached anywhere in LSA, not even the names of the current user's groups. Given that, it was pretty surprising that the noldap code is so fast compared to the getgroups version. The LDAP connection is opened once only, so the ldap request should be fast. Even with a call to LookupAccountSid and an additional call to ldap_search_st, I would understand if the getgroups version takes twice as much as the noldap version, but *8* times? After some more testing it seems LookupAccountSid is asking the Global Catalog (GC). If I switch my LDAP queries to the GC port 3286, it's getting a *lot* faster. In fact, it's suddenly not 8 times slower, but only two times, as expected. Unfortunately that doesn't help us at all, because the POSIX attributes are not duplicated to the GC by default, and I guess it's not exactly helpful to ask administrators to duplicate the POSIX attributes to the GC :( Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-25 20:25 ` Corinna Vinschen 2014-02-25 20:28 ` Achim Gratz @ 2014-03-10 18:14 ` Achim Gratz 2014-03-10 18:29 ` Corinna Vinschen 1 sibling, 1 reply; 145+ messages in thread From: Achim Gratz @ 2014-03-10 18:14 UTC (permalink / raw) To: cygwin Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes: > The fact that the shells are doing it right seems to indicate that this > isn't a generic problem. I can't debug this, though. Can you see if > you can figure out what's going on under the hood? Does strace show > anything of interest? Can we perhaps set up some joint debugging via > private mail during the next couple of days? Perl goes through stat64 that seems to explicitly check the ACL while sh uses a different codepath. I don't see anything obviously wrong with either trace. If I set up cygdrive to use the noacl option the problem goes away, apparently because the ACL check never takes place within stat64. 869 2501512 [main] perl 2604 stat64: entering 814 2502326 [main] perl 2604 normalize_posix_path: src x86 802 2503128 [main] perl 2604 cwdstuff::get: posix /cygdrive/x/install 1195 2504323 [main] perl 2604 cwdstuff::get: (/cygdrive/x/install) = cwdstuff::get (0x80000008, 32768, 1, 0), errno 0 810 2505133 [main] perl 2604 normalize_posix_path: /cygdrive/x/install/x86 = normalize_posix_path (x86) 1195 2506328 [main] perl 2604 mount_info::conv_to_win32_path: conv_to_win32_path (/cygdrive/x/install/x86) 810 2507138 [main] perl 2604 mount_info::cygdrive_win32_path: src '/cygdrive/x/install/x86', dst 'X:\install\x86' 837 2507975 [main] perl 2604 set_flags: flags: binary (0x2) 810 2508785 [main] perl 2604 mount_info::conv_to_win32_path: src_path /cygdrive/x/install/x86, dst X:\install\x86, flags 0x4022, rc 0 3329 2512114 [main] perl 2604 symlink_info::check: 0x0 = NtCreateFile (\??\X:\install\x86) 2816 2514930 [main] perl 2604 symlink_info::check: not a symlink 845 2515775 [main] perl 2604 symlink_info::check: 0 = symlink.check(X:\install\x86, 0x289888) (0x404022) 839 2516614 [main] perl 2604 path_conv::check: this->path(X:\install\x86), has_acls(1) 828 2517442 [main] perl 2604 build_fh_pc: fh 0x612DD5A0, dev 000000C3 816 2518258 [main] perl 2604 stat_worker: (\??\X:\install\x86, 0x800390D0, 0x612DD5A0), file_attributes 16 1603 2519861 [main] perl 2604 cygpsid::debug_print: get_sids_info: owner SID = S-1-5-21-2052111302-842925246-682003330-75441 818 2520679 [main] perl 2604 cygpsid::debug_print: get_sids_info: group SID = S-1-5-21-2052111302-842925246-682003330-513 818 2521497 [main] perl 2604 get_info_from_sd: ACL 0x4000, uid 75441, gid 10513 875 2522372 [main] perl 2604 fhandler_base::fstat_helper: 0 = fstat (\??\X:\install\x86, 0x800390D0) st_size=0, st_mode=0x4000, st_ino=-197262732544 4575109st_atim=531DE525.1B5BB150 st_ctim=530C5D84.1F71B690 st_mtim=52D570D0.251FE418 st_birthtim=51EFE5A9.12BDBAC0 1424 1978515 [main] sh 4736 normalize_posix_path: src x86 882 1979397 [main] sh 4736 cwdstuff::get: posix /cygdrive/x/install 918 1980315 [main] sh 4736 cwdstuff::get: (/cygdrive/x/install) = cwdstuff::get (0x20000008, 32768, 1, 0), errno 0 1647 1981962 [main] sh 4736 normalize_posix_path: /cygdrive/x/install/x86 = normalize_posix_path (x86) 1218 1983180 [main] sh 4736 mount_info::conv_to_win32_path: conv_to_win32_path (/cygdrive/x/install/x86) 932 1984112 [main] sh 4736 mount_info::cygdrive_win32_path: src '/cygdrive/x/install/x86', dst 'X:\install\x86' 2389 1986501 [main] sh 4736 set_flags: flags: binary (0x2) 1221 1987722 [main] sh 4736 mount_info::conv_to_win32_path: src_path /cygdrive/x/install/x86, dst X:\install\x86, flags 0x4022, rc 0 4069 1991791 [main] sh 4736 symlink_info::check: 0x0 = NtCreateFile (\??\X:\install\x86) 2709 1994500 [main] sh 4736 symlink_info::check: not a symlink 985 1995485 [main] sh 4736 symlink_info::check: 0 = symlink.check(X:\install\x86, 0x289408) (0x404022) 946 1996431 [main] sh 4736 path_conv::check: this->path(X:\install\x86), has_acls(1) 947 1997378 [main] sh 4736 build_fh_pc: fh 0x612DD8D8, dev 000000C3 1935 1999313 [main] sh 4736 check_file_access: flags 0x2, ret 0 971 2000284 [main] sh 4736 fhandler_base::fhaccess: returning 0 1526 2001810 [main] sh 4736 euidaccess: returning 0 Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-10 18:14 ` Achim Gratz @ 2014-03-10 18:29 ` Corinna Vinschen 2014-03-10 19:21 ` Achim Gratz 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-03-10 18:29 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 4121 bytes --] On Mar 10 17:19, Achim Gratz wrote: > Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes: > > The fact that the shells are doing it right seems to indicate that this > > isn't a generic problem. I can't debug this, though. Can you see if > > you can figure out what's going on under the hood? Does strace show > > anything of interest? Can we perhaps set up some joint debugging via > > private mail during the next couple of days? > > Perl goes through stat64 that seems to explicitly check the ACL while sh > uses a different codepath. I don't see anything obviously wrong with either > trace. If I set up cygdrive to use the noacl option the problem goes away, > apparently because the ACL check never takes place within stat64. > > 869 2501512 [main] perl 2604 stat64: entering > 814 2502326 [main] perl 2604 normalize_posix_path: src x86 > 802 2503128 [main] perl 2604 cwdstuff::get: posix /cygdrive/x/install > 1195 2504323 [main] perl 2604 cwdstuff::get: (/cygdrive/x/install) = > cwdstuff::get (0x80000008, 32768, 1, 0), errno 0 > 810 2505133 [main] perl 2604 normalize_posix_path: /cygdrive/x/install/x86 > = normalize_posix_path (x86) > 1195 2506328 [main] perl 2604 mount_info::conv_to_win32_path: > conv_to_win32_path (/cygdrive/x/install/x86) > 810 2507138 [main] perl 2604 mount_info::cygdrive_win32_path: src > '/cygdrive/x/install/x86', dst 'X:\install\x86' > 837 2507975 [main] perl 2604 set_flags: flags: binary (0x2) > 810 2508785 [main] perl 2604 mount_info::conv_to_win32_path: src_path > /cygdrive/x/install/x86, dst X:\install\x86, flags 0x4022, rc 0 > 3329 2512114 [main] perl 2604 symlink_info::check: 0x0 = NtCreateFile > (\??\X:\install\x86) > 2816 2514930 [main] perl 2604 symlink_info::check: not a symlink > 845 2515775 [main] perl 2604 symlink_info::check: 0 = > symlink.check(X:\install\x86, 0x289888) (0x404022) > 839 2516614 [main] perl 2604 path_conv::check: this->path(X:\install\x86), > has_acls(1) > 828 2517442 [main] perl 2604 build_fh_pc: fh 0x612DD5A0, dev 000000C3 > 816 2518258 [main] perl 2604 stat_worker: (\??\X:\install\x86, 0x800390D0, > 0x612DD5A0), file_attributes 16 > 1603 2519861 [main] perl 2604 cygpsid::debug_print: get_sids_info: owner > SID = S-1-5-21-2052111302-842925246-682003330-75441 > 818 2520679 [main] perl 2604 cygpsid::debug_print: get_sids_info: group > SID = S-1-5-21-2052111302-842925246-682003330-513 > 818 2521497 [main] perl 2604 get_info_from_sd: ACL 0x4000, uid 75441, gid > 10513 > 875 2522372 [main] perl 2604 fhandler_base::fstat_helper: 0 = fstat > (\??\X:\install\x86, 0x800390D0) st_size=0, st_mode=0x4000, st_ino=-197262732544 ^^^^^^^^^^^^^^ This is the important snippet, but I don't see how this could have been different before my patches. The mode is S_IFDIR and 000 permissions. That usually means: - The owner of the file, here S-1-5-21-2052111302-842925246-682003330-75441, has no ACCESS_ALLOWED_ACE in the ACL, or the owner has no FILE_READ_DATA, FILE_WRITE_DATA, and FILE_EXECUTE permissions on the file. - The group of the file, here S-1-5-21-2052111302-842925246-682003330-513 (Domain Users, apparently) has no ACCESS_ALLOWED_ACE in the ACL, or the owner has no FILE_READ_DATA, FILE_WRITE_DATA, and FILE_EXECUTE permissions on the file. - The Everyone group S-1-1-0 has no ACCESS_ALLOWED_ACE in the ACL, or the owner has no FILE_READ_DATA, FILE_WRITE_DATA, and FILE_EXECUTE permissions on the file. This stuff is entirely independent of the new passwd/group code, unless the owner and group are Samba Unix Users/Groups (S-1-22-[...]), in which case I made some changes in this area on 2014-02-27. The uid and gid values point to the fact that you're still using a passwd and group file. How are your /etc/nsswitch.conf settings and does switching to db-only make a difference? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-10 18:29 ` Corinna Vinschen @ 2014-03-10 19:21 ` Achim Gratz 2014-03-10 20:11 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Achim Gratz @ 2014-03-10 19:21 UTC (permalink / raw) To: cygwin Corinna Vinschen writes: >> (\??\X:\install\x86, 0x800390D0) st_size=0, st_mode=0x4000, st_ino=-197262732544 > ^^^^^^^^^^^^^^ > This is the important snippet, but I don't see how this could have been > different before my patches. The mode is S_IFDIR and 000 permissions. I've run the same on Cygwin64 (where I don't use the snapshot yet) and it does indeed produce the same line. It still correctly determines that I do have permission to change into (and write in) the directory, but I don't know how. > That usually means: > > - The owner of the file, here S-1-5-21-2052111302-842925246-682003330-75441, > has no ACCESS_ALLOWED_ACE in the ACL, or the owner has no FILE_READ_DATA, > FILE_WRITE_DATA, and FILE_EXECUTE permissions on the file. > > - The group of the file, here S-1-5-21-2052111302-842925246-682003330-513 > (Domain Users, apparently) has no ACCESS_ALLOWED_ACE in the ACL, or > the owner has no FILE_READ_DATA, FILE_WRITE_DATA, and FILE_EXECUTE > permissions on the file. > > - The Everyone group S-1-1-0 has no ACCESS_ALLOWED_ACE in the ACL, or > the owner has no FILE_READ_DATA, FILE_WRITE_DATA, and FILE_EXECUTE > permissions on the file. > > This stuff is entirely independent of the new passwd/group code, unless > the owner and group are Samba Unix Users/Groups (S-1-22-[...]), in which > case I made some changes in this area on 2014-02-27. The owner is me and the primary group is indeed Domain Users. As I said, the whole share (a NetApp filer) is set up to not forbid access to anyone except via extended security settings that enable access for a certain AD group (and administrative access for another). These settings are forced upon all new files via inheritance, plus if I managed to change this (there was such a loophole once, but it likely has been closed) there'd be a script to periodically remove all extra permissions. The owner and groups are not Samba Unix as far as I can tell. > The uid and gid values point to the fact that you're still using a > passwd and group file. How are your /etc/nsswitch.conf settings and > does switching to db-only make a difference? The same test without an /etc/passwd file produces a different uid (the original one in /etc/passwd was actually 85441 and I just changed it to see where it came from), I haven't yet checked if the nsswitch.conf settings make a difference. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for KORG EX-800 and Poly-800MkII V0.9: http://Synth.Stromeko.net/Downloads.html#KorgSDada -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-10 19:21 ` Achim Gratz @ 2014-03-10 20:11 ` Corinna Vinschen 2014-03-10 20:37 ` Achim Gratz 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-03-10 20:11 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 889 bytes --] On Mar 10 19:28, Achim Gratz wrote: > Corinna Vinschen writes: > >> (\??\X:\install\x86, 0x800390D0) st_size=0, st_mode=0x4000, st_ino=-197262732544 > > ^^^^^^^^^^^^^^ > > This is the important snippet, but I don't see how this could have been > > different before my patches. The mode is S_IFDIR and 000 permissions. > > I've run the same on Cygwin64 (where I don't use the snapshot yet) and > it does indeed produce the same line. It still correctly determines > that I do have permission to change into (and write in) the directory, > but I don't know how. In that case the stat call is very likely unrelated. There must be some other call involved in perl. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-10 20:11 ` Corinna Vinschen @ 2014-03-10 20:37 ` Achim Gratz 2014-03-11 11:56 ` Achim Gratz 0 siblings, 1 reply; 145+ messages in thread From: Achim Gratz @ 2014-03-10 20:37 UTC (permalink / raw) To: cygwin Corinna Vinschen writes: > In that case the stat call is very likely unrelated. There must be some > other call involved in perl. IIRC, the next call is the write that prints "no" or "yes"... there may have been something like "stat_helper" inbetween only on Cygwin64, but I'll have to check again tomorrow. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Q+, Q and microQ: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-10 20:37 ` Achim Gratz @ 2014-03-11 11:56 ` Achim Gratz 2014-03-11 12:07 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Achim Gratz @ 2014-03-11 11:56 UTC (permalink / raw) To: cygwin Achim Gratz <Stromeko <at> nexgo.de> writes: > IIRC, the next call is the write that prints "no" or "yes"... there may > have been something like "stat_helper" inbetween only on Cygwin64, but > I'll have to check again tomorrow. It's a call to stat_worker with the UNC file path and the stat_worker handle, both times returning zero. There is only one interesting difference between the release and the snapshot DLL, they are reporting the atime to be different: st_atim=531DE525.1B5BB150 (release) st_atim=531DF887.5D9B9F8 (snapshot) Whatever differences there are probably inside stat_worker, but I don't really know. I've just found out that cygcheck cuts off the group listing after exactly 16kiB. Maybe there are some other places where it's now possible to overrun some buffer (pwdgrp::fetch_account_from_windows has got all of them according top the trace) or maybe some other limit for the number of groups? The group that grants access is number 293 in the list as returned by id... I've also found that two trusted domains aren't returning an entry for cyg_ldap::fetch_posix_offset_for_domain from the DC I'm assigned to, so that probably explains the long startup times I've been seeing with earlier snapshots. Maybe of interest: 987 1803457 [main] perl 6856 alloc_sd: uid 1124017, gid 10513, attribute 0x2190 860 1804317 [main] perl 6856 cygsid::debug_print: alloc_sd: owner SID = S-1-5-21-2052111302-842925246-682003330-75441 (+) 877 1805194 [main] perl 6856 cygsid::debug_print: alloc_sd: group SID = S-1-5-21-2052111302-842925246-682003330-513 (+) 827 1806021 [main] perl 6856 alloc_sd: ACL-Size: 124 934 1806955 [main] perl 6856 alloc_sd: Created SD-Size: 200 Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-11 11:56 ` Achim Gratz @ 2014-03-11 12:07 ` Corinna Vinschen 2014-03-11 12:40 ` Achim Gratz 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-03-11 12:07 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2924 bytes --] On Mar 11 07:44, Achim Gratz wrote: > Achim Gratz <Stromeko <at> nexgo.de> writes: > > IIRC, the next call is the write that prints "no" or "yes"... there may > > have been something like "stat_helper" inbetween only on Cygwin64, but > > I'll have to check again tomorrow. > > It's a call to stat_worker with the UNC file path and the stat_worker > handle, both times returning zero. zero == success > There is only one interesting difference > between the release and the snapshot DLL, they are reporting the atime to be > different: > > st_atim=531DE525.1B5BB150 (release) > st_atim=531DF887.5D9B9F8 (snapshot) Access time. On Windows it even changes when requesting certain kinds of metadata :-P > Whatever differences there are probably inside stat_worker, but I don't > really know. stat_worker is just a wrapper. Nothing happens in there which could explain any differences. > I've just found out that cygcheck cuts off the group listing after exactly > 16kiB. That's because cygcheck uses a fixed buffer of 16K to collect the output of the `id' call. > Maybe there are some other places where it's now possible to overrun > some buffer (pwdgrp::fetch_account_from_windows has got all of them > according top the trace) or maybe some other limit for the number of groups? Not in Cygwin. The limit is set by the application in the call to getgroups: http://linux.die.net/man/2/getgroups What we have is a NGROUPS_MAX define in limits.h and the equivalent NGROUPS in sys/param.h. It's defined as 1024, but it's not used or enforced anywhere in Cygwin. > The group that grants access is number 293 in the list as returned by id... But the question is, does perl actually use that list? The strace snippets don't show anything here. If it doesn't use the access(2) call, it would have to load the full ACL of the file and match that against your token group list. This requires calls to getgroups (which would create the litany of group SIDs from fetch_account_from_windows) and acl. It's pretty unlikely that perl would do this manually. > I've also found that two trusted domains aren't returning an entry for > cyg_ldap::fetch_posix_offset_for_domain from the DC I'm assigned to, so that > probably explains the long startup times I've been seeing with earlier > snapshots. Maybe of interest: > > 987 1803457 [main] perl 6856 alloc_sd: uid 1124017, gid 10513, attribute > 0x2190 You're still using a group file? Anyway, I need the full straces. It's pretty hard to say anything, let alone isolate at least the most probable cause without them. Can you please run the simple examples under the 2014-03-09 snapshot and send URLs to the snapshots? Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-11 12:07 ` Corinna Vinschen @ 2014-03-11 12:40 ` Achim Gratz 2014-03-11 13:37 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Achim Gratz @ 2014-03-11 12:40 UTC (permalink / raw) To: cygwin Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes: > > st_atim=531DE525.1B5BB150 (release) > > st_atim=531DF887.5D9B9F8 (snapshot) > > Access time. On Windows it even changes when requesting certain > kinds of metadata :-P It's been consistent over many days of testing and the only difference between release and snapshot, so I thought I'd mention it. > But the question is, does perl actually use that list? The strace > snippets don't show anything here. If it doesn't use the access(2) > call, it would have to load the full ACL of the file and match that > against your token group list. This requires calls to getgroups (which > would create the litany of group SIDs from fetch_account_from_windows) > and acl. It's pretty unlikely that perl would do this manually. Not that I can see, it simply seems to call stat64, which it did before. I don't think it drops any groups, either. And again, just mounting cygdrive with "noacl" gets rid of the problem entirely. > You're still using a group file? I think yes, I hadn't moved it away (but no passwd file). > Anyway, I need the full straces. It's pretty hard to say anything, let > alone isolate at least the most probable cause without them. Can you > please run the simple examples under the 2014-03-09 snapshot and send > URLs to the snapshots? I've just updated to the latest snapshot w/o AD integration for testing, will revert to the other one for more testing. Not sure if I can fit that in today, will send the links via PM when I have the traces uploaded. Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-11 12:40 ` Achim Gratz @ 2014-03-11 13:37 ` Corinna Vinschen 2014-03-11 17:06 ` Achim Gratz 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-03-11 13:37 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2098 bytes --] On Mar 11 12:07, Achim Gratz wrote: > Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes: > > > st_atim=531DE525.1B5BB150 (release) > > > st_atim=531DF887.5D9B9F8 (snapshot) > > > > Access time. On Windows it even changes when requesting certain > > kinds of metadata :-P > > It's been consistent over many days of testing and the only difference > between release and snapshot, so I thought I'd mention it. Can't explain that. There are no changes at all in this piece of code. > > But the question is, does perl actually use that list? The strace > > snippets don't show anything here. If it doesn't use the access(2) > > call, it would have to load the full ACL of the file and match that > > against your token group list. This requires calls to getgroups (which > > would create the litany of group SIDs from fetch_account_from_windows) > > and acl. It's pretty unlikely that perl would do this manually. > > Not that I can see, it simply seems to call stat64, which it did before. I > don't think it drops any groups, either. And again, just mounting cygdrive > with "noacl" gets rid of the problem entirely. There's a test in perl somewhere, which fails. "noacl" only fakes permissions anyway, so it doesn't matter. > > You're still using a group file? > > I think yes, I hadn't moved it away (but no passwd file). You don't have to move them away. Just set nsswitch.conf. > > Anyway, I need the full straces. It's pretty hard to say anything, let > > alone isolate at least the most probable cause without them. Can you > > please run the simple examples under the 2014-03-09 snapshot and send > > URLs to the snapshots? > > I've just updated to the latest snapshot w/o AD integration for testing, > will revert to the other one for more testing. Not sure if I can fit that > in today, will send the links via PM when I have the traces uploaded. Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-11 13:37 ` Corinna Vinschen @ 2014-03-11 17:06 ` Achim Gratz 2014-03-11 18:14 ` Corinna Vinschen 0 siblings, 1 reply; 145+ messages in thread From: Achim Gratz @ 2014-03-11 17:06 UTC (permalink / raw) To: cygwin Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes: > You don't have to move them away. Just set nsswitch.conf. Did that and using the snapshot DLL from 2014-03-05 on top of a full snapshot install from 2014-03-10. The ACL is this: # file: x86 # owner: gratz # group: Domain Users user::--- group::--- group:admin-cygwinupload:rwx group:user-cygwinupload:rwx mask:rwx other:--- default:user::--- default:group::--- default:group:admin-cygwinupload:rwx default:group:user-cygwinupload:rwx default:mask:rwx default:other:--- With the original passwd and group file in place and nsswitch.conf set to either "files" or "files db" the test fails. With just "files" getfacl doesn't show the group ACL at all, while with "files db" I see the ACL for both the admin and the user group (both are not in the group file). Setting to just "db" the ACL is shown as before and the test from Perl now succeeds! In fact any combination that includes "files" fails. So, after some head scratching I changed the uid and gid in the passwd and group files to match the new mapping scheme and lo and behold the test is now working. The getfacl command starts to show the group ACL when I add them to the group file (with the correct gid mapping), but the test still fails with "files" only. With the correct group entries and "files db", the test also works. So, Perl somehow uses the gid/uid mapping and relies on those to be working, while bash uses a code path that doesn't and probably just uses the uid/gid directly. I guess I could make the "files" only case work by adding some more groups (no time for checking what that might be at the moment), again changing the mapping (will mkpasswd do this at some point?). Do you still need traces or does get you a test case that works in your environment? Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-11 17:06 ` Achim Gratz @ 2014-03-11 18:14 ` Corinna Vinschen 2014-03-11 18:50 ` Achim Gratz 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-03-11 18:14 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 3235 bytes --] On Mar 11 15:07, Achim Gratz wrote: > Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes: > > You don't have to move them away. Just set nsswitch.conf. > > Did that and using the snapshot DLL from 2014-03-05 on top of a full > snapshot install from 2014-03-10. The ACL is this: > > # file: x86 > # owner: gratz > # group: Domain Users > user::--- > group::--- > group:admin-cygwinupload:rwx > group:user-cygwinupload:rwx > mask:rwx > other:--- > default:user::--- > default:group::--- > default:group:admin-cygwinupload:rwx > default:group:user-cygwinupload:rwx > default:mask:rwx > default:other:--- > > With the original passwd and group file in place and nsswitch.conf set to > either "files" or "files db" the test fails. With just "files" getfacl > doesn't show the group ACL at all, How does it look with any non-AD integrated Cygwin? > while with "files db" I see the ACL for > both the admin and the user group (both are not in the group file). Setting > to just "db" the ACL is shown as before and the test from Perl now succeeds! Ok. > In fact any combination that includes "files" fails. Hmm. So you're saying that the groups in question are not in /etc/groups, but it works with the non-AD Cygwin but not with the AD-Cygwin? A group which is not in /etc/groups is, in theory, just not in the ACL with the old Cygwin. What's not in Cygwin anymore is the mapping of a non-existing account to the uid/gid -1, what would have been printed as "????????" in ls output. This automatism would have collided with the DB stuff, but maybe I have to re-introduce it if only "files" is used. This could explain what happens in the "files"-only case... ...but that doesn't explain what happens with "files db". The uid/gid values may differ from the DB values, but only if the account actually exists in the file. And then the values in the files would have precedent over the db values. I'm really wondering what perl is checking there. > So, after some head > scratching I changed the uid and gid in the passwd and group files to match > the new mapping scheme and lo and behold the test is now working. The > getfacl command starts to show the group ACL when I add them to the group > file (with the correct gid mapping), but the test still fails with "files" > only. With the correct group entries and "files db", the test also works. Erm... > So, Perl somehow uses the gid/uid mapping and relies on those to be working, Whatever it's doing there. That doesn't make sense, unless it calls getgrent maybe?!? > while bash uses a code path that doesn't and probably just uses the uid/gid > directly. Much easier. bash just calls access(2). > I guess I could make the "files" only case work by adding some > more groups (no time for checking what that might be at the moment), again > changing the mapping (will mkpasswd do this at some point?). Do you still > need traces or does get you a test case that works in your environment? Yes, please. Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-11 18:14 ` Corinna Vinschen @ 2014-03-11 18:50 ` Achim Gratz 2014-03-11 19:56 ` Achim Gratz 2014-03-12 9:23 ` Corinna Vinschen 0 siblings, 2 replies; 145+ messages in thread From: Achim Gratz @ 2014-03-11 18:50 UTC (permalink / raw) To: cygwin Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes: > > With the original passwd and group file in place and nsswitch.conf set to > > either "files" or "files db" the test fails. With just "files" getfacl > > doesn't show the group ACL at all, > > How does it look with any non-AD integrated Cygwin? ... doesn't show the group ACL until I add them to the group file. That part is consistent with the AD enabled snapshot. Actually... if I create a group file with those two groups added, the access again doesn't get granted. Which finally reveals that I also need to have the administrators group present in that file (which mkpasswd had been doing) -- then it works. I can even leave out the two ACL groups again and it still works. > Hmm. So you're saying that the groups in question are not in > /etc/groups, but it works with the non-AD Cygwin but not with the > AD-Cygwin? Exactly. But as revealed above, what was really missing is the Administrators group. Somehow, when "files" is in effect, that mapping doesn't seem to exist unless it is explicitly listed in the file. It does get auto-created when I use _only_ the "db". I hope that somehow makes sense... > > So, Perl somehow uses the gid/uid mapping and relies on those to be working, No, it seems to balk on not being able to map the Administrator group (which is my egid). Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-11 18:50 ` Achim Gratz @ 2014-03-11 19:56 ` Achim Gratz 2014-03-12 9:27 ` Corinna Vinschen 2014-05-06 20:08 ` Achim Gratz 2014-03-12 9:23 ` Corinna Vinschen 1 sibling, 2 replies; 145+ messages in thread From: Achim Gratz @ 2014-03-11 19:56 UTC (permalink / raw) To: cygwin Achim Gratz <Stromeko <at> NexGo.DE> writes: > Exactly. But as revealed above, what was really missing is the > Administrators group. Somehow, when "files" is in effect, that mapping > doesn't seem to exist unless it is explicitly listed in the file. It does > get auto-created when I use _only_ the "db". I hope that somehow makes sense... I guess it does: the mapping that gets created from AD is sometimes 1049120 instead of 544. That depends on the settings in nsswitch.conf and whether an /etc/group file exists at all or contains an entry for Administrators. I guess that once this behaviour is stable (and maps to 544 in every case) the problem that Perl is having also goes away. Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-11 19:56 ` Achim Gratz @ 2014-03-12 9:27 ` Corinna Vinschen 2014-03-12 18:00 ` Achim Gratz 2014-05-06 20:08 ` Achim Gratz 1 sibling, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-03-12 9:27 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1966 bytes --] On Mar 11 17:12, Achim Gratz wrote: > Achim Gratz <Stromeko <at> NexGo.DE> writes: > > Exactly. But as revealed above, what was really missing is the > > Administrators group. Somehow, when "files" is in effect, that mapping > > doesn't seem to exist unless it is explicitly listed in the file. It does > > get auto-created when I use _only_ the "db". I hope that somehow makes > sense... > > I guess it does: the mapping that gets created from AD is sometimes 1049120 > instead of 544. That depends on the settings in nsswitch.conf and whether > an /etc/group file exists at all or contains an entry for Administrators. Argh! What about *details* which allow to reproduce this behaviour. In my environment I *never* see the admins group being produced with a GID of 1049120. And it doesn't make any sense at all. The UID/GID values starting with 0x100000 are reserved for accounts of the primary domain. The admins account is *never* an account of the primary domain because its SID doesn't start with S-1-5-21-YOUR-DOMAIN-SID. Rather it's a builtin account with SID 1-5-32-544, which always gets converted to UID/GID 544. [...time passes...] Hmm. The only reason I can see which would allow to generate the GID 1049120 is, if your account has a primaryGroupID of 544 in AD, but that *should* be impossible. The AD user properties dialog doesn't allow to set the primary group to a builtin group... unless you set this in the attribute editor. This is totally unexpected by Cygwin. From AD it just gets a RID value, and it simply adds this to the primary domain offset 0x100000. Ok, so, here's the question. Is your primaryGroupID in AD 544? If not, you will have to explain to me how this happens. I have found no other way to reproduce this. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-12 9:27 ` Corinna Vinschen @ 2014-03-12 18:00 ` Achim Gratz 0 siblings, 0 replies; 145+ messages in thread From: Achim Gratz @ 2014-03-12 18:00 UTC (permalink / raw) To: cygwin Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes: > Ok, so, here's the question. Is your primaryGroupID in AD 544? If not, > you will have to explain to me how this happens. I have found no other > way to reproduce this. My primary group ID in AD is 513 (Domain Users), just as shown by id, I am not member of the Administrators group and I only have effective group membership when running from an elevated prompt (which I'm allowed to get to via another group membership and a group policy that's totally unrelated to the well-known Administrators group). I have no idea what happens either, none of the traces even contain a check for Administrators group or a recognizable ID. All I know is that when the mapping for Administrators group is not 544, then the test fails. Anyway, I've run out of time for the moment, I'll try to set up a clean test environment later. This whole business of various settings in nsswitch.conf and entries in passwd/group is creating a combinatorial explosion that I'm not sure how to manage yet. Then, I'll have to sanitize the traces, which will require a script if it's more than two or three. Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-11 19:56 ` Achim Gratz 2014-03-12 9:27 ` Corinna Vinschen @ 2014-05-06 20:08 ` Achim Gratz 1 sibling, 0 replies; 145+ messages in thread From: Achim Gratz @ 2014-05-06 20:08 UTC (permalink / raw) To: cygwin Achim Gratz <Stromeko <at> NexGo.DE> writes: > I guess it does: the mapping that gets created from AD is sometimes 1049120 > instead of 544. That depends on the settings in nsswitch.conf and whether > an /etc/group file exists at all or contains an entry for Administrators. I > guess that once this behaviour is stable (and maps to 544 in every case) the > problem that Perl is having also goes away. I've installed todays snapshot and removed passwd, group and nsswitch.conf. So far everything looks OK (while I'm wired to the local network, wireless and VPN testing will have to wait). The Administrators group maps to 544 as it should and Perl correctly determines accessibility of files and directories via the ACL. Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-03-11 18:50 ` Achim Gratz 2014-03-11 19:56 ` Achim Gratz @ 2014-03-12 9:23 ` Corinna Vinschen 1 sibling, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-03-12 9:23 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2138 bytes --] On Mar 11 17:05, Achim Gratz wrote: > Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes: > > > With the original passwd and group file in place and nsswitch.conf set to > > > either "files" or "files db" the test fails. With just "files" getfacl > > > doesn't show the group ACL at all, > > > > How does it look with any non-AD integrated Cygwin? > > ... doesn't show the group ACL until I add them to the group file. That > part is consistent with the AD enabled snapshot. Actually... if I create a > group file with those two groups added, the access again doesn't get > granted. Which finally reveals that I also need to have the administrators > group present in that file (which mkpasswd had been doing) -- then it works. > I can even leave out the two ACL groups again and it still works. > > > Hmm. So you're saying that the groups in question are not in > > /etc/groups, but it works with the non-AD Cygwin but not with the > > AD-Cygwin? > > Exactly. But as revealed above, what was really missing is the > Administrators group. Somehow, when "files" is in effect, that mapping > doesn't seem to exist unless it is explicitly listed in the file. It does > get auto-created when I use _only_ the "db". I hope that somehow makes sense... > > > > So, Perl somehow uses the gid/uid mapping and relies on those to be working, > > No, it seems to balk on not being able to map the Administrator group (which > is my egid). Eh, what? You never mentioned that before! So you changed your passwd file to have group 544 as primary group? That usually won't work well if UAC is enabled. That's the problem with bug reports which don't provide *details*. If you had provided full straces, your passwd and group files and your nsswitch.conf settings, we may have been a lot further. As it is now, we're discussing your perl problem for days and days and *still* don't really know what's going on in the code :( Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-13 14:44 Testers needed: New passwd/group handling in Cygwin Corinna Vinschen ` (7 preceding siblings ...) 2014-02-25 18:25 ` Achim Gratz @ 2014-02-27 19:08 ` Frank Fesevur 2014-02-27 19:38 ` Andrey Repin 2014-02-27 20:10 ` Corinna Vinschen 8 siblings, 2 replies; 145+ messages in thread From: Frank Fesevur @ 2014-02-27 19:08 UTC (permalink / raw) To: cygwin 2014-02-13 15:38 GMT+01:00 Corinna Vinschen: > This is a pretty intrusive change, in need of some serious testing, so > I'd like to ask for volunteers. The latest 2014-02-13 snapshot from > http://cygwin.com/snapshots/ contains the changes, including the latest > bugfix. I noticed another thing. Not sure if I would call it a problem, just an observation or if it is a feature. As said before, my laptop is part of a domain and I have moved the local passwd and group out of the way. When I am not connected to the domain (on the road) I get this error every time I start mintty: "id: kan geen naam vinden bij groeps-ID 1049089" I have an export LANG=en_US.utf8 in my .bashrc so since the text is in Dutch this happens before the .bashrc is executed. It basically translates to "id: unable to find name for group ID 1049089". When I start my vpn and re-open mintty the error doesn't show up anymore, even if I disconnect the vpn again. After rebooting the error message occurs again. The reason for the message seems obvious. The process can't lookup the information. But it can be confusing to users to see an error at startup. Is it error useful to the user? Can I suppress this message? Regards, Frank -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-27 19:08 ` Frank Fesevur @ 2014-02-27 19:38 ` Andrey Repin 2014-02-27 20:16 ` Corinna Vinschen 2014-02-27 20:10 ` Corinna Vinschen 1 sibling, 1 reply; 145+ messages in thread From: Andrey Repin @ 2014-02-27 19:38 UTC (permalink / raw) To: Frank Fesevur, cygwin Greetings, Frank Fesevur! >> This is a pretty intrusive change, in need of some serious testing, so >> I'd like to ask for volunteers. The latest 2014-02-13 snapshot from >> http://cygwin.com/snapshots/ contains the changes, including the latest >> bugfix. > I noticed another thing. Not sure if I would call it a problem, just > an observation or if it is a feature. > As said before, my laptop is part of a domain and I have moved the > local passwd and group out of the way. > When I am not connected to the domain (on the road) I get this error > every time I start mintty: "id: kan geen naam vinden bij groeps-ID > 1049089" > I have an export LANG=en_US.utf8 in my .bashrc so since the text is in > Dutch this happens before the .bashrc is executed. It basically > translates to "id: unable to find name for group ID 1049089". > When I start my vpn and re-open mintty the error doesn't show up > anymore, even if I disconnect the vpn again. After rebooting the error > message occurs again. > The reason for the message seems obvious. The process can't lookup the > information. But it can be confusing to users to see an error at > startup. Is it error useful to the user? Can I suppress this message? I suppose, with latest Corinna's modifications to the ways `id' lookup the data, this message would go away naturally. I don't know, if the changes are available in a public snapshot, though. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 27.02.2014, <23:13> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-27 19:38 ` Andrey Repin @ 2014-02-27 20:16 ` Corinna Vinschen 2014-02-27 23:36 ` Andrey Repin 0 siblings, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-27 20:16 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1757 bytes --] On Feb 27 23:15, Andrey Repin wrote: > Greetings, Frank Fesevur! > > >> This is a pretty intrusive change, in need of some serious testing, so > >> I'd like to ask for volunteers. The latest 2014-02-13 snapshot from > >> http://cygwin.com/snapshots/ contains the changes, including the latest > >> bugfix. > > > I noticed another thing. Not sure if I would call it a problem, just > > an observation or if it is a feature. > > > As said before, my laptop is part of a domain and I have moved the > > local passwd and group out of the way. > > > When I am not connected to the domain (on the road) I get this error > > every time I start mintty: "id: kan geen naam vinden bij groeps-ID > > 1049089" > > I have an export LANG=en_US.utf8 in my .bashrc so since the text is in > > Dutch this happens before the .bashrc is executed. It basically > > translates to "id: unable to find name for group ID 1049089". > > > When I start my vpn and re-open mintty the error doesn't show up > > anymore, even if I disconnect the vpn again. After rebooting the error > > message occurs again. > > > The reason for the message seems obvious. The process can't lookup the > > information. But it can be confusing to users to see an error at > > startup. Is it error useful to the user? Can I suppress this message? > > I suppose, with latest Corinna's modifications to the ways `id' lookup the > data, this message would go away naturally. > I don't know, if the changes are available in a public snapshot, though. http://cygwin.com/ml/cygwin/2014-02/msg00707.html ? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-27 20:16 ` Corinna Vinschen @ 2014-02-27 23:36 ` Andrey Repin 0 siblings, 0 replies; 145+ messages in thread From: Andrey Repin @ 2014-02-27 23:36 UTC (permalink / raw) To: Corinna Vinschen Greetings, Corinna Vinschen! >> >> This is a pretty intrusive change, in need of some serious testing, so >> >> I'd like to ask for volunteers. The latest 2014-02-13 snapshot from >> >> http://cygwin.com/snapshots/ contains the changes, including the latest >> >> bugfix. >> >> > I noticed another thing. Not sure if I would call it a problem, just >> > an observation or if it is a feature. >> >> > As said before, my laptop is part of a domain and I have moved the >> > local passwd and group out of the way. >> >> > When I am not connected to the domain (on the road) I get this error >> > every time I start mintty: "id: kan geen naam vinden bij groeps-ID >> > 1049089" >> > I have an export LANG=en_US.utf8 in my .bashrc so since the text is in >> > Dutch this happens before the .bashrc is executed. It basically >> > translates to "id: unable to find name for group ID 1049089". >> >> > When I start my vpn and re-open mintty the error doesn't show up >> > anymore, even if I disconnect the vpn again. After rebooting the error >> > message occurs again. >> >> > The reason for the message seems obvious. The process can't lookup the >> > information. But it can be confusing to users to see an error at >> > startup. Is it error useful to the user? Can I suppress this message? >> >> I suppose, with latest Corinna's modifications to the ways `id' lookup the >> data, this message would go away naturally. >> I don't know, if the changes are available in a public snapshot, though. > http://cygwin.com/ml/cygwin/2014-02/msg00707.html ? Nearly missed it. This was a bit messy thursday for me. -- WBR, Andrey Repin (anrdaemon@yandex.ru) 28.02.2014, <03:06> Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-27 19:08 ` Frank Fesevur 2014-02-27 19:38 ` Andrey Repin @ 2014-02-27 20:10 ` Corinna Vinschen 2014-02-28 14:10 ` Corinna Vinschen 1 sibling, 1 reply; 145+ messages in thread From: Corinna Vinschen @ 2014-02-27 20:10 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2139 bytes --] On Feb 27 20:06, Frank Fesevur wrote: > 2014-02-13 15:38 GMT+01:00 Corinna Vinschen: > > This is a pretty intrusive change, in need of some serious testing, so > > I'd like to ask for volunteers. The latest 2014-02-13 snapshot from > > http://cygwin.com/snapshots/ contains the changes, including the latest > > bugfix. > > I noticed another thing. Not sure if I would call it a problem, just > an observation or if it is a feature. > > As said before, my laptop is part of a domain and I have moved the > local passwd and group out of the way. > > When I am not connected to the domain (on the road) I get this error > every time I start mintty: "id: kan geen naam vinden bij groeps-ID > 1049089" My dutch is a bit rusty but this text is obvious. And unexpected. What was supposed to happen is this: - LookupAccountSid returns with error ==> Check if the local machine knows the domain name of the SID ==> Yes: Create a fake group name "DOMAIN+RID" No: Create a fake group name "Unknown_Group+GID" > I have an export LANG=en_US.utf8 in my .bashrc so since the text is in > Dutch this happens before the .bashrc is executed. It basically > translates to "id: unable to find name for group ID 1049089". > When I start my vpn and re-open mintty the error doesn't show up > anymore, even if I disconnect the vpn again. At this point all the required information isd apparently cashed by the local LSA. > After rebooting the error message occurs again. Cache is empty again. > The reason for the message seems obvious. The process can't lookup the > information. But it can be confusing to users to see an error at > startup. Is it error useful to the user? Can I suppress this message? No, because it comes from `id', not from Cygwin, and as I wrote above, it's in fact unexpected and unwanted. Could you try to strace this scenario? Maybe I can figure out why the account name faking didn't work. Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
* Re: Testers needed: New passwd/group handling in Cygwin 2014-02-27 20:10 ` Corinna Vinschen @ 2014-02-28 14:10 ` Corinna Vinschen 0 siblings, 0 replies; 145+ messages in thread From: Corinna Vinschen @ 2014-02-28 14:10 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2200 bytes --] On Feb 27 20:38, Corinna Vinschen wrote: > On Feb 27 20:06, Frank Fesevur wrote: > > 2014-02-13 15:38 GMT+01:00 Corinna Vinschen: > > > This is a pretty intrusive change, in need of some serious testing, so > > > I'd like to ask for volunteers. The latest 2014-02-13 snapshot from > > > http://cygwin.com/snapshots/ contains the changes, including the latest > > > bugfix. > > > > I noticed another thing. Not sure if I would call it a problem, just > > an observation or if it is a feature. > > > > As said before, my laptop is part of a domain and I have moved the > > local passwd and group out of the way. > > > > When I am not connected to the domain (on the road) I get this error > > every time I start mintty: "id: kan geen naam vinden bij groeps-ID > > 1049089" > > My dutch is a bit rusty but this text is obvious. And unexpected. > What was supposed to happen is this: > > - LookupAccountSid returns with error > ==> Check if the local machine knows the domain name of the SID > ==> Yes: Create a fake group name "DOMAIN+RID" > No: Create a fake group name "Unknown_Group+GID" > [...] > > The reason for the message seems obvious. The process can't lookup the > > information. But it can be confusing to users to see an error at > > startup. Is it error useful to the user? Can I suppress this message? > > No, because it comes from `id', not from Cygwin, and as I wrote > above, it's in fact unexpected and unwanted. Could you try to > strace this scenario? Maybe I can figure out why the account name > faking didn't work. Never mind that, I could easily reproduce the issue. Please try the latest snapshot from http://cygwin.com/snapshots/. It's supposed to fix this issue. It also introduces another new behaviour as outlined in http://cygwin.com/ml/cygwin/2014-02/msg00735.html Note that "db_cache: full" is the default for now so the new feature gets a bit more testing. The former default would have been equivalent to "db_cache: yes". Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 145+ messages in thread
end of thread, other threads:[~2014-05-06 20:08 UTC | newest] Thread overview: 145+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-02-13 14:44 Testers needed: New passwd/group handling in Cygwin Corinna Vinschen 2014-02-13 16:05 ` Steven Penny 2014-02-13 16:17 ` Corinna Vinschen 2014-02-13 16:20 ` Corinna Vinschen 2014-02-13 16:42 ` Andrey Repin 2014-02-13 18:41 ` Lord Laraby 2014-02-13 18:49 ` Larry Hall (Cygwin) 2014-02-13 19:05 ` Lord Laraby 2014-02-13 19:37 ` Corinna Vinschen 2014-02-13 19:52 ` Andrey Repin 2014-02-14 0:31 ` Warren Young 2014-02-14 7:36 ` Andrey Repin 2014-02-14 8:00 ` Warren Young 2014-02-14 9:47 ` Corinna Vinschen 2014-02-13 19:33 ` Andrey Repin 2014-02-13 19:50 ` Corinna Vinschen 2014-02-13 19:41 ` m0viefreak 2014-02-13 19:56 ` Corinna Vinschen 2014-02-13 22:20 ` David Stacey 2014-02-14 9:05 ` Dave Kilroy 2014-02-14 10:16 ` Corinna Vinschen 2014-02-14 11:00 ` Corinna Vinschen 2014-02-14 13:51 ` Kurt Franke 2014-02-14 14:05 ` Corinna Vinschen 2014-02-14 9:49 ` Corinna Vinschen 2014-02-13 23:12 ` New passwd/group handling in Cygwin - test results and observations Andrey Repin 2014-02-14 10:42 ` Corinna Vinschen 2014-02-14 12:35 ` Andrey Repin 2014-02-14 14:13 ` Corinna Vinschen 2014-02-16 14:34 ` Andrey Repin 2014-02-16 14:48 ` Corinna Vinschen 2014-02-16 15:44 ` Corinna Vinschen 2014-02-16 15:52 ` Corinna Vinschen 2014-02-16 16:35 ` Andrey Repin 2014-02-14 6:40 ` Testers needed: New passwd/group handling in Cygwin Warren Young 2014-02-14 10:48 ` Corinna Vinschen 2014-02-14 12:56 ` Andrey Repin 2014-02-14 14:15 ` Corinna Vinschen 2014-02-14 20:18 ` Warren Young 2014-02-14 21:49 ` Warren Young 2014-02-15 12:52 ` Corinna Vinschen 2014-02-16 11:48 ` Warren Young 2014-02-16 12:00 ` Corinna Vinschen 2014-02-16 14:50 ` Corinna Vinschen 2014-02-16 17:15 ` Christopher Faylor 2014-02-16 18:30 ` Corinna Vinschen 2014-02-16 16:35 ` Warren Young 2014-02-16 16:53 ` Andrey Repin 2014-02-14 10:20 ` Frank Fesevur 2014-02-14 10:50 ` Corinna Vinschen 2014-02-14 12:05 ` Corinna Vinschen 2014-02-14 14:04 ` Ken Brown 2014-02-14 14:22 ` Corinna Vinschen 2014-02-14 15:23 ` Ken Brown 2014-02-15 2:51 ` Frank Fesevur 2014-02-15 12:56 ` Corinna Vinschen 2014-02-19 9:32 ` Frank Fesevur 2014-02-17 17:25 ` Corinna Vinschen 2014-02-19 15:05 ` Andrey Repin 2014-02-19 18:50 ` Corinna Vinschen 2014-02-19 19:32 ` Andrey Repin 2014-02-19 19:56 ` Corinna Vinschen 2014-02-19 20:27 ` Andrey Repin 2014-02-20 16:02 ` Corinna Vinschen 2014-02-20 17:35 ` Andrey Repin 2014-02-21 15:33 ` Corinna Vinschen 2014-02-21 16:29 ` Ken Brown 2014-02-21 16:31 ` Corinna Vinschen 2014-02-25 18:25 ` Achim Gratz 2014-02-25 19:16 ` Andrey Repin 2014-02-25 20:04 ` Achim Gratz 2014-02-25 20:25 ` Corinna Vinschen 2014-02-25 20:28 ` Achim Gratz 2014-02-25 21:55 ` Corinna Vinschen 2014-02-25 22:59 ` Andrey Repin 2014-02-26 3:09 ` Andrey Repin 2014-02-26 10:02 ` Corinna Vinschen 2014-02-26 9:06 ` Achim Gratz 2014-02-26 10:07 ` Corinna Vinschen 2014-02-26 16:12 ` Corinna Vinschen 2014-02-26 19:14 ` Achim Gratz 2014-02-27 9:08 ` Achim Gratz 2014-02-27 9:49 ` Achim Gratz 2014-02-27 9:58 ` Corinna Vinschen 2014-02-27 13:25 ` Achim Gratz 2014-02-27 15:09 ` Corinna Vinschen 2014-02-27 23:20 ` Andrey Repin 2014-02-28 12:09 ` Corinna Vinschen 2014-02-28 20:10 ` Achim Gratz 2014-02-28 20:29 ` Corinna Vinschen 2014-02-28 21:08 ` Frank Fesevur 2014-02-28 21:13 ` Corinna Vinschen 2014-02-28 21:50 ` Corinna Vinschen 2014-03-02 13:21 ` Frank Fesevur 2014-03-03 9:21 ` Corinna Vinschen 2014-03-03 12:19 ` Frank Fesevur 2014-03-03 12:29 ` Henry S. Thompson 2014-03-03 13:19 ` Corinna Vinschen 2014-03-03 17:06 ` Andrey Repin 2014-03-04 8:07 ` Warren Young 2014-03-04 8:08 ` Andrey Repin 2014-03-04 8:10 ` Warren Young 2014-03-04 8:19 ` Corinna Vinschen 2014-03-05 16:47 ` Warren Young 2014-03-05 20:18 ` Andrey Repin 2014-03-05 23:09 ` Corinna Vinschen 2014-03-03 18:24 ` Andy Hall 2014-03-01 8:59 ` Achim Gratz 2014-02-28 22:35 ` Andrey Repin 2014-02-28 23:03 ` Andrey Repin 2014-03-03 9:23 ` Corinna Vinschen 2014-03-03 14:50 ` Andrey Repin 2014-03-03 15:07 ` Corinna Vinschen 2014-03-03 15:35 ` Andrey Repin 2014-03-03 15:41 ` Andrey Repin 2014-03-03 20:50 ` Andrey Repin 2014-03-03 22:28 ` Andrey Repin 2014-03-03 23:09 ` Andrey Repin 2014-03-04 0:37 ` Andrey Repin 2014-03-04 11:35 ` Corinna Vinschen 2014-02-28 15:49 ` Achim Gratz 2014-02-28 17:43 ` Corinna Vinschen 2014-03-10 18:14 ` Achim Gratz 2014-03-10 18:29 ` Corinna Vinschen 2014-03-10 19:21 ` Achim Gratz 2014-03-10 20:11 ` Corinna Vinschen 2014-03-10 20:37 ` Achim Gratz 2014-03-11 11:56 ` Achim Gratz 2014-03-11 12:07 ` Corinna Vinschen 2014-03-11 12:40 ` Achim Gratz 2014-03-11 13:37 ` Corinna Vinschen 2014-03-11 17:06 ` Achim Gratz 2014-03-11 18:14 ` Corinna Vinschen 2014-03-11 18:50 ` Achim Gratz 2014-03-11 19:56 ` Achim Gratz 2014-03-12 9:27 ` Corinna Vinschen 2014-03-12 18:00 ` Achim Gratz 2014-05-06 20:08 ` Achim Gratz 2014-03-12 9:23 ` Corinna Vinschen 2014-02-27 19:08 ` Frank Fesevur 2014-02-27 19:38 ` Andrey Repin 2014-02-27 20:16 ` Corinna Vinschen 2014-02-27 23:36 ` Andrey Repin 2014-02-27 20:10 ` Corinna Vinschen 2014-02-28 14:10 ` Corinna Vinschen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).