* cygwin utils can access directory and its contents, but W10 utils claim to have no access, why? @ 2024-01-22 6:18 John Ruckstuhl 2024-01-22 7:33 ` Brian Inglis 2024-01-22 9:48 ` Corinna Vinschen 0 siblings, 2 replies; 5+ messages in thread From: John Ruckstuhl @ 2024-01-22 6:18 UTC (permalink / raw) To: cygwin I am seeing a weird phenomenon, hopefully someone can illuminate and teach. Or point me to an archived thread. I have some folders that Cygwin utils can readily access, but W10 utils claim to have no access to. It feels as if * the Cygwin utils try to do what they are commanded to do, and it's up to the OS to refuse if the ACLs are insufficient. * the W10 utils are written to decline to attempt access, due to some convention or gentlemen's agreement (built into some API?). But wouldn't that lead Windows users to a false sense of protection, a false sense of security? Environment * Cygwin 3.4.9-1 on W10. 64-bit. * UAC is not active (based on EnableLUA=0 at boot) I have a 3rd-party program “yabba” that many different users run (on the same shared PC). It leaves behind a “temporary” dir (under C:/Users/username/AppData/Local/Temp), each time it’s run. Many times per day. These dirs are approx. 1000 files and 20 MB. The 3rd-party program is an exe-file compiled with pyinstaller. It unpacks a python script and a python interpreter, and runs the python script. But in our workflow, we usually terminate the execution brutally, so the pyinstaller cleanup does not happen, leaving the 20 MB behind. I want Bob, a member of the local group Administrators, to remove these folders. For example, ordinary user Alice runs yabba a couple of times and leaves behind 20 MB in each of C:/Users/Alice/AppData/Local/Temp/_MEI21002 C:/Users/Alice/AppData/Local/Temp/_MEI21282 Now for Local Administrator Bob, no impediment seeing these two dirs... $ cd C\:/Users/Alice/AppData/Local/Temp $ D1=_MEI21002 $ D2=_MEI21282 $ du -sh $D1 $D2 21M _MEI21002 21M _MEI21282 $ for D in $D1 $D2; do printf "%s\t%s\n" $D "$(find "$D" -mindepth 1 -type d -printf "dirs\n" -o -printf "files\n" | sort | uniq -c | paste - -)"; done _MEI21002 34 dirs 940 files _MEI21282 33 dirs 929 files $ getfacl $D1 $D2 # file: _MEI21002 # owner: Alice # group: Domain Users user::rwx group::--- group:OWNER RIGHTS:rwx mask::rwx other::--- # file: _MEI21282 # owner: Alice # group: Domain Users user::rwx group::--- group:OWNER RIGHTS:rwx mask::rwx other::--- And (for Local Administrator Bob) no impediment to removing them, for example, $ rm -r _MEI21282 (success!) BUT Windows utils run by Bob the Administrator on the remaining dir are blocked, like this (cmd, "Run as administrator"): C:\Users\Alice\AppData\Local\Temp>dir _MEI21002 Volume in drive C is OSDisk Volume Serial Number is 581C-10F2 Directory of C:\Users\Alice\AppData\Local\Temp\_MEI21002 File Not Found C:\Users\Alice\AppData\Local\Temp>icacls _MEI21002 _MEI21002: Access is denied. Successfully processed 0 files; Failed processing 1 files As a consequence, Windows utils run by Bob the Administrator cannot delete Alice's directory. Until they do a takeown. The first try to delete fails: C:\Users\Alice\AppData\Local\Temp>del /S /Q _MEI21002 Could Not Find C:\Users\Alice\AppData\Local\Temp\_MEI21002\* But a takeown succeeds C:\Users\Alice\AppData\Local\Temp>takeown /A /F _MEI21002 /R /D Y Now icacls has access C:\Users\Alice\AppData\Local\Temp>icacls _MEI21002 _MEI21002 BUILTIN\Administrators:(OI)(IO)(F) BUILTIN\Administrators:(CI)(F) Successfully processed 1 files; Failed processing 0 files And the second try to delete succeeds C:\Users\Alice\AppData\Local\Temp>del /S /Q _MEI21002 In summary, why is it that Bob the Administrator can Cygwin "rm.exe" to delete these folders without taking ownership, but to delete with Windows utils, he needs to take ownership first? Thanks for teaching, John Ruckstuhl ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: cygwin utils can access directory and its contents, but W10 utils claim to have no access, why? 2024-01-22 6:18 cygwin utils can access directory and its contents, but W10 utils claim to have no access, why? John Ruckstuhl @ 2024-01-22 7:33 ` Brian Inglis 2024-01-22 9:48 ` Corinna Vinschen 1 sibling, 0 replies; 5+ messages in thread From: Brian Inglis @ 2024-01-22 7:33 UTC (permalink / raw) To: cygwin On 2024-01-21 23:18, John Ruckstuhl via Cygwin wrote: > I am seeing a weird phenomenon, hopefully someone can illuminate and teach. > Or point me to an archived thread. > > I have some folders that Cygwin utils can readily access, > but W10 utils claim to have no access to. > It feels as if > * the Cygwin utils try to do what they are commanded to do, and it's > up to the OS to refuse if the ACLs are insufficient. > * the W10 utils are written to decline to attempt access, due to some > convention or gentlemen's agreement (built into some API?). > But wouldn't that lead Windows users to a false sense of protection, a > false sense of security? > > Environment > * Cygwin 3.4.9-1 on W10. 64-bit. > * UAC is not active (based on EnableLUA=0 at boot) > > I have a 3rd-party program “yabba” that many different users run (on the > same shared PC). > It leaves behind a “temporary” dir (under > C:/Users/username/AppData/Local/Temp), each time it’s run. > Many times per day. > These dirs are approx. 1000 files and 20 MB. > The 3rd-party program is an exe-file compiled with pyinstaller. > It unpacks a python script and a python interpreter, and runs the python > script. > But in our workflow, we usually terminate the execution brutally, so the > pyinstaller cleanup does not happen, leaving the 20 MB behind. > > I want Bob, a member of the local group Administrators, to remove these folders. > > For example, ordinary user Alice runs yabba a couple of times and leaves > behind 20 MB in each of > C:/Users/Alice/AppData/Local/Temp/_MEI21002 > C:/Users/Alice/AppData/Local/Temp/_MEI21282 > > Now for Local Administrator Bob, no impediment seeing these two dirs... > $ cd C\:/Users/Alice/AppData/Local/Temp > $ D1=_MEI21002 > $ D2=_MEI21282 > $ du -sh $D1 $D2 > 21M _MEI21002 > 21M _MEI21282 > $ for D in $D1 $D2; do printf "%s\t%s\n" $D "$(find "$D" -mindepth > 1 -type d -printf "dirs\n" -o -printf "files\n" | sort | uniq -c | > paste - -)"; done > _MEI21002 34 dirs 940 files > _MEI21282 33 dirs 929 files > $ getfacl $D1 $D2 > # file: _MEI21002 > # owner: Alice > # group: Domain Users > user::rwx > group::--- > group:OWNER RIGHTS:rwx > mask::rwx > other::--- > > > # file: _MEI21282 > # owner: Alice > # group: Domain Users > user::rwx > group::--- > group:OWNER RIGHTS:rwx > mask::rwx > other::--- > > And (for Local Administrator Bob) no impediment to removing them, for example, > $ rm -r _MEI21282 > (success!) > > BUT Windows utils run by Bob the Administrator on the remaining dir > are blocked, like this > (cmd, "Run as administrator"): > C:\Users\Alice\AppData\Local\Temp>dir _MEI21002 > Volume in drive C is OSDisk > Volume Serial Number is 581C-10F2 > > Directory of C:\Users\Alice\AppData\Local\Temp\_MEI21002 > > File Not Found > > C:\Users\Alice\AppData\Local\Temp>icacls _MEI21002 > _MEI21002: Access is denied. > Successfully processed 0 files; Failed processing 1 files > > As a consequence, Windows utils run by Bob the Administrator cannot > delete Alice's directory. Until they do a takeown. > > The first try to delete fails: > C:\Users\Alice\AppData\Local\Temp>del /S /Q _MEI21002 > Could Not Find C:\Users\Alice\AppData\Local\Temp\_MEI21002\* > > But a takeown succeeds > C:\Users\Alice\AppData\Local\Temp>takeown /A /F _MEI21002 /R /D Y > > Now icacls has access > C:\Users\Alice\AppData\Local\Temp>icacls _MEI21002 > _MEI21002 BUILTIN\Administrators:(OI)(IO)(F) > BUILTIN\Administrators:(CI)(F) > > Successfully processed 1 files; Failed processing 0 files > > And the second try to delete succeeds > C:\Users\Alice\AppData\Local\Temp>del /S /Q _MEI21002 > > In summary, why is it that Bob the Administrator can Cygwin "rm.exe" to > delete these folders without taking ownership, but to delete with > Windows utils, he needs to take ownership first? I normally find the problem is that a directory has no useful Windows DACLs defined, so directories or files created in it by Cygwin programs are not accessible to Windows programs. It seems to me, that when Cygwin creates directories or files in those directories, they have no Windows ACLs defaulted, but they seem to look fine to Cygwin's assumed POSIX permissions, as if on FAT drives? Better explanations would be welcome. For directories, run: $ lsattr -adl $D $ ls -adlL $D $ getfacl $D $ icalcs "$(cygpath -m "$D")" and for files, skip lsattr. This should show you all the views of permissions you need to figure out the issue. -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut -- Antoine de Saint-Exupéry ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: cygwin utils can access directory and its contents, but W10 utils claim to have no access, why? 2024-01-22 6:18 cygwin utils can access directory and its contents, but W10 utils claim to have no access, why? John Ruckstuhl 2024-01-22 7:33 ` Brian Inglis @ 2024-01-22 9:48 ` Corinna Vinschen 2024-01-22 18:59 ` John Ruckstuhl 1 sibling, 1 reply; 5+ messages in thread From: Corinna Vinschen @ 2024-01-22 9:48 UTC (permalink / raw) To: cygwin; +Cc: John.Ruckstuhl On Jan 21 22:18, John Ruckstuhl via Cygwin wrote: > I am seeing a weird phenomenon, hopefully someone can illuminate and teach. > Or point me to an archived thread. > > I have some folders that Cygwin utils can readily access, > but W10 utils claim to have no access to. > It feels as if > * the Cygwin utils try to do what they are commanded to do, and it's > up to the OS to refuse if the ACLs are insufficient. > * the W10 utils are written to decline to attempt access, due to some > convention or gentlemen's agreement (built into some API?). > But wouldn't that lead Windows users to a false sense of protection, a > false sense of security? No, this has nothing to do with security, just with a weird concept of user privileges. Here's what happens: > And (for Local Administrator Bob) no impediment to removing them, for example, > $ rm -r _MEI21282 > (success!) > > BUT Windows utils run by Bob the Administrator on the remaining dir > are blocked, like this As you know, a "root" user on a Unix system has the right to ignore file permissions. That's why root users often have "rm" aliased to "rm -i" :) Windows OTOH manages so called user privileges stored in the user's access token. One of these privileges is the right to ignore permissions while reading (SE_BACKUP_PRIVILEGE), another privilege is the right to set the ACL of a file to arbitrary permissions (SE_RESTORE_PRIVILEGE). These are the two privileges covering what a "root" user can do to files. As the names of these privileges suggest, they were originally thought of privileges you only need for backup tools. Users in the Administrators group have these privileges in their user token. Under UAC, both privileges are removed from the token. In an elevated shell, though, both privileges are present. The funny thing here is this: While both privileges are present in the token, they are disabled by default. They have to be enabled explicitely before you can exercise the privileges. Usually you do this in the same application programatically. Apart from backup tool, standard Windows tools practically *never* enable these privileges, so they fail in various circumstances. For instance, a standard "del" in the CMD shell is not able to delete a file even as administrator, if the file permissions don't allow delete access for the Administrators group. That's when you have to "take ownership" in the GUI as a last resort. In NT4 times, even that had a good chance to fail for some reason I never understood, back in the days. However, since this concept is pretty weird and not a Unix concept, the Cygwin DLL enables both privileges by default right at process startup. If the privileges are not in the user token, nothing happens. But if the privileges are present, as for an elevated process of a user in the Administrators group, then they will be enabled, and you're a "root" user in the Unix sense. And then it's "alias rm = rm -i" time again ;) HTH, Corinna ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: cygwin utils can access directory and its contents, but W10 utils claim to have no access, why? 2024-01-22 9:48 ` Corinna Vinschen @ 2024-01-22 18:59 ` John Ruckstuhl 2024-04-03 2:07 ` John Ruckstuhl 0 siblings, 1 reply; 5+ messages in thread From: John Ruckstuhl @ 2024-01-22 18:59 UTC (permalink / raw) To: cygwin, John.Ruckstuhl Thanks for the replies Brian & Corinna, I learned a lot. On Mon, Jan 22, 2024 at 1:48 AM Corinna Vinschen <corinna-cygwin@cygwin.com> wrote: > Users in the Administrators group have these privileges in their user > token. Under UAC, both privileges are removed from the token. In an > elevated shell, though, both privileges are present. > > The funny thing here is this: While both privileges are present in the > token, they are disabled by default. > > They have to be enabled explicitely before you can exercise the > privileges. Usually you do this in the same application > programatically. Now I see... Local administrator Bob reports his User Access Token info (with Windows whoami, not cygwin whoami) C:\>whoami /priv PRIVILEGES INFORMATION ---------------------- Privilege Name ... State ========================================= ... ======== ... SeBackupPrivilege ... Disabled SeRestorePrivilege ... Disabled ... C:\> Thanks very much. John Ruckstuhl ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: cygwin utils can access directory and its contents, but W10 utils claim to have no access, why? 2024-01-22 18:59 ` John Ruckstuhl @ 2024-04-03 2:07 ` John Ruckstuhl 0 siblings, 0 replies; 5+ messages in thread From: John Ruckstuhl @ 2024-04-03 2:07 UTC (permalink / raw) To: cygwin, John.Ruckstuhl On Mon, Jan 22, 2024 at 10:59 AM John Ruckstuhl <john.ruckstuhl@gmail.com> wrote: > > Thanks for the replies Brian & Corinna, I learned a lot. > > On Mon, Jan 22, 2024 at 1:48 AM Corinna Vinschen > <corinna-cygwin@cygwin.com> wrote: > > Users in the Administrators group have these privileges in their user > > token. Under UAC, both privileges are removed from the token. In an > > elevated shell, though, both privileges are present. > > > > The funny thing here is this: While both privileges are present in the > > token, they are disabled by default. > > > > They have to be enabled explicitely before you can exercise the > > privileges. Usually you do this in the same application > > programatically. > > Now I see... > Local administrator Bob reports his User Access Token info > (with Windows whoami, not cygwin whoami) > C:\>whoami /priv > > PRIVILEGES INFORMATION > ---------------------- > > Privilege Name ... State > ========================================= ... ======== > ... > SeBackupPrivilege ... Disabled > SeRestorePrivilege ... Disabled > ... > > C:\> > > Thanks very much. > John Ruckstuhl The clue about UAT and disabled privileges was crucial. Thank you again, Corinna. I'm still doing my cleanup with ordinary Python (not the Cygwin Python linked to the cygwin dll). But I wrote an EnablePrivilege function to enable the privs if if necessary. Best regards, John Ruckstuhl 1 """ 2 This module exports functions for removing old files and directories. 3 4 Functions 5 myremove -- Remove the file or dir, and return the number of bytes freed. 6 """ 7 8 # standard library imports 9 import logging 10 import os 11 import shutil 12 import subprocess 13 14 # related third party imports 15 import win32api 16 import win32security 17 18 # no local application/library specific imports 19 20 21 logger = logging.getLogger(__name__) 22 23 # this custom startupinfo is used to suppress display of the subprocess window 24 startupinfo = subprocess.STARTUPINFO() 25 startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW 26 27 28 def _remove(path): 29 """Remove the file or dir, and return the number of bytes freed.""" 30 31 ... 32 33 34 def myremove(path): 35 """Remove the file or dir, and return the number of bytes freed. 36 37 Attempt removal different ways, until successful (or defeated). 38 39 1. Attempt call _remove (without additional prep). 40 2. Attempt to enable privileges SeBackupPrivilege and 41 SeRestorePrivilege, then call _remove. 42 3. Attempt to takeown, then call _remove. 43 44 The enable privileges approach is ~6x faster than the takeown approach, 45 because takeown of an MEI dir containing ~1K files takes time (8-9 sec). 46 47 Example results, enable privs approach 48 MEI dirs: 113/113 removed, 2047.5 MB recovered in 2.0 minutes 49 50 Example results, takeown approach 51 MEI dirs: 126/126 removed, 2453.8 MB recovered in 15.0 minutes 52 53 By default, the User Access Token for a member of local group 54 Administrators has the privileges SeBackupPrivilege and SeRestorePrivilege, 55 but they are disabled. 56 57 The MEI dirs are typically ~1000 files occupying ~20 MB. 58 The MEI dirs are created by users running the pyinstaller -generated 59 executables ... 60 61 Presumably the process that calls this function is run as a user 62 which is a member of local group "Administrators". So this process will 63 have privileges SeBackupPrivilege and SeRestorePrivilege granted but 64 disabled. Also this process will be able to take ownership, if needed. 65 """ 66 67 logger.debug('%s(%r)', 'myremove', path) 68 69 # first try, attempt _remove 70 try: 71 logger.debug('%s(%r)', '_remove', path) 72 nbytes = _remove(path) 73 return nbytes 74 except WindowsError as e: 75 if e.args == (5, 'Access is denied'): 76 # report the exception and carry on to next attempt 77 logger.debug(e) 78 else: 79 # re-raise any other WindowsError 80 raise 81 82 # second try, attempt enable privs then _remove 83 try: 84 logger.debug('%s(%r)', '_remove', path) 85 EnablePrivilege(win32security.SE_BACKUP_NAME) 86 EnablePrivilege(win32security.SE_RESTORE_NAME) 87 nbytes = _remove(path) 88 return nbytes 89 except Exception as e: 90 # report the exception and carry on to next attempt 91 logger.debug(e) 92 93 # third try, attempt takeown then _remove 94 try: 95 logger.debug('%s(%r)', '_remove', path) 96 takeown(path) 97 nbytes = _remove(path) 98 return nbytes 99 except Exception as e: 100 # report the exception and carry on ... 101 logger.debug(e) 102 103 # accept defeat 104 raise 105 106 107 def takeown(path): 108 subprocess.check_call( 109 [ 110 'takeown', 111 '/A', 112 '/F', path.replace('/', '\\'), 113 '/R', 114 '/D', 'Y', 115 ], 116 startupinfo=startupinfo) 117 118 119 def EnablePrivilege(privilege): 120 """Enable a privilege, if it's already granted in the User Access Token. 121 122 From [1]: 123 When a user holds a privilege, it allows that user to do things that 124 other users without that privilege are not allowed to do. For example, 125 the SeBackupPrivilege allows a user to read any file, even if the 126 security descriptor denies access. 127 But just having the SeBackupPrivilege is not enough: it needs to be 128 enabled 129 130 [1] https://blog.didierstevens.com/2021/07/19/using-sebackupprivilege-with-python # noqa: E501 131 """ 132 hToken = win32security.OpenProcessToken( 133 win32api.GetCurrentProcess(), 134 win32security.TOKEN_ADJUST_PRIVILEGES | win32security.TOKEN_QUERY 135 ) 136 win32security.AdjustTokenPrivileges( 137 hToken, 138 0, 139 [( 140 win32security.LookupPrivilegeValue(None, privilege), 141 win32security.SE_PRIVILEGE_ENABLED 142 )] 143 ) 144 win32api.CloseHandle(hToken) ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2024-04-03 2:08 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-01-22 6:18 cygwin utils can access directory and its contents, but W10 utils claim to have no access, why? John Ruckstuhl 2024-01-22 7:33 ` Brian Inglis 2024-01-22 9:48 ` Corinna Vinschen 2024-01-22 18:59 ` John Ruckstuhl 2024-04-03 2:07 ` John Ruckstuhl
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).