* Strange paths in NTFS reparse points created by Cygwin Setup for e.g. TTF fonts [not found] <245e2446-8c1e-cbaa-a4ad-215d7e766274.ref@yahoo.de> @ 2020-11-05 21:41 ` Michael Soegtrop 2020-11-05 22:39 ` Ken Brown 2020-11-06 5:31 ` L A Walsh 0 siblings, 2 replies; 6+ messages in thread From: Michael Soegtrop @ 2020-11-05 21:41 UTC (permalink / raw) To: cygwin Dear Cyhwin Users and Team, since a while I have issues removing cygwin installations, especially the symlinks to true type fonts in /usr/share/fonts/microsoft. Looking at these links with Windows tools I get: C:\bin\cygwin\usr\share\fonts\microsoft>fsutil reparsepoint query wingding.ttf Reparse Tag Value : 0xa000001d Tag value: Microsoft Tag value: Name Surrogate Reparse Data Length: 0x00000025 Reparse Data: 0000: 02 00 00 00 2f 6d 6e 74 2f 63 2f 57 69 6e 64 6f ..../mnt/c/Windo 0010: 77 73 2f 46 6f 6e 74 73 2f 77 69 6e 67 64 69 6e ws/Fonts/wingdin 0020: 67 2e 74 74 66 g.ttf I wonder if the path "/mnt/c/Windows/Fonts/wingding.ttf" is something which should be written into a NTFS reparse point by cygwin setup. Probably not - it looks like a cygwin path and it is understandable that this confuses NTFS. Btw.: I meanwhile managed to reliably remove cygwin install folders with the below command sequence ($1 being the cygwin folder name) - sorry an ugly mix of bash and DOS tools: PATH_WINFMT="$(cygpath -w -a $1)" rm -rf "$1" && { echo "first removal run successful"; exit; } TAKEOWN /F "$PATH_WINFMT" /R /D Y ICACLS "$PATH_WINFMT" /grant <your user name>:F /T CMD /C "DIR /AL /S /B $PATH_WINFMT" | sed 's|\\|\\\\|g' | tr -d "\r" | while read f; do echo removing reparsepoint "$f"; FSUTIL REPARSEPOINT DELETE "$f"; done rm -rfv "$1" Best regards, Michael ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Strange paths in NTFS reparse points created by Cygwin Setup for e.g. TTF fonts 2020-11-05 21:41 ` Strange paths in NTFS reparse points created by Cygwin Setup for e.g. TTF fonts Michael Soegtrop @ 2020-11-05 22:39 ` Ken Brown 2020-11-08 14:40 ` Michael Soegtrop 2020-11-06 5:31 ` L A Walsh 1 sibling, 1 reply; 6+ messages in thread From: Ken Brown @ 2020-11-05 22:39 UTC (permalink / raw) To: Michael Soegtrop, cygwin On 11/5/2020 4:41 PM, Michael Soegtrop via Cygwin wrote: > Dear Cyhwin Users and Team, > > since a while I have issues removing cygwin installations, especially the > symlinks to true type fonts in /usr/share/fonts/microsoft. > > Looking at these links with Windows tools I get: > > C:\bin\cygwin\usr\share\fonts\microsoft>fsutil reparsepoint query wingding.ttf > Reparse Tag Value : 0xa000001d > Tag value: Microsoft > Tag value: Name Surrogate This is the IO_REPARSE_TAG_LX_SYMLINK tag, which WSL uses for symlinks and which Cygwin also uses by default for symlinks on systems that support it. But you can change this if you don't like it, as I said in my reply to your earlier message about this: https://cygwin.com/pipermail/cygwin/2020-September/246277.html > Reparse Data Length: 0x00000025 > Reparse Data: > 0000: 02 00 00 00 2f 6d 6e 74 2f 63 2f 57 69 6e 64 6f ..../mnt/c/Windo > 0010: 77 73 2f 46 6f 6e 74 73 2f 77 69 6e 67 64 69 6e ws/Fonts/wingdin > 0020: 67 2e 74 74 66 g.ttf > > I wonder if the path "/mnt/c/Windows/Fonts/wingding.ttf" is something which > should be written into a NTFS reparse point by cygwin setup. Probably not - it > looks like a cygwin path and it is understandable that this confuses NTFS. It's not written by setup. It's a Cygwin symlink [also a WSL symlink, as I said above], created by the postinstall script /etc/postinstall/zp_fontconfig_cache_1.sh The fact that it contains a Cygwin path as its reparse data is not at all surprising: ls -l /usr/share/fonts/microsoft/wingding.ttf lrwxrwxrwx ... /usr/share/fonts/microsoft/wingding.ttf -> /c/Windows/Fonts/wingding.ttf Ken ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Strange paths in NTFS reparse points created by Cygwin Setup for e.g. TTF fonts 2020-11-05 22:39 ` Ken Brown @ 2020-11-08 14:40 ` Michael Soegtrop 2020-11-08 16:38 ` Ken Brown 2020-11-09 2:14 ` L A Walsh 0 siblings, 2 replies; 6+ messages in thread From: Michael Soegtrop @ 2020-11-08 14:40 UTC (permalink / raw) To: Ken Brown, cygwin, L A Walsh Hi Ken, L A Walsh, > But you can change this if you don't like it, as I said in my reply to your earlier message about this: I don't know if I like it yet - I am still in the process of understanding what is going on. I maintain the Windows build of a large open source project and we also deliver scripts to setup cygwin on our user's machines. I need to understand what is going on before I make changes to settings. I am afraid that using non standard settings in the end will lead to more confusion than good - especially if people have several cygwin installations. Also I need to be able to support our users in case they have issues with this. The interesting point is that in our CI we do several 100 cygwin installations (sometimes more than 1000) per month. For 95%..99% the removal with "rm -rf" from another cygwin installation works well, in about 1%..5% of the case about 250 symlinks / reparse points remain. This happens all on the same machines with identical scripts. With my manual installations of cygwin it is about the same ratio. I had two cases where removing a cygwin installation from Windows explorer with "shift+delete" did not work cause of these about 250 symlinks - I would say I did roughly 100 installations in this time frame. What I don't understand is why it does work sometimes and sometimes not. I always use the same scripts to install and remove cygwin on the same machines and then do pretty much the same thing with this cygwin (build our open source software) before I delete it. It is unlikely that the issue is that the target files are open as L A Welsh suggested because always either all symlinks or none at all remain. The number is always the same (with recent versions afair 258). @ L A Walsh: you wrote "1) if you try to delete the file in cygwin" Why shouldn't I be able to remove symlinks with rm -rf from within a cygwin? As far as I understand the standard behavior of "rm" is to remove the symlink and not its target. What I do when I remove a cygwin installation in our CI is an "rm -rf" from a different cygwin installation. As I said in most cases this works but rarely it doesn't. In case it doesn't work the symlinks are quite hard to get rid of. FSUTIL REPARSEPOINT DELETE is the only method which works I found so far. Even after a reboot, resetting the ACLs in various ways, ... no usual method to remove these files works. I understand that the contents / path of the symlink is as expected, so I am looking out for other oddities which could explain this behavior. Best regards, Michael ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Strange paths in NTFS reparse points created by Cygwin Setup for e.g. TTF fonts 2020-11-08 14:40 ` Michael Soegtrop @ 2020-11-08 16:38 ` Ken Brown 2020-11-09 2:14 ` L A Walsh 1 sibling, 0 replies; 6+ messages in thread From: Ken Brown @ 2020-11-08 16:38 UTC (permalink / raw) To: Michael Soegtrop, cygwin, L A Walsh On 11/8/2020 9:40 AM, Michael Soegtrop wrote: > Why shouldn't I be able to remove symlinks with rm -rf from within a cygwin? As > far as I understand the standard behavior of "rm" is to remove the symlink and > not its target. What I do when I remove a cygwin installation in our CI is an > "rm -rf" from a different cygwin installation. As I said in most cases this > works but rarely it doesn't. [...] > I understand that the contents / path of the symlink is as expected, so I am > looking out for other oddities which could explain this behavior. I've seen this behavior only in the following situation. Suppose you create a symlink in one Cygwin installation that uses WSL symlinks. Then you try to delete it from a different Cygwin installation with version < 3.1.5 (which is when WSL symlinks were first supported). This will fail, because the older Cygwin installation will see a reparse point of a type that it doesn't recognize. From what you say, it sounds like this doesn't apply to your situation, so I'm at a loss to explain it. Do you get any error message when you're unable to delete a symlink in one Cygwin installation from another? If not, you could try running 'rm <cygwin root>/usr/share/fonts/microsoft/wingding.ttf' under strace to see if that give you any clues. You could also try 'file', 'ls -l', 'stat', etc., to see if Cygwin is correctly recognizing the file as a symlink. For example, I have a Cygwin installation in C:\cygwin64test. From a different Cygwin installation in which C: is mounted on /c, I see the following: $ file /c/cygwin64test/usr/share/fonts/microsoft/wingding.ttf /c/cygwin64test/usr/share/fonts/microsoft/wingding.ttf: symbolic link to /c/Windows/Fonts/wingding.ttf Now I remove that symlink under strace: $ strace -o strace.out rm /c/cygwin64test/usr/share/fonts/microsoft/wingding.ttf In the file strace.out I see 57 26112 [main] rm 39456 symlink_info::check: 0x0 = NtCreateFile (\??\C:\cygwin64test\usr\share\fonts\microsoft\wingding.ttf) 100 26212 [main] rm 39456 symlink_info::check: 29 = symlink.check(C:\cygwin64test\usr\share\fonts\microsoft\wingding.ttf, 0xFFFFB680) (mount_flags 0x4008, path_flags 0x10) 26 26238 [main] rm 39456 path_conv::check: this->path(C:\cygwin64test\usr\share\fonts\microsoft\wingding.ttf), has_acls(1) 25 26263 [main] rm 39456 _unlink_nt: Trying to delete \??\C:\cygwin64test\usr\share\fonts\microsoft\wingding.ttf, isdir = 0 218 26481 [main] rm 39456 _unlink_nt: \??\C:\cygwin64test\usr\share\fonts\microsoft\wingding.ttf, return status = 0x0 33 26514 [main] rm 39456 unlink: 0 = unlink(/c/cygwin64test/usr/share/fonts/microsoft/wingding.ttf) In your case I would expect to see some errors reported when you are unable to remove the symlink. Ken ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Strange paths in NTFS reparse points created by Cygwin Setup for e.g. TTF fonts 2020-11-08 14:40 ` Michael Soegtrop 2020-11-08 16:38 ` Ken Brown @ 2020-11-09 2:14 ` L A Walsh 1 sibling, 0 replies; 6+ messages in thread From: L A Walsh @ 2020-11-09 2:14 UTC (permalink / raw) To: Michael Soegtrop; +Cc: Ken Brown, cygwin On 2020/11/08 06:40, Michael Soegtrop wrote: > What I don't understand is why it does work sometimes and sometimes not. > I always use the same scripts to install and remove cygwin on the same > machines and then do pretty much the same thing with this cygwin (build > our open source software) before I delete it. ---- Likely because the windows settings are different on the various machines -- especially the various policies. But, first, basic, you know that windows symlinks are turned off for normal users by default when they first get it (i.e. they can't create them). Which ways work and who can create them are settings under fsultil/behavior/(set/query) symlinkEvaluation. These 4 settings control directions of symlinking There's another setting that is suppose to control where fonts can be installed, but can't find it off hand -- in the group policy editor. If I remember, it was suppose to allow/disallow fonts being installed in folders other than /windows/fonts. I think the default might be to allow, but am not sure. I thought I disabled it, and what I've seen is the same font-file hardlinked between windows/fonts and the application path. > It is unlikely that the issue is that the target files are open as L A > Welsh suggested because always either all symlinks or none at all > remain. The number is always the same (with recent versions afair 258). ---- Windows doesn't hold open all fonts -- but some subset -- they have some font cache services that may be running at times to support some apps. So could be if one of those apps has run, it started its font caching service, which could lock it. Each font system has a set of "core fonts" that it will load whether they are called upon or not -- that could create a toggle situation between a bunch being used or not. Some applications (though I don't think rm does this) will follow the symlink by default, so when they try to delete a file, it doesn't delete the symlink but tries to delete the font, which may be opened by some service. BTW, are you familiar with 'Process Hacker' (google it, its open source, and hasn't been changed for a while). It can replace the task manager and ProcessExplorer that MS helps distribute. It's more powerful than either. But specifically if you can't delete a file, you can find out what process(es) have the file open. Finding that out might enable you to find the what apps might be running and holding those files open. Symlinks and their older counterparts mountd+junction have different formats for directories and volumes. Some will work with network, some not. If they are all Microsoft windows symlinks, you might look at the fsutil settings -- as well as open files. You maybe said, but don't remember -- is there any error message when you can't delete those files? > > @ L A Walsh: you wrote "1) if you try to delete the file in cygwin" > > Why shouldn't I be able to remove symlinks with rm -rf from within a > cygwin? As far as I understand the standard behavior of "rm" is to > remove the symlink and not its target. --- With posix/linux type symlinks, yes, but am not equally sure that windows behaves the same in all circumstances. One thing you could try is to use cygwin-only symlinks and not use native symlinks -- the cygwin-only links should be more reliably just controlled by cygwin rules. If they are native links, there might be some windows settings interaction(s). > In case it doesn't work the symlinks are quite hard to get rid of. > FSUTIL REPARSEPOINT DELETE is the only method which works I found so > far. Even after a reboot, resetting the ACLs in various ways, ... no > usual method to remove these files works. --- How were these links created? Also, MSsymlinks unlike unix symlinks require the target's existence when they are setup. unix symlinks do no checks on the target. If the target is missing, some operations on that symlink (if it is a win-symlink) might not work as expected. Also note, that these two entries in my root dir look like this from unix: lrwxrwxrwx 1 20 Nov 6 2014 Prog -> /Program Files (x86)/ lrwxrwxrwx 1 13 Apr 21 2013 Prog64 -> Program Files/ Both sorta look like symlinks. But in windows, they look different because they were made differently, and they may have different effects on 'rm' from cygwin. 2014/11/06 19:45 <JUNCTION> Prog [C:\Program Files (x86)] 2013/04/21 22:53 <SYMLINKD> Prog64 [Program Files] 1. Make sure the symlinks you see in cygwin are the same type. 2. you say you have symlinks left over after you try deletion. Do they point to existing files? or not? If not, what happens if you create the target of one of them, and then try deleting the symlink? 3. you said you had to use fsutil to remove some symlinks -- have you tried just 'del' in windows? Anyway, sorry to write so much, but I'm trying to point to how there can be many differences, even when you think you are doing the same thing to delete a cygwin. -linda ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Strange paths in NTFS reparse points created by Cygwin Setup for e.g. TTF fonts 2020-11-05 21:41 ` Strange paths in NTFS reparse points created by Cygwin Setup for e.g. TTF fonts Michael Soegtrop 2020-11-05 22:39 ` Ken Brown @ 2020-11-06 5:31 ` L A Walsh 1 sibling, 0 replies; 6+ messages in thread From: L A Walsh @ 2020-11-06 5:31 UTC (permalink / raw) To: Michael Soegtrop; +Cc: cygwin On 2020/11/05 13:41, Michael Soegtrop via Cygwin wrote: > I wonder if the path "/mnt/c/Windows/Fonts/wingding.ttf" is something > which should be written into a NTFS reparse point by cygwin setup. > Probably not - it looks like a cygwin path and it is understandable that > this confuses NTFS. > ==== Uh....except that "/mnt/c/Windows/Fonts/wingding.ttf" is a valid NT pathname (file and reg). For that matter, "/mnt/\x00\x01\x04" is a valid NT pathname -- how, why?: NT uses a 'count' hidden before the string, so any character is valid. However, the above is not universally true in 'Windows' where some system libraries enforce using backslash as the separator. I'm guessing you get some permission denied messages when you try to delete some fonts. This can happen in 2 cases: 1) if you try to delete the file in cygwin or 2), if some program hardlinked the font to the same in the windows directory instead of symlinking it. With both, it comes down to your delete command trying to delete (for example) a font that is linked into the /windows/fonts dir and some program (or lib) already has it open. Generally, you can't delete *open* files in Windows. In windows, an open will lock some range of bytes on the disk. Windows "lock"s are locks on byte ranges, "on disk" that windows won't let you write to, move or delete while someone has that file open. ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-11-09 2:15 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <245e2446-8c1e-cbaa-a4ad-215d7e766274.ref@yahoo.de> 2020-11-05 21:41 ` Strange paths in NTFS reparse points created by Cygwin Setup for e.g. TTF fonts Michael Soegtrop 2020-11-05 22:39 ` Ken Brown 2020-11-08 14:40 ` Michael Soegtrop 2020-11-08 16:38 ` Ken Brown 2020-11-09 2:14 ` L A Walsh 2020-11-06 5:31 ` L A Walsh
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).