Deleting files with open handles on subst'd drives

Deleting files with open handles on subst'd drives

[David Macek]

[-- Attachment #1: Type: text/plain, Size: 1013 bytes --]

Hello everyone.

Some time ago, I encountered a problem in a script when run on Cygwin/MSYS2. To add some drama, I'll mention that the problem caused some data loss (nothing important though). The problem is that some operations on open files seem to fail on subst'd drives (but not on regular ones).

To reproduce the problem, choose some writable directory and run:
$ subst X: C:/somesubdir
$ cd /cygdrive/x/
$ touch foo
$ { rm foo; touch foo; } <foo
touch: cannot touch 'foo': Permission denied

(Do not subst to just C:/, because that somehow doesn't lead to the error.)

There's also a C program that does essentially the same thing: <https://gist.github.com/elieux/6463521192baed613099>

I compared this with the same operations running on a regular drive, but I couldn't find any difference in how `rm` deletes the file nor in how `touch` creates the file. Just the result of the last NtCreateFile is different (STATUS_SUCCESS vs. STATUS_DELETE_PENDING).

Thoughts?

-- 
David Macek


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4234 bytes --]

Re: Deleting files with open handles on subst'd drives

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1812 bytes --]

On Apr 23 01:44, David Macek wrote:
> Hello everyone.
> 
> Some time ago, I encountered a problem in a script when run on Cygwin/MSYS2. To add some drama, I'll mention that the problem caused some data loss (nothing important though). The problem is that some operations on open files seem to fail on subst'd drives (but not on regular ones).
> 
> To reproduce the problem, choose some writable directory and run:
> $ subst X: C:/somesubdir
> $ cd /cygdrive/x/
> $ touch foo
> $ { rm foo; touch foo; } <foo
> touch: cannot touch 'foo': Permission denied
> 
> (Do not subst to just C:/, because that somehow doesn't lead to the error.)
> 
> There's also a C program that does essentially the same thing: <https://gist.github.com/elieux/6463521192baed613099>
> 
> I compared this with the same operations running on a regular drive,
> but I couldn't find any difference in how `rm` deletes the file nor in
> how `touch` creates the file. Just the result of the last NtCreateFile
> is different (STATUS_SUCCESS vs. STATUS_DELETE_PENDING).

It's a result of how Cygwin tries to workaround the weird Windows
behaviour that a deleted, but still opened file is not invisible in the
filesystem, as desired per POSIX.  Cygwin then tries to move the
file out of the way, on local drives into the recycler.  This fails
on a subst drive pointing to some subdir apparently, so Cygwin just
falls back to what Windows does.

I have no simple workaround for that.  In theory there should be a
way to check the drive for being a virtual drive and then using the
recycler of the drive it's pointing to for the aforementioned operation.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Deleting files with open handles on subst'd drives

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2157 bytes --]

On Apr 23 10:15, Corinna Vinschen wrote:
> On Apr 23 01:44, David Macek wrote:
> > Hello everyone.
> > 
> > Some time ago, I encountered a problem in a script when run on Cygwin/MSYS2. To add some drama, I'll mention that the problem caused some data loss (nothing important though). The problem is that some operations on open files seem to fail on subst'd drives (but not on regular ones).
> > 
> > To reproduce the problem, choose some writable directory and run:
> > $ subst X: C:/somesubdir
> > $ cd /cygdrive/x/
> > $ touch foo
> > $ { rm foo; touch foo; } <foo
> > touch: cannot touch 'foo': Permission denied
> > 
> > (Do not subst to just C:/, because that somehow doesn't lead to the error.)
> > 
> > There's also a C program that does essentially the same thing: <https://gist.github.com/elieux/6463521192baed613099>
> > 
> > I compared this with the same operations running on a regular drive,
> > but I couldn't find any difference in how `rm` deletes the file nor in
> > how `touch` creates the file. Just the result of the last NtCreateFile
> > is different (STATUS_SUCCESS vs. STATUS_DELETE_PENDING).
> 
> It's a result of how Cygwin tries to workaround the weird Windows
> behaviour that a deleted, but still opened file is not invisible in the
> filesystem, as desired per POSIX.  Cygwin then tries to move the
> file out of the way, on local drives into the recycler.  This fails
> on a subst drive pointing to some subdir apparently, so Cygwin just
> falls back to what Windows does.
> 
> I have no simple workaround for that.  In theory there should be a
> way to check the drive for being a virtual drive and then using the
> recycler of the drive it's pointing to for the aforementioned operation.

I hacked a bit on that and it seems the solution was easier than I thought.
I uploaded new developer snaphshots to https://cygwin.com/snapshots/
Replacing the Cygwin DLL alone is sufficient for testing.  Please give it
a try.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Deleting files with open handles on subst'd drives

[Nick Hansen]

Unsubscribe

Sent from my BlackBerry 10 smartphone on the Verizon Wireless 4G LTE network.
  Original Message  
From: Corinna Vinschen - corinna-cygwin@cygwin.com
Sent: Thursday, April 23, 2015 9:25 AM
To: cygwin@cygwin.com
Reply To: cygwin@cygwin.com
Subject: Re: Deleting files with open handles on subst'd drives

On Apr 23 10:15, Corinna Vinschen wrote:
> On Apr 23 01:44, David Macek wrote:
> > Hello everyone.
> > 
> > Some time ago, I encountered a problem in a script when run on Cygwin/MSYS2. To add some drama, I'll mention that the problem caused some data loss (nothing important though). The problem is that some operations on open files seem to fail on subst'd drives (but not on regular ones).
> > 
> > To reproduce the problem, choose some writable directory and run:
> > $ subst X: C:/somesubdir
> > $ cd /cygdrive/x/
> > $ touch foo
> > $ { rm foo; touch foo; } <foo
> > touch: cannot touch 'foo': Permission denied
> > 
> > (Do not subst to just C:/, because that somehow doesn't lead to the error.)
> > 
> > There's also a C program that does essentially the same thing: <https://gist.github.com/elieux/6463521192baed613099>
> > 
> > I compared this with the same operations running on a regular drive,
> > but I couldn't find any difference in how `rm` deletes the file nor in
> > how `touch` creates the file. Just the result of the last NtCreateFile
> > is different (STATUS_SUCCESS vs. STATUS_DELETE_PENDING).
> 
> It's a result of how Cygwin tries to workaround the weird Windows
> behaviour that a deleted, but still opened file is not invisible in the
> filesystem, as desired per POSIX. Cygwin then tries to move the
> file out of the way, on local drives into the recycler. This fails
> on a subst drive pointing to some subdir apparently, so Cygwin just
> falls back to what Windows does.
> 
> I have no simple workaround for that. In theory there should be a
> way to check the drive for being a virtual drive and then using the
> recycler of the drive it's pointing to for the aforementioned operation.

I hacked a bit on that and it seems the solution was easier than I thought.
I uploaded new developer snaphshots to https://cygwin.com/snapshots/
Replacing the Cygwin DLL alone is sufficient for testing. Please give it
a try.


Thanks,
Corinna

-- 
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Deleting files with open handles on subst'd drives

[David Macek]

[-- Attachment #1: Type: text/plain, Size: 216 bytes --]

On 23. 4. 2015 15:25, Nick Hansen wrote:
> Unsubscribe

Are you trying to unsubscribe from a thread, or from the whole mailing list? Please read:

http://cygwin.com/ml/#unsubscribe-simple

-- 
David Macek


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4234 bytes --]

RE: Deleting files with open handles on subst'd drives

[Nick Hansen]

David,

I want to be unsubscribed from the mailing list and thread.  I will not click the link below because I do not know if it is a virus.  Please remove immediately.



-----Original Message-----
From: David Macek - david.macek.0@gmail.com [mailto:cygwin2.ares0.2af6b9dd8b.david.macek.0#gmail.com@ob.0sg.net] 
Sent: Thursday, April 23, 2015 10:32 AM
To: cygwin@cygwin.com
Subject: Re: Deleting files with open handles on subst'd drives

On 23. 4. 2015 15:25, Nick Hansen wrote:
> Unsubscribe

Are you trying to unsubscribe from a thread, or from the whole mailing list? Please read:

http://cygwin.com/ml/#unsubscribe-simple

-- 
David Macek



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Deleting files with open handles on subst'd drives

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1449 bytes --]

On 04/23/2015 08:36 AM, Nick Hansen wrote:
> David,
> 
> I want to be unsubscribed from the mailing list and thread.  I will not click the link below because I do not know if it is a virus.  Please remove immediately.

It is not a virus.  If it makes you feel better, manually type the web
site address directly into your browser instead of clicking on the link
in this email.

> Are you trying to unsubscribe from a thread, or from the whole mailing list? Please read:
> 
> http://cygwin.com/ml/#unsubscribe-simple

Really - YOU are the one that is supposed to be unsubscribing yourself.
 That web site has all the directions on how to do so, and it is
intentionally in the signature appended to every email sent through this
list.  Or, if you view the raw source of the email (often by 'ctrl-u',
depending on what mail client you are reading this in), you will see a
header line starting with List-Unsubscribe: that also has the direct
email address (different from the main list address) that will
unsubscribe you without visiting any web site (by the way, all GOOD
mailing lists have a List-Unsubscribe header; if you get a mail without
such a header, then you can be sure it was a spammer).   Meanwhile,
spamming hundreds of other readers about your inability to follow
directions is not helping your cause.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: Deleting files with open handles on subst'd drives

[David Macek]

[-- Attachment #1: Type: text/plain, Size: 528 bytes --]

On 23. 4. 2015 15:24, Corinna Vinschen wrote:
> I hacked a bit on that and it seems the solution was easier than I thought.
> I uploaded new developer snaphshots to https://cygwin.com/snapshots/
> Replacing the Cygwin DLL alone is sufficient for testing.  Please give it
> a try.

I'm sorry for a late reply. The snapshot seems to fix the issue. Based on a quick read of your patch, I thought I could break it with a second-level subst (subst->subst->volume), but it worked even then.

Thank you.

-- 
David Macek


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4234 bytes --]

Re: Deleting files with open handles on subst'd drives

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 752 bytes --]

On Apr 24 03:52, David Macek wrote:
> On 23. 4. 2015 15:24, Corinna Vinschen wrote:
> > I hacked a bit on that and it seems the solution was easier than I thought.
> > I uploaded new developer snaphshots to https://cygwin.com/snapshots/
> > Replacing the Cygwin DLL alone is sufficient for testing.  Please give it
> > a try.
> 
> I'm sorry for a late reply. The snapshot seems to fix the issue. Based
> on a quick read of your patch, I thought I could break it with a
> second-level subst (subst->subst->volume), but it worked even then.

Good news for the lazy dev.


Thanks!
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]