Fun with cp -R error when both foo and foo.exe exist

Fun with cp -R error when both foo and foo.exe exist

[Dan Kegel]

Everybody who uses cygwin knows that if foo and foo.exe exist, you can do
  cp foo bar
  cp foo.exe bar.exe
but not
  cp foo.exe bar.exe
  cp foo bar
because of the unavoidable magic in cygwin that lets you execute foo
when you really mean foo.exe.

Well, it just bit me again during a buildbot try build of electron.
Here's the scenario:

 argv: ['git', '--version']
 argv: ['rm', '-rf',
'/home/buildbot/slave-state/electron-oblong-1-7-x-try-win10/build']
 argv: ['git', 'fetch', '-t', 'git@gitlab.foo.com:platform/electron.git']
 argv: ['git', 'fetch', '-t',
'git@gitlab.foo.com:jisorce/electron.git', 'window_infos']
 argv: ['git', 'reset', '--hard', 'FETCH_HEAD', '--']
 argv: ['git', 'checkout', '-B', 'window_infos']
 argv: ['git', 'submodule', 'sync']
 argv: ['git', 'submodule', 'update', '--init', '--recursive',
'--force', '--checkout']
 argv: ['cp', '-R', '-P', '-p', '-v',
'/home/buildbot/slave-state/electron-oblong-1-7-x-try-win10/source',
'/home/buildbot/slave-state/electron-oblong-1-7-x-try-win10/build']
cp: cannot create regular file
'/home/buildbot/slave-state/electron-oblong-1-7-x-try-win10/build/vendor/libchromiumcontent/vendor/depot_tools/ninja':
File exists

Maybe the order of files reported via the system call cp uses to
expand a directory was unfortunate after that git operation.

The workaround was to nuke the source directory, thus forcing a clean
checkout.  (And alas that means I don't have the directory to inspect
anymore.)

A nice workaround might be for the cygwin version of cp could arrange
to wait to create .exe files until after any potential non-suffixed
file has been processed... not sure how easy that would be.

Apologies if this has already been discussed.
- Dan

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Fun with cp -R error when both foo and foo.exe exist

[Marco Atzeri]

Am 08.10.2018 um 23:24 schrieb Dan Kegel:

>
> A nice workaround might be for the cygwin version of cp could arrange
> to wait to create .exe files until after any potential non-suffixed
> file has been processed... not sure how easy that would be.
>
> Apologies if this has already been discussed.
> - Dan

it is not cp handling the magic, but the cygwin dll.

Regards
Marco


---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
https://www.avast.com/antivirus


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Fun with cp -R error when both foo and foo.exe exist

[Eric Blake]

On 10/9/18 1:22 AM, Marco Atzeri wrote:
> Am 08.10.2018 um 23:24 schrieb Dan Kegel:
> 
>>
>> A nice workaround might be for the cygwin version of cp could arrange
>> to wait to create .exe files until after any potential non-suffixed
>> file has been processed... not sure how easy that would be.
>>
>> Apologies if this has already been discussed.
>> - Dan
> 
> it is not cp handling the magic, but the cygwin dll.

Mostly the cygwin dll, but cp itself DOES have some cygwin-specific 
downstream patches to try and behave smarter.  At any rate, POSIX 
requires cp to handle command line arguments in the order given, but 
does not require recursion to copy in any specific order for files 
within a directory.  But the order that makes the most sense is the 
order that readdir() returns things, because any other order requires 
sorting the files (and thus the memory to track the files) and a longer 
window for races to occur with anything else changing directory contents 
in parallel.  Perhaps you could even argue that readdir() should visit 
'foo' and 'foo.exe' in a particular order, no matter whether the native 
ordering would display them in the opposite order.  But in general, 
whether someone patches the cygwin dll or cp, it seems like some rather 
hairy code for what is normally a rare corner case, so it probably won't 
happen unless someone actually contributes a patch.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Fun with cp -R error when both foo and foo.exe exist

[Dan Kegel]

On Tue, Oct 9, 2018 at 5:03 AM Eric Blake <eblake@redhat.com> wrote:
> whether someone patches the cygwin dll or cp, it seems like some rather
> hairy code for what is normally a rare corner case, so it probably won't
> happen unless someone actually contributes a patch.

Right.  Here's a completely untested guess patch:

--- a/winsup/cygwin/syscalls.cc
+++ b/winsup/cygwin/syscalls.cc
@@ -1363,7 +1363,15 @@ open (const char *unix_path, int flags, ...)
          opt |= PC_CTTY;
        }

-      if (!(fh = build_fh_name (unix_path, opt, stat_suffixes)))
+      suffix_info *suffices = stat_suffixes;
+      if ((opt & (O_WRONLY|O_BINARY)) == (O_WRONLY|O_BINARY))
+        {
+         /* Hack so 'cp foo bar' doesn't abort with
+             'cp: cannot create regular file bar: File exists'
+            if bar.exe already exists  */
+       suffices = NULL;
+       }
+      if (!(fh = build_fh_name (unix_path, opt, suffices)))
        __leave;                /* errno already set */
       if ((flags & O_NOFOLLOW) && fh->issymlink ())
        {

Feel free to throw rotten fruit :-)

- Dan

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Fun with cp -R error when both foo and foo.exe exist

[Eric Blake]

On 10/9/18 11:21 AM, Dan Kegel wrote:
> On Tue, Oct 9, 2018 at 5:03 AM Eric Blake <eblake@redhat.com> wrote:
>> whether someone patches the cygwin dll or cp, it seems like some rather
>> hairy code for what is normally a rare corner case, so it probably won't
>> happen unless someone actually contributes a patch.
> 
> Right.  Here's a completely untested guess patch:
> 
> --- a/winsup/cygwin/syscalls.cc
> +++ b/winsup/cygwin/syscalls.cc
> @@ -1363,7 +1363,15 @@ open (const char *unix_path, int flags, ...)
>            opt |= PC_CTTY;
>          }
> 
> -      if (!(fh = build_fh_name (unix_path, opt, stat_suffixes)))
> +      suffix_info *suffices = stat_suffixes;

suffices is a completely different word; did you mean suffixes?

> +      if ((opt & (O_WRONLY|O_BINARY)) == (O_WRONLY|O_BINARY))

O_WRONLY need not be a bitmask; it's better to use O_ACCMODE for the 
bitmask, as in
  opt & (O_ACCMODE|O_BINARY) == O_WRONLY|O_BINARY

> +        {
> +         /* Hack so 'cp foo bar' doesn't abort with
> +             'cp: cannot create regular file bar: File exists'
> +            if bar.exe already exists  */
> +       suffices = NULL;
> +       }
> +      if (!(fh = build_fh_name (unix_path, opt, suffices)))

So, with mistakes corrected, your hack is that cygwin doing auto-suffix 
probing should be suppressed in the case of opening a file O_WRONLY? Why 
does O_BINARY have to fit in the equation?

>          __leave;                /* errno already set */
>         if ((flags & O_NOFOLLOW) && fh->issymlink ())
>          {
> 
> Feel free to throw rotten fruit :-)
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Fun with cp -R error when both foo and foo.exe exist

[Dan Kegel]

Right.  My thinking was that the auto-suffix probing makes
sense when running or checking for a program, but
not when creating a program.  (I could be wrong, and I imagine
if I actually compiled and tested the hack I'd find out pretty quickly
if it was at least obviously wrong.)

O_BINARY doesn't need to be there, but I included it as
an additional filter because cp uses it.  Can leave it out.

I misremembered suffices as a britishism, sorry.
- Dan

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple