* Native symbolic link behavior is broken and makes backups using Cygwin command line tools impossible
@ 2021-01-03 4:16 Matt D.
2021-01-03 5:00 ` Brian Inglis
0 siblings, 1 reply; 11+ messages in thread
From: Matt D. @ 2021-01-03 4:16 UTC (permalink / raw)
To: cygwin
I have a folder with a lot of native Windows symbolic links. I want to
copy this folder.
I cannot rsync or cp this folder due to Cygwin being unable to create
symbolic links without also wanting to verify the link target. This
can be demonstrated:
$ ln -s a b
ln: failed to create symbolic link 'b': No such file or directory
If I create a test directory folder_a/ and folder_b/. Inside I will
"touch a" and "ln -s a b".
I cannot rsync this folder:
$ rsync -a folder_a/ folder_b/
rsync: symlink "folder_a/b" -> "a" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous
errors) (code 23) at main.c(1306) [sender=3.2.0dev]
Using "cp -a folder_a/* folder_b/" in this test case DOES work but
this is simply because files were returned in the correct order and
the link could be created.
This can be demonstrated where this works fine:
$ cp -a folder_a/a folder_a/b folder_b/
But this does not:
$ cp -a folder_a/b folder_a/b folder_b/
cp: cannot create symbolic link 'folder_b/b': No such file or directory
cp: warning: source file 'folder_a/b' specified more than once
The order in which files are returned while listing them in a
directory and necessitating their pre-existence while performing a
deep copy is impossible. It's also very normal for symbolic links to
exist which may or may not point to a valid target depending on the
observing path.
Windows does NOT require a link to be valid before creation. This can
be demonstrated with mklink:
C:\mklink b a
symbolic link created for b <<===>> a
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Native symbolic link behavior is broken and makes backups using Cygwin command line tools impossible
2021-01-03 4:16 Native symbolic link behavior is broken and makes backups using Cygwin command line tools impossible Matt D.
@ 2021-01-03 5:00 ` Brian Inglis
2021-01-04 10:36 ` Matt D.
0 siblings, 1 reply; 11+ messages in thread
From: Brian Inglis @ 2021-01-03 5:00 UTC (permalink / raw)
To: cygwin
On 2021-01-02 21:16, Matt D. via Cygwin wrote:
> I have a folder with a lot of native Windows symbolic links. I want to
> copy this folder.
>
> I cannot rsync or cp this folder due to Cygwin being unable to create
> symbolic links without also wanting to verify the link target. This
> can be demonstrated:
>
> $ ln -s a b
> ln: failed to create symbolic link 'b': No such file or directory
>
> If I create a test directory folder_a/ and folder_b/. Inside I will
> "touch a" and "ln -s a b".
>
> I cannot rsync this folder:
>
> $ rsync -a folder_a/ folder_b/
> rsync: symlink "folder_a/b" -> "a" failed: No such file or directory (2)
> rsync error: some files/attrs were not transferred (see previous
> errors) (code 23) at main.c(1306) [sender=3.2.0dev]
>
> Using "cp -a folder_a/* folder_b/" in this test case DOES work but
> this is simply because files were returned in the correct order and
> the link could be created.
>
> This can be demonstrated where this works fine:
>
> $ cp -a folder_a/a folder_a/b folder_b/
>
> But this does not:
>
> $ cp -a folder_a/b folder_a/b folder_b/
> cp: cannot create symbolic link 'folder_b/b': No such file or directory
> cp: warning: source file 'folder_a/b' specified more than once
>
> The order in which files are returned while listing them in a
> directory and necessitating their pre-existence while performing a
> deep copy is impossible. It's also very normal for symbolic links to
> exist which may or may not point to a valid target depending on the
> observing path.
>
> Windows does NOT require a link to be valid before creation. This can
> be demonstrated with mklink:
>
> C:\mklink b a
> symbolic link created for b <<===>> a
Depending on exactly what you have and what you want to do try:
$ cp -av
$ cp -dv
$ cp -Pv
or
> robocopy /sl
--
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Native symbolic link behavior is broken and makes backups using Cygwin command line tools impossible
2021-01-03 5:00 ` Brian Inglis
@ 2021-01-04 10:36 ` Matt D.
2021-01-04 12:30 ` Eliot Moss
0 siblings, 1 reply; 11+ messages in thread
From: Matt D. @ 2021-01-04 10:36 UTC (permalink / raw)
To: cygwin
Did you try any of my test cases? This can't and doesn't work for the
reasons I outlined in my previous message:
$ cp -av folder_a/a folder_b/
'folder_a/a' -> 'folder_b/a'
cp: cannot create symbolic link 'folder_b/a': No such file or directory
$ cp -dv folder_a/a folder_b/
'folder_a/a' -> 'folder_b/a'
cp: cannot create symbolic link 'folder_b/a': No such file or directory
$ cp -Pv folder_a/a folder_b/
'folder_a/a' -> 'folder_b/a'
cp: cannot create symbolic link 'folder_b/a': No such file or directory
On Sun, Jan 3, 2021 at 12:00 AM Brian Inglis
<Brian.Inglis@systematicsw.ab.ca> wrote:
>
> On 2021-01-02 21:16, Matt D. via Cygwin wrote:
> > I have a folder with a lot of native Windows symbolic links. I want to
> > copy this folder.
> >
> > I cannot rsync or cp this folder due to Cygwin being unable to create
> > symbolic links without also wanting to verify the link target. This
> > can be demonstrated:
> >
> > $ ln -s a b
> > ln: failed to create symbolic link 'b': No such file or directory
> >
> > If I create a test directory folder_a/ and folder_b/. Inside I will
> > "touch a" and "ln -s a b".
> >
> > I cannot rsync this folder:
> >
> > $ rsync -a folder_a/ folder_b/
> > rsync: symlink "folder_a/b" -> "a" failed: No such file or directory (2)
> > rsync error: some files/attrs were not transferred (see previous
> > errors) (code 23) at main.c(1306) [sender=3.2.0dev]
> >
> > Using "cp -a folder_a/* folder_b/" in this test case DOES work but
> > this is simply because files were returned in the correct order and
> > the link could be created.
> >
> > This can be demonstrated where this works fine:
> >
> > $ cp -a folder_a/a folder_a/b folder_b/
> >
> > But this does not:
> >
> > $ cp -a folder_a/b folder_a/b folder_b/
> > cp: cannot create symbolic link 'folder_b/b': No such file or directory
> > cp: warning: source file 'folder_a/b' specified more than once
> >
> > The order in which files are returned while listing them in a
> > directory and necessitating their pre-existence while performing a
> > deep copy is impossible. It's also very normal for symbolic links to
> > exist which may or may not point to a valid target depending on the
> > observing path.
> >
> > Windows does NOT require a link to be valid before creation. This can
> > be demonstrated with mklink:
> >
> > C:\mklink b a
> > symbolic link created for b <<===>> a
>
> Depending on exactly what you have and what you want to do try:
>
> $ cp -av
> $ cp -dv
> $ cp -Pv
> or
> > robocopy /sl
>
> --
> Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
>
> This email may be disturbing to some readers as it contains
> too much technical detail. Reader discretion is advised.
> [Data in binary units and prefixes, physical quantities in SI.]
> --
> Problem reports: https://cygwin.com/problems.html
> FAQ: https://cygwin.com/faq/
> Documentation: https://cygwin.com/docs.html
> Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Native symbolic link behavior is broken and makes backups using Cygwin command line tools impossible
2021-01-04 10:36 ` Matt D.
@ 2021-01-04 12:30 ` Eliot Moss
2021-01-04 15:27 ` Matt D.
0 siblings, 1 reply; 11+ messages in thread
From: Eliot Moss @ 2021-01-04 12:30 UTC (permalink / raw)
To: cygwin
On 1/4/2021 5:36 AM, Matt D. via Cygwin wrote:
> Did you try any of my test cases? This can't and doesn't work for the
> reasons I outlined in my previous message:
>
> $ cp -av folder_a/a folder_b/
> 'folder_a/a' -> 'folder_b/a'
> cp: cannot create symbolic link 'folder_b/a': No such file or directory
>
> $ cp -dv folder_a/a folder_b/
> 'folder_a/a' -> 'folder_b/a'
> cp: cannot create symbolic link 'folder_b/a': No such file or directory
>
> $ cp -Pv folder_a/a folder_b/
> 'folder_a/a' -> 'folder_b/a'
> cp: cannot create symbolic link 'folder_b/a': No such file or directory
So did you mkdir folder_b first? I don't think cp will create it for you. I
tried the commands above with folder_b not existing and got the behavior you
indicated, but when I created folder_b first, all three cp commands worked.
This overall behavior does not surprise me.
On the other hand, if I have folder_b non-existing and do (e.g.)
cp -rav folder_a folder_b
then it _does_ create folder_b, and also copies the links.
HTH - Eliot Moss
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Native symbolic link behavior is broken and makes backups using Cygwin command line tools impossible
2021-01-04 12:30 ` Eliot Moss
@ 2021-01-04 15:27 ` Matt D.
2021-01-04 15:46 ` Jeffrey Altman
` (2 more replies)
0 siblings, 3 replies; 11+ messages in thread
From: Matt D. @ 2021-01-04 15:27 UTC (permalink / raw)
Cc: cygwin
I think there is a misunderstanding with how to set up your
environment to reproduce my test cases. I did state in the subject
"native symbolic links" but I can see that this can be misinterpreted
and I should have clarified.
I am using symbolic links native to Windows. My CYGWIN environment
variable has been set to "winsymlinks:nativestrict" and my account has
permission to make symbolic links. This is an issue specifically with
Cygwin; I have no problems making links at the windows command line.
Cygwin also does not have a problem making symbolic links-- if the
target already exists. The issue is that I cannot create native
symbolic links with Cygwin for targets that DON'T exist.
The normal behavior for both Windows and Linux is to create the
symbolic link whether the target exists or not. I don't know why
Cygwin fails to do this only for native Windows symbolic links. It
does not have a problem creating links to any target with the default
Cygwin (non-Windows) symbolic links.
On Mon, Jan 4, 2021 at 7:30 AM Eliot Moss <moss@cs.umass.edu> wrote:
>
> On 1/4/2021 5:36 AM, Matt D. via Cygwin wrote:
> > Did you try any of my test cases? This can't and doesn't work for the
> > reasons I outlined in my previous message:
> >
> > $ cp -av folder_a/a folder_b/
> > 'folder_a/a' -> 'folder_b/a'
> > cp: cannot create symbolic link 'folder_b/a': No such file or directory
> >
> > $ cp -dv folder_a/a folder_b/
> > 'folder_a/a' -> 'folder_b/a'
> > cp: cannot create symbolic link 'folder_b/a': No such file or directory
> >
> > $ cp -Pv folder_a/a folder_b/
> > 'folder_a/a' -> 'folder_b/a'
> > cp: cannot create symbolic link 'folder_b/a': No such file or directory
>
> So did you mkdir folder_b first? I don't think cp will create it for you. I
> tried the commands above with folder_b not existing and got the behavior you
> indicated, but when I created folder_b first, all three cp commands worked.
> This overall behavior does not surprise me.
>
> On the other hand, if I have folder_b non-existing and do (e.g.)
>
> cp -rav folder_a folder_b
>
> then it _does_ create folder_b, and also copies the links.
>
> HTH - Eliot Moss
> --
> Problem reports: https://cygwin.com/problems.html
> FAQ: https://cygwin.com/faq/
> Documentation: https://cygwin.com/docs.html
> Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Native symbolic link behavior is broken and makes backups using Cygwin command line tools impossible
2021-01-04 15:27 ` Matt D.
@ 2021-01-04 15:46 ` Jeffrey Altman
2021-01-12 17:59 ` Matt D.
2021-01-04 16:32 ` Eliot Moss
2021-01-04 17:58 ` Achim Gratz
2 siblings, 1 reply; 11+ messages in thread
From: Jeffrey Altman @ 2021-01-04 15:46 UTC (permalink / raw)
To: cygwin
[-- Attachment #1: Type: text/plain, Size: 1257 bytes --]
On 1/4/2021 10:27 AM, Matt D. via Cygwin (cygwin@cygwin.com) wrote:
> I am using symbolic links native to Windows. My CYGWIN environment
> variable has been set to "winsymlinks:nativestrict" and my account has
> permission to make symbolic links. This is an issue specifically with
> Cygwin; I have no problems making links at the windows command line.
> Cygwin also does not have a problem making symbolic links-- if the
> target already exists. The issue is that I cannot create native
> symbolic links with Cygwin for targets that DON'T exist.
>
> The normal behavior for both Windows and Linux is to create the
> symbolic link whether the target exists or not. I don't know why
> Cygwin fails to do this only for native Windows symbolic links. It
> does not have a problem creating links to any target with the default
> Cygwin (non-Windows) symbolic links.
Windows native symlinks encode the object type of the target and the
encoded type must match that of the target or the link will not work
when the target exists.
A UNIX symlink does not encode any details of the target.
Cygwin doesn't know what type of native symlink to create if the
target does not exist.
I hope this knowledge helps.
Jeffrey Altman
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4080 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Native symbolic link behavior is broken and makes backups using Cygwin command line tools impossible
2021-01-04 15:27 ` Matt D.
2021-01-04 15:46 ` Jeffrey Altman
@ 2021-01-04 16:32 ` Eliot Moss
2021-01-04 17:58 ` Achim Gratz
2 siblings, 0 replies; 11+ messages in thread
From: Eliot Moss @ 2021-01-04 16:32 UTC (permalink / raw)
To: Matt D.; +Cc: cygwin
On 1/4/2021 10:27 AM, Matt D. via Cygwin wrote:
> I think there is a misunderstanding with how to set up your
> environment to reproduce my test cases. I did state in the subject
> "native symbolic links" but I can see that this can be misinterpreted
> and I should have clarified.
>
> I am using symbolic links native to Windows. My CYGWIN environment
> variable has been set to "winsymlinks:nativestrict" and my account has
> permission to make symbolic links. This is an issue specifically with
> Cygwin; I have no problems making links at the windows command line.
> Cygwin also does not have a problem making symbolic links-- if the
> target already exists. The issue is that I cannot create native
> symbolic links with Cygwin for targets that DON'T exist.
>
> The normal behavior for both Windows and Linux is to create the
> symbolic link whether the target exists or not. I don't know why
> Cygwin fails to do this only for native Windows symbolic links. It
> does not have a problem creating links to any target with the default
> Cygwin (non-Windows) symbolic links.
Ok, I see the behavior now that you are talking about. You can get it with ln
without any need for cp. With winsymlinks:nativestrict, if I do:
ln -s foo bar
and foo does not exist, it refuses to create the link. As you found, it also
refuses to cp it. However, I _was_ able to mv it.
Regards - Eliot
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Native symbolic link behavior is broken and makes backups using Cygwin command line tools impossible
2021-01-04 15:27 ` Matt D.
2021-01-04 15:46 ` Jeffrey Altman
2021-01-04 16:32 ` Eliot Moss
@ 2021-01-04 17:58 ` Achim Gratz
2 siblings, 0 replies; 11+ messages in thread
From: Achim Gratz @ 2021-01-04 17:58 UTC (permalink / raw)
To: cygwin
Matt D. via Cygwin writes:
> The normal behavior for both Windows and Linux is to create the
> symbolic link whether the target exists or not.
That's not the case on Windows or rather it has restrictions that can
trip you up. On Windows you must specify if the symbolic link points to
a directory or a file in order to create a symbolic link to a
non-existing file. If that information is not available, it is
obviously safer to not create the symbolic link.
Regards,
Achim.
--
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+
SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Native symbolic link behavior is broken and makes backups using Cygwin command line tools impossible
2021-01-04 15:46 ` Jeffrey Altman
@ 2021-01-12 17:59 ` Matt D.
2021-01-12 22:18 ` Andrey Repin
0 siblings, 1 reply; 11+ messages in thread
From: Matt D. @ 2021-01-12 17:59 UTC (permalink / raw)
Cc: cygwin
Operations like cp and rsync etc. should still work. The type of
symbolic link to be created can be read from the file attributes of
the one being copied.
On Mon, Jan 4, 2021 at 10:46 AM Jeffrey Altman via Cygwin
<cygwin@cygwin.com> wrote:
>
> On 1/4/2021 10:27 AM, Matt D. via Cygwin (cygwin@cygwin.com) wrote:
> > I am using symbolic links native to Windows. My CYGWIN environment
> > variable has been set to "winsymlinks:nativestrict" and my account has
> > permission to make symbolic links. This is an issue specifically with
> > Cygwin; I have no problems making links at the windows command line.
> > Cygwin also does not have a problem making symbolic links-- if the
> > target already exists. The issue is that I cannot create native
> > symbolic links with Cygwin for targets that DON'T exist.
> >
> > The normal behavior for both Windows and Linux is to create the
> > symbolic link whether the target exists or not. I don't know why
> > Cygwin fails to do this only for native Windows symbolic links. It
> > does not have a problem creating links to any target with the default
> > Cygwin (non-Windows) symbolic links.
>
> Windows native symlinks encode the object type of the target and the
> encoded type must match that of the target or the link will not work
> when the target exists.
>
> A UNIX symlink does not encode any details of the target.
>
> Cygwin doesn't know what type of native symlink to create if the
> target does not exist.
>
> I hope this knowledge helps.
>
> Jeffrey Altman
>
>
>
>
>
> --
> Problem reports: https://cygwin.com/problems.html
> FAQ: https://cygwin.com/faq/
> Documentation: https://cygwin.com/docs.html
> Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Native symbolic link behavior is broken and makes backups using Cygwin command line tools impossible
2021-01-12 17:59 ` Matt D.
@ 2021-01-12 22:18 ` Andrey Repin
2021-01-14 0:09 ` Matt D.
0 siblings, 1 reply; 11+ messages in thread
From: Andrey Repin @ 2021-01-12 22:18 UTC (permalink / raw)
To: Matt D., cygwin
Greetings, Matt D.!
Please no top posting in this list.
> On Mon, Jan 4, 2021 at 10:46 AM Jeffrey Altman via Cygwin wrote:
>>
>> On 1/4/2021 10:27 AM, Matt D. via Cygwin wrote:
>> > I am using symbolic links native to Windows. My CYGWIN environment
>> > variable has been set to "winsymlinks:nativestrict" and my account has
>> > permission to make symbolic links. This is an issue specifically with
>> > Cygwin; I have no problems making links at the windows command line.
>> > Cygwin also does not have a problem making symbolic links-- if the
>> > target already exists. The issue is that I cannot create native
>> > symbolic links with Cygwin for targets that DON'T exist.
>> >
>> > The normal behavior for both Windows and Linux is to create the
>> > symbolic link whether the target exists or not. I don't know why
>> > Cygwin fails to do this only for native Windows symbolic links. It
>> > does not have a problem creating links to any target with the default
>> > Cygwin (non-Windows) symbolic links.
>>
>> Windows native symlinks encode the object type of the target and the
>> encoded type must match that of the target or the link will not work
>> when the target exists.
>>
>> A UNIX symlink does not encode any details of the target.
>>
>> Cygwin doesn't know what type of native symlink to create if the
>> target does not exist.
> Operations like cp and rsync etc. should still work. The type of
> symbolic link to be created can be read from the file attributes of
> the one being copied.
Again, the properties are read BY SYSTEM at the moment the link is created.
In other words, Windows is unable to create a link to nonexistent
(unreachable) target (not directly, at least).
--
With best regards,
Andrey Repin
Wednesday, January 13, 2021 1:14:52
Sorry for my terrible english...
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Native symbolic link behavior is broken and makes backups using Cygwin command line tools impossible
2021-01-12 22:18 ` Andrey Repin
@ 2021-01-14 0:09 ` Matt D.
0 siblings, 0 replies; 11+ messages in thread
From: Matt D. @ 2021-01-14 0:09 UTC (permalink / raw)
To: cygwin
On Tue, Jan 12, 2021 at 5:20 PM Andrey Repin <anrdaemon@yandex.ru> wrote:
>
> Again, the properties are read BY SYSTEM at the moment the link is created.
> In other words, Windows is unable to create a link to nonexistent
> (unreachable) target (not directly, at least).
This is false. Links can be created arbitrarily both at the Windows
command line using MKLINK and programmatically with the
CreateSymbolicLink function. The information about what type of link
to make during a copy operation can be read from the link being
copied. The target does not have to exist for a link to be created.
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2021-01-14 0:10 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-03 4:16 Native symbolic link behavior is broken and makes backups using Cygwin command line tools impossible Matt D.
2021-01-03 5:00 ` Brian Inglis
2021-01-04 10:36 ` Matt D.
2021-01-04 12:30 ` Eliot Moss
2021-01-04 15:27 ` Matt D.
2021-01-04 15:46 ` Jeffrey Altman
2021-01-12 17:59 ` Matt D.
2021-01-12 22:18 ` Andrey Repin
2021-01-14 0:09 ` Matt D.
2021-01-04 16:32 ` Eliot Moss
2021-01-04 17:58 ` Achim Gratz
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).