* Re: Most git executables are hard links to git.exe?
2023-07-21 21:54 ` Jim Garrison
@ 2023-07-22 5:55 ` Larry
2023-07-22 17:33 ` Adam Dinwoodie
2023-07-31 13:36 ` Andrey Repin
2 siblings, 0 replies; 8+ messages in thread
From: Larry @ 2023-07-22 5:55 UTC (permalink / raw)
To: cygwin; +Cc: jhg
On 7/21/23 17:54, Jim Garrison via Cygwin wrote:
> On 07/21/23 14:52, Brian Inglis wrote:
>> On 2023-07-21 14:59, Jim Garrison via Cygwin wrote:
>>> Git comes with over 100 executables, mostly in /usr/libexec/git-core,
>>> that all appear to be *hard* links to /bin/git, in both Cygwin and
>>> Windows. The Windows fsutil command shows they're all hard linked:
> [snip]
>>> I'm curious to know if there's a specific reason for this implementation
>>> that would make it the choice over symbolic links.
>>
>> For the same reason you are complaining about backups not taking hardlinks
>> into account: to avoid distributing 400MB instead of 3MB.
>>
>> Cygwin backup utilities should be able to deal with these e.g. rsync -H,
>> --hard-links, although it appears xcopy and robocopy may not under Windows
>> 10; don't know about other utilities or Windows 11.
>
> But why not use symbolic links to accomplish the same thing?
If you're wondering what the limitations and complications of symbolic links
on Windows are, I'd recommend reading item 5.8 in the FAQ - "How do symbolic
links work?" (https://cygwin.com/faq.html#faq.api.symlinks) Keep in
mind that this applies to symbolic links in the Cygwin world. As long as
that's all you care about, then the various trade-offs and limitations can
certainly be mitigated in your personal environment to the extent that you
might actually prefer and choose to use Cygwin symbolic links over hard
links for your needs. But in the general case, they really don't compare to
simplicity of hard links which are fully supported in Windows on NTFS file
systems along with all Cygwin tools and transparently degrade to duplicated
files on file systems and with software that doesn't support them. In these
degraded use cases, they take up more space because the link semantics
aren't maintained but they are still 100% valid and useful. The same cannot
be said of symbolic links.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Most git executables are hard links to git.exe?
2023-07-21 21:54 ` Jim Garrison
2023-07-22 5:55 ` Larry
@ 2023-07-22 17:33 ` Adam Dinwoodie
2023-07-22 17:35 ` Jim Garrison
2023-07-31 13:36 ` Andrey Repin
2 siblings, 1 reply; 8+ messages in thread
From: Adam Dinwoodie @ 2023-07-22 17:33 UTC (permalink / raw)
To: jhg; +Cc: cygwin, Jim Garrison
On Fri, 21 Jul 2023 at 22:54, Jim Garrison via Cygwin wrote:
>
> On 07/21/23 14:52, Brian Inglis wrote:
> > On 2023-07-21 14:59, Jim Garrison via Cygwin wrote:
> >> Git comes with over 100 executables, mostly in /usr/libexec/git-core,
> >> that all appear to be *hard* links to /bin/git, in both Cygwin and
> >> Windows. The Windows fsutil command shows they're all hard linked:
> [snip]
> >> I'm curious to know if there's a specific reason for this implementation
> >> that would make it the choice over symbolic links.
> >
> > For the same reason you are complaining about backups not taking
> > hardlinks into account: to avoid distributing 400MB instead of 3MB.
> >
> > Cygwin backup utilities should be able to deal with these e.g. rsync -H,
> > --hard-links, although it appears xcopy and robocopy may not under
> > Windows 10; don't know about other utilities or Windows 11.
>
> But why not use symbolic links to accomplish the same thing?
A few reasons off the top of my head:
- This is what the Git build tooling does out of the box. Minimising
the number of changes we're making as a downstream packager makes my
life easier as package maintainer.
- This is what happens on *nix systems, and Cygwin generally
prioritises matching function with *nix systems over interoperability
with Windows tools; if you want interoperability with Windows tools,
you might be better off with Git for Windows. That's not trying to
brush you off; the reason Cygwin Git and Git for Windows both exist is
that they're both serving different user needs.
- As others have said, Windows in general has good support for
hardlinks, while it has no inherent support for Cygwin's symlinks.
That means a Windows application would need to be aware of Cygwin to
have any chance of usefully interacting with those files if they were
symlinks, whereas a Windows application doesn't need to be aware of
Cygwin at all to be able to handle hardlinks, it only needs to know
how to handle hardlinks on Windows.
- Although I've not measured it, I expect there's a small runtime cost
from using symlinks over hardlinks. Cygwin's Git is already slow, for
a variety of difficult-to-solve reasons, and I'm reluctant to do
anything that might make that worse.
- Inertia. The current situation works well for most people, and
changing things takes effort and risks breaking other folks' use
cases.
I do acknowledge that while many Windows tools *could* handle
hardlinks, many don't. I'm not at all surprised that some backup
utilities don't handle them well and back up each file separately. I
think switching to using symlinks for Cygwin's executables is the
wrong solution, though.
Instead, I'd suggest (a) finding a backup tool that can handle
hardlinks, (b) finding a backup tool that uses compression so the
"duplicate" data gets deduplicated as part of the backup process, (c)
not backing up most of Cygwin's /usr directory in the first place – in
most cases I wouldn't expect there to be anything in that folder that
couldn't be readily recovered elsewhere anyway – or (d) switching to a
disk imaging backup system rather than a file-based one if it's really
important that you have everything on disk ready to restore.
Hopefully that's all useful and/or interesting, even if it's not the
answer you were hoping for!
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Most git executables are hard links to git.exe?
2023-07-22 17:33 ` Adam Dinwoodie
@ 2023-07-22 17:35 ` Jim Garrison
2023-07-30 0:32 ` L A Walsh
0 siblings, 1 reply; 8+ messages in thread
From: Jim Garrison @ 2023-07-22 17:35 UTC (permalink / raw)
To: cygwin
On 07/22/23 10:33, Adam Dinwoodie wrote:
> On Fri, 21 Jul 2023 at 22:54, Jim Garrison via Cygwin wrote:
>>
>> On 07/21/23 14:52, Brian Inglis wrote:
>>> On 2023-07-21 14:59, Jim Garrison via Cygwin wrote:
>>>> Git comes with over 100 executables, mostly in /usr/libexec/git-core,
>>>> that all appear to be *hard* links to /bin/git, in both Cygwin and
>>>> Windows. The Windows fsutil command shows they're all hard linked:
>> [snip]
>>>> I'm curious to know if there's a specific reason for this implementation
>>>> that would make it the choice over symbolic links.
>>>
>>> For the same reason you are complaining about backups not taking
>>> hardlinks into account: to avoid distributing 400MB instead of 3MB.
>>>
>>> Cygwin backup utilities should be able to deal with these e.g. rsync -H,
>>> --hard-links, although it appears xcopy and robocopy may not under
>>> Windows 10; don't know about other utilities or Windows 11.
>>
>> But why not use symbolic links to accomplish the same thing?
>
> A few reasons off the top of my head:
>
> - This is what the Git build tooling does out of the box. Minimising
> the number of changes we're making as a downstream packager makes my
> life easier as package maintainer.
>
[snip]
> Hopefully that's all useful and/or interesting, even if it's not the
> answer you were hoping for!
Yes, that's very useful and now I understand and appreciate all the
good reasons for hard links. Thanks!
--
Jim Garrison
jhg@acm.org
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Most git executables are hard links to git.exe?
2023-07-22 17:35 ` Jim Garrison
@ 2023-07-30 0:32 ` L A Walsh
0 siblings, 0 replies; 8+ messages in thread
From: L A Walsh @ 2023-07-30 0:32 UTC (permalink / raw)
To: jhg; +Cc: cygwin, Jim Garrison
On 2023/07/22 10:35, Jim Garrison via Cygwin wrote:
> On 07/22/23 10:33, Adam Dinwoodie wrote:
>
>> On Fri, 21 Jul 2023 at 22:54, Jim Garrison via Cygwin wrote:
>>
>>> On 07/21/23 14:52, Brian Inglis wrote:
>>>
>>>> On 2023-07-21 14:59, Jim Garrison via Cygwin wrote:
>>>>
>>>>> Git comes with over 100 executables, mostly in /usr/libexec/git-core,
>>>>> that all appear to be *hard* links to /bin/git, in both Cygwin and
>>>>> Windows. The Windows fsutil command shows they're all hard linked:
>>>>>
>>> [snip]
>>>
>>>>> I'm curious to know if there's a specific reason for this implementation
>>>>> that would make it the choice over symbolic links.
>>>>>
The hardlink implementation on windows is very similar to the
implementation on linux. I'm pretty sure that utils that want to save
on space will look at the inode-number and notice that the hardlinked files
all have the same inode-number (windows has a similar concept though it is
called something else).
On linux, utils that are ignorant of inode numbers, will see hardlinked
files
as separate files -- just as windows does.
The symlink files will break if their targets move (same on lin+win).
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Most git executables are hard links to git.exe?
2023-07-21 21:54 ` Jim Garrison
2023-07-22 5:55 ` Larry
2023-07-22 17:33 ` Adam Dinwoodie
@ 2023-07-31 13:36 ` Andrey Repin
2 siblings, 0 replies; 8+ messages in thread
From: Andrey Repin @ 2023-07-31 13:36 UTC (permalink / raw)
To: Jim Garrison via Cygwin, cygwin
Greetings, Jim Garrison via Cygwin!
> On 07/21/23 14:52, Brian Inglis wrote:
>> On 2023-07-21 14:59, Jim Garrison via Cygwin wrote:
>>> Git comes with over 100 executables, mostly in /usr/libexec/git-core,
>>> that all appear to be *hard* links to /bin/git, in both Cygwin and
>>> Windows. The Windows fsutil command shows they're all hard linked:
> [snip]
>>> I'm curious to know if there's a specific reason for this implementation
>>> that would make it the choice over symbolic links.
>> > For the same reason you are complaining about backups not taking > hardlinks into account: to avoid distributing 400MB instead of 3MB.
>> > Cygwin backup utilities should be able to deal with these e.g. rsync -H, > --hard-links, although it appears xcopy and robocopy may not under > Windows 10; don't know about other utilities or Windows 11.
> But why not use symbolic links to accomplish the same thing?
Adding to the bullet list:
cmd's handling of symbolik links is such that given a symlink path, it would
actually execute the symlink target, thoroughly ruining the executable name
and associated CWD. Making them unusable as command processors in general.
--
With best regards,
Andrey Repin
Monday, July 31, 2023 16:33:49
Sorry for my terrible english...
^ permalink raw reply [flat|nested] 8+ messages in thread