Future setup regression caused by 'mkdir: always check-for-existence' commit

Future setup regression caused by 'mkdir: always check-for-existence' commit

[Stephen Provine via cygwin]

After this change (commit b0c033bf3fae810b9e5a5c69f17bd4de63725691), the Git for Windows setup (and future Cygwin setups) do not correctly configure bash features because the post-install step for configuring the /dev directory does not work any more. It used to be that "mkdir -m 755 /dev" would succeed, but now it returns a "File exists" error, after which attempts to create the 'shm' and 'mqueue' directories fail and the /dev/fd, /dev/std{in,out,err} links are not created. This causes some bash features to not work. The fix (validated on Git for Windows) would be for setups to pre-create this directory outside of the Cygwin environment before running the post-install steps.

See https://github.com/git-for-windows/git/issues/2291#issuecomment-524433693 for the in-depth analysis. Note, this is not a current issue in Cygwin, but is believed to become a FUTURE issue with the next release.

Thanks,
Stephen


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Future setup regression caused by 'mkdir: always check-for-existence' commit

[Eric Blake]

[-- Attachment #1.1: Type: text/plain, Size: 741 bytes --]

On 8/26/19 10:25 AM, Stephen Provine via cygwin wrote:
> After this change (commit b0c033bf3fae810b9e5a5c69f17bd4de63725691), the Git for Windows setup (and future Cygwin setups) do not correctly configure bash features because the post-install step for configuring the /dev directory does not work any more. It used to be that "mkdir -m 755 /dev" would succeed,

What's wrong with 'mkdir -p -m 755 /dev'?  And if so, which package
needs to be patched to use that? It was unclear from your message if
this is git's postinstall or bash's postinstall or some other script
that is using mkdir without -p.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Future setup regression caused by 'mkdir: always check-for-existence' commit

[Ken Brown]

On 8/26/2019 11:25 AM, Stephen Provine via cygwin wrote:
> After this change (commit b0c033bf3fae810b9e5a5c69f17bd4de63725691), the Git for Windows setup (and future Cygwin setups) do not correctly configure bash features because the post-install step for configuring the /dev directory does not work any more. It used to be that "mkdir -m 755 /dev" would succeed, but now it returns a "File exists" error, after which attempts to create the 'shm' and 'mqueue' directories fail and the /dev/fd, /dev/std{in,out,err} links are not created. This causes some bash features to not work. The fix (validated on Git for Windows) would be for setups to pre-create this directory outside of the Cygwin environment before running the post-install steps.
> 
> See https://github.com/git-for-windows/git/issues/2291#issuecomment-524433693 for the in-depth analysis. Note, this is not a current issue in Cygwin, but is believed to become a FUTURE issue with the next release.

It looks like you've bumped into the bug reported here:

   https://cygwin.com/ml/cygwin/2019-07/msg00152.html

This was a bug in a development snapshot, and it has already been fixed.  You 
should try the test release for cygwin-3.1.0 to confirm this.

I don't think this problem has anything to do with commit 
b0c033bf3fae810b9e5a5c69f17bd4de63725691.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Future setup regression caused by 'mkdir: always check-for-existence' commit

[Ken Brown]

[Please don't top-post on this list.  Thanks.]

On 8/26/2019 1:33 PM, Stephen Provine via cygwin wrote:
> It's possible that this issue contributes to the problem, but the fact that you can manually create the "dev" directory outside of Cygwin (just mkdir from Windows command prompt) to fix the problem tells me otherwise. If I understand the logic change in commit b0c033bf3fae810b9e5a5c69f17bd4de63725691, it would indicate to me that "mkdir 0755 /dev" in a post-install bash script (regardless of which one) would no longer work when it previously worked.

I don't think you understand the logic change in that commit.  An attempt to 
create an existing directory would already fail before that commit; the failure 
would occur in the call to fh->mkdir().  The commit in question simply 
guarantees that the error code is EEXIST.  Take a look at the commit message.

Also, what postinstall script are you referring to that calls "mkdir /dev" 
without either checking for existence or using "mkdir -p"?  Cygwin's bash 
postinstall script doesn't do this.

(You must have missed Eric's reply to your original post, in which he asked the 
same question.)

> I'd be happy to try the test release for cygwin-3.1.0, but I don't know how to find it. Can you provide a link?

Just run's Cygwin's setup program, select the cygwin package, and select the 
test release (currently 3.1.0-0.2).

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: Future setup regression caused by 'mkdir: always check-for-existence' commit

[Stephen Provine via cygwin]

It's possible that this issue contributes to the problem, but the fact that you can manually create the "dev" directory outside of Cygwin (just mkdir from Windows command prompt) to fix the problem tells me otherwise. If I understand the logic change in commit b0c033bf3fae810b9e5a5c69f17bd4de63725691, it would indicate to me that "mkdir 0755 /dev" in a post-install bash script (regardless of which one) would no longer work when it previously worked. This ultimately means that the /dev/fd directory does not exist and that files like /dev/fd/63 cannot be created (the underlying error is "Filesystem is read-only", because the "dev" directory doesn't actually exist except in Cygwin). It's quite possible that if /dev/fd/63 can be created that there are some subsequent FIFO issues, but in this case the file can't even be created in the first place.

I'd be happy to try the test release for cygwin-3.1.0, but I don't know how to find it. Can you provide a link?

Thanks,
Stephen

-----Original Message-----
From: Ken Brown <kbrown@cornell.edu> 
Sent: Monday, August 26, 2019 10:04 AM
To: cygwin@cygwin.com
Cc: Stephen Provine <stephpr@microsoft.com>
Subject: Re: Future setup regression caused by 'mkdir: always check-for-existence' commit

On 8/26/2019 11:25 AM, Stephen Provine via cygwin wrote:
> After this change (commit b0c033bf3fae810b9e5a5c69f17bd4de63725691), the Git for Windows setup (and future Cygwin setups) do not correctly configure bash features because the post-install step for configuring the /dev directory does not work any more. It used to be that "mkdir -m 755 /dev" would succeed, but now it returns a "File exists" error, after which attempts to create the 'shm' and 'mqueue' directories fail and the /dev/fd, /dev/std{in,out,err} links are not created. This causes some bash features to not work. The fix (validated on Git for Windows) would be for setups to pre-create this directory outside of the Cygwin environment before running the post-install steps.
> 
> See https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgit-for-windows%2Fgit%2Fissues%2F2291%23issuecomment-524433693&amp;data=02%7C01%7Cstephpr%40microsoft.com%7C227fabc69f4e434ae12a08d72a4763d2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637024358495313120&amp;sdata=16rboEMc6kGEAzIWDGTjjg%2FVLeuguIbBh%2F9cJMl0oZI%3D&amp;reserved=0 for the in-depth analysis. Note, this is not a current issue in Cygwin, but is believed to become a FUTURE issue with the next release.

It looks like you've bumped into the bug reported here:

   https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcygwin.com%2Fml%2Fcygwin%2F2019-07%2Fmsg00152.html&amp;data=02%7C01%7Cstephpr%40microsoft.com%7C227fabc69f4e434ae12a08d72a4763d2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637024358495313120&amp;sdata=Ua%2BocDgveO3LqqSP0SnzemAihSysyV3jvae5WMxVA1M%3D&amp;reserved=0

This was a bug in a development snapshot, and it has already been fixed.  You should try the test release for cygwin-3.1.0 to confirm this.

I don't think this problem has anything to do with commit b0c033bf3fae810b9e5a5c69f17bd4de63725691.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Future setup regression caused by 'mkdir: always check-for-existence' commit

[Achim Gratz]

Stephen Provine via cygwin writes:
> After this change (commit b0c033bf3fae810b9e5a5c69f17bd4de63725691),
> the Git for Windows setup (and future Cygwin setups) do not correctly
> configure bash features because the post-install step for configuring
> the /dev directory does not work any more.

Cygwin does not create these directories in postinstall nor does any
package provide them, setup.exe does.  So you can assume the existence
of these directories in Cygwin.

https://cygwin.com/cygwin-ug-net/using-specialnames.html

The /dev/std* links are created by bash postinstall and doing that still
works (and the link shows up in the backing filesystem, not just the
virtual one).

> It used to be that "mkdir -m 755 /dev" would succeed, but now it
> returns a "File exists" error, after which attempts to create the
> 'shm' and 'mqueue' directories fail and the /dev/fd,
> /dev/std{in,out,err} links are not created. This causes some bash
> features to not work. The fix (validated on Git for Windows) would be
> for setups to pre-create this directory outside of the Cygwin
> environment before running the post-install steps.

It seems mkdir gets spooked by the overlay of the virtual /dev over the
real one.  That's clearly a bug that needs fixing, IMHO.
You can work around by using a different path to the same directory (via
/proc/cygdrive, say).

> See
> https://github.com/git-for-windows/git/issues/2291#issuecomment-524433693
> for the in-depth analysis. Note, this is not a current issue in
> Cygwin, but is believed to become a FUTURE issue with the next
> release.

You can test that assumption by doing a fresh install of the current test version of Cygwin
3.1.0, but based on what I see on a system that has been updated to this
version I'd think Cygwin would not trigger this bug.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: Future setup regression caused by 'mkdir: always check-for-existence' commit

[Stephen Provine via cygwin]

On 8/26/2019 1:33 PM, Ken Brown wrote:
> I don't think you understand the logic change in that commit.  An attempt to create an existing directory would already fail before that commit; the failure would occur in the call to fh->mkdir().  The commit in question simply guarantees that the error code is EEXIST.  Take a look at the commit message.

I installed the current release of Cygwin with all the default packages (so cygwin package version 3.0.7-1) and tested the scenario that Git for Windows runs into, which is that prior to running its post-install script (https://github.com/git-for-windows/git-sdk-64/blob/master/etc/post-install/01-devices.post), there is no "dev" directory present on the host Windows machine (I just renamed it to "dev2" before running Cygwin.bat). Running "mkdir -m 0755 /dev" in Cygwin *succeeds* with exit code 0. I then deleted that installation and installed Cygwin again, only changing the cygwin package version to 3.1.0-0.2 (test). Tried the same scenario again and "mkdir -m 0755 /dev" fails with the error " mkdir: cannot create directory '/dev': File exists".

I should also point out that this change in behavior starts not in 3.1.0 but in 3.0.7-3, and in-between 3.0.7-1 and 3.0.7-3 the commit I mention seemed like the most reasonable reason for it.

All this said, Cygwin succeeds in both cases. And now that I look, I think the Cygwin setup program explicitly creates "/dev" as a standard directory so it avoids this change in behavior.

Given that the new behavior is obviously correct, I assume this means there is nothing for Cygwin to do, and the fix is for Git for Windows to explicitly create this directory.

Thanks,
Stephen

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Future setup regression caused by 'mkdir: always check-for-existence' commit

[Achim Gratz]

Stephen Provine via cygwin writes:
> I should also point out that this change in behavior starts not in
> 3.1.0 but in 3.0.7-3, and in-between 3.0.7-1 and 3.0.7-3 the commit I
> mention seemed like the most reasonable reason for it.

You should also point out that you are talking not about Cygwin, but the
bundled w/ GfW Msys2 here.  There is no 3.0.7-3 Cygwin.

> All this said, Cygwin succeeds in both cases. And now that I look, I
> think the Cygwin setup program explicitly creates "/dev" as a standard
> directory so it avoids this change in behavior.

Yes, as I already wrote yesterday.

> Given that the new behavior is obviously correct, I assume this means
> there is nothing for Cygwin to do, and the fix is for Git for Windows
> to explicitly create this directory.

Preferrably, yes.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple