Regression in Cygwin 3.2.0

Regression in Cygwin 3.2.0

[David Allsopp]

I'm unable to build OCaml using the mingw-w64 compilers with Cygwin 3.2.0.
Windows 10.0.19042.928 (and tried on three different machines so far)

Repro:

 - Fresh Cygwin64 installation with make, libiconv, mingw64-x86_64-gcc-core
and git added; fire up mintty
 - git clone --depth 1 --recursive https://github.com/ocaml/ocaml
 - cd ocaml && ./configure --host=x86_64-w64-mingw32 && make -j8 flexdll &&
make -j8 world.opt

This is consistently hanging with Cygwin 3.2.0 and seems possibly to have
something to do with linking (the creation of runtime/ocamlrun.exe and
stdlib/camlheader, which is also an executable, seem to be part of the
problem).

It can be immediately fixed by downgrading to Cygwin 3.1.7, cleaning the
tree and repeating the build.

I'm of course very happy to try to reduce this to a smaller repro case, but
is there an obvious culprit in Cygwin 3.2.0 to check before I do?

Many thanks,


David

Re: Regression in Cygwin 3.2.0

[Thomas Wolff]

Am 16.04.2021 um 12:17 schrieb David Allsopp via Cygwin:
> I'm unable to build OCaml using the mingw-w64 compilers with Cygwin 3.2.0.
> Windows 10.0.19042.928 (and tried on three different machines so far)
>
> Repro:
>
>   - Fresh Cygwin64 installation with make, libiconv, mingw64-x86_64-gcc-core
> and git added; fire up mintty
>   - git clone --depth 1 --recursive https://github.com/ocaml/ocaml
>   - cd ocaml && ./configure --host=x86_64-w64-mingw32 && make -j8 flexdll &&
> make -j8 world.opt
>
> This is consistently hanging with Cygwin 3.2.0 and seems possibly to have
> something to do with linking (the creation of runtime/ocamlrun.exe and
> stdlib/camlheader, which is also an executable, seem to be part of the
> problem).
>
> It can be immediately fixed by downgrading to Cygwin 3.1.7, cleaning the
> tree and repeating the build.
>
> I'm of course very happy to try to reduce this to a smaller repro case, but
> is there an obvious culprit in Cygwin 3.2.0 to check before I do?
The build fails for me with error messages but it does not hang.
You could have tried at least building without -j option, and running 
the build steps separately for better diagnostics.

>
> Many thanks,
>
>
> David
>
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

RE: Regression in Cygwin 3.2.0

[David Allsopp]

Thomas Wolff wrote:
> Am 16.04.2021 um 12:17 schrieb David Allsopp via Cygwin:
> > I'm unable to build OCaml using the mingw-w64 compilers with Cygwin
> > 3.2.0. Windows 10.0.19042.928 (and tried on three different machines
> > so far)
> >
> > Repro:
> >
> >   - Fresh Cygwin64 installation with make, libiconv,
> > mingw64-x86_64-gcc-core and git added; fire up mintty
> >   - git clone --depth 1 --recursive https://github.com/ocaml/ocaml
> >   - cd ocaml && ./configure --host=x86_64-w64-mingw32 && make -j8
> > flexdll && make -j8 world.opt
> >
> > This is consistently hanging with Cygwin 3.2.0 and seems possibly to
> > have something to do with linking (the creation of
> > runtime/ocamlrun.exe and stdlib/camlheader, which is also an
> > executable, seem to be part of the problem).
> >
> > It can be immediately fixed by downgrading to Cygwin 3.1.7, cleaning
> > the tree and repeating the build.
> >
> > I'm of course very happy to try to reduce this to a smaller repro
> > case, but is there an obvious culprit in Cygwin 3.2.0 to check before I
> > do?
> The build fails for me with error messages but it does not hang.

Thanks for confirming!

> You could have tried at least building without -j option, and running the
> build steps separately for better diagnostics.

Indeed - the repro case above was intended to be small and fast. I was still
at the "is this just me/us", stage. Suggesting a non-parallel build of OCaml
would have been mean, as it takes about an hour if it doesn't crash...!

That said, the non-parallel build I think reveals the Cygwin problem: there
are console windows constantly flashing up (it renders the workstation
unusable for the duration of the build). The first stage of the build
compiles ocamlrun.exe which is non-Cygwin in this case and then it uses
that throughout the build, so I guess it's the conpty changes which are biting.

However, the build did succeed when run without -j, however this (which will
take about 20 mins or so):

./configure --build=x86_64-w64-mingw32 --disable-native-compiler
make flexdll runtop

causes a new console Window to pop up with an OCaml prompt (which can be quit
by taking #quit;;) rather than the same prompt being connected to mintty.

With make -j, it doesn't appear to flash up all the console Windows, but it's
also reliably failing. I tried with the cygwin1.dll snapshot from this morning
which seems to be doing the same.


David

Re: Regression in Cygwin 3.2.0

[Thomas Wolff]

Am 16.04.2021 um 16:07 schrieb David Allsopp via Cygwin:
> Thomas Wolff wrote:
>> Am 16.04.2021 um 12:17 schrieb David Allsopp via Cygwin:
>>> I'm unable to build OCaml using the mingw-w64 compilers with Cygwin
>>> 3.2.0. Windows 10.0.19042.928 (and tried on three different machines
>>> so far)
>>>
>>> Repro:
>>>
>>>    - Fresh Cygwin64 installation with make, libiconv,
>>> mingw64-x86_64-gcc-core and git added; fire up mintty
>>>    - git clone --depth 1 --recursive https://github.com/ocaml/ocaml
>>>    - cd ocaml && ./configure --host=x86_64-w64-mingw32 && make -j8
>>> flexdll && make -j8 world.opt
>>>
>>> This is consistently hanging with Cygwin 3.2.0 and seems possibly to
>>> have something to do with linking (the creation of
>>> runtime/ocamlrun.exe and stdlib/camlheader, which is also an
>>> executable, seem to be part of the problem).
>>>
>>> It can be immediately fixed by downgrading to Cygwin 3.1.7, cleaning
>>> the tree and repeating the build.
>>>
>>> I'm of course very happy to try to reduce this to a smaller repro
>>> case, but is there an obvious culprit in Cygwin 3.2.0 to check before I
>>> do?
>> The build fails for me with error messages but it does not hang.
> Thanks for confirming!
>
>> You could have tried at least building without -j option, and running the
>> build steps separately for better diagnostics.
> Indeed - the repro case above was intended to be small and fast. I was still
> at the "is this just me/us", stage. Suggesting a non-parallel build of OCaml
> would have been mean, as it takes about an hour if it doesn't crash...!
>
> That said, the non-parallel build I think reveals the Cygwin problem: there
> are console windows constantly flashing up (it renders the workstation
> unusable for the duration of the build). The first stage of the build
> compiles ocamlrun.exe which is non-Cygwin in this case and then it uses
> that throughout the build, so I guess it's the conpty changes which are biting.
>
> However, the build did succeed when run without -j, however this (which will
> take about 20 mins or so):
>
> ./configure --build=x86_64-w64-mingw32 --disable-native-compiler
> make flexdll runtop
>
> causes a new console Window to pop up with an OCaml prompt (which can be quit
> by taking #quit;;) rather than the same prompt being connected to mintty.
>
> With make -j, it doesn't appear to flash up all the console Windows, but it's
> also reliably failing. I tried with the cygwin1.dll snapshot from this morning
> which seems to be doing the same.
Maybe a non-cygwin tool is injected in your build process?
What if you strip your PATH from all non-cygwin directories?

RE: Regression in Cygwin 3.2.0

[David Allsopp]

Thomas Wolff wrote:
> Am 16.04.2021 um 16:07 schrieb David Allsopp via Cygwin:
> > Thomas Wolff wrote:
> >> Am 16.04.2021 um 12:17 schrieb David Allsopp via Cygwin:
> >>> I'm unable to build OCaml using the mingw-w64 compilers with Cygwin
> >>> 3.2.0. Windows 10.0.19042.928 (and tried on three different machines
> >>> so far)
> >>>
> >>> Repro:
> >>>
> >>>    - Fresh Cygwin64 installation with make, libiconv,
> >>> mingw64-x86_64-gcc-core and git added; fire up mintty
> >>>    - git clone --depth 1 --recursive https://github.com/ocaml/ocaml
> >>>    - cd ocaml && ./configure --host=x86_64-w64-mingw32 && make -j8
> >>> flexdll && make -j8 world.opt
> >>>
> >>> This is consistently hanging with Cygwin 3.2.0 and seems possibly to
> >>> have something to do with linking (the creation of
> >>> runtime/ocamlrun.exe and stdlib/camlheader, which is also an
> >>> executable, seem to be part of the problem).
> >>>
> >>> It can be immediately fixed by downgrading to Cygwin 3.1.7, cleaning
> >>> the tree and repeating the build.
> >>>
> >>> I'm of course very happy to try to reduce this to a smaller repro
> >>> case, but is there an obvious culprit in Cygwin 3.2.0 to check
> >>> before I do?
> >> The build fails for me with error messages but it does not hang.
> > Thanks for confirming!
> >
> >> You could have tried at least building without -j option, and running
> >> the build steps separately for better diagnostics.
> > Indeed - the repro case above was intended to be small and fast. I was
> > still at the "is this just me/us", stage. Suggesting a non-parallel
> > build of OCaml would have been mean, as it takes about an hour if it
> > doesn't crash...!
> >
> > That said, the non-parallel build I think reveals the Cygwin problem:
> > there are console windows constantly flashing up (it renders the
> > workstation unusable for the duration of the build). The first stage
> > of the build compiles ocamlrun.exe which is non-Cygwin in this case
> > and then it uses that throughout the build, so I guess it's the conpty
> > changes which are biting.
> >
> > However, the build did succeed when run without -j, however this
> > (which will take about 20 mins or so):
> >
> > ./configure --build=x86_64-w64-mingw32 --disable-native-compiler make
> > flexdll runtop
> >
> > causes a new console Window to pop up with an OCaml prompt (which can
> > be quit by taking #quit;;) rather than the same prompt being connected
> > to mintty.
> >
> > With make -j, it doesn't appear to flash up all the console Windows,
> > but it's also reliably failing. I tried with the cygwin1.dll snapshot
> > from this morning which seems to be doing the same.
> Maybe a non-cygwin tool is injected in your build process?
> What if you strip your PATH from all non-cygwin directories?

There always has been a non-Cygwin tool - it's calling an ocamlrun.exe it us built at the start of the process with x86_64-w64-mingw32. Is that no longer permitted?!

However, I'm not sure how the environment has anything to do with this: everything goes back to working just by downgrading cygwin1.dll to 3.1.7.


David

Re: Regression in Cygwin 3.2.0

[Takashi Yano]

On Fri, 16 Apr 2021 11:17:50 +0100
David Allsopp wrote:
> I'm unable to build OCaml using the mingw-w64 compilers with Cygwin 3.2.0.
> Windows 10.0.19042.928 (and tried on three different machines so far)
> 
> Repro:
> 
>  - Fresh Cygwin64 installation with make, libiconv, mingw64-x86_64-gcc-core
> and git added; fire up mintty
>  - git clone --depth 1 --recursive https://github.com/ocaml/ocaml
>  - cd ocaml && ./configure --host=x86_64-w64-mingw32 && make -j8 flexdll &&
> make -j8 world.opt
> 
> This is consistently hanging with Cygwin 3.2.0 and seems possibly to have
> something to do with linking (the creation of runtime/ocamlrun.exe and
> stdlib/camlheader, which is also an executable, seem to be part of the
> problem).
> 
> It can be immediately fixed by downgrading to Cygwin 3.1.7, cleaning the
> tree and repeating the build.
> 
> I'm of course very happy to try to reduce this to a smaller repro case, but
> is there an obvious culprit in Cygwin 3.2.0 to check before I do?

Thanks for the report. I looked into this problem and found
that the caused is the combination of some race issues for
non-cygwin processes in pty and console code.

I will submit the patches fixing this issue.

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

RE: Regression in Cygwin 3.2.0

[David Allsopp]

Takashi Yano wrote:
> On Fri, 16 Apr 2021 11:17:50 +0100
> David Allsopp wrote:
> > I'm unable to build OCaml using the mingw-w64 compilers with Cygwin
> 3.2.0.
> > Windows 10.0.19042.928 (and tried on three different machines so far)
> >
> > Repro:
> >
> >  - Fresh Cygwin64 installation with make, libiconv,
> > mingw64-x86_64-gcc-core and git added; fire up mintty
> >  - git clone --depth 1 --recursive https://github.com/ocaml/ocaml
> >  - cd ocaml && ./configure --host=x86_64-w64-mingw32 && make -j8
> > flexdll && make -j8 world.opt
> >
> > This is consistently hanging with Cygwin 3.2.0 and seems possibly to
> > have something to do with linking (the creation of
> > runtime/ocamlrun.exe and stdlib/camlheader, which is also an
> > executable, seem to be part of the problem).
> >
> > It can be immediately fixed by downgrading to Cygwin 3.1.7, cleaning
> > the tree and repeating the build.
> >
> > I'm of course very happy to try to reduce this to a smaller repro
> > case, but is there an obvious culprit in Cygwin 3.2.0 to check before I
> do?
> 
> Thanks for the report. I looked into this problem and found that the
> caused is the combination of some race issues for non-cygwin processes in
> pty and console code.
> 
> I will submit the patches fixing this issue.

Fantastic, thanks - I'll keep an eye on cygwin-patches and test.


David