Bug Report: Regression in Cygwin 2.11.0-1

Bug Report: Regression in Cygwin 2.11.0-1

[Bryan Phelps]

Hello,


Thank you for all the work on Cygwin! I've been using it to spin up an environment to build the OCaml compiler / toolchain, and it was working great.


However, today, all our CI builds mysteriously started failing - at first, I suspected it was a problem with AppVeyor, but I also failures with VSTS. We use an NPM package (`esy-bash`) to spin up a Cygwin environment, and then use that to build the OCaml toolchain.


The error message we started receiving today is:

OCAML_FLEXLINK="../boot/ocamlrun ../flexdll/flexlink.exe" ../byterun/ocamlrun ../ocamlc -g -nostdlib -I ../utils -I ../parsing -I ../stdlib -I ../compilerlibs -strict-sequence -safe-string -strict-formats -w +a-4-9-41-42-44-45-48 -warn-error A -custom ocamlcommon.cma -o ocamltest.exe run_win32.o run_stubs.o ocamltest_config.cmo testlib.cmo run_command.cmo filetype.cmo filecompare.cmo backends.cmo variables.cmo environments.cmo builtin_variables.cmo builtin_modifiers.cmo actions.cmo builtin_actions.cmo tests.cmo builtin_tests.cmo tsl_ast.cmo tsl_parser.cmo tsl_lexer.cmo tsl_semantics.cmo options.cmo main.cmo
x86_64-w64-mingw32-gcc: error: ../stdlib\libcamlrun.a: No such file or directory


The complete build log is here:

https://gist.github.com/bryphe/58603ab752ecd988f78ee383fa9c9e78


After some investigation, I was able to isolate the issue - environments with the 2.10.0-1 version of Cygwin built successfully, whereas newer environments with the 2.11.0-1 version exhibit this failure. The remaining packages are in parity - importantly, all the mingw packages we install are at the same version.


Results of cygcheck -c on both environments:

- Working environment: https://gist.github.com/bryphe/b11f72f60d9d7c04a8e709e6a8eb20ff

- Failing environment: https://gist.github.com/bryphe/7cf4e6216030781eb273ac45d36590cd


I'll continue to look around for a more minimal repro, but I suspect there may be other manifestations of it. It looks like there were a few path-related patches that came in to 2.11.0-1, so perhaps this scenario was impacted.


In the meantime - is there a way we can pin the cygwin package to the 2.10.0-1 version, to unblock our builds?


Thank you!

Bryan



Sent from Outlook<http://aka.ms/weboutlook>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Andreas Hauptmann]

On Sat, 1 Sep 2018 01:24:49 +0000
Bryan Phelps wrote:

> I'll continue to look around for a more minimal repro,

The normalization of paths with backslashes has changed.

The following doesn't work any longer:

    cd /tmp
    stat "..\bin\file.exe" # or
    stat "..\\bin\\file.exe"

This however still works:
    
    stat "C:\cygwin\bin\file.exe"



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Marco Atzeri]

Am 01.09.2018 um 03:24 schrieb Bryan Phelps:
> Hello,

> In the meantime - is there a way we can pin the cygwin package to the 2.10.0-1 version, to unblock our builds?
>
>
> Thank you!
>
> Bryan

What is the problem in installing the previous version of the package ?


---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
https://www.avast.com/antivirus


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Marco Atzeri]

Am 01.09.2018 um 03:24 schrieb Bryan Phelps:
> Hello,
>
>
> Thank you for all the work on Cygwin! I've been using it to spin up an environment to build the OCaml compiler / toolchain, and it was working great.
>
>
> However, today, all our CI builds mysteriously started failing - at first, I suspected it was a problem with AppVeyor, but I also failures with VSTS. We use an NPM package (`esy-bash`) to spin up a Cygwin environment, and then use that to build the OCaml toolchain.
>
>
> The error message we started receiving today is:
>
> OCAML_FLEXLINK="../boot/ocamlrun ../flexdll/flexlink.exe" ../byterun/ocamlrun ../ocamlc -g -nostdlib -I ../utils -I ../parsing -I ../stdlib -I ../compilerlibs -strict-sequence -safe-string -strict-formats -w +a-4-9-41-42-44-45-48 -warn-error A -custom ocamlcommon.cma -o ocamltest.exe run_win32.o run_stubs.o ocamltest_config.cmo testlib.cmo run_command.cmo filetype.cmo filecompare.cmo backends.cmo variables.cmo environments.cmo builtin_variables.cmo builtin_modifiers.cmo actions.cmo builtin_actions.cmo tests.cmo builtin_tests.cmo tsl_ast.cmo tsl_parser.cmo tsl_lexer.cmo tsl_semantics.cmo options.cmo main.cmo
> x86_64-w64-mingw32-gcc: error: ../stdlib\libcamlrun.a: No such file or directory
>

Question:
is libcamlrun.a built correctly and in the same directory than before ?
This mixed "../stdlib\libcamlrun.a" slash looks strange



---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
https://www.avast.com/antivirus


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: Bug Report: Regression in Cygwin 2.11.0-1

[David Allsopp]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2458 bytes --]

Marco Atzeri wrote:
> Am 01.09.2018 um 03:24 schrieb Bryan Phelps:
> > Hello,
> >
> >
> > Thank you for all the work on Cygwin! I've been using it to spin up an
> environment to build the OCaml compiler / toolchain, and it was working great.
> >
> >
> > However, today, all our CI builds mysteriously started failing - at first, I
> suspected it was a problem with AppVeyor, but I also failures with VSTS. We
> use an NPM package (`esy-bash`) to spin up a Cygwin environment, and then
> use that to build the OCaml toolchain.
> >
> >
> > The error message we started receiving today is:
> >
> > OCAML_FLEXLINK="../boot/ocamlrun ../flexdll/flexlink.exe"
> ../byterun/ocamlrun ../ocamlc -g -nostdlib -I ../utils -I ../parsing -I ../stdlib -I
> ../compilerlibs -strict-sequence -safe-string -strict-formats -w +a-4-9-41-42-44-
> 45-48 -warn-error A -custom ocamlcommon.cma -o ocamltest.exe run_win32.o
> run_stubs.o ocamltest_config.cmo testlib.cmo run_command.cmo filetype.cmo
> filecompare.cmo backends.cmo variables.cmo environments.cmo
> builtin_variables.cmo builtin_modifiers.cmo actions.cmo builtin_actions.cmo
> tests.cmo builtin_tests.cmo tsl_ast.cmo tsl_parser.cmo tsl_lexer.cmo
> tsl_semantics.cmo options.cmo main.cmo
> > x86_64-w64-mingw32-gcc: error: ../stdlib\libcamlrun.a: No such file or
> directory
> >
> 
> Question:
> is libcamlrun.a built correctly and in the same directory than before ?

I haven't double-checked, but the build cannot have got this far if the file were really missing.

> This mixed "../stdlib\libcamlrun.a" slash looks strange

Yeah, our build system is not terribly hygienic when it comes to mixing forward and backslashes in the Windows builds (the Windows shell of course is mostly forgiving of the mixture).

It's not immediately clear to me from the announcement whether this change in behaviour is a regression or an intentional change? Only asking in terms of how quickly we need to make OCaml's build system cleaner.

It's worth noting in this instance that x86_64-w64-mingw32-gcc would be being invoked by a non-Cygwin executable here (i.e. which IIRC is slightly different path normalisation than when it's invoked from bash).


David

\0ТÒÐÐ¥\a&ö&ÆVÒ\a&W\x06÷'G3¢\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒ÷\a&ö&ÆV×2æ‡FÖÀФd\x15\x13¢\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöf\x17\x12ðФFö7VÖVçF\x17F–öã¢\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöFö72æ‡FÖÀÐ¥Vç7V'67&–&R\x06–æfó¢\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöÖÂò7Vç7V'67&–&R×6–×\x06ÆPРÐ

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Thomas Wolff]

Am 01.09.2018 um 10:10 schrieb Marco Atzeri:
> Am 01.09.2018 um 03:24 schrieb Bryan Phelps:
>> Hello,
>>
>>
>> Thank you for all the work on Cygwin! I've been using it to spin up 
>> an environment to build the OCaml compiler / toolchain, and it was 
>> working great.
>>
>>
>> However, today, all our CI builds mysteriously started failing - at 
>> first, I suspected it was a problem with AppVeyor, but I also 
>> failures with VSTS. We use an NPM package (`esy-bash`) to spin up a 
>> Cygwin environment, and then use that to build the OCaml toolchain.
>>
>>
>> The error message we started receiving today is:
>>
>> OCAML_FLEXLINK="../boot/ocamlrun ../flexdll/flexlink.exe" 
>> ../byterun/ocamlrun ../ocamlc -g -nostdlib -I ../utils -I ../parsing 
>> -I ../stdlib -I ../compilerlibs -strict-sequence -safe-string 
>> -strict-formats -w +a-4-9-41-42-44-45-48 -warn-error A -custom 
>> ocamlcommon.cma -o ocamltest.exe run_win32.o run_stubs.o 
>> ocamltest_config.cmo testlib.cmo run_command.cmo filetype.cmo 
>> filecompare.cmo backends.cmo variables.cmo environments.cmo 
>> builtin_variables.cmo builtin_modifiers.cmo actions.cmo 
>> builtin_actions.cmo tests.cmo builtin_tests.cmo tsl_ast.cmo 
>> tsl_parser.cmo tsl_lexer.cmo tsl_semantics.cmo options.cmo main.cmo
>> x86_64-w64-mingw32-gcc: error: ../stdlib\libcamlrun.a: No such file 
>> or directory
>>
>
> Question:
> is libcamlrun.a built correctly and in the same directory than before ?
> This mixed "../stdlib\libcamlrun.a" slash looks strange
It does, but remember, we're running on a mixed system. Also the problem 
is not caused by mixed syntax, the following do not work anymore:
     cd /usr; ls '..\bin'
     cd /; ls 'usr\include'
I think it's good that cygwin used to support this and it should be 
reestablished.
Note the following did not even work before, due to mount point issues:
     cd /; ls 'usr\bin'
Not sure whether it's worth fixing that seeming inconsistence as well.

Checking the change log, it might be a side effect of

- Fix a buffer underrun problem in Win32 path normalization.
   Addresses:https://cygwin.com/ml/cygwin/2018-05//msg00017.html


Checking the commit diff
https://cygwin.com/git/gitweb.cgi?p=newlib-cygwin.git;a=commitdiff;h=35998fc2fa6cbb7d761f6d88346246bd3627552b
this might be a starting point to analyse...

---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
https://www.avast.com/antivirus


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1310 bytes --]

On Sep  1 09:56, Andreas Hauptmann wrote:
> On Sat, 1 Sep 2018 01:24:49 +0000
> Bryan Phelps wrote:
> 
> > I'll continue to look around for a more minimal repro,
> 
> The normalization of paths with backslashes has changed.
> 
> The following doesn't work any longer:
> 
>     cd /tmp
>     stat "..\bin\file.exe" # or
>     stat "..\\bin\\file.exe"
> 
> This however still works:
>     
>     stat "C:\cygwin\bin\file.exe"

I know where the problem is.  A new piece of code fixing a crash due to
a buffer underflow is too aggressively guarding the path buffer against
this underflow when normalizing ".." in Win32 paths.

I'll fix this and release a 2.11.1 soon, but I still have a question:

Why do I push out test releases if nobody cares?

In terms of this OCAML build system problem:

Please fix your build system.  Do not mix POSIX and Win32 paths, use
POSIX paths only.  Backslashes are *no* drop-in replacement for slashes.
Some of the path handling is seriously crippled as soon as you start
using backslashes, and that's a delipberate decision and won't change,
even after fixing the aforementioned bug.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

RE: Bug Report: Regression in Cygwin 2.11.0-1

[David Allsopp]

Corinna Vinschen wrote:
> On Sep  1 09:56, Andreas Hauptmann wrote:
> > On Sat, 1 Sep 2018 01:24:49 +0000
> > Bryan Phelps wrote:
> >
> > > I'll continue to look around for a more minimal repro,
> >
> > The normalization of paths with backslashes has changed.
> >
> > The following doesn't work any longer:
> >
> >     cd /tmp
> >     stat "..\bin\file.exe" # or
> >     stat "..\\bin\\file.exe"
> >
> > This however still works:
> >
> >     stat "C:\cygwin\bin\file.exe"
> 
> I know where the problem is.  A new piece of code fixing a crash due to a buffer
> underflow is too aggressively guarding the path buffer against this underflow
> when normalizing ".." in Win32 paths.
> 
> I'll fix this and release a 2.11.1 soon, but I still have a question:
> 
> Why do I push out test releases if nobody cares?

Like you, we're a (very) small number of maintainers - and, as far as I'm aware, the command line setup doesn't exactly make continuous integration testing of Cygwin test versions "trivial". I'd love to have had the time within the last 22 days to service my care...

> In terms of this OCAML build system problem:
> 
> Please fix your build system.  Do not mix POSIX and Win32 paths, use POSIX
> paths only.  Backslashes are *no* drop-in replacement for slashes.

The "problem" for us is more subtle than this. The program which is generating that path is not a Cygwin executable, so it is correctly combining a path it has been supplied (the ../stdlib which has been supplied via Cygwin's make) with a filename, so it uses a backslash. It's been on my TODO for years to enhance to understand that the program it's supplying the path back to is a Cygwin executable and so sort it out properly but, well, we're a small number of maintainers! That said, WSL is forcing us to address it properly... 

> Some of the path handling is seriously crippled as soon as you start using
> backslashes, and that's a delipberate decision and won't change, even after
> fixing the aforementioned bug.

I don't quite follow this - does that mean that that path definitely work even with a new Cygwin? i.e. our existing build system is permanently broken for Cygwin 2.11+?


David

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Houder]

On Sat, 1 Sep 2018 17:54:35, Corinna Vinschen wrote:
> I'll fix this and release a 2.11.1 soon, but I still have a question:
> 
> Why do I push out test releases if nobody cares?

Yes, I know, it is a _hypothetical_question_. You do not really expect
an answer.

However it was my thought exactly. Nobody? I like to think that those
that were responsible for patches, have tested their modifications.

I did.

On the other hand, those that will find problems, the people building
packages, will only do that on "official" releases.

I am almost sure of that :-)

Unless you like creating test releases, releasing only one before you
push an "official" one, would be more efficient, I imagine.

Yes, you would run the chance that you would have to mend an official
release several times ... but is that bad?

Just a thought.

Henri


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1133 bytes --]

On Sep  1 16:41, David Allsopp wrote:
> Corinna Vinschen wrote:
> > Some of the path handling is seriously crippled as soon as you start using
> > backslashes, and that's a delipberate decision and won't change, even after
> > fixing the aforementioned bug.
> 
> I don't quite follow this - does that mean that that path definitely
> work even with a new Cygwin? i.e. our existing build system is
> permanently broken for Cygwin 2.11+?

No.  The problem we're talking about here is a regression and needs
fixing, no doubt about it.

I was just saying that win32 paths *in general* get a less complete
treatment than POSIX paths for hopefully obvious reasons.  Just be
aware that \foo\bar is not the same as /foo/bar and may even point
to another file.

As for the bug in question:  I pushed a patch which should fix this
issue.  I created new developer snapshots and uploaded them to
https://cygwin.com/snapshots/.  Please give them a try.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1247 bytes --]

On Sep  1 20:08, Houder wrote:
> On Sat, 1 Sep 2018 17:54:35, Corinna Vinschen wrote:
> > I'll fix this and release a 2.11.1 soon, but I still have a question:
> > 
> > Why do I push out test releases if nobody cares?
> 
> Yes, I know, it is a _hypothetical_question_. You do not really expect
> an answer.
> 
> However it was my thought exactly. Nobody? I like to think that those
> that were responsible for patches, have tested their modifications.
> 
> I did.
> 
> On the other hand, those that will find problems, the people building
> packages, will only do that on "official" releases.
> 
> I am almost sure of that :-)
> 
> Unless you like creating test releases, releasing only one before you
> push an "official" one, would be more efficient, I imagine.
> 
> Yes, you would run the chance that you would have to mend an official
> release several times ... but is that bad?

The idea of test releases is to avoid problem like the one we're discussing
here in official releases.  I'd like to get an official release out and
then go on vacation :)


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Bryan Phelps]

> As for the bug in question:  I pushed a patch which should fix this 
> issue.  I created new developer snapshots and uploaded them to 
> https://cygwin.com/snapshots/.  Please give them a try. 

Thank you Corinna for the quick fix and investigation!

I set up an environment to test it out: https://github.com/bryphe/test-ocaml-build and ran a build against the patched dll.
It looks like the patch fixes our scenario - the build is green with the patched dll: https://ci.appveyor.com/project/bryphe/test-ocaml-build, so the patch fixes our problem.

> On the other hand, those that will find problems, the people building
packages, will only do that on "official" releases.

I'm pretty new to the Cygwin ecosystem - we could certainly create nightly builds that test against cygwin releases, if that helps (and adjust our CLI script to grab the nightly / latest test release of Cygwin).

I wonder if, for a change like this, we could add a test case that exercises it - looking through the source, it looks like cygload.cc has a few test cases around pathing - perhaps it would be helpful to add a regression test for the minimal repro?

Thanks again for the help and investigation!
Bryan
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Houder]

On Sat, 1 Sep 2018 21:23:56, Corinna Vinschen wrote:
> > Yes, you would run the chance that you would have to mend an official
> > release several times ... but is that bad?
> 
> The idea of test releases is to avoid problem like the one we're discussing
> here in official releases.  I'd like to get an official release out and
> then go on vacation :)

Now you know it does not work that way :-) ... I would say: "Just go!" Happy
holidays.

Henri


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Andrey Repin]

Greetings, David Allsopp!

>> In terms of this OCAML build system problem:
>> 
>> Please fix your build system.  Do not mix POSIX and Win32 paths, use POSIX
>> paths only.  Backslashes are *no* drop-in replacement for slashes.

> The "problem" for us is more subtle than this. The program which is
> generating that path is not a Cygwin executable, so it is correctly
> combining a path it has been supplied (the ../stdlib which has been supplied
> via Cygwin's make) with a filename, so it uses a backslash. It's been on my
> TODO for years to enhance to understand that the program it's supplying the
> path back to is a Cygwin executable and so sort it out properly but, well,
> we're a small number of maintainers! That said, WSL is forcing us to address it properly...

You CAN just send back forward slashes. Windows don't care.
If any program would choke, it would be that program's problem after all.


-- 
With best regards,
Andrey Repin
Sunday, September 2, 2018 0:51:18

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug Report: Regression in Cygwin 2.11.0-1

[cyg Simple]

On 9/1/2018 5:52 PM, Andrey Repin wrote:
> Greetings, David Allsopp!
> 
>>> In terms of this OCAML build system problem:
>>>
>>> Please fix your build system.  Do not mix POSIX and Win32 paths, use POSIX
>>> paths only.  Backslashes are *no* drop-in replacement for slashes.
> 
>> The "problem" for us is more subtle than this. The program which is
>> generating that path is not a Cygwin executable, so it is correctly
>> combining a path it has been supplied (the ../stdlib which has been supplied
>> via Cygwin's make) with a filename, so it uses a backslash. It's been on my
>> TODO for years to enhance to understand that the program it's supplying the
>> path back to is a Cygwin executable and so sort it out properly but, well,
>> we're a small number of maintainers! That said, WSL is forcing us to address it properly...
> 
> You CAN just send back forward slashes. Windows don't care.
> If any program would choke, it would be that program's problem after all.

Certainly not "that program's problem". The problem becomes mine and
yours as we've not followed the design of the program.  While Windows at
the moment doesn't care there is always the possibility that some fix
could break that since the documentation states to use \ in paths and
not /.  So while we "CAN just send back forward slashes" we need to be
prepared for the outcome if something changes in an update to the OS and
breaks the assumptions based on observed behavior rather than the
documented behavior.

-- 
cyg Simple

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: Bug Report: Regression in Cygwin 2.11.0-1

[David Allsopp]

cyg Simple wrote:
> On 9/1/2018 5:52 PM, Andrey Repin wrote:
> > Greetings, David Allsopp!
> >
> >>> In terms of this OCAML build system problem:
> >>>
> >>> Please fix your build system.  Do not mix POSIX and Win32 paths, use
> >>> POSIX paths only.  Backslashes are *no* drop-in replacement for slashes.
> >
> >> The "problem" for us is more subtle than this. The program which is
> >> generating that path is not a Cygwin executable, so it is correctly
> >> combining a path it has been supplied (the ../stdlib which has been
> >> supplied via Cygwin's make) with a filename, so it uses a backslash.
> >> It's been on my TODO for years to enhance to understand that the
> >> program it's supplying the path back to is a Cygwin executable and so
> >> sort it out properly but, well, we're a small number of maintainers! That
> said, WSL is forcing us to address it properly...
> >
> > You CAN just send back forward slashes. Windows don't care.

That's not strictly true, as mkdir foo/bar will demonstrate in a command prompt (it is almost true when dealing with the Windows API). However, IMO, one looks like a Unix programmer who doesn't really understand Windows when your programs visibly use forward slashes everywhere. 

> > If any program would choke, it would be that program's problem after all.
> 
> Certainly not "that program's problem". The problem becomes mine and yours
> as we've not followed the design of the program.  While Windows at the
> moment doesn't care there is always the possibility that some fix could break
> that since the documentation states to use \ in paths and not /.  So while we
> "CAN just send back forward slashes" we need to be prepared for the outcome
> if something changes in an update to the OS and breaks the assumptions based
> on observed behavior rather than the documented behavior.

Just as an aside, the use of slashes is documented: see Kuhne's compulsory reading on the subject at https://blogs.msdn.microsoft.com/jeremykuhne/2016/04/21/path-normalization/ and also the notes in MSDN (https://docs.microsoft.com/en-gb/windows/desktop/FileIO/naming-a-file#maximum-path-length-limitation) which do mention that normalisation of / to \ in the API is not an accident.


David


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: Bug Report: Regression in Cygwin 2.11.0-1

[David Allsopp]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1819 bytes --]

Corinna Vinschen wrote:
> On Sep  1 16:41, David Allsopp wrote:
> > Corinna Vinschen wrote:
> > > Some of the path handling is seriously crippled as soon as you start
> > > using backslashes, and that's a delipberate decision and won't
> > > change, even after fixing the aforementioned bug.
> >
> > I don't quite follow this - does that mean that that path definitely
> > work even with a new Cygwin? i.e. our existing build system is
> > permanently broken for Cygwin 2.11+?
> 
> No.  The problem we're talking about here is a regression and needs fixing, no
> doubt about it.
> 
> I was just saying that win32 paths *in general* get a less complete treatment
> than POSIX paths for hopefully obvious reasons.  Just be aware that \foo\bar is
> not the same as /foo/bar and may even point to another file.
>
> As for the bug in question:  I pushed a patch which should fix this issue.  I
> created new developer snapshots and uploaded them to
> https://cygwin.com/snapshots/.  Please give them a try.

Thanks! That snapshot does indeed fix our build.

How feasible would it be to generate an alternate setup.ini (setup-snapshots.ini or something) and include the snapshots in the actual mirror with a switch to setup to select the alternate file? When we finally get to it with OCaml's CI, that is probably how I would configure our CI to operate (i.e. synchronise with a mirror, download the latest snapshot and then patch setup.ini), but I imagine there might be other projects which would set jobs running if it were very simple from upstream to do. 


David
\0ТÒÐÐ¥\a&ö&ÆVÒ\a&W\x06÷'G3¢\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒ÷\a&ö&ÆV×2æ‡FÖÀФd\x15\x13¢\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöf\x17\x12ðФFö7VÖVçF\x17F–öã¢\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöFö72æ‡FÖÀÐ¥Vç7V'67&–&R\x06–æfó¢\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöÖÂò7Vç7V'67&–&R×6–×\x06ÆPРÐ

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1916 bytes --]

On Sep  2 08:37, David Allsopp wrote:
> Corinna Vinschen wrote:
> > On Sep  1 16:41, David Allsopp wrote:
> > > Corinna Vinschen wrote:
> > > > Some of the path handling is seriously crippled as soon as you start
> > > > using backslashes, and that's a delipberate decision and won't
> > > > change, even after fixing the aforementioned bug.
> > >
> > > I don't quite follow this - does that mean that that path definitely
> > > work even with a new Cygwin? i.e. our existing build system is
> > > permanently broken for Cygwin 2.11+?
> > 
> > No.  The problem we're talking about here is a regression and needs fixing, no
> > doubt about it.
> > 
> > I was just saying that win32 paths *in general* get a less complete treatment
> > than POSIX paths for hopefully obvious reasons.  Just be aware that \foo\bar is
> > not the same as /foo/bar and may even point to another file.
> >
> > As for the bug in question:  I pushed a patch which should fix this issue.  I
> > created new developer snapshots and uploaded them to
> > https://cygwin.com/snapshots/.  Please give them a try.
> 
> Thanks! That snapshot does indeed fix our build.

Thanks for testing.

> How feasible would it be to generate an alternate setup.ini
> (setup-snapshots.ini or something) and include the snapshots in the
> actual mirror with a switch to setup to select the alternate file?
> When we finally get to it with OCaml's CI, that is probably how I
> would configure our CI to operate (i.e. synchronise with a mirror,
> download the latest snapshot and then patch setup.ini), but I imagine
> there might be other projects which would set jobs running if it were
> very simple from upstream to do. 

I'll release 2.11.1 today or tomorrow.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Achim Gratz]

Corinna Vinschen writes:
>> How feasible would it be to generate an alternate setup.ini
>> (setup-snapshots.ini or something) and include the snapshots in the
>> actual mirror with a switch to setup to select the alternate file?
>> When we finally get to it with OCaml's CI, that is probably how I
>> would configure our CI to operate (i.e. synchronise with a mirror,
>> download the latest snapshot and then patch setup.ini), but I imagine
>> there might be other projects which would set jobs running if it were
>> very simple from upstream to do. 
>
> I'll release 2.11.1 today or tomorrow.

I almost gave the same answer when I read that yesterday.  However, I
think David was asking for a way to let his CI pick up a snapshot when
one is available so it would automatically give an indication if there
was any problem with it.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1182 bytes --]

On Sep  3 21:12, Achim Gratz wrote:
> Corinna Vinschen writes:
> >> How feasible would it be to generate an alternate setup.ini
> >> (setup-snapshots.ini or something) and include the snapshots in the
> >> actual mirror with a switch to setup to select the alternate file?
> >> When we finally get to it with OCaml's CI, that is probably how I
> >> would configure our CI to operate (i.e. synchronise with a mirror,
> >> download the latest snapshot and then patch setup.ini), but I imagine
> >> there might be other projects which would set jobs running if it were
> >> very simple from upstream to do. 
> >
> > I'll release 2.11.1 today or tomorrow.
> 
> I almost gave the same answer when I read that yesterday.  However, I
> think David was asking for a way to let his CI pick up a snapshot when
> one is available so it would automatically give an indication if there
> was any problem with it.

The answer is no, but as soon as I have a test release, it's available
via the default setup.ini.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 829 bytes --]