cygwin fork problem maybe?

cygwin fork problem maybe?

[Linda W]

A problem I reported in trying to install "File::BOM" (module to handle files 
with Unicode Byte-Order-Marks) under CPAN originally here:
http://sourceware.org/ml/cygwin/2006-02/msg00238.html last February
is still a problem.  I bounced it off of the module maintainer, and he
believes the problem has to do with "forking", not FIFO use.

His test works under AS Perl, but he doesn't think he'll be able to
get his module to work under cygwin with these problems.

A summary snippet of the last email I got from him is included below.

The symptoms of the hung processes are still the same -- both instances
of perl running the test completely lock-up and are unkillable in the cygwin
environment

Is there any chance of getting someone to look at this problem?

I'm rather depressed at the number of Win32 modules that don't work under
cygwin.  I know this may be considered a "feature" by some, but if I wanted
a POSIXish environment that was incompatible with Win32, I'd run linux, but
I want a *nix environment that lets me access the native system's
features.  If it was only a few Win32 modules that had problems, that'd be
one thing, but I feel overwhelmed by the number of incompatibilities.

Anyway I didn't know fork was such a problem on Windows...

thanks for any help on this...
-linda



-------- Original Message --------
Subject: Re: [perl #39211] cygperl hangs on FIFO use (attempt install of CPAN 
File::BOM hangs during test using FIFO)
Date: Tue, 06 Jun 2006 12:18:09 +0100
From: "CPAN File::BOM module maintainer"

...
I would say it's definitely fork() that's the problem, not fifo support.
...
For what it's worth, the tests succeed on ActivePerl.


>> I know that fork is a bit of a bugbear for the activeperl team. forking
>> on win32 is probably always going to be a problem. *sigh*
>     ... 
Windows just doesn't support forking at all, as far as I know.
activeperl emulates forking using win32 threads. I don'tknow how cygwin
handles it, but my guess is that it's not very well :-(



>> Those exception test results seem to indicate that there's no problem
>> reading from filehandles that were opened for writing, or even reading
>> from non-existent files.
> 
...

The tests that are failing are these:
     open WRITER, '>', $tmpfile or die "Couldn't write to '$tmpfile': $!";

     throws_ok { File::BOM::_get_encoding_seekable(\*WRITER) }
	    qr/^Couldn't read from handle/,
	    "_get_encoding_seekable on unreadable handle fails";

     throws_ok { File::BOM::_get_encoding_unseekable(\*WRITER) }
		qr/^Couldn't read byte/,
		"_get_encoding_unseekable() on unreadable handle fails";


The first one, at least, returns normally, which indicates that
it succeeded in reading from the WRITER handle, which was
opened for writing.

One of the other failing tests tried to
read from a generated (non-existent) tempfile name, and also didn't die.

...

I don't think I'm going to be able to fix the tests to pass on cygwin,
other than testing $^O and skipping all forking and exception tests,
which seems a bit like cheating.

--------



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: cygwin fork problem maybe?

[Brian Dessent]

Linda W wrote:

> Windows just doesn't support forking at all, as far as I know.
> activeperl emulates forking using win32 threads. I don'tknow how cygwin
> handles it, but my guess is that it's not very well :-(

This smells like total FUD.  This person that admittedly does not use
Cygwin nor obviously know anything about Cygwin says that it's a fork
problem and that's that?  Fork may be a problem for Activestate but
Cygwin fully emulates fork just fine, as does Cygwin's perl.  The Cygwin
DLL goes to extreme lengths to make sure this work.  Just about every
Cygwin compiled app that invokes a subprogram uses fork in some form,
for example every shell command you type is a fork/exec.

"my guess is that it's not very well" without any specific indication as
to why or how fork is even related seems like nonsense to me.  I
certainly don't know why this module is failing, but just because
Activestate's implementation of fork is terrible doesn't mean Cygwin's
is too.

Brian

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: cygwin fork problem maybe?

[Yitzchak Scott-Thoennes]

On Tue, Jun 06, 2006 at 01:30:43PM -0700, Linda W wrote:
> A problem I reported in trying to install "File::BOM" (module to handle 
> files with Unicode Byte-Order-Marks) under CPAN originally here:
> http://sourceware.org/ml/cygwin/2006-02/msg00238.html last February
> is still a problem.  I bounced it off of the module maintainer, and he
> believes the problem has to do with "forking", not FIFO use.
> 
> His test works under AS Perl, but he doesn't think he'll be able to
> get his module to work under cygwin with these problems.

Can he or you reduce the problem to a non-File::BOM dependent test script?

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: cygwin fork problem maybe?

[Linda Walsh]

Brian Dessent wrote:
> Linda W wrote:
>
>   
>> Windows just doesn't support forking at all, as far as I know.
>> activeperl emulates forking using win32 threads. I don'tknow how cygwin
>> handles it, but my guess is that it's not very well :-(
>>     
>
> This smells like total FUD.  This person that admittedly does not use
> Cygwin nor obviously know anything about Cygwin says that it's a fork
> problem and that's that? 
=====
    They don't have a windows machine to test with, but they
are trying to help pinpoint a problem that works under linux
and ActiveState, which seems like the most useful response
to date.  I can't give them a Windows machine or a cygwin
install to test with.  I can run the programs and report on
the results, and I can get a copy of the source to you that
fails.  Is there anything else I could do to be helpful?

:-)
Linda


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: cygwin non-posix problems

[Linda Walsh]

Yitzchak Scott-Thoennes wrote:
> Can he or you reduce the problem to a non-File::BOM dependent test script
What part of the perl module File::BOM should I throw out before
it's no longer File::BOM?  It's just perl code.

It's freely downloadable through CPAN, so I can't make it too
much more publicly available than that.

But FWIW, the File::BOM code isn't the actual problem.  It's
the authors test routine that he decided to be "fancy" with,
and use a child process to send strings via a "FIFO" to the
test harness process.

It isn't desirable to modify "cygwin-only-failing" Perl modules
to work around problems than only happen under cygwin.  Certainly
you can see how that is "burying one's head under the sand".  Suppose
various parts of CPAN are rewritten to steer around bugs in Cygwin.
Does that make the underlying problems problems in Cygwin go away?
Does that make cygwin more stable or more compatible with other
Posix platforms? 

In my mind it eliminates test cases that are perfectly uncovering
Cygwin incompatibilities and deficiencies.

Another example is the Win32::API module?  It also
fails under cygwin -- starting about 9 months ago.  Still does.  The
problems in cygwin aren't going away.  And when module developers
get bugs reported under cygwin, they may not bother with them if
cygwin is known to have many Posix compatibility problems.
The module maintainers would like nothing more than for their module
to work w/o problems on all platforms.  Perl goes to great lengths
to ensure "it just works", "out-of-the-box" on scores of platforms.

Also, FWIW, I did report a simpler test case that came up during
their continued attempts to isolate the problem:
([perlbug #39325]: Cygperl allows reading of file descriptors open 
Write-Only)

I don't know if the above bug is somehow the "root" cause of the
problem in File::BOM but I doubt it is solely responsible for the
behaviors I'm seeing, including cygperl hanging and being
unkillable from within cygwin.

Certainly, we can agree, that a process under cygwin should not
normally hang and be unresponsive to cygwin "kill -9" signals?

linda






--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: cygwin non-posix problems

[Yitzchak Scott-Thoennes]

Linda Walsh wrote:
> Yitzchak Scott-Thoennes wrote:
>> Can he or you reduce the problem to a non-File::BOM dependent test
>> script
> What part of the perl module File::BOM should I throw out before
> it's no longer File::BOM?  It's just perl code.
>
> It's freely downloadable through CPAN, so I can't make it too
> much more publicly available than that.

The point would be to reduce the amount of code that might need
to be inspected to find the underlying problem.  Nothing to do
with publicly available.

> But FWIW, the File::BOM code isn't the actual problem.  It's
> the authors test routine that he decided to be "fancy" with,
> and use a child process to send strings via a "FIFO" to the
> test harness process.
>
> It isn't desirable to modify "cygwin-only-failing" Perl modules
> to work around problems than only happen under cygwin.  Certainly
> you can see how that is "burying one's head under the sand".  Suppose
> various parts of CPAN are rewritten to steer around bugs in Cygwin.
> Does that make the underlying problems problems in Cygwin go away?
> Does that make cygwin more stable or more compatible with other
> Posix platforms?
>
> In my mind it eliminates test cases that are perfectly uncovering
> Cygwin incompatibilities and deficiencies.

I agree with all of the above and wasn't trying to suggest modifying
the tests.

> Another example is the Win32::API module?  It also
> fails under cygwin -- starting about 9 months ago.  Still does.  The
> problems in cygwin aren't going away.  And when module developers
> get bugs reported under cygwin, they may not bother with them if
> cygwin is known to have many Posix compatibility problems.

I didn't know posix compatibility problems were at issue there?
And I don't think posix compatibility ranks high in importance
for Win32::API.

> The module maintainers would like nothing more than for their module
> to work w/o problems on all platforms.  Perl goes to great lengths
> to ensure "it just works", "out-of-the-box" on scores of platforms.
>
> Also, FWIW, I did report a simpler test case that came up during
> their continued attempts to isolate the problem:
> ([perlbug #39325]: Cygperl allows reading of file descriptors open
> Write-Only)
>
> I don't know if the above bug is somehow the "root" cause of the
> problem in File::BOM but I doubt it is solely responsible for the
> behaviors I'm seeing, including cygperl hanging and being
> unkillable from within cygwin.
>
> Certainly, we can agree, that a process under cygwin should not
> normally hang and be unresponsive to cygwin "kill -9" signals?

/bin/kill -f worked for me.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: cygwin non-posix problems

[Christopher Faylor]

On Thu, Jun 08, 2006 at 05:11:29PM -0700, Yitzchak Scott-Thoennes wrote:
>Linda Walsh wrote:
>> Yitzchak Scott-Thoennes wrote:
>>> Can he or you reduce the problem to a non-File::BOM dependent test
>>> script
>> What part of the perl module File::BOM should I throw out before
>> it's no longer File::BOM?  It's just perl code.
>>
>> It's freely downloadable through CPAN, so I can't make it too
>> much more publicly available than that.
>
>The point would be to reduce the amount of code that might need
>to be inspected to find the underlying problem.  Nothing to do
>with publicly available.
>
>> But FWIW, the File::BOM code isn't the actual problem.  It's
>> the authors test routine that he decided to be "fancy" with,
>> and use a child process to send strings via a "FIFO" to the
>> test harness process.
>>
>> It isn't desirable to modify "cygwin-only-failing" Perl modules
>> to work around problems than only happen under cygwin.  Certainly
>> you can see how that is "burying one's head under the sand".  Suppose
>> various parts of CPAN are rewritten to steer around bugs in Cygwin.
>> Does that make the underlying problems problems in Cygwin go away?
>> Does that make cygwin more stable or more compatible with other
>> Posix platforms?
>>
>> In my mind it eliminates test cases that are perfectly uncovering
>> Cygwin incompatibilities and deficiencies.
>
>I agree with all of the above and wasn't trying to suggest modifying
>the tests.

Indeed, this is standard operating procedure for debugging problems.

>> Certainly, we can agree, that a process under cygwin should not
>> normally hang and be unresponsive to cygwin "kill -9" signals?
>
>/bin/kill -f worked for me.

That would suggest that File::BOM is using blocking windows calls
directly somehow.  Gee, if only there was some way to narrow this down.

I know! It must be because fork doesn't work on a multi-threaded dual
core processor!

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: cygwin non-posix problems

[Christopher Faylor]

On Thu, Jun 08, 2006 at 10:03:58PM -0400, Christopher Faylor wrote:
>On Thu, Jun 08, 2006 at 05:11:29PM -0700, Yitzchak Scott-Thoennes wrote:
>>Linda Walsh wrote:
>>> Yitzchak Scott-Thoennes wrote:
>>>> Can he or you reduce the problem to a non-File::BOM dependent test
>>>> script
>>> What part of the perl module File::BOM should I throw out before
>>> it's no longer File::BOM?  It's just perl code.
>>>
>>> It's freely downloadable through CPAN, so I can't make it too
>>> much more publicly available than that.
>>
>>The point would be to reduce the amount of code that might need
>>to be inspected to find the underlying problem.  Nothing to do
>>with publicly available.
>>
>>> But FWIW, the File::BOM code isn't the actual problem.  It's
>>> the authors test routine that he decided to be "fancy" with,
>>> and use a child process to send strings via a "FIFO" to the
>>> test harness process.
>>>
>>> It isn't desirable to modify "cygwin-only-failing" Perl modules
>>> to work around problems than only happen under cygwin.  Certainly
>>> you can see how that is "burying one's head under the sand".  Suppose
>>> various parts of CPAN are rewritten to steer around bugs in Cygwin.
>>> Does that make the underlying problems problems in Cygwin go away?
>>> Does that make cygwin more stable or more compatible with other
>>> Posix platforms?
>>>
>>> In my mind it eliminates test cases that are perfectly uncovering
>>> Cygwin incompatibilities and deficiencies.
>>
>>I agree with all of the above and wasn't trying to suggest modifying
>>the tests.
>
>Indeed, this is standard operating procedure for debugging problems.

In case this wasn't clear, I meant that winnowing down a failure to a
minimal amount of code required to reproduce the problem is "SOP".

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: cygwin non-posix problems

[Linda Walsh]

Christopher Faylor wrote:
> On Thu, Jun 08, 2006 at 05:11:29PM -0700, Yitzchak Scott-Thoennes wrote:
>   
>> The point would be to reduce the amount of code that might need
>> to be inspected to find the underlying problem.  Nothing to do
>> with publicly available.
>>     
---
    Would that it was always so "easy", but in this case, single
stepping through the test made the problem go away.  So it isn't
entirely straight forward to narrow it down.

>> /bin/kill -f worked for me.
>>     
    Hmm....SIGEFF?  Haven't heard of that one.  At least you can
reproduce it. >>Thank you.<<  At least I know
this isn't another bug that's due to the "if (user==linda) {!?%#$;}"
pseudo code that seems to haunt me every now and then.  Even
MS's "brightest" (*cough*) support engineers can't figure those out.

>
> That would suggest that File::BOM is using blocking windows calls
> directly somehow.  Gee, if only there was some way to narrow this down.
>   
Except that it doesn't**.  It doesn't use any windows calls (at
least none that aren't included on a standard linux system; :-)).
> I know! It must be because fork doesn't work on a multi-threaded dual
> core processor!
>   
---
    It doesn't?  That sounds nasty, but unfortunately, I'm still
running on a pokey Mobile Pentium-III, no dualing cores or
multi-shredded for me! :-)

    Since I hadn't had any luck in paring down the test case,
I thought it might be possible, depending on the debugging tools
available, to recreate the "hang", then find out why the processes
don't respond to normal signals, since that shouldn't normally
happen for POSIX compliant programs, right?


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/