2.3.0: possible Cygwin flock bug (Windows 10 x86_64)

2.3.0: possible Cygwin flock bug (Windows 10 x86_64)

[Mario Roy]

Greetings,

During testing of the Perl MCE 1.608 module (uses flock), the examples
scripts hang. Downgrading the base Cygwin package from 2.3.0-1 to
2.2.1-1 resolves the issue.

Installation is not required if a Cygwin developer desires to test
file locking via the Perl MCE module.

https://cpan.metacpan.org/authors/id/M/MA/MARIOROY/MCE-1.608.tar.gz

1. extract tar file
2. cd MCE-1.608/examples
3. perl foreach.pl 10

Hangs with Cygwin 2.3.0-1
Successful with Cygwin 2.2.1-1

The upcoming Perl MCE 1.7 release does away with file locking and
works with 2.3.0-1. The next commit into GitHub (in a day or two) will
contain optimizations allowing Perl MCE on Windows and Cygwin to run
at full speed with lesser overhead.

Best regards,
Mario

p.s. Thank you for Cygwin

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 2.3.0: possible Cygwin flock bug (Windows 10 x86_64)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1399 bytes --]

Hi Mario,

On Nov 11 21:51, Mario Roy wrote:
> Greetings,
> 
> During testing of the Perl MCE 1.608 module (uses flock), the examples
> scripts hang. Downgrading the base Cygwin package from 2.3.0-1 to
> 2.2.1-1 resolves the issue.
> 
> Installation is not required if a Cygwin developer desires to test
> file locking via the Perl MCE module.
> 
> https://cpan.metacpan.org/authors/id/M/MA/MARIOROY/MCE-1.608.tar.gz
> 
> 1. extract tar file
> 2. cd MCE-1.608/examples
> 3. perl foreach.pl 10
> 
> Hangs with Cygwin 2.3.0-1
> Successful with Cygwin 2.2.1-1

Thanks for the testcase.  I found the problem.  In 2.2.1 was a 
use-after-free bug in the file locking code which I fixed in
2.3.0... just wrongly.  It fails to work in certain cases when
locking collisions occur.  I fixed this in the git repo.

I uploaded a new developer snapshot to https://cygwin.com/snapshots/
Please give it a try.

> The upcoming Perl MCE 1.7 release does away with file locking and
> works with 2.3.0-1.

Uh, please don't do this.  This is a bug in Cygwin 2.3.0, and the right
fix is to release Cygwin 2.3.1 end of this week, not to change other
packages to workaround this bug.

> p.s. Thank you for Cygwin

Thanks :)
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: 2.3.0: possible Cygwin flock bug (Windows 10 x86_64)

[Mario Roy]

Hello Corinna,

I am writing to confirm that file locking, with MCE 1.608, utilizing
the development snapshot 2015-11-12 x86_64 is passing 100%. I ran
through other test cases including mixing threads and child processes.

Currently, MCE 1.608 does not allow one to mix threads and child
processes under Cygwin. I commented out the check in
MCE-1.608/lib/MCE.pm  (lines 424 - 435).

The following (mixing threads and child processes) now runs on Cygwin
using flock. Hurray !!!

use threads;
use threads::shared;
use MCE;

sub func {
   my ($mce) = @_;
   $mce->say($mce->wid);
}

my $mce = MCE->new(
 mutex_type => 'channel',
 user_tasks =>[{
    use_threads => 0,
    max_workers => 8,
    user_func   => \&func,
 },{
    use_threads => 0,
    max_workers => 8,
    user_func   => \&func,
 },{
    use_threads => 1,
    max_workers => 8,
    user_func   => \&func,
 },{
    use_threads => 0,
    max_workers => 8,
    user_func   => \&func,
 },{
    use_threads => 1,
    max_workers => 8,
    user_func   => \&func,
 }]
)->run;


The decision with channel locking for the upcoming MCE 1.7 release is
for another reason. In Perl, each worker must obtain the file lock
handle including threads. There are ulimit restrictions for number of
open file handles. With channel locking, workers are not having to
re-open any handles. Thus allowing the process to run with more
threads.

Recently, I simplified MCE/Mutex.pm to do channel locking only. Not to
worry, I may have MCE::Mutex support both channel and file locking.
Thus, will do the following for the upcoming MCE 1.700.

MCE-1.700/lib/MCE/Mutex.pm
MCE-1.700/lib/MCE/Mutex/Channel.pm
MCE-1.700/lib/MCE/Mutex/Flock.pm

Then, one may specify the type desired.

use MCE::Mutex;

my $m1 = MCE::Mutex->new( type => 'channel' );    # default if type is
not specified
my $m2 = MCE::Mutex->new( type => 'flock' );

Channel locking supports lock and unlock only.

$m1->lock;         # similarly to LOCK_EX
$m1->unlock;     # similarly to LOCK_UN

Will add the shared method for LOCK_SH in MCE/Mutex/Flock.pm

$m2->lock;
$m2->unlock;
$m2->shared;

MCE 1.700 in Github runs well with channel locking across the board.
However, being able to obtain a shared lock (possible with file
locking) at the application level is nice. Seeing the above example
work makes me confident in bringing back file locking and know that it
will work across platforms including Cygwin.

Thank you for fixing flock in Cygwin.dll.

Regards,
Mario

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 2.3.0: possible Cygwin flock bug (Windows 10 x86_64)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 896 bytes --]

On Nov 13 13:25, Mario Roy wrote:
> Hello Corinna,
> 
> I am writing to confirm that file locking, with MCE 1.608, utilizing
> the development snapshot 2015-11-12 x86_64 is passing 100%. I ran
> through other test cases including mixing threads and child processes.

Thanks for testing and confirming.  I uploaded the new Cygwin 2.3.1
already on Saturday.

> [...]
> MCE 1.700 in Github runs well with channel locking across the board.
> However, being able to obtain a shared lock (possible with file
> locking) at the application level is nice. Seeing the above example
> work makes me confident in bringing back file locking and know that it
> will work across platforms including Cygwin.

Sounds good.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]