perl-5.10.0-4 -> perl-5.10.0-5 site_perl update needed

perl-5.10.0-4 -> perl-5.10.0-5 site_perl update needed

[Reini Urban]

2008/7/9 Reini Urban:
> 2008/7/9 Reini Urban:
>> 2008/7/8 Reini Urban:
>>> 2008/7/8 Gary R. Van Sickle:
>>>> Hi Reini,
>>>>
>>>> It looks like something changed between perl 5.10.0-4 and -5 which breaks
>>>> the DateTime module from CPAN (version 0.4302, latest).  The attached Simple
>>>> Test Case(tm) results in only the following errors:
>>>>
>>>> Use of uninitialized value $id in pattern match (m//) at
>>>> /usr/lib/perl5/site_perl/5.10/DateTime/Locale.pm line 65.
>>>
>>> Bad side-effect with a third party module in site_perl.
>>> Strange that I didn't see this in my tests, because I tested over
>>> thousand CPAN modules with this release.
>>> Oh, I see, I reported that it failed on June 5.
>>>
>>> $ cpan DateTime
>>> requires DateTime::Locale
>>>
>>> DateTime::Locale cannot be installed because of this error,
>>> but since you upgraded perl we have to fix DateTime::Locale now.
>>>
>>> Oops. This seems to affect more modules.
>>> Params::Validate with type => SCALAR seems to be broken.
>>>  http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Params%3A%3AValidate
>>>
>>> I have to go now, but will provide a patch soon.
>>> As workaround I would disable the validation in DateTime::Locale::_register
>>> but this also doesn't work as expected.
>>>
>>>    if (! defined $p{id} or !$id) {
>>>        # cygperl5 error in Params::Validate?
>>>        #warn "Broken Params::Validate. Given ".join(",",@_);
>>>        $p{id} = $_[0];
>>>        $p{en_language} = $_[1];
>>>        my @keys = qw(en_script en_territory en_variant native_language
>>> native_script native_territory native_variant class replace);
>>>        for my $i (2 .. @_) {
>>>            $p{$keys[$i]} = $_[$i];
>>>        }
>>>    }
>>
>> Update on this issue:
>> As you can see, there's only one machine with this error, which is my laptop.
>> http://cpantesters.perl.org/show/DateTime-Locale.html#DateTime-Locale-0.4001
>> My normal devel machine passed the DateTime::Locale test, which is weird.
>> I also pass the Params::Validate test on my laptop, which is weirder,
>> because this fails for DateTime::Locale on my laptop.
>>
>> Anyway, until I find out the differences between my laptop and my
>> devel machine,
>> the short summary:
>>
>> Params::Validate fails in an BEGIN block, in a type => SCALAR check only on
>> certain machines with 5.10.0 patch 34065 on Cygwin. (Tested only
>> release and patch 34065).
>> Reproducible with DateTime::Locale.
>> Within the debugger everything works ok. (Probably because of the BEGIN block)
>> The other DateTime::Locale reports show no clue.
>> There's a Darwin report on the same patchlevel which passed.
>>
>> Within DateTime-Locale-0.4001 after make
>> even perl -Mblib -cw t/01basic.t fails with the same errors.
>>
>> Moving the register call from BEGIN to INIT also does not help.
>
> The adhoc workaround for perl-5.10.0-5 is to move away the Validate XS module:
>
> mv /usr/lib/perl5/site_perl/5.10/i686-cygwin/auto/Params/Validate/Validate.dll \
>  /usr/lib/perl5/site_perl/5.10/i686-cygwin/auto/Params/Validate/_Validate.dll
>
> Now it works, because Params::Validate has a slower pure-perl fallback.

I've analyzed the symbols in the dll's and found the explanation and
solution for this update problems. p5p and module authors need not to
be concerned. It's a simple cygwin perl update problem.

In short:
All old perl-5.10.0-4 generated binary site_perl modules need to be
recompiled with perl-5.10.0-5.
They are not ABI cross-compatible with the new vendor and archlib modules.

Explanation:
perl-5.10.0-5 is based on patchlevel 34065, perl-5.10.0-4 was based on
plain 5.10.0 with a lot of patches.

perl-5.10.0-5 adds a new utf8 handling of strings via a new
newSVpvn_flags function
sv.h: #define newSVpvn_utf8(s, len, u) newSVpvn_flags((s), (len), (u)
? SVf_UTF8 : 0)
perl-5.10.0-4 does not contain newSVpvn_utf8 nor newSVpvn_flags.

So any new XS module using strings may not call an old perl-5.10.0-4 dll, which
does not contain the _utf8  and _flags functions.

Solution:
# delete all old site packages with dll's and reinstall them
for pack in $(find /usr/lib/perl5/site_perl/5.10/i686-cygwin/auto \
  -name .packlist \! -newer /bin/perl.exe);
do
  grep .dll $pack && rm -f $(cat $pack)
done | \
perl -MCPAN -ne'
s|/usr/lib/perl5/site_perl/5.10/i686-cygwin/auto/(.+?)/\w+\.dll$|$1|;
s|/|::|g; push @p, $_; END {install(@p) }'

should recompile them all.

-- 
Reini Urban
http://phpwiki.org/ http://murbreak.at/

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: perl-5.10.0-4 -> perl-5.10.0-5 site_perl update needed

[Reini Urban]

2008/7/10 Reini Urban:
> I've analyzed the symbols in the dll's and found the explanation and
> solution for this update problems. p5p and module authors need not to
> be concerned. It's a simple cygwin perl update problem.
>
> In short:
> All old perl-5.10.0-4 generated binary site_perl modules need to be
> recompiled with perl-5.10.0-5.
> They are not ABI cross-compatible with the new vendor and archlib modules.
>
> Explanation:
> perl-5.10.0-5 is based on patchlevel 34065, perl-5.10.0-4 was based on
> plain 5.10.0 with a lot of patches.
>
> perl-5.10.0-5 adds a new utf8 handling of strings via a new
> newSVpvn_flags function
> sv.h: #define newSVpvn_utf8(s, len, u) newSVpvn_flags((s), (len), (u)
> ? SVf_UTF8 : 0)
> perl-5.10.0-4 does not contain newSVpvn_utf8 nor newSVpvn_flags.
>
> So any new XS module using strings may not call an old perl-5.10.0-4 dll, which
> does not contain the _utf8  and _flags functions.

Well, newSVpvn_utf8 is just a macro.

> Solution:
> # delete all old site packages with dll's and reinstall them
> for pack in $(find /usr/lib/perl5/site_perl/5.10/i686-cygwin/auto \
>  -name .packlist \! -newer /bin/perl.exe);
> do
>  grep .dll $pack && rm -f $(cat $pack)
> done | \
> perl -MCPAN -ne'
> s|/usr/lib/perl5/site_perl/5.10/i686-cygwin/auto/(.+?)/\w+\.dll$|$1|;
> s|/|::|g; push @p, $_; END {install(@p) }'
>
> should recompile them all.

Oops, I forgot the final \n. chomp is needed.
Removing the .packlist would also be good.

This is the script, needed if you installed binary cpan packages with
-4 or older 5.10 packages

<<<< ~/bin/perl-site-update-4 >>>>
for pack in $(find /usr/lib/perl5/site_perl/5.10/i686-cygwin/auto \
 -name .packlist \! -newer /bin/perl.exe);
do
 grep .dll $pack && rm -f $(cat $pack) && rm -f $pack
done | \
  perl -MCPAN -ne'
s|/usr/lib/perl5/site_perl/5.10/i686-cygwin/auto/(.+?)/\w+\.dll$|$1|;
s|/|::|g; chomp; push @p, $_; END {install(@p) }'

>>>> <<<<

-- 
Reini Urban
http://phpwiki.org/ http://murbreak.at/

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: perl-5.10.0-4 -> perl-5.10.0-5 site_perl update needed

[Yaakov (Cygwin Ports)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Reini Urban wrote:
| In short:
| All old perl-5.10.0-4 generated binary site_perl modules need to be
| recompiled with perl-5.10.0-5.
| They are not ABI cross-compatible with the new vendor and archlib modules.

Say WHAT?  I need to recompile all binary modules in Ports?!?!?


Yaakov
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEAREIAAYFAkh2cNoACgkQpiWmPGlmQSOIewCfYyiRYrLNLVg2DYEw3m3avo4b
Z20An0l2s9waHF89WWIidTQiNKuYHkaW
=f4de
-----END PGP SIGNATURE-----

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: perl-5.10.0-4 -> perl-5.10.0-5 site_perl update needed

[Jan Dubois]

On Thu, 10 Jul 2008, Reini Urban wrote:
> I've analyzed the symbols in the dll's and found the explanation and
> solution for this update problems. p5p and module authors need not to
> be concerned. It's a simple cygwin perl update problem.
> 
> In short:
> All old perl-5.10.0-4 generated binary site_perl modules need to be
> recompiled with perl-5.10.0-5.

This should not be true: if the modules were compiled with 5.10.0-4, then
they should work with 5.10.0-5.

> They are not ABI cross-compatible with the new vendor and archlib modules.
> 
> Explanation:
> perl-5.10.0-5 is based on patchlevel 34065, perl-5.10.0-4 was based on
> plain 5.10.0 with a lot of patches.
> 
> perl-5.10.0-5 adds a new utf8 handling of strings via a new
> newSVpvn_flags function
> sv.h: #define newSVpvn_utf8(s, len, u) newSVpvn_flags((s), (len), (u)
> ? SVf_UTF8 : 0)
> perl-5.10.0-4 does not contain newSVpvn_utf8 nor newSVpvn_flags.
> 
> So any new XS module using strings may not call an old perl-5.10.0-4 dll, which
> does not contain the _utf8  and _flags functions.

So how do you end up with a site_perl compiled against 5.10.0-5, but
running with 5.10.0-4? You installed the new site_perl into an older
Perl installation?

> Solution:
> # delete all old site packages with dll's and reinstall them
> for pack in $(find /usr/lib/perl5/site_perl/5.10/i686-cygwin/auto \
>   -name .packlist \! -newer /bin/perl.exe);
> do
>   grep .dll $pack && rm -f $(cat $pack)
> done | \
> perl -MCPAN -ne'
> s|/usr/lib/perl5/site_perl/5.10/i686-cygwin/auto/(.+?)/\w+\.dll$|$1|;
> s|/|::|g; push @p, $_; END {install(@p) }'
> 
> should recompile them all.

But that would compile them with 5.10.0-4 again, right. Wouldn't it be
better to replace your 5.10.0-4 dll with the 5.10.0-5 dll?

I guess I'm confused about how you ended up in this situation in the
first place.

Cheers,
-Jan


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: perl-5.10.0-4 -> perl-5.10.0-5 site_perl update needed

[Reini Urban]

Yaakov (Cygwin Ports) schrieb:
> Reini Urban wrote:
> | In short:
> | All old perl-5.10.0-4 generated binary site_perl modules need to be
> | recompiled with perl-5.10.0-5.
> | They are not ABI cross-compatible with the new vendor and archlib 
> modules.
> 
> Say WHAT?  I need to recompile all binary modules in Ports?!?!?

That should not have happened, because the MAINT maintainer (Nicholas 
Clark) must not have applied patches, which break the 5.10 binary ABI.

Anyway, it happened now and I wonder what would be best.
Continue as with the 5.8.x mess where the same utf8 disaster happened, 
or go back to a -6 based on -4 based on quilt.

I wait for an official statement and proposed fixes from p5p for a few days.
My 5.10 patch queue was getting really big, and I was almost forced to 
switch my build system to be quilt based. That's why I switched to a 
MAINT patchlevel version.
-- 
Reini Urban
http://phpwiki.org/  http://murbreak.at/

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: perl-5.10.0-4 -> perl-5.10.0-5 site_perl update needed

[Reini Urban]

Reini Urban schrieb:
> Yaakov (Cygwin Ports) schrieb:
>> Reini Urban wrote:
>> | In short:
>> | All old perl-5.10.0-4 generated binary site_perl modules need to be
>> | recompiled with perl-5.10.0-5.
>> | They are not ABI cross-compatible with the new vendor and archlib 
>> modules.
>>
>> Say WHAT?  I need to recompile all binary modules in Ports?!?!?
> 
> That should not have happened, because the MAINT maintainer (Nicholas 
> Clark) must not have applied patches, which break the 5.10 binary ABI.

Wrong.

> Anyway, it happened now and I wonder what would be best.
> Continue as with the 5.8.x mess where the same utf8 disaster happened, 
> or go back to a -6 based on -4 based on quilt.
> 
> I wait for an official statement and proposed fixes from p5p for a few 
> days.
> My 5.10 patch queue was getting really big, and I was almost forced to 
> switch my build system to be quilt based. That's why I switched to a 
> MAINT patchlevel version.

The p5p statement was that it is legal to ADD functions, and it is 
illegal to REMOVE functions during the 5.10 cycle.
So it's not comparable to the 5.8 case, where structs were changed.

So breakage only occurs if a new XS library (here DateTime::Locale) 
calls an old XS library (here Params::Validate) without this new function.
This would have happened with 5.10.1 also.

A fix could be, that CPAN should refuse to accept XS packages with the 
same version number but built with older perl versions in its dependency 
checker. So that those old XS packages would get recompiled 
automatically. I'll have a chat with Andreas König about that.
-- 
Reini Urban
http://phpwiki.org/  http://murbreak.at/

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: perl-5.10.0-4 -> perl-5.10.0-5 site_perl update needed

[Yaakov (Cygwin Ports)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Reini Urban wrote:
| The p5p statement was that it is legal to ADD functions, and it is
| illegal to REMOVE functions during the 5.10 cycle.
| So it's not comparable to the 5.8 case, where structs were changed.

OK, that makes sense.

| So breakage only occurs if a new XS library (here DateTime::Locale)
| calls an old XS library (here Params::Validate) without this new function.
| This would have happened with 5.10.1 also.

I did notice that a gtk2-perl program did run after the upgrade to -5.
So how can I detect if an XS module needs to rebuilt or not?


Yaakov

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEAREIAAYFAkh37lgACgkQpiWmPGlmQSP2MwCggjVR1nQATzLdqI1BKQjGXydA
5YMAoKOAwLo0S4vGuHyr+BN89bIIye4b
=ngKI
-----END PGP SIGNATURE-----

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: perl-5.10.0-4 -> perl-5.10.0-5 site_perl update needed

[Reini Urban]

2008/7/12 Yaakov (Cygwin Ports):
> Reini Urban wrote:
> | The p5p statement was that it is legal to ADD functions, and it is
> | illegal to REMOVE functions during the 5.10 cycle.
> | So it's not comparable to the 5.8 case, where structs were changed.
>
> OK, that makes sense.

Such CPAN site incompatibilities are so common, there there exists
even a simplier solution for this:

perl -MCPAN -e'recompile'

Quoting perldoc /usr/lib/perl5/5.10/CPAN.pm

 recompile() is a very special command in that it takes no argument and
       runs the make/test/install cycle with brute force over all installed
       dynamically loadable extensions (aka XS modules) with 'force' in
       effect. The primary purpose of this command is to finish a network
       installation. Imagine, you have a common source tree for two different
       architectures. You decide to do a completely independent fresh
       installation. You start on one architecture with the help of a Bundle
       file produced earlier. CPAN installs the whole Bundle for you, but when
       you try to repeat the job on the second architecture, CPAN responds
       with a "Foo up to date" message for all modules. So you invoke CPAN's
       recompile on the second architecture and you're done.

       Another popular use for "recompile" is to act as a rescue in case your
       perl breaks binary compatibility. If one of the modules that CPAN uses
       is in turn depending on binary compatibility (so you cannot run CPAN
       commands), then you should try the CPAN::Nox module for recovery.

> | So breakage only occurs if a new XS library (here DateTime::Locale)
> | calls an old XS library (here Params::Validate) without this new function.
> | This would have happened with 5.10.1 also.
>
> I did notice that a gtk2-perl program did run after the upgrade to -5.
> So how can I detect if an XS module needs to rebuilt or not?

Hmm, I would like to know that also.
So far I only saw the Params::Validate problem with DateTime-Locale.

It could be any of these current issues, but most have different failures.

test FAIL ack-1.84  (permission problem)
test FAIL PAR-Packer-0.980
test FAIL Archive-Rar-2.02
test FAIL Class-MOP-0.62
test FAIL DateTime-Locale-0.4001 - old Params::Validate in site
test FAIL Devel-Tokenizer-C-0.07
test FAIL Devel-TypeCheck-1.2
test FAIL GraphViz-2.03
test FAIL Log-Dispatch-2.21
test FAIL Log-Log4perl-1.16
test FAIL Log-Report-0.18
test FAIL MP3-Find-0.03
test FAIL Net-Daemon-0.43
test FAIL Perl-Critic-1.088
test FAIL Text-RecordParser-v1.2.1
test FAIL Readonly-XS-1.04   - This looks suspect also
test FAIL PerlIO-gzip-0.18

But
$ for p in $(egrep "test FAIL.*perl-5.10.0 patch 34065" reports-sent.db |\
  perl -ane'print $F[2],"\n"'); do grep "test PASS $p" reports-sent.db; done

only gives me:

test PASS DateTime-Locale-0.4001 (perl-5.10.0 patch 34065)
cygwin-thread-multi-64int 1.5.25(0.15642)

So the recompile of all my binary site XS modules only fixed the
DateTime-Locale-0.4001 issue.
-- 
Reini Urban
http://phpwiki.org/ http://murbreak.at/

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: perl-5.10.0-4 -> perl-5.10.0-5 site_perl update needed

[Yaakov (Cygwin Ports)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Reini Urban wrote:
| Such CPAN site incompatibilities are so common, there there exists
| even a simplier solution for this:
|
| perl -MCPAN -e'recompile'

That's good to know.

| Hmm, I would like to know that also.
| So far I only saw the Params::Validate problem with DateTime-Locale.
|
| It could be any of these current issues, but most have different failures.
| [snip]
| But
| $ for p in $(egrep "test FAIL.*perl-5.10.0 patch 34065" reports-sent.db |\
|   perl -ane'print $F[2],"\n"'); do grep "test PASS $p"
reports-sent.db; done
|
| only gives me:
|
| test PASS DateTime-Locale-0.4001 (perl-5.10.0 patch 34065)
| cygwin-thread-multi-64int 1.5.25(0.15642)
|
| So the recompile of all my binary site XS modules only fixed the
| DateTime-Locale-0.4001 issue.

The DateTime modules were just updated anyway, so I'll be sure to bump.
~ I know gtk2-perl modules are still working, so I think that I'll let
this sit otherwise unless something comes up.


Yaakov
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEAREIAAYFAkh7gFwACgkQpiWmPGlmQSMYdwCfRXjv0dWQg9FCdoScpv88TroP
KZYAmwURQouh8OlVs7FeTopVJc3zmnbD
=Wg71
-----END PGP SIGNATURE-----

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/