Latest perl JSON::XS failing when threaded

Latest perl JSON::XS failing when threaded

[Wayne Davison]

I updated cygwin today, and now perl JSON::XS fails to be able to
decode json data if it is run in a multi-threaded script.  I tested
both x86_64 and x86 32-bit versions of cygwin, and they both have the
issue: once I launch a new thread, JSON::XS refuses to parse anything,
even in the primary thread and even if the launched thread didn't do
any json work.  The error is similar to this for a decode_json() call:

Thread 1 terminated abnormally: JSON text must be an object or array [...]

Note that it is complaining about the arg not being an object when
decoding takes a string. If I pass it an object, it accepts the
wrong-typed var and fails with a parsing error.

I forcefully uninstalled JSON::XS and the bug goes away.

I'll append a simple test script that tries to do a decrypt_json() in a thread.

..wayne..
--------------------------------------------------------------------------------
#!/usr/bin/perl
use strict;
use warnings;
use threads;
use JSON::XS;

print "Normal:\n"; # This works:
test_json_xs();

print "Threaded:\n"; # This throws an error:
my $t = threads->create(\&test_json_xs);
$t->detach();
exit;

sub test_json_xs
{
    my $test = <<EOT;
  {
    "testing": [ "one", "two", "three" ]
  }
EOT
    my $try = decode_json($test);
    if (ref $try->{testing} eq 'ARRAY') {
        print "OK!\n";
    } else {
        print "FAIL!\n";
    }
}

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Latest perl JSON::XS failing when threaded

[Achim Gratz]

Wayne Davison <wayne <at> opencoder.net> writes:
> I updated cygwin today, and now perl JSON::XS fails to be able to
> decode json data if it is run in a multi-threaded script.

That's the expected outcome, see:
http://search.cpan.org/dist/JSON-XS/XS.pm#THREADS

> I forcefully uninstalled JSON::XS and the bug goes away.

Uh, check the documentation for how to use the PP instead of the XS backend
with JSON.  Or maybe try Test::Without::Module.

Regards,
Achim.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Latest perl JSON::XS failing when threaded

[Achim Gratz]

Achim Gratz writes:
> Wayne Davison <wayne <at> opencoder.net> writes:
>> I updated cygwin today, and now perl JSON::XS fails to be able to
>> decode json data if it is run in a multi-threaded script.
>
> That's the expected outcome, see:
> http://search.cpan.org/dist/JSON-XS/XS.pm#THREADS

Since you mentioned it happens after the update of Cygwin, what was the
last version of JSON::XS that worked?


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Latest perl JSON::XS failing when threaded

[Achim Gratz]

Wayne Davison writes:
> Note that it is complaining about the arg not being an object when
> decoding takes a string. If I pass it an object, it accepts the
> wrong-typed var and fails with a parsing error.

The error message is about the return value of the decode and it is the
result of a new test that obviously does the wrong thing when the
JSON::XS object is being cloned before the thread gets set up.

> I forcefully uninstalled JSON::XS and the bug goes away.

You should be able to use the previous version (3.01) until this gets
sorted out upstream.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Latest perl JSON::XS failing when threaded

[Achim Gratz]

Wayne Davison writes:
> I updated cygwin today, and now perl JSON::XS fails to be able to
> decode json data if it is run in a multi-threaded script.

As said before, the use of JSON::XS with threads is not supported by
upstream.  That said, I've found a patch at least for the scenario that
your test case uses.  Some situations would have degraded performance,
however, although these have seemingly not been supported in earlier
versions of JSON::XS, so no regressions in existing code are expected to
occur.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Latest perl JSON::XS failing when threaded

[Wayne Davison]

On Sun, Mar 13, 2016 at 10:21 AM, Achim Gratz <Stromeko@nexgo.de> wrote:
> I've found a patch at least for the scenario that your test case uses.

Thanks -- the new version does indeed get things back to "normal" (how
things were behaving before).

As for thread support, it looks like I need to start using
Cpanel::JSON::XS (which might be nice to get added into cygwin).  I
noticed that the newly added JSON::MaybeXS will choose
Cpanel::JSON::XS first if it is around (unlike "JSON", which just
seems to choose between JSON::XS and JSON::PP).

..wayne..

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Latest perl JSON::XS failing when threaded

[Achim Gratz]

Wayne Davison writes:
> As for thread support, it looks like I need to start using
> Cpanel::JSON::XS (which might be nice to get added into cygwin).  I
> noticed that the newly added JSON::MaybeXS will choose
> Cpanel::JSON::XS first if it is around (unlike "JSON", which just
> seems to choose between JSON::XS and JSON::PP).

These distribution have been added to Cygwin.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple