[64bit] python3 vs. threads

[64bit] python3 vs. threads

[Yaakov (Cygwin/X)]

Jason,

As part of the 64bit bootstrap process, I packaged python3, but had to 
configure it --without-threads due to a runtime error.  However, I am 
finding now that this is not a regularly used configuration; several 
major Python modules use PyGILState_*() APIs unconditionally, which 
aren't exported when so configured.  Are you able to take a look at the 
runtime failure and track it down?


Yaakov

Re: [64bit] python3 vs. threads

[Jason Tishler]

Yaakov,

On Mon, Jun 03, 2013 at 11:30:28PM -0500, Yaakov (Cygwin/X) wrote:
> As part of the 64bit bootstrap process, I packaged python3, but had
> to configure it --without-threads due to a runtime error.  However,
> I am finding now that this is not a regularly used configuration;
> several major Python modules use PyGILState_*() APIs
> unconditionally, which aren't exported when so configured.  Are you
> able to take a look at the runtime failure and track it down?

Unfortunately, I don't have access to 64-bit Windows.

Jason

Re: [64bit] python3 vs. threads

[Corinna Vinschen]

Hi Jason,

On Jun  7 19:07, Jason Tishler wrote:
> Yaakov,
> 
> On Mon, Jun 03, 2013 at 11:30:28PM -0500, Yaakov (Cygwin/X) wrote:
> > As part of the 64bit bootstrap process, I packaged python3, but had
> > to configure it --without-threads due to a runtime error.  However,
> > I am finding now that this is not a regularly used configuration;
> > several major Python modules use PyGILState_*() APIs
> > unconditionally, which aren't exported when so configured.  Are you
> > able to take a look at the runtime failure and track it down?
> 
> Unfortunately, I don't have access to 64-bit Windows.

You can download a Windows 2012 Evaluation ISO for free from the MSDN
homepage:

http://msdn.microsoft.com/en-us/evalcenter/hh708764.aspx

Windows 2012 is always 64 bit, and the evaluation versions works for
180 days for, well, evaluation.  It runs fine in VMs.

Does that help?


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

Re: [64bit] python3 vs. threads

[Corinna Vinschen]

On Jun 10 10:32, Corinna Vinschen wrote:
> Hi Jason,
> 
> On Jun  7 19:07, Jason Tishler wrote:
> > Yaakov,
> > 
> > On Mon, Jun 03, 2013 at 11:30:28PM -0500, Yaakov (Cygwin/X) wrote:
> > > As part of the 64bit bootstrap process, I packaged python3, but had
> > > to configure it --without-threads due to a runtime error.  However,
> > > I am finding now that this is not a regularly used configuration;
> > > several major Python modules use PyGILState_*() APIs
> > > unconditionally, which aren't exported when so configured.  Are you
> > > able to take a look at the runtime failure and track it down?
> > 
> > Unfortunately, I don't have access to 64-bit Windows.
> 
> You can download a Windows 2012 Evaluation ISO for free from the MSDN
> homepage:
> 
> http://msdn.microsoft.com/en-us/evalcenter/hh708764.aspx
> 
> Windows 2012 is always 64 bit, and the evaluation versions works for
> 180 days for, well, evaluation.  It runs fine in VMs.
> 
> Does that help?

There's also a shorter running (90 days) evaluation version of Windows 8
available, btw:

http://technet.microsoft.com/en-US/evalcenter/hh699156.aspx

but usually it's enough to check on the server system.  The differences
between client and server usually(*) don't affect Cygwin.


Corinna

(*) Except they did when Vista was new.  Some difference in exception
    handling on 32 bit foiled Cygwin's exception handling on 2008, but
    not on Vista.  But that's the only one I remember.

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

Re: [64bit] python3 vs. threads

[Yaakov (Cygwin/X)]

On 2013-06-03 23:30, Yaakov (Cygwin/X) wrote:
> Jason,
>
> As part of the 64bit bootstrap process, I packaged python3, but had to
> configure it --without-threads due to a runtime error.

I think I tracked down the source of the problem:

The PyThread TLS APIs, added in 3.2 via issue 9786 (namely, 
PyThread_*_key and PyThread_*_key_value in Include/pythread.h and 
Python/thread_pthread.h) assume sizeof(pthread_key_t) == sizeof(int). 
However, on x86_64-cygwin, pthread_key_t is a pointer and so the native 
calls fail when passed the incorrect values.

As for fixing it, we could s/int/long/g those APIs, but I'm not sure 
what the ramifications of that might be.  Can you think of a better 
solution?


Yaakov

Re: [64bit] python3 vs. threads

[Yaakov (Cygwin/X)]

On 2013-07-30 17:54, Yaakov (Cygwin/X) wrote:
> On 2013-06-03 23:30, Yaakov (Cygwin/X) wrote:
>> Jason,
>>
>> As part of the 64bit bootstrap process, I packaged python3, but had to
>> configure it --without-threads due to a runtime error.
>
> I think I tracked down the source of the problem:
>
> The PyThread TLS APIs, added in 3.2 via issue 9786 (namely,
> PyThread_*_key and PyThread_*_key_value in Include/pythread.h and
> Python/thread_pthread.h) assume sizeof(pthread_key_t) == sizeof(int).
> However, on x86_64-cygwin, pthread_key_t is a pointer and so the native
> calls fail when passed the incorrect values.
>
> As for fixing it, we could s/int/long/g those APIs, but I'm not sure
> what the ramifications of that might be.  Can you think of a better
> solution?

I have patchsets ready for 2.7.5 (to fix the parsetuple issue) and 3.2.5 
in Ports git; see 3.2-thread-cygwin64.patch for my proposed solution, 
which would have to be rebased to a newer version and expanded to all 
the other platform-specific implementations before pushing upstream.

If you're okay with these, I have packages ready for x86_64.

http://cygwin-ports.git.sourceforge.net/git/gitweb.cgi?p=cygwin-ports/python;a=tree
http://cygwin-ports.git.sourceforge.net/git/gitweb.cgi?p=cygwin-ports/python3;a=tree


Yaakov

Re: [64bit] python3 vs. threads

[Yaakov (Cygwin/X)]

On 2013-07-30 20:52, Yaakov (Cygwin/X) wrote:
> I have patchsets ready for 2.7.5 (to fix the parsetuple issue) and 3.2.5
> in Ports git; see 3.2-thread-cygwin64.patch for my proposed solution,
> which would have to be rebased to a newer version and expanded to all
> the other platform-specific implementations before pushing upstream.
>
> If you're okay with these, I have packages ready for x86_64.
>
> http://cygwin-ports.git.sourceforge.net/git/gitweb.cgi?p=cygwin-ports/python;a=tree
> http://cygwin-ports.git.sourceforge.net/git/gitweb.cgi?p=cygwin-ports/python3;a=tree

Ping?


Yaakov

Re: [64bit] python3 vs. threads

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 904 bytes --]


Jason?  Is there any chance you could pick this up?


On Aug  9 00:38, Yaakov (Cygwin/X) wrote:
> On 2013-07-30 20:52, Yaakov (Cygwin/X) wrote:
> >I have patchsets ready for 2.7.5 (to fix the parsetuple issue) and 3.2.5
> >in Ports git; see 3.2-thread-cygwin64.patch for my proposed solution,
> >which would have to be rebased to a newer version and expanded to all
> >the other platform-specific implementations before pushing upstream.
> >
> >If you're okay with these, I have packages ready for x86_64.
> >
> >http://cygwin-ports.git.sourceforge.net/git/gitweb.cgi?p=cygwin-ports/python;a=tree
> >http://cygwin-ports.git.sourceforge.net/git/gitweb.cgi?p=cygwin-ports/python3;a=tree
> 
> Ping?
> 
> 
> Yaakov


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [64bit] python3 vs. threads

[Yaakov (Cygwin/X)]

On 2013-08-15 10:32, Corinna Vinschen wrote:
> Jason?  Is there any chance you could pick this up?

I NMU'd both python and python3 for x86_64 earlier this week as a 
prerequisite for the GNOME 3.8 update; so far they're both working well.


Yaakov

Re: [64bit] python3 vs. threads

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 520 bytes --]

On Aug 15 12:50, Yaakov (Cygwin/X) wrote:
> On 2013-08-15 10:32, Corinna Vinschen wrote:
> >Jason?  Is there any chance you could pick this up?
> 
> I NMU'd both python and python3 for x86_64 earlier this week as a
> prerequisite for the GNOME 3.8 update; so far they're both working
> well.

That sounds good, basically, but what means "NMU'd? :}


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [64bit] python3 vs. threads

[Yaakov (Cygwin/X)]

On 2013-08-15 13:23, Corinna Vinschen wrote:
> On Aug 15 12:50, Yaakov (Cygwin/X) wrote:
>> I NMU'd both python and python3 for x86_64 earlier this week as a
>> prerequisite for the GNOME 3.8 update; so far they're both working
>> well.
>
> That sounds good, basically, but what means "NMU'd? :}

"Non-maintainer upload" (Debian lingo, I'm not aware of a similar term 
in Fedora-land).


Yaakov

Re: [64bit] python3 vs. threads

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 657 bytes --]

On Aug 15 13:48, Yaakov (Cygwin/X) wrote:
> On 2013-08-15 13:23, Corinna Vinschen wrote:
> >On Aug 15 12:50, Yaakov (Cygwin/X) wrote:
> >>I NMU'd both python and python3 for x86_64 earlier this week as a
> >>prerequisite for the GNOME 3.8 update; so far they're both working
> >>well.
> >
> >That sounds good, basically, but what means "NMU'd? :}
> 
> "Non-maintainer upload" (Debian lingo, I'm not aware of a similar
> term in Fedora-land).

Uh, thanks.  I didn't know this term.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [64bit] python3 vs. threads

[Yaakov (Cygwin/X)]

On 2013-08-15 12:50, Yaakov (Cygwin/X) wrote:
> On 2013-08-15 10:32, Corinna Vinschen wrote:
>> Jason?  Is there any chance you could pick this up?
>
> I NMU'd both python and python3 for x86_64 earlier this week as a
> prerequisite for the GNOME 3.8 update; so far they're both working well.

I just added a -3 with a patch for the uuid module:

http://bugs.python.org/issue18784

Jason, any chance of matching updates for x86?


Yaakov

Re: [64bit] python3 vs. threads

[Jason Tishler]

Yaakov,

On Wed, Oct 02, 2013 at 11:53:55PM -0500, Yaakov (Cygwin/X) wrote:
> On 2013-08-15 12:50, Yaakov (Cygwin/X) wrote:
> >On 2013-08-15 10:32, Corinna Vinschen wrote:
> >>Jason?  Is there any chance you could pick this up?
> >
> >I NMU'd both python and python3 for x86_64 earlier this week as a
> >prerequisite for the GNOME 3.8 update; so far they're both working well.
> 
> I just added a -3 with a patch for the uuid module:
> 
> http://bugs.python.org/issue18784
> 
> Jason, any chance of matching updates for x86?

Yes, but I would like to work on getting my 64-bit environment set up
first.

Jason

Re: [64bit] python3 vs. threads

[Aron Ahmadia]

Hi Folks,

On Tue, Oct 15, 2013 at 8:28 PM, Jason Tishler <jason@tishler.net
> wrote:

> Yaakov,
>
> On Wed, Oct 02, 2013 at 11:53:55PM -0500, Yaakov (Cygwin/X) wrote:
> > On 2013-08-15 12:50, Yaakov (Cygwin/X) wrote:
> > >On 2013-08-15 10:32, Corinna Vinschen wrote:
> > >>Jason?  Is there any chance you could pick this up?
> > >
> > >I NMU'd both python and python3 for x86_64 earlier this week as a
> > >prerequisite for the GNOME 3.8 update; so far they're both working well.
> >
> > I just added a -3 with a patch for the uuid module:
> >
> > http://bugs.python.org/issue18784
> >
> > Jason, any chance of matching updates for x86?
>
> Yes, but I would like to work on getting my 64-bit environment set up
> first.
>
> Jason
>

Thanks for all of your work on this, but I don't think the Python uuid
issue is fixed yet.  The Python interpreter relies on the stubs in
libuuid-devel, and this dependency is not indicated in the setup.ini file
for Python or Python 3 (this is an issue for both codes).

Let me know if there's anything I can do to help!

-Aron

References:

https://sourceware.org/ml/cygwin/2014-04/msg00375.html

https://stackoverflow.com/questions/18947163/uuid-python-import-fails-on-cygwin-64bits

https://github.com/kennethreitz/requests/issues/1547

https://github.com/pypa/pip/issues/1448