python3 mod_wsgi and patchwork

python3 mod_wsgi and patchwork

[Siddhesh Poyarekar]

Hi,

I'm trying to upgrade the patchwork instance on sourceware and ran into
an issue with package versions on the system.  Patchwork (and django
1.6+) needs at least python2.7 when running on the python2 interpreter
and the latest python2 is 2.6 on sourceware.  Using python3 is not
possible because the installed and configured mod_wsgi is too old (needs
3.4+ AFAICT) and is linked with python2.

Can we upgrade these packages to enable the upgrade?  It would be ideal
if we do a full OS upgrade to a newer version because I suspect I'll run
into more package version issues as I upgrade to newer versions of
patchwork; I'm updating a year at a time (i.e. now I'm trying the 2015
version) and I've got a few years worth of patchwork updates to go
through :)

Siddhesh

Re: python3 mod_wsgi and patchwork

[Frank Ch. Eigler]

Hi -

> [...] Can we upgrade these packages to enable the upgrade?  It would
> be ideal if we do a full OS upgrade to a newer version because I
> suspect I'll run into more package version issues as I upgrade to
> newer versions of patchwork; I'm updating a year at a time (i.e. now
> I'm trying the 2015 version) and I've got a few years worth of
> patchwork updates to go through :)

The current sourceware server runs up-to-date RHEL 6, and is limited
to packages distributed for that OS by RH or via Fedora EPEL.
Updating the complete OS in situ is not practical, but the new
sourceware server is under construction that will use CentOS 8 - so
python3 will come with it.

EPEL does include a build of python 3.4, so we might try it, but if it
interferes with the system, we can't hold onto it.  Are you sure that
e.g. mod_wsgi is necessary here, as opposed to reverse-proxying or
calling out to the patchwork code via normal cgi?

- FChE

Re: python3 mod_wsgi and patchwork

[Siddhesh Poyarekar]

On 01/12/19 10:29 pm, Frank Ch. Eigler wrote:
> Hi -
> 
>> [...] Can we upgrade these packages to enable the upgrade?  It would
>> be ideal if we do a full OS upgrade to a newer version because I
>> suspect I'll run into more package version issues as I upgrade to
>> newer versions of patchwork; I'm updating a year at a time (i.e. now
>> I'm trying the 2015 version) and I've got a few years worth of
>> patchwork updates to go through :)
> 
> The current sourceware server runs up-to-date RHEL 6, and is limited
> to packages distributed for that OS by RH or via Fedora EPEL.
> Updating the complete OS in situ is not practical, but the new
> sourceware server is under construction that will use CentOS 8 - so
> python3 will come with it.

That sounds great, do you know when the new sourceware server will be
deployed?

> EPEL does include a build of python 3.4, so we might try it, but if it
> interferes with the system, we can't hold onto it.  Are you sure that
> e.g. mod_wsgi is necessary here, as opposed to reverse-proxying or
> calling out to the patchwork code via normal cgi?
> 

AFAICT, python3 is already installed, just that the installed mod_wsgi
cannot use it since it does not support python3.  I didn't give cgi a
thought for the deployment; let me try that with python3 and get back to
you.

Siddhesh

Re: python3 mod_wsgi and patchwork

[Frank Ch. Eigler]

Hi -

> [...]
> > Updating the complete OS in situ is not practical, but the new
> > sourceware server is under construction that will use CentOS 8 - so
> > python3 will come with it.
> 
> That sounds great, do you know when the new sourceware server will be
> deployed?

Sometime in the next few months.

> AFAICT, python3 is already installed, just that the installed mod_wsgi
> cannot use it since it does not support python3.  I didn't give cgi a
> thought for the deployment; let me try that with python3 and get back to
> you.

OK, good luck!

- FChE

Re: python3 mod_wsgi and patchwork

[Yaakov Selkowitz]

On Sun, 2019-12-01 at 11:59 -0500, Frank Ch. Eigler wrote:
> Hi -
> 
> > [...] Can we upgrade these packages to enable the upgrade?  It would
> > be ideal if we do a full OS upgrade to a newer version because I
> > suspect I'll run into more package version issues as I upgrade to
> > newer versions of patchwork; I'm updating a year at a time (i.e. now
> > I'm trying the 2015 version) and I've got a few years worth of
> > patchwork updates to go through :)
> 
> The current sourceware server runs up-to-date RHEL 6, and is limited
> to packages distributed for that OS by RH or via Fedora EPEL.

Note that the python27 SCL is already installed; can you use that?

-- 
Yaakov Selkowitz
Senior Software Engineer - Platform Enablement
Red Hat, Inc.

Re: python3 mod_wsgi and patchwork

[Siddhesh Poyarekar]

On 01/12/19 10:55 pm, Frank Ch. Eigler wrote:
>> AFAICT, python3 is already installed, just that the installed mod_wsgi
>> cannot use it since it does not support python3.  I didn't give cgi a
>> thought for the deployment; let me try that with python3 and get back to
>> you.
> 
> OK, good luck!

I'v made some amount of progress since; I was able to update django to
1.6.x and also fast-forward the sourceware patchwork version to about
2015 without losing any data or users.  I did lose Carlos' patch to
pwclient which converts the code to python3, but I think we can bring
that back at a later date.

This is the point (i.e. an upgrade to django 1.7.x) at which the sources
either need python 2.7+ or python3.  I tried running an independent
server using python3 but it fails due to missing modules.  Instead of
requesting for individual modules to install using yum, do you think it
would make sense to instead just install python34-pip?  I can then use
pip to install modules in user mode only for the patchwork user.

Siddhesh

Re: python3 mod_wsgi and patchwork

[Frank Ch. Eigler]

Hi -

> [...] Instead of requesting for individual modules to install using
> yum, do you think it would make sense to instead just install
> python34-pip?  I can then use pip to install modules in user mode
> only for the patchwork user.

OK, python34-pip is a go.

- FChE

Re: python3 mod_wsgi and patchwork

[Siddhesh Poyarekar]

Hello folks,

So I've wrestled with this for a while and it looks like there's no easy
way to do this upgrade without upgrading the OS.  The latest patchwork
and django use features in python 3.7 and those are not available on
RHEL-6.  I tried a few combinations of older patchwork/django versions
and all of them broke in different ways at different stages.  I'd have
to do something similar for the modules they use too, e.g.
rest-framework, flup, etc.

I think I'll just wait for the upgraded server to try this out, or maybe
explore another option with the community, perhaps by piloting on
patchwork.siddhesh.in again and then migrating once the new sourceware
server is ready.

Siddhesh

Re: python3 mod_wsgi and patchwork

[Siddhesh Poyarekar]

On 10/12/19 3:55 am, Frank Ch. Eigler wrote:
> Would it be crazy to try to build your own copy of python3.7 and
> install it under /home/patchwork/SOMETHING privately?

It's doable but would be quite a bit of work because I'll have to figure
out and build the whole dependency tree for both patchwork and django.
I have an instance from OSCI to play with for now.  I reckon it's
actually better to try things out there instead of tinkering and
breaking patchwork here.

Siddhesh