Proposal to remove Python 2 support

Proposal to remove Python 2 support

[Tom Tromey]

Hi.  Python 2 officially reached its sunset date this year:

    https://www.python.org/doc/sunset-python-2/

I think gdb should follow suit starting in gdb 11 -- that is, my
proposal is to keep Python 2 support for the upcoming gdb 10 release,
but remove said support from git master.

In the short term this would amount to some relatively minor code
cleanups.

In the longer term, my preferred approach to handling inferior control
from Python is to base it on async/await, which IIUC is only available
in Python 3.  So, this would be an enabling step.

Let me know what you think.

Tom

Re: Proposal to remove Python 2 support

[Paul Koning]

> On Sep 15, 2020, at 4:46 PM, Tom Tromey <tom@tromey.com> wrote:
> 
> Hi.  Python 2 officially reached its sunset date this year:
> 
>    https://www.python.org/doc/sunset-python-2/
> 
> I think gdb should follow suit starting in gdb 11 -- that is, my
> proposal is to keep Python 2 support for the upcoming gdb 10 release,
> but remove said support from git master.
> 
> In the short term this would amount to some relatively minor code
> cleanups.

That makes sense to me.

> In the longer term, my preferred approach to handling inferior control
> from Python is to base it on async/await, which IIUC is only available
> in Python 3.  So, this would be an enabling step.

"await" appears in Python 3.5.  Do you want to drop support for 3.4 and earlier?  I guess they are no longer supported so that is arguably a valid thing to do, but I wonder how many packaged OS releases still come with versions that old.

	paul

Re[2]: Proposal to remove Python 2 support

[fedor_qd]

That has no sense for me because Python 2 doesn't disappear from distros.
--
Отправлено из myMail для Android вторник, 15 сентября 2020г., 23:59 +03:00 от Paul Koning via Gdb  gdb@sourceware.org :

>
>
> 
>On Sep 15, 2020, at 4:46 PM, Tom Tromey < tom@tromey.com> wrote:
>
> Hi.  Python 2 officially reached its sunset date this year:
>
> https://www.python.org/doc/sunset-python-2/
>
> I think gdb should follow suit starting in gdb 11 -- that is, my
> proposal is to keep Python 2 support for the upcoming gdb 10 release,
> but remove said support from git master.
>
> In the short term this would amount to some relatively minor code
> cleanups.
>
>That makes sense to me.
>
> In the longer term, my preferred approach to handling inferior control
> from Python is to base it on async/await, which IIUC is only available
> in Python 3.  So, this would be an enabling step.
>
>"await" appears in Python 3.5.  Do you want to drop support for 3.4 and earlier?  I guess they are no longer supported so that is arguably a valid thing to do, but I wonder how many packaged OS releases still come with versions that old.
>
>paul
>

Re: Proposal to remove Python 2 support

[Joel Brobecker]

> That has no sense for me because Python 2 doesn't disappear from distros.

I don't know how long this is going to be True, but "distros" have
been shipping Python 3.x for quite a while now, so lack of Python 2
support wouldn't prevent you from building GDB with Python support
enabled.

-- 
Joel

Re: Proposal to remove Python 2 support

[Andrew Burgess]

* Joel Brobecker <brobecker@adacore.com> [2020-09-16 06:00:13 -0700]:

> > That has no sense for me because Python 2 doesn't disappear from distros.
> 
> I don't know how long this is going to be True, but "distros" have
> been shipping Python 3.x for quite a while now, so lack of Python 2
> support wouldn't prevent you from building GDB with Python support
> enabled.

But a user might potentially have a significant body of Python 2 code
that they run through GDB, so it's not as simple as "just" building
with Python 3 and off they go.

I think once distros start to drop Python 2 then there's a really good
argument that GDB should drop Python 2 also, but before then we need
to go in with our eyes open, if the distros still ship Python 2 then
users might be annoyed if they have to rewrite their scripts.

Thanks,
Andrew

Re: Proposal to remove Python 2 support

[Paul Koning]

> On Sep 16, 2020, at 9:53 AM, Andrew Burgess <andrew.burgess@embecosm.com> wrote:
> 
> * Joel Brobecker <brobecker@adacore.com> [2020-09-16 06:00:13 -0700]:
> 
>>> That has no sense for me because Python 2 doesn't disappear from distros.
>> 
>> I don't know how long this is going to be True, but "distros" have
>> been shipping Python 3.x for quite a while now, so lack of Python 2
>> support wouldn't prevent you from building GDB with Python support
>> enabled.
> 
> But a user might potentially have a significant body of Python 2 code
> that they run through GDB, so it's not as simple as "just" building
> with Python 3 and off they go.

An interesting experiment would be to take some significant Python 2 based GDB scripts and see (a) if they run unchanged with Python 3, (b) if they run correctly if processed through the 2to3 tool.

For many situations it's easy to write a bilingual Python script, and in scripts that don't use "print" statements the differences are often tiny or non-existent.

	paul

Re: Proposal to remove Python 2 support

[Joel Brobecker]

> > I don't know how long this is going to be True, but "distros" have
> > been shipping Python 3.x for quite a while now, so lack of Python 2
> > support wouldn't prevent you from building GDB with Python support
> > enabled.
> 
> But a user might potentially have a significant body of Python 2 code
> that they run through GDB, so it's not as simple as "just" building
> with Python 3 and off they go.

That is true, and I understand that. I have been involved in quite
a few transitions from Python 2 to Python 3, and it's been a lot
of work for a number of projects.

That being said, we need to be careful and try to find the right
balance between the users who are still stuck on Python 2, those
who have made the effort to transition in time, and the cost of
keeping Python 2 support for a little longer. If the cost remains
reasonable, then of course, let's help everyone.

Here, what we are talking about, is the fact that keeping Python 2
support would delay the implementation of a feature (or force us
to implement it in a way that's more complex -- something I don't
think Tom is interested in doing).

The question therefore is: Do we wait a little more to see what
entities outside our community decide to do re: Python 2.x support,
at the cost of delaying some features and improvements?

I don't have a strong opinion on this. My feeling is that the end
of Python 2.x is something we have known about for a very long time,
and those still stuck on it have a workaround: They can continue
using older versions of GDB.

> I think once distros start to drop Python 2 then there's a really good
> argument that GDB should drop Python 2 also, but before then we need
> to go in with our eyes open, if the distros still ship Python 2 then
> users might be annoyed if they have to rewrite their scripts.

-- 
Joel

Re: Proposal to remove Python 2 support

[Andrew Burgess]

* Joel Brobecker <brobecker@adacore.com> [2020-09-16 08:23:29 -0700]:

> > > I don't know how long this is going to be True, but "distros" have
> > > been shipping Python 3.x for quite a while now, so lack of Python 2
> > > support wouldn't prevent you from building GDB with Python support
> > > enabled.
> > 
> > But a user might potentially have a significant body of Python 2 code
> > that they run through GDB, so it's not as simple as "just" building
> > with Python 3 and off they go.
> 
> That is true, and I understand that. I have been involved in quite
> a few transitions from Python 2 to Python 3, and it's been a lot
> of work for a number of projects.
> 
> That being said, we need to be careful and try to find the right
> balance between the users who are still stuck on Python 2, those
> who have made the effort to transition in time, and the cost of
> keeping Python 2 support for a little longer. If the cost remains
> reasonable, then of course, let's help everyone.
> 
> Here, what we are talking about, is the fact that keeping Python 2
> support would delay the implementation of a feature (or force us
> to implement it in a way that's more complex -- something I don't
> think Tom is interested in doing).

Something that I think might be related here, or at least worth
mentioning, is guile support.

Without knowing (a) what the new feature is, or (b) how Tom plans to
implement...

Can the feature be added to guile in the same super clean / clever
way?  If not are we taking the position that this feature just
wouldn't be available on guile?  Or would we potentially end up
implementing the feature the "other way" anyway?

Additionally there's always the option of supporting a particular
feature only when the user compiles against a particular Python
version.  I guess this would depend on how tightly coupled the feature
ends up being to the rest of the GDB/Python code.

Thanks,
Andrew

Re: Proposal to remove Python 2 support

[Eli Zaretskii]

> Date: Wed, 16 Sep 2020 08:23:29 -0700
> From: Joel Brobecker <brobecker@adacore.com>
> Cc: Paul Koning <paulkoning@comcast.net>, Tom Tromey <tom@tromey.com>,
>  gdb@sourceware.org
> 
> Here, what we are talking about, is the fact that keeping Python 2
> support would delay the implementation of a feature (or force us
> to implement it in a way that's more complex -- something I don't
> think Tom is interested in doing).

That shouldn't be a problem: it's perfectly fine to say that some GDB
feature requires Python 3.x.  We already have something similar with
Guile, I believe.

> My feeling is that the end of Python 2.x is something we have known
> about for a very long time, and those still stuck on it have a
> workaround: They can continue using older versions of GDB.

That is not always possible, though.  One may need a GDB feature that
was introduced in a newer version, but still have good reason to be
"stuck" with old Python versions.

Re: Proposal to remove Python 2 support

[André Pönitz]

On Wed, Sep 16, 2020 at 02:53:12PM +0100, Andrew Burgess wrote:
> * Joel Brobecker <brobecker@adacore.com> [2020-09-16 06:00:13 -0700]:
> 
> > > That has no sense for me because Python 2 doesn't disappear from distros.
> > 
> > I don't know how long this is going to be True, but "distros" have
> > been shipping Python 3.x for quite a while now, so lack of Python 2
> > support wouldn't prevent you from building GDB with Python support
> > enabled.
> 
> But a user might potentially have a significant body of Python 2 code
> that they run through GDB, so it's not as simple as "just" building
> with Python 3 and off they go.
>
> I think once distros start to drop Python 2 then there's a really good
> argument that GDB should drop Python 2 also, but before then we need
> to go in with our eyes open, if the distros still ship Python 2 then
> users might be annoyed if they have to rewrite their scripts.

I am not sure whether this is significant, but I have a data point here:

I have ~13kLOC of Python code to be used with GDB that I currently need
to keep compatible with Python 2 and 3, as this may be used with any GDB.

Dropping Python 2 would certainly make /my/ life simpler.

Regards,
André

Re: Proposal to remove Python 2 support

[Tom Tromey]

Andrew> I think once distros start to drop Python 2 then there's a really good
Andrew> argument that GDB should drop Python 2 also, but before then we need
Andrew> to go in with our eyes open, if the distros still ship Python 2 then
Andrew> users might be annoyed if they have to rewrite their scripts.

At least Ubuntu and Fedora ship a gdb that is linked against Python 3.
I didn't check the dates, but Joel said Ubuntu has done this since 18.04
(2.5 years ago).

Tom

Re: Proposal to remove Python 2 support

[Tom Tromey]

>>>>> "Andrew" == Andrew Burgess <andrew.burgess@embecosm.com> writes:

Andrew> Something that I think might be related here, or at least worth
Andrew> mentioning, is guile support.

Andrew> Without knowing (a) what the new feature is, or (b) how Tom plans to
Andrew> implement...

Andrew> Can the feature be added to guile in the same super clean / clever
Andrew> way?  If not are we taking the position that this feature just
Andrew> wouldn't be available on guile?  Or would we potentially end up
Andrew> implementing the feature the "other way" anyway?

Normally features are added to the Python or Guile layer based on
developer interest.  So, they aren't really at parity in general.

I don't really know if Guile has anything that Python is missing.  Maybe
something to do with I/O -- I'm not super familiar with it.

But IIRC Guile doesn't have frame filters, unwinder support, or TUI
windows.  Maybe more is missing there as well.

I don't know if inferior control can be added to the Guile layer easily,
but I would assume so, because Scheme generally has better support for
continuation-like things than other languages.

Andrew> Additionally there's always the option of supporting a particular
Andrew> feature only when the user compiles against a particular Python
Andrew> version.  I guess this would depend on how tightly coupled the feature
Andrew> ends up being to the rest of the GDB/Python code.

I don't really like to do this kind of thing because the user experience
is bad -- gdb might or might not work.  Though of course we're already
in that situation since the entire scripting layer is optional.

Tom

Re: Proposal to remove Python 2 support

[Joel Brobecker]

> Andrew> Additionally there's always the option of supporting a particular
> Andrew> feature only when the user compiles against a particular Python
> Andrew> version.  I guess this would depend on how tightly coupled the feature
> Andrew> ends up being to the rest of the GDB/Python code.
> 
> I don't really like to do this kind of thing because the user experience
> is bad -- gdb might or might not work.  Though of course we're already
> in that situation since the entire scripting layer is optional.

I agree we should be very careful about that, and keep things clear
for the users. One the one hand, a full on/off switch, with a clear
message "your GDB was configured without Python support" is simple
for users to understand. On the other hand, having the API itself
depend on what version of Python GDB was built with would make things
pretty confusing, in my opinion.

-- 
Joel

Re: Proposal to remove Python 2 support

[Eli Zaretskii]

> Date: Thu, 17 Sep 2020 10:49:40 -0700
> From: Joel Brobecker <brobecker@adacore.com>
> Cc: Paul Koning <paulkoning@comcast.net>, gdb@sourceware.org
> 
> I agree we should be very careful about that, and keep things clear
> for the users. One the one hand, a full on/off switch, with a clear
> message "your GDB was configured without Python support" is simple
> for users to understand. On the other hand, having the API itself
> depend on what version of Python GDB was built with would make things
> pretty confusing, in my opinion.

But that kind of thing is inevitable when one relies on external
libraries for some of our features.  For example, suppose that the
source-highlight package learns to highlight sources better -- these
improvements will only available to users if they upgrade their
installed source-highlight library before building GDB.  Granted,
these differences are smaller than entire commands or features
missing, but they still do exist, and always will.

So I don't think we should be too worried about this.

Re: Proposal to remove Python 2 support

[Jan Kratochvil]

On Thu, 17 Sep 2020 20:03:02 +0200, Eli Zaretskii wrote:
> But that kind of thing is inevitable when one relies on external
> libraries for some of our features.  For example, suppose that the
> source-highlight package learns to highlight sources better -- these
> improvements will only available to users if they upgrade their
> installed source-highlight library before building GDB.

That is different.

There will always be installed only one version of source-highlight package.

With Python there can be (and currently commonly is) installed both Python
2 & 3 and it is sometimes tricky to convince a package to use the other
version than the one it uses by default.


Jan

Re: Proposal to remove Python 2 support

[Joel Brobecker]

> > I agree we should be very careful about that, and keep things clear
> > for the users. One the one hand, a full on/off switch, with a clear
> > message "your GDB was configured without Python support" is simple
> > for users to understand. On the other hand, having the API itself
> > depend on what version of Python GDB was built with would make things
> > pretty confusing, in my opinion.
> 
> But that kind of thing is inevitable when one relies on external
> libraries for some of our features.  For example, suppose that the
> source-highlight package learns to highlight sources better -- these
> improvements will only available to users if they upgrade their
> installed source-highlight library before building GDB.  Granted,
> these differences are smaller than entire commands or features
> missing, but they still do exist, and always will.
> 
> So I don't think we should be too worried about this.

I do not agree with this sentiment in general, but this is making me
realize that it's not so easy to categorize things. We can have
guidelines, but each probably deserves to be considered individually.

Taking Tom's feature proposal, however, I feel it would be a mistake
to have it only conditionally based on the version of Python being
used to build GDB.

-- 
Joel

Re: Proposal to remove Python 2 support

[Eli Zaretskii]

> Date: Thu, 17 Sep 2020 20:10:05 +0200
> From: Jan Kratochvil <jan.kratochvil@redhat.com>
> Cc: Joel Brobecker <brobecker@adacore.com>, paulkoning@comcast.net,
> 	tom@tromey.com, gdb@sourceware.org
> 
> On Thu, 17 Sep 2020 20:03:02 +0200, Eli Zaretskii wrote:
> > But that kind of thing is inevitable when one relies on external
> > libraries for some of our features.  For example, suppose that the
> > source-highlight package learns to highlight sources better -- these
> > improvements will only available to users if they upgrade their
> > installed source-highlight library before building GDB.
> 
> That is different.
> 
> There will always be installed only one version of source-highlight package.
> 
> With Python there can be (and currently commonly is) installed both Python
> 2 & 3 and it is sometimes tricky to convince a package to use the other
> version than the one it uses by default.

Sorry, I don't think I understand how what you say is relevant to what
I wrote.

What I tried to convey is that having some features in GDB that only
work if you build GDB with Python 3 is not an acute problem.

Re: Proposal to remove Python 2 support

[vincent Dupaquis]

Some vendors are still delivering gdb with python2 by default, for
instance ARM in their arm-none-eabi delivery
(https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm/downloads).

Le 17/09/2020 à 20:45, Eli Zaretskii a écrit :
>> Date: Thu, 17 Sep 2020 20:10:05 +0200
>> From: Jan Kratochvil <jan.kratochvil@redhat.com>
>> Cc: Joel Brobecker <brobecker@adacore.com>, paulkoning@comcast.net,
>> 	tom@tromey.com, gdb@sourceware.org
>>
>> On Thu, 17 Sep 2020 20:03:02 +0200, Eli Zaretskii wrote:
>>> But that kind of thing is inevitable when one relies on external
>>> libraries for some of our features.  For example, suppose that the
>>> source-highlight package learns to highlight sources better -- these
>>> improvements will only available to users if they upgrade their
>>> installed source-highlight library before building GDB.
>> That is different.
>>
>> There will always be installed only one version of source-highlight package.
>>
>> With Python there can be (and currently commonly is) installed both Python
>> 2 & 3 and it is sometimes tricky to convince a package to use the other
>> version than the one it uses by default.
> Sorry, I don't think I understand how what you say is relevant to what
> I wrote.
>
> What I tried to convey is that having some features in GDB that only
> work if you build GDB with Python 3 is not an acute problem.
-- 

*Vincent Dupaquis*
Software security & Cryptography expert
06 24 58 17 05
/Europarc de Pichaury Bâtiment B8 1330 rue Guillibert de la Lauzière
13290 Aix-en-Provence/

www.trusted-objects.com <http://www.trusted-objects.com>

Re: Proposal to remove Python 2 support

[Eli Zaretskii]

> From: vincent Dupaquis <v.dupaquis@trusted-objects.com>
> Date: Fri, 18 Sep 2020 09:15:35 +0200
> 
> Some vendors are still delivering gdb with python2 by default, for
> instance ARM in their arm-none-eabi delivery
> (https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm/downloads).

To clarify, I didn't mean to say that we should decide not to have new
features supported for Python 2.  What I wrote was written in response
to an opinion that keeping Python 2 supported might induce some heavy
development and maintenance costs.  I then said that we could decide
not to support some of the newer features for Python 2, because doing
that would be better than dropping Python 2 support completely.

FTR, I'm the last person to be happy with dropping Python 2 support: I
have it installed, and I'm building GDB with Python 2.  I even
volunteered to be the tester of Python 2.6 usage in GDB, because that
is even older.

Re: Proposal to remove Python 2 support

[Pedro Alves]

On 9/15/20 9:46 PM, Tom Tromey wrote:
> Hi.  Python 2 officially reached its sunset date this year:
> 
>     https://www.python.org/doc/sunset-python-2/
> 
> I think gdb should follow suit starting in gdb 11 -- that is, my
> proposal is to keep Python 2 support for the upcoming gdb 10 release,
> but remove said support from git master.
> 
> In the short term this would amount to some relatively minor code
> cleanups.
> 
> In the longer term, my preferred approach to handling inferior control
> from Python is to base it on async/await, which IIUC is only available
> in Python 3.  So, this would be an enabling step.

Dropping Python 2 support is going to be an issue for us, given we will
still have a couple years more of DTS releases for RHEL 7, which is
Python 2.7 based.

If possible, we would prefer if Python 2 support was kept around a
while longer.

Thanks,
Pedro Alves

RE: Proposal to remove Python 2 support

[Mikhail.Terekhov]

> -----Original Message-----
> From: Gdb <gdb-bounces@sourceware.org> On Behalf Of Pedro Alves via Gdb
> Sent: Friday, September 18, 2020 11:59 AM
> To: Tom Tromey; 
> Subject: Re: Proposal to remove Python 2 support
>
> On 9/15/20 9:46 PM, Tom Tromey wrote:
> > Hi.  Python 2 officially reached its sunset date this year:
> >
> >     https://www.python.org/doc/sunset-python-2/
> > 
> > I think gdb should follow suit starting in gdb 11 -- that is, my 
> > proposal is to keep Python 2 support for the upcoming gdb 10 release, 
> > but remove said support from git master.
> > 
> > In the short term this would amount to some relatively minor code 
> > cleanups.
> > 
> > In the longer term, my preferred approach to handling inferior control 
> > from Python is to base it on async/await, which IIUC is only available 
> > in Python 3.  So, this would be an enabling step.
>
> Dropping Python 2 support is going to be an issue for us, given we will still have a couple years more of DTS releases for RHEL 7, which is Python 2.7 based.
>
> If possible, we would prefer if Python 2 support was kept around a while longer.

RHEL 7 comes with Python 3.6 as well.

Regards,
Mikhail

Re: Proposal to remove Python 2 support

[Tom Tromey]

Pedro> Dropping Python 2 support is going to be an issue for us, given
Pedro> we will still have a couple years more of DTS releases for RHEL
Pedro> 7, which is Python 2.7 based.

>> RHEL 7 comes with Python 3.6 as well.

I think the issue must be that gdb is built against Python 2 on RHEL 7.

Does anyone know if there are other distros with LTS releases that do
this?  I guess it makes sense to hold off for those; at least major
distros anyway.

Tom

RE: Proposal to remove Python 2 support

[Mikhail.Terekhov]

> From: Gdb <gdb-bounces@sourceware.org> On Behalf Of Tom Tromey
>
> Pedro> Dropping Python 2 support is going to be an issue for us, given 
> Pedro> we will still have a couple years more of DTS releases for RHEL 
> Pedro> 7, which is Python 2.7 based.
>
> >> RHEL 7 comes with Python 3.6 as well.
>
> I think the issue must be that gdb is built against Python 2 on RHEL 7.

If they would want to update to the new major version of gdb they would build it with Python3. What is the problem?

Re: Proposal to remove Python 2 support

[Pedro Alves]

On 9/18/20 6:13 PM, Mikhail.Terekhov--- via Gdb wrote:
>> From: Gdb <gdb-bounces@sourceware.org> On Behalf Of Tom Tromey
>>
>> Pedro> Dropping Python 2 support is going to be an issue for us, given 
>> Pedro> we will still have a couple years more of DTS releases for RHEL 
>> Pedro> 7, which is Python 2.7 based.
>>
>>>> RHEL 7 comes with Python 3.6 as well.
>>
>> I think the issue must be that gdb is built against Python 2 on RHEL 7.
> 
> If they would want to update to the new major version of gdb they would build it with Python3. What is the problem?

It would break compatibility with pretty printers, user scripts, and
require Python3 versions of Python libraries scripts may load.

Pedro Alves

Re: Proposal to remove Python 2 support

[Jeffrey Walton]

On Fri, Sep 18, 2020 at 12:56 PM Tom Tromey <tom@tromey.com> wrote:
>
> Pedro> Dropping Python 2 support is going to be an issue for us, given
> Pedro> we will still have a couple years more of DTS releases for RHEL
> Pedro> 7, which is Python 2.7 based.
>
> >> RHEL 7 comes with Python 3.6 as well.
>
> I think the issue must be that gdb is built against Python 2 on RHEL 7.
>
> Does anyone know if there are other distros with LTS releases that do
> this?  I guess it makes sense to hold off for those; at least major
> distros anyway.

Ubuntu 16.04 is LTS, and it is also ESM. ESM is Extended Security
Maintenance. The ESM means it will probably be around for 2 or 3 years
beyond the 5 years provided by LTS (i.e., 2023 or so).

It looks like Ubuntu 16.04 provides both Python 2.7 and 3.5.
https://packages.ubuntu.com/xenial/python/.

It looks like Ubuntu 16's GDB is built against Python 3.5.
https://packages.ubuntu.com/xenial/devel/gdb.

Jeff