[PATCH] Convert all Python scripts to Python 3

[PATCH] Convert all Python scripts to Python 3

[Alistair Francis]

Convert all of the Python scripts in tree to explicity use Python 3.

I have not explicity tested all of the scripts. Makefiles generally
explicitly use $(PYTHON) so the scripts should already support Python3.
---
 benchtests/scripts/bench.py             | 2 +-
 benchtests/scripts/compare_bench.py     | 2 +-
 benchtests/scripts/compare_strings.py   | 2 +-
 benchtests/scripts/import_bench.py      | 2 +-
 benchtests/scripts/validate_benchout.py | 2 +-
 conform/glibcconform.py                 | 2 +-
 conform/linknamespace.py                | 2 +-
 conform/list-header-symbols.py          | 2 +-
 math/gen-libm-test.py                   | 2 +-
 math/gen-tgmath-tests.py                | 2 +-
 10 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/benchtests/scripts/bench.py b/benchtests/scripts/bench.py
index aeec5ccd23..a54ec6f7e3 100755
--- a/benchtests/scripts/bench.py
+++ b/benchtests/scripts/bench.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 # Copyright (C) 2014-2020 Free Software Foundation, Inc.
 # This file is part of the GNU C Library.
 #
diff --git a/benchtests/scripts/compare_bench.py b/benchtests/scripts/compare_bench.py
index a215ebc9bd..f174161ed3 100755
--- a/benchtests/scripts/compare_bench.py
+++ b/benchtests/scripts/compare_bench.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 # Copyright (C) 2015-2020 Free Software Foundation, Inc.
 # This file is part of the GNU C Library.
 #
diff --git a/benchtests/scripts/compare_strings.py b/benchtests/scripts/compare_strings.py
index 2b2edbf10d..06f7c6e1c8 100755
--- a/benchtests/scripts/compare_strings.py
+++ b/benchtests/scripts/compare_strings.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 # Copyright (C) 2017-2020 Free Software Foundation, Inc.
 # This file is part of the GNU C Library.
 #
diff --git a/benchtests/scripts/import_bench.py b/benchtests/scripts/import_bench.py
index 08681aafdc..2264da0037 100644
--- a/benchtests/scripts/import_bench.py
+++ b/benchtests/scripts/import_bench.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 # Copyright (C) 2015-2020 Free Software Foundation, Inc.
 # This file is part of the GNU C Library.
 #
diff --git a/benchtests/scripts/validate_benchout.py b/benchtests/scripts/validate_benchout.py
index 715218aff8..6b4877a67b 100755
--- a/benchtests/scripts/validate_benchout.py
+++ b/benchtests/scripts/validate_benchout.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 # Copyright (C) 2014-2020 Free Software Foundation, Inc.
 # This file is part of the GNU C Library.
 #
diff --git a/conform/glibcconform.py b/conform/glibcconform.py
index 6075745574..1440dedc83 100644
--- a/conform/glibcconform.py
+++ b/conform/glibcconform.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 # Shared code for glibc conformance tests.
 # Copyright (C) 2018-2020 Free Software Foundation, Inc.
 # This file is part of the GNU C Library.
diff --git a/conform/linknamespace.py b/conform/linknamespace.py
index 87cd17b1ce..1d27e4cfba 100644
--- a/conform/linknamespace.py
+++ b/conform/linknamespace.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 # Check that use of symbols declared in a given header does not result
 # in any symbols being brought in that are not reserved with external
 # linkage for the given standard.
diff --git a/conform/list-header-symbols.py b/conform/list-header-symbols.py
index e43c12ec40..bfa463a303 100644
--- a/conform/list-header-symbols.py
+++ b/conform/list-header-symbols.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 # Print a list of symbols exported by some headers that would
 # otherwise be in the user's namespace.
 # Copyright (C) 2018-2020 Free Software Foundation, Inc.
diff --git a/math/gen-libm-test.py b/math/gen-libm-test.py
index ec263397d8..0142c0f332 100755
--- a/math/gen-libm-test.py
+++ b/math/gen-libm-test.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 # Generate tests for libm functions.
 # Copyright (C) 2018-2020 Free Software Foundation, Inc.
 # This file is part of the GNU C Library.
diff --git a/math/gen-tgmath-tests.py b/math/gen-tgmath-tests.py
index ef207dd852..c225b64db1 100755
--- a/math/gen-tgmath-tests.py
+++ b/math/gen-tgmath-tests.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 # Generate tests for <tgmath.h> macros.
 # Copyright (C) 2017-2020 Free Software Foundation, Inc.
 # This file is part of the GNU C Library.
-- 
2.25.0

Re: [PATCH] Convert all Python scripts to Python 3

[Zack Weinberg]

On Fri, Jan 31, 2020 at 12:10 PM Alistair Francis
<alistair.francis@wdc.com> wrote:>
> Convert all of the Python scripts in tree to explicity use Python 3.

More accurate to say "Change all of the #! lines on Python scripts in
tree to reference /usr/bin/python3."

> I have not explicity tested all of the scripts. Makefiles generally
> explicitly use $(PYTHON) so the scripts should already support Python3.

I think this change needs manual explicit testing that all of the
scripts do in fact work with Python 3 (and not just any Python 3, but
specifically version 3.4, since that's the oldest version that
install.texi says should work).  We should also verify that the
autoconf logic that sets $(PYTHON) will bomb out if the only available
version of Python is 2.x. (It looks to me like it _should_ do that,
but I want it tested in an environment with no trace of a Python 3
installation.)  And the alternative of using bare "python" for
$(PYTHON) should be removed at the same time as the #! lines are
changed.

zw

Re: [PATCH] Convert all Python scripts to Python 3

[Alistair Francis]

On Fri, Jan 31, 2020 at 9:29 AM Zack Weinberg <zackw@panix.com> wrote:
>
> On Fri, Jan 31, 2020 at 12:10 PM Alistair Francis
> <alistair.francis@wdc.com> wrote:>
> > Convert all of the Python scripts in tree to explicity use Python 3.
>
> More accurate to say "Change all of the #! lines on Python scripts in
> tree to reference /usr/bin/python3."
>
> > I have not explicity tested all of the scripts. Makefiles generally
> > explicitly use $(PYTHON) so the scripts should already support Python3.
>
> I think this change needs manual explicit testing that all of the
> scripts do in fact work with Python 3 (and not just any Python 3, but
> specifically version 3.4, since that's the oldest version that

I don't have a way to test all of the scripts, especially with Python 3.4.

My original patch just updated math/gen-libm-test.py as I have tested
that with Python 3 (although not 3.4). I don't know if I will be able
to explicitly test all of the scripts with Python3 (definitely not
with 3.4, where do you even get that?). The painful part with Python
testing is you actually need to run the code, not just run the script
once.

With Python2 EOL and the assumption that everyone is already using
Python3 anyway (and calling this scripts with $(PYTHON)) I was hoping
this would be enough.

> install.texi says should work).  We should also verify that the
> autoconf logic that sets $(PYTHON) will bomb out if the only available
> version of Python is 2.x. (It looks to me like it _should_ do that,
> but I want it tested in an environment with no trace of a Python 3
> installation.)  And the alternative of using bare "python" for
> $(PYTHON) should be removed at the same time as the #! lines are
> changed.

I also have no way to test a setup with Python 2.

On Arch "python" is Python3, so I don't think we should error if
"python" is set.

Alistair

>
> zw

Re: [PATCH] Convert all Python scripts to Python 3

[Zack Weinberg]

On Fri, Jan 31, 2020 at 6:51 PM Alistair Francis <alistair23@gmail.com> wrote:
> On Fri, Jan 31, 2020 at 9:29 AM Zack Weinberg <zackw@panix.com> wrote:
> > I think this change needs manual explicit testing that all of the
> > scripts do in fact work with Python 3 (and not just any Python 3, but
> > specifically version 3.4, since that's the oldest version that
>
> I don't have a way to test all of the scripts, especially with Python 3.4.
>
> My original patch just updated math/gen-libm-test.py as I have tested
> that with Python 3 (although not 3.4). I don't know if I will be able
> to explicitly test all of the scripts with Python3 (definitely not
> with 3.4, where do you even get that?). The painful part with Python
> testing is you actually need to run the code, not just run the script
> once.
>
> With Python2 EOL and the assumption that everyone is already using
> Python3 anyway (and calling this scripts with $(PYTHON)) I was hoping
> this would be enough.

I may have misunderstood the situation.  I thought we were allowing
use of either Python 2 or 3 for these scripts, but on further
investigation, it looks more like we were already requiring them to be
run with v3.  If so, your proposed change does not need testing,
because they must already work with v3.  Joseph, you've been
spearheading a lot of the use of Python - can you confirm that Python
3 is already being used for all these scripts?

> I also have no way to test a setup with Python 2.

The simplest way to test with a Python 2-only environment (or a
3.4-only environment) would probably be to use 'schroot'
(https://wiki.debian.org/Schroot) together with appropriately old
versions of Debian.  I don't know off the top of my head what those
versions are.

> On Arch "python" is Python3

That is a bug in Arch.  The bare command name "python" and the #! path
/usr/bin/python need to be permanently reserved for Python 2.  As long
as "python3" is also available, we don't actually care, but we should
actively refuse to support an environment where there is no "python3"
command.

zw

Re: [PATCH] Convert all Python scripts to Python 3

[Alistair Francis]

On Tue, Feb 4, 2020 at 10:48 AM Zack Weinberg <zackw@panix.com> wrote:
>
> On Fri, Jan 31, 2020 at 6:51 PM Alistair Francis <alistair23@gmail.com> wrote:
> > On Fri, Jan 31, 2020 at 9:29 AM Zack Weinberg <zackw@panix.com> wrote:
> > > I think this change needs manual explicit testing that all of the
> > > scripts do in fact work with Python 3 (and not just any Python 3, but
> > > specifically version 3.4, since that's the oldest version that
> >
> > I don't have a way to test all of the scripts, especially with Python 3.4.
> >
> > My original patch just updated math/gen-libm-test.py as I have tested
> > that with Python 3 (although not 3.4). I don't know if I will be able
> > to explicitly test all of the scripts with Python3 (definitely not
> > with 3.4, where do you even get that?). The painful part with Python
> > testing is you actually need to run the code, not just run the script
> > once.
> >
> > With Python2 EOL and the assumption that everyone is already using
> > Python3 anyway (and calling this scripts with $(PYTHON)) I was hoping
> > this would be enough.
>
> I may have misunderstood the situation.  I thought we were allowing
> use of either Python 2 or 3 for these scripts, but on further
> investigation, it looks more like we were already requiring them to be
> run with v3.  If so, your proposed change does not need testing,
> because they must already work with v3.  Joseph, you've been
> spearheading a lot of the use of Python - can you confirm that Python
> 3 is already being used for all these scripts?

That was my understanding as well.

>
> > I also have no way to test a setup with Python 2.
>
> The simplest way to test with a Python 2-only environment (or a
> 3.4-only environment) would probably be to use 'schroot'
> (https://wiki.debian.org/Schroot) together with appropriately old
> versions of Debian.  I don't know off the top of my head what those
> versions are.
>
> > On Arch "python" is Python3
>
> That is a bug in Arch.  The bare command name "python" and the #! path
> /usr/bin/python need to be permanently reserved for Python 2.  As long
> as "python3" is also available, we don't actually care, but we should
> actively refuse to support an environment where there is no "python3"
> command.

Ah ok. I agree that we can throw and error if there is no "python3".

Alistair

>
> zw

Re: [PATCH] Convert all Python scripts to Python 3

[Joseph Myers]

On Tue, 4 Feb 2020, Zack Weinberg wrote:

> I may have misunderstood the situation.  I thought we were allowing
> use of either Python 2 or 3 for these scripts, but on further
> investigation, it looks more like we were already requiring them to be
> run with v3.  If so, your proposed change does not need testing,
> because they must already work with v3.  Joseph, you've been
> spearheading a lot of the use of Python - can you confirm that Python
> 3 is already being used for all these scripts?

All the Python scripts run from the makefiles are run with Python 3.

Of the others, I confirmed when I wrote math/gen-tgmath-tests.py that it 
worked with both Python 2 and Python 3; I haven't tried those benchtests/ 
scripts not run from the makefiles.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [PATCH] Convert all Python scripts to Python 3

[Zack Weinberg]

On Thu, Feb 6, 2020 at 4:20 PM Joseph Myers <joseph@codesourcery.com> wrote:
> On Tue, 4 Feb 2020, Zack Weinberg wrote:
> > Joseph, you've been
> > spearheading a lot of the use of Python - can you confirm that Python
> > 3 is already being used for all these scripts?
>
> All the Python scripts run from the makefiles are run with Python 3.
>
> Of the others, I confirmed when I wrote math/gen-tgmath-tests.py that it
> worked with both Python 2 and Python 3; I haven't tried those benchtests/
> scripts not run from the makefiles.

OK, so we need to leave the manually-run benchtests/ scripts alone
until someone can confirm that they work with Python 3.

zw

Re: [PATCH] Convert all Python scripts to Python 3

[Alistair Francis]

On Thu, Feb 6, 2020 at 2:07 PM Zack Weinberg <zackw@panix.com> wrote:
>
> On Thu, Feb 6, 2020 at 4:20 PM Joseph Myers <joseph@codesourcery.com> wrote:
> > On Tue, 4 Feb 2020, Zack Weinberg wrote:
> > > Joseph, you've been
> > > spearheading a lot of the use of Python - can you confirm that Python
> > > 3 is already being used for all these scripts?
> >
> > All the Python scripts run from the makefiles are run with Python 3.
> >
> > Of the others, I confirmed when I wrote math/gen-tgmath-tests.py that it
> > worked with both Python 2 and Python 3; I haven't tried those benchtests/
> > scripts not run from the makefiles.
>
> OK, so we need to leave the manually-run benchtests/ scripts alone
> until someone can confirm that they work with Python 3.

I just sent a v2.

Alistair

>
> zw