python 2 check & cleaning

python 2 check & cleaning

[Marco Atzeri]

Hi All,

the current python test packages remove the link python->python2.7
and replace with an alternative appraouch.

Before making the switch, I would like to review the packages that could 
be broken, so I looked at all packages, not in "python*-*" form,
that pull one of the "python", "python2" and "python27" using

    $ cygcheck-dep -O -S -q -n python..

as base.

There are almost 200 binary packages
(of the almost 9000 currently present on 64 bit)

The list with source package and maintainer is here:

https://drive.google.com/file/d/1xaPzIZ78JzHtJXcdzB77qJoeNlvVQRJq/view?usp=sharing

I know that Mercurial need rebuild, and I assume the same
for subversion (that I plan to adopt).

Could you please check your packages if they will work with
preferred python3.8 ?

Texlive and fontforge seem the most urgent, IMHO, for a verification.

As python2 is officially dead, a fast switch is nice to have
without braking too many critical packages.

Thanks in advance & Regards
Marco

Re: python 2 check & cleaning

[Brian Inglis]

On 2021-01-17 22:33, Marco Atzeri via Cygwin-apps wrote:
> Could you please check your packages if they will work with
> preferred python3.8?

@ units
...
requires: cygwin findutils libreadline7 python3 python36 python36-requests
...
depends2: cygwin, findutils, libreadline7, python36, python36-requests
build-depends: autoconf, binutils, coreutils, cygport, cygwin-devel, gawk, 
gcc-core, grep, libncurses-devel, libreadline-devel, perl_base, python3, 
python3-devel, python3-requests, sed, texinfo

$ head -1 `which units_cur`
#!/usr/bin/python3.6m

How can I ensure units depends2 and requires, and the script hash-bang uses only 
the generic python3, python3-devel, python3-requests like build-depends, and 
avoids using the "current" specific version 36/3.6m?

And how can I ensure that units_cur is tested using only python38 modules?
Will the hash-bang override the command line run python38 `which units_cur`?

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]

Re: python 2 check & cleaning

[Marco Atzeri]

On 18.01.2021 08:21, Brian Inglis wrote:
> On 2021-01-17 22:33, Marco Atzeri via Cygwin-apps wrote:
>> Could you please check your packages if they will work with
>> preferred python3.8?
> 
> @ units
> ...
> requires: cygwin findutils libreadline7 python3 python36 python36-requests
> ...
> depends2: cygwin, findutils, libreadline7, python36, python36-requests
> build-depends: autoconf, binutils, coreutils, cygport, cygwin-devel, 
> gawk, gcc-core, grep, libncurses-devel, libreadline-devel, perl_base, 
> python3, python3-devel, python3-requests, sed, texinfo
> 
> $ head -1 `which units_cur`
> #!/usr/bin/python3.6m
> 
> How can I ensure units depends2 and requires, and the script hash-bang 
> uses only the generic python3, python3-devel, python3-requests like 
> build-depends, and avoids using the "current" specific version 36/3.6m?

Hi Brian,

python3-requests for the time being will pull python36-requests.

That is fine for your
   #!/usr/bin/python3.6m

Cygport has hard coded the OBSOLETES, this is something
we need to change in cygport:

 >>> python36-requests OBSOLETES: python3-requests

> And how can I ensure that units_cur is tested using only python38 modules?
> Will the hash-bang override the command line run python38 `which 
> units_cur`?
> 

rebuilding from your source I had

units requires: cygwin findutils libreadline7 python38 python38-requests

It seems the "inherit python3" is pulling the current choice
of my system

$ alternatives --display python3
python3 - status is auto.
  link currently points to /usr/bin/python3.8
/usr/bin/python3.7 - priority 37
/usr/bin/python3.6 - priority 36
/usr/bin/python3.8 - priority 38
Current `best' version is /usr/bin/python3.8.

Regards
Marco

Re: python 2 check & cleaning

[Brian Inglis]

On 2021-01-18 04:16, Marco Atzeri via Cygwin-apps wrote:
> On 18.01.2021 08:21, Brian Inglis wrote:
>> On 2021-01-17 22:33, Marco Atzeri via Cygwin-apps wrote:
>>> Could you please check your packages if they will work with
>>> preferred python3.8?
>>
>> @ units
>> ...
>> requires: cygwin findutils libreadline7 python3 python36 python36-requests
>> ...
>> depends2: cygwin, findutils, libreadline7, python36, python36-requests
>> build-depends: autoconf, binutils, coreutils, cygport, cygwin-devel, gawk, 
>> gcc-core, grep, libncurses-devel, libreadline-devel, perl_base, python3, 
>> python3-devel, python3-requests, sed, texinfo
>>
>> $ head -1 `which units_cur`
>> #!/usr/bin/python3.6m
>>
>> How can I ensure units depends2 and requires, and the script hash-bang uses 
>> only the generic python3, python3-devel, python3-requests like build-depends, 
>> and avoids using the "current" specific version 36/3.6m?

> python3-requests for the time being will pull python36-requests.
> 
> That is fine for your
>    #!/usr/bin/python3.6m
> 
> Cygport has hard coded the OBSOLETES, this is something
> we need to change in cygport:
> 
>  >>> python36-requests OBSOLETES: python3-requests

>> And how can I ensure that units_cur is tested using only python38 modules?
>> Will the hash-bang override the command line run python38 `which units_cur`?

> rebuilding from your source I had
> 
> units requires: cygwin findutils libreadline7 python38 python38-requests
> 
> It seems the "inherit python3" is pulling the current choice
> of my system
> 
> $ alternatives --display python3
> python3 - status is auto.
>   link currently points to /usr/bin/python3.8
> /usr/bin/python3.7 - priority 37
> /usr/bin/python3.6 - priority 36
> /usr/bin/python3.8 - priority 38
> Current `best' version is /usr/bin/python3.8.

Same on mine as of package release time.

I still have direct symlinks from installs:

$ llrt -go /bin/python{,[23]*}
-rwxr-xr-x  1 9.1K Apr 10  2020 /bin/python3.7m.exe*
lrwxrwxrwx  1   14 Apr 11  2020 /bin/python3.7 -> python3.7m.exe*
-rwxr-xr-x  1 9.1K May 23  2020 /bin/python2.7.exe*
-rwxr-xr-x  1 3.4K May 23  2020 /bin/python3.8-config*
-rwxr-xr-x+ 1 9.1K May 23  2020 /bin/python3.8.exe*
-rwxr-xr-x  1 3.4K Jun  8  2020 /bin/python3.6m-config*
-rwxr-xr-x  1 9.6K Jun  8  2020 /bin/python3.6m.exe*
lrwxrwxrwx  1   14 Jun 26  2020 /bin/python3.6 -> python3.6m.exe*
lrwxrwxrwx  1   13 Jun 26  2020 /bin/python -> python2.7.exe*
lrwxrwxrwx  1   13 Jun 26  2020 /bin/python2 -> python2.7.exe*
lrwxrwxrwx  1   17 Jun 26  2020 /bin/python3.6-config -> python3.6m-config*
lrwxrwxrwx  1   13 Jan  2 13:36 /bin/python3 -> python3.8.exe*
lrwxrwxrwx  1   16 Jan  2 13:39 /bin/python3-config -> python3.8-config*

Problem is if units goes quiet for a while, as it has sometimes for 2-3 years, 
averaging a release a year, it may require python3.X packages to stay around 
long after really required, even if obsoleted by many later releases, when it 
really just needs to depend on python3{,-requests}.
At the moment, python 3.6 has to stay around until the next units release!

I would rather not have to generate package releases per python release (or few) 
just for this, but I don't mind testing each.

Should I hack the build files next release to avoid resolving the symlinks and 
retain the base version?

[Units Version Release Dates

Version	  Released	Days
1.53    1997-01-14      183
1.54    1997-09-10      239
1.55    1999-07-30      688
1.74    2001-06-13      684
1.80    2002-06-17      369
1.85    2005-05-20     1068
1.86    2006-11-11      540
1.87    2007-09-26      319
1.88    2010-02-15      873
2.00    2012-06-28      864
2.01    2012-10-24      118
2.02    2013-07-11      260
2.10    2014-03-26      258
2.11    2014-04-02        7
2.12    2015-10-14      560
2.13    2016-06-21      251
2.14    2017-03-08      260
2.15    2017-10-23      229
2.16    2017-10-31        8
2.17    2018-06-25      237
2.18    2018-10-20      117
2.19    2019-05-31      223
2.20    2020-09-30      488
2.21    2020-11-15       46

    min 7  avg 370  max 1068]
-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]

Re: python 2 check & cleaning

[Ken Brown]

On 1/18/2021 12:33 AM, Marco Atzeri via Cygwin-apps wrote:
> Hi All,
> 
> the current python test packages remove the link python->python2.7
> and replace with an alternative appraouch.
> 
> Before making the switch, I would like to review the packages that could be 
> broken, so I looked at all packages, not in "python*-*" form,
> that pull one of the "python", "python2" and "python27" using
> 
>     $ cygcheck-dep -O -S -q -n python..
> 
> as base.
> 
> There are almost 200 binary packages
> (of the almost 9000 currently present on 64 bit)
> 
> The list with source package and maintainer is here:
> 
> https://drive.google.com/file/d/1xaPzIZ78JzHtJXcdzB77qJoeNlvVQRJq/view?usp=sharing
> 
> I know that Mercurial need rebuild, and I assume the same
> for subversion (that I plan to adopt).
> 
> Could you please check your packages if they will work with
> preferred python3.8 ?
> 
> Texlive and fontforge seem the most urgent, IMHO, for a verification.

I think TeX Live should be fine.  All of the python scripts provided by the 
texlive-collection-* packages are identical across platforms, so any python3 
incompatibilities would almost certainly have been reported and fixed by now.  I 
think all I have to do is rebuild the collections that contain python scripts to 
make sure I pick up the latest versions of the upstream packages, and also to 
let cygport update the dependencies.  I can do that quickly as soon as cygport 
is ready for the new system.

My other package listed in your spreadsheet, bzr-fastimport, is of very low 
priority.  It can be dealt with when and if bzr is updated.  Since there hasn't 
been an upstream bzr release in 5 years, updating bzr might simply be a matter 
of changing the shebang in /usr/bin/bzr to specify python2.  But that's up to Jari.

Ken

Re: python 2 check & cleaning

[Ken Brown]

On 1/18/2021 1:25 PM, Ken Brown via Cygwin-apps wrote:
> On 1/18/2021 12:33 AM, Marco Atzeri via Cygwin-apps wrote:
>> Hi All,
>>
>> the current python test packages remove the link python->python2.7
>> and replace with an alternative appraouch.
>>
>> Before making the switch, I would like to review the packages that could be 
>> broken, so I looked at all packages, not in "python*-*" form,
>> that pull one of the "python", "python2" and "python27" using
>>
>>     $ cygcheck-dep -O -S -q -n python..
>>
>> as base.
>>
>> There are almost 200 binary packages
>> (of the almost 9000 currently present on 64 bit)
>>
>> The list with source package and maintainer is here:
>>
>> https://drive.google.com/file/d/1xaPzIZ78JzHtJXcdzB77qJoeNlvVQRJq/view?usp=sharing 
>>
>>
>> I know that Mercurial need rebuild, and I assume the same
>> for subversion (that I plan to adopt).
>>
>> Could you please check your packages if they will work with
>> preferred python3.8 ?
>>
>> Texlive and fontforge seem the most urgent, IMHO, for a verification.
> 
> I think TeX Live should be fine.  All of the python scripts provided by the 
> texlive-collection-* packages are identical across platforms, so any python3 
> incompatibilities would almost certainly have been reported and fixed by now.  I 
> think all I have to do is rebuild the collections that contain python scripts to 
> make sure I pick up the latest versions of the upstream packages, and also to 
> let cygport update the dependencies.  I can do that quickly as soon as cygport 
> is ready for the new system.

I've started doing some trial rebuilds on a system with the test python3 
packages installed.  I'm finding that cygport doesn't pick up any python 
dependency for some packages (such as texlive-collection-binextra-doc) even 
though there's a script with a shebang "#!/usr/bin/env python".  Marco, you've 
probably said this already, but could you confirm that this is to be expected 
until cygport is modified?

Ken

Re: python 2 check & cleaning

[Marco Atzeri]

On 19.01.2021 16:10, Ken Brown via Cygwin-apps wrote:
> On 1/18/2021 1:25 PM, Ken Brown via Cygwin-apps wrote:
>> On 1/18/2021 12:33 AM, Marco Atzeri via Cygwin-apps wrote:
>>> Hi All,
>>>

> 
> I've started doing some trial rebuilds on a system with the test python3 
> packages installed.  I'm finding that cygport doesn't pick up any python 
> dependency for some packages (such as texlive-collection-binextra-doc) 
> even though there's a script with a shebang "#!/usr/bin/env python".  
> Marco, you've probably said this already, but could you confirm that 
> this is to be expected until cygport is modified?
> 
> Ken

Hi Ken,
it is expected as python is not anymore reporting to any package.

Just add python38 or python3 as dependency.
I plan to put python38 that obsoletes python3

Regards
Marco

Re: python 2 check & cleaning

[Lemures Lemniscati]

On Mon, 18 Jan 2021 06:33:09 +0100, Marco Atzeri via Cygwin-apps
> the current python test packages remove the link python->python2.7
> and replace with an alternative appraouch.
> 
> Before making the switch, I would like to review the packages that could be broken, so I looked at all packages, not in "python*-*" form,
> that pull one of the "python", "python2" and "python27" using
> 
>     $ cygcheck-dep -O -S -q -n python..
> 
> as base.
> 
> There are almost 200 binary packages
> (of the almost 9000 currently present on 64 bit)
> 
> The list with source package and maintainer is here:
> 
> https://drive.google.com/file/d/1xaPzIZ78JzHtJXcdzB77qJoeNlvVQRJq/view?usp=sharing
> 
> Could you please check your packages if they will work with
> preferred python3.8 ?
> 
> Texlive and fontforge seem the most urgent, IMHO, for a verification.
> 

I've promoted fontforge packages from test to current, which pass ctests.

Regards,

Lem