Inconsistent handling of python3-module vs python3x-module packages

Inconsistent handling of python3-module vs python3x-module packages

[Adam Dinwoodie]

While wrangling a bunch of Python packages for my Cygwin installation,
I've noticed an inconsistency about how python3 vs python3x packages are
installed.

As an example: the python3 package itself describes itself as a
meta-package; the package itself is almost empty, and the key thing
selecting the package does is depend on the latest python3x package,
currently python39.  The same behaviour is in place with, for example,
python3-tkinter.

However the python3-pytest package is marked as obsolete, and is
obsoleted by python36-pytest.  If I select python3-pytest for
installation, the package that's actually installed is python36-pytest,
even though there's a python39-pytest package available.

This inconsistency means that someone naively installing python3-tkinter
and python3-pytest will end up with both python3.9 and python3.6
installed, with neither installation having access to both the pytest
and tkinter modules.

I think this inconsistency is liable to cause confusion; it certainly
confused me until I worked out what was going on.  In my ideal world,
we'd be in a situation where I could specify `python3-foo` to an install
script and it'd automatically pick up the latest Python version
available; I could specify `python3x-foo` if I wanted a specific older
release.  But at the very least, I'd really like to see these packages
being handled consistently one way or another.

Adam

Re: Inconsistent handling of python3-module vs python3x-module packages

[marco atzeri]

On Wed, Feb 16, 2022 at 12:11 PM Adam Dinwoodie  wrote:
>
> While wrangling a bunch of Python packages for my Cygwin installation,
> I've noticed an inconsistency about how python3 vs python3x packages are
> installed.

Only one ? ;-)

> As an example: the python3 package itself describes itself as a
> meta-package; the package itself is almost empty, and the key thing
> selecting the package does is depend on the latest python3x package,
> currently python39.  The same behaviour is in place with, for example,
> python3-tkinter.

Yes, these are real meta packages that pull the latest version

> However the python3-pytest package is marked as obsolete, and is
> obsoleted by python36-pytest.  If I select python3-pytest for
> installation, the package that's actually installed is python36-pytest,
> even though there's a python39-pytest package available.
>
> This inconsistency means that someone naively installing python3-tkinter
> and python3-pytest will end up with both python3.9 and python3.6
> installed, with neither installation having access to both the pytest
> and tkinter modules.

This is an artifact of of cygport creating python3-xxxx packages
also for packages that never had a real package called like that.
We had  a discussion if it was worth to make the  python3-xxxx pulling the
 python39-xxxx instead of python39-xxxx and it was decided against it.

> I think this inconsistency is liable to cause confusion; it certainly
> confused me until I worked out what was going on.  In my ideal world,
> we'd be in a situation where I could specify `python3-foo` to an install
> script and it'd automatically pick up the latest Python version
> available; I could specify `python3x-foo` if I wanted a specific older
> release.  But at the very least, I'd really like to see these packages
> being handled consistently one way or another.

I expect no one looking for a package will look for obsolete packages.

> Adam

Regards
Marco

Re: Inconsistent handling of python3-module vs python3x-module packages

[Jon Turney]

On 16/02/2022 11:45, marco atzeri wrote:
> On Wed, Feb 16, 2022 at 12:11 PM Adam Dinwoodie  wrote:
>>
>> While wrangling a bunch of Python packages for my Cygwin installation,
>> I've noticed an inconsistency about how python3 vs python3x packages are
>> installed.
> 
> Only one ? ;-)
> 
>> As an example: the python3 package itself describes itself as a
>> meta-package; the package itself is almost empty, and the key thing
>> selecting the package does is depend on the latest python3x package,
>> currently python39.  The same behaviour is in place with, for example,
>> python3-tkinter.
> 
> Yes, these are real meta packages that pull the latest version
> 
>> However the python3-pytest package is marked as obsolete, and is
>> obsoleted by python36-pytest.  If I select python3-pytest for
>> installation, the package that's actually installed is python36-pytest,
>> even though there's a python39-pytest package available.
>>
>> This inconsistency means that someone naively installing python3-tkinter
>> and python3-pytest will end up with both python3.9 and python3.6
>> installed, with neither installation having access to both the pytest
>> and tkinter modules.
> 
> This is an artifact of of cygport creating python3-xxxx packages
> also for packages that never had a real package called like that.
> We had  a discussion if it was worth to make the  python3-xxxx pulling the
>   python39-xxxx instead of python39-xxxx and it was decided against it.

I think this is more the emergent behaviour due to various changes than 
the result of a considered decision.

>> I think this inconsistency is liable to cause confusion; it certainly
>> confused me until I worked out what was going on.  In my ideal world,
>> we'd be in a situation where I could specify `python3-foo` to an install
>> script and it'd automatically pick up the latest Python version
>> available; I could specify `python3x-foo` if I wanted a specific older
>> release.  But at the very least, I'd really like to see these packages
>> being handled consistently one way or another.
> 
> I expect no one looking for a package will look for obsolete packages.

It makes 'install python3-foo' a moving target for scripts.

Patches to cygport to make this work better welcome!

Re: Inconsistent handling of python3-module vs python3x-module packages

[Adam Dinwoodie]

On Thu, Feb 17, 2022 at 03:42:56PM +0000, Jon Turney wrote:
> On 16/02/2022 11:45, marco atzeri wrote:
> > On Wed, Feb 16, 2022 at 12:11 PM Adam Dinwoodie  wrote:
> > > 
> > > While wrangling a bunch of Python packages for my Cygwin installation,
> > > I've noticed an inconsistency about how python3 vs python3x packages are
> > > installed.
> > 
> > Only one ? ;-)
> > 
> > > As an example: the python3 package itself describes itself as a
> > > meta-package; the package itself is almost empty, and the key thing
> > > selecting the package does is depend on the latest python3x package,
> > > currently python39.  The same behaviour is in place with, for example,
> > > python3-tkinter.
> > 
> > Yes, these are real meta packages that pull the latest version
> > 
> > > However the python3-pytest package is marked as obsolete, and is
> > > obsoleted by python36-pytest.  If I select python3-pytest for
> > > installation, the package that's actually installed is python36-pytest,
> > > even though there's a python39-pytest package available.
> > > 
> > > This inconsistency means that someone naively installing python3-tkinter
> > > and python3-pytest will end up with both python3.9 and python3.6
> > > installed, with neither installation having access to both the pytest
> > > and tkinter modules.
> > 
> > This is an artifact of of cygport creating python3-xxxx packages
> > also for packages that never had a real package called like that.
> > We had  a discussion if it was worth to make the  python3-xxxx pulling the
> >   python39-xxxx instead of python39-xxxx and it was decided against it.
> 
> I think this is more the emergent behaviour due to various changes than the
> result of a considered decision.

My assumption here was that this was emergent behaviour, which is
clearly not to say there wasn't a discussion at the time about whether
it was worth finding a better solution.  My hope here is to think about
what would be necessary to improve things, not least because this is
causing me (admittedly very minor) pain now.

> > > I think this inconsistency is liable to cause confusion; it certainly
> > > confused me until I worked out what was going on.  In my ideal world,
> > > we'd be in a situation where I could specify `python3-foo` to an install
> > > script and it'd automatically pick up the latest Python version
> > > available; I could specify `python3x-foo` if I wanted a specific older
> > > release.  But at the very least, I'd really like to see these packages
> > > being handled consistently one way or another.
> > 
> > I expect no one looking for a package will look for obsolete packages.
> 
> It makes 'install python3-foo' a moving target for scripts.

Exactly this.  By default, I'd set up packages I maintain that require
Python packages -- for both build-time and run-time dependencies -- to
look for the latest and greatest Python interpreter and Pyhton modules,
and only pin to a specific Python version where there's a clear reason
to do so.  At the moment I can do that with the python3 package, but not
with python3-pytest.

Most users are probably not looking at obsolete packages, but if you're
trying to automate things by calling setup with the -P option, whether
the package you're looking at is marked as obsolete or not doesn't
actually get considered.  Running `setup.exe -P python3,python3-pytest`
seems reasonable, but will hit the problems I've described above.

> Patches to cygport to make this work better welcome!

I _think_ most -- if not all -- the cygport infrastructure is already in
place: cygport clearly already supports both virtual packages and
"provides" listings, which are the only bits of this that I think are
actually necessary.  The difficult part is just either (a) updating the
cygport files for all the packages that would benefit, or (b) finding a
way to have cygport work out that this is something it should do
automatically.

I'll have a look at some of the existing cygport files, and cygport
itself, and see if I can work out what it would take...

Re: Inconsistent handling of python3-module vs python3x-module packages

[Achim Gratz]

Adam Dinwoodie writes:
>> Patches to cygport to make this work better welcome!
>
> I _think_ most -- if not all -- the cygport infrastructure is already in
> place: cygport clearly already supports both virtual packages and
> "provides" listings, which are the only bits of this that I think are
> actually necessary.

This was implemented for Perl mainly and the corresponding patch to
cygport to request the correct virtual Perl package is still not
upstream.

> The difficult part is just either (a) updating the
> cygport files for all the packages that would benefit, or (b) finding a
> way to have cygport work out that this is something it should do
> automatically.

The real problem is that python is multi-version on Cygwin and what's
considered the "main" version is going to change independently of
package releases, so the way it's done for Perl (or any other single
version package that has synchronized updates of dependent packages)
won't work.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves