setup and 'provides:'

setup and 'provides:'

[Ken Brown]

[-- Attachment #1: Type: text/plain, Size: 1191 bytes --]

I've been experimenting with setup's support for the 'provides:' tag, and it's 
not behaving the way I expect [*].  I'm not sure if something in setup's 
interface with libsolv needs to be tweaked or if I'm just misunderstanding how 
this should work.  Here's what I tried:

I created a test repo with packages A, B, and C.  I made A require foo (not a 
package), and I made B and C provide foo.  The attached script does all this 
[**].  I then ran setup and selected A for installation.

The result was that libsolv simply chose B for installation, and setup showed 
this in the "Confirm" dialog.  What I expected was that libsolv would report a 
problem ("A requires foo but no selected or installed packages provide it"), 
with two possible solutions ("Install B or C").  Is that expectation unreasonable?

I plan to look into the libsolv sources to see if I can fix this, but I'm hoping 
that someone already knows the answer.

Ken

[*] I actually have a use in mind for 'provides:' in the emacs packaging, so I'd 
like to get this working.

[**] This requires using mksetupini from the calm git repo.  The released 
version of calm doesn't support 'provides:'.

[-- Attachment #2: make_test.sh --]
[-- Type: text/plain, Size: 979 bytes --]

#!/bin/sh
arch=x86_64
mkdir -p testrepo/${arch}/release/{A,B,C}
pushd testrepo/${arch}/release/A
tar -Jcf A-1-1.tar.xz --files-from /dev/null
tar -Jcf A-1-1-src.tar.xz --files-from /dev/null
cat > A-1-1.hint << EOF
sdesc: "Test package A"
ldesc: "Test package A"
category: Libs
depends: foo
EOF
popd
pushd testrepo/${arch}/release/B
tar -Jcf B-1-1.tar.xz --files-from /dev/null
tar -Jcf B-1-1-src.tar.xz --files-from /dev/null
cat > B-1-1.hint << EOF
sdesc: "Test package B"
ldesc: "Test package B"
category: Libs
depends: cygwin
provides: foo
EOF
popd
pushd testrepo/${arch}/release/C
tar -Jcf C-1-1.tar.xz --files-from /dev/null
tar -Jcf C-1-1-src.tar.xz --files-from /dev/null
cat > C-1-1.hint << EOF
sdesc: "Test package C"
ldesc: "Test package C"
category: Libs
depends: cygwin
provides: foo
EOF
popd
cd testrepo
mksetupini --arch ${arch} --inifile=${arch}/setup.ini --releasearea=. \
	   --disable-check=missing-depended-package 
xz -c ${arch}/setup.ini > ${arch}/setup.xz

Re: setup and 'provides:'

[cyg Simple]

On 10/7/2018 6:02 PM, Ken Brown wrote:
> I've been experimenting with setup's support for the 'provides:' tag, and it's 
> not behaving the way I expect [*].  I'm not sure if something in setup's 
> interface with libsolv needs to be tweaked or if I'm just misunderstanding how 
> this should work.  Here's what I tried:
> 

Shouldn't there be a need for a 'conflicts:' tag as well?

> I created a test repo with packages A, B, and C.  I made A require foo (not a 
> package), and I made B and C provide foo.  The attached script does all this 
> [**].  I then ran setup and selected A for installation.
> 

So C:foo conflicts with B:foo.

> The result was that libsolv simply chose B for installation, and setup showed 
> this in the "Confirm" dialog.  What I expected was that libsolv would report a 
> problem ("A requires foo but no selected or installed packages provide it"), 
> with two possible solutions ("Install B or C").  Is that expectation unreasonable?
> 

Not unreasonable but what happens if B:foo is already installed and
C:foo is the required because the functionality is slightly different?

-- 
cyg Simple

Re: setup and 'provides:'

[Ken Brown]

On 10/8/2018 10:41 AM, cyg Simple wrote:
> On 10/7/2018 6:02 PM, Ken Brown wrote:
>> I've been experimenting with setup's support for the 'provides:' tag, and it's
>> not behaving the way I expect [*].  I'm not sure if something in setup's
>> interface with libsolv needs to be tweaked or if I'm just misunderstanding how
>> this should work.  Here's what I tried:
>>
> 
> Shouldn't there be a need for a 'conflicts:' tag as well?

setup does support a 'conflicts:' tag, but I don't see why I should need 
it here.

>> I created a test repo with packages A, B, and C.  I made A require foo (not a
>> package), and I made B and C provide foo.  The attached script does all this
>> [**].  I then ran setup and selected A for installation.
>>
> 
> So C:foo conflicts with B:foo.

I'm not sure what you mean by C:foo and B:foo.  My intention is that 
"foo" is an identifier for a single requirement, which can be provided 
by B or C or both.  I explicitly want to allow both to be installed, so 
I don't want B and C to conflict with one another.

Here's an example (modeled on what Fedora does): Cygwin has four 
packages that provide emacs binaries: emacs, emacs-X11, emacs-lucid, and 
emacs-w32.  Users can install any or all of these if they want to be 
able to run emacs.  The differences are in the UI.  These packages don't 
conflict with one another.

If some other package requires an emacs binary, I would like it to be 
able to require "emacs-bin".  [This plays the role of "foo" in my test 
case.]  Then I would like to be able to say that all four emacs packages 
above provide "emacs-bin".

>> The result was that libsolv simply chose B for installation, and setup showed
>> this in the "Confirm" dialog.  What I expected was that libsolv would report a
>> problem ("A requires foo but no selected or installed packages provide it"),
>> with two possible solutions ("Install B or C").  Is that expectation unreasonable?
>>
> 
> Not unreasonable but what happens if B:foo is already installed and
> C:foo is the required because the functionality is slightly different?

See above.

Ken

Re: setup and 'provides:'

[cyg Simple]

On 10/8/2018 11:05 AM, Ken Brown wrote:
> On 10/8/2018 10:41 AM, cyg Simple wrote:
>> On 10/7/2018 6:02 PM, Ken Brown wrote:
>>> I've been experimenting with setup's support for the 'provides:' tag, and it's
>>> not behaving the way I expect [*].  I'm not sure if something in setup's
>>> interface with libsolv needs to be tweaked or if I'm just misunderstanding how
>>> this should work.  Here's what I tried:
>>>
>>
>> Shouldn't there be a need for a 'conflicts:' tag as well?
> 
> setup does support a 'conflicts:' tag, but I don't see why I should need 
> it here.
> 

You might not, others might.  See below ...

>>> I created a test repo with packages A, B, and C.  I made A require foo (not a
>>> package), and I made B and C provide foo.  The attached script does all this
>>> [**].  I then ran setup and selected A for installation.
>>>
>>
>> So C:foo conflicts with B:foo.
> 
> I'm not sure what you mean by C:foo and B:foo.  My intention is that 
> "foo" is an identifier for a single requirement, which can be provided 
> by B or C or both.  I explicitly want to allow both to be installed, so 
> I don't want B and C to conflict with one another.

"A single requirement" is the issue.  B and C both provide foo but the
foo in B and C are different and conflict with one another so which one
is correct to satisfy the package A requirement?

> 
> Here's an example (modeled on what Fedora does): Cygwin has four 
> packages that provide emacs binaries: emacs, emacs-X11, emacs-lucid, and 
> emacs-w32.  Users can install any or all of these if they want to be 
> able to run emacs.  The differences are in the UI.  These packages don't 
> conflict with one another.
> 

How do they overcome the conflict?

> If some other package requires an emacs binary, I would like it to be 
> able to require "emacs-bin".  [This plays the role of "foo" in my test 
> case.]  Then I would like to be able to say that all four emacs packages 
> above provide "emacs-bin".
> 

That's fine but how do an installation of multiple emacs-bin not create
a conflict?

>>> The result was that libsolv simply chose B for installation, and setup showed
>>> this in the "Confirm" dialog.  What I expected was that libsolv would report a
>>> problem ("A requires foo but no selected or installed packages provide it"),
>>> with two possible solutions ("Install B or C").  Is that expectation unreasonable?
>>>
>>
>> Not unreasonable but what happens if B:foo is already installed and
>> C:foo is the required because the functionality is slightly different?
> 
> See above.

You're above didn't answer the question.

-- 
cyg Simple

Re: setup and 'provides:'

[Ken Brown]

On 10/8/2018 11:17 AM, cyg Simple wrote:
> On 10/8/2018 11:05 AM, Ken Brown wrote:
>> Here's an example (modeled on what Fedora does): Cygwin has four
>> packages that provide emacs binaries: emacs, emacs-X11, emacs-lucid, and
>> emacs-w32.  Users can install any or all of these if they want to be
>> able to run emacs.  The differences are in the UI.  These packages don't
>> conflict with one another.
>>
> 
> How do they overcome the conflict?

They use different names for the emacs binaries: emacs-nox.exe, 
emacs-X11.exe, emacs-lucid.exe, and emacs-w32.exe.  The "alternatives" 
system then creates a symlink /usr/bin/emacs that resolves to whichever 
binary the user wants to use by default.  It's been this way for many 
years, with no problems.

>> If some other package requires an emacs binary, I would like it to be
>> able to require "emacs-bin".  [This plays the role of "foo" in my test
>> case.]  Then I would like to be able to say that all four emacs packages
>> above provide "emacs-bin".
>>
> 
> That's fine but how do an installation of multiple emacs-bin not create
> a conflict?

"emacs-bin" is not a thing that can be installed.  It's simply a name 
for a requirement, and that requirement can be met by installing any 
package that declares that it provides "emacs-bin".  At least that's my 
understanding of how it works in package managers like rpm.  As I said 
at the very beginning, it's quite possible that I'm misunderstanding how 
'provides:' is supposed to work.

Ken

Re: setup and 'provides:'

[cyg Simple]

On 10/8/2018 12:24 PM, Ken Brown wrote:
> On 10/8/2018 11:17 AM, cyg Simple wrote:
>> On 10/8/2018 11:05 AM, Ken Brown wrote:
>>> Here's an example (modeled on what Fedora does): Cygwin has four
>>> packages that provide emacs binaries: emacs, emacs-X11, emacs-lucid, and
>>> emacs-w32.  Users can install any or all of these if they want to be
>>> able to run emacs.  The differences are in the UI.  These packages don't
>>> conflict with one another.
>>>
>>
>> How do they overcome the conflict?
> 
> They use different names for the emacs binaries: emacs-nox.exe, 
> emacs-X11.exe, emacs-lucid.exe, and emacs-w32.exe.  The "alternatives" 
> system then creates a symlink /usr/bin/emacs that resolves to whichever 
> binary the user wants to use by default.  It's been this way for many 
> years, with no problems.
> 

I assumed that this was the case.  But the symlink is a conflict and I
assume that if one exists already the package management system would
not recreate one or would ask the user if it should be overwritten.

>>> If some other package requires an emacs binary, I would like it to be
>>> able to require "emacs-bin".  [This plays the role of "foo" in my test
>>> case.]  Then I would like to be able to say that all four emacs packages
>>> above provide "emacs-bin".
>>>
>>
>> That's fine but how do an installation of multiple emacs-bin not create
>> a conflict?
> 
> "emacs-bin" is not a thing that can be installed.  It's simply a name 
> for a requirement, and that requirement can be met by installing any 
> package that declares that it provides "emacs-bin".  At least that's my 
> understanding of how it works in package managers like rpm.  As I said 
> at the very beginning, it's quite possible that I'm misunderstanding how 
> 'provides:' is supposed to work.

And I understand that emacs-bin is a pseudo name identifying a class of
product the package provides.  The RPM system allows for defining
Requires, Provides, Conflicts and Obsoletes.  The Arch Linux pacman
allows for depends, makedepends, checkdepends, optdepends, provides,
replaces and conflicts.  As you can see other package managers see
*conflicts* as an important item because of the global namespace issue.

-- 
cyg Simple

Re: setup and 'provides:'

[Ken Brown]

On 10/9/2018 11:10 AM, cyg Simple wrote:
> On 10/8/2018 12:24 PM, Ken Brown wrote:
>> On 10/8/2018 11:17 AM, cyg Simple wrote:
>>> On 10/8/2018 11:05 AM, Ken Brown wrote:
>>>> Here's an example (modeled on what Fedora does): Cygwin has four
>>>> packages that provide emacs binaries: emacs, emacs-X11, emacs-lucid, and
>>>> emacs-w32.  Users can install any or all of these if they want to be
>>>> able to run emacs.  The differences are in the UI.  These packages don't
>>>> conflict with one another.
>>>>
>>>
>>> How do they overcome the conflict?
>>
>> They use different names for the emacs binaries: emacs-nox.exe,
>> emacs-X11.exe, emacs-lucid.exe, and emacs-w32.exe.  The "alternatives"
>> system then creates a symlink /usr/bin/emacs that resolves to whichever
>> binary the user wants to use by default.  It's been this way for many
>> years, with no problems.
>>
> 
> I assumed that this was the case.  But the symlink is a conflict and I
> assume that if one exists already the package management system would
> not recreate one or would ask the user if it should be overwritten.

No, that's not how it works.  The symlink is created and managed by the 
alternatives system via commands in postinstall/preremove scripts and in 
user-callable scripts /usr/bin/set-emacs-default-*.  There is no conflict.  The 
various emacs packages can happily coexist.

>>>> If some other package requires an emacs binary, I would like it to be
>>>> able to require "emacs-bin".  [This plays the role of "foo" in my test
>>>> case.]  Then I would like to be able to say that all four emacs packages
>>>> above provide "emacs-bin".
>>>>
>>>
>>> That's fine but how do an installation of multiple emacs-bin not create
>>> a conflict?
>>
>> "emacs-bin" is not a thing that can be installed.  It's simply a name
>> for a requirement, and that requirement can be met by installing any
>> package that declares that it provides "emacs-bin".  At least that's my
>> understanding of how it works in package managers like rpm.  As I said
>> at the very beginning, it's quite possible that I'm misunderstanding how
>> 'provides:' is supposed to work.
> 
> And I understand that emacs-bin is a pseudo name identifying a class of
> product the package provides.  The RPM system allows for defining
> Requires, Provides, Conflicts and Obsoletes.

So does Cygwin's setup.

>  The Arch Linux pacman
> allows for depends, makedepends, checkdepends, optdepends, provides,
> replaces and conflicts.  As you can see other package managers see
> *conflicts* as an important item because of the global namespace issue.

I agree that it can be useful, and I'm glad setup supports it.

Ken

Re: setup and 'provides:'

[Achim Gratz]

cyg Simple writes:
> I assumed that this was the case.  But the symlink is a conflict and I
> assume that if one exists already the package management system would
> not recreate one or would ask the user if it should be overwritten.

The symlink to the actual executable is not a conflict since it isn't
packaged.  It is created / altered via alternatives in a postinstall
script (and potentially altered and/or removed by a pre-remove script).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: setup and 'provides:'

[Ken Brown]

On 10/7/2018 6:02 PM, Ken Brown wrote:
> I've been experimenting with setup's support for the 'provides:' tag, and it's
> not behaving the way I expect [*].  I'm not sure if something in setup's
> interface with libsolv needs to be tweaked or if I'm just misunderstanding how
> this should work.  Here's what I tried:
> 
> I created a test repo with packages A, B, and C.  I made A require foo (not a
> package), and I made B and C provide foo.  The attached script does all this
> [**].  I then ran setup and selected A for installation.
> 
> The result was that libsolv simply chose B for installation, and setup showed
> this in the "Confirm" dialog.  What I expected was that libsolv would report a
> problem ("A requires foo but no selected or installed packages provide it"),
> with two possible solutions ("Install B or C").  Is that expectation unreasonable?
> 
> I plan to look into the libsolv sources to see if I can fix this, but I'm hoping
> that someone already knows the answer.

It turns out that this question has been asked before:

https://bugzilla.redhat.com/show_bug.cgi?id=1266761#c3
https://github.com/openSUSE/libsolv/issues/107

AFAICT, libsolv will by default pick the alphabetically first package 
when there are several equally good candidates to meet the requirements. 
  According to the second link I cited, "libsolv offers methods to query 
for the encountered ambiguities after the solving, so that the user can 
be presented with a list of choices and then select different packages."

I may look into this further, but I'm beginning to wonder if it's worth 
the trouble.

Ken

Re: setup and 'provides:'

[Achim Gratz]

Ken Brown writes:
> I created a test repo with packages A, B, and C.  I made A require foo (not a 
> package), and I made B and C provide foo.  The attached script does all this 
> [**].  I then ran setup and selected A for installation.

That really only makes sense if several packages provide the same
feature(s).  So we'd need to bolt on a feature space in setup as well.

> The result was that libsolv simply chose B for installation, and setup showed 
> this in the "Confirm" dialog.  What I expected was that libsolv would report a 
> problem ("A requires foo but no selected or installed packages provide it"), 
> with two possible solutions ("Install B or C").  Is that expectation unreasonable?

Zypper would present you with a list of several potential solutions and
let you (incrementally if necessary) pick one of them.  I doubt that
will work nicely with the current setup UI.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: setup and 'provides:'

[Ken Brown]

On 10/8/2018 4:04 PM, Achim Gratz wrote:
> Ken Brown writes:
>> I created a test repo with packages A, B, and C.  I made A require foo (not a
>> package), and I made B and C provide foo.  The attached script does all this
>> [**].  I then ran setup and selected A for installation.
> 
> That really only makes sense if several packages provide the same
> feature(s).

Right.  I gave a potential example elsewhere in this thread, in which the 
feature emacs-bin is provided by the packages emacs, emacs-X11, emacs-lucid, and 
emacs-w32.  Fedora does something like this, except that it calls the feature 
emacs(bin).

>>  So we'd need to bolt on a feature space in setup as well.

I'm not sure what you mean by this.  setup doesn't need to know whether a 
requirement is a feature or a package.  It simply passes the dependencies (and 
the provides) to libsolv, which handles them just fine.

The only thing that I thought wasn't working right is libsolv's behavior when 
more than one package provides a certain feature and libsolv has to choose one 
for installation.  But I realize now that libsolv just doesn't do what I 
expected, and some non-trivial work on setup would be required in order to get 
the behavior I was expecting.  See my other messages in this thread.

Ken

Re: setup and 'provides:'

[Achim Gratz]

Ken Brown writes:
>>>  So we'd need to bolt on a feature space in setup as well.
>
> I'm not sure what you mean by this.  setup doesn't need to know whether a 
> requirement is a feature or a package.  It simply passes the dependencies (and 
> the provides) to libsolv, which handles them just fine.

If we have features, then setup would need to show them somewhere and it
would be awkward if that info was empty for most packages.  Then people
will want to install features instead of packages anyway, so that is
another nudge into "featurizing" the whole distribution.  In other
words, once we start using "provides:" in that way, I think it should
become mandatory.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada