[setup topic/libsolv] Packages contained in multiple repositories

[setup topic/libsolv] Packages contained in multiple repositories

[Ken Brown]

[-- Attachment #1: Type: text/plain, Size: 781 bytes --]

When a package (with specified version) is contained in multiple 
repositories, we register in packagemeta the last one we see.  But if 
libsolv decides that the package needs to be installed, its solution may 
arbitrarily specify one from a different repo.  This caused me some 
confusion when debugging an unrelated issue, and I created the attached 
patch to "fix" it.

In retrospect, I'm not sure this patch is right, but I'm sending it 
anyway for the sake of discussion.  My hesitation comes from the fact 
that libsolv might have a good reason for preferring the one it chose, 
e.g., if we've assigned priorities to the repos.  On the other hand, if 
we've gone to the trouble of assigning priorities, shouldn't packagemeta 
reflect our choice?

I'm of two minds here.

Ken

[-- Attachment #2: 0001-Prefer-the-packageversion-registered-in-packagemeta.patch --]
[-- Type: text/plain, Size: 1932 bytes --]

From 2c0c1edbecad7cdce69a02cef0506b93fe5d7981 Mon Sep 17 00:00:00 2001
From: Ken Brown <kbrown@cornell.edu>
Date: Wed, 18 Oct 2017 09:24:48 -0400
Subject: [PATCH] Prefer the packageversion registered in packagemeta

When a packageversion that is contained in multiple repositories is
being installed, the solver has no way to know which one we prefer.
Change the solution, if necessary, to use the one we registered in
packagemeta when we parsed the setup.ini files.
---
 libsolv.cc | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/libsolv.cc b/libsolv.cc
index 3a244d4..3750867 100644
--- a/libsolv.cc
+++ b/libsolv.cc
@@ -21,6 +21,8 @@
 
 #include "LogSingleton.h"
 #include  <iomanip>
+#include <set>
+#include <assert.h>
 
 // ---------------------------------------------------------------------------
 // Utility functions for mapping between Operators and Relation Ids
@@ -762,10 +764,21 @@ SolverSolution::update(SolverTasks &tasks, updateMode update, bool use_test_pack
 
   for (int i = 0; i < t->steps.count; i++)
     {
-      Id id = t->steps.elements[i];
       SolverTransaction::transType tt = type(t, i);
+      SolvableVersion sv = SolvableVersion(t->steps.elements[i], pool.pool);
+      // If we are installing a package that is contained in multiple
+      // repositories, we want to use the one registered in
+      // packagemeta.
+      if (tt == SolverTransaction::transInstall)
+	{
+	  packagedb db;
+	  packagemeta *pkg = db.findBinary(PackageSpecification(sv.Name()));
+	  std::set <packageversion>::iterator j = pkg->versions.find(sv);
+	  assert (j != pkg->versions.end());
+	  sv = *j;
+	}
       if (tt != SolverTransaction::transIgnore)
-        trans.push_back(SolverTransaction(SolvableVersion(id, pool.pool), tt));
+        trans.push_back(SolverTransaction(sv, tt));
     }
 
   // add install and remove tasks for anything marked as reinstall
-- 
2.14.2

Re: [setup topic/libsolv] Packages contained in multiple repositories

[Achim Gratz]

Ken Brown writes:
> In retrospect, I'm not sure this patch is right, but I'm sending it
> anyway for the sake of discussion.  My hesitation comes from the fact
> that libsolv might have a good reason for preferring the one it chose,
> e.g., if we've assigned priorities to the repos.  On the other hand,
> if we've gone to the trouble of assigning priorities, shouldn't
> packagemeta reflect our choice?

Extrapolating from my experience with zypper, libsolv should stick with
the repo the installed package comes from even if some other repo has a
newer version.  The whole purpose of the "dup" command in zypper is to
lift that restriction (compared to the normal "up") and consider the
highest version from any repo as the preferred package (unless more
specific dependencies would yield a lower version or repo priorities
override the default algorithm).

This is often used for example to update just a single application to
something different from the main distribution: chose an extra repo,
install just one of many applications from that repo and then keep
updating the system normally.  The updates will come from your install
repo and just that single application will be updated from the extra
repo instead.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

Re: [setup topic/libsolv] Packages contained in multiple repositories

[Jon Turney]

On 18/10/2017 17:41, Achim Gratz wrote:
> Ken Brown writes:
>> In retrospect, I'm not sure this patch is right, but I'm sending it
>> anyway for the sake of discussion.  My hesitation comes from the fact
>> that libsolv might have a good reason for preferring the one it chose,
>> e.g., if we've assigned priorities to the repos.  On the other hand,
>> if we've gone to the trouble of assigning priorities, shouldn't
>> packagemeta reflect our choice?
> 
> Extrapolating from my experience with zypper, libsolv should stick with
> the repo the installed package comes from even if some other repo has a
> newer version.

Unfortunately, we are limited by the fact that installed.db doesn't 
record which repo an installed package came from.

We map repo setup.ini release: labels to package vendor names, so we 
assume it's 'Cygwin' for an installed package (package_db.cc:115), and 
run the solver with SOLVER_FLAG_ALLOW_VENDORCHANGE on (libsolv.cc:745) 
to allow it to get upgraded by a package in the repo it actually came 
from, but we've forgotten.

I'm not overly concerned about this: we don't define what happens in 
this situation currently, and if the packages are identical, it's no 
problem.

If the packages are the same version but have different contents, you're 
asking for problems, although I guess it would be nice to do something 
defined in that situation.

Re: [setup topic/libsolv] Packages contained in multiple repositories

[Achim Gratz]

Jon Turney writes:
>> Extrapolating from my experience with zypper, libsolv should stick with
>> the repo the installed package comes from even if some other repo has a
>> newer version.
>
> Unfortunately, we are limited by the fact that installed.db doesn't
> record which repo an installed package came from.

Which is something we might eventually change, but yes, we can't use
that information at the moment if we can't figure out the sourtce repo
by looking at the version.

> We map repo setup.ini release: labels to package vendor names, so we
> assume it's 'Cygwin' for an installed package (package_db.cc:115), and
> run the solver with SOLVER_FLAG_ALLOW_VENDORCHANGE on (libsolv.cc:745)
> to allow it to get upgraded by a package in the repo it actually came
> from, but we've forgotten.

Well actually zypper does the same thing if you have a package
installed, but it's gone from the repos (we'd call that an orphan
package): it'll show up as "@System" instead of the repo it came from
originally.

> I'm not overly concerned about this: we don't define what happens in
> this situation currently, and if the packages are identical, it's no
> problem.

As long as the repos we're installing from use coordinated
version/release numbers, then we could map them correctly as long as the
packages are not orphaned.  That would also allow to correctly
"transfer" a package from a hypothetical "test" repo into the "current"
repo, regardless of when it was originally installed.

> If the packages are the same version but have different contents,
> you're asking for problems, although I guess it would be nice to do
> something defined in that situation.

I think that the multiple repositories situation for zypper is still
expected to provide either different versions or the same content.  At
least all the repositories that I'm aware of (that are supposed to work
together) keep things that way.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [setup topic/libsolv] Packages contained in multiple repositories

[Ken Brown]

On 10/19/2017 9:36 AM, Jon Turney wrote:
> On 18/10/2017 17:41, Achim Gratz wrote:
>> Ken Brown writes:
>>> In retrospect, I'm not sure this patch is right, but I'm sending it
>>> anyway for the sake of discussion.  My hesitation comes from the fact
>>> that libsolv might have a good reason for preferring the one it chose,
>>> e.g., if we've assigned priorities to the repos.  On the other hand,
>>> if we've gone to the trouble of assigning priorities, shouldn't
>>> packagemeta reflect our choice?
>>
>> Extrapolating from my experience with zypper, libsolv should stick with
>> the repo the installed package comes from even if some other repo has a
>> newer version.
> 
> Unfortunately, we are limited by the fact that installed.db doesn't 
> record which repo an installed package came from.
> 
> We map repo setup.ini release: labels to package vendor names, so we 
> assume it's 'Cygwin' for an installed package (package_db.cc:115), and 
> run the solver with SOLVER_FLAG_ALLOW_VENDORCHANGE on (libsolv.cc:745) 
> to allow it to get upgraded by a package in the repo it actually came 
> from, but we've forgotten.
> 
> I'm not overly concerned about this: we don't define what happens in 
> this situation currently, and if the packages are identical, it's no 
> problem.

Here's a situation where I think it might be a problem: 
fixup_source_package_ids only considers packageversions that are 
registered in packagedb.  If libsolv later selects a packageversion from 
a different repo and the user has requested the source, it might not be 
found.

Ken