[PATCH setup] Add view mode "Unneeded"

[PATCH setup] Add view mode "Unneeded"

[Christian Franke]

[-- Attachment #1: Type: text/plain, Size: 500 bytes --]

In long standing cygwin installations, many no longer needed 
automatically installed packages (e.g. libicuNN) accumulate. This patch 
adds a new view which is possibly helpful to cleanup packages manually.

Some possible later enhancements:
- automatically refresh this view (a few seconds) after the user changed 
a package status as this may add or remove entries.
- add a keyboard shortcut (^U) to the list view for "Uninstall this 
package and then select next package"

-- 
Regards,
Christian


[-- Attachment #2: 0001-Add-view-mode-Unneeded.patch --]
[-- Type: text/plain, Size: 4818 bytes --]

From 07a2e561b7ba19817a0bebb2cbf542a9b18a6d4e Mon Sep 17 00:00:00 2001
From: Christian Franke <christian.franke@t-online.de>
Date: Tue, 2 Aug 2022 13:54:30 +0200
Subject: [PATCH] Add view mode "Unneeded"

This view shows installed packages which could be safely removed.
---
 PickView.cc   | 47 ++++++++++++++++++++++++++++++++++++++++++++++-
 PickView.h    |  1 +
 res/en/res.rc |  1 +
 res/fr/res.rc |  1 +
 resource.h    |  1 +
 5 files changed, 50 insertions(+), 1 deletion(-)

diff --git a/PickView.cc b/PickView.cc
index c961b9f..b7a37b5 100644
--- a/PickView.cc
+++ b/PickView.cc
@@ -17,6 +17,7 @@
 #include "PickPackageLine.h"
 #include "PickCategoryLine.h"
 #include <algorithm>
+#include <set>
 #include <limits.h>
 #include <shlwapi.h>
 
@@ -28,6 +29,40 @@
 #include "LogSingleton.h"
 #include "Exception.h"
 
+// Scan installed or desired packages and collect the names of packages
+// which provide the dependencies of other packages or are member of
+// category "Base".
+static void FindNeededPackages (const packagedb & db, std::set<std::string> & needed)
+{
+  std::map<std::string, std::string> providedBy;
+  for (const auto & p : db.packages)
+    {
+      const packagemeta & pkg = *p.second;
+      if (!pkg.isBinary ())
+        continue;
+      if (!(pkg.desired && (pkg.installed || pkg.picked ())))
+        continue;
+      for (const PackageSpecification *s : pkg.desired.provides ())
+        providedBy.insert ({s->packageName (), pkg.name});
+    }
+  for (const auto & p : db.packages)
+    {
+      const packagemeta & pkg = *p.second;
+      if (!pkg.isBinary ())
+        continue;
+      if (pkg.categories.count ("Base"))
+        needed.insert (pkg.name);
+      if (!(pkg.desired && (pkg.installed || pkg.picked ())))
+        continue;
+      for (const PackageSpecification *s : pkg.desired.depends ()) {
+        const auto i = providedBy.find (s->packageName ());
+        if (i == providedBy.end ())
+          continue;
+        needed.insert (i->second);
+      }
+    }
+}
+
 void
 PickView::setViewMode (views mode)
 {
@@ -47,6 +82,10 @@ PickView::setViewMode (views mode)
     }
   else
     {
+      std::set<std::string> needed;
+      if (view_mode == PickView::views::PackageUnneeded)
+        FindNeededPackages (db, needed);
+
       // iterate through every package
       for (packagedb::packagecollection::iterator i = db.packages.begin ();
             i != db.packages.end (); ++i)
@@ -77,7 +116,11 @@ PickView::setViewMode (views mode)
 
               // "UserPick" : installed packages that were picked by user
               || (view_mode == PickView::views::PackageUserPicked &&
-                  (pkg.installed && pkg.user_picked)))
+                  (pkg.installed && pkg.user_picked))
+
+              // "Unneeded" : installed packages which could be safely removed
+              || (view_mode == PickView::views::PackageUnneeded &&
+                  (pkg.installed && !needed.count (pkg.name))))
             {
               // Filter by package name
               if (packageFilterString.empty ()
@@ -111,6 +154,8 @@ PickView::mode_caption (views mode)
       return IDS_VIEW_NOTINSTALLED;
     case views::PackageUserPicked:
       return IDS_VIEW_PICKED;
+    case views::PackageUnneeded:
+      return IDS_VIEW_UNNEEDED;
     case views::Category:
       return IDS_VIEW_CATEGORY;
     default:
diff --git a/PickView.h b/PickView.h
index 1e14a74..f2dbfa9 100644
--- a/PickView.h
+++ b/PickView.h
@@ -36,6 +36,7 @@ public:
     PackageKeeps,
     PackageSkips,
     PackageUserPicked,
+    PackageUnneeded,
     Category,
   };
 
diff --git a/res/en/res.rc b/res/en/res.rc
index 644b252..9517bb2 100644
--- a/res/en/res.rc
+++ b/res/en/res.rc
@@ -578,6 +578,7 @@ BEGIN
     IDS_VIEW_UPTODATE "Up To Date"
     IDS_VIEW_NOTINSTALLED "Not Installed"
     IDS_VIEW_PICKED "Picked"
+    IDS_VIEW_UNNEEDED "Unneeded"
     IDS_VIEW_CATEGORY "Category"
     IDS_COLUMN_PACKAGE "Package"
     IDS_COLUMN_CURRENT "Current"
diff --git a/res/fr/res.rc b/res/fr/res.rc
index a0a7909..d787fdb 100644
--- a/res/fr/res.rc
+++ b/res/fr/res.rc
@@ -564,6 +564,7 @@ BEGIN
     IDS_VIEW_UPTODATE "À jour"
     IDS_VIEW_NOTINSTALLED "Non installé"
     IDS_VIEW_PICKED "Sélectionné"
+    // IDS_VIEW_UNNEEDED "XXX: missing translation"
     IDS_VIEW_CATEGORY  "Catégorie"
     IDS_COLUMN_PACKAGE "Paquet"
     IDS_COLUMN_CURRENT "Actuel"
diff --git a/resource.h b/resource.h
index 2668dd9..8c33794 100644
--- a/resource.h
+++ b/resource.h
@@ -105,6 +105,7 @@
 #define IDS_FILE_INUSE_MSG               1208
 #define IDS_DEPRECATED_WINDOWS_VERSION   1209
 #define IDS_DEPRECATED_WINDOWS_ARCH      1210
+#define IDS_VIEW_UNNEEDED                1211
 
 #define IDS_HELPTEXT_COMPACTOS           1500
 #define IDS_HELPTEXT_PUBKEY              1501
-- 
2.37.1

Re: [PATCH setup] Add view mode "Unneeded"

[Jon Turney]

On 02/08/2022 13:17, Christian Franke wrote:
> In long standing cygwin installations, many no longer needed 
> automatically installed packages (e.g. libicuNN) accumulate. This patch 
> adds a new view which is possibly helpful to cleanup packages manually.
> 
> Some possible later enhancements:
> - automatically refresh this view (a few seconds) after the user changed 
> a package status as this may add or remove entries.
> - add a keyboard shortcut (^U) to the list view for "Uninstall this 
> package and then select next package"
> 

Thanks.  This looks good.

I think perhaps a better approach would be a view showing all packages 
which aren't user_picked, or a dependency of a user_picked package.

(If I've read the code correctly your implementation has the weakness 
that if e.g. appA -> libbB -> libC, which is then changed to appA -> 
libD -> libE, it will only show libC as unneeded, then libB on the next 
run?)

> +// Scan installed or desired packages and collect the names of packages
> +// which provide the dependencies of other packages or are member of
> +// category "Base".
> +static void FindNeededPackages (const packagedb & db, std::set<std::string> & needed)
> +{
> +  std::map<std::string, std::string> providedBy;
> +  for (const auto & p : db.packages)
> +    {
> +      const packagemeta & pkg = *p.second;
> +      if (!pkg.isBinary ())
> +        continue;
> +      if (!(pkg.desired && (pkg.installed || pkg.picked ())))
> +        continue;

This seems redundant.  Why can't this be just !pkg.desired?

This should also update the tooltip for the view dropdown 
(IDS_VIEWBUTTON_TOOLTIP) to describe the new view.

Re: [PATCH setup] Add view mode "Unneeded"

[Christian Franke]

[-- Attachment #1: Type: text/plain, Size: 2560 bytes --]

Jon Turney wrote:
> On 02/08/2022 13:17, Christian Franke wrote:
>> In long standing cygwin installations, many no longer needed 
>> automatically installed packages (e.g. libicuNN) accumulate. This 
>> patch adds a new view which is possibly helpful to cleanup packages 
>> manually.
>>
>> Some possible later enhancements:
>> - automatically refresh this view (a few seconds) after the user 
>> changed a package status as this may add or remove entries.
>> - add a keyboard shortcut (^U) to the list view for "Uninstall this 
>> package and then select next package"
>>
>
> Thanks.  This looks good.
>
> I think perhaps a better approach would be a view showing all packages 
> which aren't user_picked, or a dependency of a user_picked package.

This would drop the ability to easily clean up user_picked packages 
without later conflicts. The attached new patch splits this into two 
views "Removable" (not "Uninstallable" due to possible ambiguity with 
"cannot be installed") and "Unneeded" (or "Stale" ?).


>
> (If I've read the code correctly your implementation has the weakness 
> that if e.g. appA -> libbB -> libC, which is then changed to appA -> 
> libD -> libE, it will only show libC as unneeded, then libB on the 
> next run?)

I'm not sure for this case. It may be correct again after the view is 
refreshed during the same run. In general, this ad-hoc algorithm does 
not handle all corner cases. It should be sufficient if installation 
cleanup is done in a separate run.


>
>
>> +// Scan installed or desired packages and collect the names of packages
>> +// which provide the dependencies of other packages or are member of
>> +// category "Base".
>> +static void FindNeededPackages (const packagedb & db, 
>> std::set<std::string> & needed)
>> +{
>> +  std::map<std::string, std::string> providedBy;
>> +  for (const auto & p : db.packages)
>> +    {
>> +      const packagemeta & pkg = *p.second;
>> +      if (!pkg.isBinary ())
>> +        continue;
>> +      if (!(pkg.desired && (pkg.installed || pkg.picked ())))
>> +        continue;
>
> This seems redundant.  Why can't this be just !pkg.desired?

Yes, fixed. I originally wanted to handle the "install source package 
without the binary" case here. During development of "Ctrl+I/R/U" patch, 
I learned that this could not happen.


> This should also update the tooltip for the view dropdown 
> (IDS_VIEWBUTTON_TOOLTIP) to describe the new view.
>

Done.

Open issue: An easy way to refresh the views after Uninstall requests 
(Ctrl+L ?).


[-- Attachment #2: 0001-Add-view-modes-Removable-and-Unneeded.patch --]
[-- Type: text/plain, Size: 6748 bytes --]

From b31674d809a71bf17bb621c74e5ba7b3df3cd80a Mon Sep 17 00:00:00 2001
From: Christian Franke <christian.franke@t-online.de>
Date: Mon, 15 Aug 2022 14:21:36 +0200
Subject: [PATCH] Add view modes "Removable" and "Unneeded"

These views show user picked or automatically installed packages which
could be safely removed.
---
 PickView.cc   | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 PickView.h    |  2 ++
 res/en/res.rc | 12 +++++++++++-
 res/fr/res.rc |  4 ++++
 resource.h    |  2 ++
 5 files changed, 71 insertions(+), 2 deletions(-)

diff --git a/PickView.cc b/PickView.cc
index c961b9f..7d60d8a 100644
--- a/PickView.cc
+++ b/PickView.cc
@@ -17,6 +17,7 @@
 #include "PickPackageLine.h"
 #include "PickCategoryLine.h"
 #include <algorithm>
+#include <set>
 #include <limits.h>
 #include <shlwapi.h>
 
@@ -28,6 +29,39 @@
 #include "LogSingleton.h"
 #include "Exception.h"
 
+// Scan desired packages and collect the names of packages which provide the
+// dependencies of other desired packages or are member of category "Base".
+static void FindNeededPackages (const packagedb & db, std::set<std::string> & needed)
+{
+  std::map<std::string, std::string> providedBy;
+  for (const auto & p : db.packages)
+    {
+      const packagemeta & pkg = *p.second;
+      if (!pkg.isBinary ())
+        continue;
+      if (!pkg.desired)
+        continue;
+      for (const PackageSpecification *s : pkg.desired.provides ())
+        providedBy.insert ({s->packageName (), pkg.name});
+    }
+  for (const auto & p : db.packages)
+    {
+      const packagemeta & pkg = *p.second;
+      if (!pkg.isBinary ())
+        continue;
+      if (pkg.categories.count ("Base"))
+        needed.insert (pkg.name);
+      if (!pkg.desired)
+        continue;
+      for (const PackageSpecification *s : pkg.desired.depends ()) {
+        const auto i = providedBy.find (s->packageName ());
+        if (i == providedBy.end ())
+          continue;
+        needed.insert (i->second);
+      }
+    }
+}
+
 void
 PickView::setViewMode (views mode)
 {
@@ -47,6 +81,11 @@ PickView::setViewMode (views mode)
     }
   else
     {
+      std::set<std::string> needed;
+      if (view_mode == PickView::views::PackageRemovable
+          || view_mode == PickView::views::PackageUnneeded)
+        FindNeededPackages (db, needed);
+
       // iterate through every package
       for (packagedb::packagecollection::iterator i = db.packages.begin ();
             i != db.packages.end (); ++i)
@@ -77,7 +116,15 @@ PickView::setViewMode (views mode)
 
               // "UserPick" : installed packages that were picked by user
               || (view_mode == PickView::views::PackageUserPicked &&
-                  (pkg.installed && pkg.user_picked)))
+                  (pkg.installed && pkg.user_picked))
+
+              // "Removable" : user picked packages that could be safely removed
+              || (view_mode == PickView::views::PackageRemovable &&
+                  (pkg.installed && pkg.user_picked && !needed.count (pkg.name)))
+
+              // "Unneeded" : auto installed packages that could be safely removed
+              || (view_mode == PickView::views::PackageUnneeded &&
+                  (pkg.installed && !pkg.user_picked && !needed.count (pkg.name))))
             {
               // Filter by package name
               if (packageFilterString.empty ()
@@ -111,6 +158,10 @@ PickView::mode_caption (views mode)
       return IDS_VIEW_NOTINSTALLED;
     case views::PackageUserPicked:
       return IDS_VIEW_PICKED;
+    case views::PackageRemovable:
+      return IDS_VIEW_REMOVABLE;
+    case views::PackageUnneeded:
+      return IDS_VIEW_UNNEEDED;
     case views::Category:
       return IDS_VIEW_CATEGORY;
     default:
diff --git a/PickView.h b/PickView.h
index 1e14a74..4833414 100644
--- a/PickView.h
+++ b/PickView.h
@@ -36,6 +36,8 @@ public:
     PackageKeeps,
     PackageSkips,
     PackageUserPicked,
+    PackageRemovable,
+    PackageUnneeded,
     Category,
   };
 
diff --git a/res/en/res.rc b/res/en/res.rc
index 644b252..fc61e59 100644
--- a/res/en/res.rc
+++ b/res/en/res.rc
@@ -537,7 +537,15 @@ BEGIN
        "and haven't been selected for installation.\n"
        "\n"
        "Picked: Show installed packages that were selected, not installed "
-       "as a dependency."
+       "as a dependency.\n"
+       "\n"
+       "Removable: Show installed packages that were selected and could be "
+       "safely removed.  No other installed or selected packages depend on "
+       "these packages.\n"
+       "\n"
+       "Unneeded: Show automatically installed packages that could now be "
+       "safely removed.  Other installed or selected packages no longer "
+       "depend on these packages."
     IDS_HIDEOBS_TOOLTIP     "If selected, setup will hide packages in categories "
        "with names that begin with '_'.  Such packages are usually empty "
        "placeholders for packages that have been removed or renamed, or are "
@@ -578,6 +586,8 @@ BEGIN
     IDS_VIEW_UPTODATE "Up To Date"
     IDS_VIEW_NOTINSTALLED "Not Installed"
     IDS_VIEW_PICKED "Picked"
+    IDS_VIEW_REMOVABLE "Removable"
+    IDS_VIEW_UNNEEDED "Unneeded"
     IDS_VIEW_CATEGORY "Category"
     IDS_COLUMN_PACKAGE "Package"
     IDS_COLUMN_CURRENT "Current"
diff --git a/res/fr/res.rc b/res/fr/res.rc
index a0a7909..681aea4 100644
--- a/res/fr/res.rc
+++ b/res/fr/res.rc
@@ -524,6 +524,8 @@ BEGIN
        "\n"
        "Choisi : montre les paquets installés qui ont été sélectionnés, et non installés "
        "pour résoudre une dépendance."
+       // "Removable: ... XXX: Missing translation"
+       // "Unneeded: ... XXX: Missing translation"
     IDS_HIDEOBS_TOOLTIP     "L'assistant cachera les paquets des catégories dont "
        "le nom commence par '_'.  Ces paquets sont vides et sont des emplacements "
        "pour des paquets éliminés ou renommés, ou encore des paquets "
@@ -564,6 +566,8 @@ BEGIN
     IDS_VIEW_UPTODATE "À jour"
     IDS_VIEW_NOTINSTALLED "Non installé"
     IDS_VIEW_PICKED "Sélectionné"
+    // IDS_VIEW_REMOVABLE "XXX: missing translation"
+    // IDS_VIEW_UNNEEDED "XXX: missing translation"
     IDS_VIEW_CATEGORY  "Catégorie"
     IDS_COLUMN_PACKAGE "Paquet"
     IDS_COLUMN_CURRENT "Actuel"
diff --git a/resource.h b/resource.h
index 2668dd9..cfe860b 100644
--- a/resource.h
+++ b/resource.h
@@ -105,6 +105,8 @@
 #define IDS_FILE_INUSE_MSG               1208
 #define IDS_DEPRECATED_WINDOWS_VERSION   1209
 #define IDS_DEPRECATED_WINDOWS_ARCH      1210
+#define IDS_VIEW_REMOVABLE               1211
+#define IDS_VIEW_UNNEEDED                1212
 
 #define IDS_HELPTEXT_COMPACTOS           1500
 #define IDS_HELPTEXT_PUBKEY              1501
-- 
2.37.1

Re: [PATCH setup] Add view mode "Unneeded"

[Jon Turney]

On 15/08/2022 14:04, Christian Franke wrote:
> Jon Turney wrote:
>> On 02/08/2022 13:17, Christian Franke wrote:
>>> In long standing cygwin installations, many no longer needed 
>>> automatically installed packages (e.g. libicuNN) accumulate. This 
>>> patch adds a new view which is possibly helpful to cleanup packages 
>>> manually.
>>>
>>> Some possible later enhancements:
>>> - automatically refresh this view (a few seconds) after the user 
>>> changed a package status as this may add or remove entries.
>>> - add a keyboard shortcut (^U) to the list view for "Uninstall this 
>>> package and then select next package"
>>>
>>
>> Thanks.  This looks good.
>>
>> I think perhaps a better approach would be a view showing all packages 
>> which aren't user_picked, or a dependency of a user_picked package.
> 
> This would drop the ability to easily clean up user_picked packages 
> without later conflicts. The attached new patch splits this into two 
> views "Removable" (not "Uninstallable" due to possible ambiguity with 
> "cannot be installed") and "Unneeded" (or "Stale" ?).
>  
>>
>> (If I've read the code correctly your implementation has the weakness 
>> that if e.g. appA -> libbB -> libC, which is then changed to appA -> 
>> libD -> libE, it will only show libC as unneeded, then libB on the 
>> next run?)
> 
> I'm not sure for this case. It may be correct again after the view is 
> refreshed during the same run. In general, this ad-hoc algorithm does 
> not handle all corner cases. It should be sufficient if installation 
> cleanup is done in a separate run.
> 

Thanks.  I applied this.

I'm not sure these are the perfect filters, but they are a lot better 
than the nothing we have at the moment :)

Some time ago, I thought about replacing the "view"s list with a 
drop-down menu which would let you choose how to filter and show the 
package list, but I don't think I got around to writing anything...

e.g

---
[x] Installed
[✓] Changed
[ ] Picked
[ ] Needed
---
[✓] Category
[ ] List
---

(I think the filter menu items would need to be tri-stateable so they 
can represent "true", "false" and "don't care", so this might be complex 
to implement...)

Later: actually it looks like I started writing some of this [1], but 
didn't get very far...

[1] 
https://github.com/jon-turney/cygwin-setup/tree/split-view-filter-and-style

>>> +// Scan installed or desired packages and collect the names of packages
>>> +// which provide the dependencies of other packages or are member of
>>> +// category "Base".
>>> +static void FindNeededPackages (const packagedb & db, 
>>> std::set<std::string> & needed)
>>> +{
>>> +  std::map<std::string, std::string> providedBy;
>>> +  for (const auto & p : db.packages)
>>> +    {
>>> +      const packagemeta & pkg = *p.second;
>>> +      if (!pkg.isBinary ())
>>> +        continue;
>>> +      if (!(pkg.desired && (pkg.installed || pkg.picked ())))
>>> +        continue;
>>
>> This seems redundant.  Why can't this be just !pkg.desired?
> 
> Yes, fixed. I originally wanted to handle the "install source package 
> without the binary" case here. During development of "Ctrl+I/R/U" patch, 
> I learned that this could not happen.
> 
> 
>> This should also update the tooltip for the view dropdown 
>> (IDS_VIEWBUTTON_TOOLTIP) to describe the new view.
>>
> 
> Done.