[PATCH setup 10/10] Reserve paths starting "." for package metadata

[PATCH setup 10/10] Reserve paths starting "." for package metadata

[Jon Turney]

Reserve pathnames starting "." (i.e. dotfiles in the root directory) for
package metadata, so don't extract them.

There are no current uses of these pathnames
---
 install.cc | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/install.cc b/install.cc
index f54acdc..8aad3a5 100644
--- a/install.cc
+++ b/install.cc
@@ -471,6 +471,15 @@ Installer::installOne (packagemeta &pkgm, const packageversion &ver,
   while ((fn = tarstream->next_file_name ()).size ())
     {
       std::string canonicalfn = prefixPath + fn;
+
+      // pathnames starting "." (i.e. dotfiles in the root directory) are
+      // reserved for package metadata.  Don't extract them.
+      if (fn[0] == '.')
+        {
+          tarstream->skip_file ();
+          continue;
+        }
+
       Progress.SetText3 (canonicalfn.c_str ());
       Log (LOG_BABBLE) << "Installing file " << prefixURL << prefixPath
           << fn << endLog;
-- 
2.8.3

[PATCH setup 06/10] Remove obsolete installed_from member from packagemeta

[Jon Turney]

This just stores a made-up tarfile name read from installed.db, and is never
used.
---
 package_db.cc   |  2 +-
 package_meta.cc |  2 +-
 package_meta.h  | 11 ++---------
 3 files changed, 4 insertions(+), 11 deletions(-)

diff --git a/package_db.cc b/package_db.cc
index 73dfbf6..87da922 100644
--- a/package_db.cc
+++ b/package_db.cc
@@ -96,7 +96,7 @@ packagedb::packagedb ()
 		  packagemeta *pkg = findBinary (PackageSpecification(pkgname));
 		  if (!pkg)
 		    {
-		      pkg = new packagemeta (pkgname, inst);
+		      pkg = new packagemeta (pkgname);
 		      packages.insert (packagedb::packagecollection::value_type(pkgname, pkg));
 		      /* we should install a new handler then not check this...
 		       */
diff --git a/package_meta.cc b/package_meta.cc
index 34ff78c..21b21ef 100644
--- a/package_meta.cc
+++ b/package_meta.cc
@@ -91,7 +91,7 @@ packagemeta::_actions::caption ()
 }
 
 packagemeta::packagemeta (packagemeta const &rhs) :
-  name (rhs.name), key (rhs.name), installed_from (), 
+  name (rhs.name), key (rhs.name),
   categories (rhs.categories), versions (rhs.versions),
   installed (rhs.installed), prev (rhs.prev),
   curr (rhs.curr),
diff --git a/package_meta.h b/package_meta.h
index b24d4fc..3d6ccd2 100644
--- a/package_meta.h
+++ b/package_meta.h
@@ -35,17 +35,11 @@ public:
   static void ScanDownloadedFiles (bool);
   packagemeta (packagemeta const &);
   packagemeta (const std::string& pkgname)
-  : name (pkgname), key(pkgname), installed_from (), user_picked (false),
+  : name (pkgname), key(pkgname), user_picked (false),
     architecture (), priority(), visited_(false)
   {
   }
 
-  packagemeta (const std::string& pkgname, const std::string& installedfrom)
-  : name (pkgname), key(pkgname), installed_from (installedfrom),
-    user_picked (false), architecture (), priority(), visited_(false)
-  {
-  }
-
   ~packagemeta ();
 
   void add_version (packageversion &);
@@ -118,8 +112,7 @@ public:
 
   std::string name;			/* package name, like "cygwin" */
   std::string key;
-  /* legacy variable used to output data for installed.db versions <= 2 */
-  std::string installed_from;
+
   /* true if package was selected on command-line. */
   bool isManuallyWanted() const;
   /* true if package was deleted on command-line. */
-- 
2.8.3

[PATCH setup 00/10] Various setup patches

[Jon Turney]

Jon Turney (10):
  Remove stray execute permissions
  Prevent libtool warning that a getopt++ shared library cannot be built
  Add lex and yacc generated files to .gitignore
  Downgrade "Running preremove script" logging to debug
  Properly report progress in PrereqChecker::isMet
  Remove obsolete installed_from member from packagemeta
  Remove unused fn member from cygpackage
  Track if a package was installed by user, or as a dependency
  Add an additional filter view, showing packages which were user picked
  Reserve paths starting "." for package metadata

 .gitignore              |   3 +
 KeysSetting.cc          |   0
 KeysSetting.h           |   0
 PickView.cc             |  16 +++--
 PickView.h              |   4 +-
 crypto.cc               |   0
 crypto.h                |   0
 cyg-pubkey.h            |   0
 cygpackage.cc           |   3 -
 cygpackage.h            |   8 +--
 cygwin.pub              | Bin
 gpg-packet.cc           |   0
 gpg-packet.h            |   0
 ini.cc                  |   4 ++
 install.cc              |  11 +++-
 libgetopt++/Makefile.am |   3 +-
 package_db.cc           | 152 +++++++++++++++++++++++++++++++++++++++++-------
 package_db.h            |   3 +
 package_meta.cc         |   6 +-
 package_meta.h          |  11 +---
 prereq.cc               |   1 +
 res.rc                  |   5 +-
 tree-minus.bmp          | Bin
 tree-plus.bmp           | Bin
 24 files changed, 181 insertions(+), 49 deletions(-)
 mode change 100755 => 100644 KeysSetting.cc
 mode change 100755 => 100644 KeysSetting.h
 mode change 100755 => 100644 crypto.cc
 mode change 100755 => 100644 crypto.h
 mode change 100755 => 100644 cyg-pubkey.h
 mode change 100755 => 100644 cygwin.pub
 mode change 100755 => 100644 gpg-packet.cc
 mode change 100755 => 100644 gpg-packet.h
 mode change 100755 => 100644 tree-minus.bmp
 mode change 100755 => 100644 tree-plus.bmp

-- 
2.8.3

[PATCH setup 07/10] Remove unused fn member from cygpackage

[Jon Turney]

This is set, but never used.
---
 cygpackage.cc | 3 ---
 cygpackage.h  | 8 +++-----
 package_db.cc | 2 +-
 3 files changed, 4 insertions(+), 9 deletions(-)

diff --git a/cygpackage.cc b/cygpackage.cc
index 4b05cb3..71816f9 100644
--- a/cygpackage.cc
+++ b/cygpackage.cc
@@ -39,7 +39,6 @@ name (),
 vendor (),
 packagev (),
 canonical (),
-fn (),
 sdesc (),
 ldesc (),
 status (package_installed),
@@ -66,14 +65,12 @@ cygpackage::createInstance (const std::string& pkgname,
 
 packageversion
 cygpackage::createInstance (const std::string& pkgname,
-                            const std::string& filename,
                             const std::string& version,
 			    package_status_t const newstatus,
 			    package_type_t const newtype)
 {
   cygpackage *temp = new cygpackage;
   temp->name = pkgname;
-  temp->fn = filename;
   temp->status = newstatus;
   temp->type = newtype;
   temp->setCanonicalVersion (version);
diff --git a/cygpackage.h b/cygpackage.h
index 0b08e0d..991072a 100644
--- a/cygpackage.h
+++ b/cygpackage.h
@@ -70,10 +70,9 @@ public:
   static packageversion createInstance (const std::string& pkgname,
                                         const package_type_t type);
 
-  static packageversion createInstance (const std::string& ,
-                                        const std::string& , 
-                                        const std::string& ,
-					package_status_t const, 
+  static packageversion createInstance (const std::string& pkgname,
+                                        const std::string& version,
+					package_status_t const,
 					package_type_t const);
 
 private:
@@ -83,7 +82,6 @@ private:
   std::string vendor;
   std::string packagev;
   std::string canonical;
-  std::string fn;
   std::string sdesc, ldesc;
   char getfilenamebuffer[CYG_PATH_MAX];
 
diff --git a/package_db.cc b/package_db.cc
index 87da922..f437daf 100644
--- a/package_db.cc
+++ b/package_db.cc
@@ -105,7 +105,7 @@ packagedb::packagedb ()
 		    }
 
 		  packageversion binary = 
-		    cygpackage::createInstance (pkgname, inst, f.ver,
+		    cygpackage::createInstance (pkgname, f.ver,
 	    					package_installed,
 	    					package_binary);
 
-- 
2.8.3

[PATCH setup 04/10] Downgrade "Running preremove script" logging to debug

[Jon Turney]

This is emitted for every package, regardless of it has any scripts or not.
Actual script execution is logged separately.

Also a cosmetic fix to remove a doubled space.
---
 install.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/install.cc b/install.cc
index 2b714bc..f54acdc 100644
--- a/install.cc
+++ b/install.cc
@@ -159,7 +159,7 @@ Installer::preremoveOne (packagemeta & pkg)
 {
   Progress.SetText1 ("Running preremove script...");
   Progress.SetText2 (pkg.name.c_str());
-  Log (LOG_PLAIN) << "Running preremove script for  " << pkg.name << endLog;
+  Log (LOG_BABBLE) << "Running preremove script for " << pkg.name << endLog;
   const unsigned numexts = 4;
   const char* exts[numexts] = { ".dash", ".sh", ".bat", ".cmd" };
   for (unsigned i = 0; i < numexts; i++)
-- 
2.8.3

[PATCH setup 03/10] Add lex and yacc generated files to .gitignore

[Jon Turney]

---
 .gitignore | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/.gitignore b/.gitignore
index 2186aac..8b81166 100644
--- a/.gitignore
+++ b/.gitignore
@@ -9,3 +9,6 @@ make.out
 CVS
 tags
 autoconf.h.in*
+inilex.cc
+iniparse.cc
+iniparse.hh
-- 
2.8.3

[PATCH setup 01/10] Remove stray execute permissions

[Jon Turney]

---
 KeysSetting.cc |   0
 KeysSetting.h  |   0
 crypto.cc      |   0
 crypto.h       |   0
 cyg-pubkey.h   |   0
 cygwin.pub     | Bin
 gpg-packet.cc  |   0
 gpg-packet.h   |   0
 tree-minus.bmp | Bin
 tree-plus.bmp  | Bin
 10 files changed, 0 insertions(+), 0 deletions(-)
 mode change 100755 => 100644 KeysSetting.cc
 mode change 100755 => 100644 KeysSetting.h
 mode change 100755 => 100644 crypto.cc
 mode change 100755 => 100644 crypto.h
 mode change 100755 => 100644 cyg-pubkey.h
 mode change 100755 => 100644 cygwin.pub
 mode change 100755 => 100644 gpg-packet.cc
 mode change 100755 => 100644 gpg-packet.h
 mode change 100755 => 100644 tree-minus.bmp
 mode change 100755 => 100644 tree-plus.bmp

diff --git a/KeysSetting.cc b/KeysSetting.cc
old mode 100755
new mode 100644
diff --git a/KeysSetting.h b/KeysSetting.h
old mode 100755
new mode 100644
diff --git a/crypto.cc b/crypto.cc
old mode 100755
new mode 100644
diff --git a/crypto.h b/crypto.h
old mode 100755
new mode 100644
diff --git a/cyg-pubkey.h b/cyg-pubkey.h
old mode 100755
new mode 100644
diff --git a/cygwin.pub b/cygwin.pub
old mode 100755
new mode 100644
diff --git a/gpg-packet.cc b/gpg-packet.cc
old mode 100755
new mode 100644
diff --git a/gpg-packet.h b/gpg-packet.h
old mode 100755
new mode 100644
diff --git a/tree-minus.bmp b/tree-minus.bmp
old mode 100755
new mode 100644
diff --git a/tree-plus.bmp b/tree-plus.bmp
old mode 100755
new mode 100644
-- 
2.8.3

[PATCH setup 09/10] Add an additional filter view, showing packages which were user picked

[Jon Turney]

Add an additional filter view, showing installed packages which were
selected for installation by the user, not installed as dependencies.

Future work:

Why is PickView::views is not an enum?

The view button would make more sense as a pop-up menu, allowing a specific
filter view to be directly selected, rather than cycling around the possible
filter views (and this situation is made worse by adding another filter
view)
---
 PickView.cc | 16 ++++++++++++----
 PickView.h  |  4 ++--
 res.rc      |  5 ++++-
 3 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/PickView.cc b/PickView.cc
index 4630ee9..c784a2a 100644
--- a/PickView.cc
+++ b/PickView.cc
@@ -57,8 +57,9 @@ const PickView::views PickView::views::Unknown (0);
 const PickView::views PickView::views::PackageFull (1);
 const PickView::views PickView::views::Package (2);
 const PickView::views PickView::views::PackageKeeps (3);
-const PickView::views PickView::views::PackageSkips = PickView::views (4);
-const PickView::views PickView::views::Category (5);
+const PickView::views PickView::views::PackageSkips (4);
+const PickView::views PickView::views::PackageUserPicked (5);
+const PickView::views PickView::views::Category (6);
 
 ATOM PickView::WindowClassAtom = 0;
 
@@ -92,7 +93,8 @@ PickView::set_header_column_order (views vm)
   if (vm == views::Unknown)
     return -1;
   else if (vm == views::PackageFull || vm == views::Package
-      || vm == views::PackageKeeps || vm == views::PackageSkips)
+           || vm == views::PackageKeeps || vm == views::PackageSkips
+           || vm == views::PackageUserPicked)
     {
       headers = pkg_headers;
       current_col = 0;
@@ -197,7 +199,11 @@ PickView::setViewMode (views mode)
 
               // "Not installed"
               || (view_mode == PickView::views::PackageSkips &&
-                  (!pkg.desired && !pkg.installed)))
+                  (!pkg.desired && !pkg.installed))
+
+              // "UserPick" : installed packages that were picked by user
+              || (view_mode == PickView::views::PackageUserPicked &&
+                  (pkg.installed && pkg.user_picked)))
             {
               // Filter by package name
               if (packageFilterString.empty ()
@@ -246,6 +252,8 @@ PickView::views::caption ()
     case 4:
       return "Not Installed";
     case 5:
+      return "Picked";
+    case 6:
       return "Category";
     default:
       return "";
diff --git a/PickView.h b/PickView.h
index 0ce7581..fd20dc9 100644
--- a/PickView.h
+++ b/PickView.h
@@ -104,15 +104,15 @@ public:
     static const views Package;
     static const views PackageKeeps;
     static const views PackageSkips;
+    static const views PackageUserPicked;
     static const views Category;
-    static const views NView;
       views ():_value (0)
     {
     };
     views (int aInt)
     {
       _value = aInt;
-      if (_value < 0 || _value > 5)
+      if (_value < 0 || _value > 6)
 	_value = 0;
     }
     views & operator++ ();
diff --git a/res.rc b/res.rc
index f1cf406..de75da8 100644
--- a/res.rc
+++ b/res.rc
@@ -551,7 +551,10 @@ BEGIN
        "are at the desired version already.\r\n"
        "\r\n"
        "Not installed: Show packages that are are not currently installed "
-       "and haven't been selected for installation."
+       "and haven't been selected for installation.\r\n"
+       "\r\n"
+       "Picked: Show installed packages that were selected, not installed "
+       "as a dependency."
     IDS_HIDEOBS_TOOLTIP     "If selected, setup will hide packages in categories "
        "with names that begin with '_'.  Such packages are usually empty "
        "placeholders for packages that have been removed or renamed, or are "
-- 
2.8.3

[PATCH setup 08/10] Track if a package was installed by user, or as a dependency

[Jon Turney]

Update the installed.db file format to version 3:
- Write installed version as a version, rather than as a notional filename.
- Also write user_picked flag

This extends the semantics of user_pick somewhat: currently it is only used
for UI purposes, to record if a package was picked in the current session.

Now we also use it to record if an installed package has ever been picked
via the UI (otherwise it is only installed because it is a dependency).

So, we are careful not to set it when a currently installed package has it's
installed version adjusted via the GUI.

We also arrange for user_pick to be set when a package was selected for
installation via CLI.

Add a heuristic to initially populate user_pick when upgrading from older
installed.db formats: All non-base installed packages which aren't
dependenies are assumed to be user_pick-ed.

Note: other tools (e.g. cygcheck) which read the installed.db file will need
updating appropriately
---
 ini.cc          |   4 ++
 package_db.cc   | 148 +++++++++++++++++++++++++++++++++++++++++++++++++-------
 package_db.h    |   3 ++
 package_meta.cc |   4 +-
 4 files changed, 140 insertions(+), 19 deletions(-)

diff --git a/ini.cc b/ini.cc
index 456eb6e..f925bf5 100644
--- a/ini.cc
+++ b/ini.cc
@@ -51,6 +51,7 @@
 #include "compress.h"
 #include "Exception.h"
 #include "crypto.h"
+#include "package_db.h"
 
 extern ThreeBarProgressPage Progress;
 
@@ -351,6 +352,9 @@ do_ini_thread (HINSTANCE h, HWND owner)
   else
     ini_count = do_remote_ini (owner);
 
+  packagedb db;
+  db.upgrade();
+
   if (ini_count == 0)
     return false;
 
diff --git a/package_db.cc b/package_db.cc
index f437daf..18f4f37 100644
--- a/package_db.cc
+++ b/package_db.cc
@@ -42,6 +42,7 @@ static const char *cvsid =
 #include "package_meta.h"
 #include "Exception.h"
 #include "Generic.h"
+#include "LogSingleton.h"
 
 using namespace std;
 
@@ -55,23 +56,26 @@ packagedb::packagedb ()
       installeddbread = 1;
       if (!db)
 	return;
-      /* flush_local_db_package_data */
-      char line[1000], pkgname[1000], inst[1000];
-      int instsz;
+      char line[1000], pkgname[1000];
 
       if (db->gets (line, 1000))
 	{
+	  /* Look for header line (absent in version 1) */
+	  int instsz;
 	  int dbver;
 	  sscanf (line, "%s %d", pkgname, &instsz);
-	  if (!strcasecmp (pkgname, "INSTALLED.DB") && instsz == 2)
-	    dbver = 2;
+	  if (!strcasecmp (pkgname, "INSTALLED.DB") && instsz <= 3)
+	    dbver = instsz;
 	  else
 	    dbver = 1;
 	  delete db;
 	  db = 0;
+
 	  /* Later versions may not use installed.db other than to record the version. */
 	  if (dbver == 1 || dbver == 2)
 	    {
+	      char inst[1000];
+
 	      db =
 		io_stream::open ("cygfile:///etc/setup/installed.db", "rt", 0);
 	      if (dbver == 2)
@@ -106,19 +110,63 @@ packagedb::packagedb ()
 
 		  packageversion binary = 
 		    cygpackage::createInstance (pkgname, f.ver,
-	    					package_installed,
-	    					package_binary);
+						package_installed,
+						package_binary);
+
+		  pkg->add_version (binary);
+		  pkg->set_installed (binary);
+		  pkg->desired = pkg->installed;
+		}
+	      delete db;
+	      db = 0;
+	    }
+	  else if (dbver == 3)
+	    {
+	      char ver[1000];
+
+	      db = io_stream::open ("cygfile:///etc/setup/installed.db", "rt", 0);
+
+	      // skip over already-parsed header line
+	      db->gets (line, 1000);
+
+	      while (db->gets (line, 1000))
+		{
+		  pkgname[0] = '\0';
+		  ver[0] = '\0';
+		  int user_picked = 0;
+
+		  /*
+		     In an INSTALLED.DB 3, the lines consist of:
+		     packagename installed-version user-picked
+		   */
+		  int res = sscanf (line, "%s %s %d", pkgname, ver, &user_picked);
 
+		  if (res < 3 || pkgname[0] == '\0' || ver[0] == '\0')
+			continue;
+
+		  packagemeta *pkg = findBinary (PackageSpecification(pkgname));
+		  if (!pkg)
+		    {
+		      pkg = new packagemeta (pkgname);
+		      packages.insert (packagedb::packagecollection::value_type(pkgname, pkg));
+		    }
+
+		  packageversion binary = cygpackage::createInstance (pkgname, ver, package_installed, package_binary);
 		  pkg->add_version (binary);
 		  pkg->set_installed (binary);
 		  pkg->desired = pkg->installed;
+		  pkg->user_picked = (user_picked != 0);
 		}
 	      delete db;
 	      db = 0;
 	    }
 	  else
-	    // unknown dbversion
-	    exit (1);
+	    {
+	      Log (LOG_PLAIN) << "unknown INSTALLED.DB version " << dbver << endLog;
+	      exit (1);
+	    }
+
+	  installeddbver = dbver;
 	}
     }
 }
@@ -138,21 +186,17 @@ packagedb::flush ()
   if (!ndb)
     return errno ? errno : 1;
 
-  ndb->write ("INSTALLED.DB 2\n", strlen ("INSTALLED.DB 2\n"));
+  ndb->write ("INSTALLED.DB 3\n", strlen ("INSTALLED.DB 3\n"));
   for (packagedb::packagecollection::iterator i = packages.begin ();
        i != packages.end (); ++i)
     {
       packagemeta & pkgm = *(i->second);
       if (pkgm.installed)
 	{
-	  /* size here is irrelevant - as we can assume that this install source
-	   * no longer exists, and it does not correlate to used disk space
-	   * also note that we are writing a fictional install source 
-	   * to keep cygcheck happy.               
-	   */
 	  std::string line;
-	  line = pkgm.name + " " + pkgm.name + "-" + 
-	    std::string(pkgm.installed.Canonical_version()) + ".tar.bz2 0\n";
+	  line = pkgm.name + " " +
+	    std::string(pkgm.installed.Canonical_version()) + " " +
+	    (pkgm.user_picked ? "1" : "0") + "\n";
 	  ndb->write (line.c_str(), line.size());
 	}
     }
@@ -166,6 +210,18 @@ packagedb::flush ()
   return 0;
 }
 
+void
+packagedb::upgrade()
+{
+  if (installeddbver < 3)
+    {
+      /* Guess which packages were user_picked.  This has to take place after
+         setup.ini has been parsed as it needs dependency information. */
+      guessUserPicked();
+      installeddbver = 3;
+    }
+}
+
 packagemeta *
 packagedb::findBinary (PackageSpecification const &spec) const
 {
@@ -199,13 +255,13 @@ packagedb::findSource (PackageSpecification const &spec) const
 /* static members */
 
 int packagedb::installeddbread = 0;
+int packagedb::installeddbver = 0;
 packagedb::packagecollection packagedb::packages;
 packagedb::categoriesType packagedb::categories;
 packagedb::packagecollection packagedb::sourcePackages;
 PackageDBActions packagedb::task = PackageDB_Install;
 std::vector <packagemeta *> packagedb::dependencyOrderedPackages;
 
-#include "LogSingleton.h"
 #include <stack>
 
 class
@@ -449,3 +505,59 @@ packagedb::defaultTrust (trusts trust)
         packagedb::categories.erase (n++);
       }
 }
+
+void
+packagedb::guessUserPicked()
+{
+  /*
+    Assume that any non-base installed package which is a dependency of an
+    installed package wasn't user_picked
+
+    i.e. only installed packages which aren't in the base category, and aren't
+    a dependency of any installed package are user_picked
+  */
+
+  /* First mark all installed non-base packages */
+  for (packagedb::packagecollection::iterator i = packages.begin ();
+       i != packages.end (); ++i)
+    {
+      packagemeta & pkgm = *(i->second);
+
+      if (pkgm.categories.find ("Base") != pkgm.categories.end ())
+	continue;
+
+      if (pkgm.installed)
+	pkgm.user_picked = TRUE;
+    }
+
+  /* Then clear the mark for all dependencies of all installed packages */
+  for (packagedb::packagecollection::iterator i = packages.begin ();
+       i != packages.end (); ++i)
+    {
+      packagemeta & pkgm = *(i->second);
+
+      if (!pkgm.installed)
+	continue;
+
+      /* walk through each and clause */
+      vector <vector <PackageSpecification *> *>::const_iterator dp = pkgm.installed.depends()->begin();
+      while (dp != pkgm.installed.depends()->end())
+	{
+	  /* check each or clause for an installed match */
+	  vector <PackageSpecification *>::const_iterator i = find_if ((*dp)->begin(), (*dp)->end(), checkForInstalled);
+	  if (i != (*dp)->end())
+	    {
+	      const packagedb::packagecollection::iterator n = packages.find((*i)->packageName());
+	      if (n != packages.end())
+		{
+		  packagemeta *pkgm2 = n->second;
+		  pkgm2->user_picked = FALSE;
+		}
+	      /* skip to next and clause */
+	      ++dp;
+	      continue;
+	    }
+	  ++dp;
+	}
+    }
+}
diff --git a/package_db.h b/package_db.h
index bc828a1..6a99398 100644
--- a/package_db.h
+++ b/package_db.h
@@ -65,6 +65,7 @@ public:
   packagedb ();
   /* 0 on success */
   int flush ();
+  void upgrade ();
   packagemeta * findBinary (PackageSpecification const &) const;
   packagemeta * findSource (PackageSpecification const &) const;
   PackageDBConnectedIterator connectedBegin();
@@ -84,8 +85,10 @@ public:
   static PackageDBActions task;
 private:
   static int installeddbread;	/* do we have to reread this */
+  static int installeddbver;
   friend class ConnectedLoopFinder;
   static std::vector <packagemeta *> dependencyOrderedPackages;
+  void guessUserPicked(void);
 };
 
 #endif /* SETUP_PACKAGE_DB_H */
diff --git a/package_meta.cc b/package_meta.cc
index 21b21ef..3923b13 100644
--- a/package_meta.cc
+++ b/package_meta.cc
@@ -458,7 +458,8 @@ packagemeta::set_action (trusts const trust)
   else
     desired = packageversion ();
   /* Memorize the fact that the user picked at least once. */
-  user_picked = true;
+  if (!installed)
+    user_picked = true;
 }
 
 int
@@ -510,6 +511,7 @@ packagemeta::set_action (_actions action, packageversion const &default_version)
 	  if (desired != installed)
 	    if (desired.accessible ())
 	      {
+		user_picked = true;
 		desired.pick (true, this);
 		desired.sourcePackage ().pick (false, NULL);
 	      }
-- 
2.8.3

[PATCH setup 05/10] Properly report progress in PrereqChecker::isMet

[Jon Turney]

Properly report progress in PrereqChecker::isMet after additional dependency
work is found.
---
 prereq.cc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/prereq.cc b/prereq.cc
index bdc609e..a5083ed 100644
--- a/prereq.cc
+++ b/prereq.cc
@@ -238,6 +238,7 @@ PrereqChecker::isMet ()
                     {
                       // newly found dependency: add to worklist
                       todo.push (dep);
+                      max++;
                     }
                   unmet[dep].push_back (pack);
                 }
-- 
2.8.3

[PATCH setup 02/10] Prevent libtool warning that a getopt++ shared library cannot be built

[Jon Turney]

libtool: warning: undefined symbols not allowed in i686-w64-mingw32 shared libraries; building static only
---
 libgetopt++/Makefile.am | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/libgetopt++/Makefile.am b/libgetopt++/Makefile.am
index f820d71..34dc6fd 100644
--- a/libgetopt++/Makefile.am
+++ b/libgetopt++/Makefile.am
@@ -28,7 +28,7 @@ libgetopt___la_SOURCES = src/GetOption.cc src/Option.cc src/BoolOption.cc \
 	src/OptionSet.cc \
 	src/StringArrayOption.cc src/StringOption.cc
 
-libgetopt___la_LDFLAGS = -version-info 1:1:0
+libgetopt___la_LDFLAGS = -version-info 1:1:0 -no-undefined
 
 getoptinclude_HEADERS = include/getopt++/Option.h \
   include/getopt++/BoolOption.h \
@@ -49,4 +49,3 @@ tests_BoolOptionTest_LDADD = libgetopt++.la
 
 tests_OptionSet_SOURCES = tests/OptionSet.cc
 tests_OptionSet_LDADD = libgetopt++.la
-
-- 
2.8.3

Re: [PATCH setup 00/10] Various setup patches

[Achim Gratz]

Jon Turney writes:
>   Track if a package was installed by user, or as a dependency
>   Add an additional filter view, showing packages which were user picked

As a suggestion (and I won't have time for implementation help at the
moment): Please consider keeping /etc/setup/installed.db at version 2
and instead move the new-style database(s) to somewhere under
/var/setup.  For some time we would have to generate both the old and
new databases from setup of course until everything has switched over to
the new locations.  The format of the new database is up for discussion
I think, but besides the distinction between picked and non-picked I
think there should be a way to record version locks or preferences for
prev/curr/test.

I would also like to add checksums to the package lists, provided we can
find a way to ignore the changes due to rebasing, so it becomes easier to
audit an installation for changes.

>   Reserve paths starting "." for package metadata

What did you envision here?  In general I like the idea, but when we
start to have a structured package format I think we should move to some
other naming convention than .tar.xz, like .cyg or .cpm perhaps.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: [PATCH setup 00/10] Various setup patches

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2901 bytes --]

On Aug  3 09:10, Achim Gratz wrote:
> Jon Turney writes:
> >   Track if a package was installed by user, or as a dependency
> >   Add an additional filter view, showing packages which were user picked
> 
> As a suggestion (and I won't have time for implementation help at the
> moment): Please consider keeping /etc/setup/installed.db at version 2
> and instead move the new-style database(s) to somewhere under
> /var/setup.

*If* we do that, the setup files should go under /var/lib/setup.

However, why would this make sense exactly?  Yes, I know LSB and yada
yada, but why *exactly* would that make sense in *our* situation?

> For some time we would have to generate both the old and
> new databases from setup of course until everything has switched over to
> the new locations. 

Moving from /etc/ to /var requires to move all files.  It's useless
if the lst files stay in /etc while the db and rc files go to /var.

That means, a move to /var requires to copy/move the lst files over in a
single step, even (especially) the old ones, otherwise you'd have a
broken database, with files in /etc and /var.

Who's going to do this?  Setup itself would have to do it since a
post-install script would be too late.  So you'd have to write code
into setup for just this move to /var.  Code which runs exactly once.

And then writing *both* at the same time?  What packages are the
consumers of the data in /etc/setup?  AFAICS, cygwin itself (cygcheck),
cygcheck-dep, and _autorebase only.

Wouldn't it make more sense either to stick to /etc/setup, or to
change all three packages to check for /var/lib/setup and use that
if it exists, /etc/setup otherwise?

> The format of the new database is up for discussion
> I think, but besides the distinction between picked and non-picked I
> think there should be a way to record version locks or preferences for
> prev/curr/test.

I think the old "size" field should become a flag field instead, with
picked/non-picked having the bit value 1.  That covers a lot of info
already.

> I would also like to add checksums to the package lists, provided we can
> find a way to ignore the changes due to rebasing, so it becomes easier to
> audit an installation for changes.

Feel free to add stuff.  Just make sure the (hopefully only) three
dependent packages can work with the new format before we introduce the
new format.

> >   Reserve paths starting "." for package metadata
> 
> What did you envision here?  In general I like the idea, but when we
> start to have a structured package format I think we should move to some
> other naming convention than .tar.xz, like .cyg or .cpm perhaps.

.cpm sounds a bit... old-fashioned ;)


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH setup 00/10] Various setup patches

[Achim Gratz]

Corinna Vinschen writes:
> *If* we do that, the setup files should go under /var/lib/setup.

Yes.

> However, why would this make sense exactly?  Yes, I know LSB and yada
> yada, but why *exactly* would that make sense in *our* situation?

In this case it's just a clean way to separate the old and new databases.

>> For some time we would have to generate both the old and
>> new databases from setup of course until everything has switched over to
>> the new locations. 
>
> Moving from /etc/ to /var requires to move all files.  It's useless
> if the lst files stay in /etc while the db and rc files go to /var.

The way we're installing at the moment we could just hardlink.

> And then writing *both* at the same time?  What packages are the
> consumers of the data in /etc/setup?  AFAICS, cygwin itself (cygcheck),
> cygcheck-dep, and _autorebase only.
>
> Wouldn't it make more sense either to stick to /etc/setup, or to
> change all three packages to check for /var/lib/setup and use that
> if it exists, /etc/setup otherwise?

Maybe we could just rename installed.db to installed.db3 or seomthing
similar.

>> The format of the new database is up for discussion
>> I think, but besides the distinction between picked and non-picked I
>> think there should be a way to record version locks or preferences for
>> prev/curr/test.
>
> I think the old "size" field should become a flag field instead, with
> picked/non-picked having the bit value 1.  That covers a lot of info
> already.

Yes, but you'll still have to come up with some encoding and it would be
awfully nice if the new file format was a bit more forward-thinking when
it comes to extensibility.

> Feel free to add stuff.  Just make sure the (hopefully only) three
> dependent packages can work with the new format before we introduce the
> new format.

That would be either supplemental files with the hashes or some new .lst
format which could/should use a different extension anyway since the
transition period will be long.

>> >   Reserve paths starting "." for package metadata
>> 
>> What did you envision here?  In general I like the idea, but when we
>> start to have a structured package format I think we should move to some
>> other naming convention than .tar.xz, like .cyg or .cpm perhaps.
>
> .cpm sounds a bit... old-fashioned ;)

I still have one of these, not that I've run it in the last few years:
http://www.robotron-net.de/pc_s.html#BIC

Too "boot" the CP/M clone you just have to insert a floppy with an emtpy
file SCPX5105.SYS since actually it's in ROM (otherwise it starts
BASIC).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: [PATCH setup 08/10] Track if a package was installed by user, or as a dependency

[Jon Turney]

[-- Attachment #1: Type: text/plain, Size: 138 bytes --]


Attached is an updated version of the [08/10] patch which doesn't 
gratuitously break compatibility with other readers of installed.db.


[-- Attachment #2: 0008-Track-if-a-package-was-installed-by-user-or-as-a-dep.patch --]
[-- Type: text/plain, Size: 10909 bytes --]

From 280d51aa8c159164637039e7a4de47eb53b4fe39 Mon Sep 17 00:00:00 2001
From: Jon Turney <jon.turney@dronecode.org.uk>
Date: Tue, 2 Aug 2016 12:19:09 +0100
Subject: [PATCH setup 08/10] Track if a package was installed by user, or as a
 dependency

Update the installed.db file format to version 3, containing the user_picked
flag.

This extends the semantics of user_pick somewhat: currently it is only used
for UI purposes, to record if a package was picked in the current session.

Now we also use it to record if an installed package has ever been picked
via the UI (otherwise it is only installed because it is a dependency).

So, we are careful not to set it when a currently installed package has it's
installed version adjusted via the GUI.

We also arrange for user_pick to be set when a package was selected for
installation via CLI.

Add a heuristic to initially populate user_pick when upgrading from older
installed.db formats: All non-base installed packages which aren't
dependenies are assumed to be user_pick-ed.

Note: other tools (e.g. cygcheck) which read the installed.db file will need
updating appropriately

v2:
Don't gratuitously break compatiblity with other existing readers of
installed.db, keep recording version as a notional filename.

Don't treat unknown future installed.db format versions as format version 1,
instead treat setup downgrading as a fatal error.
---
 ini.cc          |   4 ++
 package_db.cc   | 136 ++++++++++++++++++++++++++++++++++++++++++++------------
 package_db.h    |   3 ++
 package_meta.cc |   4 +-
 res.rc          |   1 +
 resource.h      |   1 +
 6 files changed, 120 insertions(+), 29 deletions(-)

diff --git a/ini.cc b/ini.cc
index 456eb6e..f925bf5 100644
--- a/ini.cc
+++ b/ini.cc
@@ -51,6 +51,7 @@
 #include "compress.h"
 #include "Exception.h"
 #include "crypto.h"
+#include "package_db.h"
 
 extern ThreeBarProgressPage Progress;
 
@@ -351,6 +352,9 @@ do_ini_thread (HINSTANCE h, HWND owner)
   else
     ini_count = do_remote_ini (owner);
 
+  packagedb db;
+  db.upgrade();
+
   if (ini_count == 0)
     return false;
 
diff --git a/package_db.cc b/package_db.cc
index f437daf..79f2f93 100644
--- a/package_db.cc
+++ b/package_db.cc
@@ -42,6 +42,8 @@ static const char *cvsid =
 #include "package_meta.h"
 #include "Exception.h"
 #include "Generic.h"
+#include "LogSingleton.h"
+#include "resource.h"
 
 using namespace std;
 
@@ -55,37 +57,44 @@ packagedb::packagedb ()
       installeddbread = 1;
       if (!db)
 	return;
-      /* flush_local_db_package_data */
-      char line[1000], pkgname[1000], inst[1000];
-      int instsz;
+      char line[1000], pkgname[1000];
 
       if (db->gets (line, 1000))
 	{
+	  /* Look for header line (absent in version 1) */
+	  int instsz;
 	  int dbver;
 	  sscanf (line, "%s %d", pkgname, &instsz);
-	  if (!strcasecmp (pkgname, "INSTALLED.DB") && instsz == 2)
-	    dbver = 2;
+	  if (!strcasecmp (pkgname, "INSTALLED.DB"))
+	    dbver = instsz;
 	  else
 	    dbver = 1;
 	  delete db;
 	  db = 0;
-	  /* Later versions may not use installed.db other than to record the version. */
-	  if (dbver == 1 || dbver == 2)
+
+	  Log (LOG_BABBLE) << "INSTALLED.DB version " << dbver << endLog;
+
+	  if (dbver <= 3)
 	    {
+	      char inst[1000];
+
 	      db =
 		io_stream::open ("cygfile:///etc/setup/installed.db", "rt", 0);
-	      if (dbver == 2)
+
+	      // skip over already-parsed header line
+	      if (dbver >= 2)
 		db->gets (line, 1000);
+
 	      while (db->gets (line, 1000))
 		{
 		  int parseable;
-		  int ign;
+		  int user_picked = 0;
 		  pkgname[0] = '\0';
 		  inst[0] = '\0';
 
-		  sscanf (line, "%s %s %d", pkgname, inst, &ign);
+		  int res = sscanf (line, "%s %s %d", pkgname, inst, &user_picked);
 
-		  if (pkgname[0] == '\0' || inst[0] == '\0')
+		  if (res < 3 || pkgname[0] == '\0' || inst[0] == '\0')
 			continue;
 
 		  fileparse f;
@@ -98,27 +107,29 @@ packagedb::packagedb ()
 		    {
 		      pkg = new packagemeta (pkgname);
 		      packages.insert (packagedb::packagecollection::value_type(pkgname, pkg));
-		      /* we should install a new handler then not check this...
-		       */
-		      //if (!pkg)
-		      //die badly
 		    }
 
 		  packageversion binary = 
 		    cygpackage::createInstance (pkgname, f.ver,
-	    					package_installed,
-	    					package_binary);
+						package_installed,
+						package_binary);
 
 		  pkg->add_version (binary);
 		  pkg->set_installed (binary);
 		  pkg->desired = pkg->installed;
+
+		  if (dbver == 3)
+		    pkg->user_picked = (user_picked != 0);
 		}
 	      delete db;
 	      db = 0;
 	    }
 	  else
-	    // unknown dbversion
-	    exit (1);
+	    {
+              fatal(NULL, IDS_INSTALLEDB_VERSION);
+	    }
+
+	  installeddbver = dbver;
 	}
     }
 }
@@ -138,21 +149,22 @@ packagedb::flush ()
   if (!ndb)
     return errno ? errno : 1;
 
-  ndb->write ("INSTALLED.DB 2\n", strlen ("INSTALLED.DB 2\n"));
+  ndb->write ("INSTALLED.DB 3\n", strlen ("INSTALLED.DB 3\n"));
   for (packagedb::packagecollection::iterator i = packages.begin ();
        i != packages.end (); ++i)
     {
       packagemeta & pkgm = *(i->second);
       if (pkgm.installed)
 	{
-	  /* size here is irrelevant - as we can assume that this install source
-	   * no longer exists, and it does not correlate to used disk space
-	   * also note that we are writing a fictional install source 
-	   * to keep cygcheck happy.               
-	   */
+          /*
+            In INSTALLED.DB 3, lines are: 'packagename version user-picked',
+            where version is encoded in a notional filename for backwards
+            compatibility.
+          */
 	  std::string line;
-	  line = pkgm.name + " " + pkgm.name + "-" + 
-	    std::string(pkgm.installed.Canonical_version()) + ".tar.bz2 0\n";
+	  line = pkgm.name + " " +
+	    pkgm.name + "-" + std::string(pkgm.installed.Canonical_version()) + ".tar.bz2 " +
+	    (pkgm.user_picked ? "1" : "0") + "\n";
 	  ndb->write (line.c_str(), line.size());
 	}
     }
@@ -166,6 +178,18 @@ packagedb::flush ()
   return 0;
 }
 
+void
+packagedb::upgrade()
+{
+  if (installeddbver < 3)
+    {
+      /* Guess which packages were user_picked.  This has to take place after
+         setup.ini has been parsed as it needs dependency information. */
+      guessUserPicked();
+      installeddbver = 3;
+    }
+}
+
 packagemeta *
 packagedb::findBinary (PackageSpecification const &spec) const
 {
@@ -199,13 +223,13 @@ packagedb::findSource (PackageSpecification const &spec) const
 /* static members */
 
 int packagedb::installeddbread = 0;
+int packagedb::installeddbver = 0;
 packagedb::packagecollection packagedb::packages;
 packagedb::categoriesType packagedb::categories;
 packagedb::packagecollection packagedb::sourcePackages;
 PackageDBActions packagedb::task = PackageDB_Install;
 std::vector <packagemeta *> packagedb::dependencyOrderedPackages;
 
-#include "LogSingleton.h"
 #include <stack>
 
 class
@@ -449,3 +473,59 @@ packagedb::defaultTrust (trusts trust)
         packagedb::categories.erase (n++);
       }
 }
+
+void
+packagedb::guessUserPicked()
+{
+  /*
+    Assume that any non-base installed package which is a dependency of an
+    installed package wasn't user_picked
+
+    i.e. only installed packages which aren't in the base category, and aren't
+    a dependency of any installed package are user_picked
+  */
+
+  /* First mark all installed non-base packages */
+  for (packagedb::packagecollection::iterator i = packages.begin ();
+       i != packages.end (); ++i)
+    {
+      packagemeta & pkgm = *(i->second);
+
+      if (pkgm.categories.find ("Base") != pkgm.categories.end ())
+	continue;
+
+      if (pkgm.installed)
+	pkgm.user_picked = TRUE;
+    }
+
+  /* Then clear the mark for all dependencies of all installed packages */
+  for (packagedb::packagecollection::iterator i = packages.begin ();
+       i != packages.end (); ++i)
+    {
+      packagemeta & pkgm = *(i->second);
+
+      if (!pkgm.installed)
+	continue;
+
+      /* walk through each and clause */
+      vector <vector <PackageSpecification *> *>::const_iterator dp = pkgm.installed.depends()->begin();
+      while (dp != pkgm.installed.depends()->end())
+	{
+	  /* check each or clause for an installed match */
+	  vector <PackageSpecification *>::const_iterator i = find_if ((*dp)->begin(), (*dp)->end(), checkForInstalled);
+	  if (i != (*dp)->end())
+	    {
+	      const packagedb::packagecollection::iterator n = packages.find((*i)->packageName());
+	      if (n != packages.end())
+		{
+		  packagemeta *pkgm2 = n->second;
+		  pkgm2->user_picked = FALSE;
+		}
+	      /* skip to next and clause */
+	      ++dp;
+	      continue;
+	    }
+	  ++dp;
+	}
+    }
+}
diff --git a/package_db.h b/package_db.h
index bc828a1..6a99398 100644
--- a/package_db.h
+++ b/package_db.h
@@ -65,6 +65,7 @@ public:
   packagedb ();
   /* 0 on success */
   int flush ();
+  void upgrade ();
   packagemeta * findBinary (PackageSpecification const &) const;
   packagemeta * findSource (PackageSpecification const &) const;
   PackageDBConnectedIterator connectedBegin();
@@ -84,8 +85,10 @@ public:
   static PackageDBActions task;
 private:
   static int installeddbread;	/* do we have to reread this */
+  static int installeddbver;
   friend class ConnectedLoopFinder;
   static std::vector <packagemeta *> dependencyOrderedPackages;
+  void guessUserPicked(void);
 };
 
 #endif /* SETUP_PACKAGE_DB_H */
diff --git a/package_meta.cc b/package_meta.cc
index 21b21ef..3923b13 100644
--- a/package_meta.cc
+++ b/package_meta.cc
@@ -458,7 +458,8 @@ packagemeta::set_action (trusts const trust)
   else
     desired = packageversion ();
   /* Memorize the fact that the user picked at least once. */
-  user_picked = true;
+  if (!installed)
+    user_picked = true;
 }
 
 int
@@ -510,6 +511,7 @@ packagemeta::set_action (_actions action, packageversion const &default_version)
 	  if (desired != installed)
 	    if (desired.accessible ())
 	      {
+		user_picked = true;
 		desired.pick (true, this);
 		desired.sourcePackage ().pick (false, NULL);
 	      }
diff --git a/res.rc b/res.rc
index f1cf406..8292750 100644
--- a/res.rc
+++ b/res.rc
@@ -569,4 +569,5 @@ BEGIN
     IDS_NO_LOCALDIR    "Local package directory %s not found.\nYou can still use setup-%s.exe to remove installed\npackages, but there "
       "will be nothing to install.\n\nPress OK if that's what you wanted\nor Cancel to choose a different directory."
     IDS_ELEVATED       "Hand installation over to elevated child process."
+    IDS_INSTALLEDB_VERSION "Unknown INSTALLED.DB version"
 END
diff --git a/resource.h b/resource.h
index 48f03ab..68e8023 100644
--- a/resource.h
+++ b/resource.h
@@ -39,6 +39,7 @@
 #define IDS_CANT_MKDIR                    137
 #define IDS_NO_LOCALDIR			  138
 #define IDS_ELEVATED			  139
+#define IDS_INSTALLEDB_VERSION            140
 
 // Dialogs
 
-- 
2.8.3

Re: [PATCH setup 00/10] Various setup patches

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2276 bytes --]

On Aug  2 16:30, Jon Turney wrote:
> Jon Turney (10):
>   Remove stray execute permissions
>   Prevent libtool warning that a getopt++ shared library cannot be built
>   Add lex and yacc generated files to .gitignore
>   Downgrade "Running preremove script" logging to debug
>   Properly report progress in PrereqChecker::isMet
>   Remove obsolete installed_from member from packagemeta
>   Remove unused fn member from cygpackage
>   Track if a package was installed by user, or as a dependency
>   Add an additional filter view, showing packages which were user picked
>   Reserve paths starting "." for package metadata
> 
>  .gitignore              |   3 +
>  KeysSetting.cc          |   0
>  KeysSetting.h           |   0
>  PickView.cc             |  16 +++--
>  PickView.h              |   4 +-
>  crypto.cc               |   0
>  crypto.h                |   0
>  cyg-pubkey.h            |   0
>  cygpackage.cc           |   3 -
>  cygpackage.h            |   8 +--
>  cygwin.pub              | Bin
>  gpg-packet.cc           |   0
>  gpg-packet.h            |   0
>  ini.cc                  |   4 ++
>  install.cc              |  11 +++-
>  libgetopt++/Makefile.am |   3 +-
>  package_db.cc           | 152 +++++++++++++++++++++++++++++++++++++++++-------
>  package_db.h            |   3 +
>  package_meta.cc         |   6 +-
>  package_meta.h          |  11 +---
>  prereq.cc               |   1 +
>  res.rc                  |   5 +-
>  tree-minus.bmp          | Bin
>  tree-plus.bmp           | Bin
>  24 files changed, 181 insertions(+), 49 deletions(-)
>  mode change 100755 => 100644 KeysSetting.cc
>  mode change 100755 => 100644 KeysSetting.h
>  mode change 100755 => 100644 crypto.cc
>  mode change 100755 => 100644 crypto.h
>  mode change 100755 => 100644 cyg-pubkey.h
>  mode change 100755 => 100644 cygwin.pub
>  mode change 100755 => 100644 gpg-packet.cc
>  mode change 100755 => 100644 gpg-packet.h
>  mode change 100755 => 100644 tree-minus.bmp
>  mode change 100755 => 100644 tree-plus.bmp

ACK for the series with v2 of patch 8.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH setup 00/10] Various setup patches

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3596 bytes --]

On Aug  3 11:51, Achim Gratz wrote:
> Corinna Vinschen writes:
> > *If* we do that, the setup files should go under /var/lib/setup.
> 
> Yes.
> 
> > However, why would this make sense exactly?  Yes, I know LSB and yada
> > yada, but why *exactly* would that make sense in *our* situation?
> 
> In this case it's just a clean way to separate the old and new databases.
> 
> >> For some time we would have to generate both the old and
> >> new databases from setup of course until everything has switched over to
> >> the new locations. 
> >
> > Moving from /etc/ to /var requires to move all files.  It's useless
> > if the lst files stay in /etc while the db and rc files go to /var.
> 
> The way we're installing at the moment we could just hardlink.

If the filesystem supports it.  We *might* get away with the assumption
the underlying filesystem is NTFS or similar, but even a newer FS like
ReFS has no hardlinks :(

> > And then writing *both* at the same time?  What packages are the
> > consumers of the data in /etc/setup?  AFAICS, cygwin itself (cygcheck),
> > cygcheck-dep, and _autorebase only.
> >
> > Wouldn't it make more sense either to stick to /etc/setup, or to
> > change all three packages to check for /var/lib/setup and use that
> > if it exists, /etc/setup otherwise?
> 
> Maybe we could just rename installed.db to installed.db3 or seomthing
> similar.

In the current state of Jon's patch that shouldn't be necessary, but
whatever you do, you don't drop the requirement that tools like
cygcheck, cygcheck-dep or _autorebase have to adapt.

> >> The format of the new database is up for discussion
> >> I think, but besides the distinction between picked and non-picked I
> >> think there should be a way to record version locks or preferences for
> >> prev/curr/test.
> >
> > I think the old "size" field should become a flag field instead, with
> > picked/non-picked having the bit value 1.  That covers a lot of info
> > already.
> 
> Yes, but you'll still have to come up with some encoding and it would be
> awfully nice if the new file format was a bit more forward-thinking when
> it comes to extensibility.

I don't think that's really necessary as long as you *add* information.
What's really important is to check and, if required, change cygcheck
and friends to be able to skip information they don't evaluate, rather
than choking on it.

> > Feel free to add stuff.  Just make sure the (hopefully only) three
> > dependent packages can work with the new format before we introduce the
> > new format.
> 
> That would be either supplemental files with the hashes or some new .lst
> format which could/should use a different extension anyway since the
> transition period will be long.

Why?  The transition period can be very much shortened if we do what
I wrote above.

> >> >   Reserve paths starting "." for package metadata
> >> 
> >> What did you envision here?  In general I like the idea, but when we
> >> start to have a structured package format I think we should move to some
> >> other naming convention than .tar.xz, like .cyg or .cpm perhaps.

In terms of all of the above, I'd like to see some input from Jon, Yaakov
et al.

> > .cpm sounds a bit... old-fashioned ;)
> 
> I still have one of these, not that I've run it in the last few years:
> http://www.robotron-net.de/pc_s.html#BIC

Wow!


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH setup 00/10] Various setup patches

[Achim Gratz]

Corinna Vinschen writes:
>> That would be either supplemental files with the hashes or some new .lst
>> format which could/should use a different extension anyway since the
>> transition period will be long.
>
> Why?  The transition period can be very much shortened if we do what
> I wrote above.

People tend to not re-install their whole set of packages just because
some new version of setup is announced, so I'm going to assume that for
quite some time a mix of old and new .lst files (for instance) exists on
the majority of installations and whatever we do (in cygcheck, say)
needs to work with that.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [PATCH setup 00/10] Various setup patches

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1174 bytes --]

On Aug  3 20:27, Achim Gratz wrote:
> Corinna Vinschen writes:
> >> That would be either supplemental files with the hashes or some new .lst
> >> format which could/should use a different extension anyway since the
> >> transition period will be long.
> >
> > Why?  The transition period can be very much shortened if we do what
> > I wrote above.
> 
> People tend to not re-install their whole set of packages just because
> some new version of setup is announced,

Uhm?  If you download a new setup, but then don't update your packages,
why did you download the latest setup at all?  If you don't run this
new setup, you won't get new-style files.

> so I'm going to assume that for
> quite some time a mix of old and new .lst files (for instance) exists on
> the majority of installations and whatever we do (in cygcheck, say)
> needs to work with that.

1. Provide new versions of cygwin, cygcheck-dep and _autorebase
2. time passes (2 weeks or so)
3. Provide a new version of setup


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH setup 00/10] Various setup patches

[Achim Gratz]

Corinna Vinschen writes:
>> People tend to not re-install their whole set of packages just because
>> some new version of setup is announced,
>
> Uhm?  If you download a new setup, but then don't update your packages,
> why did you download the latest setup at all?  If you don't run this
> new setup, you won't get new-style files.

The checksum information has to come from somewhere and that somewhere
requires a package install or update.  Together with that new setup we might
have an update of cygccheck and some unrelated packages that happen to
have been rebuilt recently, but certainly not the whole distribution.

>> so I'm going to assume that for
>> quite some time a mix of old and new .lst files (for instance) exists on
>> the majority of installations and whatever we do (in cygcheck, say)
>> needs to work with that.
>
> 1. Provide new versions of cygwin, cygcheck-dep and _autorebase
> 2. time passes (2 weeks or so)
> 3. Provide a new version of setup

That doesn't cut it, unless you want to require the whole distribution
to be rebuilt and everything re-installed with that change.  Cygwin10
1609 or something like that?  :-)


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [PATCH setup 00/10] Various setup patches

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1679 bytes --]

On Aug  3 21:52, Achim Gratz wrote:
> Corinna Vinschen writes:
> >> People tend to not re-install their whole set of packages just because
> >> some new version of setup is announced,
> >
> > Uhm?  If you download a new setup, but then don't update your packages,
> > why did you download the latest setup at all?  If you don't run this
> > new setup, you won't get new-style files.
> 
> The checksum information has to come from somewhere and that somewhere
> requires a package install or update.  Together with that new setup we might
> have an update of cygccheck and some unrelated packages that happen to
> have been rebuilt recently, but certainly not the whole distribution.

Why is that important?  The checksums will collect over time.

> >> so I'm going to assume that for
> >> quite some time a mix of old and new .lst files (for instance) exists on
> >> the majority of installations and whatever we do (in cygcheck, say)
> >> needs to work with that.
> >
> > 1. Provide new versions of cygwin, cygcheck-dep and _autorebase
> > 2. time passes (2 weeks or so)
> > 3. Provide a new version of setup
> 
> That doesn't cut it, unless you want to require the whole distribution
> to be rebuilt and everything re-installed with that change.  Cygwin10
> 1609 or something like that?  :-)

I don't think I understand what you're up to.  Why is it important
to add the extra information all at once?  And if that's the case,
couldn't we use a runonce postinstall script for that?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH setup 00/10] Various setup patches

[Achim Gratz]

Corinna Vinschen writes:
>> The checksum information has to come from somewhere and that somewhere
>> requires a package install or update.  Together with that new setup we might
>> have an update of cygccheck and some unrelated packages that happen to
>> have been rebuilt recently, but certainly not the whole distribution.
>
> Why is that important?  The checksums will collect over time.

That's the plan, but then you'll have to deal with missing checksums
while that particular strand of hair grows out.

> I don't think I understand what you're up to.  Why is it important
> to add the extra information all at once?  And if that's the case,
> couldn't we use a runonce postinstall script for that?

I was interpreting your reply that you wanted the tools to only support
the new list formats.  I'm not sure what you suppose the postinstall
script should do, but it certainly shouldn't be trying to checksum the
existing installation and using _that_ information instead.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [PATCH setup 00/10] Various setup patches

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1217 bytes --]

On Aug  4 19:56, Achim Gratz wrote:
> Corinna Vinschen writes:
> >> The checksum information has to come from somewhere and that somewhere
> >> requires a package install or update.  Together with that new setup we might
> >> have an update of cygccheck and some unrelated packages that happen to
> >> have been rebuilt recently, but certainly not the whole distribution.
> >
> > Why is that important?  The checksums will collect over time.
> 
> That's the plan, but then you'll have to deal with missing checksums
> while that particular strand of hair grows out.
> 
> > I don't think I understand what you're up to.  Why is it important
> > to add the extra information all at once?  And if that's the case,
> > couldn't we use a runonce postinstall script for that?
> 
> I was interpreting your reply that you wanted the tools to only support
> the new list formats.

Oh, no, sorry if I was unclear but I would like the tools just to
adapt so that they don't choke on new formats while still working
happily on the old formats.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH setup 00/10] Various setup patches

[Achim Gratz]

Corinna Vinschen writes:
> Oh, no, sorry if I was unclear but I would like the tools just to
> adapt so that they don't choke on new formats while still working
> happily on the old formats.

Oh goody, then we're on the same page.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html