[PATCH] setup.exe

[PATCH] setup.exe

[Achim Gratz]

[-- Attachment #1: Type: text/plain, Size: 118 bytes --]

As requested by Corinna on the Cygwin list, here's a patch to document
some recent changes in the build environment.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-README-document-some-recent-changes-in-the-build-env.patch --]
[-- Type: text/x-patch, Size: 2002 bytes --]

From 3dd23c6063a3edb8bfd1874f5b3c68baf0a89ec4 Mon Sep 17 00:00:00 2001
From: Achim Gratz <Achim.Gratz@Infineon.com>
Date: Fri, 18 Jan 2013 14:24:13 +0100
Subject: [PATCH 1/3] README: document some recent changes in the build
 environment

* setup/README (HOW TO BUILD): Cross compiler package is now named
  mingw-gcc-g++, also mention package upx as an optional dependency.
  Document the requirement of libgetopt++ as a subdirectory in the
  main source tree.  Change the bootstrap stanza to take place in the
  source tree since an out-of-tree build doesn't seem to work (if it
  does, it apparently has additional requirements that I haven't
  figured out).
---
 setup/README | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/setup/README b/setup/README
index 2f36fb8..d5b0240 100755
--- a/setup/README
+++ b/setup/README
@@ -5,7 +5,7 @@ HOW TO BUILD:
 -------------
 Setup should build out-of-the-box on any Cygwin environment that has all the
 required packages installed:
-  - gcc-mingw-g++
+  - mingw-gcc-g++
   - make
   - mingw-bzip2
   - mingw-libgcrypt-devel
@@ -13,6 +13,7 @@ required packages installed:
   - mingw-zlib
   - and all packages that are dependencies of the above, i.e.  gcc-mingw-core,
     mingw-runtime, binutils, w*api, etc.
+  - upx (optional)
 
 The following additional packages are required if building from CVS, not from
 a source tarball, or if you want to make changes to the build system.
@@ -22,9 +23,14 @@ a source tarball, or if you want to make changes to the build system.
   - flex
   - bison
 
+Additionally, libgetopt++ (also available from the cygwin-apps CVS at
+sourceware.org) must be available directly as a subdirectory
+libgetopt++ within the setup source directory.
+
 Build commands:
 1) Configure using this option
-   $ /path/to/setup/bootstrap.sh
+   $ cd /path/to/setup
+   $ ./bootstrap.sh
    This will automatically rebuild configure files and run configure in the
    current directory.
 2) $ make
-- 
1.8.1


[-- Attachment #3: Type: text/plain, Size: 88 bytes --]


A second patch adds two new targets for the makefile to strip and
compress setup.exe.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0002-Makefile-additional-targets-strip-and-upx.patch --]
[-- Type: text/x-patch, Size: 1778 bytes --]

From ba454956b3e934cf767c9cc57ffbe090acd9437e Mon Sep 17 00:00:00 2001
From: Achim Gratz <Achim.Gratz@Infineon.com>
Date: Fri, 18 Jan 2013 14:33:29 +0100
Subject: [PATCH 2/3] Makefile: additional targets "strip" and "upx"

* setup/Makefile.am: Provide new targets "strip" and "upx" to remove
  debugging symbols and compress the executable using UPX,
  respectively.  Check for an executable "upx" in path and bail with a
  warning message if not found.

* setup/README: Change the description of how to produce stripped and
  compressed binaries to use the new make targets.
---
 setup/Makefile.am | 11 +++++++++++
 setup/README      |  9 +++++----
 2 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/setup/Makefile.am b/setup/Makefile.am
index 7bd4546..6d06f01 100755
--- a/setup/Makefile.am
+++ b/setup/Makefile.am
@@ -299,3 +299,14 @@ setup-src:
 	sort | tar -T - -cjf ${CURDIR}/$$ver-src.tar.bz2;\
 	echo $$ver-src.tar.bz2; exec rm -f $$ver
 
+# optional: strip and compress executable
+.PHONY:	strip upx
+
+strip:	all
+	$(STRIP) -s setup$(EXEEXT)
+upx:	strip
+	@if [ -e `which upx` ]; then\
+		upx -9 setup$(EXEEXT) ;\
+	else \
+		echo "UPX doesn't seem to be installed, cannot compress setup$(EXEEXT)." ;\
+	fi
diff --git a/setup/README b/setup/README
index d5b0240..a0846be 100755
--- a/setup/README
+++ b/setup/README
@@ -36,10 +36,11 @@ Build commands:
 2) $ make
 
 3) Wondering why your binary is so much bigger than the official releases?
-   Remove debugging symbols:
-   $ strip -s setup.exe
-   Compress using UPX:
-   $ upx -9 setup.exe
+   This removes debugging symbols:
+   $ make strip
+   This additionally compresses it using UPX
+   (requires package upx to be installed):
+   $ make upx
 
 CODING GUIDELINES:
 ------------------
-- 
1.8.1


[-- Attachment #5: Type: text/plain, Size: 535 bytes --]


I#ll also offer a third patch that adds an option to setup.exe to enable
the use of (local) ini files with a different basename than "setup".
This is useful to offer (in the same install hierarchy) multiple
installations, either for having different configurations or to allow
individual installations to be rolled back.  The idea is to copy the
setup.ini to something like release_2013-01-18.ini whenever a release is
made and let setup.ini keep going forward for testing without having to
duplicate the complete install hierarchy.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #6: 0003-Allow-a-different-basename-instead-of-setup.patch --]
[-- Type: text/x-patch, Size: 2604 bytes --]

From a69118718bd5e7ac0ca22d36480fefa992da449c Mon Sep 17 00:00:00 2001
From: Achim Gratz <Achim.Gratz@Infineon.com>
Date: Fri, 18 Jan 2013 17:05:52 +0100
Subject: [PATCH 3/3] Allow a different basename (instead of "setup")

* setup/ini.h: Modify macro definition to pick up name from external
  variable SetupBaseName instead of string constant.

* setup/main.cc: New string option "-I" aka "--ini-basename" to feed
  basename into setup.  Copy resulting string to the exported
  variable SetupBaseName.
---
 setup/ini.h   | 6 ++++--
 setup/main.cc | 5 +++++
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/setup/ini.h b/setup/ini.h
index 7276e0a..cf26aa1 100755
--- a/setup/ini.h
+++ b/setup/ini.h
@@ -39,8 +39,10 @@ typedef enum
 } excludes;
 
 extern bool is_legacy;
-#define SETUP_INI_FILENAME (is_legacy ? "setup-legacy.ini" : "setup.ini")
-#define SETUP_BZ2_FILENAME (is_legacy ? "setup-legacy.bz2" : "setup.bz2")
+
+#define SETUP_INI_FILENAME (is_legacy ? "setup-legacy.ini" : (std::string(SetupBaseName)+".ini").c_str())
+#define SETUP_BZ2_FILENAME (is_legacy ? "setup-legacy.bz2" : (std::string(SetupBaseName)+".bz2").c_str())
+extern std::string SetupBaseName;
 
 /* The following three vars are used to facilitate error handling between the
    parser/lexer and its callers, namely ini.cc:do_remote_ini() and
diff --git a/setup/main.cc b/setup/main.cc
index dc73936..0a66e1f 100755
--- a/setup/main.cc
+++ b/setup/main.cc
@@ -67,6 +67,7 @@ static const char *cvsid =
 
 #include "getopt++/GetOption.h"
 #include "getopt++/BoolOption.h"
+#include "getopt++/StringOption.h"
 
 #include "Exception.h"
 #include <stdexcept>
@@ -91,6 +92,8 @@ bool is_legacy;
 static BoolOption UnattendedOption (false, 'q', "quiet-mode", "Unattended setup mode");
 static BoolOption PackageManagerOption (false, 'M', "package-manager", "Semi-attended chooser-only mode");
 static BoolOption HelpOption (false, 'h', "help", "print help");
+static StringOption SetupBaseNameOpt ("blafasel", 'I', "ini-basename", "Use a different basename instead of setup", false);
+std::string SetupBaseName;
 static BOOL WINAPI (*dyn_AttachConsole) (DWORD);
 static BOOL WINAPI (*dyn_GetLongPathName) (LPCTSTR, LPTSTR, DWORD);
 
@@ -289,6 +292,8 @@ WinMain (HINSTANCE h,
     if (unattended_mode || HelpOption)
       set_cout ();
 
+    SetupBaseName = SetupBaseNameOpt;
+
     LogSingleton::SetInstance (*(theLog = LogFile::createLogFile ()));
     const char *sep = isdirsep (local_dir[local_dir.size () - 1]) ? "" : "\\";
     theLog->setFile (LOG_BABBLE, local_dir + sep + "setup.log.full", false);
-- 
1.8.1


[-- Attachment #7: Type: text/plain, Size: 305 bytes --]


This is certainly not the most elegant way to do this, but I was more
concerned to not change most code.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [PATCH] setup.exe

[Ken Brown]

On 1/18/2013 12:23 PM, Achim Gratz wrote:
> +Additionally, libgetopt++ (also available from the cygwin-apps CVS at
> +sourceware.org) must be available directly as a subdirectory
> +libgetopt++ within the setup source directory.

I don't understand why you need this part.  Checking out setup 
automatically gives you the libgetopt++ subdirectory.

Ken

Re: [PATCH] setup.exe

[Christopher Faylor]

On Fri, Jan 18, 2013 at 06:23:49PM +0100, Achim Gratz wrote:
>As requested by Corinna on the Cygwin list, here's a patch to document
>some recent changes in the build environment.

You'd really be much better served to submit one patch at a time.

From 3dd23c6063a3edb8bfd1874f5b3c68baf0a89ec4 Mon Sep 17 00:00:00 2001
>From: Achim Gratz <Achim.Gratz@Infineon.com>
>Date: Fri, 18 Jan 2013 14:24:13 +0100
>Subject: [PATCH 1/3] README: document some recent changes in the build
> environment
>
>* setup/README (HOW TO BUILD): Cross compiler package is now named
>  mingw-gcc-g++, also mention package upx as an optional dependency.
>  Document the requirement of libgetopt++ as a subdirectory in the
>  main source tree.  Change the bootstrap stanza to take place in the
>  source tree since an out-of-tree build doesn't seem to work (if it
>  does, it apparently has additional requirements that I haven't
>  figured out).
>---
> setup/README | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
>diff --git a/setup/README b/setup/README
>index 2f36fb8..d5b0240 100755
>--- a/setup/README
>+++ b/setup/README
>@@ -5,7 +5,7 @@ HOW TO BUILD:
> -------------
> Setup should build out-of-the-box on any Cygwin environment that has all the
> required packages installed:
>-  - gcc-mingw-g++
>+  - mingw-gcc-g++
>   - make
>   - mingw-bzip2
>   - mingw-libgcrypt-devel
>@@ -13,6 +13,7 @@ required packages installed:
>   - mingw-zlib
>   - and all packages that are dependencies of the above, i.e.  gcc-mingw-core,
>     mingw-runtime, binutils, w*api, etc.
>+  - upx (optional)
> 
> The following additional packages are required if building from CVS, not from
> a source tarball, or if you want to make changes to the build system.
>@@ -22,9 +23,14 @@ a source tarball, or if you want to make changes to the build system.
>   - flex
>   - bison
> 
>+Additionally, libgetopt++ (also available from the cygwin-apps CVS at
>+sourceware.org) must be available directly as a subdirectory
>+libgetopt++ within the setup source directory.
>+
> Build commands:
> 1) Configure using this option
>-   $ /path/to/setup/bootstrap.sh
>+   $ cd /path/to/setup
>+   $ ./bootstrap.sh

If you look at bootstrap.sh, you will see why I'm puzzled why this should be
needed.  The script is intended to be run from the build directory which
should be separate from the source directory.

>A second patch adds two new targets for the makefile to strip and
>compress setup.exe.

"strip" as a target is really not a good idea.

>I#ll also offer a third patch that adds an option to setup.exe to enable
>the use of (local) ini files with a different basename than "setup".
>This is useful to offer (in the same install hierarchy) multiple
>installations, either for having different configurations or to allow
>individual installations to be rolled back.  The idea is to copy the
>setup.ini to something like release_2013-01-18.ini whenever a release is
>made and let setup.ini keep going forward for testing without having to
>duplicate the complete install hierarchy.

Who is the target audience for this?  It doesn't seem like it is worth
complicating setup for this type of feature.

cgf

Re: [PATCH] setup.exe

[Achim Gratz]

Christopher Faylor writes:
> You'd really be much better served to submit one patch at a time.

Noted.

> If you look at bootstrap.sh, you will see why I'm puzzled why this should be
> needed.  The script is intended to be run from the build directory which
> should be separate from the source directory.

In all the combinations I tried, this didn't result in a working build
environment, but let me try again.  It is possible I got fooled by
CVSgrab and there's still something missing in my source tree.

> "strip" as a target is really not a good idea.

I'm not wedded to the name and I really only intend to use "upx" myself
(whose name is also up for grabs if you don't like it).

> Who is the target audience for this?

I'm in the process of updating our age-old "IT approved" Cygwin
installation to 1.7.x.  One of the things I need to provide is the
ability to "freeze" and "roll back" installations to some known state
and do it from a script.  I have a (not quite finished yet) script
infrastructure that collects the necessary files into an install
hierarchy and re-writes setup.ini to match that.  The current plan is to
rename setup.ini to some unique name upon freeze/release and make sure
that the files referenced by all these ini files are kept along with any
updates that come with a new setup.ini.  That way I hope to fulfill that
requirement without unduly needing to duplicate gigabytes of data for
each release.  All this of course only works if "setup.ini" isn't
hard-coded into setup.

> It doesn't seem like it is worth complicating setup for this type of
> feature.

If you are talking about the interactive installation method I tend to
agree.

This change however is intended to add some support for non-interactive
installations (and I will likely request some more support for that
later on, like deleting packages from the command line).  The
alternative would be to put each setup.ini in its own directory and link
the install hierachy (that works, I've tested it), but the master copy
I'm producing will be replicated to some 45 distribution servers around
the world in a two- or sometimes three-stage process and I'm trying to
avoid testing the chances of the links surviving the replication (over
which I have no control other than saying "do it now" and I can't check
most of those servers easily).

But you are free of course to not wanting to support that and drop the
patch, in which case I'll carry it forward locally.  No big deal.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: [PATCH] setup.exe

[Christopher Faylor]

On Sat, Jan 19, 2013 at 08:41:05AM +0100, Achim Gratz wrote:
>Christopher Faylor writes:
>> You'd really be much better served to submit one patch at a time.
>
>Noted.
>
>> If you look at bootstrap.sh, you will see why I'm puzzled why this should be
>> needed.  The script is intended to be run from the build directory which
>> should be separate from the source directory.
>
>In all the combinations I tried, this didn't result in a working build
>environment, but let me try again.  It is possible I got fooled by
>CVSgrab and there's still something missing in my source tree.
>
>> "strip" as a target is really not a good idea.
>
>I'm not wedded to the name and I really only intend to use "upx" myself
>(whose name is also up for grabs if you don't like it).
>
>> Who is the target audience for this?
>
>I'm in the process of updating our age-old "IT approved" Cygwin
>installation to 1.7.x.  One of the things I need to provide is the
>ability to "freeze" and "roll back" installations to some known state
>and do it from a script.

I'm really not too keen on adding hacks to setup so that people can
use it for other than its intended purpose.  The code is already
a complicated obtuse mishmash and adding more stuff which would
be used by only one person doesn't seem like a good idea.

cgf

Re: [PATCH] setup.exe

[Achim Gratz]

Christopher Faylor writes:
> I'm really not too keen on adding hacks to setup so that people can
> use it for other than its intended purpose.

If there is another method to do an unattended Cygwin install, I'm all
ears.  I have briefly pondered to script a standalone installer, but it
requires too much infrastructure and I don't see much value in trying to
re-do what setup.exe already does.  The question of doing unattended
installs comes up often enough that official support would be nice,
IMHO.

> The code is already a complicated obtuse mishmash and adding more
> stuff which would be used by only one person doesn't seem like a good
> idea.

A self-fulfilling prophecy: nobody is doing it because nobody is doing
it.  I'll have to add on whatever I need, if it's useless to you then
just don't use it.



Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables

Re: [PATCH] setup.exe

[Christopher Faylor]

On Sat, Jan 19, 2013 at 09:47:31PM +0100, Achim Gratz wrote:
>Christopher Faylor writes:
>>I'm really not too keen on adding hacks to setup so that people can use
>>it for other than its intended purpose.
>
>If there is another method to do an unattended Cygwin install, I'm all
>ears.  I have briefly pondered to script a standalone installer, but it
>requires too much infrastructure and I don't see much value in trying
>to re-do what setup.exe already does.  The question of doing unattended
>installs comes up often enough that official support would be nice,
>IMHO.

I was referring to your change which handled versioned setup.ini's.
That is not a requirement for an unattended Cygwin install.

>>The code is already a complicated obtuse mishmash and adding more stuff
>>which would be used by only one person doesn't seem like a good idea.
>
>A self-fulfilling prophecy: nobody is doing it because nobody is doing
>it.  I'll have to add on whatever I need, if it's useless to you then
>just don't use it.

If there was a general hue and cry for the feature that you want to add,
I'd consider adding it.  AFAIK, there isn't.  If we did add it we would
be obligated to support it so, if there is a need for this type of
feature, I'd like to see a lot more discussion about why it's necessary
and the best way to implement it.

cgf

Re: [PATCH] setup.exe

[green fox]

Achim, reguardless of if this code getting into cygwin (or not), could
you be able to provide a copy of this on public git/whatever server?
I really like this.

Just for note, in the past, unattended instalation/update using
automated package management, we resorted into using apt-cyg written
by Stephen Jungels.
(http://code.google.com/apt-cyg)

However, I had requirements for closed network instalation, and (back
then nobody supported that) added options for --localdiskcache,
--force, --ForceUpdate and md5sum file check... etc. The modified
script is floating in superuser.com
(superuser.com/questions/99198/cygwin-offline-installer) if anyone
wants it.

Its just that we have duplication on effort here, not just me, Achim,
the cyg-apt(diffrent guy), more then few "Alternative" installers on
the net, each admin having their own way of ghosting/cloneing/setting
up cygwin...

It may not be the intended use-case of the cygwin dev, however with
all respect, there is a need for a solid cui with /short and sweet
options/automated install/update/package management/use of
proxy(s)/cache/use of mirror/alternative dir as source/dl from
multiple servers/offline/version check/hash check, as a bare minimum
requirement when managing multilpe clients.

And setup.exe has very good GUI, with new and improved commandlines,
its just that it is not suitable for day to day administration
purpose. However we all have diffrent ways of providing the package
data. Nfs/mirror/rsync/ftp/p2p/samba/dvd/ghost/proxy, etc,etc...
Trying to provide "Everything" in setup.exe is cauing the code to get
into spagetti.

Could we somehow share and concatinate the effort?

My 2cents worth of thought tells me, providing setup.exe as front
end/stub/gui to call a solid back end package manager is a reasonable
way to go. For admin purpose using the backend directly. With well
difined porotocol on package version control, we could start writeing
/ or even re-use existing package managers. Just a thought.

GreenFox


On 1/20/13, Christopher Faylor wrote:
> On Sat, Jan 19, 2013 at 09:47:31PM +0100, Achim Gratz wrote:
>>Christopher Faylor writes:
>>>I'm really not too keen on adding hacks to setup so that people can use
>>>it for other than its intended purpose.
>>
>>If there is another method to do an unattended Cygwin install, I'm all
>>ears.  I have briefly pondered to script a standalone installer, but it
>>requires too much infrastructure and I don't see much value in trying
>>to re-do what setup.exe already does.  The question of doing unattended
>>installs comes up often enough that official support would be nice,
>>IMHO.
>
> I was referring to your change which handled versioned setup.ini's.
> That is not a requirement for an unattended Cygwin install.
>
>>>The code is already a complicated obtuse mishmash and adding more stuff
>>>which would be used by only one person doesn't seem like a good idea.
>>
>>A self-fulfilling prophecy: nobody is doing it because nobody is doing
>>it.  I'll have to add on whatever I need, if it's useless to you then
>>just don't use it.
>
> If there was a general hue and cry for the feature that you want to add,
> I'd consider adding it.  AFAIK, there isn't.  If we did add it we would
> be obligated to support it so, if there is a need for this type of
> feature, I'd like to see a lot more discussion about why it's necessary
> and the best way to implement it.
>
> cgf
>

Re: [PATCH] setup.exe

[Christopher Faylor]

On Sun, Jan 20, 2013 at 12:35:32PM +0900, green fox wrote:
>Achim, reguardless of if this code getting into cygwin (or not), could
>you be able to provide a copy of this on public git/whatever server?
>I really like this.

What *specifically* do you really like?

cgf

Re: [PATCH] setup.exe

[Achim Gratz]

green fox writes:
> Achim, reguardless of if this code getting into cygwin (or not), could
> you be able to provide a copy of this on public git/whatever server?

I plan to publish my infrastructure, but haven't yet since it a) isn't
feature complete and b) I need to clean up a few things.  I don't want
to fork setup.exe if I can help it.

> I really like this.

Thanks.

> Just for note, in the past, unattended instalation/update using
> automated package management, we resorted into using apt-cyg written
> by Stephen Jungels.

I've looked at apt-cyg and all the other alternative installers I could
find, but they could either not bootstrap Cygwin or would require too
much infrastructure for installation.  That's why I decided to stick
with setup.exe (also because it is the supported Cygwin installer).

Let me briefly describe how it works: I currently use lftp to mirror
Cygwin and Cygport locally (in a pinch you could use setup.exe itself to
do this).  I have locally built packages and (sometimes) the Cygwin
snapshots re-packaged into Cygwin packages as additional package
sources.  I then use a Perl script to combine all package sources, pull
in dependencies of the selected packages and write this out to a new
setup.ini and install hierarchy (optionally with removing unreferenced
files from earlier versions).  I'm injecting additional groups into the
new setup.ini so that I can do different installs from the same
setup.ini easily (I currently have CLI, GUI, Devel, CygDev - but you can
define whatever you want).  This is what then gets distributed to the
software servers and installed via batch files, or you could even put it
on a flash drive or burn a DVD.

This is working and it doesn't require any changes to setup.exe.  The
users can use setup.exe to alter their installations if they think they
know what they're doing and I can keep providing updated versions
without messing up all those installations each time.

As I said, I have additional requirements to provide different releases,
which for the sake of discussion can be simplified to being able to take
a setup.ini and "freeze" it along with all the package files it
references and then be able to install it again in exactly that state at
a later time.

> Its just that we have duplication on effort here, not just me, Achim,
> the cyg-apt(diffrent guy), more then few "Alternative" installers on
> the net, each admin having their own way of ghosting/cloneing/setting
> up cygwin...

I guess anyone trying to get Cygwin into a corporate network is in the
same boat.

> It may not be the intended use-case of the cygwin dev, however with
> all respect, there is a need for a solid cui with /short and sweet
> options/automated install/update/package management/use of
> proxy(s)/cache/use of mirror/alternative dir as source/dl from
> multiple servers/offline/version check/hash check, as a bare minimum
> requirement when managing multilpe clients.

So far, the only things I would want to change in setup.exe is to name
alternative setup.ini files (for which I sent the patch), but given the
resistance I've met I'll check again if I can work around this without
duplicating whole directory trees or using links/junctions.  If it's
possible to do the moral equivalent by just changing the directory
structure, then I'll happily drop the patch.

The remaining wishes are to add an option to again provide the
"install+update" functionality that was briefly present but then got
changed into either "install" or "update" (which currently requires to
start setup.exe two times in a row) and the ability to delete packages
from the command line.  The functionality to do this already is in
setup.exe, there are just no controls for this on the command line.
Maybe an option to suppress the GUI completely would be nice, but I
personally like to have the progress status on screen.

> And setup.exe has very good GUI, with new and improved commandlines,
> its just that it is not suitable for day to day administration
> purpose. However we all have diffrent ways of providing the package
> data. Nfs/mirror/rsync/ftp/p2p/samba/dvd/ghost/proxy, etc,etc...
> Trying to provide "Everything" in setup.exe is cauing the code to get
> into spagetti.

As I said, this is all dealt with outside setup.exe already and I agree
that this should _not_ be bolted onto setup.exe at all.

> My 2cents worth of thought tells me, providing setup.exe as front
> end/stub/gui to call a solid back end package manager is a reasonable
> way to go.

That would be a different setup.exe than we have today.  If Cygwin wants
to go that route, a more capable package backend would be nice, libzypp
anyone?

> For admin purpose using the backend directly. With well
> difined porotocol on package version control, we could start writeing
> / or even re-use existing package managers. Just a thought.

While I'd love to see something like this, see above why the current
state isn't that far from what is needed, at least in the short term.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [PATCH] setup.exe

[Achim Gratz]

Christopher Faylor writes:
> I was referring to your change which handled versioned setup.ini's.
> That is not a requirement for an unattended Cygwin install.

I do have that requirement, not that I love it very much.

> If there was a general hue and cry for the feature that you want to add,
> I'd consider adding it.  AFAIK, there isn't.

The mere existence of multiple "alternative" installers should speak to
you.

> If we did add it we would be obligated to support it so, if there is a
> need for this type of feature, I'd like to see a lot more discussion
> about why it's necessary and the best way to implement it.

I have one last idea of how to live without that particular change.  If
it works, I'll drop the patch.  I will propose at least two other
changes that I don't think have workarounds that wouldn't require to
duplicate large swathes of functionality that already is in setup.exe,
but simply not accessible from the command line.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: [PATCH] setup.exe

[green fox]

On 1/20/13, Christopher Faylor wrote:
>
> What *specifically* do you really like?
>
> cgf
>

+1 for being able to specify custom setup.ini
Not a happy moment when you realize some package is missing, having
spent 2 housr to distribute the blob on the network.

Only a _workaround_, but If I can just push out the new
setup-<machine>-<epoch-timestamp>.ini
or something, and have a bat script automate the setup depend on the
specific machine group,
it is much better than the current situation.
Haven't tried it out, but it seems promising.

Not the best way, having a solid package manager is best...

fox

Re: [PATCH] setup.exe

[Christopher Faylor]

On Mon, Jan 21, 2013 at 04:03:04PM +0900, green fox wrote:
>On 1/20/13, Christopher Faylor wrote:
>>What *specifically* do you really like?
>
>+1 for being able to specify custom setup.ini Not a happy moment when
>you realize some package is missing, having spent 2 housr to distribute
>the blob on the network.

If you are missing a package then sending out a new setup.ini which
includes the missing package should fix the problem.  I don't see how
being able to specify something other than setup.ini helps.

Btw, I'm not looking for votes.  I'm looking for explanations for why
this option is useful.  It seems like the described functionality could
be handled with just a shell script wrapper.

cgf

Re: [PATCH] setup.exe

[green fox]

On 1/20/13, Achim Gratz wrote:
> I plan to publish my infrastructure, but haven't yet since it a) isn't
> feature complete and b) I need to clean up a few things.  I don't want
> to fork setup.exe if I can help it.

Agrred. No one likes to fork, and that includes myself.
And believe me, clean up of code will never happen. We've got a
rollout script that noone dare touches it to clean it up. And it
grows, too.

The lack for any good (standard) bootstrap method for mass deployment
is a problem.
And everybody result in writing one'sown in-house version.

> Let me briefly describe how it works: I currently use lftp to mirror
> Cygwin and Cygport locally (in a pinch you could use setup.exe itself to
> do this).  I have locally built packages and (sometimes) the Cygwin
> snapshots re-packaged into Cygwin packages as additional package
> sources.  I then use a Perl script to combine all package sources, pull
> in dependencies of the selected packages and write this out to a new
> setup.ini and install hierarchy (optionally with removing unreferenced
> files from earlier versions).  I'm injecting additional groups into the
> new setup.ini so that I can do different installs from the same
> setup.ini easily (I currently have CLI, GUI, Devel, CygDev - but you can
> define whatever you want).  This is what then gets distributed to the

Diffrent vehicle(rsync), but same here.
Diffrence is, we gave up on the setup.ini route in the past, but if
the setup.ini thing works nicely, this is cleaner approach than what
we've got.
Having to build and deploy custom packages, wrote a quick and dirty
script that makes a self extracting archive blob. Then it pushes
out/exec on remote. Very dirty, but we had a existing base for pushing
out binary blobs/remove executon anyway.

> This is working and it doesn't require any changes to setup.exe.
Very cool :-)

> As I said, I have additional requirements to provide different releases,
> which for the sake of discussion can be simplified to being able to take
> a setup.ini and "freeze" it along with all the package files it
> references and then be able to install it again in exactly that state at
> a later time.

Actually this is the most important thing lacking from the curernt situation.

We have a mirror that holds packages of (current and past )specific
versions, and
the freeze/save state is performed by the (avobe mentioned) blob maker
(that can re-create the same blob , as long as supplied with same list
).
But this is not the best way to do things. It's jsut a workaroud,
that's been used for years.

> I guess anyone trying to get Cygwin into a corporate network is in the
> same boat.
>

really need a solid package manager for cygwin.

> start setup.exe two times in a row) and the ability to delete packages
> from the command line.  The functionality to do this already is in
> setup.exe, there are just no controls for this on the command line.

+1 on providing a solid option
As a minimum having install|remove|update|show|find|depends would be needed.

And if possible
-mirror <url> : set mirror to download package( can specify multiple)
-cache <dir>  : set cache directory to use
     (for packages that may aready been downloaded/partial download in progress)
    Something like combination of the exsiting [--local-install]
[--local-package-dir]
-file, -f <file>  : read package list to install/remove from file
([-P] from a file)
-force            : Force install/remove

> Maybe an option to suppress the GUI completely would be nice, but I
> personally like to have the progress status on screen.
>
I believe there is [--quiet-mode]

>> My 2cents worth of thought tells me, providing setup.exe as front
>> end/stub/gui to call a solid back end package manager is a reasonable
>> way to go.
>
> That would be a different setup.exe than we have today.  If Cygwin wants
> to go that route, a more capable package backend would be nice, libzypp
> anyone?
>

It was just a dream.
But really, having only curr|prev|test for version control is not funny.

Re: [PATCH] setup.exe

[green fox]

On 1/21/13, Christopher Faylor wrote:
> On Mon, Jan 21, 2013 at 04:03:04PM +0900, green fox wrote:
>>On 1/20/13, Christopher Faylor wrote:
>>>What *specifically* do you really like?
>>
>>+1 for being able to specify custom setup.ini Not a happy moment when
>>you realize some package is missing, having spent 2 housr to distribute
>>the blob on the network.
>
> If you are missing a package then sending out a new setup.ini which
> includes the missing package should fix the problem.  I don't see how
> being able to specify something other than setup.ini helps.
>
> Btw, I'm not looking for votes.  I'm looking for explanations for why
> this option is useful.  It seems like the described functionality could
> be handled with just a shell script wrapper.
>
> cgf
>

Because we do not have a decent package manager

fox

Re: [PATCH] setup.exe

[Christopher Faylor]

On Mon, Jan 21, 2013 at 06:46:00PM +0900, green fox wrote:
>On 1/21/13, Christopher Faylor wrote:
>> On Mon, Jan 21, 2013 at 04:03:04PM +0900, green fox wrote:
>>>On 1/20/13, Christopher Faylor wrote:
>>>>What *specifically* do you really like?
>>>
>>>+1 for being able to specify custom setup.ini Not a happy moment when
>>>you realize some package is missing, having spent 2 housr to distribute
>>>the blob on the network.
>>
>> If you are missing a package then sending out a new setup.ini which
>> includes the missing package should fix the problem.  I don't see how
>> being able to specify something other than setup.ini helps.
>>
>> Btw, I'm not looking for votes.  I'm looking for explanations for why
>> this option is useful.  It seems like the described functionality could
>> be handled with just a shell script wrapper.
>
>Because we do not have a decent package manager

And, that's because we don't have any "decent" developers willing to
write one.  Why haven't you stepped up?

But, anyway, you are apparently missing the point.  I'm not looking for
complaints.  I'm looking for actual reasoned arguments for why a patch
should be applied.  If you are just knee-jerk responding to someone
saying "unattended setup" then you're going to have to actually read and
understand what the patch does and indicate why it would be useful.

cgf

Re: [PATCH] setup.exe

[Achim Gratz]

Achim Gratz writes:
> I have one last idea of how to live without that particular change.  If
> it works, I'll drop the patch.

I've done some tests and I can implement my minimum requirement by
changing the directory structure to something like:

repo/release/... (packages directory)
repo/setup_current/setup.ini
repo/setup_2013-01-21/setup.ini

This works for me since I'm rewriting setup.ini anyway and each install
entry now simply gets a "../" prepended.  This isn't quite as neat as
simply telling setup.exe where to find an alternative setup.ini, but as
long as you have write access to the repo it gets the job done with
minimum fuzz.  I guess if you could put absolute paths into your
setup.ini otherwise, but then you'll have to re-create it again with
relative paths on release.  I'm keeping the patch locally since it has
proven useful to me, but for the "official" setup.exe I can use the
above workaround.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [PATCH] setup.exe

[green fox]

Chris, language please.

The problem in lack of decent package manager is something most of us here
know for a long time. Hope everyone agrees on this point.

Here we're just focusing to concentrate the work,
refactor what key items we need, or can leave out with, re-use and such.

At the same time, things must be kept running.
So a workaround (for the mean time) is needed.
We know that setup.exe is not enough.
And as clealy noted, this setup-<whatever>.ini thing is a _workaround_
We need a package manager to allow routine package management tasks to
be accomplished in a single command.

We need
1- a consistent user interface
2- automated package installation
3- upgrading and searching
4- package removal
5- package builder
6- lookup of package dependency
7- bootstrap cygwin to a clean env
8- generate/modify setup.ini and customize

 *And everyting must be script-ready*

let me know if I left something out

The existing and runing system does not allow one to specify a version
for a given package/
example)
what if package foo 3.1.6 had changed ABI, and 3.1.x (other thatn 6)
restored the old ABI.
if we have a dependency from bar, either
A) if bar is compiled with 3.1.6 , then make sure foo is against 3.1.6 as well
B) or check that foo != 3.1.6 , and double check that bar is compiled
for version != 3.1.6

Version control using  prev|curr|test is not the right way to do things, IMHO

I hope this is clear enough example for _one_ of the many reasons why
we need and write workarounds.

As a side note
libzypp =>Depends on mingw boost C++  gtk e2fsporg and so forth...
I really like the algorithm they have, but we need it running quick,
cheap, and simple.
have not looked into it, but if it matches our requirements, prahaps a
option. not sure yet.

One of the reasons we sill have perl/awk/shell scripts to roll things out, is
when things break, its not that hard to fix/patch a workaround.

fox

Re: [PATCH] setup.exe

[Christopher Faylor]

On Tue, Jan 22, 2013 at 04:32:22AM +0900, green fox wrote:
>Chris, language please.

Sorry for being unclear.  I was writing in English.

>The problem in lack of decent package manager is something most of us here
>know for a long time. Hope everyone agrees on this point.

Reiteration of this point in all of your messages does not serve any
useful purpose.

A quick scan of your reply seemed to show that you are asking for
features and outlining well-known shortcomings.  Unless you are
volunteering to do that work, this is likely YA wishlist which will go
nowhere.

For your amusement:

http://www.cygwin.com/ml/cygwin-apps/2004-12/msg00019.html

>I hope this is clear enough example for _one_ of the many reasons why
>we need and write workarounds.

I was asking why the *specific* change needed to be implemented.  You
did not answer my question.  But that's ok.  This particular case is
closed anyway.

>As a side note
>libzypp =>Depends on mingw boost C++  gtk e2fsporg and so forth...
>I really like the algorithm they have, but we need it running quick,
>cheap, and simple.
>have not looked into it, but if it matches our requirements, prahaps a
>option. not sure yet.

No idea what libzypp is.

**cgf googles

http://doc.opensuse.org/projects/libzypp/HEAD/

If someone has ported rpm to mingw that would certainly be interesting.
Otherwise, this is a cart-before-the-horse situation.

>One of the reasons we sill have perl/awk/shell scripts to roll things
>out, is when things break, its not that hard to fix/patch a workaround.

I love perl/awk/shell scripts.  I use them quite frequently.  I was
actually suggesting that they would be an alternative to the suggested
patch.  I think that, in general, wrapping your installation needs in a
wrapper is a fine idea.

cgf

Re: [PATCH 0/4] setup.exe

[Achim Gratz]

Here is a reworked patch series, this time as single patches in separate
mail.  For the sake of completeness I'm sending all patches again.  They
should all apply seperately if necessary.  Some comments below.


[PATCH 1/4] README: document some recent changes in the build environment

I've clarified that libgetopt++ should be present after a CVS checkout
and that this should be checked and perhaps manually corrected when
using a different method of obtaining the sources.  I've checked with a
completely fresh source tree that an out-of-tree build does indeed work
(don't know why it didn't when I first tried this).


[PATCH 2/4] Makefile: additional targets "strip" and "upx"

These targets have been quite useful to me.  Feel free to rename them.


[PATCH 3/4] Allow delete and reinstall from CLI, re-implement install

This is the patch that was missing from the earlier series and also the
one that I really want to go into the official sources.

It adds two options to specify deletion of packages and entire
categories from the CLI and re-implements the handling of these and the
options for adding packages or categories.  As a bonus, there is no need
to discern between unattended and other install methods anymore and you
can switch to chooser mode and inspect what packages are selected with a
particular invocation.  If the same package is both removed and added
from the CLI, it will get reinstalled.  This is a no-op if the package
wasn't installed, this could be detected and changed to install instead,
but the current behaviour is IMHO more useful since it allows
reinstallation of a whole category without also installing all packages
from that category that weren't already present.

Some usage examples (may require additional options):

# reinstall tar
setup.exe -P tar -x tar

# install or upgrade packages xz and zoo without touching the rest of
# the installed packages
setup.exe -P xz,zoo

# remove package xz and zoo
setup.exe -x xz,zoo

# remove all package in category Devel
setup.exe -c Devel

# upgrade installed packages and install all missing packages from
# category Devel, enter chooser mode to inspect and maybe alter the
# package selection
setup.exe --upgrade -C Devel -M

# upgrade installed packages, remove orphaned packages and enter chooser
# mode
setup.exe -goM


[PATCH 4/4] Allow a different basename (instead of "setup")

I still think this is a useful option to have.  The workaround I've been
using for about a week now requires extra complexity.  I can absorb that
in the scripts I'm using, but it is not easy to do this directly on the
command line.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: [PATCH 1/4] setup.exe

[Achim Gratz]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: 0001-README-document-some-recent-changes-in-the-build-env.patch --]
[-- Type: text/x-patch, Size: 2241 bytes --]

From 991a4e1455b73abfffef6651583edae0079c077f Mon Sep 17 00:00:00 2001
From: Achim Gratz
Date: Fri, 25 Jan 2013 13:23:10 +0100
Subject: [PATCH 1/4] README: document some recent changes in the build environment

* setup/README (HOW TO BUILD): Cross compiler package is now named
  mingw-gcc-g++, also mention package upx as an optional dependency.
  Document the requirement of libgetopt++ as a subdirectory in the
  main source tree for user of CVSgrab and similar programs to access
  the CVS sources.  Mention that the build directory should be outside
  the source tree.
---
 setup/README | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/setup/README b/setup/README
index 2f36fb8..f6eb520 100755
--- a/setup/README
+++ b/setup/README
@@ -5,7 +5,7 @@ HOW TO BUILD:
 -------------
 Setup should build out-of-the-box on any Cygwin environment that has all the
 required packages installed:
-  - gcc-mingw-g++
+  - mingw-gcc-g++
   - make
   - mingw-bzip2
   - mingw-libgcrypt-devel
@@ -13,6 +13,7 @@ required packages installed:
   - mingw-zlib
   - and all packages that are dependencies of the above, i.e.  gcc-mingw-core,
     mingw-runtime, binutils, w*api, etc.
+  - upx (optional)
 
 The following additional packages are required if building from CVS, not from
 a source tarball, or if you want to make changes to the build system.
@@ -22,11 +23,18 @@ a source tarball, or if you want to make changes to the build system.
   - flex
   - bison
 
+Additionally, libgetopt++ (also available from the cygwin-apps CVS at
+sourceware.org) must be available directly as a subdirectory
+libgetopt++ within the setup source directory.  It should be
+automatically populated by CVS, but if you are using tools like
+CVSgrab, you may need to fetch this yourself.
+
 Build commands:
 1) Configure using this option
    $ /path/to/setup/bootstrap.sh
-   This will automatically rebuild configure files and run configure in the
-   current directory.
+   This will automatically rebuild configure files and run configure
+   in the current directory.  The current directory must be outside
+   the source tree.
 2) $ make
 
 3) Wondering why your binary is so much bigger than the official releases?
-- 
1.8.1.1


[-- Attachment #2: Type: text/plain, Size: 177 bytes --]



Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: [PATCH 2/4] setup.exe

[Achim Gratz]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: 0002-Makefile-additional-targets-strip-and-upx.patch --]
[-- Type: text/x-patch, Size: 1753 bytes --]

From a8ffe947b5e64b9f29cecc8c404d0d508ab7be67 Mon Sep 17 00:00:00 2001
From: Achim Gratz
Date: Fri, 18 Jan 2013 14:33:29 +0100
Subject: [PATCH 2/4] Makefile: additional targets "strip" and "upx"

* setup/Makefile.am: Provide new targets "strip" and "upx" to remove
  debugging symbols and compress the executable using UPX,
  respectively.  Check for an executable "upx" in path and bail with a
  warning message if not found.

* setup/README: Change the description of how to produce stripped and
  compressed binaries to use the new make targets.
---
 setup/Makefile.am | 11 +++++++++++
 setup/README      |  9 +++++----
 2 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/setup/Makefile.am b/setup/Makefile.am
index 7bd4546..6d06f01 100755
--- a/setup/Makefile.am
+++ b/setup/Makefile.am
@@ -299,3 +299,14 @@ setup-src:
 	sort | tar -T - -cjf ${CURDIR}/$$ver-src.tar.bz2;\
 	echo $$ver-src.tar.bz2; exec rm -f $$ver
 
+# optional: strip and compress executable
+.PHONY:	strip upx
+
+strip:	all
+	$(STRIP) -s setup$(EXEEXT)
+upx:	strip
+	@if [ -e `which upx` ]; then\
+		upx -9 setup$(EXEEXT) ;\
+	else \
+		echo "UPX doesn't seem to be installed, cannot compress setup$(EXEEXT)." ;\
+	fi
diff --git a/setup/README b/setup/README
index f6eb520..e70965f 100755
--- a/setup/README
+++ b/setup/README
@@ -38,10 +38,11 @@ Build commands:
 2) $ make
 
 3) Wondering why your binary is so much bigger than the official releases?
-   Remove debugging symbols:
-   $ strip -s setup.exe
-   Compress using UPX:
-   $ upx -9 setup.exe
+   This removes debugging symbols:
+   $ make strip
+   This additionally compresses it using UPX
+   (requires package upx to be installed):
+   $ make upx
 
 CODING GUIDELINES:
 ------------------
-- 
1.8.1.1


[-- Attachment #2: Type: text/plain, Size: 188 bytes --]



Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: [PATCH 0/4] setup.exe

[Achim Gratz]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: 0003-Allow-delete-and-reinstall-from-CLI-re-implement-ins.patch --]
[-- Type: text/x-patch, Size: 10510 bytes --]

From a246270621b2b4fbd476aa386fc217ed14f3fe10 Mon Sep 17 00:00:00 2001
From: Achim Gratz
Date: Fri, 25 Jan 2013 12:12:41 +0100
Subject: [PATCH 3/4] Allow delete and reinstall from CLI, re-implement install from CLI

* choose.cc: Add new CLI option -g/--upgrade-also, considers all
  installed packages as candidates for upgrades (default when no CLI
  package or category options have been given).
* choose.cc: Add new CLI option -o/--delete-orphans, considers
  installed packages that do not exist anymore in the package
  repositories as candidates for deletion.
* choose.cc (createListview): remove superflous and detrimental
  default trust setting.  This has already been set correctly in
  OnInit.
* choose.cc (OnInit): Re-implement package handling depending on
  options given on CLI using package actions instead of package_meta
  low-level functions.  When no CLI package or category options have
  been given, assume --upgrade-also in accordance with the previous
  behaviour.  No such assumption is made when packages are to be added
  or removed from the CLI, but this behaviour can be requested with
  --upgrade-also.  A package that is requested to be removed and also
  added at the same time gets re-installed or upgraded (when version
  curr != installed).
* choose.h: Declare extern bool PackageCategoryOptions from
  package_meta.cc.
* package_meta.h (packagemeta): add new method bool
  isManuallyDeleted() to indicate that a particular package has been
  deleted from the CLI.
* package_meta.cc: Add new CLI option -x/--remove-packages, packages
  listed here are considered candidates for deletion.
* package_meta.cc: Add new CLI option -c/--remove-categories, packages
  belonging to categories listed here are considered candidates for
  deletion.
* package_meta.cc: Additional bool PackageCategoryOptions to record if
  any manual installations or deletions have been requested.
* package_meta.cc (isManuallyDeleted): Implement along the same lines
  as isManuallyWanted, only for deletion candidates.
* package_db.h (packagedb): Remove method addCommandLinePackages(), it
  isn't needed anymore.
* package_db.cc: Remove implementation for method
  addCommandLinePackages(), it isn't needed anymore.
---
 setup/choose.cc       | 52 +++++++++++++++++++++---------------------------
 setup/choose.h        |  1 +
 setup/package_db.cc   | 17 ----------------
 setup/package_db.h    |  1 -
 setup/package_meta.cc | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++
 setup/package_meta.h  |  2 ++
 6 files changed, 80 insertions(+), 48 deletions(-)

diff --git a/setup/choose.cc b/setup/choose.cc
index cf917cd..70aafaf 100755
--- a/setup/choose.cc
+++ b/setup/choose.cc
@@ -61,6 +61,10 @@ static const char *cvsid =
 
 #include "UserSettings.h"
 
+#include "getopt++/BoolOption.h"
+static BoolOption UpgradeAlsoOption (false, 'g', "upgrade-also", "also upgrade installed packages");
+static BoolOption CleanOrphansOption (false, 'o', "delete-orphans", "remove orphaned packages");
+
 using namespace std;
 
 extern ThreeBarProgressPage Progress;
@@ -148,11 +152,6 @@ ChooserPage::createListview ()
     log (LOG_BABBLE) << "Failed to set View button caption %ld" <<
 	 GetLastError () << endLog;
 
-  for (packagedb::packagecollection::iterator i = db.packages.begin(); i != db.packages.end(); i++)
-    {
-      i->second->set_requirements(chooser->deftrust);
-    }
-
   /* FIXME: do we need to init the desired fields ? */
   static int ta[] = { IDC_CHOOSE_KEEP, IDC_CHOOSE_CURR, IDC_CHOOSE_EXP, 0 };
   rbset (GetHWND (), ta, IDC_CHOOSE_CURR);
@@ -244,33 +243,26 @@ ChooserPage::OnInit ()
   packagedb db;
   db.setExistence ();
   db.fillMissingCategory ();
-  bool bCommandLineAddedPackages = db.addCommandLinePackages();
 
-  // in unattended mode, if packages were selected on the command line using the --packages
-  // or --categories options, just install those selected packages and don't upgrade all others
-  // (we always install all packages in the Base or Misc categories; packages selected on the
-  // command line are added to the Base category)
-  if ((unattended_mode == unattended) && (bCommandLineAddedPackages))
-    {
-      for (packagedb::packagecollection::iterator i = db.packages.begin ();
-           i != db.packages.end (); ++i)
-        {
-          packagemeta & pkg = *(i->second);
-          if (pkg.installed)
-            {
-              pkg.desired = pkg.installed;
-            }
-          else if (pkg.categories.find ("Base") != pkg.categories.end ()
-                   || pkg.categories.find ("Misc") != pkg.categories.end ())
-            {
-              pkg.desired = pkg.trustp(TRUST_CURR);
-              pkg.desired.pick(TRUE, &pkg);
-            }
-        }
-    }
-  else
+  for (packagedb::packagecollection::iterator i = db.packages.begin ();
+       i != db.packages.end (); ++i)
     {
-      db.defaultTrust (TRUST_CURR);
+      packagemeta & pkg = *(i->second);
+      bool wanted    = pkg.isManuallyWanted();
+      bool deleted   = pkg.isManuallyDeleted();
+      bool current   = pkg.curr || CleanOrphansOption;
+      bool upgrade   =  wanted || UpgradeAlsoOption || !PackageCategoryOptions;
+      bool install   =  wanted && !deleted && !pkg.installed;
+      bool reinstall =  wanted && deleted;
+      bool uninstall = !wanted && deleted;
+      if (install)
+	pkg.set_action( packagemeta::Install_action, pkg.curr );
+      else if (reinstall)
+	pkg.set_action( packagemeta::Reinstall_action, pkg.curr );
+      else if (uninstall)
+	pkg.set_action( packagemeta::Uninstall_action, packageversion() );
+      else
+	pkg.set_action( packagemeta::Default_action, ((upgrade && current) ? pkg.curr : pkg.installed) );
     }
 
   ClearBusy ();
diff --git a/setup/choose.h b/setup/choose.h
index b24aefc..86e9dc6 100755
--- a/setup/choose.h
+++ b/setup/choose.h
@@ -21,6 +21,7 @@
 #include "package_meta.h"
 #include "PickView.h"
 
+extern bool PackageCategoryOptions;
 
 class ChooserPage:public PropertyPage
 {
diff --git a/setup/package_db.cc b/setup/package_db.cc
index 1da931c..3578033 100755
--- a/setup/package_db.cc
+++ b/setup/package_db.cc
@@ -422,23 +422,6 @@ packagedb::fillMissingCategory ()
     }
 }
 
-bool
-packagedb::addCommandLinePackages ()
-{
-  bool bReturn = false;
-
-  for (packagedb::packagecollection::iterator i = packages.begin(); i != packages.end(); i++)
-    {
-      if (i->second->isManuallyWanted())
-        {
-          i->second->addToCategoryBase();
-          bReturn = true;
-        }
-    }
-
-  return bReturn;
-}
-
 void
 packagedb::defaultTrust (trusts trust)
 {
diff --git a/setup/package_db.h b/setup/package_db.h
index 63753aa..bc828a1 100755
--- a/setup/package_db.h
+++ b/setup/package_db.h
@@ -70,7 +70,6 @@ public:
   PackageDBConnectedIterator connectedBegin();
   PackageDBConnectedIterator connectedEnd();
   void fillMissingCategory();
-  bool addCommandLinePackages();
   void defaultTrust (trusts trust);
   void markUnVisited();
   void setExistence();
diff --git a/setup/package_meta.cc b/setup/package_meta.cc
index d044eff..ac20e7d 100755
--- a/setup/package_meta.cc
+++ b/setup/package_meta.cc
@@ -51,8 +51,11 @@ using namespace std;
 
 using namespace std;
 
+static StringArrayOption DeletePackageOption ('x', "remove-packages", "Specify packages to uninstall");
+static StringArrayOption DeleteCategoryOption ('c', "remove-categories", "Specify categories to uninstall");
 static StringArrayOption PackageOption ('P', "packages", "Specify packages to install");
 static StringArrayOption CategoryOption ('C', "categories", "Specify entire categories to install");
+bool PackageCategoryOptions = 0;
 
 /*****************/
 
@@ -308,7 +311,9 @@ bool packagemeta::isManuallyWanted() const
 {
   static bool parsed_yet = false;
   static std::set<string> parsed_names;
+  PackageCategoryOptions |= parsed_names.size ();
   static std::set<string> parsed_categories;
+  PackageCategoryOptions |= parsed_categories.size ();
   bool bReturn = false;
 
   /* First time through, we parse all the names out from the 
@@ -352,6 +357,56 @@ bool packagemeta::isManuallyWanted() const
   return bReturn;
 }
 
+bool packagemeta::isManuallyDeleted() const
+{
+  static bool parsed_yet = false;
+  static std::set<string> parsed_delete;
+  PackageCategoryOptions |= parsed_delete.size ();
+  static std::set<string> parsed_delete_categories;
+  PackageCategoryOptions |= parsed_delete_categories.size ();
+  bool bReturn = false;
+
+  /* First time through, we parse all the names out from the
+    option string and store them away in an STL set.  */
+  if (!parsed_yet)
+  {
+    vector<string> delete_options   = DeletePackageOption;
+    vector<string> categories_options = DeleteCategoryOption;
+    for (vector<string>::iterator n = delete_options.begin ();
+		n != delete_options.end (); ++n)
+      {
+	parseNames (parsed_delete, *n);
+      }
+    for (vector<string>::iterator n = categories_options.begin ();
+		n != categories_options.end (); ++n)
+      {
+	parseNames (parsed_delete_categories, *n);
+      }
+    parsed_yet = true;
+  }
+
+  /* Once we've already parsed the option string, just do
+    a lookup in the cache of already-parsed names.  */
+  bReturn = parsed_delete.find(name) != parsed_delete.end();
+
+  /* If we didn't select the package manually, did we select any
+     of the categories it is in? */
+  if (!bReturn && parsed_delete_categories.size ())
+    {
+      std::set<std::string, casecompare_lt_op>::iterator curcat;
+      for (curcat = categories.begin (); curcat != categories.end (); curcat++)
+	if (parsed_delete_categories.find (*curcat) != parsed_delete_categories.end ())
+	  {
+	    log (LOG_PLAIN) << "Found category " << *curcat << " in package " << name << endLog;
+	    bReturn = true;
+	  }
+    }
+
+  if (bReturn)
+    log (LOG_PLAIN) << "Deleted manual package " << name << endLog;
+  return bReturn;
+}
+
 const std::string
 packagemeta::SDesc () const
 {
diff --git a/setup/package_meta.h b/setup/package_meta.h
index 64a77d9..2da4a65 100755
--- a/setup/package_meta.h
+++ b/setup/package_meta.h
@@ -106,6 +106,8 @@ public:
   std::string installed_from;
   /* true if package was selected on command-line. */
   bool isManuallyWanted() const;
+  /* true if package was deleted on command-line. */
+  bool isManuallyDeleted() const;
   /* SDesc is global in theory, across all package versions. 
      LDesc is not: it can be different per version */
   const std::string SDesc () const;
-- 
1.8.1.1


[-- Attachment #2: Type: text/plain, Size: 199 bytes --]



Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [PATCH 4/4] setup.exe

[Achim Gratz]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: 0004-Allow-a-different-basename-instead-of-setup.patch --]
[-- Type: text/x-patch, Size: 2599 bytes --]

From ead437489d0873f983103cef24b708af18a9a391 Mon Sep 17 00:00:00 2001
From: Achim Gratz
Date: Fri, 18 Jan 2013 17:05:52 +0100
Subject: [PATCH 4/4] Allow a different basename (instead of "setup")

* setup/ini.h: Modify macro definition to pick up name from external
  variable SetupBaseName instead of string constant.

* setup/main.cc: New string option "-I" aka "--ini-basename" to feed
  basename into setup and default to "setup".  Copy resulting string
  to the exported variable SetupBaseName.
---
 setup/ini.h   | 6 ++++--
 setup/main.cc | 5 +++++
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/setup/ini.h b/setup/ini.h
index 7276e0a..cf26aa1 100755
--- a/setup/ini.h
+++ b/setup/ini.h
@@ -39,8 +39,10 @@ typedef enum
 } excludes;
 
 extern bool is_legacy;
-#define SETUP_INI_FILENAME (is_legacy ? "setup-legacy.ini" : "setup.ini")
-#define SETUP_BZ2_FILENAME (is_legacy ? "setup-legacy.bz2" : "setup.bz2")
+
+#define SETUP_INI_FILENAME (is_legacy ? "setup-legacy.ini" : (std::string(SetupBaseName)+".ini").c_str())
+#define SETUP_BZ2_FILENAME (is_legacy ? "setup-legacy.bz2" : (std::string(SetupBaseName)+".bz2").c_str())
+extern std::string SetupBaseName;
 
 /* The following three vars are used to facilitate error handling between the
    parser/lexer and its callers, namely ini.cc:do_remote_ini() and
diff --git a/setup/main.cc b/setup/main.cc
index dc73936..b71cb63 100755
--- a/setup/main.cc
+++ b/setup/main.cc
@@ -67,6 +67,7 @@ static const char *cvsid =
 
 #include "getopt++/GetOption.h"
 #include "getopt++/BoolOption.h"
+#include "getopt++/StringOption.h"
 
 #include "Exception.h"
 #include <stdexcept>
@@ -91,6 +92,8 @@ bool is_legacy;
 static BoolOption UnattendedOption (false, 'q', "quiet-mode", "Unattended setup mode");
 static BoolOption PackageManagerOption (false, 'M', "package-manager", "Semi-attended chooser-only mode");
 static BoolOption HelpOption (false, 'h', "help", "print help");
+static StringOption SetupBaseNameOpt ("setup", 'I', "ini-basename", "Use a different basename instead of setup", false);
+std::string SetupBaseName;
 static BOOL WINAPI (*dyn_AttachConsole) (DWORD);
 static BOOL WINAPI (*dyn_GetLongPathName) (LPCTSTR, LPTSTR, DWORD);
 
@@ -289,6 +292,8 @@ WinMain (HINSTANCE h,
     if (unattended_mode || HelpOption)
       set_cout ();
 
+    SetupBaseName = SetupBaseNameOpt;
+
     LogSingleton::SetInstance (*(theLog = LogFile::createLogFile ()));
     const char *sep = isdirsep (local_dir[local_dir.size () - 1]) ? "" : "\\";
     theLog->setFile (LOG_BABBLE, local_dir + sep + "setup.log.full", false);
-- 
1.8.1.1


[-- Attachment #2: Type: text/plain, Size: 183 bytes --]



Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [PATCH 1/4] setup.exe

[Jon TURNEY]

On 25/01/2013 22:10, Achim Gratz wrote:
>  Build commands:
>  1) Configure using this option
>     $ /path/to/setup/bootstrap.sh
> -   This will automatically rebuild configure files and run configure in the
> -   current directory.
> +   This will automatically rebuild configure files and run configure
> +   in the current directory.  The current directory must be outside
> +   the source tree.

I don't think this is needed.  I know cgf said "the build directory should be
separate from the source directory", but it seems to me that setup configures
and builds correctly in the source directory.

Re: [PATCH 0/4] setup.exe

[Jon TURNEY]

On 25/01/2013 22:11, Achim Gratz wrote:
> From a246270621b2b4fbd476aa386fc217ed14f3fe10 Mon Sep 17 00:00:00 2001
> From: Achim Gratz
> Date: Fri, 25 Jan 2013 12:12:41 +0100
> Subject: [PATCH 3/4] Allow delete and reinstall from CLI, re-implement install from CLI

> -  bool bCommandLineAddedPackages = db.addCommandLinePackages();
>  
> -  // in unattended mode, if packages were selected on the command line using the --packages
> -  // or --categories options, just install those selected packages and don't upgrade all others
> -  // (we always install all packages in the Base or Misc categories; packages selected on the
> -  // command line are added to the Base category)
> -  if ((unattended_mode == unattended) && (bCommandLineAddedPackages))
> -    {
> -      for (packagedb::packagecollection::iterator i = db.packages.begin ();
> -           i != db.packages.end (); ++i)
> -        {
> -          packagemeta & pkg = *(i->second);
> -          if (pkg.installed)
> -            {
> -              pkg.desired = pkg.installed;
> -            }
> -          else if (pkg.categories.find ("Base") != pkg.categories.end ()
> -                   || pkg.categories.find ("Misc") != pkg.categories.end ())
> -            {
> -              pkg.desired = pkg.trustp(TRUST_CURR);
> -              pkg.desired.pick(TRUE, &pkg);
> -            }
> -        }
> -    }
> -  else
> +  for (packagedb::packagecollection::iterator i = db.packages.begin ();
> +       i != db.packages.end (); ++i)
>      {
> -      db.defaultTrust (TRUST_CURR);
> +      packagemeta & pkg = *(i->second);
> +      bool wanted    = pkg.isManuallyWanted();
> +      bool deleted   = pkg.isManuallyDeleted();
> +      bool current   = pkg.curr || CleanOrphansOption;
> +      bool upgrade   =  wanted || UpgradeAlsoOption || !PackageCategoryOptions;
> +      bool install   =  wanted && !deleted && !pkg.installed;
> +      bool reinstall =  wanted && deleted;
> +      bool uninstall = !wanted && deleted;
> +      if (install)
> +	pkg.set_action( packagemeta::Install_action, pkg.curr );
> +      else if (reinstall)
> +	pkg.set_action( packagemeta::Reinstall_action, pkg.curr );
> +      else if (uninstall)
> +	pkg.set_action( packagemeta::Uninstall_action, packageversion() );
> +      else
> +	pkg.set_action( packagemeta::Default_action, ((upgrade && current) ? pkg.curr : pkg.installed) );
>      }

It looks to me that this might be removing the code which causes the Base
category to get installed by default on the first install.  Have you tested
that still happens?

>  
>    ClearBusy ();
> diff --git a/setup/choose.h b/setup/choose.h
> index b24aefc..86e9dc6 100755
> --- a/setup/choose.h
> +++ b/setup/choose.h
> @@ -21,6 +21,7 @@
>  #include "package_meta.h"
>  #include "PickView.h"
>  
> +extern bool PackageCategoryOptions;

I think this could have a name which better matches what it does
(areManualPackageSelections?)

Since this is a variable name, not a type name, maybe it should start with a
lower case letter?

Re: [PATCH 1/4] setup.exe

[marco atzeri]

On 2/1/2013 3:40 PM, Jon TURNEY wrote:
> On 25/01/2013 22:10, Achim Gratz wrote:
>>   Build commands:
>>   1) Configure using this option
>>      $ /path/to/setup/bootstrap.sh
>> -   This will automatically rebuild configure files and run configure in the
>> -   current directory.
>> +   This will automatically rebuild configure files and run configure
>> +   in the current directory.  The current directory must be outside
>> +   the source tree.
>
> I don't think this is needed.  I know cgf said "the build directory should be
> separate from the source directory", but it seems to me that setup configures
> and builds correctly in the source directory.
>

I prefer cgf's point of view, it is much simpler to produce patches
if the build tree is separated.

Regards
Marco

Re: [PATCH 1/4] setup.exe

[Achim Gratz]

Jon TURNEY writes:
> I don't think this is needed.  I know cgf said "the build directory
> should be separate from the source directory", but it seems to me that
> setup configures and builds correctly in the source directory.

I tried to build inside the source tree first, but the bootstrap
wouldn't finish correctly.  I can try again in a freshly checked out
tree because maybe I had a different problem that prevented this from
working, but I think it's much easier to simply advise to build outside
the source tree.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [PATCH 0/4] setup.exe

[Achim Gratz]

Jon TURNEY writes:
> It looks to me that this might be removing the code which causes the Base
> category to get installed by default on the first install.  Have you tested
> that still happens?

I think there's a single combination of options where this indeed
occurs: a Base package is not installed nor requested _and_ a manual
package selection is in effect _and_ the "--upgrade-also" option has not
been given.  One could argue that this actually is what the user
requested to do, but since this is different from what happened with the
original code, I'll add a check for this situation (unless someone
argues that the old behaviour wasn't intended anyway).  For any other
combination of options, packagemeta::Default_action will arrange for the
installation/upgrade of those packages even when they're not explicitly
requested.

> I think this could have a name which better matches what it does
> (areManualPackageSelections?)

I've worked from the existing names for the two (now four) options, all
ending with "Option", I can certainly change the name.

> Since this is a variable name, not a type name, maybe it should start with a
> lower case letter?

All options related variable names already started with upper-case
letters, so I left it at that.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html

Re: [PATCH 1/4] setup.exe

[Christopher Faylor]

On Fri, Feb 01, 2013 at 04:11:22PM +0100, marco atzeri wrote:
>On 2/1/2013 3:40 PM, Jon TURNEY wrote:
>> On 25/01/2013 22:10, Achim Gratz wrote:
>>>   Build commands:
>>>   1) Configure using this option
>>>      $ /path/to/setup/bootstrap.sh
>>> -   This will automatically rebuild configure files and run configure in the
>>> -   current directory.
>>> +   This will automatically rebuild configure files and run configure
>>> +   in the current directory.  The current directory must be outside
>>> +   the source tree.
>>
>> I don't think this is needed.  I know cgf said "the build directory should be
>> separate from the source directory", but it seems to me that setup configures
>> and builds correctly in the source directory.
>
>I prefer cgf's point of view, it is much simpler to produce patches
>if the build tree is separated.

So do I but I don't think the intent was that it wouldn't work at all.
If that is the case it probably should be fixed.

cgf

Re: [PATCH 5/4] setup.exe

[Achim Gratz]

[-- Attachment #1: Type: text/plain, Size: 235 bytes --]

Here's a patch on top of that that implements your suggestion.  This is
untested since I only have a i686-w64-mingw32 cross-compilation
toolchain available right now.  I can revise the patch or the whole
series next week if you want.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0005-Rename-variable-and-re-implement-original-behaviour-.patch --]
[-- Type: text/x-patch, Size: 3797 bytes --]

From 7bbdad05e58fb2365bae5b4ed61c2cf4af9619c7 Mon Sep 17 00:00:00 2001
From: Achim Gratz
Date: Fri, 1 Feb 2013 20:22:21 +0100
Subject: [PATCH 5/5] Rename variable and re-implement original behaviour
 exactly

* choose.cc (OnInit): When a package is from category "Base" or "Misc"
  is found uninstalled, select it for installation.  This
  re-implements the previous behaviour for this situation when there
  are manual selections on the CLI and "--upgrade-also" is not given.
* choose.h: Rename PackageCategoryOptions to hasManualSelections.
* choose.cc (OnInit): Rename PackageCategoryOptions to
  hasManualSelections.
* package_meta.cc (isManuallyWanted): Rename PackageCategoryOptions to
  hasManualSelections.
* package_meta.cc (isManuallyDeleted): Rename PackageCategoryOptions
  to hasManualSelections.
---
 setup/choose.cc       |  5 ++++-
 setup/choose.h        |  2 +-
 setup/package_meta.cc | 10 +++++-----
 3 files changed, 10 insertions(+), 7 deletions(-)

diff --git a/setup/choose.cc b/setup/choose.cc
index 70aafaf..57d06f2 100755
--- a/setup/choose.cc
+++ b/setup/choose.cc
@@ -248,10 +248,13 @@ ChooserPage::OnInit ()
        i != db.packages.end (); ++i)
     {
       packagemeta & pkg = *(i->second);
+      bool basemisc  = !pkg.installed
+	&& (    categories.find ("Base") != categories.end ()
+	     || categories.find ("Misc") != categories.end ());
       bool wanted    = pkg.isManuallyWanted();
       bool deleted   = pkg.isManuallyDeleted();
       bool current   = pkg.curr || CleanOrphansOption;
-      bool upgrade   =  wanted || UpgradeAlsoOption || !PackageCategoryOptions;
+      bool upgrade   =  wanted || basemisc || UpgradeAlsoOption || !hasManualSelections;
       bool install   =  wanted && !deleted && !pkg.installed;
       bool reinstall =  wanted && deleted;
       bool uninstall = !wanted && deleted;
diff --git a/setup/choose.h b/setup/choose.h
index 86e9dc6..9dc5882 100755
--- a/setup/choose.h
+++ b/setup/choose.h
@@ -21,7 +21,7 @@
 #include "package_meta.h"
 #include "PickView.h"
 
-extern bool PackageCategoryOptions;
+extern bool hasManualSelections;
 
 class ChooserPage:public PropertyPage
 {
diff --git a/setup/package_meta.cc b/setup/package_meta.cc
index ac20e7d..fbb7a68 100755
--- a/setup/package_meta.cc
+++ b/setup/package_meta.cc
@@ -55,7 +55,7 @@ static StringArrayOption DeletePackageOption ('x', "remove-packages", "Specify p
 static StringArrayOption DeleteCategoryOption ('c', "remove-categories", "Specify categories to uninstall");
 static StringArrayOption PackageOption ('P', "packages", "Specify packages to install");
 static StringArrayOption CategoryOption ('C', "categories", "Specify entire categories to install");
-bool PackageCategoryOptions = 0;
+bool hasManualSelections = 0;
 
 /*****************/
 
@@ -311,9 +311,9 @@ bool packagemeta::isManuallyWanted() const
 {
   static bool parsed_yet = false;
   static std::set<string> parsed_names;
-  PackageCategoryOptions |= parsed_names.size ();
+  hasManualSelections |= parsed_names.size ();
   static std::set<string> parsed_categories;
-  PackageCategoryOptions |= parsed_categories.size ();
+  hasManualSelections |= parsed_categories.size ();
   bool bReturn = false;
 
   /* First time through, we parse all the names out from the 
@@ -361,9 +361,9 @@ bool packagemeta::isManuallyDeleted() const
 {
   static bool parsed_yet = false;
   static std::set<string> parsed_delete;
-  PackageCategoryOptions |= parsed_delete.size ();
+  hasManualSelections |= parsed_delete.size ();
   static std::set<string> parsed_delete_categories;
-  PackageCategoryOptions |= parsed_delete_categories.size ();
+  hasManualSelections |= parsed_delete_categories.size ();
   bool bReturn = false;
 
   /* First time through, we parse all the names out from the
-- 
1.8.1.2


[-- Attachment #3: Type: text/plain, Size: 190 bytes --]



Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [PATCH 3/4] setup.exe

[Achim Gratz]

[-- Attachment #1: Type: text/plain, Size: 313 bytes --]

Here's the third patch again with all the changes requested by John
integrated (and the subject line corrected).  This is now fully tested
(I hope I didn't forget anything).  Additionally I have arranged that
packages in categories "Base" and "Misc" can not be deleted, an attempt
to do so will re-install them.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0003-Allow-delete-and-reinstall-from-CLI-re-implement-ins.patch --]
[-- Type: text/x-patch, Size: 10827 bytes --]

From 0527144754297f1bb552c4ee337044b8a16f3548 Mon Sep 17 00:00:00 2001
From: Achim Gratz <Achim.Gratz@Infineon.com>
Date: Mon, 4 Feb 2013 14:28:26 +0100
Subject: [PATCH 3/5] Allow delete and reinstall from CLI, re-implement install
 from CLI

* choose.cc: Add new CLI option -g/--upgrade-also, considers all
  installed packages as candidates for upgrades (default when no CLI
  package or category options have been given).
* choose.cc: Add new CLI option -o/--delete-orphans, considers
  installed packages that do not exist anymore in the package
  repositories as candidates for deletion.
* choose.cc (createListview): remove superflous and detrimental
  default trust setting.  This has already been set correctly in
  OnInit.
* choose.cc (OnInit): Re-implement package handling depending on
  options given on CLI using package actions instead of package_meta
  low-level functions.  When no CLI package or category options have
  been given, assume --upgrade-also in accordance with the previous
  behaviour.  No such assumption is made when packages are to be added
  or removed from the CLI, but this behaviour can be requested with
  --upgrade-also.  A package that is requested to be removed and also
  added at the same time gets re-installed or upgraded (when version
  curr != installed).  Uninstalled packages in categories "Base" or
  "Misc" are always selected for installation; installed packages in
  these categories are not eligible for deletion and will be
  reinstalled instead.
* choose.h: Declare extern bool PackageCategoryOptions from
  package_meta.cc.
* package_meta.h (packagemeta): add new method bool
  isManuallyDeleted() to indicate that a particular package has been
  deleted from the CLI.
* package_meta.cc: Add new CLI option -x/--remove-packages, packages
  listed here are considered candidates for deletion.
* package_meta.cc: Add new CLI option -c/--remove-categories, packages
  belonging to categories listed here are considered candidates for
  deletion.
* package_meta.cc: Additional bool PackageCategoryOptions to record if
  any manual installations or deletions have been requested.
* package_meta.cc (isManuallyDeleted): Implement along the same lines
  as isManuallyWanted, only for deletion candidates.
* package_db.h (packagedb): Remove method addCommandLinePackages(), it
  isn't needed anymore.
---
 setup/choose.cc       | 54 ++++++++++++++++++++++----------------------------
 setup/choose.h        |  1 +
 setup/package_db.cc   | 17 ----------------
 setup/package_db.h    |  1 -
 setup/package_meta.cc | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++
 setup/package_meta.h  |  2 ++
 6 files changed, 82 insertions(+), 48 deletions(-)

diff --git a/setup/choose.cc b/setup/choose.cc
index cf917cd..f48f189 100755
--- a/setup/choose.cc
+++ b/setup/choose.cc
@@ -61,6 +61,10 @@ static const char *cvsid =
 
 #include "UserSettings.h"
 
+#include "getopt++/BoolOption.h"
+static BoolOption UpgradeAlsoOption (false, 'g', "upgrade-also", "also upgrade installed packages");
+static BoolOption CleanOrphansOption (false, 'o', "delete-orphans", "remove orphaned packages");
+
 using namespace std;
 
 extern ThreeBarProgressPage Progress;
@@ -148,11 +152,6 @@ ChooserPage::createListview ()
     log (LOG_BABBLE) << "Failed to set View button caption %ld" <<
 	 GetLastError () << endLog;
 
-  for (packagedb::packagecollection::iterator i = db.packages.begin(); i != db.packages.end(); i++)
-    {
-      i->second->set_requirements(chooser->deftrust);
-    }
-
   /* FIXME: do we need to init the desired fields ? */
   static int ta[] = { IDC_CHOOSE_KEEP, IDC_CHOOSE_CURR, IDC_CHOOSE_EXP, 0 };
   rbset (GetHWND (), ta, IDC_CHOOSE_CURR);
@@ -244,33 +243,28 @@ ChooserPage::OnInit ()
   packagedb db;
   db.setExistence ();
   db.fillMissingCategory ();
-  bool bCommandLineAddedPackages = db.addCommandLinePackages();
 
-  // in unattended mode, if packages were selected on the command line using the --packages
-  // or --categories options, just install those selected packages and don't upgrade all others
-  // (we always install all packages in the Base or Misc categories; packages selected on the
-  // command line are added to the Base category)
-  if ((unattended_mode == unattended) && (bCommandLineAddedPackages))
-    {
-      for (packagedb::packagecollection::iterator i = db.packages.begin ();
-           i != db.packages.end (); ++i)
-        {
-          packagemeta & pkg = *(i->second);
-          if (pkg.installed)
-            {
-              pkg.desired = pkg.installed;
-            }
-          else if (pkg.categories.find ("Base") != pkg.categories.end ()
-                   || pkg.categories.find ("Misc") != pkg.categories.end ())
-            {
-              pkg.desired = pkg.trustp(TRUST_CURR);
-              pkg.desired.pick(TRUE, &pkg);
-            }
-        }
-    }
-  else
+  for (packagedb::packagecollection::iterator i = db.packages.begin ();
+       i != db.packages.end (); ++i)
     {
-      db.defaultTrust (TRUST_CURR);
+      packagemeta & pkg = *(i->second);
+      bool wanted    = pkg.isManuallyWanted();
+      bool deleted   = pkg.isManuallyDeleted();
+      bool basemisc  = (pkg.categories.find ("Base") != pkg.categories.end ()
+		     || pkg.categories.find ("Misc") != pkg.categories.end ());
+      bool current   = pkg.curr || CleanOrphansOption;
+      bool upgrade   =  wanted  || (!pkg.installed && basemisc) || UpgradeAlsoOption || !hasManualSelections;
+      bool install   =   wanted  && !deleted && !pkg.installed;
+      bool reinstall =  (wanted  || basemisc ) && deleted;
+      bool uninstall = !(wanted  || basemisc ) && deleted;
+      if (install)
+	pkg.set_action( packagemeta::Install_action, pkg.curr );
+      else if (reinstall)
+	pkg.set_action( packagemeta::Reinstall_action, pkg.curr );
+      else if (uninstall)
+	pkg.set_action( packagemeta::Uninstall_action, packageversion() );
+      else
+	pkg.set_action( packagemeta::Default_action, ((upgrade && current) ? pkg.curr : pkg.installed) );
     }
 
   ClearBusy ();
diff --git a/setup/choose.h b/setup/choose.h
index b24aefc..9dc5882 100755
--- a/setup/choose.h
+++ b/setup/choose.h
@@ -21,6 +21,7 @@
 #include "package_meta.h"
 #include "PickView.h"
 
+extern bool hasManualSelections;
 
 class ChooserPage:public PropertyPage
 {
diff --git a/setup/package_db.cc b/setup/package_db.cc
index 1da931c..3578033 100755
--- a/setup/package_db.cc
+++ b/setup/package_db.cc
@@ -422,23 +422,6 @@ packagedb::fillMissingCategory ()
     }
 }
 
-bool
-packagedb::addCommandLinePackages ()
-{
-  bool bReturn = false;
-
-  for (packagedb::packagecollection::iterator i = packages.begin(); i != packages.end(); i++)
-    {
-      if (i->second->isManuallyWanted())
-        {
-          i->second->addToCategoryBase();
-          bReturn = true;
-        }
-    }
-
-  return bReturn;
-}
-
 void
 packagedb::defaultTrust (trusts trust)
 {
diff --git a/setup/package_db.h b/setup/package_db.h
index 63753aa..bc828a1 100755
--- a/setup/package_db.h
+++ b/setup/package_db.h
@@ -70,7 +70,6 @@ public:
   PackageDBConnectedIterator connectedBegin();
   PackageDBConnectedIterator connectedEnd();
   void fillMissingCategory();
-  bool addCommandLinePackages();
   void defaultTrust (trusts trust);
   void markUnVisited();
   void setExistence();
diff --git a/setup/package_meta.cc b/setup/package_meta.cc
index d044eff..fbb7a68 100755
--- a/setup/package_meta.cc
+++ b/setup/package_meta.cc
@@ -51,8 +51,11 @@ using namespace std;
 
 using namespace std;
 
+static StringArrayOption DeletePackageOption ('x', "remove-packages", "Specify packages to uninstall");
+static StringArrayOption DeleteCategoryOption ('c', "remove-categories", "Specify categories to uninstall");
 static StringArrayOption PackageOption ('P', "packages", "Specify packages to install");
 static StringArrayOption CategoryOption ('C', "categories", "Specify entire categories to install");
+bool hasManualSelections = 0;
 
 /*****************/
 
@@ -308,7 +311,9 @@ bool packagemeta::isManuallyWanted() const
 {
   static bool parsed_yet = false;
   static std::set<string> parsed_names;
+  hasManualSelections |= parsed_names.size ();
   static std::set<string> parsed_categories;
+  hasManualSelections |= parsed_categories.size ();
   bool bReturn = false;
 
   /* First time through, we parse all the names out from the 
@@ -352,6 +357,56 @@ bool packagemeta::isManuallyWanted() const
   return bReturn;
 }
 
+bool packagemeta::isManuallyDeleted() const
+{
+  static bool parsed_yet = false;
+  static std::set<string> parsed_delete;
+  hasManualSelections |= parsed_delete.size ();
+  static std::set<string> parsed_delete_categories;
+  hasManualSelections |= parsed_delete_categories.size ();
+  bool bReturn = false;
+
+  /* First time through, we parse all the names out from the
+    option string and store them away in an STL set.  */
+  if (!parsed_yet)
+  {
+    vector<string> delete_options   = DeletePackageOption;
+    vector<string> categories_options = DeleteCategoryOption;
+    for (vector<string>::iterator n = delete_options.begin ();
+		n != delete_options.end (); ++n)
+      {
+	parseNames (parsed_delete, *n);
+      }
+    for (vector<string>::iterator n = categories_options.begin ();
+		n != categories_options.end (); ++n)
+      {
+	parseNames (parsed_delete_categories, *n);
+      }
+    parsed_yet = true;
+  }
+
+  /* Once we've already parsed the option string, just do
+    a lookup in the cache of already-parsed names.  */
+  bReturn = parsed_delete.find(name) != parsed_delete.end();
+
+  /* If we didn't select the package manually, did we select any
+     of the categories it is in? */
+  if (!bReturn && parsed_delete_categories.size ())
+    {
+      std::set<std::string, casecompare_lt_op>::iterator curcat;
+      for (curcat = categories.begin (); curcat != categories.end (); curcat++)
+	if (parsed_delete_categories.find (*curcat) != parsed_delete_categories.end ())
+	  {
+	    log (LOG_PLAIN) << "Found category " << *curcat << " in package " << name << endLog;
+	    bReturn = true;
+	  }
+    }
+
+  if (bReturn)
+    log (LOG_PLAIN) << "Deleted manual package " << name << endLog;
+  return bReturn;
+}
+
 const std::string
 packagemeta::SDesc () const
 {
diff --git a/setup/package_meta.h b/setup/package_meta.h
index 64a77d9..2da4a65 100755
--- a/setup/package_meta.h
+++ b/setup/package_meta.h
@@ -106,6 +106,8 @@ public:
   std::string installed_from;
   /* true if package was selected on command-line. */
   bool isManuallyWanted() const;
+  /* true if package was deleted on command-line. */
+  bool isManuallyDeleted() const;
   /* SDesc is global in theory, across all package versions. 
      LDesc is not: it can be different per version */
   const std::string SDesc () const;
-- 
1.8.1.2


[-- Attachment #3: Type: text/plain, Size: 197 bytes --]



Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: [PATCH 1/4] setup.exe

[Jon TURNEY]

On 25/01/2013 22:10, Achim Gratz wrote:
> Subject: [PATCH 1/4] README: document some recent changes in the build environment

I've applied the uncontroversial part of this patch.

In future, please use the ChangeLog format linked to in [1]. (emacs protip:
C-x 4 a :-))

[1] http://cygwin.com/contrib.html

> +Additionally, libgetopt++ (also available from the cygwin-apps CVS at
> +sourceware.org) must be available directly as a subdirectory
> +libgetopt++ within the setup source directory.  It should be
> +automatically populated by CVS, but if you are using tools like
> +CVSgrab, you may need to fetch this yourself.

I had a go at rewriting this, but "The libgetopt++ CVS module should be
automatically checked out in a subdirectory, unless you use a tool which
doesn't" didn't seem that helpful an observation :-)

Re: [PATCH 1/4] setup.exe

[Achim Gratz]

Jon TURNEY writes:
> I've applied the uncontroversial part of this patch.

Thank you.

> In future, please use the ChangeLog format linked to in [1]. (emacs protip:
> C-x 4 a :-))

Noted.

> I had a go at rewriting this, but "The libgetopt++ CVS module should be
> automatically checked out in a subdirectory, unless you use a tool which
> doesn't" didn't seem that helpful an observation :-)

I tried this again from home (where no firewall interferes with getting
at the CVS repository server directly).  Using cvs directly, this
directory is populated while doing the checkout, no extra steps
necessary.  On the other hand, git cvsimport also does not pull the
libgetopt++ subdirectory by default (I've created it as a submodule
instead and did another cvsimport there).  Just like CVSgrab it doesn't
seem to understand the concept of a CVS module within another module or
it simply doesn't care to look up CVSROOT/modules where thst gets
defined.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [PATCH 1/4] setup.exe

[Christopher Faylor]

On Fri, Feb 08, 2013 at 10:30:21PM +0100, Achim Gratz wrote:
>Jon TURNEY writes:
>> I've applied the uncontroversial part of this patch.
>
>Thank you.
>
>> In future, please use the ChangeLog format linked to in [1]. (emacs protip:
>> C-x 4 a :-))
>
>Noted.
>
>> I had a go at rewriting this, but "The libgetopt++ CVS module should be
>> automatically checked out in a subdirectory, unless you use a tool which
>> doesn't" didn't seem that helpful an observation :-)
>
>I tried this again from home (where no firewall interferes with getting
>at the CVS repository server directly).  Using cvs directly, this
>directory is populated while doing the checkout, no extra steps
>necessary.  On the other hand, git cvsimport also does not pull the
>libgetopt++ subdirectory by default (I've created it as a submodule
>instead and did another cvsimport there).  Just like CVSgrab it doesn't
>seem to understand the concept of a CVS module within another module or
>it simply doesn't care to look up CVSROOT/modules where thst gets
>defined.

Achim,
Would you consider rebundling your patch so that it only includes
--remove-packages and --remove-categories options?  Those are obviously
missing command-line functionality.

Regarding the autorebase changes, I am not against the idea of implementing
a general purpose mechanism to have setup.exe "do something" when it notices
certain file patterns being added or deleted.  This would move the current
'autodep' functionality from the "upset" program which creates setup.ini into
setup.exe itself.

This is the contents of _autorebase's setup.hint:

sdesc: "Run rebaseall automatically"
#external-source: rebase
category: _PostInstallLast
requires: rebase dash
autodep: .*/[^/]*\.(?:dll|so|oct)$
incver_ifdep: yes

I'm willing to move the autodep into setup.ini if someone is willing to
handle regex parsing like that in setup.  It would probably need to look
something like:

autodep: .*/[^/]*\.(?:dll|so|oct)$
autorun: /path/to/program

If we do something like this we can clean up the handling of .info files
too.

cgf

Re: [PATCH 1/4] setup.exe

[Achim Gratz]

Christopher Faylor writes:
> Would you consider rebundling your patch so that it only includes
> --remove-packages and --remove-categories options?  Those are obviously
> missing command-line functionality.

I'll consider anything that makes them more acceptable.  BTW, I've just
renamed the "--remove-packages" short option to "-p" so that it rhymes
more easily with the opposite "-P" (like it's already the case for
categories).

> Regarding the autorebase changes, I am not against the idea of implementing
> a general purpose mechanism to have setup.exe "do something" when it notices
> certain file patterns being added or deleted.

That actually is another useful thing, yes.  But I specifically wanted
something that does not depend on setup.ini changing since that lets me
do some housekeeping tasks even when setup.exe would otherwise determine
that it hasn't anything to do.  Again, this is maybe not generally
useful for a single user, but with central deployment onto machines that
can have very different starting points it helps a lot.

This could of course be implemented in a different way (I think it
should become simpler in the code actually), like having a special
post-setup run directory to which to install such scripts or providing
another setup.ini key to mark these packages.

>  This would move the current
> 'autodep' functionality from the "upset" program which creates setup.ini into
> setup.exe itself.
>
> This is the contents of _autorebase's setup.hint:
>
> sdesc: "Run rebaseall automatically"
> #external-source: rebase
> category: _PostInstallLast
> requires: rebase dash
> autodep: .*/[^/]*\.(?:dll|so|oct)$
> incver_ifdep: yes

I've been wondering about this before... the other autodependency is on
cygwin, IIRC.

> I'm willing to move the autodep into setup.ini if someone is willing to
> handle regex parsing like that in setup.  It would probably need to look
> something like:
>
> autodep: .*/[^/]*\.(?:dll|so|oct)$
> autorun: /path/to/program
>
> If we do something like this we can clean up the handling of .info files
> too.

I'll have a look.  I don't remember seeing any regex stuff already there
that I could piggyback on (a simple list of extensions to test against
might be easier than pulling in a globbing library), but I found to my
surprise that setup.exe would be perfectly capable of resolving
versioned dependencies (although I'm not quite certain exactly how the
syntax would look and it has probably never been tested).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [PATCH 0/3] setup: implement CLI options

[Achim Gratz]

Christopher Faylor writes:
> Would you consider rebundling your patch so that it only includes
> --remove-packages and --remove-categories options?  Those are obviously
> missing command-line functionality.

I hope I understood correctly what you were asking, here's what I
prepared.  The meat of the patch (a split from patch 3/4 of the earlier
series and with some small changes) is the first one, to be applied on
top of CVS 2.788.  This implements _only_ the deletion of packages or
categories from the command line and nothing else.

The second patch is much smaller and implements an option to upgrade the
installed packages along with potentially installing or deleting others.
This actually was the original behaviour of setup.exe but that quickly
drew a negative response from folks that didn't want to upgrade all of
Cygwin while installing a single extra package.  After the
implementation was changed to their current behaviour, there were a
handful of responses on the Cygwin mailing list to get the old behaviour
back.  This switch implements it again for those who want it.

The third patch, again rather small, implements an option to remove
packages that are installed, but no longer available from the package
source.  This is useful when switching between different package sources
or for cleaning up very old installations that have missed the
obsoletion of packages.  This should be used together with a complete
set of categories or packages to install in order to keep the
installation consistent.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: [PATCH 1/3] setup: implement CLI options

[Achim Gratz]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: 0001-Allow-delete-and-reinstall-from-CLI-re-implement-ins.patch --]
[-- Type: text/x-patch, Size: 9745 bytes --]

From c06d2094f372ef1426b723231397532c69866390 Mon Sep 17 00:00:00 2001
From: Achim Gratz
Date: Tue, 12 Feb 2013 16:33:40 +0100
Subject: [PATCH 1/3] Allow delete and reinstall from CLI, re-implement install
 from CLI

	* package_meta.h (DeletePackageOption): Add new CLI option
	-p/--remove-packages, packages specified are candidates for
	deletion.
	(DeleteCategoryOption): Add new CLI option -c/--remove-categories
	all packages in the category specified are candidates for
	deletion.
	(isManuallyDeleted): Add new method to check if a particular
	package is requested to be deleted from the CLI.
	(hasManualSelections): Additional boolean to record if any manual
	installations or deletions have been requested from the CLI.
	* choose.cc (hasManualSelections): Declare extern, import from
	package_meta.cc.
	(createListview): remove superflous and detrimental default trust
	setting.  This has already been set correctly in OnInit.
	(OnInit): Re-implement package handling depending on options given
	on CLI using package actions instead of low-level functions.  When
	no CLI package or category options have been given, upgrade
	installed packages.  Do not upgrade existing packages when
	packages are added or removed from the CLI.  A package that is
	requested to be removed and also added at the same time gets
	re-installed or upgraded as package accessibility dictates.
	Uninstalled packages in categories "Base" or "Misc" are always
	selected for installation; already installed packages in these
	categories are not eligible for deletion and will be reinstalled
	or upgraded instead.
	* package_db.h (addCommandLinePackages): Remove unused method.
---
 setup/choose.cc       | 50 +++++++++++++++++++---------------------------
 setup/choose.h        |  1 +
 setup/package_db.cc   | 17 ----------------
 setup/package_db.h    |  1 -
 setup/package_meta.cc | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++
 setup/package_meta.h  |  2 ++
 6 files changed, 78 insertions(+), 48 deletions(-)

diff --git a/setup/choose.cc b/setup/choose.cc
index cf917cd..0f260f7 100755
--- a/setup/choose.cc
+++ b/setup/choose.cc
@@ -148,11 +148,6 @@ ChooserPage::createListview ()
     log (LOG_BABBLE) << "Failed to set View button caption %ld" <<
 	 GetLastError () << endLog;
 
-  for (packagedb::packagecollection::iterator i = db.packages.begin(); i != db.packages.end(); i++)
-    {
-      i->second->set_requirements(chooser->deftrust);
-    }
-
   /* FIXME: do we need to init the desired fields ? */
   static int ta[] = { IDC_CHOOSE_KEEP, IDC_CHOOSE_CURR, IDC_CHOOSE_EXP, 0 };
   rbset (GetHWND (), ta, IDC_CHOOSE_CURR);
@@ -244,33 +239,28 @@ ChooserPage::OnInit ()
   packagedb db;
   db.setExistence ();
   db.fillMissingCategory ();
-  bool bCommandLineAddedPackages = db.addCommandLinePackages();
 
-  // in unattended mode, if packages were selected on the command line using the --packages
-  // or --categories options, just install those selected packages and don't upgrade all others
-  // (we always install all packages in the Base or Misc categories; packages selected on the
-  // command line are added to the Base category)
-  if ((unattended_mode == unattended) && (bCommandLineAddedPackages))
-    {
-      for (packagedb::packagecollection::iterator i = db.packages.begin ();
-           i != db.packages.end (); ++i)
-        {
-          packagemeta & pkg = *(i->second);
-          if (pkg.installed)
-            {
-              pkg.desired = pkg.installed;
-            }
-          else if (pkg.categories.find ("Base") != pkg.categories.end ()
-                   || pkg.categories.find ("Misc") != pkg.categories.end ())
-            {
-              pkg.desired = pkg.trustp(TRUST_CURR);
-              pkg.desired.pick(TRUE, &pkg);
-            }
-        }
-    }
-  else
+  for (packagedb::packagecollection::iterator i = db.packages.begin ();
+       i != db.packages.end (); ++i)
     {
-      db.defaultTrust (TRUST_CURR);
+      packagemeta & pkg = *(i->second);
+      bool wanted    = pkg.isManuallyWanted();
+      bool deleted   = pkg.isManuallyDeleted();
+      bool basemisc  = (pkg.categories.find ("Base") != pkg.categories.end ()
+		     || pkg.categories.find ("Misc") != pkg.categories.end ());
+      bool current   = pkg.curr;
+      bool upgrade   =  wanted  || (!pkg.installed && basemisc) || !hasManualSelections;
+      bool install   =   wanted  && !deleted && !pkg.installed;
+      bool reinstall =  ((wanted  || basemisc ) && deleted) && pkg.installed.accessible ();
+      bool uninstall = !(wanted  || basemisc ) && deleted;
+      if (install)
+	pkg.set_action( packagemeta::Install_action, pkg.curr );
+      else if (reinstall)
+	pkg.set_action( packagemeta::Reinstall_action, pkg.curr );
+      else if (uninstall)
+	pkg.set_action( packagemeta::Uninstall_action, packageversion() );
+      else
+	pkg.set_action( packagemeta::Default_action, ((upgrade && current) ? pkg.curr : pkg.installed) );
     }
 
   ClearBusy ();
diff --git a/setup/choose.h b/setup/choose.h
index b24aefc..9dc5882 100755
--- a/setup/choose.h
+++ b/setup/choose.h
@@ -21,6 +21,7 @@
 #include "package_meta.h"
 #include "PickView.h"
 
+extern bool hasManualSelections;
 
 class ChooserPage:public PropertyPage
 {
diff --git a/setup/package_db.cc b/setup/package_db.cc
index 1da931c..3578033 100755
--- a/setup/package_db.cc
+++ b/setup/package_db.cc
@@ -422,23 +422,6 @@ packagedb::fillMissingCategory ()
     }
 }
 
-bool
-packagedb::addCommandLinePackages ()
-{
-  bool bReturn = false;
-
-  for (packagedb::packagecollection::iterator i = packages.begin(); i != packages.end(); i++)
-    {
-      if (i->second->isManuallyWanted())
-        {
-          i->second->addToCategoryBase();
-          bReturn = true;
-        }
-    }
-
-  return bReturn;
-}
-
 void
 packagedb::defaultTrust (trusts trust)
 {
diff --git a/setup/package_db.h b/setup/package_db.h
index 63753aa..bc828a1 100755
--- a/setup/package_db.h
+++ b/setup/package_db.h
@@ -70,7 +70,6 @@ public:
   PackageDBConnectedIterator connectedBegin();
   PackageDBConnectedIterator connectedEnd();
   void fillMissingCategory();
-  bool addCommandLinePackages();
   void defaultTrust (trusts trust);
   void markUnVisited();
   void setExistence();
diff --git a/setup/package_meta.cc b/setup/package_meta.cc
index d044eff..c4d9b34 100755
--- a/setup/package_meta.cc
+++ b/setup/package_meta.cc
@@ -51,8 +51,11 @@ using namespace std;
 
 using namespace std;
 
+static StringArrayOption DeletePackageOption ('p', "remove-packages", "Specify packages to uninstall");
 static StringArrayOption PackageOption ('P', "packages", "Specify packages to install");
+static StringArrayOption DeleteCategoryOption ('c', "remove-categories", "Specify categories to uninstall");
 static StringArrayOption CategoryOption ('C', "categories", "Specify entire categories to install");
+bool hasManualSelections = 0;
 
 /*****************/
 
@@ -308,7 +311,9 @@ bool packagemeta::isManuallyWanted() const
 {
   static bool parsed_yet = false;
   static std::set<string> parsed_names;
+  hasManualSelections |= parsed_names.size ();
   static std::set<string> parsed_categories;
+  hasManualSelections |= parsed_categories.size ();
   bool bReturn = false;
 
   /* First time through, we parse all the names out from the 
@@ -352,6 +357,56 @@ bool packagemeta::isManuallyWanted() const
   return bReturn;
 }
 
+bool packagemeta::isManuallyDeleted() const
+{
+  static bool parsed_yet = false;
+  static std::set<string> parsed_delete;
+  hasManualSelections |= parsed_delete.size ();
+  static std::set<string> parsed_delete_categories;
+  hasManualSelections |= parsed_delete_categories.size ();
+  bool bReturn = false;
+
+  /* First time through, we parse all the names out from the
+    option string and store them away in an STL set.  */
+  if (!parsed_yet)
+  {
+    vector<string> delete_options   = DeletePackageOption;
+    vector<string> categories_options = DeleteCategoryOption;
+    for (vector<string>::iterator n = delete_options.begin ();
+		n != delete_options.end (); ++n)
+      {
+	parseNames (parsed_delete, *n);
+      }
+    for (vector<string>::iterator n = categories_options.begin ();
+		n != categories_options.end (); ++n)
+      {
+	parseNames (parsed_delete_categories, *n);
+      }
+    parsed_yet = true;
+  }
+
+  /* Once we've already parsed the option string, just do
+    a lookup in the cache of already-parsed names.  */
+  bReturn = parsed_delete.find(name) != parsed_delete.end();
+
+  /* If we didn't select the package manually, did we select any
+     of the categories it is in? */
+  if (!bReturn && parsed_delete_categories.size ())
+    {
+      std::set<std::string, casecompare_lt_op>::iterator curcat;
+      for (curcat = categories.begin (); curcat != categories.end (); curcat++)
+	if (parsed_delete_categories.find (*curcat) != parsed_delete_categories.end ())
+	  {
+	    log (LOG_PLAIN) << "Found category " << *curcat << " in package " << name << endLog;
+	    bReturn = true;
+	  }
+    }
+
+  if (bReturn)
+    log (LOG_PLAIN) << "Deleted manual package " << name << endLog;
+  return bReturn;
+}
+
 const std::string
 packagemeta::SDesc () const
 {
diff --git a/setup/package_meta.h b/setup/package_meta.h
index 64a77d9..2da4a65 100755
--- a/setup/package_meta.h
+++ b/setup/package_meta.h
@@ -106,6 +106,8 @@ public:
   std::string installed_from;
   /* true if package was selected on command-line. */
   bool isManuallyWanted() const;
+  /* true if package was deleted on command-line. */
+  bool isManuallyDeleted() const;
   /* SDesc is global in theory, across all package versions. 
      LDesc is not: it can be different per version */
   const std::string SDesc () const;
-- 
1.8.1.2


[-- Attachment #2: Type: text/plain, Size: 199 bytes --]



Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [PATCH 2/3] setup: implement CLI options

[Achim Gratz]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: 0002-additional-option-g-upgrade-also-to-upgrade-installe.patch --]
[-- Type: text/x-patch, Size: 1753 bytes --]

From 4343022ae575fc7836cf084797ba02fdc5f04d01 Mon Sep 17 00:00:00 2001
From: Achim Gratz
Date: Tue, 12 Feb 2013 16:42:57 +0100
Subject: [PATCH 2/3] additional option -g/--upgrade-also to upgrade installed
 packages in the presence of manual selections from the CLI

	* choose.cc (UpgradeAlsoOption): Add option -g/--upgrade-also
	to indicate that installed packages are candidates for
	upgrade, even when manual selection from the CLI are in
	effect.  This avoids having to run setup.exe two times, once
	to add any new packages and then again to perform any upgrades
	and instead performing the two tasks in a single run.
---
 setup/choose.cc | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/setup/choose.cc b/setup/choose.cc
index 0f260f7..acae63b 100755
--- a/setup/choose.cc
+++ b/setup/choose.cc
@@ -61,6 +61,8 @@ static const char *cvsid =
 
 #include "UserSettings.h"
 
+#include "getopt++/BoolOption.h"
+static BoolOption UpgradeAlsoOption (false, 'g', "upgrade-also", "also upgrade installed packages");
 using namespace std;
 
 extern ThreeBarProgressPage Progress;
@@ -249,7 +251,7 @@ ChooserPage::OnInit ()
       bool basemisc  = (pkg.categories.find ("Base") != pkg.categories.end ()
 		     || pkg.categories.find ("Misc") != pkg.categories.end ());
       bool current   = pkg.curr;
-      bool upgrade   =  wanted  || (!pkg.installed && basemisc) || !hasManualSelections;
+      bool upgrade   =  wanted  || (!pkg.installed && basemisc) || UpgradeAlsoOption || !hasManualSelections;
       bool install   =   wanted  && !deleted && !pkg.installed;
       bool reinstall =  ((wanted  || basemisc ) && deleted) && pkg.installed.accessible ();
       bool uninstall = !(wanted  || basemisc ) && deleted;
-- 
1.8.1.2


[-- Attachment #2: Type: text/plain, Size: 179 bytes --]



Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [PATCH 3/3] setup: implement CLI options

[Achim Gratz]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: 0003-implement-option-o-clean-orphans-to-delete-packages-.patch --]
[-- Type: text/x-patch, Size: 1813 bytes --]

From 4c4e8b0f0f80d73bd78e49ca92648b4180be99de Mon Sep 17 00:00:00 2001
From: Achim Gratz
Date: Tue, 12 Feb 2013 16:51:18 +0100
Subject: [PATCH 3/3] implement option -o/-clean-orphans to delete packages
 that are no longer available in the package source

	* choose.cc (CleanOrphansOption): Add option to indicate that
	packages which are no longer present in setup.ini are
	candidates for deletion.  This can be used when package
	sources have been switched or to clean up old installations
	that are missing several generations of package obsolescense.
	This option should be accompanied by an installation request
	for all desired categories or packages.
---
 setup/choose.cc | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/setup/choose.cc b/setup/choose.cc
index acae63b..d6a2eec 100755
--- a/setup/choose.cc
+++ b/setup/choose.cc
@@ -63,6 +63,8 @@ static const char *cvsid =
 
 #include "getopt++/BoolOption.h"
 static BoolOption UpgradeAlsoOption (false, 'g', "upgrade-also", "also upgrade installed packages");
+static BoolOption CleanOrphansOption (false, 'o', "delete-orphans", "remove orphaned packages");
+
 using namespace std;
 
 extern ThreeBarProgressPage Progress;
@@ -250,7 +252,7 @@ ChooserPage::OnInit ()
       bool deleted   = pkg.isManuallyDeleted();
       bool basemisc  = (pkg.categories.find ("Base") != pkg.categories.end ()
 		     || pkg.categories.find ("Misc") != pkg.categories.end ());
-      bool current   = pkg.curr;
+      bool current   = pkg.curr || CleanOrphansOption;
       bool upgrade   =  wanted  || (!pkg.installed && basemisc) || UpgradeAlsoOption || !hasManualSelections;
       bool install   =   wanted  && !deleted && !pkg.installed;
       bool reinstall =  ((wanted  || basemisc ) && deleted) && pkg.installed.accessible ();
-- 
1.8.1.2


[-- Attachment #2: Type: text/plain, Size: 190 bytes --]



Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [PATCH 1/4] setup.exe

[Achim Gratz]

Christopher Faylor writes:
> Regarding the autorebase changes, I am not against the idea of implementing
> a general purpose mechanism to have setup.exe "do something" when it notices
> certain file patterns being added or deleted.  This would move the current
> 'autodep' functionality from the "upset" program which creates setup.ini into
> setup.exe itself.

I've had a brief look, I would not want to add another (regex) library
to the sources, but if std::regex can be used (I haven't checked which
version of gcc first had it), this should not be too difficult to
implement.

Another question is how to call this, I'd rather use "autorun" rather
than "autodep" (but I haven't really made up my mind).  The semantics
would roughly be that the package plus its dependencies must be
installed and then whenever the autorun regex matches, a script that has
been installed with the package is run.  Those scripts should not be in
/etc/postinstall and for general use it would probably be useful to be
able to chose if it gets run before or after the postinstall scripts.



Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: [PATCH 1/4] setup.exe

[Christopher Faylor]

On Wed, Feb 13, 2013 at 08:10:25PM +0100, Achim Gratz wrote:
>Christopher Faylor writes:
>> Regarding the autorebase changes, I am not against the idea of implementing
>> a general purpose mechanism to have setup.exe "do something" when it notices
>> certain file patterns being added or deleted.  This would move the current
>> 'autodep' functionality from the "upset" program which creates setup.ini into
>> setup.exe itself.
>
>I've had a brief look, I would not want to add another (regex) library
>to the sources, but if std::regex can be used (I haven't checked which
>version of gcc first had it), this should not be too difficult to
>implement.
>
>Another question is how to call this, I'd rather use "autorun" rather
>than "autodep" (but I haven't really made up my mind).  The semantics
>would roughly be that the package plus its dependencies must be
>installed and then whenever the autorun regex matches, a script that has
>been installed with the package is run.  Those scripts should not be in
>/etc/postinstall and for general use it would probably be useful to be
>able to chose if it gets run before or after the postinstall scripts.

The autodep/autorun wasn't an either/or.  We need the autodep and, while
you may need the ability to unconditionally run programs whenever
setup.exe is executed, the Cygwin project don't.  However, you could get
the behavior you want with autodep with something like:

autodep: .

cgf

Re: [PATCH 1/4] setup.exe

[Achim Gratz]

Christopher Faylor writes:
> The autodep/autorun wasn't an either/or.  We need the autodep and, while

As I understand (and this might be wrong as I don't know what upset is
doing), autodep adds that package to the requires of another package if
the dependency regex matches for any files installed by that other
package, unless the regex in noautodep also matches.  Additionally, the
version of the autodep'ed package is incremented if incver_ifdep is set
to "yes", which has the effect that the package will be re-installed
(setup.exe sees it has a newer version) and hence the postinstall script
will run again.

When doing this from within setup.exe, without manipulating the version
in setup.ini, there are again two things to take care of: the autodep
part, that is adding the autodep'ed package to the requires list so that
the package will be installed; then the "consider this package updated"
part that will lead to re-installation.

However, the sole purpose of making the package appear updated is (today
at least) to run the postinstall script again.  My suggestion was that
the part of running a script whenever another package installs certain
file patterns should be a separate (additional) functionality.  You
couldn't do this with upset, but you can when it moves into setup.exe.
I was not suggesting to drop the autodep functionality, although
currently it would have no real use (cygwin doesn't need it since it is
already special cased in setup.exe via the "base" category).

> you may need the ability to unconditionally run programs whenever
> setup.exe is executed, the Cygwin project don't.  However, you could get
> the behavior you want with autodep with something like:
>
> autodep: .

Please forget the red herring of this always-run script for a moment, it
really is another discussion.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [PATCH 1/4] setup.exe

[Christopher Faylor]

On Wed, Feb 13, 2013 at 10:24:50PM +0100, Achim Gratz wrote:
>Christopher Faylor writes:
>> The autodep/autorun wasn't an either/or.  We need the autodep and, while
>
>As I understand (and this might be wrong as I don't know what upset is
>doing), autodep adds that package to the requires of another package if
>the dependency regex matches for any files installed by that other
>package, unless the regex in noautodep also matches.  Additionally, the
>version of the autodep'ed package is incremented if incver_ifdep is set
>to "yes", which has the effect that the package will be re-installed
>(setup.exe sees it has a newer version) and hence the postinstall script
>will run again.

There is no reason to worry about how upset manages things currently.
The point of adding this functionality is to not have upset do anything.

>When doing this from within setup.exe, without manipulating the version
>in setup.ini, there are again two things to take care of: the autodep
>part, that is adding the autodep'ed package to the requires list so that
>the package will be installed; then the "consider this package updated"
>part that will lead to re-installation.

There is no need for setup.exe to add anything to a requires list due
to autodep.

>However, the sole purpose of making the package appear updated is (today
>at least) to run the postinstall script again.  My suggestion was that
>the part of running a script whenever another package installs certain
>file patterns should be a separate (additional) functionality.  You
>couldn't do this with upset, but you can when it moves into setup.exe.
>I was not suggesting to drop the autodep functionality, although
>currently it would have no real use (cygwin doesn't need it since it is
>already special cased in setup.exe via the "base" category).

We have at least two specific needs for for the ability to run a script
when certain files are installed: .info files and dlls.

Just to reiterate what I'm asking for (which I think is converging on
what you are saying):

Whenever setup sees a file which matches pattern X being installed, flag
that we have to run program Y before running postinstall scripts.  Then
when everything has been installed run all program Y's.  It might be
nice if the programs received filename arguments indicating all of the
recently installed files but that is not necessary for the current
needs.  And, it might be nice, as you mention, to be able to run the
script after the postinstalls.

My proposal is that the rebase package will have a setup.hint which
contains (at the simplest)

autodep: .*/[^/]*\.(?:dll|so|oct)$
autorun: /usr/bin/rebaseall

I wanted two lines for the ease in parsing.

However, I guess that could also be:

autorun: /usr/bin/rebaseall .*/[^/]*\.(?:dll|so|oct)$

Maybe that's what you're saying.  I was thinking that two lines would
make parsing easier but I suppose that you could also just assume that
the first "program to run" pattern doesn't contain any spaces so the
parsing would be the equivalent of ([\S+])\s*(.*)$ .  An empty regex
string could be allowed to match everything.

cgf

Re: [PATCH 1/4] setup.exe

[Achim Gratz]

Christopher Faylor writes:
> There is no need for setup.exe to add anything to a requires list due
> to autodep.

It needs to ensure that the package providing the script is installed,
which means installation, updating or doing nothing; depending on what
state the installation is in.  In terms of setup.ini this is "adding to
the requires list", in terms of setup.exe this list has been absorbed
into the package_db state at that point already.

> We have at least two specific needs for for the ability to run a script
> when certain files are installed: .info files and dlls.
>
> Just to reiterate what I'm asking for (which I think is converging on
> what you are saying):

I think so, too.

> Whenever setup sees a file which matches pattern X being installed, flag
> that we have to run program Y before running postinstall scripts.  Then
> when everything has been installed run all program Y's.  It might be
> nice if the programs received filename arguments indicating all of the
> recently installed files but that is not necessary for the current
> needs.  And, it might be nice, as you mention, to be able to run the
> script after the postinstalls.
>
> My proposal is that the rebase package will have a setup.hint which
> contains (at the simplest)
>
> autodep: .*/[^/]*\.(?:dll|so|oct)$
> autorun: /usr/bin/rebaseall
>
> I wanted two lines for the ease in parsing.
>
> However, I guess that could also be:
>
> autorun: /usr/bin/rebaseall .*/[^/]*\.(?:dll|so|oct)$
>
> Maybe that's what you're saying.  I was thinking that two lines would
> make parsing easier but I suppose that you could also just assume that
> the first "program to run" pattern doesn't contain any spaces so the
> parsing would be the equivalent of ([\S+])\s*(.*)$ .  An empty regex
> string could be allowed to match everything.


Yes, splitting the parsing over two lines should be slightly easier, but
probably not so much as to lose sleep over.

The bigger problem is that the std::regex library has not been
implemented for gcc yet, so we'd either have to use an extra library or
dispense with full regex parsing.  It seems not very likely that even
the 4.8 gcc series changes this, so suggestions on what to do instead
are welcome (using boost looks possible, but is said to be buggy).

The two current applications of autodep however could be implemented by
either matching path components or the extension which could be
implemented with the existing string functions and perhaps using a
slightly different syntax in setup.ini:

autoext: dll so oct
autopath: /usr/share/info /usr/info

I don't know if you had other applications for autodep in mind where
these two wouldn't be sufficient and a real regex parser would be
needed, though.



Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [PATCH 1/4] setup.exe

[Christopher Faylor]

On Thu, Feb 14, 2013 at 09:30:59PM +0100, Achim Gratz wrote:
>Christopher Faylor writes:
>> There is no need for setup.exe to add anything to a requires list due
>> to autodep.
>
>It needs to ensure that the package providing the script is installed,
>which means installation, updating or doing nothing; depending on what
>state the installation is in.  In terms of setup.ini this is "adding to
>the requires list", in terms of setup.exe this list has been absorbed
>into the package_db state at that point already.

Actually, it needs to detect when a DLL is being installed.  AFAIK,
that's it.  I don't think detecting when a package is being deleted
really matters.

>> autorun: /usr/bin/rebaseall .*/[^/]*\.(?:dll|so|oct)$
>>
>> Maybe that's what you're saying.  I was thinking that two lines would
>> make parsing easier but I suppose that you could also just assume that
>> the first "program to run" pattern doesn't contain any spaces so the
>> parsing would be the equivalent of ([\S+])\s*(.*)$ .  An empty regex
>> string could be allowed to match everything.
>
>
>Yes, splitting the parsing over two lines should be slightly easier, but
>probably not so much as to lose sleep over.
>
>The bigger problem is that the std::regex library has not been
>implemented for gcc yet, so we'd either have to use an extra library or
>dispense with full regex parsing.  It seems not very likely that even
>the 4.8 gcc series changes this, so suggestions on what to do instead
>are welcome (using boost looks possible, but is said to be buggy).

You can certainly parse regex's in c++ without relying on std::regex.

>The two current applications of autodep however could be implemented by
>either matching path components or the extension which could be
>implemented with the existing string functions and perhaps using a
>slightly different syntax in setup.ini:
>
>autoext: dll so oct
>autopath: /usr/share/info /usr/info
>
>I don't know if you had other applications for autodep in mind where
>these two wouldn't be sufficient and a real regex parser would be
>needed, though.

Huh?  That would not help with the very case that we're talking about -
dlls.

cgf

Re: [PATCH 1/4] setup.exe

[Achim Gratz]

Christopher Faylor writes:
> Actually, it needs to detect when a DLL is being installed.  AFAIK,
> that's it.

The first part (detecting when a file of a certain type or in a certain
location gets installed) was never in question.  But you also need to
ensure that the package that contains the autorun script is actually
installed at the most recent version.  At least that is the behaviour
that autodep+upset does produce and my assumption is that any
integration into setup.exe needs to keep it that way.

>  I don't think detecting when a package is being deleted
> really matters.

I don't think I talked about deinstallation.  In any case it wouldn't
matter for autodep; at least not if it continues to work like it already
does.

> You can certainly parse regex's in c++ without relying on std::regex.

Sure, but do you have a suggestion on exactly how?  Whatever library
gets chosen it would need to be linked into the executable.

> Huh?  That would not help with the very case that we're talking about -
> dlls.

Autodep is used three times (if I counted correctly) in the current
distribution.  One is for cygwin, which we don't really need it since it
is already taken care of in a different way in setup.exe.  The other two
are for _autorebase and _update_info.  Of these, one regex matches on
files having certain extensions and the other matches on files installed
into directories having certain path prefixes.  So having a way to
specify these two cases without using a regex would satisfy the current
uses of autodep — or am I missing something?


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [PATCH 1/4] setup.exe

[Christopher Faylor]

On Fri, Feb 15, 2013 at 08:51:46PM +0100, Achim Gratz wrote:
>Christopher Faylor writes:
>> Actually, it needs to detect when a DLL is being installed.  AFAIK,
>> that's it.
>
>The first part (detecting when a file of a certain type or in a certain
>location gets installed) was never in question.  But you also need to
>ensure that the package that contains the autorun script is actually
>installed at the most recent version.  At least that is the behaviour
>that autodep+upset does produce and my assumption is that any
>integration into setup.exe needs to keep it that way.

Are you trying to say that the package containing an autorun script
must run its postinstall.sh before anything else?  I don't really see
the need for a postinstall.sh for any of the current use cases but,
ok, yes, that would be a wrinkle for this scenario.  However, I'm
comfortable with imposing a "autorun packages should not require
a postinstall.sh script to operate" rule.[

>>I don't think detecting when a package is being deleted really matters.
>
>I don't think I talked about deinstallation.  In any case it wouldn't
>for autodep; at least not if it continues to work like it already does.

We're talking about something that *I* suggested.  I'm laying out the
ground rules.  It actually *does* matter that something should be run
when a .info file is deleted but it's not a crucial thing to implement
right now.

>> You can certainly parse regex's in c++ without relying on std::regex.
>
>Sure, but do you have a suggestion on exactly how?  Whatever library
>gets chosen it would need to be linked into the executable.

It doesn't have to be a library.  Cygwin's regexec.c would probably
suffice.

>> Huh?  That would not help with the very case that we're talking about -
>> dlls.
>
>Autodep is used three times (if I counted correctly) in the current
>distribution.  One is for cygwin, which we don't really need it since it
>is already taken care of in a different way in setup.exe.

I removed the autorun from cygwin's setup a couple of days ago since it
has outlived its usefulness.  It actually did do something that wasn't
covered by setup.exe but that no longer matters.

>The other two are for _autorebase and _update_info.  Of these, one
>regex matches on files having certain extensions and the other matches
>on files installed into directories having certain path prefixes.  So
>satisfy the current uses of autodep ???  or am I missing something?

I actually posted the current autodep line for the _autorebase package.
It did not (and should not) rely on file prefixes.

cgf

Re: [PATCH 1/4] setup.exe

[Achim Gratz]

Christopher Faylor writes:
> Are you trying to say that the package containing an autorun script
> must run its postinstall.sh before anything else?

No, only that it must be installed so that the script to autorun is
accessible when it triggers.  This could of course be handled by
stipulating that all packages defining autoruns must be in the "Base"
category.  If that is not desired, then obviously there's the
possibility that the package isn't installed (or outdated) when the
autorun rule it defines triggers and setup.exe should IMHO install or
update it in this case.

>  I don't really see the need for a postinstall.sh for any of the
> current use cases but, ok, yes, that would be a wrinkle for this
> scenario.  However, I'm comfortable with imposing a "autorun packages
> should not require a postinstall.sh script to operate" rule.[

Currently a side effect of how things are implemented is indeed that the
autorun script is actually a postinstall script and the package name
determines when it is run.  But no, I wouldn't want to keep it that way.
The package providing the autorun script should not need or have a
postinstall script, precisely because we would otherwise need to
carefully orchestrate the order in which to run the postinstall
scripts.

> We're talking about something that *I* suggested.  I'm laying out the
> ground rules.  It actually *does* matter that something should be run
> when a .info file is deleted but it's not a crucial thing to implement
> right now.

OK, I didn't understand where that was coming from, since the first part
of your comment seemed to be directed at autodep and it currently
doesn't care about deletions.

Yes, when things get removed that might be another reason to autorun
something (to free up address space in the rebase DB for instance), but
it wouldn't necessarily need to run the same script.  So we would either
need to have a separate keyword or the autorun script should be invoked
with an argument so that it can distinguish between these two cases.

> It doesn't have to be a library.  Cygwin's regexec.c would probably
> suffice.

I'll have a look.

> I actually posted the current autodep line for the _autorebase package.
> It did not (and should not) rely on file prefixes.

No, of course not; it matches one of three file extensions, something
that is trivial to implement without a regex parser.  I still think that
it would be safer if we didn't try to parse arbitrary regex, and
certainly more performant if there are many such rules.  But if there's
a genuine need for full regex parsing…


Regarding another comment you've made earlier: for triggering the
autorun script we'd only need to check until the first match is found
and could then skip checking for that particular trigger later on.  If
on the other hand you would want to be able to get a list of file names
that triggered the rule, we'd need to always check them all and also
collect them into a list so that for instance they could become an
argument to the script.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [PATCH 1/4] setup.exe

[Corinna Vinschen]

On Feb 16 21:10, Achim Gratz wrote:
> Christopher Faylor writes:
> > It doesn't have to be a library.  Cygwin's regexec.c would probably
> > suffice.
> 
> I'll have a look.

What about just using std::regex?  Setup is using the stdc++ library
anyway, so you would get std::regex for free.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

Re: [PATCH 1/4] setup.exe

[Christopher Faylor]

On Sat, Feb 16, 2013 at 10:16:15PM +0100, Corinna Vinschen wrote:
>On Feb 16 21:10, Achim Gratz wrote:
>> Christopher Faylor writes:
>> > It doesn't have to be a library.  Cygwin's regexec.c would probably
>> > suffice.
>> 
>> I'll have a look.
>
>What about just using std::regex?  Setup is using the stdc++ library
>anyway, so you would get std::regex for free.

It's already been suggested and I thought it wasn't available:

http://cygwin.com/ml/cygwin-apps/2013-02/msg00146.html

Re: [PATCH 1/4] setup.exe

[Corinna Vinschen]

On Feb 17 12:15, Christopher Faylor wrote:
> On Sat, Feb 16, 2013 at 10:16:15PM +0100, Corinna Vinschen wrote:
> >On Feb 16 21:10, Achim Gratz wrote:
> >> Christopher Faylor writes:
> >> > It doesn't have to be a library.  Cygwin's regexec.c would probably
> >> > suffice.
> >> 
> >> I'll have a look.
> >
> >What about just using std::regex?  Setup is using the stdc++ library
> >anyway, so you would get std::regex for free.
> 
> It's already been suggested and I thought it wasn't available:
> 
> http://cygwin.com/ml/cygwin-apps/2013-02/msg00146.html

Sorry, I didn't follow the entire thread.  I'm not big in libstdc++
stuff, but I thought std::regex is part of it by default.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

Re: [PATCH 1/4] setup.exe

[Christopher Faylor]

On Sun, Feb 17, 2013 at 06:43:00PM +0100, Corinna Vinschen wrote:
>On Feb 17 12:15, Christopher Faylor wrote:
>> On Sat, Feb 16, 2013 at 10:16:15PM +0100, Corinna Vinschen wrote:
>> >On Feb 16 21:10, Achim Gratz wrote:
>> >> Christopher Faylor writes:
>> >> > It doesn't have to be a library.  Cygwin's regexec.c would probably
>> >> > suffice.
>> >> 
>> >> I'll have a look.
>> >
>> >What about just using std::regex?  Setup is using the stdc++ library
>> >anyway, so you would get std::regex for free.
>> 
>> It's already been suggested and I thought it wasn't available:
>> 
>> http://cygwin.com/ml/cygwin-apps/2013-02/msg00146.html
>
>Sorry, I didn't follow the entire thread.  I'm not big in libstdc++
>stuff, but I thought std::regex is part of it by default.

Me too.  Can someone definitively confirm or deny this?

cgf

Re: [PATCH 1/4] setup.exe

[Achim Gratz]

Christopher Faylor writes:
>>Sorry, I didn't follow the entire thread.  I'm not big in libstdc++
>>stuff, but I thought std::regex is part of it by default.
>
> Me too.  Can someone definitively confirm or deny this?

The current toolchain based on cygwin definitely doesn't support it and
the status of the upstream implementation can be checked here (scroll
down to #28):

http://gcc.gnu.org/onlinedocs/libstdc++/manual/status.html#status.iso.200x

The compiler produces the correct symbols (and has been for some time),
but the library doesn't implement those functions and you cannot link.
There is another status page for g++ that say much the same, sorry I
can't find that link right now.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [PATCH 1/4] setup.exe

[Corinna Vinschen]

On Feb 17 19:44, Achim Gratz wrote:
> Christopher Faylor writes:
> >>Sorry, I didn't follow the entire thread.  I'm not big in libstdc++
> >>stuff, but I thought std::regex is part of it by default.
> >
> > Me too.  Can someone definitively confirm or deny this?
> 
> The current toolchain based on cygwin definitely doesn't support it and
> the status of the upstream implementation can be checked here (scroll
> down to #28):
> 
> http://gcc.gnu.org/onlinedocs/libstdc++/manual/status.html#status.iso.200x
> 
> The compiler produces the correct symbols (and has been for some time),
> but the library doesn't implement those functions and you cannot link.
> There is another status page for g++ that say much the same, sorry I
> can't find that link right now.

That's really a pity.  If we really need regex'es, the most simple code
could be fetched from NetBSD.  It has the advantage to be single-byte
only.  Paths in the distro shouldn't use multi-byte strings anyway.

OTOH, it might be sufficent for quite some time, if the autodep strings
are only evaluated as simple filename suffixes:

    autodep: .oct .so .dll

    autodep: .info .info.gz

Wouldn't that be sufficent and very easy to implement?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

Re: [PATCH 1/4] setup.exe

[Christopher Faylor]

On Sun, Feb 17, 2013 at 08:20:48PM +0100, Corinna Vinschen wrote:
>On Feb 17 19:44, Achim Gratz wrote:
>> Christopher Faylor writes:
>> >>Sorry, I didn't follow the entire thread.  I'm not big in libstdc++
>> >>stuff, but I thought std::regex is part of it by default.
>> >
>> > Me too.  Can someone definitively confirm or deny this?
>> 
>> The current toolchain based on cygwin definitely doesn't support it and
>> the status of the upstream implementation can be checked here (scroll
>> down to #28):
>> 
>> http://gcc.gnu.org/onlinedocs/libstdc++/manual/status.html#status.iso.200x
>> 
>> The compiler produces the correct symbols (and has been for some time),
>> but the library doesn't implement those functions and you cannot link.
>> There is another status page for g++ that say much the same, sorry I
>> can't find that link right now.
>
>That's really a pity.  If we really need regex'es, the most simple code
>could be fetched from NetBSD.  It has the advantage to be single-byte
>only.  Paths in the distro shouldn't use multi-byte strings anyway.

I just added a regex object to setup.exe.  I tested it on Windows and
it seems to work for the simple case.  Nothing is using it right now
of course.

(My suggestion of using Cygwin's didn't work well since it relied on
too many pieces from /usr/include)

>OTOH, it might be sufficent for quite some time, if the autodep strings
>are only evaluated as simple filename suffixes:
>
>    autodep: .oct .so .dll
>
>    autodep: .info .info.gz
>
>Wouldn't that be sufficent and very easy to implement?

This has also already been covered.

We only care about .info files which show up in /usr/share/info.  Would
it matter right now for the distro?  Probably not.  But, there is no
reason to set the bar low if we're adding a new feature.  Using regex
allows much more flexibility than the above and doesn't limit future
potential uses.

If using regex is too big a chore then I'll add this feature myself.

cgf

Re: [PATCH 1/4] setup.exe

[Christopher Faylor]

On Sun, Feb 17, 2013 at 04:47:01PM -0500, Christopher Faylor wrote:
>I just added a regex object to setup.exe.  I tested it on Windows and
>it seems to work for the simple case.  Nothing is using it right now
>of course.

I just did a little configury cleanup in setup to remove the warnings
about old usage.

And, I added a autodep stub.  setup.exe should now parse an autodep
line like:

autodep: string string

Since quoted strings are allowed, either of those can be the script
to run.  I think it probably makes sense for this to be, e.g.,

autodep: "usr/(?:share/)?info/.*" /usr/sbin/update-info-dir

This is the opposite of what I previously suggested.  I didn't remember
how the setup.exe parsing worked before.

The above is an example.  I will add something like that to texinfo if
the functionality shows up in setup.exe.

Achim, if you are actively working on this please let me know.
Otherwise, I'll probably just do this myself in the next several days.

cgf

Re: [PATCH 1/4] setup.exe

[Achim Gratz]

Christopher Faylor writes:
> And, I added a autodep stub.  setup.exe should now parse an autodep
> line like:
>
> autodep: string string
>
> Since quoted strings are allowed, either of those can be the script
> to run.  I think it probably makes sense for this to be, e.g.,
>
> autodep: "usr/(?:share/)?info/.*" /usr/sbin/update-info-dir
>
> This is the opposite of what I previously suggested.  I didn't remember
> how the setup.exe parsing worked before.

Great, thanks.  May I suggest adding a third string parameter to select
running the autodep script before or after the postinstall scripts?

> The above is an example.  I will add something like that to texinfo if
> the functionality shows up in setup.exe.
>
> Achim, if you are actively working on this please let me know.
> Otherwise, I'll probably just do this myself in the next several days.

I haven't done anything serious yet since it seemed that the discussion
on what exactly this feature should do in setup.exe wasn't quite
finished.  I did find where to plug the matcher into, but haven't yet
made up my mind how and where to collect the information from both the
autodep stanza in setup.ini and then from running the matcher(s) into.
In any case I don't think I'll have time to do anything before the
weekend, so if you implement something earlier this week that is OK with
me.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds