setup 2.885 release candidate - please test

setup 2.885 release candidate - please test

[Jon Turney]

A new setup release candidate is available at:

   https://cygwin.com/setup/setup-2.885.x86.exe    (32 bit version)
   https://cygwin.com/setup/setup-2.885.x86_64.exe (64 bit version)

Since this contains many internal changes, I think this could use some 
wider testing before being deployed. Please test and report any problems 
here.

This is not the place for setup feature requests.

Changes compared to 2.884:

User visible changes:
- 'Current' is replaced by 'Best' (which is slightly different in ways 
it's difficult to summarize briefly) and 'Sync' (which exposes the 
--force-current option through the UI).  These are modified by a 'Test' 
checkbox, which allows test packages to be used.
- The "prereq" page showing dependencies which will be added is replaced 
by "problems" page showing problems found by the dependency solver, with 
default solutions.
- A "confirm" page is added showing all the changes which will be made.

Internal changes:
- Uses the libsolv dependency solver, rather than a home-made one.
- Add support for 'depends2: package (relation version) [...]', in a 
version section in setup.ini
- Add support for 'obsoletes:' in setup.ini, likewise
- Add support for 'replace-versions:' in a package section setup.ini, to 
indicate problematic versions.

Other:
- Query the user for action to take if a corrupt local file is found
- Validate package hash after download
- Any MessageBox shown during setup.ini parsing is now modal

A big 'thank you' to Ken Brown for all his work on this.

Re: setup 2.885 release candidate - please test

[Achim Gratz]

Jon Turney writes:
> Since this contains many internal changes, I think this could use some
> wider testing before being deployed. Please test and report any
> problems here.

I've built these myself, but I don't think that changes anything below.

> User visible changes:
> - 'Current' is replaced by 'Best' (which is slightly different in ways
> it's difficult to summarize briefly) and 'Sync' (which exposes the
> --force-current option through the UI).  These are modified by a
> 'Test' checkbox, which allows test packages to be used.

I am always running setup with options to install a selected category w/
orphan removal and removal of non-selected packages.  The selected
category comprises _all_ packages that are supposed to be installed, so
no dependencies need to be found.

In order to see what's going on I also selected chooser mode (the normal
install just does whatever it's told to do).  That still works
apparently, but whenever I click anywhere to change the mode from "Best"
to something else and then back the transaction list gets emptied.  As I
said, I normally don't do this and clicking on those buttons serves no
useful purpose for me in that situation, but I think the result is still
wrong.  I think that maybe the command line parameters are forgotten
when doing this.

> - The "prereq" page showing dependencies which will be added is
> replaced by "problems" page showing problems found by the dependency
> solver, with default solutions.
> - A "confirm" page is added showing all the changes which will be made.

I've not actually commenced the installation yet due to other things I
want to fix first, so I can't say whether these two pages work.  I guess
I should not see them with my normal install script.  The interesting
part would be if they are skipped when non-interactive mode is given and
there was something to add due to dependencies?

> - Add support for 'depends2: package (relation version) [...]', in a
> version section in setup.ini

Those lines don't seem to get generated for all packages yet.  I
currently merge with requires: to produce a working setup.ini re-write
and will switch to using requires: when I find no depends2:.  Can I
assume that all versions have a depends2: line when I find one for
[curr]?

I don't like the syntax with the commas, could we just drop the space
between the package name and the paren for the version expression and
keep the version-relations separated by plain whitespace?

> - Add support for 'obsoletes:' in setup.ini, likewise

These don't appear to be produced by calm yet.  Also, it would be useful
if the obsoleted package showed the replacement package more explicitly,
so maybe "obsoleted-by:" instead of requires:/depends2:

> - Add support for 'replace-versions:' in a package section setup.ini,
> to indicate problematic versions.

Any examples for this yet?

As you surely know, all the new syntax is not yet described in
https://sourceware.org/cygwin-apps/setup.ini.html


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: setup 2.885 release candidate - please test

[Achim Gratz]

Achim Gratz writes:
>> - Add support for 'depends2: package (relation version) [...]', in a
>> version section in setup.ini
>
> Those lines don't seem to get generated for all packages yet.  I
> currently merge with requires: to produce a working setup.ini re-write
> and will switch to using requires: when I find no depends2:.  Can I
> assume that all versions have a depends2: line when I find one for
> [curr]?

…and of course these are a result of the latest officially available
calm version not having those changes, so my local packages are still
using requires: lines.  Any chance you could relese a new calm version?

Other than that I think I've fixed up my setup rewriter so it can deal
with the new format correctly now and I've even managed to implement
explicit version pinning (which I had on TODO for almost three years,
but since before I've only had test, curr and prev for testing I've put
it up each time I looked at it).

Another thing I noticed today is that when packages get upgraded the
transaction list that gets printed to the console seems to always show
the removal of the old package _after_ the installation of the new
version.  I guess that will not happen in this order, but it looks
positiviely wierd on screen.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: setup 2.885 release candidate - please test

[Jon Turney]

On 30/01/2018 18:49, Achim Gratz wrote:
> Achim Gratz writes:
>>> - Add support for 'depends2: package (relation version) [...]', in a
>>> version section in setup.ini
>>
>> Those lines don't seem to get generated for all packages yet.  I
>> currently merge with requires: to produce a working setup.ini re-write
>> and will switch to using requires: when I find no depends2:.  Can I
>> assume that all versions have a depends2: line when I find one for
>> [curr]?
> 
> …and of course these are a result of the latest officially available
> calm version not having those changes, so my local packages are still
> using requires: lines.  Any chance you could relese a new calm version?

Sure. I uploaded calm-20180130-1.

I really need to automate that as part of the deploy :)

> Other than that I think I've fixed up my setup rewriter so it can deal
> with the new format correctly now and I've even managed to implement
> explicit version pinning (which I had on TODO for almost three years,
> but since before I've only had test, curr and prev for testing I've put
> it up each time I looked at it).
> 
> Another thing I noticed today is that when packages get upgraded the
> transaction list that gets printed to the console seems to always show
> the removal of the old package _after_ the installation of the new
> version.  I guess that will not happen in this order, but it looks
> positively weird on screen.

Yeah, I think we are reversing the order given by the solver.  This 
should be benign, as we do all the uninstalls before installs.

I'll fix that :)

Re: setup 2.885 release candidate - please test

[Achim Gratz]

Jon Turney writes:
> Sure. I uploaded calm-20180130-1.

Thanks.  Maybe I will actually do another update rollout this week,
then.

> I really need to automate that as part of the deploy :)

One step after the other.  :-)

> Yeah, I think we are reversing the order given by the solver.  This
> should be benign, as we do all the uninstalls before installs.

OK, so let's see what blows up when I hit the red button tomorrow.
It'll be my own 32bit installation first, followed by the 64bit one and
then if I still live I'll update the Cygwin server.  If that works I'll
roll out for my users and duck for cover.

> I'll fix that :)

Thanks.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: setup 2.885 release candidate - please test

[Jon Turney]

On 30/01/2018 20:18, Jon Turney wrote:
> On 30/01/2018 18:49, Achim Gratz wrote:
>> Another thing I noticed today is that when packages get upgraded the
>> transaction list that gets printed to the console seems to always show
>> the removal of the old package _after_ the installation of the new
>> version.  I guess that will not happen in this order, but it looks
>> positively weird on screen.
> 
> Yeah, I think we are reversing the order given by the solver.  This 
> should be benign, as we do all the uninstalls before installs.
> 
> I'll fix that :)

Hmm.. actually, I don't know what's going here.  We are preserving the 
order produced by the solver, and it produces a more sensible looking 
ordering for some more complex transactions, so I don't know why 
upgrades look backwards...

Re: setup 2.885 release candidate - please test

[Jon Turney]

On 29/01/2018 19:19, Achim Gratz wrote:
> Jon Turney writes:
>> Since this contains many internal changes, I think this could use some
>> wider testing before being deployed. Please test and report any
>> problems here.
> 
> I've built these myself, but I don't think that changes anything below.

Thanks for taking a look.

>> User visible changes:
>> - 'Current' is replaced by 'Best' (which is slightly different in ways
>> it's difficult to summarize briefly) and 'Sync' (which exposes the
>> --force-current option through the UI).  These are modified by a
>> 'Test' checkbox, which allows test packages to be used.
> 
> I am always running setup with options to install a selected category w/
> orphan removal and removal of non-selected packages.  The selected
> category comprises _all_ packages that are supposed to be installed, so
> no dependencies need to be found.
> 
> In order to see what's going on I also selected chooser mode (the normal
> install just does whatever it's told to do).  That still works
> apparently, but whenever I click anywhere to change the mode from "Best"
> to something else and then back the transaction list gets emptied.  As I
> said, I normally don't do this and clicking on those buttons serves no
> useful purpose for me in that situation, but I think the result is still
> wrong.  I think that maybe the command line parameters are forgotten
> when doing this.

I think this is the behaviour in previous versions, as well.

When command line and UI settings are in conflict, I don't think it's 
unreasonable that the last one changed wins.

>> - The "prereq" page showing dependencies which will be added is
>> replaced by "problems" page showing problems found by the dependency
>> solver, with default solutions.
>> - A "confirm" page is added showing all the changes which will be made.
> 
> I've not actually commenced the installation yet due to other things I
> want to fix first, so I can't say whether these two pages work.  I guess
> I should not see them with my normal install script.  The interesting
> part would be if they are skipped when non-interactive mode is given and
> there was something to add due to dependencies?

These should be added (and default solutions applied where the solver 
identified problems) in non-interactive mode.

>> - Add support for 'depends2: package (relation version) [...]', in a
>> version section in setup.ini
> 
> Those lines don't seem to get generated for all packages yet.  I
> currently merge with requires: to produce a working setup.ini re-write
> and will switch to using requires: when I find no depends2:.  Can I
> assume that all versions have a depends2: line when I find one for
> [curr]?

Yes, with the proviso that empty depends2: lines are currently 
suppressed (this might be a mistake)

> I don't like the syntax with the commas, could we just drop the space
> between the package name and the paren for the version expression and
> keep the version-relations separated by plain whitespace?
> 
>> - Add support for 'obsoletes:' in setup.ini, likewise
> 
> These don't appear to be produced by calm yet.  Also, it would be useful

calm can write these lines, if obsoletes: is present in the .hint, but 
cygport is not yet capable of generating those.

> if the obsoleted package showed the replacement package more explicitly,
> so maybe "obsoleted-by:" instead of requires:/depends2:
> 
>> - Add support for 'replace-versions:' in a package section setup.ini,
>> to indicate problematic versions.
> 
> Any examples for this yet?

A separate email on this is in the works.

> As you surely know, all the new syntax is not yet described in
> https://sourceware.org/cygwin-apps/setup.ini.html

Thanks for the reminder,  I pushed the update I wrote for this.

Re: setup 2.885 release candidate - please test

[Jon Turney]

On 30/01/2018 20:18, Jon Turney wrote:
> On 29/01/2018 19:19, Achim Gratz wrote:
>> Jon Turney writes:
[...]>>> - The "prereq" page showing dependencies which will be added is
>>> replaced by "problems" page showing problems found by the dependency
>>> solver, with default solutions.
>>> - A "confirm" page is added showing all the changes which will be made.
>>
>> I've not actually commenced the installation yet due to other things I
>> want to fix first, so I can't say whether these two pages work.  I guess
>> I should not see them with my normal install script.  The interesting
>> part would be if they are skipped when non-interactive mode is given and
>> there was something to add due to dependencies?
> 
> These should be added (and default solutions applied where the solver 
> identified problems) in non-interactive mode.

It seems I missed the part to add default problem solutions in 
non-interactive mode.

>>> - Add support for 'depends2: package (relation version) [...]', in a
>>> version section in setup.ini
>>
>> Those lines don't seem to get generated for all packages yet.  I
>> currently merge with requires: to produce a working setup.ini re-write
>> and will switch to using requires: when I find no depends2:.  Can I
>> assume that all versions have a depends2: line when I find one for
>> [curr]?
> 
> Yes, with the proviso that empty depends2: lines are currently 
> suppressed (this might be a mistake)

Yeah, suppressing these is definitely a mistake, as we need to indicate 
(in a small number of cases) a package version which has no depends: 
when other versions do have them.

Unfortunately, fixing that reveals that the setup parser doesn't 
currently permit an empty requires: or depends: list (which I guess 
explains why they have been suppressed historically)