setup's response to a "corrupt local copy"

setup's response to a "corrupt local copy"

[Ken Brown]

This is a followup to the discussion started here:

   https://cygwin.com/ml/cygwin/2017-12/msg00088.html

When setup is preparing to download files and it finds a corrupt copy in 
the local cache, it issues a fatal error message telling the user to 
remove the corrupt file and retry.  Steven said that setup should 
silently delete the corrupt file, while I argued in favor of the current 
behavior, on the grounds that setup shouldn't be deleting user files if 
it doesn't know where they came from.

There is a middle ground: setup could query the user.  Additionally, as 
suggested by cyg Simple, there could be an option that directs setup to 
silently remove corrupt files.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: setup's response to a "corrupt local copy"

[Hans-Bernhard Bröker]

Am 13.12.2017 um 14:28 schrieb Ken Brown:

> When setup is preparing to download files and it finds a corrupt copy in 
> the local cache, it issues a fatal error message telling the user to 
> remove the corrupt file and retry.  Steven said that setup should 
> silently delete the corrupt file, while I argued in favor of the current 
> behavior, on the grounds that setup shouldn't be deleting user files if 
> it doesn't know where they came from.

I agree with the latter approach, primarily because those files must 
have still been OK the last time setup was run successfully, or not have 
been there at all.  Otherwise this run of setup wouldn't be the one 
where this suddenly popped up, would it.  So the real question is: how 
could that status have changed from then to now?

First off, I hope we agree that files very rarely change their content 
just by lying around somewhere, particularly in a local cache folder 
structure like this, which will usually not be touched by anything other 
than setup itself.  So the odds should really be negligible that the 
files just corrupted themselves.  If those are sizable odds on a given 
machine, the ease of further cygwin installs done onto it are the least 
of your worries.

That leaves two primary possibilities how this change of state might 
have occurred:

1) File contents changed on purpose, probably by manual overwrite with 
locally built archives.

2) setup's idea of what a correct file is changed from one run of 
setup.exe to the next, mostly likely by loading a newer setup.ini

> There is a middle ground: setup could query the user.  Additionally, as 
> suggested by cyg Simple, there could be an option that directs setup to 
> silently remove corrupt files.

I agree: this is essentially the same situation as a merge conflict in 
CVS/SVN/git: upstream (setup.ini) and local working copy (archive) are 
now in conflict, and you really _have_to_ ask the user about what to do 
about it.  The query should contain relevant details (CRC expected vs. 
observed, timestamps, whatever) to allow the user to make an informed 
decision, and it might better offer an extra wide selection of answers, 
such as Back to selection, Delete this, Delete all, Keep this, Keep all, 
and possibly even "Back up local file for later inspection".

A command line switch really won't do, because its setting would be 
decided either way too early or slightly too late for that decision to 
have any reliable relation to what the user needs to happen in the case 
at hand.  It would unavoidably trigger irate user feedback like

	"This switch solved some arcane problem I don't even remember any more, 
years ago, so I hardcoded it in the start script; and _now_ you tell me 
that's what killed my local, irreplacable cygwin packages?"

one way, or

	"If just some junk file needed to be deleted, why on earth does that 
mean I have to step through that entire, tedious setup and package 
selection _again_!!??!@"

the other.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: setup's response to a "corrupt local copy"

[Andrey Repin]

Greetings, Ken Brown!

> This is a followup to the discussion started here:

>    https://cygwin.com/ml/cygwin/2017-12/msg00088.html

> When setup is preparing to download files and it finds a corrupt copy in 
> the local cache, it issues a fatal error message telling the user to 
> remove the corrupt file and retry.  Steven said that setup should 
> silently delete the corrupt file, while I argued in favor of the current 
> behavior, on the grounds that setup shouldn't be deleting user files if 
> it doesn't know where they came from.

The point being, if this is a "Download" Setup mode, the files are NOT "User"
files, but a local setup cache. And all files therein SHOULD be valid package
archives.
There's of course situations, when setup.ini on server become corrupted or
otherwise out of sync. But being rare, they should not interfere too much.

> There is a middle ground: setup could query the user.  Additionally, as 
> suggested by cyg Simple, there could be an option that directs setup to 
> silently remove corrupt files.

Make it mode dependent.
If it's a "download[ and install]" mode, cleanup and redownload.
If redownloaded file still does not match the setup.ini hash or if it's an
"install from local cache" mode, leave file alone for investigation and notify
the user.


-- 
With best regards,
Andrey Repin
Thursday, December 14, 2017 22:38:11

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: setup's response to a "corrupt local copy"

[Ken Brown]

On 12/14/2017 2:46 PM, Andrey Repin wrote:
> Greetings, Ken Brown!
> 
>> This is a followup to the discussion started here:
> 
>>     https://cygwin.com/ml/cygwin/2017-12/msg00088.html
> 
>> When setup is preparing to download files and it finds a corrupt copy in
>> the local cache, it issues a fatal error message telling the user to
>> remove the corrupt file and retry.  Steven said that setup should
>> silently delete the corrupt file, while I argued in favor of the current
>> behavior, on the grounds that setup shouldn't be deleting user files if
>> it doesn't know where they came from.
> 
> The point being, if this is a "Download" Setup mode, the files are NOT "User"
> files, but a local setup cache. And all files therein SHOULD be valid package
> archives.
> There's of course situations, when setup.ini on server become corrupted or
> otherwise out of sync. But being rare, they should not interfere too much.
> 
>> There is a middle ground: setup could query the user.  Additionally, as
>> suggested by cyg Simple, there could be an option that directs setup to
>> silently remove corrupt files.
> 
> Make it mode dependent.
> If it's a "download[ and install]" mode, cleanup and redownload.
> If redownloaded file still does not match the setup.ini hash or if it's an
> "install from local cache" mode, leave file alone for investigation and notify
> the user.

You've misunderstood the context.  The error is only shown in download or 
download/install mode.  And, as I said, it happens when setup is *preparing* to 
download files and finds a corrupt copy already present in the local cache.  In 
that context, setup has no idea where the file came from.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: setup's response to a "corrupt local copy"

[Steven Penny]

On Thu, 14 Dec 2017 14:57:14, Ken Brown wrote:
> And, as I said, it happens when setup is *preparing* to download files and
> finds a corrupt copy already present in the local cache. In that context,
> setup has no idea where the file came from.

the point is, *it doesnt matter*. it is not and shouldnt be setup.exe job to
worry about the origin of files in the Local Package Directory.

Officially this directory, and the files below, are only created by setup.exe
itself. If people are creating their own custom archives, then putting them in
the Local Package Directory, *then* expecting them to work without a modified
setup.ini, that is an error in judgment on their part.

my original point, and one that i stand by as ive seen no reasonable counter
yet, is that setup.exe should assume control of the entire Local Package
Directory and its contents. this includes removing rogue files that are in
conflict with the current setup.ini. As Hans said:

http://cygwin.com/ml/cygwin/2017-12/msg00126.html

my viewpoint would be in concert with a typical "Git" workflow:

1. make changes to working directory
2. add those changes to the index

with step 2 being the analogue to creating a custom setup.ini. setup.ini should
always have the final say on what a correct file is; so if you want a custom
archive then you better tell setup.ini about it or otherwise risk it being
nuked.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: setup's response to a "corrupt local copy"

[Vince Rice]

> On Dec 15, 2017, at 7:33 AM, Steven Penny wrote:
> 
> On Thu, 14 Dec 2017 14:57:14, Ken Brown wrote:
>> And, as I said, it happens when setup is *preparing* to download files and
>> finds a corrupt copy already present in the local cache. In that context,
>> setup has no idea where the file came from.
> 
> the point is, *it doesnt matter*. it is not and shouldnt be setup.exe job to
> worry about the origin of files in the Local Package Directory.
> 
> Officially this directory, and the files below, are only created by setup.exe
> itself. If people are creating their own custom archives, then putting them in
> the Local Package Directory, *then* expecting them to work without a modified
> setup.ini, that is an error in judgment on their part.
> 
> my original point, and one that i stand by as ive seen no reasonable counter
> yet, is that setup.exe should assume control of the entire Local Package
> Directory and its contents. this includes removing rogue files that are in
> conflict with the current setup.ini. As Hans said:
> 
> http://cygwin.com/ml/cygwin/2017-12/msg00126.html
> 
> my viewpoint would be in concert with a typical "Git" workflow:
> 
> 1. make changes to working directory
> 2. add those changes to the index
> 
> with step 2 being the analogue to creating a custom setup.ini. setup.ini should
> always have the final say on what a correct file is; so if you want a custom
> archive then you better tell setup.ini about it or otherwise risk it being
> nuked.

No, the point is it doesn't matter *to you*. As always, what you want doesn't necessarily reflect
what everyone else wants, no matter how often you repeat yourself or try to deflect others'
opinions as not being a "reasonable counter".

It's my computer. I don't want setup (or anything else) replacing files on it it doesn't know about
without at least asking whether that's what I want it to do. Setup's current behavior is exactly
what it should be, IMO. If, as has been mentioned, someone wants to offer an *option* to replace,
either with or without a question, then great. But the default should be to leave something it
doesn't know about alone.
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: setup's response to a "corrupt local copy"

[Steven Penny]

On Fri, 15 Dec 2017 09:50:44, Vince Rice wrote:
> It's my computer. I don't want setup (or anything else) replacing files on it
> it doesn't know about without at least asking whether that's what I want it to
> do. Setup's current behavior is exactly what it should be, IMO. If, as has
> been mentioned, someone wants to offer an *option* to replace, either with or
> without a question, then great. But the default should be to leave something
> it doesn't know about alone.

Thankfully we dont need your opinion on this matter, as we can simply fall back
to facts, as I will now do. Let us bring some sanity to this thread, shall we?

Exhibit A
====================
Package manager: APT
Reference distro: Lubuntu http://lubuntu.me
Cache: /var/cache/apt

Exhibit B
====================
Package manager: DNF
Reference distro: Fedora http://spins.fedoraproject.org/lxde
Cache: /var/cache/dnf
Option: keepcache=True

Exhibit C
=====================
Package manager: ZYpp
Reference distro: Gecko LXQt http://geckolinux.github.io
Cache: /var/cache/zypp
Option: --keep-packages

In all cases, these package managers remove invalid archives without warning.
Why should setup.exe be any different? I eagerly await your referenced and
reasoned rebuttal.

Cheers


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: setup's response to a "corrupt local copy"

[Frank Redeker]

Am 16.12.2017 um 05:00 schrieb Steven Penny:
> On Fri, 15 Dec 2017 09:50:44, Vince Rice wrote:
>> It's my computer. I don't want setup (or anything else) replacing
>> files on it
>> it doesn't know about without at least asking whether that's what I
>> want it to
>> do. Setup's current behavior is exactly what it should be, IMO. If, as
>> has
>> been mentioned, someone wants to offer an *option* to replace, either
>> with or
>> without a question, then great. But the default should be to leave
>> something
>> it doesn't know about alone.
> 
> Thankfully we dont need your opinion on this matter, as we can simply
> fall back
> to facts, as I will now do. Let us bring some sanity to this thread,
> shall we?

Hello Steven Penny,

This is exactly the disagreeable tone in most Linux forums that keeps
people from switching from Windows to Linux. In almost all the threads I
read you here, you get personal.

> 
> Exhibit A
> ====================
> Package manager: APT
> Reference distro: Lubuntu http://lubuntu.me
> Cache: /var/cache/apt
> 
> Exhibit B
> ====================
> Package manager: DNF
> Reference distro: Fedora http://spins.fedoraproject.org/lxde
> Cache: /var/cache/dnf
> Option: keepcache=True
> 
> Exhibit C
> =====================
> Package manager: ZYpp
> Reference distro: Gecko LXQt http://geckolinux.github.io
> Cache: /var/cache/zypp
> Option: --keep-packages
> 
> In all cases, these package managers remove invalid archives without
> warning.
> Why should setup.exe be any different? I eagerly await your referenced and
> reasoned rebuttal.
> 

If 100 people jump off the bridge, will you jump after them? Who says
that automatic deletion without warning is the only true solution? You?

I agree with Vince on that. If setup. exe detects that a packet is
invalid, it should ask what to do with it instead of deleting it silently.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: setup's response to a "corrupt local copy"

[Vince Rice]

> On Dec 16, 2017, at 11:00 AM, Steven Penny wrote:
> 
> On Fri, 15 Dec 2017 09:50:44, Vince Rice wrote:
>> It's my computer. I don't want setup (or anything else) replacing files on it
>> it doesn't know about without at least asking whether that's what I want it to
>> do. Setup's current behavior is exactly what it should be, IMO. If, as has
>> been mentioned, someone wants to offer an *option* to replace, either with or
>> without a question, then great. But the default should be to leave something
>> it doesn't know about alone.
> 
> Thankfully we dont need your opinion on this matter, as we can simply fall back
> to facts, as I will now do. Let us bring some sanity to this thread, shall we?

No, you never need any opinion that is in conflict with yours. I'm not going to
argue with you for a variety of reasons, not least because the decision is not
yours. You have shown you don't care what anyone else thinks on this or any
other issue, and believe me, the feeling is reciprocated. I offered my thoughts
for the people whose decision this really is, and I leave it with them.
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: setup's response to a "corrupt local copy"

[Steven Penny]

On Sat, 16 Dec 2017 14:50:56, Vince Rice wrote:
> No, you never need any opinion that is in conflict with yours. I'm not going
> to argue with you for a variety of reasons, not least because the decision is
> not yours. You have shown you don't care what anyone else thinks on this or
> any other issue, and believe me, the feeling is reciprocated. I offered my
> thoughts for the people whose decision this really is, and I leave it with
> them.

sure i do - here is example where i screwed up - and fixed a problem on my end:

http://cygwin.com/ml/cygwin/2017-01/msg00236.html

i do care about "the other side", when said side presents a referenced and
reasoned response. you have failed to do that. you presented a use case where
you were creating custom archives, then putting them in local package directory,
then not updating setup.ini. wait no, that was Ken:

http://cygwin.com/ml/cygwin/2017-12/msg00092.html

you presented no evidence or references really:

http://cygwin.com/ml/cygwin/2017-12/msg00145.html

mostly and opinion post. its possible that this issue doesnt even apply to you.
with ken, he is building custom archives so it applies to him. with me, i have
a package manager that runs in concert with setup.exe, so it applies to me:

http://github.com/svnpenn/sage

do you actually have an personal experience in this matter, or did you just want
to slight me and offer an opinion of someone with no experience of the issue at
hand. i fear the latter. APT, DNF and ZYpp are very popular package managers,
with APT being used by hundreds of thousands of people:

http://stackexchange.com/leagues/76

so it makes sense that setup.exe should behave in a manner similar to APT,
unless a very good reason is discovered why it shouldnt. more likely the
setup.exe behavior with corrupt files was decided long ago, and without
consideration to other package managers. and now that I am considering them, we
maybe have some people dissenting just to dissent, with no personal experience
of the issue. we are not talking removing files from:

C:\Users\Vince\Documents

we are talking about removing files from:

C:\http%3a%2f%2fcygwin.mirror.constant.com%2f

or similar, a directory specifically created by setup.exe. It is not
unreasonable to expect that similar to how setup.exe has the power to remove
anything from:

C:\cygwin64

that it would and should have the power to remove from a directory that it
created.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: setup's response to a "corrupt local copy"

[Ken Brown]

On 12/16/2017 9:40 AM, Steven Penny wrote:
> you presented a use case 
> where
> you were creating custom archives, then putting them in local package 
> directory,
> then not updating setup.ini. wait no, that was Ken

Actually that's not what I said.  You jumped to a conclusion based on an 
admittedly sketchy description of my use case.  But it's an obscure use 
case anyway, and I'm not interested in pursuing it.

I'm currently working on a patch that implements my suggestion (query 
the user).  I expect to send it to cygwin-apps within the next day or so.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: setup's response to a "corrupt local copy"

[Andrey Repin]

Greetings, Ken Brown!

> On 12/14/2017 2:46 PM, Andrey Repin wrote:
>> Greetings, Ken Brown!
>> 
>>> This is a followup to the discussion started here:
>> 
>>>     https://cygwin.com/ml/cygwin/2017-12/msg00088.html
>> 
>>> When setup is preparing to download files and it finds a corrupt copy in
>>> the local cache, it issues a fatal error message telling the user to
>>> remove the corrupt file and retry.  Steven said that setup should
>>> silently delete the corrupt file, while I argued in favor of the current
>>> behavior, on the grounds that setup shouldn't be deleting user files if
>>> it doesn't know where they came from.
>> 
>> The point being, if this is a "Download" Setup mode, the files are NOT "User"
>> files, but a local setup cache. And all files therein SHOULD be valid package
>> archives.
>> There's of course situations, when setup.ini on server become corrupted or
>> otherwise out of sync. But being rare, they should not interfere too much.
>> 
>>> There is a middle ground: setup could query the user.  Additionally, as
>>> suggested by cyg Simple, there could be an option that directs setup to
>>> silently remove corrupt files.
>> 
>> Make it mode dependent.
>> If it's a "download[ and install]" mode, cleanup and redownload.
>> If redownloaded file still does not match the setup.ini hash or if it's an
>> "install from local cache" mode, leave file alone for investigation and notify
>> the user.

> You've misunderstood the context.  The error is only shown in download or 
> download/install mode.

Where did I say, when this error is shown currently?

> And, as I said, it happens when setup is *preparing* to
> download files and finds a corrupt copy already present in the local cache. In
> that context, setup has no idea where the file came from.

Please forget about current behavior and read my post again.


-- 
With best regards,
Andrey Repin
Friday, December 15, 2017 15:44:59

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: setup's response to a "corrupt local copy"

[Brian Inglis]

On 2017-12-14 12:57, Ken Brown wrote:
> On 12/14/2017 2:46 PM, Andrey Repin wrote:
>>> This is a followup to the discussion started here:
>>>     https://cygwin.com/ml/cygwin/2017-12/msg00088.html
>>> When setup is preparing to download files and it finds a corrupt copy in
>>> the local cache, it issues a fatal error message telling the user to
>>> remove the corrupt file and retry.  Steven said that setup should
>>> silently delete the corrupt file, while I argued in favor of the current
>>> behavior, on the grounds that setup shouldn't be deleting user files if
>>> it doesn't know where they came from.
>> The point being, if this is a "Download" Setup mode, the files are NOT "User"
>> files, but a local setup cache. And all files therein SHOULD be valid package
>> archives.
>> There's of course situations, when setup.ini on server become corrupted or
>> otherwise out of sync. But being rare, they should not interfere too much.
>>> There is a middle ground: setup could query the user.  Additionally, as
>>> suggested by cyg Simple, there could be an option that directs setup to
>>> silently remove corrupt files.
>> Make it mode dependent.
>> If it's a "download[ and install]" mode, cleanup and redownload.
>> If redownloaded file still does not match the setup.ini hash or if it's an
>> "install from local cache" mode, leave file alone for investigation and notify
>> the user.
> You've misunderstood the context.  The error is only shown in download or
> download/install mode.  And, as I said, it happens when setup is *preparing* to
> download files and finds a corrupt copy already present in the local cache.  In
> that context, setup has no idea where the file came from.

Setup knows the file came from
	.../%encoded-mirror-url/arch/release/package/.../pvr....tar.xz
so it could warn the user of the validation failure, rename the file failing
validation if the cache is (one of) the currently selected mirror(s) and inform
the user of the rename, then add the package for redo of download/install from
(one of) the currently selected mirror(s), similar to a package prereq.
This would DTRT in most cases and not blow away the questionable cached file.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple