From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 60014 invoked by alias); 5 Jan 2018 23:26:47 -0000 Mailing-List: contact cygwin-apps-help@cygwin.com; run by ezmlm Precedence: bulk Sender: cygwin-apps-owner@cygwin.com List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Mail-Followup-To: cygwin-apps@cygwin.com Received: (qmail 60000 invoked by uid 89); 5 Jan 2018 23:26:45 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.1 required=5.0 tests=AWL,BAYES_00,SPF_HELO_PASS,SPF_PASS,T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 spammy=Download, offer, site X-HELO: limerock01.mail.cornell.edu Received: from limerock01.mail.cornell.edu (HELO limerock01.mail.cornell.edu) (128.84.13.241) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 05 Jan 2018 23:26:43 +0000 X-CornellRouted: This message has been Routed already. Received: from authusersmtp.mail.cornell.edu (granite3.serverfarm.cornell.edu [10.16.197.8]) by limerock01.mail.cornell.edu (8.14.4/8.14.4_cu) with ESMTP id w05NQfni030446; Fri, 5 Jan 2018 18:26:41 -0500 Received: from [10.13.22.4] (50-192-26-108-static.hfc.comcastbusiness.net [50.192.26.108]) (authenticated bits=0) by authusersmtp.mail.cornell.edu (8.14.4/8.12.10) with ESMTP id w05NQdGr013800 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 5 Jan 2018 18:26:40 -0500 Subject: Re: [PATCH setup] Query the user if a corrupt local file is found To: Jon Turney Cc: cygwin-apps@cygwin.com References: <20171216183731.10888-1-kbrown@cornell.edu> <86ba78c3-de97-6edf-613b-8483b3920851@dronecode.org.uk> From: Ken Brown Message-ID: <0f9e9636-b821-d7ab-f2e3-e0f8af31c378@cornell.edu> Date: Fri, 05 Jan 2018 23:26:00 -0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <86ba78c3-de97-6edf-613b-8483b3920851@dronecode.org.uk> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-PMX-Cornell-Gauge: Gauge=X X-PMX-CORNELL-AUTH-RESULTS: dkim-out=none; X-IsSubscribed: yes X-SW-Source: 2018-01/txt/msg00009.txt.bz2 On 1/5/2018 10:00 AM, Jon Turney wrote: > On 16/12/2017 18:37, Ken Brown wrote: >> If a corrupt file is found in one of the selected mirror site >> directories, offer to delete it instead of making this a fatal error. >> Do this only on the first call to check_for cached().  If the corrupt >> file is still there on the second call, then the deletion failed, and >> the user will have to fix the problem. >> >> See https://cygwin.com/ml/cygwin/2017-12/msg00122.html for discussion. > > This is a nice idea.  But I think there are some structural problems > with this, though. e.g. validateCachedPackage() only checks the package > size, not hash (which happens in the install phase) > > I'm also concerned about masking problems with how we got into this > state in the first place: I think either (i) a corrupt download was > stored into the cache, (ii) the valid size was changed between runs, or > (iii) the files contents actually got corrupted somehow. > > (i) indicates another problem in setup > > Uploading replacement packages, which would cause (ii) was permitted > historically (but of course, didn't work well), but now should be > forbidden by calm.  This could, of course, still happen with a private > package repo, and should be handled sanely. > > (iii) seems unlikely, barring deliberate action. > > I guess the ideal solution looks something like: > > Download: > - verify size/hash of cached packages, offer to remove corrupt ones > - download packages, verifying size/hash > > Install: > - verify size/hash of cached packages, skip corrupt ones > - install packages > > with some memory so that we don't verify size/hash for the same package > file more than once... I agree. It's strange that the code for size checking is currently in download.cc, and the code for hash checking is currently in install.cc. All of this checking should probably be done in a packagesource::validate() function, which can be called during both the download and install stages. I'll work on this and send a new patch. Ken