From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 101700 invoked by alias); 7 Nov 2018 20:19:04 -0000 Mailing-List: contact cygwin-apps-help@cygwin.com; run by ezmlm Precedence: bulk Sender: cygwin-apps-owner@cygwin.com List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Mail-Followup-To: cygwin-apps@cygwin.com Received: (qmail 101601 invoked by uid 89); 7 Nov 2018 20:19:03 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-1.7 required=5.0 tests=AWL,BAYES_00,SPF_PASS,TVD_RCVD_IP autolearn=ham version=3.3.2 spammy=H*Ad:U*cygwin-apps, doctor, displayed, H*RU:sk:ripple. X-HELO: ripple.fruitbat.org Received: from 173-228-5-241.dsl.static.fusionbroadband.com (HELO ripple.fruitbat.org) (173.228.5.241) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 07 Nov 2018 20:19:01 +0000 Received: from ming.fruitbat.org (ming.fruitbat.org [192.168.55.2]) by ripple.fruitbat.org (8.14.4/8.14.4/fruitwall-1.7) with ESMTP id wA7KHkAX007387 for ; Wed, 7 Nov 2018 12:17:46 -0800 Received: from ming.fruitbat.org (localhost [127.0.0.1]) by ming.fruitbat.org (8.14.4/8.14.4/fruithub-1.11) with ESMTP id wA7KHkEd025630 for ; Wed, 7 Nov 2018 12:17:46 -0800 Received: (from doctor@localhost) by ming.fruitbat.org (8.14.4/8.14.4/Submit) id wA7KHjYF025629 for cygwin-apps@cygwin.com; Wed, 7 Nov 2018 12:17:45 -0800 Date: Wed, 07 Nov 2018 20:19:00 -0000 From: "Peter A. Castro" To: Cygwin Apps Subject: Problem with Setup and multiple URLs with same host Message-ID: <20181107201745.GD3598@ming.fruitbat.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.10.0 (2018-05-17) X-SW-Source: 2018-11/txt/msg00000.txt.bz2 Greetings, All, I tend to just be a lurker on the Cygwin lists, but a user of the Cygwin Time Machine had a problem with the latest Setup version (2.893) that made me think a little about how Setup treats URLs. I'd email Jon Turney off list, and, after some discussion, felt it should be brought up for discussion. The user, call him "Mike", was getting a failure trying to fetch a specific circa of Cygwin from the Cygwin Time Machine. The setup.log showed something I hadn't seen before, which was that multiple URLs were being accessed. Here's a piece of the log: 2018/11/05 17:10:45 Starting cygwin install, version 2.893 2018/11/05 17:10:45 User has backup/restore rights 2018/11/05 17:10:45 Current Directory: C:\cygwin-install\cygwin-time-machine 2018/11/05 17:10:45 Could not open service McShield for query, start and stop. McAfee may not be installed, or we don't have access. 2018/11/05 17:10:49 source: download 2018/11/05 17:10:50 Selected local directory: C:\cygwin-install\cygwin-time-machine 2018/11/05 17:10:51 net: Direct 2018/11/05 17:11:24 site: http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/64bit/ 2018/11/05 17:11:24 site: http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/64bit/2014/04/06/120212/ 2018/11/05 17:11:24 site: http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/64bit/index.html / 2018/11/05 17:11:27 HTTP status 404 fetching http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/64bit/2014/04/06/120212/x86_64/setup.xz.sig 2018/11/05 17:11:28 HTTP status 404 fetching http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/64bit/index.html /x86_64/setup.xz.sig 2018/11/05 17:11:28 HTTP status 404 fetching http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/64bit/index.html /x86_64/setup.xz 2018/11/05 17:11:28 HTTP status 404 fetching http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/64bit/index.html /x86_64/setup.bz2.sig 2018/11/05 17:11:29 HTTP status 404 fetching http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/64bit/index.html /x86_64/setup.bz2 2018/11/05 17:11:29 HTTP status 404 fetching http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/64bit/index.html /x86_64/setup.ini.sig 2018/11/05 17:11:29 HTTP status 404 fetching http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/64bit/index.html /x86_64/setup.ini 2018/11/05 17:11:29 mbox note: Unable to get setup from 2018/11/05 17:11:33 Ending cygwin install Now, I did not know that Mike had previously tried adding several incorrect URLs before, but the end result was that he'd added three URLs all having the same location: http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/64bit/ http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/64bit/2014/04/06/120212/ http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/64bit/index.html / These were recorded, together, in setup.rc under "last-mirror", because they were added as User mirrors, but they are displayed in the Chooser as only one location. Now, as it turns out, two of these will resolve to a valid setup.ini{.xz} (being qualified with "/x86_64"), but Setup seems to insist that all three be valid, and, of course, the last one with "/index.html /" wouldn't have ever worked. But, also, he really only wanted the one for 2014/04/06/120212. A solution for Mike was to edit setup.rc and remove all three URLs and then re-add only the correct one. But, that was not obvious and required the use of an external, native editor. But this brings up a preceived deficiency in how Setup handles mirror URLs and failures. Specifically, that multiple URLs with the same host & protocol are treated as one virtual entity. I can understand the intent to allow a mirror to be split up between multiple paths, but the problem is that that knowledge is being hidden from the user. It also appears that Setup lacks an effective way to edit the list of mirrors in a find-grained manner. Historically, mirrors supply only one URL for a repository, and I realize that the Cygwin Time Machine necessarily deviates from that norm. But, there really is no other choice in how to present individual time slices of Cygwin such that Setup can access them. I'd like to suggest that Setup needs to perform come kind of URL validation at the time a user provided URL is added. Something simple like getting the head of setup.ini at that time of addition, perhaps. That might help prevent bad mirrors from being added by the user. I realize that that doesn't prevent the mirror path from being removed later, but, that leads to my next point: Better detail and management. Setup hides the full URL from the user when it displays URLs in most any way, and I feel this is a disservice. It really should display the full URL, particularly when managing the list of mirrors, but also when choosing a mirror to use. I'm not sure where the requirement to group multiple URLs with the same host, together, came from (haven't checked the email archives, yet), but I can see both pros and cons for this functionality. However it really should be under the users control and not assumed. As it is now, the failure Mike observed didn't give him any real clue as to what or why, which is why he email'd me. Having users refer to the setup.log to do diagnostics isn't very user-friendly, and there's no obvious logic or notice that multiple URLs will be used, so it was a bit baffling to me to see that too. Now, of course, the quick fix is to "not enter mutliple/bad URLs", but that belies the user experience with Setup, which doesn't do a whole lot of validation of user values until they are tried to be accessed much later on, which is much too late. Anyway, just wanted to bring this to light and see what people think. I haven't had time to look at the code for Setup but I'm hoping that the good people who are most familiar with it can give a quick evaluation if some of this is easy to implement or change. Yes, I know, "PTC" :) -- --=> Peter A. Castro Email: doctor at fruitbat dot org / Peter dot Castro at oracle dot com "Cats are just autistic Dogs" -- Dr. Tony Attwood