From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 25242 invoked by alias); 16 Apr 2006 17:24:47 -0000 Received: (qmail 25235 invoked by uid 22791); 16 Apr 2006 17:24:46 -0000 X-Spam-Check-By: sourceware.org Received: from ACCESS1.CIMS.NYU.EDU (HELO access1.cims.nyu.edu) (128.122.81.155) by sourceware.org (qpsmtpd/0.31) with ESMTP; Sun, 16 Apr 2006 17:24:44 +0000 Received: from localhost (localhost [127.0.0.1]) by access1.cims.nyu.edu (8.13.6+Sun/8.13.6) with ESMTP id k3GHOeit018242 for ; Sun, 16 Apr 2006 13:24:40 -0400 (EDT) Date: Sun, 16 Apr 2006 17:24:00 -0000 From: Igor Peshansky Reply-To: The Cygwin-Talk Malingering List To: The Cygwin-Talk Maiming List Subject: Re: Mirroring and stage and tmp dirs In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-IsSubscribed: yes Mailing-List: contact cygwin-talk-help@cygwin.com; run by ezmlm Precedence: bulk List-Subscribe: List-Post: List-Help: , Sender: cygwin-talk-owner@cygwin.com Reply-To: The Cygwin-Talk Malingering List X-SW-Source: 2006-q2/txt/msg00029.txt.bz2 On Sun, 16 Apr 2006, Robert Pendell wrote: > Igor Peshansky wrote: > > On Sun, 16 Apr 2006, Robert Pendell wrote: > > > >> Should we be ignoring these folders during mirroring? I was mirroring > >> them but recently started getting permission denied errors during > >> transfers for those folders so they are now excluded from rsync > >> transfers (i.e. not on my public mirror). What is the stage folder used > >> for? > > > > The /stage folder is used to build up parts of the changed directory > > structure and transfer them over in one swell foop (to avoid a race > > with the upset cron job, where it could catch the uploader in > > mid-update and result in a broken setup.ini). > > > > In short, you should not mirror that directory, as nothing in there is > > going to be useful on a mirror. > > > >> rsync: send_files failed to open "/stage/tcp_wrappers-7.6-2.sh" (in > >> cygwin-ftp): Permission denied (13) > >> rsync: send_files failed to open "/stage/tcp_wrappers_7.6.tar.gz" (in > >> cygwin-ftp): Permission denied (13) > >> > >> And from earlier on... > >> rsync: send_files failed to open "/stage/monotone-0.26-1.sh" (in > >> cygwin-ftp): Permission denied (13) Heh, I just looked closely at the names of the files... Looks like someone unpacked a source archive there -- not part of your normal "create files/directories and move them to /release" sequence... Don't know why this happened, but my explanation below is probably not correct for this particular case. > > You probably caught the uploader in the act of moving files from > > /stage to /release. > > Igor > > Ok. I updated the cron job last night to exclude those folders from now > on. I've been trying to optimized the command so it causes as few 404's > as possible. It is setup to delete files after getting the new ones but > there are still a few occuring. > > Oh and an interesting statistic. So far this month I have had more hits > from people using Wget then the regular setup program. _Alot_ more hits. > > Wget (various versions) - 76841 > Cygwin Setup - 26035 > > This is versus last month where Cygwin and Wget was very close. I > wonder if people are starting to branch (2nd tier) off my mirror... I doubt it. Can you also check whether the hits are for the current binary tarballs, or for the older source/binary ones? If the former, it might indicate the more prominent usage of clean_setup.pl (which builds lists of wget commands). If the latter, people may be doing what I occasionally do -- grabbing the whole older tarball to extract an older version of some file... Also, are some tarballs hit more than others? Igor -- http://cs.nyu.edu/~pechtcha/ |\ _,,,---,,_ pechtcha@cs.nyu.edu | igor@watson.ibm.com ZZZzz /,`.-'`' -. ;-;;,_ Igor Peshansky, Ph.D. (name changed!) |,4- ) )-,_. ,\ ( `'-' old name: Igor Pechtchanski '---''(_/--' `-'\_) fL a.k.a JaguaR-R-R-r-r-r-.-.-. Meow! "Las! je suis sot... -Mais non, tu ne l'es pas, puisque tu t'en rends compte." "But no -- you are no fool; you call yourself a fool, there's proof enough in that!" -- Rostand, "Cyrano de Bergerac"