From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 3069 invoked by alias); 12 Apr 2004 21:05:59 -0000 Mailing-List: contact overseers-help@sources.redhat.com; run by ezmlm Precedence: bulk List-Archive: List-Post: List-Help: , Sender: overseers-owner@sources.redhat.com Received: (qmail 3059 invoked from network); 12 Apr 2004 21:05:56 -0000 Received: from unknown (HELO timesys.com) (66.30.22.40) by sources.redhat.com with SMTP; 12 Apr 2004 21:05:56 -0000 Received: by timesys.com (Postfix, from userid 201) id 021654005FD; Mon, 12 Apr 2004 17:05:55 -0400 (EDT) Date: Mon, 12 Apr 2004 21:28:00 -0000 From: Christopher Faylor To: "Frank Ch. Eigler" Cc: Sourceware Overseers Subject: Re: search engines crawling bugzilla Message-ID: <20040412210555.GA23495@coe.bosbc.com> Mail-Followup-To: "Frank Ch. Eigler" , Sourceware Overseers References: <20040412205728.GF23436@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040412205728.GF23436@redhat.com> User-Agent: Mutt/1.4.1i X-SW-Source: 2004-q2/txt/msg00171.txt.bz2 On Mon, Apr 12, 2004 at 04:57:28PM -0400, Frank Ch. Eigler wrote: >Part of the high load on sourceware in the last few hours has been >due to a large number of bugzilla/buglist.cgi perl scripts running. >Many of these in turn are being invoked by google. Assuming this >was unintentional, are there any objections to my listing the >bugzilla cgi directories for exclusion in the robots.txt files? I'm sure this was unintentional. I see that there is already a robots.txt file. I wonder why it is so sparse. cgf