From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 2161) id 1BC953858408; Fri, 1 Mar 2024 13:38:09 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 1BC953858408 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1709300289; bh=EyNWJmILSd4E3lZMDxPlo9pduuVDkK3GKP0okuTT6r4=; h=To:Subject:Date:From:From; b=L+MiVZ6Mjkx2jQAfbOZ2KfKA0Pz4bBFhSjZXw382P4CJbR4WVV3EtqWvujTHZk2Yw 4CSacC/bLFWFrxe08DUftAkpWW01jb7QqzFX1ucrrt8cPS4mnEzv/nKcZPl8NNt9yd VFgWTZU318/z4/orXv+A+OQ3uyVessw1DWVqT1nc= To: gcc-cvs-wwwdocs@gcc.gnu.org Subject: gcc-wwwdocs branch master updated. 509d9104a9ae8246122c3d97b3e33ddf30416f93 X-Git-Refname: refs/heads/master X-Git-Reftype: branch X-Git-Oldrev: 57e980697644832bf9e35a3722ae286ebdca3587 X-Git-Newrev: 509d9104a9ae8246122c3d97b3e33ddf30416f93 Message-Id: <20240301133809.1BC953858408@sourceware.org> Date: Fri, 1 Mar 2024 13:38:09 +0000 (GMT) From: Mark Wielaard List-Id: This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "gcc-wwwdocs". The branch, master has been updated via 509d9104a9ae8246122c3d97b3e33ddf30416f93 (commit) from 57e980697644832bf9e35a3722ae286ebdca3587 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 509d9104a9ae8246122c3d97b3e33ddf30416f93 Author: Mark Wielaard Date: Fri Mar 1 14:33:39 2024 +0100 robots.txt: Disallow various wiki actions It is fine for robots to crawl the wiki pages, but they should perform actions, generate huge diffs, search/highlight pages or generate calendars. diff --git a/htdocs/robots.txt b/htdocs/robots.txt index 057c5899..36be4d13 100644 --- a/htdocs/robots.txt +++ b/htdocs/robots.txt @@ -14,4 +14,8 @@ Disallow: /bugzilla/show_bug.cgi*ctype=xml* Disallow: /bugzilla/attachment.cgi Disallow: /bugzilla/showdependencygraph.cgi Disallow: /bugzilla/showdependencytree.cgi +Disallow: /wiki/*?action=* +Disallow: /wiki/*?diffs=* +Disallow: /wiki/*?highlight=* +Disallow: /wiki/*?calparms=* Crawl-Delay: 60 ----------------------------------------------------------------------- Summary of changes: htdocs/robots.txt | 4 ++++ 1 file changed, 4 insertions(+) hooks/post-receive -- gcc-wwwdocs