gcc-wwwdocs branch master updated. 9b040396d6931afd408d17f6dee698a12b70114b

gcc-wwwdocs branch master updated. 9b040396d6931afd408d17f6dee698a12b70114b

[Mark Wielaard]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "gcc-wwwdocs".

The branch, master has been updated
       via  9b040396d6931afd408d17f6dee698a12b70114b (commit)
      from  c7784b7b76dcb2881c32f6fa9b3fe2ff00390e5e (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
commit 9b040396d6931afd408d17f6dee698a12b70114b
Author: Mark Wielaard <mark@klomp.org>
Date:   Sat Dec 9 01:40:17 2023 +0100

    Disallow /cgit for web robots
    
    Although cgit is more efficient than gitweb it still is not great
    for bots to go through it.

diff --git a/htdocs/robots.txt b/htdocs/robots.txt
index c650057b..b9fc830d 100644
--- a/htdocs/robots.txt
+++ b/htdocs/robots.txt
@@ -6,6 +6,7 @@ User-agent: *
 Disallow: /viewvc/
 Disallow: /viewcvs
 Disallow: /git/
+Disallow: /cgit/
 Disallow: /svn
 Disallow: /cgi-bin/
 Disallow: /bugzilla/buglist.cgi

-----------------------------------------------------------------------

Summary of changes:
 htdocs/robots.txt | 1 +
 1 file changed, 1 insertion(+)


hooks/post-receive
-- 
gcc-wwwdocs