* [gcc-wwwdocs COMMITTED] Disallow /cgit for web robots
@ 2023-12-09 0:43 Mark Wielaard
0 siblings, 0 replies; only message in thread
From: Mark Wielaard @ 2023-12-09 0:43 UTC (permalink / raw)
To: gcc-patches; +Cc: Gerald Pfeifer, Mark Wielaard
Although cgit is more efficient than gitweb it still is not great
for bots to go through it.
---
htdocs/robots.txt | 1 +
1 file changed, 1 insertion(+)
diff --git a/htdocs/robots.txt b/htdocs/robots.txt
index c650057b..b9fc830d 100644
--- a/htdocs/robots.txt
+++ b/htdocs/robots.txt
@@ -6,6 +6,7 @@ User-agent: *
Disallow: /viewvc/
Disallow: /viewcvs
Disallow: /git/
+Disallow: /cgit/
Disallow: /svn
Disallow: /cgi-bin/
Disallow: /bugzilla/buglist.cgi
--
2.39.3
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2023-12-09 0:44 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-12-09 0:43 [gcc-wwwdocs COMMITTED] Disallow /cgit for web robots Mark Wielaard
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).