* robots.txt
@ 1999-03-09 15:04 Gerald Pfeifer
[not found] ` < Pine.GSO.4.10.9902150100390.9254-100000@alphard.dbai.tuwien.ac.at >
1999-03-31 23:46 ` robots.txt Gerald Pfeifer
0 siblings, 2 replies; 4+ messages in thread
From: Gerald Pfeifer @ 1999-03-09 15:04 UTC (permalink / raw)
To: Jason Molenda, egcs
I noticed that we don't (yet) have a robots.txt file for the egcs web
server.
How about the following for now?
User-Agent: *
Disallow: /cgi-bin/
If there are no objections, I'll commit that in a couple of days...
Gerald
--
Gerald Pfeifer (Jerry) Vienna University of Technology
pfeifer@dbai.tuwien.ac.at http://www.dbai.tuwien.ac.at/~pfeifer/
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: robots.txt
[not found] ` < Pine.GSO.4.10.9902150100390.9254-100000@alphard.dbai.tuwien.ac.at >
@ 1999-03-11 14:11 ` Gerald Pfeifer
1999-03-31 23:46 ` robots.txt Gerald Pfeifer
0 siblings, 1 reply; 4+ messages in thread
From: Gerald Pfeifer @ 1999-03-11 14:11 UTC (permalink / raw)
To: Jason Molenda, egcs; +Cc: egcs-patches
On Wed, 10 Mar 1999, Gerald Pfeifer wrote:
> How about the following for now?
I have now installed the following robots.txt file for our egcs.cygnus.com
site as wwwdocs/htdocs/robots.txt.
---- cut ----
# See http://info.webcrawler.com/mak/projects/robots/norobots.html
# for information about the file format.
# Contact egcs@egcs.cygnus.com for questions.
User-Agent: *
Disallow: /cgi-bin/
---- cut ----
Thanks to Jason for his comments...
Gerald
--
Gerald "Jerry" pfeifer@dbai.tuwien.ac.at http://www.dbai.tuwien.ac.at/~pfeifer/
^ permalink raw reply [flat|nested] 4+ messages in thread
* robots.txt
1999-03-09 15:04 robots.txt Gerald Pfeifer
[not found] ` < Pine.GSO.4.10.9902150100390.9254-100000@alphard.dbai.tuwien.ac.at >
@ 1999-03-31 23:46 ` Gerald Pfeifer
1 sibling, 0 replies; 4+ messages in thread
From: Gerald Pfeifer @ 1999-03-31 23:46 UTC (permalink / raw)
To: Jason Molenda, egcs
I noticed that we don't (yet) have a robots.txt file for the egcs web
server.
How about the following for now?
User-Agent: *
Disallow: /cgi-bin/
If there are no objections, I'll commit that in a couple of days...
Gerald
--
Gerald Pfeifer (Jerry) Vienna University of Technology
pfeifer@dbai.tuwien.ac.at http://www.dbai.tuwien.ac.at/~pfeifer/
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: robots.txt
1999-03-11 14:11 ` robots.txt Gerald Pfeifer
@ 1999-03-31 23:46 ` Gerald Pfeifer
0 siblings, 0 replies; 4+ messages in thread
From: Gerald Pfeifer @ 1999-03-31 23:46 UTC (permalink / raw)
To: Jason Molenda, egcs; +Cc: egcs-patches
On Wed, 10 Mar 1999, Gerald Pfeifer wrote:
> How about the following for now?
I have now installed the following robots.txt file for our egcs.cygnus.com
site as wwwdocs/htdocs/robots.txt.
---- cut ----
# See http://info.webcrawler.com/mak/projects/robots/norobots.html
# for information about the file format.
# Contact egcs@egcs.cygnus.com for questions.
User-Agent: *
Disallow: /cgi-bin/
---- cut ----
Thanks to Jason for his comments...
Gerald
--
Gerald "Jerry" pfeifer@dbai.tuwien.ac.at http://www.dbai.tuwien.ac.at/~pfeifer/
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~1999-03-31 23:46 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
1999-03-09 15:04 robots.txt Gerald Pfeifer
[not found] ` < Pine.GSO.4.10.9902150100390.9254-100000@alphard.dbai.tuwien.ac.at >
1999-03-11 14:11 ` robots.txt Gerald Pfeifer
1999-03-31 23:46 ` robots.txt Gerald Pfeifer
1999-03-31 23:46 ` robots.txt Gerald Pfeifer
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).