Re: sourceware on search engines

[Christopher Faylor <cgf-use-the-mailinglist-please@sourceware.org>]

On Thu, Nov 11, 2010 at 07:50:34PM +0000, Jonathan Larmour wrote:
>On 11/11/10 18:09, Frank Ch. Eigler wrote:
>> Hi -
>> 
>>> Frank> They generally are indexed (not included in robots.txt).  Can you give
>>> Frank> an example of what you see missing?
>>>
>>> I've seen this too.
>>> Almost any search that I would expect to hit on sourceware.org instead
>>> pulls up results from elsewhere, often cygwin.ru.
>> 
>> google has funny heuristics about which copy of a mirror to present for
>> any given query.
>
>It seems like this is indeed the origin of my belief too. If I look much
>later in search results I do eventually see sourceware aliases crop up.
>For example a google for "gdb cortex registers" does eventually show a
>result straight from sourceware, but only on p.10.
>
>>> E.g., search for "systemtap signedness roland" on Google.
>>> This shows cygwin.ru, nabble.com, but not sourceware.
>>> Now add "site:sourceware.org" -- I see no hits.
>> 
>> OTOH, sourceware.org/ml/* is indexed by google, and for other queries,
>> we get hits just fine.  So whatever the problem is, it's not as simple
>> as it being blocked.  It's more about freshness or crawling rate or
>> something.
>
>Perhaps it's the presence of all the site aliases? I vaguely recall that
>google lowers the rank of sites that have aliases pointing to the same
>pages - that's a ploy that people have done in the past to try and improve
>their search ranking. Maybe cygwin.com should only have links to the
>cygwin* mailing lists, ecos.sourceware.org to the ecos* lists, and so on?

And don't forget: sources.redhat.com.

http://google.com/search?q=systemtap+signedness+roland+site:sources.redhat.com

cgf