[Haskell-cafe] haskell wiki indexing

[Henning Thielemann]

On Tue, 22 May 2007, Vincent Kraeutler wrote:

> as was pointed out on the programming reddit [1], crawling of the
> haskell wiki is forbidden, since http://www.haskell.org/robots.txt contains
>
> User-agent: *
> Disallow: /haskellwiki/
>
> and indeed, a google search gives the old wiki
> http://www.google.ch/search?q=haskell+wiki
> i.e.
> http://haskell.org/hawiki/FrontPage
> rather than
> http://haskell.org/haskellwiki/Haskell

This also applies to Haskell mailing lists as I mentioned recently:
 http://www.haskell.org/pipermail/haskell-cafe/2007-April/025006.html