Slow down the MSNBot

Johannes Beus
(Author)
13.06.2004

    According to an article on webmasterworld.com, you are now able to limit the MSNBot:

    User-Agent: msnbot
    Crawl-Delay: 20

    where20 is the amount of seconds that the MSNBot is supposed to wait between two pageviews.

    Johannes Beus
    (Author)
    13.06.2004
    Related posts
    13.8% of all page views go to searchengines
    At least that is what Hitwise, which uses userprofiles from several ISPs for their statistical data, is claiming. Google’s...
    Johannes Beus
    5. June 2004
    Yahoo!-Japan is also replacing Google
    After Yahoo! has already switched from their former index-supplier Google to their own technology in many different countries,...
    Johannes Beus
    3. June 2004
    PageRank and Backlink Update at Google
    ...
    Johannes Beus
    31. May 2004
    Comments
    Comments will be closed 30 days after the post was published.