Slow down the MSNBot

Johannes Beus
Published: 13.06.2004
Johannes Beus
Johannes Beus ist Gründer und Geschäftsführer von SISTRIX.
0 Comments 0

According to an article on webmasterworld.com, you are now able to limit the MSNBot:

User-Agent: msnbot
Crawl-Delay: 20

where20 is the amount of seconds that the MSNBot is supposed to wait between two pageviews.

Related posts
13.8% of all page views go to searchengines
At least that is what Hitwise, which uses userprofiles from several ISPs for their statistical data, is claiming. Google’s...
Johannes Beus
5. June 2004
Yahoo!-Japan is also replacing Google
After Yahoo! has already switched from their former index-supplier Google to their own technology in many different countries,...
Johannes Beus
3. June 2004
PageRank and Backlink Update at Google
...
Johannes Beus
31. May 2004
Comments
Comments will be closed 30 days after the post was published.