Tool for the Robots.txt

Johannes Beus
Johannes Beus
(Author)
08.02.2006

    Google is offering a tool though Google Sitemaps that checks the robots.txt. After reading out the file, the tool will display the allowed and restricted directories.

    Johannes Beus
    Johannes Beus
    (Author)
    08.02.2006
    Related posts
    Canonical Addresses
    Christoph Schneegans is giving some tips on how to avoid duplicate content: Canonical Addresses
    Johannes Beus
    Johannes Beus
    7. February 2006
    Example 2.0
    After the harsh crackdown on doorwaypages at automobile.de, Google has removed bmw.de from the index. Even though it was...
    Johannes Beus
    Johannes Beus
    4. February 2006
    Web.de wants to start local searchengine
    Web.de announced to start a local search for restaurants, hotels, etc. just in time for this years CeBit. The database will...
    Johannes Beus
    Johannes Beus
    1. February 2006