Johannes Beus
(Author)
27.03.2006
While taking a look at the logfiles for a FTP-server I noticed that the Googlebot (recently?) is also interested in FTP-servers and registers there:
Mar 26 06:54:40 pure-ftpd: (?@crawl-66-249-65-35.googlebot.com) [INFO] Neue Verbindung von crawl-66-249-65-35.googlebot.com.
Mar 26 06:54:40 pure-ftpd: (?@crawl-66-249-65-35.googlebot.com) [INFO] Anonymer Benutzer angemeldet.
Mar 26 06:54:41 pure-ftpd: (ftp@crawl-66-249-65-35.googlebot.com) [INFO] Logout.
In a few cases, Google even downloaded some files:
Mar 26 19:15:02 pure-ftpd: (ftp@crawl-66-249-65-35.googlebot.com) [NOTICE] /home/ftp/xxx/xxx.html heruntergeladen. (5828 bytes, 400.05KB/sec)
Is this going on for a while and I just did not notice? And are those files added to the Index?
