Google crawls FTP

Johannes Beus
Johannes Beus
Johannes Beus ist Gründer und Geschäftsführer von SISTRIX.

While taking a look at the logfiles for a FTP-server I noticed that the Googlebot (recently?) is also interested in FTP-servers and registers there:

Mar 26 06:54:40 pure-ftpd: (?@crawl-66-249-65-35.googlebot.com) [INFO] Neue Verbindung von crawl-66-249-65-35.googlebot.com.
Mar 26 06:54:40 pure-ftpd: (?@crawl-66-249-65-35.googlebot.com) [INFO] Anonymer Benutzer angemeldet.
Mar 26 06:54:41 pure-ftpd: (ftp@crawl-66-249-65-35.googlebot.com) [INFO] Logout.

In a few cases, Google even downloaded some files:

Mar 26 19:15:02 pure-ftpd: (ftp@crawl-66-249-65-35.googlebot.com) [NOTICE] /home/ftp/xxx/xxx.html heruntergeladen. (5828 bytes, 400.05KB/sec)

Is this going on for a while and I just did not notice? And are those files added to the Index?

Related posts
Comments
Comments will be closed 30 days after the post was published.