Neu registrieren / Login

Can I disallow crawling of my CSS and JavaScript files?

 
 

Video Video

You need Flash player 8+ and JavaScript enabled to view this video.
 
 

Abstract Abstract

Question: “Would I cause problems for the Googlebot if I disallow crawling of my CSS and JavaScript files?”

Answer: Google would prefer to have access to those files just in case they have to do a quality check on the site. Most of the times, they will not even fetch those files and since they do not consume much bandwidth, they should be accessible.
0:00
A fun question from SEOmofo in Simi Valley. They ask: „If i externalize all CSS style definitions and JavaScripts and disallow all user agents from accessing these external files (via robots.txt), would this cause problems for Googlebot? Does Googlebot need access to these files?”.

0:21
I personally would recommend not blocking that. So for example, the White House recently rolled out a new robots.txt and I think they blocked the images directory or CSS or JavaScript or something like that. You really don't need to do that and in fact, sometimes it can be very helpful if we think something spammy is going on with JavaScript, if somebody is doing a sneaky redirect or something like that.

0:44
So my personal advice would be to let Googlebot go ahead and crawl that and then its not like these files are huge anyways so it doesn't consume a lot of bandwidth. So my personal advice: “just go ahead and let Googlebot have access to all that stuff” and then, most of the time, we won't ever fetch it but in the rare occasion when were doing a quality check on behalf of someone or we receive a spamreport, then we can go ahead and fetch that and make sure that your site is clean and not not having any sorts of problems.