• SEO-Blog
  • Startseite
  • RSS-Feed, complete Feed
  • About the Blog, Blogcharts
  • Archive: 2003, 2004, 2005, 2006, 2007, 2008, 2009
• Weitere Postings
  • Is Google planning datacenters in Germany?
  • Microsoft purchases Yahoo
  • Has “trust” become too strong as a rating-factor?
  • Visibility of German projects in the Googleindex
  • Results displayed directly in the Google-SERPs
  • PDF: searchengine-optimization essentials
  • Microsoft's and Google's referrer-spam
  • Hosting with own IP-address – necessary for SEO?
  • Brockhaus gains insight
  • Marion's cookbook – a SEO-reflection
•  

New Tag in Town: X-Robots

After Google followed Yahoo's announcement to also support metatags in the HTTP-headers in the future, it is high time to start thinking about where this can be applied and what the consequences could be. Fundamentally this is about directions, which searchengine-crawlers, up until now, received from the meta-tags in the HTML-page, that can now also be transmitted through the HTTP-header.

The difference between the HTTP-header and the head-area of the HTML-file is that, as a rule, the former is even more “invisible” to the visitor. While you only have to look at the sourcecode of the page through the browser (Ctrl+U for Firefox) to view the head-area, you need external plugins or tools to do the same for the HTTP-header. For Firefox you can get the highly recommended plugin LiveHTTPHeaders and for an onlinetool we have a HTTP-Header-Viewer in our tools area. To get a feeling for the general integration of the X-Robots-tag, I put an example-file online: x-robots-demo.php (PHP-Source)

What advantages does this Header-tag give us now? Up until now it has been the case that meta-tags could only be used when a HTML-head-area was present – for many filetypes like PDF's, picture or downloadable files this method for robot-control was non existent on a file level. When you wanted to keep PDF's from being indexed you regularly had to make an extra folder for them that was then blocked via the robots.txt. Advanced controlling options like the “noindex, follow” were not possible – the X-Robots-tag enables, for the first time, a control of the indexing on a file level. Admittedly you have to consider that this adds another point for a link to inherit “LinkJuice” to searchengines which has to be checked for. While, up until now, it was sufficient to test for “rel='nofollow'” on the link-level as well as “meta-nofollow” on the page-level, now you have to additionally test the HTTP-Headers.