IndexWatch 06/2009
The first half of the year is through and we are on the home stretch towards Christmas. Here we have the latest IndexWatch-numbers for the previous month. As always, we start out with the winners in the Googleindex:
Winners
| # | Domain | Change | |
|---|---|---|---|
| 1 | weg24.de | > +1.000% |  |
| 2 | travel-links.de | > +1.000% | |
| 3 | over-blog.de | > +1.000% | |
| 4 | doesntexist.org | > +1.000% | |
| 5 | readster.de | > +1.000% | |
| 6 | isik.de | > +1.000% | |
| 7 | kelkoo.de | > +1.000% | |
| 8 | counter-gratis.com | > +1.000% | |
| 9 | fluege.de | > +1.000% | |
| 10 | simplyhired.de | +536% | |
| 11 | infopirat.com | +440% | |
| 12 | linguee.de | +417% | |
| 13 | beate-uhse.com | +367% | |
| 14 | viamichelin.ch | +290% | |
| 15 | bluray-disc.de | +246% | |
This time there are (once again) travel-sites as well as parasite-hosting-domains on the winning side: over-blog.de is a bloghosting-service which was able to gain significant ground over the last few weeks thats to some “users”, doesntexist.org is a DynDns domain which gives out free subdomains and is now ranking for words that I have never heard of before. There are rumors that something like this was supposed to have worked well a few years back but I am rather sure that Google will put a stop to it in a timely manner. We also have a few domains that escaped a Google-penalty (Visibilityindex): kelkoo.de, readster.de and infopirat.com – lets give it a while and see how long it will take them to get put back in penalty-land.
A great example for duplicate-content and how Google deals with it, are Beate-uhse.com on the winning side and Beate-uhse.de on the loosing end. It seems that all the content of the “www”-subdomain could be reached through both domains for a while and for a few weeks, Google decided that wanted to list the .de, while now it seems that they have changed their mind. By now, the problem has been fixed but you can still track it nicely through the Visibilityindex for the two hostnames.
Losers
| # | Domain | Change | |
|---|---|---|---|
| 1 | beate-uhse.de | -100% | |
| 2 | cwanswers.de | -100% | |
| 3 | poplexikon.com | -100% | |
| 4 | logomarket.com | -99% | |
| 5 | new-video.de | -99% | |
| 6 | truveo.com | -97% | |
| 7 | urlaubsreisen.eu | -97% | |
| 8 | datingjungle.de | -96% | |
| 9 | lastminute-reisepreisvergle... | -95% | |
| 10 | mallorca-today.de | -95% | |
| 11 | kundenserver.de | -93% | |
| 12 | celeblounge.net | -88% | |
| 13 | electronicscout24.de | -87% | |
| 14 | gamona.de | -86% | |
| 15 | juris.de | -86% | |
On the losing end we have quite the colorful picture. For many of the domains listed, I would guess the reason behind the loss would be incorrect linkbuilding. You would think that, by now, everyone on the Internet (and their mothers) knew that linkbuilding methods from 2004 are not working anymore and that they are being sanctioned by Google accordingly. So it is even more startling to see how many still try to go this way, which means that sooner or later they run into the (penalty)-wall.
Truveo.com is a video meta-searchengine that queries different video-portals like You Tube. Sadly, no one there seems to have read anywhere that Google does not like to list “SERPs-in-SERPs” – it seems that now, they were manually delisted.
Kundenserver.de is a service domain belonging to the 1&1 webhoster, where they offer HTTPS-encrypted sites through the “ssl.kundenserver.de” subdomain, for those domainowners that are unable/unwilling to pay for their own certificate. For quite a while, these sites ranked rather well,, seeing how they were rather “normal” shops, that profited a little bit from the power of that domain. A few weeks ago, all this stopped, as can be seen nicely in the Visibilityindex for the hostname. This becomes more graphic in this example: here we have the rankings for three domains for the keyword “Lebensversicherung Kaufen” (buy life-insurance). The green line is the Kundenserver.de-domain, which was constantly at position 1 until it disappeared from the ranking from one day to the next. I cannot really grasp the reason for this as all the pages are still in the index, it is just that Google does not like them anymore.
Did Google disown Burda?
Hubert Burda is publicly venting his grievance in the FAZ: Searchengines like Google are “silently disowning” content-producers. His reasoning is as follows: While quality-media (two of Burdas papers are the tabloids Superillu and Glücks-Revue) spend a lot of money to create great content, searchengines just take this content and keep a major share of the advertising profits. It is surely no coincidence, that he suddenly remembers this during an economic- as well as (print)-advertising recession. To change all this, he wants the government to intervene: they should adapt the laws so that publishing houses get a larger piece of the profits. It might not have been the smartest idea to use the music-industry as his role model in this.It is rather difficult to give an appropriate answer to something like this – while in and of itself, these arguments might make sense because they mirror the self-conception of the post-WWII publishing houses in this country. Just because the print-market held double digit returns for many years, does not mean that they have any claim to similar return in this new interwebs. The simply slept through new developments for years and now, that the pie is already carved up, they suddenly realize that they won't do as well as they used to.
At the same time, no other sector get such preferential treatment from Google as publishing houses do: ACAP, kind of a vastly extended robots.txt was created in close partnership with publishing houses and things like “First-Click-Free” show a lot of good will towards the content-producers. But then again, it still seems to be much easier to seek the problems with other than with themselves which makes Google the perfect scapegoat. Chris Anderson from Wired shows that this does not have to be the consensus. He writes: “I consider that a gift, but newspapers consider it theft.”
Keyword-density: relevant SEO-basics or negligible thing of the past?
There are a multitude of different opinions when it comes to keyword-density being a ranking-factor: for some, it's nothing more than a reminder of the early years of this century, where you were able to get far ahead in the Fireball searchengine by simply repeating the keyword numerous times in the text and filename. Others are still measuring their SEO-work by the percentage that the keyword takes up in the text. Seeing how, like so often, the truth can probably be found somewhere between those extremes, I pondered about this and came up with a test, with which I will try to get to the bottom of this or at least find some partial truth.The fundamental idea of the following test is the fact, that Google is ranking sites with a higher keyword-density better than sites with a lower one (under the condition that all other factors stay the same). With that in mind, I set up this test: a bunch of useless text with the keyword-density as their only distinguishing factor – all values between 0,1 and 10% are encompassed herein. I have tried to keep all other factors constant. Whats interesting to us, is which sites will show up first when we query the keyword? Surprisingly, we really do see the texts with a high keyword-density show up first: all results on the first resultspage have a keyword-density between 7,0 to 9,8%. When we take a look at the first 60 positions, we get this picture:
On first glance, it seems to be a pretty straightforward trend: the higher the keyword-density, the better the ranking. Though that would have been too easy, right? Sadly, that is true. When I entered the rest of the results into my Excel-table, I got these results:
Once we go past position 60, the picture starts to look rather frantic. What I think happens here, is this: we all know that Google is trying their best to prevent any manipulation within the SERPs. They are using a multitude of methods to achieve this, the well-known “filters” being one of them. If a site gets flagged as showing too much negative behavior – regardless of how they get the negative attention – they are pushed back in the rankings. A popular penalty is the repositioning of a result beyond position 60, which is also called a “penalty +60”. And that is exactly what seems to have happened here. Thanks to some kind of misbehavior, those pages have managed to trigger one or more filters, which moved them to the back of the rankings. A while ago, I wrote something about how Google detects changes from the index 'norm', titled “is this still normal?” and maybe that is exactly what is happening here.
So what conclusions can we draw from this test? For one, the keyword-density is definitely a ranking-factor. More seems to be better and it seems that, at this moment, there is not a (sensible) upwards limit. Thought the higher the keyword-density gets, the more likely it is, that the site will get caught in a filter, which will cause it to be passed to the back – the test shows that a value between 1,0% and 4,0% is a secure parameter and I would honestly be quite surprised if you could sensibly top that percentage, while still keeping the text legible.
PageRank-update 06/2009
Google is a Diva. It has not been a month since the last update but tonight, Google has already pushed new PageRank-values onto their datacenters. It seems kind of as though they are trying to get back into their “old” monthly rhythm. Those new values should have already arrived at most of the datacenters and should be visible through the toolbar.This time, the are somewhat fewer changes than with the last update – which should not come as much of a surprise, if we remember that the time interval between the last two updates was quite a bit shorter than to the one before that. Some of the changes for popular domains are these: adobe.com lost their PR10 for their startpage while domains like zdf.de or ard.de both gained one point and went from a PR7 to PR8. A surprise is the strong decline of Immowelt.de – in their case, they (once again) went from a PR7 down to PR4.
Noodp not through x-robots-header
Usually, this subject would not be worth its own blogpost but it ended up being too long for the few characters allotted by Twitter. A while ago, the large searchengines (reads: „Google“) have opened the possibility of sending crawler-commands not only through a meta-tag but also through the HTTP-header. I wrote something on this matter here. In principle, this works wonderfully well and I like this feature and use it often.Though today, I noticed that the implementation seems not to be done too well: while commands like “noarchive” are implemented without a problem, Google is ignoring a “noodp”, which is supposed to make sure, that the snippet is generated from the sites' content and not from its Dmoz-text. We can also see this nicely for this blog, where first, we have the HTTP-header with the x-robots-instruction:
and then we have the actual snippet, that is shown for this blog in the Google-SERPs, which includes the Dmoz-text:
Mistake or deliberation? I, for one, cannot come up with a plausible reason for the second possibility.
PageRank Sculpting
Seeing how Matt Cutts explained his idea of PageRank sculpting a little more in-depth in a blogpost, it is time to summarize the discussion on this topic and draw up some conclusions.About two years ago, the idea of PageRank sculpting, which means the purposeful arrangement of PageRank flow within a page, came into existence – back then, I wrote something about it in my post “Controlling the internal linking with nofollow”. The basic idea is to take the available PageRank (which will be used synonymously with “linkjuice” from here on out) and direct it to the important parts of the Website, while keeping it out of things like the legal notice. This was based on the theory, that Google splits the PageRank equally along the links on a page. This would mean, that the less links you have, the more PageRank those links would get. Seeing how the removal of links is usually not very pretty from a usability point-of-view, you would use the nofollow-attribute for this instead.
Change of rules
Through their mouthpiece Matt Cutts, Google now announced, that the rules have changed: While, in the past, the PageRank was split between all the links that were not marked nofollow, the blogposting now announced, that, for about a year, linkjuice is split between all the links on a page – while the nofollow-links are still not flowing any PageRank, making them useless. Here a visualization of the status, before and after:
While before, the nofollow-link was not counted towards the PageRank distribution, which meant that the flowed PageRank was divided by four, now we get a different picture: now, even the nofollow-link gets his calculated amount, though, thanks to the nofollow-attribute, that PageRank will not be allocated but will just “disappear”.
Implications and consequences
So what are the consequences of this change? Those pages, that “moderately” used PageRank sculpting to keep sites like the legal-notice and shopping-carts or similar pages from getting PageRank, don't have to make any changes – for them it still makes no sense to hoard PageRank.
Those projects that have (massively) used nofollow-links to cure deficits in the structure of their site will have to rethink their method: for these sites, there will be no way to avoid a fundamental change as to control the linkjuice through the sites structure rather than through omitting certain links (Jens wrote some wise words on this topic). In context, I am not much of a friend of trying to find ways to hide links through JavaScript or Flash: Google has changed the rules and trying to find a loophole will not do you much good because, eventually, it will be found-out and closed.
One thing that I consider to be (very) problematic, is the way this change will affect if and how external links are permitted. While, until now, it was the case, that the worth of a link, that I posted in a blogposting, did not change with the number of comments that post got. It seems that now something else is the case: the posted link will loose some of its worth with every comment (=nofollow-link). As far as I am concerned, I don't much care in this case but I am sure that, in the future, we will get a number of blogs, forums and similar sites, that will limit the external linking they allow.
Is Google increasing the size of their (primary) index?
Usually, the days in which searchengines tried to impress users with the size of its index should be over: about a year ago, Google announced, that their crawler had discovered the one trillionth website on the Internet. Since then, it has been rather quiet on the index-size-front and even the usual indicators have not shown any significant changes.For the last few days though, one of these indicators is acting up: for these last few days, a “site-query” will return a noticeably increased amount of results for a number of domains and directories when compared to the same query a month prior: if the numbers for Amazon go up by 164%, those for Spiegel.de increase by 157% and those for faz.net skyrocket by a whopping 452%, it is rather unlikely that those numbers are purely coincidental.
At the moment, there are two possible explanations for this, as far as I am concerned: on the one hand, Google may have changed either the way in which they “estimate” the score for site-queries or what they base that score on. Though I think this to be rather unlikely, seeing how the change did not happen all at once but went on steadily throughout the last month.
On the other hand, I think it's plausible to assume, that Google though it was, once again, time to increase the size of their index. Some of you might remember, how, quite a while ago, Google had two outward indexes: the “normal” index for current and often-accessed content and the so-called “supplemental index” for all the other stuff. While this clear (outwards) division has ceased to exist after an infrastructure-update, I would be quite surprised, if internally, Google was not still working with different classifications. If we assume, that the site-query will (mainly?) use results from the first index, then the increased amount of returned pages could be an indication, that Google is enlarging that part of the index and is either accommodating more pages or is “hefting” up content from the other parts of its index.
IndexWatch 05/2009
Another month has passed and here are the current IndexWatch-numbers. Just as a short reminder: these charts are based on our Visibilityindex and show the winners and losers of the (organic) Googleindex. As usual, we start with the winners:winners
| # | Domain | Change | |
|---|---|---|---|
| 1 | linguee.de | +1.000% | |
| 2 | bizrate.de | +1.000% | |
| 3 | reise.de | +1.000% | |
| 4 | tiscover.com | +564% | |
| 5 | servicepoint.de | +454% | |
| 6 | fondsdiscount.de | +196% | |
| 7 | touristikpresse.net | +192% | |
| 8 | euroeyes.de | +178% | |
| 9 | wetter.info | +146% | |
| 10 | misterspex.de | +145% | |
| 11 | kinkaa.de | +135% | |
| 12 | sprueche-universum.de | +124% | |
| 13 | hot-maps.de | +109% | |
| 14 | kaeuferportal.de | +107% | |
| 15 | bab.la | +103% | |
It seems that Reise.de has recuperated from a penalty, which was probably put in place out of nowhere, due to the too rapid link- and ranking-growth in the weeks prior. After the heavy loss in July of 2008, Hot-Maps gained some ground once again, though this subject is annoying enough due to the Universal-Search-integration of Google Maps. Otherwise, it seems to me, that the reason for most of the other winners, can be found in (hopefully) solid SEO-work; we will see. Now we move on to last month losers:
losers
| # | Domain | Change | |
|---|---|---|---|
| 1 | kisseo.de | -100% | |
| 2 | abendkleid.net | -99% | |
| 3 | technikmonster.de | -99% | |
| 4 | cesar.de | -97% | |
| 5 | mallorca-today.de | -95% | |
| 6 | unsertag.de | -92% | |
| 7 | wohnmobilforum.de | -92% | |
| 8 | autoaid.de | -91% | |
| 9 | votello.de | -88% | |
| 10 | olimar.de | -88% | |
| 11 | mofapower.de | -88% | |
| 12 | folkd.com | -86% | |
| 13 | stern-tours.de | -84% | |
| 14 | isik.de | -84% | |
| 15 | infopirat.com | -82% | |
It seems that, as far as the losers are concerned, Google picked out the one or other not completely organically grown link-network: in any case, the disproportional amount of travel-sites points in that direction. Many domains that already harbored a penalty in the past, have been hit once again.
SEO-regulars' table Bonn
Most of the conventions and meets before the summer-break are through, which means its about time to hold another SEO-regulars' table in Bonn. I picked out Thursday the 18th of June for the date. Everyone who is interested in the subject matter and who will not blatantly misbehave is welcome to attend :-) Please use the sign-up if you are interested. I will send out the exact details a few days beforehand by e-mail.Bing – first impressions
Its official: Microsoft will not go with “Kumo” but with “Bing” in their renewed attempt to snatch away some of Google's giant market share. In my eyes, the name might not be the perfect fit but it's definitely a better pick than the present “Live Search”.Just to be clear: Bing is no “Google Killer”. With Bing, Microsoft put together a searchengine with a solid quality of results, which caught up to Google in large parts but which can only best the market leader in a few areas. I was able to take a look at Bing before the launch and I want to show a few features and new ideas with the help of some screenshots. We start with the homepage; it seems that they are changing the “background image” on a daily basis to add some variety:
Sadly, Microsoft repeats the same mistake they made years ago: at the moment, Bing is only fully functional in the USA. Other, not-so-minor markets like Germany will have to make do with a noticeably reduced version, which reminds you more of the current Live-search. They are working on fixing this at full speed but they are not going to make it in time for the launch date. The following screenshots are therefore from the US version of Bing, coupled with the hopes, that we will also get these features in Europe in the foreseeable future.
In the search for “Bonn” (here the screenshot of the full page, 325kb), we will first notice, how the results are arranged into two columns. To the left of the actual results, we are shown further perimeters – if, like for this example, some are found that fit exceptionally well, they will be sorted like a navigational bar (Images, Map, Weather, etc.), otherwise we will only be shown the “Related Searches”. Below those, you can see the last few queries you typed in, so you are able to quickly jump back to previous searches. It is commendable, that you are able to disable the storing of this data right there; with Google, you have to sift through numerous menus to turn off the annoying Webhistory.
The actual SERPs, framed by specifically marked advertisements, start out with five fitting pictures from the imagesearch. Right there, you are able to narrow down your search further, if you are only looking for large pictures or those in black and white.
After the images, there are five results from the “normal” index. Microsoft came up with something new for Wikipedia-results: instead of showing the classical cache-version, they are now showing a link to the Wikipedia-content in the Bing-layout, which is marked as “Enhanced View”; the same is also linked in the navigation through “Reference”. When you move the mouse over one of the results, you will get a continuation of the SERPs textsnippet to the right of it. This is a nice idea as far as searchengines are concerned, though siteoperators will now have to come up with a way to distribute relevant content, so that those searching will still have a reason to visit the site, instead of just getting the answers they want right in the SERPs.
Below those results, there are three results each from the searches that have already been highlighted in the navigation: Bonn Maps, Bonn Weather, Bonn Airport, Bonn Hotels and Bonn History. From the user point-of-view, I like this because it adds quite some variety to the results; from a SEO-point-of-view this is naturally not so nice, seeing how this means one thing more than ever: only positions 1-3 count. Other features are, in parts, closely related to the big role model: the local results for “Bonn Hotels” should be familiar to Google-users and the integrated Weather-information is also not something completely unknown. The shopping-area, which makes sense not to be shown for this search, is nicely enriched though data from Ciao, which was bought by Microsoft last year: prices, testimonies, comparisons and in the USA also the popular cashbacks.
All in all, this is a noticeable advancement from the current state of Microsoft's searchengine and if these nice features, which we can only marvel at in the USA, at the moment, find their way to our side of the Atlantic, then I can imagine that this, in combination with the large rumored advertisement budget that Microsoft is allocating for Bing, will cause their marketshare to grow by quite a bit.