Register / Login

Is Google increasing the size of their (primary) index?

Usually, the days in which searchengines tried to impress users with the size of its index should be over: about a year ago, Google announced, that their crawler had discovered the one trillionth website on the Internet. Since then, it has been rather quiet on the index-size-front and even the usual indicators have not shown any significant changes.

For the last few days though, one of these indicators is acting up: for these last few days, a “site-query” will return a noticeably increased amount of results for a number of domains and directories when compared to the same query a month prior: if the numbers for Amazon go up by 164%, those for Spiegel.de increase by 157% and those for faz.net skyrocket by a whopping 452%, it is rather unlikely that those numbers are purely coincidental.

At the moment, there are two possible explanations for this, as far as I am concerned: on the one hand, Google may have changed either the way in which they “estimate” the score for site-queries or what they base that score on. Though I think this to be rather unlikely, seeing how the change did not happen all at once but went on steadily throughout the last month.

On the other hand, I think it's plausible to assume, that Google though it was, once again, time to increase the size of their index. Some of you might remember, how, quite a while ago, Google had two outward indexes: the “normal” index for current and often-accessed content and the so-called “supplemental index” for all the other stuff. While this clear (outwards) division has ceased to exist after an infrastructure-update, I would be quite surprised, if internally, Google was not still working with different classifications. If we assume, that the site-query will (mainly?) use results from the first index, then the increased amount of returned pages could be an indication, that Google is enlarging that part of the index and is either accommodating more pages or is “hefting” up content from the other parts of its index.
Johannes Beus - on Mon (06/08/2009) at 23:10 PM

Comments closedComments closed
This posting is older than 30 days and therefore closed for new comments.