250 million New Keywords for the Toolbox

In the last week, we started the process of enabling, across 27 countries, about 250 million new keywords. The new keywords will have an impact on some of the metrics across countries so here's an overview of the improvements that will take place.

The new keywords that will be added to our Toolbox will be mainly focused on long-tail searches. These search terms have relatively low search volume but, when considered in big quantities, can generate significant traffic. Since the lower positions are of even lower important on their own, we’re only recording the first page of search results for each of the new keywords. Only the first 10 organic search results will be collected but we’ll continue to pull-in the Ads and other SERPs features as we have before. As always, the Visibility Index continues to be based on the core set of keywords. The new keywords don’t have any impact there.

Google has migrated much of its search index to the Mobile Crawler this year and in most countries the majority of searches are now happening on mobile. With this in mind we’ll be crawling only the mobile search results. The mobile keyword database has therefore become the biggest and most important keyword database for nearly every country. In England, Germany and Spain that’s not currently the case but over the next few weeks we’ll be switching to a ‘Mobile First’ system in these countries too.

At the time of writing, the integration of all the new keywords is not complete but you can already see changes in the total number of keywords ranking. Previously, theguardian.com had ranked for just over 3 million keywords. Today there are 3.8 million ranking keywords available to view in the toolbox.

When the implementation is complete we’ll have, for example, an additional 10 million keywords for the Google UK search index, an additional 20 million for US and 10 million for the NL index. Don’t forget that we’ve recently added Ireland, Australia and Canada to the list of ‘EN’ countries too. While we’re talking about new countries, we should mention that Mexico and Russia also came on-line this week.

The extended keywords will be updated a minimum of once per month. We’ll also continue the process of adding new keywords, and removing old, irrelevant keywords.

It’s important to remember that these ‘extended’ keyword sets have no impact on the historical data-sets. The Visibility Index and graphical representations for the number of ranking keywords and URLs stay the same and continue to be usable as comparison data across the complete range of historical records.