Data ‘scraping’ is core to Google’s crawling process so if you block that, you’ll lose visibility. Following measures taken by Elon Musk are evidence that the rate-limtiing of access to twitter.com resulted in it being removed from significant keyword rankings. Yankees, reddit, roblox and the one that no self-respecting media company wants to lose – britney spears.
Based on live data used at the time, the US market was showing a 32% loss in visibility. Lost rankings included extremely significant search terms for which Twitter was a key part of the results for searchers.
The keyword “yankees” has an estimated monthly click volume, in the USA, of 1.8 million clicks. The lost position three was worth 168K clicks per month and has been given to espn.com. Disney and Hearst will be happy about that.
The data exposed an initial 2200 keywords that were lost from page 1 search results – and that’s with preliminary data, just from Google US. Data on the following day took that into the tens of thousands of lost page-1 rankings.
By limiting access he turned off millions of clicks to the platform.
Similar losses were seen across key countries. Here are some VI figures from different countries. (14:41 BST 3rd July.)
- UK – 28% loss
- FR – 17% loss
- IT – 26% loss
- DE – 25% loss
- ES – 14% loss
Significant keywords lost from the top 10 in the USA include yankees, deviantart, ralphs, metropcs, roblox, reddit, indeed, white house, britney spears, steak n shakered, kanye west, gm, lakers, pink, krispy kreme, colts, gmc, tennis, compass, lady, patriots, transformers.
4th July Update
Losses continued into the 4th of July. One big example is “trump” whose twitter account was lost from the top 10.
The top-25 of lost keywords looked like this (2nd – 4th July, USA only.) In total, that’s over 12 million monthly clicks lost. We registered 59,262 keywords that lost top 10 rankings. (Live data: 11:00 CEST.)
Twitter Competitors have gained
When a very visible domain gets removed from SERPs like this, everything else shuffles up a place, at least initially. The following (all.com) domains have gained in US search visibility in the last 48 hours.
- theguardian
- cbssports
- usnews
- rottentomatoes
- spotify
- espn
There are no big surprises there but they’re mostly close competitors to Twitter. Major gains are currently in the 1-3% points range.
Twitter’s return to SERPs – 6th July
The graphics show a return to previous highs although it’s going to take a few more days before Google returns the rankings. It follows the path of previous examples we’ve seen where Google has been patient with the provider and allowed it to return once crawling re-started and the URLs were present as before.
