Why does the amount of indexed pages fluctuate so much?

From:

SISTRIX Content Team

Editor: Steve Paine

Modified: 22.01.2021

Google-Index, Google-Bot and the Crawling Process

• What is the Google Everflux?
• Knowledge Graph - How Google Understands Things and How Often Is It Used?
• What is a User-Agent?
• What is the Mobile First Index?
• Google Search and Entities
• Our web site is no longer in the index - have we lost our rankings?
• Crawl Budget: What does this mean?
• Crawling and Indexing for extensive websites
• Rich Snippets: What Are the Advantages?
• What is the X-Robots-Tag?
• Robots meta tag vs. robots.txt: what are the main differences?
• Web Crawlers: How do They Work?
• Can the Google-Bot fill out and crawl forms?
• What is Google Search Console and How To Get Started
• These are the CTR's For Various Types of Google Search Result
• Google SERP Features: Result Types in the Search Results
• Why does the amount of indexed pages fluctuate so much?
• Why does a blocked, noindex URL show up in the search results?
• Why does a Google search with quotes sometimes deliver more results than the same search without it?
• Find out how many pages of a domain are indexed by Google
• How can I prevent the Google-Bot from crawling my website?
• How can I remove a URL on my website from the Google Index?
• Why am I getting different values for indexed pages in the Google search, the GSC and SISTRIX?
• The consequences of negative user-signals on Google's rankings
• How can I quickly get a new page into Google's index?
• Back to overview

The amount of indexed pages in the SISTRIX Toolbox

Within the SISTRIX Toolbox, we monitor the amount of indexed pages daily, but will only create a new data-point in the chart if there is a change in the average number of indexed pages. Because of that, there may be multiple data points daily, weekly, or even monthly. A Google site:-Request is simply a rough estimate.

• You will find additional information in our article: Why am I getting different values for indexed pages in the Google search, the GSC and SISTRIX?

The source of these fluctuation and possible evaluations

Large fluctuations in the indexed pages usually point to an indexing- and/or crawling problem. This can happen because of, for example, Duplicate Content (DC), 302 Redirects instead of 301 Redirects, Searches in Searches, Sessions IDs as well as a strong ranking fluctuation and so on.

Duplicate Content is one of the most frequent reasons. In online shops, for example, you can often open the detailed product pages directly (and Google will often also index them), even without going through the category and product pages.

• http://www.onlineshop-domain.com/category/productpage/productdetailpage
• http://www.onlineshop-domain.com/productdetailpage

Here, the evaluations and alerts of the SISTRIX Optimizer module, as well as the Google Search Console (indexing status) can also offer additional help. It is always recommended to include both sources of data into your evaluation.

This method offers additional steps of how to analyse this more in depth:

• Use your web analytics solution to show all the subpages that had at least one visitor from Google (organic) in the last 30 days and see if the number of these pages has really changed dramatically.
• Alternatively, you can check the history of the number of keywords found for the domain within SISTRIX. Did something change? If so, find out what changed and why. If not, hope that there will not be a reason in the future.