With the rise of AI Search, a new category of bots has emerged: AI userbots. User agents such as ChatGPT-User or Perplexity-User access a website at the exact moment an AI is compiling information for a specific user query.
Within the SEO industry, these visits are increasingly being treated as a new currency. The logic sounds compelling: if an AI userbot crawls my site, my content feeds into the generated response. More bot visits mean more AI visibility.
This interpretation is understandable, but it falls short. When analysing AI userbot data, there are four significant pitfalls that make the picture considerably more complex.
1. The Most Widely Used Systems Don’t Use Userbots at All
Google AI Overviews and AI Mode are by far the most widely used AI search systems on the market. They operate primarily on the existing search index. When responding to a query, a live visit from a dedicated userbot generally does not take place. The entire process remains invisible in server log files.
This means: anyone measuring their AI search performance exclusively through userbot visits is completely overlooking the largest channel by far.
2. Access Does Not Equal Inclusion
As part of RAG (Retrieval Augmented Generation), an LLM can retrieve numerous sources in parallel to increase the information density for a response. However, the fact that a page was crawled in this process does not necessarily mean it will ultimately be cited as a source or included in the result. The model filters and re-ranks in a final step.
A bot visit in the logs is therefore, at best, a signal that a page was considered as a candidate. Whether it made it into the final answer cannot be inferred from this alone.
3. Validation Rather Than Generation
The key decisions about the content of a response are made within the foundation model itself. In many cases, userbots serve only to back up or validate decisions already made by the model with current facts.
The bot visit is therefore often a downstream verification mechanism, not a primary driver of the response. The causality works differently than the log data suggests: it is not the visit to the website that leads to the answer, but rather the planned answer that leads to the visit to the website.
4. Caching Distorts the Data
To reduce latency and cut costs, AI search systems rely on caching mechanisms. A single bot visit can form the basis for thousands of identical user queries without any further log entries being generated.
This distorts the data in both directions: pages with few bot visits may still appear heavily in AI responses. Pages with many bot visits may have only been consulted for a single query that was subsequently cached.
A Historical Parallel: The Meta-Search Engine Comparison
AI userbots are reminiscent of the early days of meta-search engines. These queried multiple search engines in parallel following a user request and laboriously aggregated the results in real time. The principle worked, but it was inefficient and quickly became obsolete once Google had built a sufficiently comprehensive and up-to-date index that information could be delivered directly from a single central source.
Conclusion
AI userbot visits are a data point, but not a reliable indicator of AI visibility. They represent only a fraction of the actual AI search landscape, say nothing about whether content is genuinely included in responses, and are systematically distorted by caching effects.
My prediction: AI Search will develop in a similar way. Live access via userbot will become a special case reserved for volatile, time-critical data — such as “Is this flight still available in booking class XY right now?” For the vast majority of information, the model will draw directly on its trained knowledge or a highly efficient, pre-crawled index.
