INSIGHTS / B2B GEO RESEARCH

Do More AI Bot Visits Mean More AI Visibility?

No. A rise in AI bot visits is not enough to claim better AI visibility. A bot may fetch a page for training, search or a user's request. The access log records the request and response; it does not show what an answer later said about your business.

6 min read

TL;DR

  • More AI bot requests show more recorded activity. They do not, on their own, show more citations, buyers or revenue.
  • Training crawls, search crawls and user-triggered page visits serve different jobs. Sort them by purpose first.
  • Research on crawl volume and purpose helps explain the gap, but network-wide numbers are not a benchmark for your site.
  • Use logs to find access failures. Use saved answers to check citations and separate analytics to check human visits.

Separate a request from an answer

Ask what changed and what the log can help fix. A blocked search crawler and a busy training crawler can produce very different work priorities, even when both appear under an “AI traffic” heading.

Why One AI Traffic Total Can Mislead

Cloudflare's August 2025 analysis classified 80% of AI crawling as training, 18% as search and 2% as user actions over its reported twelve-month period. Cloudflare used bot owners' statements and industry sources to group the requests. These are past network results. They do not show the current mix on your site.

Why One AI Traffic Total Can Mislead
Classified purposeShare of AI crawling
Training80%
Search18%
User actions2%

Source: Cloudflare, August 29, 2025, “Crawling by purpose.” Twelve-month view reported in the article; not the separate six-month breakdown.

The mix explains why more requests need not mean more buyers. Requests grouped under one label can serve tasks far removed from a buyer's search. The chart also does not mean that search crawls turn into clicks at a fixed rate. Each stage needs its own evidence.

Figure 1. AI Crawling by Purpose in Cloudflare's ReportShare of AI crawling (%) · twelve-month view reported August 2025
Training80%
Search18%
User actions2%
050100

Historical network data classified using operator disclosures and industry sources. This is not the current mix on your site or the separate six-month breakdown. Cloudflare, August 2025.

Big Crawl Counts Measure Activity, Not Results

In Vercel and MERJ's December 2024 study, GPTBot made 569 million requests across Vercel's network in the prior month. The report also recorded 370 million for Claude and 24.4 million for PerplexityBot. Those volumes describe requests, not unique buyers or citations. The study tracked Vercel's network and a few sites. It is old data. It cannot set current rules for how every bot works.

Big Crawl Counts Measure Activity, Not Results
Direct researchObservationActual resultLimit for a B2B report
Cloudflare, August 2025AI crawl activity grouped by purposeTraining 80%, search 18%, user actions 2%Purpose shares do not establish answer use or referral rates
Vercel and MERJ, December 2024Requests across Vercel's network in the preceding monthGPTBot 569M; Claude 370M; PerplexityBot 24.4MFetch counts do not identify unique people or useful citations

The studies add two views. One shows why the traffic exists. The other shows how large request counts can be. Neither tells you how many crawls turn into cited answers. Do not combine their figures into a ranking of which platform sends the best leads.

Current bot roles also matter. OpenAI's crawler documentation distinguishes OAI-SearchBot for search, GPTBot for content that may be used in model training, and ChatGPT-User for certain user-triggered actions. Search and training controls are separate, and OpenAI notes that a crawl may serve both uses when both are allowed. So a bot label alone does not prove how the page was used.

Turn the Log into a Useful Decision

Before you explain a spike, check who sent the request. Then check the URL, status and time window. Use the bot owner's current guide to check its identity. A name in a user-agent string can be faked. Keep retries, errors and repeated fetches visible rather than counting each as a new opportunity.

Use this table to choose the next check. These are our suggestions. They are not results from the network studies.

Turn the Log into a Useful Decision
Observed signalWhat it can supportWhat it cannot establishNext check
Verified search bot receives an access errorA possible access problem for that requestThat the page would otherwise be citedInspect the error and the affected content with the site owner
Training bot requests increaseMore recorded fetch activity by that botMore search appearances or buyersReview purpose and operational cost separately
User-triggered fetch receives a full pageThe tool could access the response in that requestA human click, a recommendation or a saleInspect the saved answer if available
A public answer links to the pageA citation in that recorded answerReliable support for every nearby claimCheck the cited passage and preserve the prompt and date

Worked example — fictional engineering supplier: a report calls a rise in GPTBot requests an “AI visibility gain.” The analyst separates bot types and finds no saved buyer answers in the report. She changes the claim to “training crawler activity increased.” In a separate check, a search bot received an error on a capability page. The site owner investigates that error. The team has found a real issue. It has not claimed a gain it cannot show.

Our AI search citation diagnostic covers the wider source check. Logs show whether a request got through. Keep the saved answer as well. The buyer's question tells you whether that answer matters.

AI Bot Traffic: Common Questions

Should all AI bots be allowed?

That is a choice for the site owner. It depends on what the bot does and what your business needs. A crawl report alone cannot settle it. Review current operator documentation with the site owner before changing access rules, and keep search access separate from training choices.

Does a successful response prove that the bot read the whole page?

No. A successful status code shows that a response was served. Inspect what the response contained and, where possible, what the tool actually used. A fetched shell with little text is different from a full product specification.

Can we divide referrals by crawls to get an AI conversion rate?

Not a buyer conversion rate. Those counts may involve different bots, purposes, pages and people. A crawl-to-referral ratio can describe a defined traffic relationship, but it does not track one person from a crawl to a sale.

What is a sound success claim after fixing an access issue?

State what you checked: the right bot could fetch the right page in your test. Check later citations and human visits separately. Do not turn an access fix into an unmeasured claim of greater AI reach.

Review the signals behind the dashboard

If a bot-traffic dashboard is driving content decisions without clear buyer-answer evidence, bring the relevant page URLs and a redacted log summary to SemShift's AI visibility audit. We can discuss which access issues and source gaps deserve a focused review.