TL;DR
- An application programming interface (API) tests a named technical setup. It does not automatically reproduce the answer a buyer sees on a website.
- Two independent studies found gaps between external tests and consumer-facing sources. The size depends on the task, setup and measure.
- Keep retrieved pages, displayed citations, domains and exact URLs in separate fields. They answer different questions.
- Use API checks for a defined monitoring task, and validate important buyer questions on the surface you intend to report.
Check what the report actually describes
Before you buy or act on an AI visibility report, check its scope. Which product was tested? Was search enabled? Which source layer was counted? A precise label makes a report more useful than a broad claim of “ChatGPT visibility.”
How Much Do Web and API Sources Overlap?
A September 2026 product-recommendation audit found mean domain overlap of 12.0% for ChatGPT and 14.8% for Gemini between each web interface and its API condition. Exact-page overlap was 4.8% and 11.9%, respectively. The team tested 117 physical-product queries. Each was repeated three times across five setups. The study analyzed 1,536 responses.
| Paired conditions | Domain overlap | Exact-page overlap |
|---|---|---|
| ChatGPT web and API | 12.0% | 4.8% |
| Gemini web and API | 14.8% | 11.9% |
Source: Auditing AI-Generated Product Recommendations, September 2026, RQ3. Mean Jaccard overlap: sources shared by both sets divided by their union. Logged-out web conditions used a Netherlands location. These are task-specific observations, not current B2B accuracy rates.
The study kept paired requests close in time. Model settings and other parts of the product could still differ. The study could not show that API access alone caused the gap. Check whether the test method matches the product the report claims to describe. The findings do not mean that all API-based services lack value.
The two levels also matter. A domain match may point to different pages with different facts. If the task is to check a product limit, the exact cited page matters more than a match on the company name or hostname.
September 2026 audit of physical-product questions. Logged-out web conditions used a Netherlands location. The two marks compare granularity, not dates; these are not B2B accuracy rates. Product-recommendation audit, 2026.
A Broader Proxy Can Still Miss the Cited Page
An independent October 2026 study by Jiang and colleagues compared ChatGPT Web with Google Search and the OpenAI Responses API. The combined external candidate set recovered 28.3% and 26.2% of cited URLs for its 5.3 and 5.5 Instant settings. Hostname recovery was 50.3% and 48.7%. The researchers collected those settings on the same 80 questions, with 20 runs per question, during March–July 2026.
Here, the study's cited set includes selected reference URLs and their attached supporting pages. That source layer differs from a list of final inline links alone.
| Direct research | Scope and measure | Actual finding | What a report should disclose |
|---|---|---|---|
| Product-recommendation audit, September 2026 | Physical-product queries; paired web/API source sets; mean Jaccard overlap | Domains: ChatGPT 12.0%, Gemini 14.8% | Product surface, model setup and whether pages or domains were compared |
| Jiang and colleagues, October 2026 | 80 mixed-intent Chinese/English questions; external-candidate recovery of Web citations | URL recovery 28.3%/26.2%; hostname recovery 50.3%/48.7% | The candidate set and the exact citation set it was tested against |
Do not compare the percentages as tool rankings. Jaccard overlap measures how much two sets share relative to their combined size. Recovery asks how much of a reference set a proxy found. A large candidate list can improve recovery without making its final answer a close match to the web answer.
Both studies show why the label on a metric matters. They do not supply a correction factor for your own B2B score. Their questions, languages, locations and model settings are limited. The services can change too. A useful monitoring plan names those conditions and checks the scope again when the setup changes.
Ask for a Monitoring Record You Can Inspect
Use this table to define the claim before choosing a tool or reviewing an agency report. These are scope checks, not a promise that two products will agree.
| Field | What a useful record states | Why it matters |
|---|---|---|
| Product surface | Web interface or named API endpoint and model | Prevents an API result being described as a web observation |
| Search and context | Tool availability, mode, location, language and conversation history | Shows the conditions under which the answer was produced |
| Source layer | Retrieved candidates, displayed links or final inline citations | Prevents a fetched page being counted as a visible citation |
| Match unit | Brand name, hostname or exact URL | Prevents a broad domain match masking the wrong source page |
| Evidence and limits | Prompt, answer, timestamp, repeat plan and unresolved gaps | Lets another reviewer inspect what the reported number means |
Worked example — fictional software firm: an API response returns a candidate list containing the firm's guide, but the final answer cites a rival. The dashboard counts the guide as “visible in ChatGPT.” The analyst checks the raw record and changes the field to “present in API retrieval candidates.” She logs final citations separately and checks the same buyer question on the intended web surface. The report now makes a claim the saved evidence can support.
That distinction is useful even if the team keeps the API test. You can compare results from the same named setup. Keep doing repeat checks and use clear labels. For the wider sampling plan, see how to measure AI visibility. For deciding whether a visible link supports a claim, use our citation support guide.
API and Web AI Monitoring: Common Questions
Should we stop using API monitoring?
No. Use it for the task it actually measures. It can support a defined test setup and a record you can inspect. If the report claims to describe a web product, ask how that claim was checked against the web experience.
Can a domain match stand in for an exact URL match?
Only when the task is truly about domain presence. If you need to check which specification, price or case study was cited, save the exact URL. Report the match level so readers can judge what it establishes.
Does enabling web search make an API identical to the website?
No. It enables a search capability within that API setup. It does not show that the web product uses the same model settings, search process or source display. Treat any claimed equivalence as something to test.
Can these studies tell us which monitoring tool is best?
No. They test research conditions, not every commercial service or your buyer questions. Compare tools on disclosed coverage, saved evidence, repeatability and fit with the decision you need to make. A larger citation count is not enough on its own.
Review the scope behind the score
If a visibility report is prompting page changes but does not say which product or source layer it measured, bring a sample export and the buyer questions to SemShift's AI visibility audit. We can discuss the reporting gaps and the checks needed before those numbers guide content work.