INSIGHTS / B2B GEO RESEARCH

Is llms.txt Worth Prioritizing for B2B AI Search?

For most B2B marketing teams, llms.txt should be a low-priority task unless a specific tool or customer workflow needs it. The file may help an agent find useful material, but that is a different goal from winning a citation in an AI search answer.

6 min read

TL;DR

  • Treat llms.txt as an optional resource for a defined tool or agent, rather than a default AI search project.
  • Google says these files do not help or harm visibility in Google Search, including its generative AI features.
  • Request logs and citation studies measure different outcomes. Neither a file's presence nor a bot fetch proves that an answer used it.
  • Fix access to useful pages and gaps in buyer evidence first. Give any file experiment a clear owner and success check.

Ask what the file is meant to change

Before paying for a file or treating its absence as an urgent audit failure, ask what it is meant to change. A document can be valid, get fetched and still have no demonstrated effect on the outcome you care about.

What is the file supposed to do?

The llms.txt proposal, updated in August 2026, describes a Markdown guide that gives agents context and links to useful material. It can cover a whole site or a section such as software docs. It serves a different purpose from robots.txt, which communicates access rules to automated tools.

That distinction matters for a B2B site. A guide to your service pages cannot fix a broken service page. It cannot turn a short claim into evidence or replace the actual access controls for private material. Judge it by the task it helps a real consumer complete.

For Google Search specifically, the guidance is direct: llms.txt and similar special files are not used to improve search visibility. Google may crawl or index such a file as ordinary content. That is not evidence that it receives special treatment. Other agents and services must be assessed separately.

What request logs reveal

Ahrefs examined 137,210 domains using its analytics tools in May 2026. About 28% published an llms.txt file. Among domains with a valid file, 97% received no requests to it that month. Its sample is more technical and SEO-aware than the wider web.

What request logs reveal
File request status in May 2026Share of domains with a valid file
No requests97%
At least one request3%

Source: Ahrefs, June 2026. The base is the roughly 38,000 domains with a valid file, not all 137,210 domains or all websites.

The study also classified 96% of requests that did reach the files as bot traffic. That percentage counts requests, not files. It does not mean that 96% of files were read by AI search tools; the bot category includes other kinds of software.

These findings weaken the case for treating a file as an urgent route to AI visibility. They do not prove that no agent can use one. A month of request logs also cannot tell us how a later version of the proposal performs in a specific workflow.

Figure 1. Requests to valid llms.txt files in one study% of domains with a valid file in the May 2026 study
No requests97%
At least one request3%
050100

About 38,000 domains with valid files in an analytics customer sample. A request is not evidence that an answer used the file. Ahrefs, June 2026.

Presence, requests and citations are different checks

SE Ranking's report found llms.txt on 10.13% of nearly 300,000 domains. Its domain-level analysis reported no useful relationship between file presence and citation frequency in the tested data. It used statistical and predictive methods, not a randomized rollout of files.

Presence, requests and citations are different checks
EvidenceResult and measured objectWhat it supportsWhat remains open
Ahrefs, June 202697% of valid files had no requests in May; logs from an analytics customer sampleCheck whether a proposed resource is being fetched at allWhether a fetch changes an answer or citation
SE Ranking, October 202610.13% adoption; no useful citation relationship reported in its domain sampleFile presence is not an established citation tactic in that analysisA causal effect for a defined tool, site or file version

The studies do not form a before-and-after trend. They cover different site populations and outcomes. Both vendors sell search tools, and neither can establish how every agent will behave. Read them as a reason to demand a clear use case and evidence, rather than a universal verdict on all machine-readable guides.

Where should the next piece of work go?

Use this editorial decision table to separate a useful delivery from an unsupported promise. The checks describe work you can inspect, not guaranteed ranking factors.

Where should the next piece of work go?
Proposed taskUseful deliverableCompletion checkOutcome to measure separately
Create llms.txt for a known agentA maintained guide to the material that agent needsThe named workflow can find and use the right materialSearch citations or business results
Fix a public service pageA working page with readable scope and proofThe intended visitor can reach and understand itWhether search systems index or cite it
Review crawler accessA record of relevant rules and actual responsesThe intended crawler can fetch the intended public URLWhether the page is selected as a source
Improve a weak claimA supported answer with its conditionsA reader can trace the evidenceWhether it changes qualified demand

Consider a fictional industrial supplier. Its audit flags a missing llms.txt file, but its main service URL returns an error and its scope document is out of date. Generating the file would leave both buyer problems in place.

The useful first job is to restore the public page and confirm the scope with the service owner. If the team also has an agent-based customer help workflow, it can test a guide for that task afterwards. The acceptance check should name the question, the correct source and the expected answer. This example shows a priority decision, not a measured client result.

llms.txt and AI Search: Common Questions

Should we delete a file that already exists?

There is no need to delete a useful, maintained resource solely because it is not a Google Search tactic. Check who uses it and what it costs to keep accurate. Google's guidance distinguishes search visibility from uses by other services.

Can llms.txt block a crawler?

Do not rely on it as an access-control mechanism. The proposal distinguishes it from robots.txt. Public crawler rules and protection of private content require their own checks.

Is a bot request a successful test?

It proves only that a request reached the recorded URL under those conditions. To test use, inspect whether the intended workflow found the right source and produced the right answer. A search citation needs a separate observation.

What evidence should an implementation proposal include?

Ask for the intended consumer, the material it needs, the maintenance owner and a concrete test. If the promised result is more AI citations, ask how that change will be distinguished from other work or normal answer variation.

Review the proposed priority

If an audit makes llms.txt urgent without explaining the buyer or agent task, bring the finding and the affected pages to SemShift. We can discuss which work removes a real barrier and which claims still need a test.