Google Says You Don't Need llms.txt. Here's the Catch.
Google says llms.txt is unnecessary. Chrome Lighthouse audits it anyway. We fact-checked five studies on what llms.txt really does for AI visibility in 2026.
Jun 12, 2026•19 min read
Pixelmojo Labs
Research Report / April 2026
State of AI Visibility 2026
Aggregated from 110 real Radar platform audits across 6 industries, scored on 6 AI readiness dimensions. The average score was 48/100. Most businesses are invisible to AI search engines.
110 domains audited6 industries6 audit dimensionsPublished April 12, 2026Data through June 12, 2026
48/100
Average AI Readiness Score across 110 domains
Median: 46.5 | Range: 2–92 | Std Dev: 20.3
Score Distribution
Most domains cluster in the 30-69 range. Only 8% score 70 or above. Only 1 domain has scored an A (90+).
0-29
28 domains (25%)
30-49
34 domains (31%)
50-69
39 domains (35%)
70-89
8 domains (7%)
90-100
1 domains (1%)
A
1
1%
B
8
7%
C
39
35%
D
34
31%
F
28
25%
AI Readiness by Industry
Healthcare leads at 52/100. Services trails at 36/100. The gap between best and worst is 16 points.
| Industry | Avg Score | Domains | Grade Breakdown |
|---|---|---|---|
| Healthcare | 52/100 | 8 | B: 1C: 4D: 2F: 1 |
| SaaS | 49/100 | 12 | B: 1C: 6D: 3F: 2 |
| Enterprise Tech | 44/100 | 11 | B: 1C: 6D: 4F: 2 |
| Retail | 44/100 | 11 | B: 1C: 1D: 8F: 1 |
| Travel | 43/100 | 6 | B: 1C: 1D: 2F: 2 |
| Services | 36/100 | 13 | B: 2C: 4D: 1F: 6 |
Where Domains Score Highest and Lowest
Each domain was evaluated across 6 dimensions. The widest gap is between the best-performing dimension (AI Bot Crawlability at 75/100) and the worst (Answer Engine Optimization at 39/100).
AI Bot Crawlability75/100
AI Readiness Score73/100
Robots.txt Configuration60/100
Schema Markup Quality58/100
llms.txt Implementation57/100
Answer Engine Optimization39/100
Key Findings
56%
of domains scored D or F
These businesses are effectively invisible to AI search engines. Their content cannot be discovered, cited, or recommended by ChatGPT, Perplexity, Claude, or Gemini.
0 domains
scored an A (90+/100)
Not a single domain in our audit achieved excellent AI readiness. Even the highest-scoring domains have significant gaps in their AI visibility infrastructure.
57/100
average llms.txt score
llms.txt is the simplest way to communicate with AI systems, yet most domains either don't have one or have a poorly structured implementation.
16 pts
gap between best and worst industry
Healthcare (52/100) vs. Services (36/100). AI readiness varies dramatically by vertical.
Methodology
Each domain was audited using Radar's AI Visibility Platform across 6 dimensions. Scores are computed independently per dimension and averaged into a unified 0-100 score with a letter grade (A-F).
| Dimension | What It Measures |
|---|---|
| AI Bot Crawlability | Can GPTBot, ClaudeBot, PerplexityBot, and other AI crawlers access the site? Tests 14 AI-specific user agents against the live site. |
| Robots.txt Configuration | Does robots.txt explicitly allow or block AI bots? Checks for AI-specific directives vs. blanket blocks that inadvertently hide the site. |
| llms.txt Implementation | Does the domain publish an llms.txt file? Evaluates structure, completeness, and whether it provides useful context for LLMs. |
| AI Readiness Score | Composite metric combining crawl accessibility, structured data presence, content depth, and LLM-friendly formatting. |
| Schema Markup Quality | Evaluates JSON-LD structured data: Organization, Article, FAQPage, Product schemas. Tests for completeness and correctness. |
| Answer Engine Optimization | Analyzes page structure for AEO signals: answer-first formatting, FAQ sections, table usage, heading hierarchy, and content depth. |
Data collection period: March 2026 – June 2026. 110 unique domains across 169 total audit runs. Domains were selected from real user audits on the Radar platform, representing businesses that actively explored their AI visibility. No domain names or identifiable data are published. Industry categories are anonymized.
The Measurement Limit: Why Single-URL Audits Misrepresent Brands
Every audit in this report measures a single URL. That is the industry default, and it is the category’s biggest blind spot. Multi-subdomain brands systematically under-score at the homepage level because their real AI surface lives elsewhere.
Three structural problems with URL-level AI audits
- 1. Category error. A search engine, a SaaS dashboard, a marketing page, and a newsroom all get judged by the same rubric. Technically correct, contextually meaningless. A content-optimized rubric applied to a JS-heavy tool returns scores that reflect the site type, not the brand’s AI visibility.
- 2. Single-URL scoring misrepresents multi-domain brands. Google’s real AI surface is blog.google, developers.google.com, about.google, support.google, and thousands of content-heavy subdomains. Scoring google.com tells you almost nothing about Google’s actual citability. The same applies to Microsoft, Amazon, Shopify, and most enterprise brands with distributed content architectures.
- 3. No site-type awareness. An e-commerce product page, a developer docs site, a B2B landing page, and a recipe blog have fundamentally different AI visibility mechanics. A single unified score averages across dimensions that should be weighted differently by site type.
Worked example: google.com scores 32/100 on our rubric
Google’s homepage is a search box, not a content page. It has almost no crawlable content (JS-heavy SPA with an input field). Its robots.txt is restrictive by design because Google blocks bots from hitting infrastructure paths. It has zero incentive to adopt llms.txt because it is the LLM via Gemini.
The score is algorithmically correct. Each metric measures what it claims to measure. But the unit of analysis is wrong. A scan of blog.google would likely return a score in the 80s. The brand is highly AI-visible. The homepage is not.
What URL audits tell you
Whether a specific page is technically retrievable by AI crawlers. Whether that page follows AEO best practices. Whether the HTML, schema, and meta are structured for extraction. Useful for fixing a specific page, not for ranking brands.
What URL audits miss
Aggregate brand visibility across every subdomain. Entity recognition across Knowledge Graphs. Citation frequency in live LLM responses. Whether AI models have enough corroborated data to recommend the brand. The category has been mis-measuring the thing that matters.
Where the category goes next: brand-level measurement
URL-level auditing is table stakes. The next frontier is brand-level AI visibility: crawling the top 10 to 20 subdomains of a brand, aggregating scores, and reporting a citation-weighted brand score. This is what AI models actually see when they decide whether to recommend a brand.
In our next report, we plan to publish brand-level scores for the Fortune 500 and compare them to their homepage scores. The gap will be large. The headline will be simple: the tools measuring AI visibility today are measuring the wrong unit.
Where does your domain stand?
Run a free AI visibility audit on your domain. Get your score across all 6 dimensions in under 60 seconds.
