Your llms.txt Is Already Stale. Here's How to Fix It.

We Maintain a Static llms.txt. It Goes Stale Every Week.

Pixelmojo publishes 2 to 4 blog posts per week. Every time we ship a new article, our llms.txt file at public/llms.txt is immediately out of date. The file lists our blog URLs, product descriptions, and company context for AI systems. But since it is a static text file committed to git, someone has to remember to update it manually.

Nobody remembers.

This is the dirty secret behind the 844,000+ websites that have adopted llms.txt since Jeremy Howard proposed the standard in September 2024. Most of them deployed a static file once, and it has been decaying ever since. The content drifts. New pages are missing. Old pages that have been removed still appear. The file becomes a snapshot of a moment in time, not a representation of what the site actually contains.

We are going to fix this. But first, we need to talk about what llms.txt actually does, because the SEO industry has gotten this completely wrong. (If you are new to the AI search landscape, start with our primer on why Google traffic dropped 33% and what replaced it.)

Step 3 of the GEO Playbook covers llms.txt as part of technical infrastructure for getting cited by ChatGPT, Perplexity, Claude, and Gemini. This post is the deep dive on the static-vs-dynamic decision.

What llms.txt Actually Is (and Is Not)

Jeremy Howard, the co-founder of Answer.AI and former president of fast.ai, proposed the /llms.txt standard in September 2024. The idea is simple: place a markdown file at your site root that gives LLMs a structured overview of your content. The official specification lives at llmstxt.org.

The format looks like this:

# Company Name

> Brief description

## Section

- [Page Title](https://example.com/page): Description
- [Another Page](https://example.com/other): Description

There is also llms-full.txt, which compiles every page on your site into a single markdown document. Think of llms.txt as the table of contents and llms-full.txt as the complete book.

The standard is intentionally minimal. No complex schemas. No XML namespaces. Just markdown that both humans and machines can read.

What the SEO industry thinks it does

The GEO (Generative Engine Optimization) community immediately latched onto llms.txt as a potential ranking signal. The reasoning: if AI search engines can read your llms.txt, they will cite your content more often. Blog posts and conference talks started framing it alongside robots.txt and schema markup as a "must-have" for AI visibility.

What the data actually shows

Three independent studies have tested this assumption. All three reached the same conclusion.

Sources: SE Ranking study, ALLMO.ai report, Search Engine Land 180-day study, OtterlyAI GEO experiment.

The Search Engine Land study is particularly telling. Of the two sites that saw AI traffic growth after adding llms.txt, one had launched 27 downloadable AI templates simultaneously, and the other had concurrent PR coverage from major publications, restructured product pages, and published 12 new FAQ pages. The llms.txt file had nothing to do with their results.

OtterlyAI's experiment adds the technical evidence: their llms.txt file received only 84 AI bot visits in 90 days, while the average page on the same site received 265 AI bot visits. The AI crawlers are not even looking at the file.

The Google Contradiction

Here is where the story gets interesting.

In June 2025, John Mueller was unequivocal: llms.txt is useless. No AI system uses it. Server logs prove it.

Six months later, in December 2025, SEO professional Lidia Infante spotted an llms.txt file on Google's own Search Central developer documentation portal. She posted her discovery on Bluesky with a pointed question: "Is this an endorsement of LLMs.txt or are you trolling us, John?"

Mueller's response was a cryptic "hmmn :-/".

The file disappeared within hours of being discovered.

The most likely explanation is that different teams within Google operate independently. A documentation platform engineer probably deployed llms.txt as part of a broader tooling update, completely unaware of Mueller's public stance. This is how large organizations work: the left hand rarely knows what the right hand is doing.

But the incident matters because it reveals something important. Even inside Google, individual teams see value in making their documentation LLM-friendly, even if the search team does not use it as a ranking signal.

Where llms.txt Actually Works: Developer Tooling

If llms.txt does not help with search rankings, why have Anthropic, Cloudflare, Stripe, Vercel, and 844,000 other sites implemented it?

Because the real consumer of llms.txt is not an AI search engine. It is a developer sitting in their IDE.

When you use Claude Code, Cursor, or Windsurf to write code, these tools need context about the libraries and APIs you are working with. They fetch documentation to understand function signatures, configuration options, and best practices. And the fastest way to load that context is to grab a single markdown file that summarizes everything. (We wrote about how context engineering shapes AI coding agent performance.)

This is why Mintlify's November 2024 rollout was so significant. When Mintlify auto-generated llms.txt across every documentation site it hosts, thousands of developer tools instantly gained LLM-friendly documentation. Anthropic's Claude docs. Cursor's docs. Stripe's API reference. All of them became loadable as a single context window for a coding assistant.

Yoast SEO recognized this shift when they added llms.txt auto-generation in version 25.3 (June 2025), making it the first major WordPress SEO plugin to support the standard. Their implementation automatically selects cornerstone content and updates weekly.

The Context7 pattern

There is another emerging pattern worth noting. Tools like Context7 (which we use in our own development) resolve library names to documentation and serve them directly to coding agents via MCP (Model Context Protocol). This is essentially a dynamic, API-driven version of what llms.txt does as a static file.

The difference: Context7 queries documentation on demand and returns only the relevant sections. llms.txt dumps everything into one file and hopes the model figures out what it needs.

Both approaches solve the same problem. But one scales and one does not.

The Real Problem: Static Files Decay

Here is what our static llms.txt looks like in practice:

1. Monday: We publish a new blog post about Claude Code hooks.
2. Tuesday: Someone asks Perplexity about Claude Code hooks. Perplexity does not check our llms.txt (no AI search engine does). But a developer using Claude Code to implement hooks might benefit from our documentation context, except the new post is not in llms.txt yet.
3. Wednesday: We publish another post. llms.txt is now two posts behind.
4. Thursday: We update our pricing page. llms.txt still references old pricing.
5. Friday: Someone remembers to update llms.txt manually. They add the new posts but miss the pricing change.

This pattern repeats every week. The file is perpetually out of sync with reality.

For documentation sites like Mintlify or Vercel's Next.js docs, this problem is solved at the platform level. The documentation framework auto-generates the file. But for custom-built sites running Next.js with a CMS or markdown-based content system, you need to build this yourself.

Building a Dynamic llms.txt in Next.js

The solution is straightforward: instead of a static file in public/, create a Next.js route handler that generates llms.txt from your actual content at request time.

Option 1: Contentlayer-powered (what we use)

If your blog runs on Contentlayer (like ours does), you can pull all published posts and generate the file dynamically:

// app/llms.txt/route.ts
import { allPosts } from 'contentlayer/generated'

export async function GET() {
  const posts = allPosts
    .filter(post => post.date)
    .sort((a, b) => new Date(b.date).getTime() - new Date(a.date).getTime())

  const lines = [
    '# Pixelmojo',
    '',
    '> AI product studio. Vector qualifies leads.',
    '> Hive orchestrates agents. You own the code.',
    '',
    '## Products',
    '',
    '- [Vector](/vector): 12-dimension lead qualification',
    '- [Hive](/hive): Multi-agent AI co-workers',
    '',
    '## Blog Posts',
    '',
    ...posts.map(
      post => `- [${post.title}](/blogs/${post.slug}): ${post.description}`
    ),
    '',
    '## Contact',
    '',
    '- Email: founders@pixelmojo.io',
    '- Location: Makati, Philippines',
  ]

  return new Response(lines.join('\n'), {
    headers: {
      'Content-Type': 'text/plain; charset=utf-8',
      'Cache-Control': 'public, max-age=3600, s-maxage=86400',
    },
  })
}

This route regenerates the file from Contentlayer's data every time it is requested, with a 1-hour client cache and 24-hour CDN cache. New blog posts appear automatically after the next Contentlayer build.

Option 2: CMS-powered (Sanity, Prisma, Supabase)

If your content lives in a headless CMS or database, query it directly:

// app/llms.txt/route.ts
import { client } from '@/lib/sanity'

export async function GET() {
  const posts = await client.fetch(
    `*[_type == "post" && !(_id in path("drafts.**"))]
     | order(publishedAt desc)
     { title, slug, description }`
  )

  const lines = [
    '# Your Company',
    '',
    '> Your description',
    '',
    '## Articles',
    '',
    ...posts.map(
      (p: { title: string; slug: { current: string }; description: string }) =>
        `- [${p.title}](/blog/${p.slug.current}): ${p.description}`
    ),
  ]

  return new Response(lines.join('\n'), {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' },
  })
}

export const revalidate = 3600

Option 3: ISR with on-demand revalidation

For sites that need instant updates when content changes, combine the route handler with Next.js ISR and a webhook from your CMS:

// app/llms.txt/route.ts
export const revalidate = false // Only revalidate on demand

// app/api/revalidate/route.ts
import { revalidatePath } from 'next/cache'

export async function POST(request: Request) {
  const secret = request.headers.get('x-revalidate-secret')
  if (secret !== process.env.REVALIDATE_SECRET) {
    return new Response('Unauthorized', { status: 401 })
  }

  revalidatePath('/llms.txt')
  return Response.json({ revalidated: true })
}

Point your CMS webhook at /api/revalidate and your llms.txt updates within seconds of publishing new content.

After building your llms.txt, validate it with the llms.txt Validator. For the full testing workflow (bot access, citations, Reddit monitoring, and llms.txt validation), see our free AI visibility tools guide.

The Ecosystem: Who Is Building What

The llms.txt ecosystem is splitting into two camps: platforms that generate it automatically and sites that maintain it manually. The automatic camp is winning.

BuiltWith now tracks llms.txt adoption across the web, reporting over 844,000 implementations as of late 2025. But a significant percentage of these are static files that were deployed once and never updated.

Related standards

llms.txt is not the only file competing for this space:

• robots.txt handles crawler permissions (allow/block). AI-specific directives like User-agent: GPTBot and User-agent: ClaudeBot control training data access. We covered AI crawler configuration in our GEO Playbook.
• ai.txt (proposed by Spawning AI in 2023) addresses training consent and data permissions, complementing robots.txt rather than replacing it.
• sitemap.xml provides URL discovery and change frequency hints.
• Schema.org / JSON-LD adds structured semantic data that AI systems already consume.

llms.txt fills a specific gap: giving LLMs a human-readable, markdown-formatted summary of what your site actually does. None of the other standards do this well.

What You Should Actually Do

Based on the evidence, here is a practical framework for deciding how to handle llms.txt:

If you are a documentation site or developer tool

Implement llms.txt dynamically. This is where the standard delivers real value. Coding assistants will use it, and your developers (and your users' developers) benefit directly. Mintlify handles this automatically. If you are self-hosting docs, build a route handler.

If you are a B2B SaaS or content-heavy site

Build a dynamic llms.txt from your CMS as future infrastructure. It costs almost nothing to implement (one route handler), and it ensures your file stays current if and when AI systems start consuming it. Do not expect it to improve your AI search citations today.

If you are a small business or personal site

Skip it. Your time is better spent on structured data (FAQ schema, Article schema), clean HTML, and content quality. These are the factors that actually influence AI citations. We covered the full tactical playbook in our GEO guide.

What matters more than llms.txt

The irony of the llms.txt conversation is that it distracts from the things that actually work for AI visibility:

1. Answer-first content structure. Start sections with direct answers, then expand with evidence. AI search engines extract the first clear answer they find.
2. FAQ schema (JSON-LD). This is already consumed by Google AI Overviews and surfaces in AI-generated answers.
3. Clean, crawlable HTML. No JavaScript-rendered content that crawlers cannot read. No content hidden behind authentication.
4. Topical authority. Publish deep, interconnected content clusters. Our AI Search Playbook series is an example of this approach.
5. External citations. When other authoritative sites reference your content, AI systems pick up on those signals.

We covered all five tactics in detail in our GEO implementation case study.

Our Approach: We Shipped It

We replaced our static public/llms.txt with a dynamic route handler at app/llms.txt/route.ts that generates from Contentlayer data. The static file is gone. The route handler pulls every published blog post automatically, includes all company and product context, and caches at the CDN layer for 24 hours.

You can see the result at pixelmojo.io/llms.txt. The blog section at the bottom updates itself every time we publish a new article. No manual edits. No stale data. No forgetting to update a file at 11pm on a Friday.

The implementation took about 30 minutes. The maintenance burden dropped to zero.

That is the real value proposition of llms.txt done right: not as a search ranking hack, but as infrastructure that represents your site accurately to any AI system that asks. Whether that system is ChatGPT searching for answers, Claude Code loading documentation context, or something that does not exist yet.

Build it dynamically. Let it run. Focus your energy on the content itself.

The Bottom Line

llms.txt is not what the SEO industry wanted it to be. It is not a ranking signal. It is not a citation driver. It is not the next robots.txt.

It is developer infrastructure. And like all infrastructure, it works best when it stays current automatically.

If you maintain a static llms.txt, it is already out of date. Build a dynamic version that pulls from your actual content, or accept that the file will always be a stale approximation of your site.

We are doing both: keeping our static file for backward compatibility while shipping a dynamic route handler that regenerates from Contentlayer on every request. Total implementation time: 30 minutes. Ongoing maintenance: zero.

That is the kind of investment worth making.

Related reading from our AI Search Playbook series:

• Your Google Traffic Dropped 33%. Here's Where It Went (Part 1)
• SEO vs GEO vs AEO: The Only Guide That Actually Makes Sense (Part 2)
• The GEO Playbook: How to Get Cited by ChatGPT, Perplexity, and Claude (Part 3)
• We Optimized for AI Search. Here's What Changed. (Part 4)

Building AI systems that need accurate, real-time context?

• Vector: Lead qualification that reads 12 dimensions of intent
• Hive: Multi-agent AI co-workers that share intelligence
• Contact Us: Talk to us about your AI infrastructure