The AI Discoverability Stack: Four Features That Make Our Site Machine-Readable

You Optimized for SEO. AI Search Engines Still Ignore You.

You have llms.txt. You have structured data. You even allowed GPTBot in your robots.txt. But when someone asks ChatGPT or Perplexity about your product, your site doesn't show up. The answer comes from some blog that paraphrased your content three months ago.

The problem is not visibility. The problem is that AI search engines are scraping your site the same way they scrape everyone else's. That is the real barrier to getting cited by ChatGPT, Perplexity, and Claude. You are one of billions of pages in a parse queue. There is no signal that says "this site has a direct knowledge API you can query instead of guessing from HTML."

We built four features on pixelmojo.io that changed this. Together, they form what we call the AI Discoverability Stack: a set of machine-readable declarations and APIs that turn your site from a scraped source into a primary citation target. One of them, the MCP server, works differently: it serves AI clients that someone has connected to it, and does not by itself change how AI search engines find or cite your pages.

This is not theory. We implemented it, tested it, and the responses came back with our URLs as sources. This is Part 6 of The AI Search Playbook, our series on moving from traditional SEO to AI-native discoverability.

Updated September 2026: Feature 2 originally recommended ai-plugin.json, the manifest format of the original ChatGPT plugins beta, which OpenAI wound down in 2024. It now covers the access points we actually serve: an MCP server and the knowledge API.

This builds on Step 3 of the GEO Playbook. The playbook covers the five steps; this post is the deep dive on the four-feature discoverability stack we built.

The Core Problem: Scraped vs. Primary Source

Most websites have a passive relationship with AI search engines. A bot crawls your page, parses the HTML, extracts what it can, and moves on. If your content is clear enough, it might get paraphrased into an AI-generated answer. If you are lucky, there is a link back to you.

This is being scraped. You have no control over how your content is represented, no guarantee of attribution, and no way to correct misunderstandings. If AI engines are already saying inaccurate things about your brand, see Why does ChatGPT say wrong things about my company? for the diagnostic + correction playbook.

Being a primary source is fundamentally different. The AI system discovers that you have a knowledge API, queries it with a structured request, and gets back a structured response with your answer, your source URLs, and your entity context. The citation is built into the response format.

The difference is not about content quality. It is about infrastructure. A site with a knowledge API, declared capabilities, and connected structured data tells AI systems: "You can query me directly. Here is exactly how." We covered the foundations in our knowledge graph implementation guide. The Discoverability Stack builds on that foundation. It also assumes AI links your name to the right entity in the first place; when it does not, that is a brand disambiguation failure to fix before any other optimization compounds.

Feature 1: Connected JSON-LD with @id References

Most sites have JSON-LD that works but doesn't connect. Your product page has SoftwareApplication schema. Your about page has Organization schema. Your blog posts have Article schema. But they don't reference each other. For the standalone how-to on Organization schema specifically, see How do I add Organization schema for AI search?. For the head-to-head between the two signal types, see Schema.org vs llms.txt: which AI search signal matters most?.

The Problem with Isolated Schemas

When each page's structured data exists in isolation, AI systems treat them as separate data points. They cannot infer that Vector is a product made by Pixelmojo, that Hive is built on Vector, or that your blog posts are written by your organization's founder.

The Fix: @id URIs

Adding @id to your schemas creates anchors that other schemas can reference:

{
  "@type": "SoftwareApplication",
  "@id": "https://www.pixelmojo.io/#vector",
  "url": "https://www.pixelmojo.io/vector",
  "name": "Vector by Pixelmojo",
  "provider": {
    "@type": "Organization",
    "@id": "https://www.pixelmojo.io/#organization"
  }
}

Now when your Hive product page references Vector, it uses the @id instead of just a URL:

{
  "isBasedOn": {
    "@type": "SoftwareApplication",
    "@id": "https://www.pixelmojo.io/#vector"
  }
}

AI systems can now traverse your entire entity graph. Product references organization. Product B references Product A. Blog posts reference both. Everything connects.

BreadcrumbList for Hierarchy

We also added BreadcrumbList schema to product pages:

{
  "@type": "BreadcrumbList",
  "itemListElement": [
    { "position": 1, "name": "Home", "item": "https://www.pixelmojo.io" },
    {
      "position": 2,
      "name": "Products",
      "item": "https://www.pixelmojo.io/products"
    },
    {
      "position": 3,
      "name": "Vector",
      "item": "https://www.pixelmojo.io/vector"
    }
  ]
}

This tells both Google and AI systems exactly where each page sits in your site architecture. For a deeper dive on how BreadcrumbList and entity linking work together, see our knowledge graph and LLM visibility guide.

Feature 2: Explicit Agent Access Through MCP

An AI client can only query your site directly if you publish an access point it knows how to call. We serve two: an MCP server at /api/mcp, covered here, and the REST knowledge API at /api/ask, covered in Feature 3.

The Model Context Protocol (MCP) is, in its own documentation's words, "an open-source standard for connecting AI applications to external systems." AI applications that support it, which that documentation lists (as of September 2026) as including Claude and ChatGPT, can call the tools a server exposes.

What Ours Exposes

One tool, ask_pixelmojo. It answers factual questions about Pixelmojo from reviewed public information and, where an answer draws on a published page, links that page as its source. It changes nothing on your behalf and does not handle pricing, checkout or account actions.

curl -X POST https://www.pixelmojo.io/api/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"ask_pixelmojo","arguments":{"question":"What is Radar?"}}}'

What MCP Does Not Do

In our reading of the MCP documentation, which describes AI applications connecting to servers they have been set up to use, publishing an MCP server does not by itself make AI search engines find your site or cite it. Someone has to connect an AI client to the server first. Treat it as a clean, explicit way in for the clients that are connected, not as a ranking or discovery signal.

A Note on ai-plugin.json

The first version of this post recommended ai-plugin.json, a file at /.well-known/ai-plugin.json describing a site's capabilities. It was the manifest format of the original ChatGPT plugins beta (OpenAI's own plugins quickstart ships one), and OpenAI wound that beta down in 2024; the quickstart now notes that "Plugins have been superseded by GPTs." A post on OpenAI's developer community forum quotes the announcement: "On March 19, 2024, you will no longer be able to install new plugins or create new conversations with existing plugins," and existing plugin conversations could continue "until April 9, 2024."

We still serve ours, because it lists our llms.txt, MCP and REST endpoints for any client that reads it. We no longer recommend building one.

Feature 3: The Knowledge API (/api/ask)

This is the centerpiece of the stack. Instead of making AI systems parse your HTML, you give them a structured endpoint to query directly.

How It Works

This section describes the endpoint as of September 2026.

Input: A POST request with a question.

curl -X POST https://www.pixelmojo.io/api/ask \
  -H 'Content-Type: application/json' \
  -d '{"question": "What is Vector?"}'

Context retrieval: The API turns the question into an embedding and looks up the closest knowledge graph entities, blog posts, answer pages and hand-written product notes in a Postgres vector index (pgvector). Weak matches are dropped, with one exception: when a question names Pixelmojo or compares its products, the matching product notes and a company overview are kept too. The closest matches become the context, each with the URL it came from.

LLM call: The context goes to GPT-4o with strict instructions: answer from context only, never fabricate, return structured JSON.

Output: A structured response with everything an AI system needs to cite you.

Every response includes:

• answer: 1-3 sentence answer grounded in your actual content
• sources: Blog post URLs that the answer draws from
• relatedEntities: Products, methodologies, and services connected to the question
• confidence: How well the context matched the question (0 to 1)

Rate Limiting

As of September 2026, the endpoint uses the same shared rate limiter as our other API routes, backed by Upstash Redis so the limits hold across serverless instances: 10 requests per minute and 50 per day per IP. Enough for legitimate AI agent queries, restrictive enough to prevent abuse.

Start Without a Vector Database

You don't need one on day one. For a small site, keyword matching works: tokenize the question into words, match against entity keywords and post metadata, and rank by overlap count. That is how our knowledge API started. Since May 2026 it has retrieved with embeddings, now stored in Postgres (pgvector), which finds related content even when the question uses different words than the page. Start simple, and add embeddings when keyword matches start missing.

Feature 4: FAQ Schema From One Source

FAQPage structured data marks up the questions and answers a page already shows. It is optional: Google stopped showing FAQ rich results on May 7, 2026, and no major AI engine documents a citation benefit from it. We keep it because it is cheap and keeps our Q&A machine-readable, and because FAQ markup has to match what the reader can see.

That last rule is why we changed how we build it. Until October 2026, each post's FAQ schema was written by hand in the frontmatter, separately from the visible FAQ, and an earlier script could generate schema with GPT-4o-mini for posts that had none. Kept apart, the two drifted: in an October 2026 check, the schema's questions or answers differed from the visible FAQ in 56 of our 98 posts, and one post shipped FAQ schema for a FAQ that never rendered.

One List, Two Outputs

Each post now keeps its FAQ as one typed list in its frontmatter:

1. One faqs list per post (question and answer pairs), validated by Contentlayer at build time
2. The visible FAQ (<BlogFAQ />) renders from that list
3. The FAQPage JSON-LD renders from the same list, never separately
4. The build fails on drift: a post with the FAQ tag but no list (or the reverse), a hand-written FAQPage block, or FAQ props in the body stops both the build and the pre-commit check

We retired the generator. Schema written by a model for a FAQ the reader cannot see is exactly the mismatch the rule exists to prevent.

The Numbers

• 98 blog posts migrated in one pass (October 2026)
• 56 had schema that differed from the visible FAQ
• 1 shipped schema for a FAQ that did not render

The Bot Strategy: Selective on Training, Open on Browsing

This stack works because of a deliberate robots.txt strategy that many sites get wrong. We covered the full bot segmentation approach in our GEO Playbook, but here is the summary.

Our policy evolved. When we first published this, we blocked all training bots to protect IP. As our citation strategy matured, we realized brand recognition in future model generations matters more than blanket IP protection for a new brand still earning ground in AI answers. We now split training bots into two buckets.

Allowed training bots (these feed AI engines we want citing us):

• GPTBot (OpenAI), ClaudeBot (Anthropic), CCBot (Common Crawl), Google-Extended (Gemini), Applebot-Extended (Apple Intelligence)

These bots collect content for model training. Allowing them is a long-term bet on brand recognition in future models, not a measured citation lever.

Still blocked (data brokers and adversarial scrapers):

• cohere-ai, Meta-ExternalAgent, FacebookBot, Bytespider, Diffbot, Omgili, img2dataset

Data brokers resell scraped content. Bytespider feeds ByteDance products with limited Western reach. We judged blocking these worth the trade-off.

Always allowed (search and user agents):

• OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User

These agents read your content at query time to generate answers. When an answer cites you, it can link back to your site. This is the pipe that sends traffic and citations. You can verify which bots have access to your site using our free robots.txt Analyzer for AI. The rest of this stack gives agents that query you a structured route alongside the HTML.

“Allow the training bots feeding AI engines you want citing you. Block the data brokers. Always keep browsing bots open.”

The AI Bot Strategy

Implementation Checklist

If you want to build this for your own site, here is the order:

1. Connect Your JSON-LD (30 minutes)

Add @id URIs to existing schemas. Add url properties. Make provider reference your global Organization by @id. Add BreadcrumbList to product and service pages.

2. Expose an MCP Server (optional)

Wrap your knowledge API as an MCP tool so AI clients that support MCP can call it. Keep its answers limited to reviewed facts, and return the sources with each answer.

3. Build the Knowledge API (2 hours)

POST endpoint with rate limiting. Context retrieval from whatever content system you use: keyword matching to start, embeddings once it grows. LLM call with strict grounding instructions. Structured JSON response with sources and entities.

4. Build FAQ Schema From One Source (1 hour)

Keep each post's FAQ as one list, render both the visible FAQ and the FAQPage JSON-LD from it, and fail the build on drift.

Total effort: approximately half a day. The ongoing cost is near zero: a small model such as GPT-4o-mini costs fractions of a cent per query, and everything else is static.

Testing Your Stack

Test the MCP Server

curl -X POST https://yoursite.com/api/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

Should list your tools. Then call one with tools/call and check the answer carries its sources.

Test the Knowledge API

curl -X POST https://yoursite.com/api/ask \
  -H 'Content-Type: application/json' \
  -d '{"question": "What does your company do?"}'

Should return a structured response with answer, sources, and entities.

Verify JSON-LD

View page source on your product pages. Look for @id properties and BreadcrumbList schemas.

Audit Overall Visibility

Use our free AI Crawl Checker to test your bot access, structured data, and llms.txt in one scan. The AI Readiness Score combines all signals into a single 0-100 score.

What This Means for Your Site

The AI Discoverability Stack is not about gaming AI search engines. It is about making your site a better, more structured source of information that AI systems can reliably query and cite.

Every feature serves a clear purpose:

• Connected JSON-LD makes your entity relationships machine-readable
• An MCP server gives connected AI clients an explicit way to query you instead of parsing HTML
• The Knowledge API gives AI agents a direct line instead of forcing them to parse HTML
• One-source FAQ schema keeps every post's Q&A machine-readable and identical to what readers see (optional; no documented citation benefit)

The combined effect: your site becomes a primary source that AI search engines query and cite, rather than one of billions of pages they might scrape and paraphrase. If you are earlier in the journey, start with our SEO vs GEO vs AEO guide to understand the landscape, then work through the GEO Playbook for tactical foundations before building this stack.

Want to see how your site scores for AI discoverability?

• AI Crawl Checker: Test your bot access, structured data, and llms.txt
• AI Readiness Score: Get a unified 0-100 score across all AI visibility signals
• Contact Us: We build AI discoverability stacks for companies that want to own their narrative in AI search