How Should an Agency Scope an AEO Audit?

What Should an Agency Agree Before an AEO Audit Runs?

An AEO audit is only as useful as its scope: the written agreement, made before anything runs, about which client decision the audit informs, which pages and buyer questions it covers, which checks and engines it uses, and what it leaves out. Without that agreement, two audits of the same site can measure different things and still look comparable.

When an audit runs before anyone agrees what it covers, the scope gets argued afterwards. A client hears "the AI audit says you are invisible", and the next meeting is spent finding out which questions were asked, about which pages, on which engines, in which market. By then the audit has already shaped the conversation, and the parts it never tested have quietly been treated as tested.

AI visibility makes this harder than a classic SEO audit, because the questions are part of the measurement. A crawl report reads the same robots.txt whoever runs it. An AI-answer check depends on what you asked, how you worded it and which engines you asked, and changes in any of them change the answers.

This guide covers the decisions that come before an AEO audit: what to agree with the client, what to write down and what to keep fixed for the recheck. The six decisions below are our framework, in the order we make them. Radar's actual inputs serve as the worked example, including a mistake from our own runs: across 50 Citations runs on our own site, the category field that sets Radar's questions held eight distinct phrases.

What Client Decision Should an AEO Audit Support?

An AEO audit should support one decision the client actually has to make, written as a single sentence before any check runs. The decision sets everything else: which pages matter, which questions to ask and which checks are worth paying for.

Auditing has words for this. ISO 19011, the international guideline for auditing management systems, defines audit scope as the boundaries of an audit, and audit criteria as the requirements the evidence is compared against. An audit with no criteria can only describe; it cannot conclude.

“audit scope: extent and boundaries of an audit”

ISO 19011:2018, Guidelines for auditing management systems, clause 3.5

For an AEO audit, we read the criteria as two things: a set of reference facts the answers are compared with, and the questions buyers ask. Agree the reference facts explicitly. Websites go stale and contradict themselves, so the reference should be the current, client-approved facts and where they live. If the reference is simply the website as it stands on the audit date, say so, and describe the result as consistency with those pages rather than accuracy.

Different decisions lead to different audits. A client deciding whether to fix crawler access before a content project needs a different scope from one deciding whether AI answers describe the business correctly, and both differ from a client deciding whether to invest in being recommended for a category. The table maps common client questions to the scope each one needs, using Radar's checks as the example.

Write the decision so that the audit can change it. "Understand our AI visibility" is not a decision. "Decide whether to rewrite the two service pages before the first quarter" is, because a result could plausibly push the answer either way. If no result would change the decision, the audit is a report, and the client should know that before paying for it.

Which Pages Should an Agency Include in the Audit?

Include the pages the client decision depends on, and check which pages each tool actually reads, because most checks do not read the whole site. A scope that says "the website" without naming pages will be measured on whichever pages the tools happen to pick.

Radar is a useful example because its checks read different parts of a site. As of October 2026, from the shipped code:

The last row matters most for scoping. As of October 2026, Radar's answer-comparison checks compare what AI engines say with the homepage and up to six top-level pages from the sitemap. They deliberately skip blog posts and deep paths. If there is no usable sitemap, they fall back to common pages such as /about and /services (and /pricing for Hallucination Check).

So if the client decision is about a service page three levels deep, or a guide on the blog, those checks will not compare answers against it by default. Running the AEO Audit on that page checks its structure, but it does not add the page to the answer comparison. If the client's question is whether AI answers match facts that live only on that page, scope a separate comparison against those facts, or write the question into the exclusions.

For every page in scope, write one line saying why it is there: "homepage, because AI answers about the brand draw on it", "services page, because the decision is whether to rewrite it". For every important page left out, write that down too. A page that was never audited should never appear in the report as if it passed.

Which Buyer Questions Belong in the Test Set?

Include the questions real buyers ask about the decision at hand, worded the way they ask them, and label any question the agency invented as a hypothesis. The questions decide what the AI answers are about, so they deserve more care than any other part of the scope.

Where do real buyer questions come from?

Real buyer questions come from places where buyers already speak in their own words. Look at sales-call notes and recordings, the sales and support inbox, the questions in requests for proposals, the client's own FAQ page, on-site search logs and the client's own Search Console queries. Keep a buyer's exact wording where you have it, and treat FAQ entries, paraphrased notes and search queries as candidate questions unless you know a buyer asked them.

Then cut the list to the decision. A client deciding whether to rewrite two service pages needs the questions those pages should answer, not every question anyone has ever asked. Five well-chosen questions tied to the decision tell the client more than fifty loosely related ones.

Why does exact wording matter?

Wording is part of the measurement, not a detail. In research published at ICLR 2024, Sclar and colleagues found that formatting changes alone, with the meaning kept the same, moved one open model's accuracy by up to 76 points in few-shot tests.

That research measured accuracy on benchmark tasks, not brand recommendations, so the exact number does not transfer. We read the direction as clear: if formatting alone can move results that much, a differently worded question is a different test, and results from different wordings should not be compared.

How does Radar turn a category into questions?

In Radar, most AI-answer checks do not take free-form questions. As of October 2026, Citations, Prompt SOV and Source Influence fill fixed question templates with the category phrase you enter. Prompt SOV, for example, asks What are the best {category} companies or tools available in 2026? with your phrase in place of the placeholder, then looks for the brand in the answers. Citations also uses the brand: of its four templates, one asks about the brand, two compare providers in the category and one asks about reputation.

So the category phrase is the question wording. "AI visibility audit services" and "AI product development studios" produce questions about two different markets, even for the same site on the same day. Citation Test is the exception: as of October 2026, run on its own, it takes a question you type, between 10 and 500 characters. In a full audit, it asks What is {brand} and what do they specialize in?, with the brand filled in.

What happens when nobody writes the phrase down?

We found out on our own site. Between 5 September and 28 October 2026, Radar's Citations check ran 50 times against pixelmojo.io, most of them product tests rather than a client audit. Across those 50 runs, the category field held eight distinct phrases.

Prompt SOV and Source Influence show the same eight phrases over the same period. A run filed under "AI visibility audit services" asked about a different kind of provider than one filed under "AI product development studios". Lining those runs up as a trend would compare two different tests.

The fix is simple. Agree one category phrase with the client, worded the way a buyer would say it, write it in the scope sheet and paste it into every run. If the business genuinely sells into two categories, treat them as two question sets and report them separately.

When Does the Scope Need Technical Checks, AI-Answer Checks, or Both?

Use technical checks when the client decision is about access, page structure or markup, and add AI-answer checks when the decision is about how the business is described or recommended. Technical checks detect potential access, parsing and structured-data issues; AI-answer checks record what the measured engines actually said.

For Google's own AI features, Google's guidance points to the same fundamentals. Its documentation on AI features says there are no additional requirements to appear in AI Overviews or AI Mode.

“There are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary.”

Google Search Central, "AI features and your website" (updated 10 December 2025, read 30 October 2026)

The fundamentals it lists include allowing crawling in robots.txt and by any CDN or hosting infrastructure, internal links, important content in text, and structured data that matches the visible page. Technical checks look for problems with things like these. They do not prove what any engine has retrieved or will cite, but checking for crawler blocks or broken markup first reduces the risk of mistaking an access problem for a content problem.

Radar splits its checks the same way. As of October 2026, it runs 13 checks: six free technical checks and seven paid checks that read AI answers or the sources behind them.

That split gives the scoping rule we recommend: start with the technical checks on the pages in scope, because they cost nothing and look for the most basic problems. Then add only the answer checks the client decision needs, not all seven by default. A client deciding whether AI describes their pricing consistently with their own facts needs Hallucination Check; they may not need Source Influence at all.

Which Engines and Answer Conditions Should Be Included?

Choose the engines and conditions that match where the client's buyers ask, and write down exactly what was measured, because a conclusion only holds for the conditions that were tested.

Start with the interface. As of October 2026, Radar queries the provider APIs: OpenAI gpt-6-luna, Anthropic Claude Haiku, Perplexity Sonar and Google Gemini 2.5 Flash. Grok is added as report-only when that rollout is switched on. Radar's own disclosure is explicit that consumer apps such as chatgpt.com can differ, because they can use different models, memory, personalization and live web search. A scope should say "the measured APIs", not "ChatGPT", unless it really tested the app. If the client cares about the consumer app specifically, that is a separate test to scope.

Does the locale setting change where the question comes from?

Market comes next. Radar's Citations, Prompt SOV and Source Influence accept a locale: global, US, UK, EU, Asia Pacific, the Philippines or Latin America. As of October 2026, the setting adds an instruction asking the model to answer as if the user were in that region and to prioritize that market. It does not send the query from that country, and the questions themselves stay in English. Hallucination Check and Brand Disambiguation take no locale.

Language is a separate condition, and research suggests it does not come for free. Qi, Fernández and Bisazza (EMNLP 2023) found that larger multilingual models answered factual questions more accurately but not more consistently across languages. A client selling in English and Arabic should either test both or write "Arabic answers: not tested" in the exclusions.

What about model changes between runs?

Model versions belong in the scope because they change underneath you. On 16 October 2026, Radar moved its ChatGPT checks from a retiring gpt-4o-mini snapshot to gpt-6-luna, and runs before and after that release are not a like-for-like baseline. We wrote up what changed and why scores may move. If the model changed between two audits, the scope sheet is where the comparison records it.

How Should an Agency Limit Scope When Time or Budget Is Tight?

Cut breadth, never definition. Keep fewer pages and fewer questions, but write each one down along with every exclusion. A small audit that can be repeated exactly is more useful than a broad one nobody can rerun.

When the budget is tight, we recommend working through the cuts in this order:

The order follows from the client decision. Pages and questions closest to the decision stay; everything else becomes a written exclusion. The free technical checks stay in almost every case, because they cost nothing and look for basic access and markup problems before you pay for answer checks.

What you cannot cut is the record. Every exclusion goes in the scope sheet in plain words: "not tested: blog guides, Arabic answers, competitor X". That line protects the agency as much as the client. When someone later asks why the audit missed a page, the answer is already agreed and signed.

What Can a Small Audit Sample Establish?

A small audit can establish what the measured engines said, on a date, to the questions in scope. It cannot establish what those engines always say, or what a real buyer sees in a consumer app, which can add memory, personalization and live web search, as Radar's own disclosure notes.

The reason is sampling. By default, as of October 2026, Radar sends each question to each measured engine once per run, and users cannot set a repeat count. Under that default, each successful response gives one observation for that question and engine.

Atil and colleagues tested five language models configured to be deterministic, across eight tasks and ten runs each. Accuracy varied by up to 15% across runs, and the models rarely produced identical output twice. The figure comes from benchmark tasks, so it does not measure Radar's variability. We treat it as a reason not to assume brand answers will repeat exactly.

So agree the sample in the scope, the same way you agree the pages. Write down how many runs each question set gets, over what window, and what happens when an engine fails to respond. More comparable runs can support firmer statements about the agreed window; a single run supports a dated observation and no more. Our evidence standard covers how a report should then disclose those observations and their uncertainty.

What Must Stay Comparable in a Later Audit?

A recheck compares only when the scope stayed the same: the same category phrase and questions, the same pages, the same locale, the same engines and model versions, and the same scoring method and question templates. If any of those changed, agree in advance that the change is reported before the score.

Radar saves part of that record. As of October 2026, each saved run keeps its domain, brand, category, locale, date and results, and the run history can compare a past run with the current one, tool by tool, with the old score, the new score and the difference. The rest, such as the exact question templates, the pages an answer check read and any scoring change between runs, belongs in the scope sheet. The compare view shows numbers; the sheet shows the conditions behind them.

One product detail makes the sheet essential. As of October 2026, when a run starts without a confirmed brand, Radar suggests the brand and category from the site itself, and falls back to a generic category if it finds none. That suggestion is a starting point, not a scope. Paste the agreed phrase from the sheet into every run, including the recheck.

Agreeing this in advance is what keeps a recheck honest. A change in the test gets reported as a change in the test, not as a change in the results, and the client hears about it before they see a number.

What Should the Agency and Client Agree Before the Audit Starts?

The agency and the client should agree a one-page scope sheet before the audit starts: the decision, the pages, the reference facts, the exact question wording, the checks, the engines and conditions, the sample, the exclusions, the baseline and the evidence to keep. Both sides sign it off, and the recheck reuses it unchanged.

Here is our recommended scope sheet. The example column is illustrative, not a client case.

The sheet sits before the findings, not after them. Once the audit runs, each finding becomes a task through an action brief, and the brief is easier to write because every finding already traces back to a page and a question the client agreed to test. If the audit concerns identity, run our identity check for AI reports on the findings before they reach the client.

How Does Radar Fit Into a Scoped Audit?

Radar runs the checks; the scope decides which checks, which pages and which wording. The tool cannot choose the client decision or the buyer questions for you, and a scope that leaves those to the tool's defaults is not a scope.

As of October 2026, an agency agrees these inputs for each Radar run: the domain, the brand, the category phrase, the audit locale and which tools to run. Our Radar user guide shows where each one goes. Before any paid run, an identity step works out which business the domain belongs to from the site's own pages, and anything short of high confidence has to be confirmed by the user. Saved runs keep the domain, brand, category, locale, date and results, and the history can compare runs.

Scoping can also happen in two stages. First agree a small scope for the free technical checks, which as of October 2026 need only a URL: which pages, and what you are looking for. Run them. Then agree the paid scope with that evidence on the table: which questions are worth paying to ask, about which pages, on which engines and over how many runs. Each stage is written down before it runs.

For how Radar turns checks into scores, read how Radar scores AI visibility. Measurement is the narrow part. The audit only means something when someone has written down, in advance, what it was for.