What a Wrong-Company Audit Taught Us About AI Visibility

An AI visibility audit can be accurate about the wrong company

An AI visibility score is only as trustworthy as the identity established before the measurement begins.

A platform can query the right engines, calculate every metric correctly, and produce a credible-looking report while analyzing an organization that is not yours. Nothing in the output announces the error. The scores arrive formatted, sourced, and plausible.

We know because a customer found it in ours.

He filed it as a test case:

Input:            https://wreb.de
Expected entity:  WREB.DEsign GmbH (Germany)
Detected entity:  Western Regional Examining Board (USA)

"As a result, the tool evaluates reviews, citations and brand signals for an unrelated organization. Since the entity resolution fails at this stage, several subsequent analysis results appear to be based on the wrong company." Peter Berwanger, WREB.DEsign GmbH, 28 July 2026.

That is the whole defect, stated more precisely than our own instrumentation managed. Our production monitoring, our test suite, and our existing validation all missed it. A customer with access to none of them found it in an afternoon and wrote a reproduction case.

The larger lesson applies to every AI visibility platform, including the ones that have not hit this yet: entity resolution is not one check inside an audit. It is the prerequisite for every brand-dependent check that follows.

What actually happened

Radar asked a language model to name the company behind a domain, accepted the answer, and never checked it against the website. The site had published the correct answer all along.

The WREB.DEsign GmbH homepage was not ambiguous about its identity. Our resolver simply never read it.

That page publishes a schema.org graph containing an Organization node typed as Organization, ProfessionalService, and LocalBusiness. Inside it: legalName set to WREB.DEsign GmbH, an address in Bielefeld with addressCountry DE, a service area, and social profile links. The same identity is repeated in og:site_name and in the page title. The site is fully crawlable, with Allow: / in robots.txt.

Every signal we needed was machine-readable, first-party, and correct. Our original resolver asked a language model "what company operates this domain?" and used the answer. It produced the bare token WREB and classified the industry as Education.

Why did the wrong report still look credible?

Because only part of the audit depended on the company name. The rest analyzed the correct website, which produced a blended report that looked plausible instead of obviously broken.

The original run returned a composite of 37 out of 100. Low, but not absurd for a small regional agency. That is what made it dangerous. A report that fails loudly invites investigation. A report that returns a believable D invites action.

Here is the split, from two runs against the same site hours apart. The corrected run was requested at www.wreb.de rather than the bare hostname, which makes the website-only scores worth reading closely: they came back identical across both.

Read the bottom six rows again. Every tool that only reads the website returned an identical score across both runs. Nothing about the site changed between them, so those tools acted as a control group. The only variable was which company we told the engines to ask about.

This is also the diagnostic you can run on any audit report you receive. If the technical scores look reasonable and every brand-dependent score has collapsed, suspect the entity before you accept the findings.

Why does an AI tool pick the wrong company?

Because language models are most confident about the entities they have seen most often, and confidence is not the same as correctness. This is a documented property of how these models store knowledge, not a quirk of one vendor.

Kandpal and colleagues found a strong correlational and causal relationship between how many pretraining documents are associated with a question and how accurately a model answers it, concluding that models struggle to learn long-tail knowledge. Sun and colleagues built an 18K-question benchmark across 16 models and found them far from perfect on torso-to-tail entities specifically.

More directly relevant to what happened to us, Lehmann and colleagues showed that when models choose between entities, entity popularity, mention order and semantic co-occurrence can override the model's own knowledge. Their phrasing is that heuristics "largely override principled reasoning" in smaller models.

Now apply that to a four-letter token. A regional agency in Bielefeld with a few hundred pages on the open web is a tail entity. A US examining board that administered dental licensure exams for decades, referenced across university pages, state board documents, and forums, is not. Ask a model which company "WREB" is, and popularity wins. It is not hallucinating in the usual sense. It is answering a different question well.

None of this is an exotic edge case. It has a name, named entity disambiguation, and dedicated benchmarks: the Namesakes dataset alone catalogs 58,862 mentions across 4,148 entities and their namesakes.

Brand ambiguity and resolver failure are not the same problem

There are two reasons an AI engine might describe the wrong company, and an audit that cannot separate them will blame the customer for the vendor's defect.

In the original run, our brand disambiguation tool received the already-wrong name. It then dutifully reported that four engines confused the brand with another entity. Technically true. Completely useless, because the confusion originated with us.

A tool that tests how the market understands your identity has to establish your identity from first-party evidence first. Otherwise it measures its own input error and reports it as your problem.

The bigger risk was not measurement. It was remediation.

A wrong score wastes a customer's time. A wrong recommendation can actively damage their website. Our audit did the second thing.

The resolved brand name did not stay inside the scoring layer. It flowed into the outputs customers are meant to act on: generated Organization schema, generated llms.txt content, the implementation prompts we hand to engineering teams, and the webhook payloads that push results into a customer's own Slack or Notion.

Concretely, the audit generated a schema markup block instructing WREB.DEsign GmbH to publish name: "WREB" on a homepage that already declared legalName: "WREB.DEsign GmbH" correctly. Had he acted on it, he could have introduced an inaccurate or conflicting identity signal alongside the correct one already published on the site.

This is the line between a measurement bug and a product-integrity bug. It is also why the fix could not stop at better inference. When confidence is low, the correct behavior is not to guess more confidently. It is to stop and ask.

How we rebuilt entity resolution

We replaced domain-name inference with site-first resolution: collect evidence from the website, reconcile it, and treat disagreement as a reason to lower confidence rather than something to resolve silently.

The resolution ladder

First-party evidence outranks inference. Inference only ever sees fetched page text.

Five things changed beyond the ordering.

Evidence is reconciled, not accepted

A website can carry stale schema, several Organization nodes, a parent company name, a publisher record, or the agency that built the site credited in the footer. So the resolver collects every organization node it finds and ranks them, preferring the node whose own URL matches the audited domain and penalizing nodes that explicitly point elsewhere. Agreement between signals raises confidence. Contradiction lowers it.

Low confidence stops the run

Anything short of high confidence requires confirmation before the brand-dependent tools execute, and before any credit is charged. A guess can still proceed, but only after a human has looked at it and said yes.

A human decision is permanent

Once a user confirms or edits the entity, that value is authoritative. It is bound to the domain it belongs to and survives reruns, scheduled refreshes, report regeneration, and webhook dispatch. No later automatic resolution may overwrite it.

Unverified names cannot enter remediation

Generated schema, llms.txt, and implementation prompts now refuse to assert a name unless it came from high-confidence first-party evidence or an explicit confirmation. Below that bar they emit a placeholder and a warning telling the customer to supply their exact registered name.

The audit records its own provenance

Every run now stores the resolved brand, the legal name, which signal it came from, the confidence level, the resolver version, the timestamp, the original automatic detection, whether a human confirmed it, the underlying evidence the decision was based on, and if the entity was later invalidated, when and why.

That last group matters more than it sounds. The reason this bug survived undetected was that no run recorded where its entity came from, so a poor brand score was indistinguishable from a resolution failure. The audit is now auditable.

We are deliberately not claiming that wrong entities are now impossible. First-party data can be stale, incomplete, or describe a parent company. The defensible guarantee is narrower and stronger: a wrong entity cannot silently proceed into a paid audit or into generated remediation without sufficient first-party evidence or explicit user confirmation.

What changed in the corrected audit

The corrected run did not simply produce a better number. It replaced findings that belonged to another organization with findings WREB.DEsign GmbH can act on.

Brand disambiguation moved from 20 to 100, and the four high-severity confusion flags went to zero. Perplexity moved from describing "a not-for-profit dental testing agency" to "a web design and internet agency based in Bielefeld, Germany." URL citations appeared for the first time: Perplexity cited the site in half its queries and Gemini in a quarter, against zero from every engine in the original run.

The composite moved from 37 to 49. Still a D. That is the honest part.

What the corrected report surfaced instead:

• Zero share of voice across five category prompts and four engines.
• Nine factual errors from three engines, now that they are describing the right company. Two invented a founding year. One claimed the company publishes no pricing when it publishes three tiers. One attributed services it does not offer.
• Structured data stronger than extractable content. The schema is good, which is why the correction was possible at all. The prose is not structured for engines to lift.
• No presence in any engine's source set for the category.

None of those were visible while we were measuring a dental board. Correcting the entity did not make the report flattering. It made the report true, which is the only version worth paying for.

Credit where it is due

Peter had no access to our code, telemetry, or audit architecture. He compared the report against the company he actually operates, and documented the mismatch clearly enough for us to reproduce it. That single email surfaced not one bad audit but a systemic weakness in how every ambiguous domain was resolved.

We are grateful to Peter Berwanger and WREB.DEsign GmbH for identifying the issue and allowing us to share the case so that others can learn from it.

What should you ask an AI visibility platform?

Ask how it established the identity it measured, before you look at any score. Six questions separate a platform that knows what it measured from one that guessed.

These are not edge-case questions. Entity resolution sits upstream of every brand-dependent metric, so a vendor who cannot answer them cannot defend a single one of those numbers. We could not answer them a week ago either. That is why we rebuilt it.

What this series covers

This is Part 6 of The AI Visibility Stack, a six-part series on building and operating an AI visibility platform in public. Each part builds on the last.

In Part 2, we wrote about AI engines lying about our own brand, and how that became the reason we built Radar. This part is the mirror image. Our tool misidentified a customer's brand, and the fix required rebuilding the layer that decides who we are measuring. In Part 4, we argued that orchestrated audits catch cross-tool conflicts that individual tools miss. This incident is the strongest evidence for that claim we have, because the conflict was inside our own orchestration.

Conclusion

The most useful product failures are the ones a team does not explain away.

Peter Berwanger paid us for an audit, recognized that it was measuring the wrong organization, and documented the failure clearly enough for us to reproduce it. Our monitoring did not catch it. Our tests did not catch it. Peter's report did.

Our brand-disambiguation check also returned a severe 20 out of 100 conflict, but we treated it as another score instead of a reason to stop the audit. That was a second failure.

We rebuilt entity resolution across the platform, gated low-confidence identities behind confirmation, stopped unverified names from reaching generated recommendations, and made every run carry the provenance of its own subject.

The corrected audit is still a grade D. It is now a grade D about the right company, which is the only kind worth acting on.

In AI visibility, trust does not begin with the score. It begins one step earlier, with knowing exactly which entity is being measured.

Want to know whether AI engines have you confused with someone else?

• Brand Disambiguation Check - See which same-named entity engines mistake you for
• AI Visibility Services - Fix the signals, not just the score
• Contact Us - Talk through your audit results