How to check an AI visibility report before you act on it

Someone sends you a report saying AI is getting your company wrong. It names a competitor page, counts the bad claims, and recommends a fix. What should you check before sending it to your content or product team?

We learned this the uncomfortable way. One of our own outbound briefs opened with 21 unverified claims supposedly learned from one competitor page. The section below said that page carried six claims, then showed three claims traced to another site. The headline and its evidence were telling different stories.

The error came from how we built the brief. Its headline selected the source with the most claims in the underlying run. The detailed section selected a source using a different ranking, then counted the claims that passed its checks within the report’s time window. We fixed the brief so the visible evidence section supplies the headline’s count and source. If it cannot support a number, the headline has no number.

That failure gives us a useful checklist for any AI visibility report, including ours.

1. Find the answer behind the headline

The report should let you read the saved AI answer and the exact sentence it calls wrong. A summary such as “AI says you lack automation” may leave out a condition or combine several claims. Check what the buyer asked, which AI system answered, and when the answer was captured.

If the report says the problem is recurring, look for multiple answers to comparable buyer questions. One answer shows an observed error. It does not establish a trend.

2. Check the source attached to that claim

An answer can cite several pages. A citation in the footer is not proof that a particular page supplied the sentence at issue. Open the cited page, find the passage the report relies on, and check whether the citation is actually tied to the disputed claim.

Our review found a claim filed under one source even though a link inside the quoted answer pointed to another. We now keep that claim out of the exhibit and its count when the sources conflict. The honest answer in that case is “we cannot attribute this claim yet.”

3. Make the numbers agree with the exhibit

If the headline says 21 claims, you should be able to understand what was counted: distinct claims or repeated appearances, which answers, and which time period. The source and count in the headline should match the section that lets you inspect them.

Be especially careful with rankings and money estimates. In the same brief, a sentence said one gap was doing most of the damage while its table showed other issues reaching far more answers. A large number in a report can guide a decision only when its denominator and evidence are visible.

4. Check the proposed fix against your site

The brief also recommended making a page the company already had. The real question was whether that page was easy to find, current, and useful to a buyer asking the disputed question. “Create a page” and “improve or promote the page you have” are different assignments.

Before acting, confirm the current product fact with the team that owns it. Then inspect the pages you already publish. If the report cannot connect the wrong answer to a source and a specific correction, treat its proposed fix as a hypothesis.

An AI claim investigation should save your team time, not hand it a confident task list built on mismatched evidence. Ask for the answer, the claim, the source, and the count. If any link in that chain is missing, ask the report author to narrow the conclusion.