# Why Most AI Visibility Dashboards Measure the Wrong Thing

> Most AI visibility dashboards report one run per prompt and count mentions. Track share of answers across repeated runs, per engine, and who is cited instead.

Source: https://www.fame.so/post/ai-visibility-dashboards-wrong-metric  ·  Tom Hunt  ·  November 18, 2026

## Key takeaways

- One run of a prompt is a sample, because AI answers and their cited sources change between runs.
- A mention count has no denominator, so report share of answers instead.
- Keep mentions and citations apart, and report branded and unbranded questions separately.
- Report each engine on its own row, since a blended score averages away the differences.
- The most useful view is who was cited instead of you and the exact page, because it tells you what to do next.

Most AI visibility dashboards report a single score built from one run of each prompt, counted in mentions. That measures the wrong thing three times over: one run is a sample, a mention count has no denominator, and a score tells you nothing about who was cited in your place. The number worth tracking is your share of answers across repeated runs, per engine, which is how we built our own [AI visibility tracker](https://www.fame.so/search/ai-visibility-tracker).

This is a criticism of a common design, and I am not pointing at any particular product. Plenty of tools can be set up to measure well. The trouble is that the default view tends to flatter you, and a flattering number is the one that ends up in the board deck.

## What do most dashboards actually show?

The typical screen has a headline "visibility score", a line that goes up and to the right, a count of mentions this month and a list of prompts with a green or red dot beside each one.

Each piece hides a decision somebody made for you:

- How many times each prompt was run before the dot turned green.
- Whether the prompts include your brand name.
- Whether a "mention" means you were recommended, listed fifth or merely linked in the sources.
- Whether the score blends several engines into one.
- Whether anyone looked at the competitors in the same answers.

I understand why the design exists. A single score is easy to explain and easy to put in a monthly report. But the questions a marketing lead needs answered are plainer: are we in the answers our buyers see, are we there more often than last quarter, and if we are not, who is?

None of those choices is wrong in itself. The problem is that the dashboard rarely shows them, so you cannot tell a real change from an artifact of the setup.

## Why is a single check not a measurement?

Because AI answers change between runs. Ask the same question twice and you can get different companies named and different sources cited.

The best public data on this comes from [Ahrefs, which asked five AI tools the same questions daily for 20 days](https://ahrefs.com/blog/how-stable-are-ai-citations/) and tracked about 37,500 citations. On ChatGPT, 59% of citations were gone by day two and only 4.0% lasted all 20 days. The study also found about 75% of citations that dropped out came back within the period. Ahrefs sells an AI tracking product and the data covers one month, so treat the exact figures as a guide.

The point holds anyway. If more than half of citations can vanish overnight and most of them return, then a single check can show either a win or a loss by chance. A dashboard that runs each prompt once a day and draws a line through the results is largely charting noise. A monthly screenshot is worse.

Repeating the question is the fix. Even then, be honest about small numbers. With three runs per question, a question moves in 33-point steps. Combine results by question group over several weeks before you call anything a trend.

## Why is a mention count the wrong number?

A count has no denominator. "We were mentioned 140 times this month" could mean you appeared in nearly every answer or in a small fraction of them. It depends on how many answers were collected, and that number often changes when someone adds prompts.

Share of answers fixes this. It is the percentage of tracked AI answers that name or cite your company: answers naming you, divided by total answers. Add 50 prompts and the count jumps, while the share only moves if you are actually present more often.

Three more problems sit inside the word "mention":

**Branded prompts inflate it.** [Analyze AI's vendor study](https://www.tryanalyze.ai/blog/state-of-ai-search-in-b2b-2026) found that naming the brand in a prompt adds about 60 percentage points to mention rates. A panel full of branded prompts will flatter almost anyone. Report prompted and unprompted results separately.

**Mentions and citations are different.** In the Ahrefs data, the cited site's brand went unnamed in 54% of ChatGPT cases. A dashboard that adds the two together cannot tell you whether you are being recommended or only used as a source.

**Name variants get miscounted.** Decide up front whether "Acme", "Acme Inc" and "acme.com" all count, or a rebrand will look like a collapse.

## Why does one blended score hide the problem?

Because the engines disagree with each other. Analyze AI found the brand missing from all three engines it tested 47.2% of the time on generic prompts, and present in all three only 14.6% of the time. An average across engines can sit still while you gain on one and disappear from another.

Empty answers distort a blend too. In the Ahrefs study, 64% of Gemini answers carried no citations at all. If your citation rate is calculated against every answer, Gemini will drag it down for reasons unrelated to your site. Calculate citation rate as answers citing your domain divided by answers citing anyone, and do it per engine.

The fix costs nothing: report one row per engine. A longer table is less satisfying than a single score and far more useful. It also tells you where to spend effort, since the sources one engine leans on are not always the ones another does.

## What is the most useful thing a dashboard can show?

Who was cited instead of you, and the exact page.

This is the piece most score-first dashboards leave out, and I think it matters more than the score. A percentage tells you whether to worry. The list of pages cited in your place tells you what to do on Monday.

If the answer to "best payroll software for a 200-person company" cites a competitor's pricing page, a list on a trade site and a review profile, you have three specific jobs: publish clearer pricing, earn a place on that list and correct your review profile. No visibility score can give you that. When a competitor rises across a group of questions, opening the sources cited for them usually shows which page or list to look at next.

The same view catches errors. If an engine states your price or your product wrongly, the cited source is usually where the error came from.

## What should a better scorecard contain?

Here is the common design next to what I would replace it with.

Common metricWhat goes wrongBetter versionOne check per promptReports chance as changeSeveral runs per question, every weekMention countNo denominator, grows when prompts are addedShare of answers: answers naming you divided by total answersMentions and citations combinedHides being a source without being namedMention rate and citation rate kept apartBranded and unbranded prompts mixedFlatters youMostly unbranded, with branded reported separatelyOne blended scoreAverages away engine differencesOne row per engineYour brand onlyNo context and no next stepCompetitor share, and the pages cited instead of youDaily line chartInvites reaction to noiseMulti-week trends by question group

I would add one more row for accuracy: the share of fact-check answers free of errors on price, product or positioning. Being named with the wrong price is not a win.

You can build this in a spreadsheet. The full routine is in [how to track AI visibility](https://www.fame.so/post/track-ai-visibility), and our [share of answers calculator](https://www.fame.so/tools/share-of-answers-calculator) does the arithmetic.

## What should you ask about the dashboard you already have?

You do not need to replace your tool to get better numbers. Ask these questions of it first:

- How many times is each question run before a result is reported?
- How does it access each engine, and which plan or model does it use?
- Does it report mentions and citations separately?
- Can I see results per engine, not only a blended score?
- Who wrote the questions, and what share of them contain our brand name?
- Can I export the raw answers and every cited URL?
- Does it show which competitors and which pages appear when we do not?

If the answers are good, change the default view and keep the tool. If you cannot get answers, treat the score as decoration. We compare the options in [our guide to AI visibility tools](https://www.fame.so/post/best-ai-visibility-tools) and in [AI visibility tool or agency](https://www.fame.so/post/ai-visibility-tool-or-agency).

Remember too that tracking alone changes nothing. A perfect scorecard with no one doing the page work is an expensive way to watch a competitor win.

## How does Fame track it?

Fame publishes this article and sells a tracking-plus-execution service, so weigh this section accordingly.

Each week we ask ChatGPT, Perplexity, Claude and Gemini every core buyer question, several times, and record who gets named and cited. The questions are agreed with the client at the start and drawn from their sales calls, forms and market. The report shows share of answers per engine and per question, which competitors are cited instead, and the exact pages being quoted. Google clicks and indexing are tracked alongside through Search Console.

The only result I can show you is our own. On fame.so, the share of tracked AI answers citing the site went from 38% on June 1 to 76% on October 30, 2026, measured across the same panel of questions and engines on both dates. Fame has no client results to report yet, and our own numbers do not predict yours. For how measurement fits with everything else, see [the complete guide to B2B AI search](https://www.fame.so/post/b2b-ai-search-guide).

## More on B2B AI search

- [B2B AI Search: The Complete Guide (2026)](https://www.fame.so/post/b2b-ai-search-guide)
- [The 13 Best AI Search Marketing Agencies for B2B (2026)](https://www.fame.so/post/best-ai-search-agencies)
- [What to Look For When Hiring an AI Search Agency (12 Questions to Ask)](https://www.fame.so/post/compare-ai-search-agencies)
- [13 Tactics to Take Your AI Search Campaigns to the Next Level](https://www.fame.so/post/ai-search-tactics)
- [Your Buyers' Shortlist Is Written Before They Visit Your Site](https://www.fame.so/post/ai-shortlist-before-site-visit)
- [Third-Party Lists Are the New Backlinks](https://www.fame.so/post/third-party-lists-new-backlinks)
- [What We Learned Running AI Search on Our Own Site for 150 Days](https://www.fame.so/post/ai-search-own-site-150-days)

**Want your company named in AI answers?** [Fame Search](https://www.fame.so/search) is our AI search agency for B2B companies: weekly tracking across four AI engines and one action on your site every day, for $2,000 a month.

## FAQ

**What is share of answers?**

The percentage of tracked AI answers that name or cite your company: answers naming you, divided by total answers. Report it per engine.

**How many times should each question be run?**

Several times. Three runs per engine is a practical compromise and not a standard, and more repeats give steadier numbers. With three runs a question moves in 33-point steps, so combine results over several weeks.

**Do I need to replace my AI visibility tool?**

Not necessarily. Ask how many runs it does per question, whether it separates mentions from citations, whether it reports per engine and whether you can export raw answers. If the answers are good, change the default view.
