The trouble with a single score
Plenty of AI visibility tools open with one big number, something like “Your AI visibility: 72 out of 100”. It looks precise and it’s easy to put in a slide. But try to answer a few simple questions about it:
- Which questions were asked?
- Which AI answered them, and was it searching the web?
- How many answers is the number built from?
- What would have to change in those answers for it to become 73?
Usually you can’t tell. A score out of 100 bundles together dozens of choices: which engines count, how much a first-place mention is worth against a fifth-place one, how tone is weighed. Change any of those choices and the score changes, even though not a single answer did. The number is real arithmetic, but you can’t see what it is a count of.
That matters more for AI answers than for most marketing metrics, because the answers themselves move. OpenAI’s own documentation says its models are non-deterministic by default: the same request can produce different outputs. A score that moves by a few points may just be the answers moving, and a single number gives you no way to tell.
Count, and keep the total beside it
The alternative is simple: count, and always say what you counted.
Named in [n] of the [N] answers we recorded.
That one line carries everything a reader needs. The first number is what you found. The second is how many chances there were. And because the line says “answers we recorded”, it describes your search, not the whole internet.
It also stays honest when the result is bad. If you weren’t named, the line becomes “not named in any of these [N] answers”, which is exactly what you know. “Invisible to AI” claims far more than any search can show.
Every number names its population
A count is only meaningful next to the population it was counted in. So every number in an AI visibility report should say, on the same line:
- Which engine and model answered, and whether web search was on
- Which questions: all of them, or one product line, or one buying stage
- When the answers were recorded
This matters most when you divide one number by another. A ratio is only valid if both numbers were counted over the same answers. Suppose, as a made-up example, a run recorded 60 answers, 21 of them about protein bars, and a brand was named in 12 of the protein-bar answers. “12 of 60” mixes two populations. “Named in 12 of the 21 answers about protein bars” doesn’t. Write each count with its own total.
Good published research does this too. When Ahrefs reports how often a domain appears in ChatGPT citations, it defines its “mention share” as a domain’s citations as a percentage of the summed citations of the top sources. That’s a stated denominator: you know exactly what the share is a share of, and that it would change if the list of top sources changed.
Read each question’s asks together
If you ask each question three times, read the three answers as a group:
| Named in | What it means for that question |
|---|---|
| All three asks | Settled: you’re on the list for this question |
| None of the three | Settled gap: not named in any of these three answers |
| One or two | Unsettled: you’re at the edge of the list |
This is far more useful than an average. It tells you which questions to leave alone, which to work on, and where small changes could move you onto the list. (Why answers vary at all is explained in why ChatGPT gives a different answer every time.)
Positions only within these answers
It’s natural to want a rank: “we’re third”. If you report one, say what it’s a rank of: “third most named in these answers”. That’s a fact about your recorded answers. “We rank third on ChatGPT” is a claim about every possible answer to every possible question, which no measurement can support.
The same caution applies to citation counts. Even Bing, describing its own AI Performance report, says its citation counts reflect how often pages are cited, not page importance, ranking or placement.
Over time: same questions, and a difference is not a trend
Measuring again is where most of the value is, and where most mistakes happen.
- Ask the same questions. If the questions change between runs, the counts aren’t comparable. You’re measuring two different things. Lock the wording.
- Keep the same engine and settings, or say clearly that they changed.
- Two runs show a difference, not a trend. A made-up example: “named in 9 of 60 answers, then 14 of 60” tells you what changed between two dates. It doesn’t show a direction, and it doesn’t prove that your work caused it, because answers move on their own.
- More runs tell you more. Weekly runs on the same questions show which results hold steady and which keep changing. Over time that is how you tell a real shift from ordinary movement.
Keep every answer
The last rule makes the others possible: store the full text of every answer, with its date and the sources it cited. Then any number in your report can be opened and checked by someone who doubts it. That includes your CEO, your agency, and you, three months from now.
If a number can’t be traced back to the answers it counts, treat it as an opinion.
What to ask any AI visibility tool
Before you trust a number from any tool, Citationly included, ask:
- Which engine and model produced these answers? Was web search on?
- How many answers is this number built from?
- Can I read the answers behind it, word for word?
- Are the questions the same every time you measure?
- If there’s a score, what exactly goes into it, and with what weights?
A good tool answers all five without hesitating.
How Citationly helps
Citationly reports counts, never a score: “named in [n] of [N] answers we recorded”, with the model, date and web search setting on the report’s method page. Every count opens to the answers behind it, and the questions stay locked so a second run compares like with like. See what’s in a report, or book a demo.