Methodology
Most AI visibility scores are noise with a number attached.
Ask an assistant the same question twice and you can get different brands. A panel of twenty-five prompts sampled once a day produces a figure whose margin of error is wider than the range it is meant to move within. It will show a change every month, and none of those changes will mean anything.
- 120+
- prompts, frozen
- Precision comes from the number of distinct prompts, not from re-asking one.
- ×5
- repetitions
- Enough to characterise the noise. Beyond roughly ten, repetition buys almost nothing.
- per engine
- reporting
- Engines retrieve differently and move at different times.
- ±
- confidence interval
- So you can tell real movement from sampling noise.
- declared
- instrument
- API sampling, consumer interface and search-result capture are three different measurements.
- onboarding
- noise floor
- On your category and an approved competitor set, before any movement is reported.
How noisy, specifically
Three figures set the floor for any measurement design:
- 65% of AI-cited sources change from one day to the next
- ±44 points of error on a single prompt asked once
- 57.8% of ChatGPT runs return zero citations at all
We watched this happen while building our own tooling. Running one commercial query twice, forty minutes apart, returned two differently worded answers naming a different number of vendors. The underlying citation pattern held. The prose did not.
The design
| Choice | Why |
|---|---|
| At least 120 prompts, frozen and approved in writing | Precision comes from the number of distinct prompts, not from re-asking one. Freezing the set is what makes month two comparable to month one. |
| Five repetitions per prompt per engine | Enough to characterise the noise. Beyond roughly ten, repetition buys almost nothing. |
| Reported per engine, never blended | Engines retrieve differently and move at different times. A blended figure hides which engine changed. |
| A confidence interval on every figure | So you can tell real movement from sampling noise. |
| The instrument declared on every number | API sampling, consumer interface and search-result capture are three different measurements. We label which produced the figure. |
| Noise floor measured at onboarding | On your category and an approved competitor set, before any movement is reported. The floor differs by market. |
Prompts are written for your positioning, not your category
A prompt set built around a category label instead of what you sell will under-report you, and the resulting "you are invisible" finding will be an artefact of the prompt rather than a fact about your brand.
We have measured this on live sites. One vendor appeared third with a generic description on a broad category prompt. On a prompt written from their own positioning, they appeared second, and the answer misquoted their pricing by roughly twenty dollars a month, sourced from a third-party video. Same company, same day, different question.
Three instruments, and every figure says which
The full panel is sampled through each engine's own API where one exists, five repetitions per prompt, per engine, and the figure names the API it came from. AI Overviews have no API, and they are frequently absent from fetched HTML, appearing only when a page is rendered. So each wave a smaller sample from the same frozen panel is captured by a person in a real browser, logged out. On Google's AI surfaces that is search-result capture, since the answer sits inside a results page. In Perplexity and ChatGPT it is consumer-interface capture. Logged-in sessions personalise results and invalidate comparison between periods. The three instruments are reported side by side and never averaged into one figure.
Any tool that reports AI Overview presence from a plain HTTP fetch will under-report it, and will do so silently. That produces confident false negatives, which are worse than no data because they look like good news.
What we can prove, and what we cannot
| Confidence | Covers |
|---|---|
| Provable | Indexation coverage. Crawler behaviour verified from your server logs. Search Console impressions, clicks and position on a frozen query set. Conversions on target pages. |
| Sampled | Movement in AI answers. Reported with intervals, per engine, against the noise floor, never as a single score. |
| Not claimed | Revenue attributed to an AI citation. The industry cannot currently do this cleanly, and we will not present a number that implies otherwise. |
Flat months are reported as flat
Most months, nothing measurable changed. That is what the report will say. A retainer that only ever reports gains cannot be checked, and a client who cannot check the reporting has no way to know whether the work is real.
Your baseline starts with the audit
The prompt panel is built and frozen during the diagnostic audit. Every later reading is measured against it.