Personalised AI search visibility is not one metric. It is the measurable difference in answer presence, citations, recommendations or framing between defined user contexts.
That definition matters because AI answers vary even when no meaningful personalisation is present. Retrieval can change, models can phrase the same conclusion differently, sources can rotate and answer modes can alter the format. A brand mentioned in one answer and absent from the next has not automatically gained or lost personalised visibility.
The practical task is to separate a context-linked pattern from normal answer variance. That requires controlled comparisons, repeated observations and careful language in the final report.
This framework is designed for SEO practitioners and content teams assessing logged-in versus logged-out experiences, remembered preferences, conversation history, devices, locations, language settings and model configurations. It applies to answer engines and AI search features, but each platform exposes different controls. Start by documenting what can actually be observed rather than assuming every interface uses the same personalisation signals.
Define visibility before testing personalisation
A useful measurement programme separates the stages that are often collapsed into a single “rank”:
- Answer inclusion: Is the brand, product, page or claim present?
- Citation visibility: Is an owned or earned source linked, cited or named as support?
- Recommendation visibility: Is the brand presented as a viable option, a leading option or the selected option for the stated need?
- Answer framing: What attributes, caveats, audience fit and competitors surround the mention?
- Conversion evidence: Where measurable, does an exposed user later take an attributable action?
The distinction is more than reporting hygiene. A citation can appear without a recommendation; a recommendation can appear without an owned citation; and a favourable mention can be conditional on a user profile. Use the definitions in the AI search measurement taxonomy to keep those outcomes separate.
For each query, define an eligibility rule before collecting results. For example, a cybersecurity vendor may be eligible only when the answer gives a shortlist for enterprise buyers in a named market. It is not eligible when the prompt asks for free tools for a personal device. Counting every mention as a missed or won opportunity produces misleading rates.
Map context as testable variables, not vague audience segments
“Personalisation” is a convenient label for several mechanisms. Turn them into explicit variables with a known value in every test record.
| Variable | Example states | What to hold constant |
|---|---|---|
| Login and account state | Logged out; logged in; account with stated preferences | Query, language, location and model mode |
| Conversation history | Fresh chat; neutral prior turn; relevant prior turn | Exact final prompt and account where possible |
| Locale and language | UK English; US English; local-language interface | Device, query intent and time window |
| Device and interface | Desktop web; mobile web; app | Account, locale and model setting |
| Model and search mode | Default model; research mode; web search enabled | All other context variables |
Do not treat a profile attribute as a fact merely because it was entered in a chat. A model may acknowledge it, ignore it or use it only to shape wording. The evidence is the observed answer difference across repeated, controlled conditions—not an inference about an underlying platform system.
Platform documentation can help establish visible product capabilities and crawler requirements, but it does not prove a result was personalised. Review Google’s guidance on AI features in Search for implementation context, and record the date and interface version used in your study.
Build a cohort matrix and a control condition
The most defensible design uses paired cohorts. A cohort is a repeatable bundle of settings, such as “logged out, fresh session, US English, desktop, default mode” or “logged in, prior conversation states a preference for low-cost tools, US English, desktop, default mode.”
Create a baseline cohort with no deliberately introduced personal context. Then change one variable at a time. If the study changes login state, location, conversation history and device simultaneously, any observed difference is real at the bundle level but cannot be assigned to one cause.
For sensitive categories, avoid using real customer accounts or identifiable personal details. Synthetic, non-sensitive test profiles are usually sufficient for observing answer behaviour. Record only the minimum account and session information needed to reproduce the setup.
Run equivalent prompt sets through every cohort, rotate their order and distribute collection over comparable time windows. Rotation reduces the risk that a transient retrieval shift affects only one condition. If a platform limits automation or prohibits particular collection methods, respect those limits. Technical accessibility is not permission to test at scale.
Use repeated prompts to estimate ordinary answer variance
A single response is a screenshot, not a measurement. Repeat every prompt in the baseline condition before declaring a context effect. Then repeat it in each test cohort.
For each run, capture the full prompt, timestamp, platform, model or mode label, account state, interface, locale, prior turns, answer text, citations, recommendation order and screenshots or exports where allowed. Preserve source URLs and the text surrounding each citation; a URL alone is rarely enough to evaluate what it supports.
There is no universal repeat count. Begin with a modest pilot, inspect how often outcomes change, then expand the sample for volatile or commercially important query groups. The goal is not a ceremonial number of runs. It is enough observations to distinguish a recurring cohort pattern from a pattern also common in the control. See this guide to AI search prompt sampling and the companion guide to visibility confidence intervals for planning principles.
Where the platform permits deterministic settings, document them. Do not assume they eliminate retrieval or source-selection variance.
Code answers at claim level
Use a codebook that another reviewer can apply. At minimum, code each eligible answer for inclusion, owned citation, third-party citation, recommendation class, listed order, sentiment or qualification, and material factual error.
Recommendation classification deserves particular care:
- Not recommended: mentioned only as background, excluded or warned against.
- Conditional recommendation: presented as suitable for a defined need, budget, market or capability.
- Unqualified recommendation: included positively without a material limitation in the answer.
- Primary recommendation: selected as the top or clearest choice, where the answer explicitly establishes one.
Do not infer primary status simply because a brand appears first in an unordered paragraph. Likewise, do not count a citation as supportive until the cited page entails the nearby claim. A citation may be adjacent to a brand without validating its capabilities, pricing, availability or suitability. Apply a claim-to-source review process such as the citation accuracy and entailment framework.
Worked example: a context comparison, not a product finding
Hypothetical example. A B2B software team wants to test whether a stated preference changes recommendation visibility. Its eligible query is: “What are suitable project-management platforms for a 200-person UK consultancy that needs client access and SSO?”
The control cohort is logged out, fresh session, UK English, desktop and default search mode. The test cohort is identical except for one prior user message: “We prefer tools with strong client collaboration and want to avoid complex administration.” Each cohort runs the final prompt 12 times in an alternating sequence.
The coding rule says Brand A receives inclusion = 1 when named; owned citation = 1 when its domain is cited; conditional recommendation = 1 when the answer connects it to the stated consultancy requirements; primary recommendation = 1 only when explicitly ranked first or called the best fit.
In this hypothetical dataset, Brand A appears in 5 of 12 control answers and 9 of 12 context answers. It is conditionally recommended in 3 control answers and 7 context answers. Its owned domain is cited in 2 and 4 answers respectively. Two citations in the context cohort point to pages that describe the product generally but do not support the claimed SSO feature, so those citations are retained as visibility observations but flagged as unsupported for evidence-quality reporting.
The inclusion rate is 5/12 = 41.7% in control and 9/12 = 75.0% in the context cohort: a descriptive difference of 33.3 percentage points. The conditional-recommendation difference is also 33.3 points. The team should report that difference as observed under this specific test setup, not as proof that all users with that preference will see Brand A more often. With only 12 runs per cohort, the result warrants replication across dates, adjacent prompts and possibly other interfaces.
Interpret results with an evidence ladder
Use four labels in dashboards and stakeholder reports:
- Observed difference: a rate differs between cohorts.
- Repeated context pattern: the difference recurs across runs and planned replications.
- Supported context association: one-variable controls and evidence review make a context explanation plausible.
- Unresolved variance: differences exist but controls, sample size or concurrent changes do not support attribution.
This language can feel cautious, but it is operationally useful. It tells content teams whether to improve an evidence block, create a locale-specific page, investigate an inaccurate claim or simply keep monitoring. It also prevents a dashboard from converting a handful of volatile answers into a fictional audience truth.
Report absolute counts beside percentages, define the eligible denominator and show the dates, cohorts and platform version. Separate answer visibility from citation quality, and citation quality from commercial impact. If you monitor over time, preserve the same core cohort definitions so that a change in methodology is not mistaken for visibility drift.
Turn findings into content and measurement actions
A repeatable context pattern can reveal useful gaps. If answers for a local cohort cite third-party sources but omit your official regional documentation, review whether your pages provide clear availability, pricing, support and evidence for that market. If answers respond to a conversation preference with unsupported product claims, prioritise correction and source clarity before treating the extra mention as a win.
Structured data can improve machine-readable interpretation when it accurately represents visible page content, but it is not a personalisation control or a citation guarantee. Google’s structured data documentation is a useful implementation reference. For platforms with explicit crawler guidance, such as Perplexity, check the current bot documentation alongside your own access and indexing diagnostics.
My practical recommendation is to keep a small, high-value context panel rather than attempt to simulate every possible person. Prioritise contexts that materially alter eligibility: country, language, regulated market, buyer type, budget band and explicit product constraints. Expand only when repeated evidence shows that the added cohort changes decisions.
FAQ and conclusion
Can a logged-in versus logged-out test prove personalisation?
No. It can show an observed difference between those conditions. Repeated paired tests, stable controls and a documented account state make a context association more credible, but they do not reveal every internal system decision.
Should we personalise content to match every AI user profile?
No. Build accurate pages for real audience needs, markets and use cases. Test whether those distinctions affect eligible answer opportunities before creating narrowly tailored content.
What is the most useful headline metric?
For many teams, use eligible-answer recommendation rate, reported with inclusion rate, owned-citation rate, the number of runs and the cohort definition. No single rate captures answer quality or evidence support.
How should results be presented to leadership?
Show the context tested, the absolute counts, the percentage-point difference, evidence-quality flags and a confidence label such as observed difference or repeated pattern. Avoid claims about all users or future traffic unless separate evidence supports them.
Conclusion: Personalised AI search visibility is measurable when context is treated as an experimental condition rather than a story about an individual user. Define eligibility, isolate variables, repeat prompts, code answers consistently and preserve citation evidence. The strongest report does not promise certainty. It shows what changed, under which conditions, how often it changed and what should be tested next.
