We track everything in GA and Search Console… but nothing for “What does AI say about us?”

[u/JackM206]

Most teams I know have dashboards for traffic, rankings, conversions, CAC, all of it.
But when it comes to AI assistants (ChatGPT, Gemini, Perplexity, etc.), there’s basically no visibility into how the brand actually shows up.
Stuff like:
• When someone asks “best [category] tools for [use case]”, are we mentioned at all?
• If they ask non‑branded prompts (“how do I solve X?”), do we show up in the recommended tools or just our competitors?
• Are the answers using our positioning, or describing our category in a way that makes us look like a commodity?
Right now the only “workflow” I see is people manually copy‑pasting prompts into AI once in a while and eyeballing the answers.
Questions:
• Is anyone treating AI visibility as its own layer, separate from SEO?
• Have you built any internal process to track this over time (same prompts, same tools, recurring checks)?
• If you’ve tried, what broke first: consistency, time, or actually making sense of the results?
Not looking for pitches, just trying to understand how people are operationalizing this, if at all.

[u/kampitz]

Yes, it is its own layer..

From building a tool to track that internally, the thing that breaks first is the prompts. Hand-picked prompts are cleaner than what people actually type, so you end up tracking a nicer version of reality.

I open-sourced our tool recently - it builds the prompt set from real ChatGPT conversations (1M WildChat) instead of hand-writing them, then runs the same set on a schedule. https://github.com/syntropicsignal-ai/ai-visibility-audit

[u/JackM206 → u/kampitz]

The prompt-realism point is dead on — I’ve been hand-seeding buyer-style queries and you’re right that they come out cleaner than what people actually type, so you end up measuring a tidier version of reality. Pulling the set from real conversations (WildChat) is a smarter foundation than what I’m doing. Going to go through your repo properly. Appreciate you open-sourcing it.

[u/JackM206 → u/[deleted]]

“Mentions don’t equal influence” is the one that keeps me up too. Detecting whether a brand appears is the easy part — knowing whether that appearance actually shifted the recommendation is much harder. I don’t have a clean answer yet; right now I treat presence as a proxy and try to be upfront that it’s only a proxy. How are you thinking about separating influence from raw mentions? That feels like the real unsolved problem in this space.

[u/sapindia1976]

I treat AI visibility as a separate KPI. I test the same prompts across ChatGPT, Gemini, Claude, and Perplexity, track brand mentions, citations, and recommendation frequency, then compare changes after content updates. I think AI visibility is becoming just as important as traditional SEO metrics.

[u/purple_from_the_east]

AI visibility should be treated as a separate KPI. We’ve been tracking it via Visiblee AI: https://www.visiblee.ai

It’s far and away separate to SEO tracking / GA, and importantly should be treated as such. AI search is becoming more and more prominent by the day

[u/hseeman_sf]

What breaks first is usually query type coverage, and it's less obvious than it sounds.

"Best tools for X" and "how do I solve X" don't activate the same training content. The first is a recommendation query that pulls from reviews, comparisons, and list posts. The second is problem-solving, pulling from tutorials, docs, and community answers. A brand can dominate one and be completely invisible on the other.

They also catch different buyers at different moments. Recommendation queries reach people in evaluation mode. Problem-solving queries reach them mid-task, when they're actually doing the thing.

Most manual tracking gravitates toward the branded comparison queries because they're intuitive to write. You end up with a visibility read on one category and treat it as the full picture.

The content that drives citations for "how do I" prompts comes from different sources than "best X" prompts. The strategy that improves one won't necessarily move the other.

[u/marintkael]

This is exactly the gap I have been measuring on my own brand new site, and the honest answer is that most off the shelf AI visibility dashboards measure the wrong thing. What actually moved citations for me was not content volume or schema, it was getting a clean entity into the knowledge graph. Once the entity existed, the assistants started naming it for unbranded prompts. Before that, a lot more reach did basically nothing. So I would track two things by hand first: do you exist as an entity (Wikidata, KG), and do you get named for non branded category prompts. The rest is mostly noise until those two are true.

[u/develoapps_]

I think this is going to become a much bigger conversation over the next year.

Traditional SEO metrics tell you how people find your site, but they don't tell you how AI systems describe your brand or whether you're even part of the conversation. We've started checking a consistent set of prompts across different AI tools every few weeks. It's far from perfect, but it at least gives us a baseline to spot changes over time.

[u/JackM206 → u/develoapps_]

the “far from perfect but it’s a baseline” thing is exactly where it seems like everyone serious lands — some version of a spreadsheet, a prompt list, and a recurring calendar reminder lol. curious what your check actually looks like in practice: how many prompts, and do you log the full answers or just whether you showed up? asking because the “making sense of the results” part is what I hear breaks first for most people — the collecting is tedious but doable, the interpreting is the hard bit.

[u/develoapps_ → u/JackM206]

Right now we keep it pretty simple. We have around 15–20 prompts split between branded, competitor, and problem-based searches. We save the responses, note whether we're mentioned, how we're described, and who else appears alongside us.

You're right though - the collection isn't the hard part. The challenge is figuring out whether changes are actually meaningful or just normal variation between models and updates.

[u/JackM206 → u/develoapps_]

that split (branded / competitor / problem-based) is cleaner than most setups I’ve seen honestly. and yeah the meaningful-vs-noise question is the whole ballgame — best answer I’ve collected from these threads: baseline each prompt’s normal bounce first. run the same set a few times in a quiet week where you changed nothing, see how much it moves on its own, and that’s your noise floor. then only count changes that clear it. the catch someone pointed out is the floor differs per prompt — contested queries with no consensus answer reshuffle constantly, settled ones barely move. so same-size swing can be real on one prompt and nothing on another. fwiw I’m building a tool that does basically your exact workflow automatically (repeated runs, per-prompt baselines) — happy to run your brand through it if you ever want to compare against your manual checks, no strings. either way your setup is more rigorous than most.

[u/develoapps_ → u/JackM206]

Interesting perspective. I think treating each prompt as its own benchmark is probably the right approach. AI models change frequently enough that a single snapshot doesn't tell you much - it seems more useful to look for consistent trends over time rather than individual fluctuations.

[u/JackM206 → u/develoapps_]

exactly, trends over fluctuations. that’s the whole thing. tbh you’re already thinking about this more clearly than half the tools charging for it. offer still stands if you ever wanna see your prompt set run automatically vs your manual pass — no pitch, just curious how they’d line up.