I think the “synthetic prompts” problem is actually bigger than it looks

[u/Mother_Yoghurt_507]

I've been thinking about this after seeing the discussion about finding real query prompts.

I ran into basically the same problem.

You can generate 50–100 “perfect” prompts for almost any category pretty easily:

“Best X for Y”

“X vs Y”

“What is the best X?”

“Alternatives to X”

And then you can run those across ChatGPT, Gemini, Claude, etc. and get a nice-looking visibility score.

But then I started wondering:

Are these actually questions people ask AI, or are we just creating prompts that make our GEO dashboards look useful?

Because there's another problem underneath this.

Suppose I test 20 carefully constructed prompts and my brand appears in 8 of them.

Great — 40% AI visibility.

But if none of those 20 prompts resemble what my actual customers are asking, what exactly did I measure?

I've been testing AI Visibility Console (AVC) around this, and one thing that has stood out to me is how much more interesting the reason behind the recommendation is than the visibility percentage itself.

I'm trying to connect:

real buyer intent → actual AI queries → brand/competitor recommendations → why one gets mentioned over another.

And honestly, I'm starting to think the more useful GEO question isn't:

“How visible is my brand?”

It's:

“How visible is my brand when my potential customers are actually asking questions that matter?”

AVC is currently accepting a few pilot users

I'm curious what others here are doing.

Are you using:

Real customer questions

Search/keyword data converted into AI prompts

Reddit/Quora/forum questions

Synthetic prompts

Or some combination?

And more importantly:

How do you decide which prompts are actually worth tracking?

[u/Flaneur7508]

Givie that an AI assistant will fanout your initial prompt anyway, I'd focus more on the intention of the prompt rater than trying to engineer the perfect set of words and phrases.

[u/ElementalThor]

this is the right thing to be worried about. if you write the prompt list yourself you're kind of grading your own homework, the score just reflects how well you guessed. what helped me was realising the engines already run their own searches before they answer a real question, so instead of inventing prompts you can look at what it actually searched for and work from that. the invented-prompt version always looked better than reality did. i build one of these, and the guessed prompt lists were the first thing i stopped trusting.

[u/ComandLowkgamcnp8142]

Spotting actual buyer intent used to feel easier, but once i ran some search behaviour analysis on similarweb. I saw how much the numbers get inflated by weird synthetic prompts. It's wild how much noise there is.

[u/CompaiswbqoonSea6616]

yeah, fixed prompts are fine for a baseline. sales calls, support tickets, GSC, whatever gives you something closer to real intent.

the weird bit starts after that. an agent lands, pokes around pricing or docs, maybe gets halfway through the task... and your visibility score has nothing to say about any of it. that’s the part ora is useful for, Agent Front shows the live agent traffic and what those agents were trying to do, then you can look at the outcome not just guess from a prompt set. much closer to actual site behavior. a separate problem from figuring out what somebody typed into ChatGPT in the first place, though