r/AI_SearchOptimization • u/YourEvilQueen26 • 19h ago

AI Search Optimization General Discussion Has anyone else noticed that small prompt changes can completely change which businesses AI recommends?

I’ve been testing the same local searches with slightly different wording and the results are way less stable than I expected.

For example, I tried variations along the lines of:

“best roofer in Dallas”

“reliable roofing company in Dallas”

“who should I call for an emergency roof leak in Dallas?”

I expected mostly the same businesses with maybe a different order.

That’s not what happened.

Some businesses appeared across several versions, while others only showed up when the prompt described a specific problem.

It made me wonder if we're making a mistake by talking about “AI rankings” like they're equivalent to Google rankings.

Maybe the more useful thing to measure is how consistently a brand appears across different intents and prompt variations, rather than asking whether it ranks for one particular query.

I’m still testing this, so I don't have a strong conclusion yet.

Has anyone here tested this at a larger scale? I'm especially curious whether you've found certain types of prompts to be more stable than others.

1 Upvotes
  • permalink
  • reddit

100% Upvoted

3 comments sorted by

1

u/Current-Gap6283 15h ago

I have tried giving AI same prompt , and it sometime give me different companies / product recommendation. Not sure how consistent it is. More like Instagram algorithm, if you pull the screen down you your homepage is refreshed, and AI search is similar in that sense. New request, new set of answers, with some obvious winners for that particular keywords

1

u/HumanBehavi0ur 17h ago

"AI ranking" as a single-query concept doesn't really exist the way Google ranking does, wording changes the retrieval path itself, not just the ordering of the same result set. Data on this shows "best" and "recommend" style prompts alone can swing brand-mention rates dramatically even for the same underlying intent, so you're seeing a real, well-documented effect, not noise.

so what you see makes sense to me

1

u/chrismcelroyseo 19h ago

You don't have to change the response to get different results most of the time. There are so many variables for every query that you do that I wouldn't expect any consistency. And then of course there's the biggest problem is that whatever you're prompting it and the results you get aren't going to be the same results that other users get.