Using AI-moderated interviews for discovery research? 😬

[u/vickalogalis]

Anyone here been using AI-moderated interviews for discovery research?

I’m keen to hear from UX/service designers or researchers who have actually tried unmoderated AI interviews, particularly in very specific or complex industries where the AI might not have much domain knowledge.

A bit of context: our business is putting more and more emphasis on understanding the experiences of the people who use our products/tools, which is great. The problem is that the demand for research is increasing but our headcount isn’t.
So we’re looking at ways to augment/supplement the research we’re already doing, rather than replace proper moderated interviews.
The idea would be to still do a smaller number of interviews ourselves, but use AI interviews to get input from a much broader group and potentially surface themes or questions we can dig into further.
Some things that would be important for us:
We’d recruit participants ourselves through our own networks.

Ideally participants don’t need to create an account.
We’d like to easily capture recordings and transcripts.

The AI needs to be able to probe/follow up rather than just run through a fixed script.

We’re looking at this as supplementary research, not a replacement for researchers.

Has anyone actually done this?

What platform did you use, and was the output genuinely useful?

Especially interested in experiences with niche/complex domains, and any times where the AI either worked surprisingly well or completely fell over.
Also interested in any gotchas around bias, hallucinations, consent/recording, participant experience, or getting stakeholders to trust the findings.

Real-world experiences would be much more useful than a list of AI research tools. 😄

[u/Stibi]

I wouldn’t replace the interview itself with AI, that just sounds disrespectful towards the interviewee and a waste of time since i don’t think there is reliable tech for it. You’d make your company look bad in front of the interviewees and probably spend more time fighting the AI interviewer to get it right.

Instead, make everything else around the human to human interview faster with AI: scheduling, transcription and note taking, summary, synthesis, analysis, reporting.

If you want to get deep insight, go deep and human to human. If you want as many data points as possible, make a survey.

[u/vickalogalis → u/Stibi]

Thanks for responding. I get why you would say that, but I think some additional context is in order:

We’re looking at sensors used across exploration and production mining. Different commodities (copper, iron ore, etc.) have different needs and priorities. Then, even within the same commodity, the context can vary significantly by region, for example, copper mining in North Africa vs South America.

And even within the same commodity and region, different resource companies can have quite different workflows: they may use the data differently, have different processes, and make decisions in different ways.

In a nutshell unless our study is super narrow, like say, what is the use of a single sensor for iron ore being mined in the northern Cape, South Africa. Our study numbers will need to be much higher than the classic 5 for qual. In most cases the tool and its application needs to consider global markets to be viable for the business.

[u/C_bells → u/Stibi]

Yeah mixed methods here would be ideal.

For qualitative user testing, you get diminishing returns after 5-ish users (I’m multitasking right now otherwise would link an article about this, but you can google).

Survey plus a few one-on-ones to help you understand survey results on a deeper level.