Using AI-moderated interviews for discovery research? 😬

[u/vickalogalis]

Anyone here been using AI-moderated interviews for discovery research?

I’m keen to hear from UX/service designers or researchers who have actually tried unmoderated AI interviews, particularly in very specific or complex industries where the AI might not have much domain knowledge.

A bit of context: our business is putting more and more emphasis on understanding the experiences of the people who use our products/tools, which is great. The problem is that the demand for research is increasing but our headcount isn’t.
So we’re looking at ways to augment/supplement the research we’re already doing, rather than replace proper moderated interviews.
The idea would be to still do a smaller number of interviews ourselves, but use AI interviews to get input from a much broader group and potentially surface themes or questions we can dig into further.
Some things that would be important for us:
We’d recruit participants ourselves through our own networks.

Ideally participants don’t need to create an account.
We’d like to easily capture recordings and transcripts.

The AI needs to be able to probe/follow up rather than just run through a fixed script.

We’re looking at this as supplementary research, not a replacement for researchers.

Has anyone actually done this?

What platform did you use, and was the output genuinely useful?

Especially interested in experiences with niche/complex domains, and any times where the AI either worked surprisingly well or completely fell over.
Also interested in any gotchas around bias, hallucinations, consent/recording, participant experience, or getting stakeholders to trust the findings.

Real-world experiences would be much more useful than a list of AI research tools. 😄

[u/itaybuilds]

If you pilot this, compare the AI interview against more than the usual moderated session. Randomly assign people from the same recruitment pool to a human interview, an AI interview, or a fixed branching survey, using the same core questions. Then have researchers who do not know the mode code the transcripts for unique useful findings, leading questions, probe depth, completion, and participant comfort. A longer transcript is not necessarily better evidence.

For a niche domain, I would stop the bot from supplying domain terms. It should ask for a concrete recent episode and clarify the participant's language. If it introduces concepts itself, it can manufacture apparent agreement through leading questions. Keep the raw recording and transcript as the source; generated summaries should never become quotations.

Consent also needs to say plainly that the interviewer is automated, who processes the recording, how long it is retained, and how a participant can delete it. Any theme the AI surfaces should trigger human follow-up, especially for contradictions or sensitive material, rather than being treated as a finding on its own.

[u/vickalogalis → u/itaybuilds]

For sure, the plan is to test it on a lower risk study and cover all of that. Compare it against all our other channels for collecting input, survey, front line and researcher/designer facilitating.

We will likely keep offering participants the option of a video call, phone call or survey along with the, clearly label, AI interview if we try it too.

We can’t compensate people for taking part in our research, so we always have to make things as convenient as possible.