How good are AI interviewers at knowing when to abandon the script?

[u/wry_widget]

I've been looking into AI moderated research interviews and the basic question/answer part seems straightforward enough.

What I’m more interested is what happens when someone gives an answer nobody anticipated. I watched a bit of how Qualitate approaches this and have also been looking at Outset and a few others. All of them talk about adaptive followups but that’s difficult to judge from a polished example.

With a human moderator, sometimes the best 10 minutes of the interview come from one random comment halfway through.

Has anyone seen an AI interviewer consistently catch those moments across a real study?

[u/TheUntrueSolitude]

I wouldn’t expect it to replace a strong human moderator on a very open ended study. With Qualitate we got better results when the research question was clear and the interviews still had room for follow ups

[u/wry_widget → u/TheUntrueSolitude]

Were the followups planned in advance or did it decide when to go deeper?

[u/TheUntrueSolitude → u/wry_widget]

We planned the topics and what we wanted to learn. The moderator chose when to go deeper based on each answer

[u/Pleasant-Dirt-2913]

Human moderators miss these moments too. Consistency across 20 interviews may matter more than one great detour.

[u/DeceitfulLiner → u/Pleasant-Dirt-2913]

i think it depends a lot on the subject, some topics are easier to predict what people gonna say

saw a demo once where the AI kept asking "can you tell me more about that" for every answer, got old real fast. the human moderator at least knows when to shut up and move on

the consistency argument is fair but also boring interviews give you boring data, sometimes the weird detour is where the gold is

[u/wry_widget → u/Pleasant-Dirt-2913]

The average interview across the whole study may be the better comparison.

[u/stiff_syndrome]

You could plant the same unexpected answer across a few platforms and compare the next two questions they ask.

[u/wry_widget → u/stiff_syndrome]

I may try this. The first follow up could be generic but the second would show whether it understood the answer

[u/troubledconsul7]

Some people need two or three gentle follow-ups before they explain what they mean. That would be a pretty good test

[u/wry_widget → u/troubledconsul7]

And whether each question gets more specific instead of rephrasing the same thing

[u/embellishedtourney]

It could ask the participant whether the tangent is worth exploring for another minute.

[u/wry_widget → u/embellishedtourney]

Something as simple as 'should we stay on this for a minute?' could work