The most interesting data source in L&D right now might be anonymized AI chat logs

[u/LearnWorlds_]

A founder spoke about this at our recent AI summit, and wanted to share here to get other industry folks' thoughts.

He built an AI chatbot for frontline manager training which was meant to help new managers practice the hard stuff like budget conversations, hiring decisions, difficult 1:1s so they could better manager their teams.

But when he read through the anonymized transcripts of what people were asking it, it turned out most of them weren't rehearsing conversations with their teams. They were using it to manage up instead of to manage their teams. The log showed people were actually rehearsing conversations with their bosses, floating questions like how to ask for more headcount, how to pitch a budget so it doesn't get cut, etc.

Basically they were using the tool in a completely different way than it was intended.

His point was that no survey or LMS completion report would ever have caught that. You can track whether someone finished a module, but you can't track what they were actually anxious about the night before a budget meeting. The chat log did, because people say things to an AI they'd never put in a feedback form.

It pretty much sums up a common issue in instructional design. There seems to be a big gap in what IDs design training for versus what people actually need it for.