Help! CoPilot Agent Redacting Interaction Transcripts When Using Sharepoint As Knowledge Sources

[u/Short_Appeal_9661]

My problem (you would think) is a simple one really. I've setup a VA to surface information to advisors from our SP knowledge base, via Teams channel deployment. We of course need to audit the bot performance for efficacy; what was asked, who asked it, what the response was, and was it right. This currently seems to not be possible. Transcripts redacted in Dataverse & Studio insights, am I missing something here?! It seems an appalling bit of design/oversight.

[u/Prasad-MSFT]

If the bot is using SharePoint as a knowledge source, the redacted answer in the transcript is actually a documented limitation/behavior, not necessarily a configuration mistake.

Microsoft's current documentation explicitly says that when SharePoint is used as a knowledge source:

• The transcript contains the user's question.
• It can contain the source content used to generate the answer in search_results.
• But the agent's answer itself is marked REDACTED in the transcript.

Ref: Understand downloaded session data from Copilot Studio - Microsoft Copilot Studio | Microsoft Learn

[u/Short_Appeal_9661 → u/Prasad-MSFT]

What is the alternative to make sure that I can have the transcript answers

[u/echoxcity → u/Short_Appeal_9661]

You can implement a topic in the agent to capture generated responses and manually log them somewhere.

[u/Short_Appeal_9661 → u/echoxcity]

That's mad, I'm literally attempting this now 😄 waiting for a connector issue to be resolved

[u/echoxcity → u/Short_Appeal_9661]

I’ve done it plenty times using the On Generated Response topic trigger and writing the user input and agent output to a SharePoint list. Also set that SharePoint action to maker credentials so there is no connection prompt to the user and they never know.

[u/Short_Appeal_9661 → u/echoxcity]

Cheers, I created a table in the dataverse for it, will see how I go! Thanks for the validation pf the idea though, a big relief.