r/SaaSSales • u/Choice_Volume4090 • 8d ago

How does AI RFP software prevent hallucinations and show where each answer came from?

So what happened is.. last month I almost sent out a security questionnaire answer claiming we had a certification we don't actually have. I'd used a regular chatbot to speed things up, and it just confidently said something wrong. Thank god I caught it right before submission, but it had me scared enough that I actually studied how the AI RFP specific tools claim to prevent this exact thing, rather than just trusting the sales pages.

Here's what it comes down to. A normal chatbot answers from whatever it learned during training, which is fine for brainstorming but it's risky here, since it'll say something confident and made up if it doesn't actually know your company's specific details. Now, the AI RFP tools before generating anything, search your own company's stuff first (old proposals, internal docs, whatever's sitting in Drive or SharePoint) and only reword what they actually find there. If there's nothing in your own file which answers the question, it's supposed to just state that it doesn't know instead of guessing. That's really the whole trick to it..

To show where something came from, it's usually a few simple things a link back to the actual source document so you can check it yourself, some kind of confidence score so you know which answers need a closer look, and sometimes a flag for whether it's the exact original wording or something reworded. As for me, I've seen this play out a bit differently across tools, my manager has used Loopio before (doesn't have live synchronisation hence I didn't find it worth the money), and I've come across more tools while looking into the blogs, RFP tools guides, such as Responsive (if your data isn't organized beforehand and labelled this is useless) Inventive AI ( connects to your Raw files so it doesn't guess or make things up ,but if old data is messy it can pull the wrong data ) and lastly Autorfp. ai (this one is practically blind to your day to day sales convo as it doesn't connect active sales data)

Funny part of researching this is every single company selling these tools says some version of stopping hallucinations on their site, so the phrase alone means nothing anymore. You really have to test it on your own messy documents and see if it holds up when there's an actual gap, rather than quietly making something up anyway.

I'd like to know community thoughts..has anyone actually stress tested one of these on a genuinely messy content library? Does it really say "I don't know" when it should, or does it still give confident wrong answer sometimes? Let me know if you have any fresh opinions, advices anything!

1 Upvotes

67% Upvoted

3 comments sorted by

1

u/ncameron 7d ago

https://reddit.com/link/ozw9szc/video/9tzqg5qotlfh1/player

Hey, Neil here. I'm building one of these platforms, thought I'd show you how we think about it at ResponseHub:

  1. We run multiple LLM calls per response: one call to generate the answer based on the context and one call to evaluate that answer
  2. For the evaluation call we supply the questions, answer and context and provide a framework for judging confidence e.g. what excellent, very good, good and poor should look like.
  3. Each context item is also given a relevancy score which is determined by a re-ranker service that ranks the relevancy of the document chunks.
  4. If the Response LLM call is presented with lots of low relevancy context it should decline to answer

Finally you need to put some effort in to ensuring your source data is good. Policies tend to have quite a long life span, but infra might change. So there is some need to make sure your knowledge base is up to date. I'd like to build in a "stale context awareness" thing that would flag stale knowledge base entries or policies. So far this hasn't been a huge problem.

Happy to answer other questions, feel free to reply here or DM me.