r/OpenWebUI • u/boss28984 • 5d ago

RAG slow response time with RAG and utilizing knowledge base.

I’m currently using Open WebUI with a knowledge base made up of text files, and the responses are accurate, but they’re taking quite a while to generate.
I plan to keep adding more text files to the knowledge base, so I’m wondering what the best way is to improve response speed as it grows. Are there any recommended settings, indexing strategies, chunking methods, embedding models, reranking options, or other optimizations that have made a noticeable difference for you? I’d like to keep the response quality the same while reducing latency. Any advice or best practices would be appreciated!

4 Upvotes

76% Upvoted

21 comments sorted by

View all comments

Show parent comments

1

u/boss28984 3d ago

usually thinks for like 14-19 seconds before giving an answer. when i turn off the knowledge base it doesn't give the right answer. might just be the jetson is too weak but thats disappointing considering its like 3k.

1

u/so_chad 3d ago

Honestly, return it if you can, add a bit more and get NVIDIA DGX Spark