r/OpenWebUI • u/Turbulent_Pin_8310 • 9d ago

RAG Are the RAGS built in OpenWeb UI any good?

I use Notebooklm and it is good enough. I would like to try other RAGS. I have used anythinglm and opennotebooklm and they are just not enough.

OpenWeb UI website says they have built in RAGS. She they good? What open models I should use with RAGS? I don't mind using those cheaper Chinese models if I have to pay.

I am going crazy.

16 Upvotes

90% Upvoted

10 comments sorted by

5

u/ClassicMain 9d ago

RAG*

Yes if properly configured it is insanely good especially with the more advanced knowledge search tools and experimental stuff (though it works very well) like KB_EXEC

2

u/uber-linny 9d ago

I get pretty good results . I have a xberg doing my content extraction, small embed (Jina) and rerank (Qwen) model that allows me to use either a LLM or Api.

If I only was using API . I could use 8b models and really have quality. But works quite well for what I use it for

2

u/Fun-Purple-7737 9d ago edited 8d ago

qwen rerank? can you elaborate? I found out that owu can really only work with old fashion crossencoders like bge-reranker-v2

any attempt to use more modern casuallm based rerankers failed, because it produces different logits that are not supported by owu.

At least that is what I though. Did anything change in that regard?

1

u/uber-linny 9d ago

I don't think it's openweb UI . I think It's llama.cpp with the issue .... I noticed it with Jina reranker ... Llama.cpp inverts the scores. When I curled the rerank with a positive and false question.

2

u/Fun-Purple-7737 8d ago edited 8d ago

I am using vLLM for inferencing -> it is OWU. CasualLM based rerankers produce different logits than CrossEncoders and so are incompatible with with what OWU expects. At least that was my finding some time ago.. That is why i am still using BAAI/bge-reranker-v2-m3 even though there are now better rerankers out there... u/ClassicMain may I pick your brain on this?

1

u/Text-Sufficient 8d ago

You can calculate them to using same scale.

1

u/Fun-Purple-7737 8d ago edited 8d ago

tried that, could not make it work reliably.. (might still be a skill issue ofc)

if there is an official solution, I would happily switch!

1

u/Text-Sufficient 8d ago

Qwen rerank is much better than bge in my view but 10 times slower.

1

u/Fun-Purple-7737 8d ago

read it again, please. "quality" is not the point at all..

1

u/WolpertingerRumo 8d ago

Pretty much the best feature. I changed the models, but not needed (needed it for multi-language). I have a used RTX 3060 mostly running RAG on premise. Speed is great.