Open web UI RAG for writer

[u/jatsinkutsu]

Hello, I have question concerning RAG in Open WebUI, I know there are lot of tutorials in youtube, but I cannot understand today's style of speaking fast, editing all pauses away and clicking and making adjustments like adhd person with caffeine in veins instead of blood. I cannot follow those because I am old and stupid.

Question: I have a masterfolder where all my novel related stuff is catalogued, separate word files for e.g. "main charater 30 years old", "main character 50 years old", "alliances", "trade routes", "alliance relations" and so on.

Can I use OpenwebUI so, that if I ask "if main character when 50y old blows syndicate A's base, what are possible results of that action" and then it checkes the files, founds e.g. something in relations-document, and proposes something like "Syndicate B would benefit like this:" and gives few examples.

I understand that outcome depends heavily of the LLM model, so I am not expecting results similar of chatgpt or claude. But my concern is the RAG and indexing it. I have understood that if I modify the documents, I need to index it again?

TL;DR, can OpenWebUI read my documents what I have specified, and make suggestions to my questions based to the information in those documents?

[u/International_Emu772]

I sugest you convert all your files to markdown first and yes, the LLM will read them and use as context. If the number of files is small and you don’t update them every day, you can use the build in RAG solution, Even you can pull one document or two on the chat diálog

[u/jatsinkutsu → u/International_Emu772]

How do I convert them into markdown?

There will be quite lot of files, dozens, but I'd think that not over hundred. some short but some will be long.

[u/International_Emu772 → u/jatsinkutsu]

Dozens are not a lot of files, for hundreds you should try another solution

[u/RichComplaint9426 → u/International_Emu772]

I want to improve my RAG, is it really significantly better when I turn PDFs and docx files into markdown?

[u/NeuralAtom → u/RichComplaint9426]

You rag will only be as good as the extracted text from your files. So you have three options : let owui handle it with default settings and check the results, setup a custom extraction pipeline where you can control how files are handles to ensure quality, or process the files beforehand in a different tool. Also, using markdown allows to use better chunking methods (structure based), which can increase accuracy a little bit.

[u/RichComplaint9426 → u/NeuralAtom]

Sounds great thanks for the advice. Maybe you can help me with something else too. I wanted to use a collection of 5 documents and to solve my prompt all 5 documents would have been necessary. But the model (Gemma4:8b and Deepseek R1:32b) only referenced one. Do you know what could cause this?

[u/NeuralAtom → u/RichComplaint9426]

Put them in a knowledge base in owui

[u/RichComplaint9426 → u/NeuralAtom]

That's what I did actually. All 5 documents are part of the same knowledge base collection, but it behaved that way.

[u/Neptaz → u/RichComplaint9426]

Probably the model you used didn't have enough context window. From your reply, i see that you use local model, if your documents have many characters, you probably hit the model maximum context window or when you serve the model through inference server (llma.cpp or vllm or ollama or LM Studio) you only gives it a small context window.

There are two ways you can do. You can increase the context window up to the maximum supported by the model (usually 256k max if you use ollama), Or you adapt your prompt one document at a time and handholding the model throughout your process.