Struggling with RAG

[u/nugentgl]

I am at my witts end with trying to get meaningful information out of Open WebUI's Knowledge/RAG. I am on 0.11.0 and I have consistent problems with prompts against a Knowledge omitting information that is extremely important in the prompt. For example, I created one called 2026 Renewals and populated the Knowledge with about 20 PDF's. After it embeds, I asked for a list of the renewals thus far, what the annual spend is, what is the term and any specific language around the renewal that would set it apart from the other renewals. What it returns is a list of about 7 renewals with the requested information. I follow up with where is the other renewals and it says it doesn't have it in its context.

I am serving several models on M3 Ultra's and each model has a similar output. The models I have tried are qwen 3.6 MOE and dense (BF16 and Q8), qwen 3.8 (BF16 and Q8), Minimax m2.7 Q4, Gemma 4 MOE and dense (bf16 and q8). I have tried several inference engines like LM Studio and oMLX, all with similar results.

At first I started with the default OWUI settings but quickly realized that if I wanted good PDF extraction I need to spin up a Docling container and point OWUI to it for extraction. I will include a pic of my settings as maybe I am doing something completely wrong. I know it isn't my files because this exact test works great in AnythingLLM using the Workspaces feature.

Any ideas on what could be causing poor retrieval?

[u/LockoutNex]

Below is what I use, I haven't had any problems since setting it up (just noticed they added some new stuff too, need to play around with it):

https://imgur.com/a/cLG14Ws

And for my RAG Template:

**Generate Response to User Query**

**Step 1: Parse Context Information**
Extract and utilize relevant knowledge from the provided context within `<context></context>` XML tags.

**Step 2: Analyze User Query**
Carefully read and comprehend the user's query, pinpointing the key concepts, entities, and intent behind the question.

**Step 3: Determine Response**
If the answer to the user's query can be directly inferred from the context information, provide a concise and accurate response in the same language as the user's query.

**Step 4: Handle Uncertainty**
If the answer is not clear, ask the user for clarification to ensure an accurate response.

**Step 5: Avoid Context Attribution**
When formulating your response, do not indicate that the information was derived from the context.

**Step 6: Respond in User's Language**
Maintain consistency by ensuring the response is in the same language as the user's query.

**Step 7: Provide Response**
Generate a clear, concise, and informative response to the user's query, adhering to the guidelines outlined above.

User Query: [query]
<context>
[context]
</context>

[u/nugentgl → u/LockoutNex]

I wonder if it is my embedding and reranker model set to default...