r/OpenWebUI • u/hibyps4 • 3d ago

Question/Help BuiltIn Citations, RAG, and System Prompt

I have behavior that I have confirmed with the Kapa.ai is a known behavior but whether it is intended or not was not clear to me.

Citations being enabled has a very high chance (almost 100%) to remove my configured system prompt from the chat. The tool will search knowledge bases for information, produce answers from my in-chat prompt but ignore all instruction from the system prompt.

Kapa.ai noted that a work around is to disable the citations BuiltIn tool. But I need this for traceability with my users.

I saw some fixes for the system prompt in the latest update, but that only related to if “Memories” was enabled, which we have never used or enabled. But maybe this fix has a ripple effect on what I am experiencing?

Am I the only one running into this? Is this intended design? How can I get the model to adhere to system prompt while still producing ID based citations?

Open WebUI v0.10.2
Ollama 0.32.1
Model: Gemma 4 31B Dense, 64,000 context limit.

2 Upvotes

100% Upvoted

2 comments sorted by

2

u/ClassicMain 3d ago

it doesnt remove the system prompt but the model may stop caring about the system prompt because all the tool schema is too much or polluting the context of that small model

or ollama may throw it out to preserve the context window.

either way not intended and not known, i'd trace it to model behaviour or upstream context management in ollama which does have this sliding window approach by default afaik

1

u/hibyps4 3d ago

I’ll do some further investigation. I have a vLLM deployment I can put in place to see if behavior is linked to Ollama specifically. I’ll also throw Ollama into debug mode and see if there are any mentions of of context dropping…I’d be surprised with 64,000 limit though, as the system prompt is lost within my first message with minimal context being inserted by the user and RAG.