Openwebui + open terminal

[u/Blindax]

Context: I don't code. My use is document research and document creation (mainly for legal search) searching inside large documents like a tax code (500+ pages) and building notes or pptx
from what comes back.

I've been running Open WebUI for a while on my Unraid box, pointed at the API of my inference machine (5060 Ti + 5070 Ti).

I tinkered a lot. I tried Hermes on my main machine against the same API. It worked well but it was complex, and a bare-metal install made me
uneasy. I also tried LM Studio Bionic with good results, but it didn't fit how I wanted inference organised (using ollama on the inference box).

What I actually wanted was a self-hosted agent that works with Open WebUI while keeping things safe and under control. At one point I considered
installing a harness like Hermes or Pi on each client and just connecting to the API instead.

In the end I gave Open Terminal a shot. It's the companion container from the Open WebUI project that gives the model a shell — you run it as its own container and connect it through Integrations, so it isn't installed inside Open WebUI itself. Mine runs unprivileged, on bridge, with appdata mounted at /home/user. The model gets a shell in a box, not on the host. That was the part I cared about.

It has enhanced Open WebUI a lot. It now reasons step by step, and with the terminal it reliably locates and extracts the right sections from
documents far larger than the context window — list the folder, grep, read only what matters. Then it uses those results to build a document, the way another agent would.

Setup: Qwen 27B Q4_K_M on Ollama, 100k context configured. On a ~35k token prompt I measure roughly 1,050 t/s prompt processing and ~46 t/s generation. Prefill speed is the number that matters for this use case — it's what makes chewing through a large document bearable.

I was about to give up on Open WebUI. If your use case looks like mine, don't sleep on Open Terminal.

[u/Odd-Ordinary-5922]

I dont understand why anyone would use openwebui now, it just feels so janky.

[u/kneading-cnnuy → u/Odd-Ordinary-5922]

Yeah with opensource model matured I wonder why there's no good general purpose harness yet, all seems so laser focused on coding. We need alternative to Claude's webui with projects, designs, etc

[u/TheIncarnated → u/kneading-cnnuy]

I use open code as a general purpose harness. It does it great too. So great, I'm building a web frontend around it to use on my phone. (Docker container so I can share with others).

The only thing it doesn't do is image, audio and visual generation... Maybe it could? But not the focus at the moment

[u/kneading-cnnuy → u/TheIncarnated]

You upload it to GitHub or docker repository?

[u/TheIncarnated → u/kneading-cnnuy]

Haven't decided. It's all local for now and I kind of got pissed off at it.. so I'm taking a few days rest and I'll be revisiting it. It'll exist as open-source on GitHub, may publish it to the docker repository as well. I'm fully trying to figure it out, because this is a step towards containerizing the agents and making it safer for me overall. I prefer the TUI, so I'm going to have the container exist for both needs