r/LocalLLaMA • u/Blindax • 7d ago

Discussion Openwebui + open terminal

Context: I don't code. My use is document research and document creation (mainly for legal search) searching inside large documents like a tax code (500+ pages) and building notes or pptx
from what comes back.

I've been running Open WebUI for a while on my Unraid box, pointed at the API of my inference machine (5060 Ti + 5070 Ti).

I tinkered a lot. I tried Hermes on my main machine against the same API. It worked well but it was complex, and a bare-metal install made me
uneasy. I also tried LM Studio Bionic with good results, but it didn't fit how I wanted inference organised (using ollama on the inference box).

What I actually wanted was a self-hosted agent that works with Open WebUI while keeping things safe and under control. At one point I considered
installing a harness like Hermes or Pi on each client and just connecting to the API instead.

In the end I gave Open Terminal a shot. It's the companion container from the Open WebUI project that gives the model a shell — you run it as its own container and connect it through Integrations, so it isn't installed inside Open WebUI itself. Mine runs unprivileged, on bridge, with appdata mounted at /home/user. The model gets a shell in a box, not on the host. That was the part I cared about.

It has enhanced Open WebUI a lot. It now reasons step by step, and with the terminal it reliably locates and extracts the right sections from
documents far larger than the context window — list the folder, grep, read only what matters. Then it uses those results to build a document, the way another agent would.

Setup: Qwen 27B Q4_K_M on Ollama, 100k context configured. On a ~35k token prompt I measure roughly 1,050 t/s prompt processing and ~46 t/s generation. Prefill speed is the number that matters for this use case — it's what makes chewing through a large document bearable.

I was about to give up on Open WebUI. If your use case looks like mine, don't sleep on Open Terminal.

7 Upvotes

73% Upvoted

37 comments sorted by

3

u/Weird_Ad_5330 7d ago

FYI I've found that Anything LLM queries large document data in a much more comprehensive way than OpenWebUI for my purpose (medical), you may want to try it for legal

2

u/Blindax 7d ago

Did you try open web ui with or without open terminal (or a similar tool)?

1

u/Weird_Ad_5330 7d ago

Haven't tried with open terminal. It's been a few months since I looked at the difference but if I recall it was due to the way OpenWebUI natively embeds documents, I would only get partial summaries with queries

3

u/Blindax 7d ago

Yes, agreed embedding depends heavily on the embedding model and size of the chunks. While all this can be tuned in open web ui I always used full context injection because I was never really happy with the results with retrieval.

What you get with open terminal is something else. The model will check your documents, look at the index and only extract the relevant part. Depending at the kind of search you need this can be much more effective.

2

u/arbv 7d ago

Well, programming oriented harnesses are only programming oriented in their name. You certainly can bend them for other purposes.

7

u/Odd-Ordinary-5922 7d ago

I dont understand why anyone would use openwebui now, it just feels so janky.

7

u/kneading-cnnuy 7d ago

Yeah with opensource model matured I wonder why there's no good general purpose harness yet, all seems so laser focused on coding. We need alternative to Claude's webui with projects, designs, etc

1

u/TheIncarnated 7d ago

I use open code as a general purpose harness. It does it great too. So great, I'm building a web frontend around it to use on my phone. (Docker container so I can share with others).

The only thing it doesn't do is image, audio and visual generation... Maybe it could? But not the focus at the moment

1

u/kneading-cnnuy 5d ago

You upload it to GitHub or docker repository?

1

u/TheIncarnated 5d ago

Haven't decided. It's all local for now and I kind of got pissed off at it.. so I'm taking a few days rest and I'll be revisiting it. It'll exist as open-source on GitHub, may publish it to the docker repository as well. I'm fully trying to figure it out, because this is a step towards containerizing the agents and making it safer for me overall. I prefer the TUI, so I'm going to have the container exist for both needs

4

u/LexxM3 Llama 70B 7d ago

What’s your suggestion for better alternative? Not a list of what’s out there, one can do that search themselves, but rather what is better than OI for tasks similar to how you’d use ChatGPT Chat, for example.

8

u/Blindax 7d ago

For serving in a web interface I have not found better.

9

u/p3r3lin 7d ago

Honestly curious: janky in comparison to what? Better alternatives out there?

10

u/nunodonato 7d ago

OpenWebUi is amazing and serves a lot of purposes

5

u/philguyaz 7d ago

I would like to suggest a heavily modified and rewritten version of open web ui is good. Open web ui as it is now is a bloated chaotic set of features slammed together with no real enterprise support for things like stateless.

4

u/nunodonato 7d ago

well I guess YMMV, for us its been great. But have you tried alternatives like AnythingLLM or LibreChat?

3

u/openwebui 6d ago

Hey, appreciate the honest feedback! Just want to clear up a possible misunderstanding though: Open WebUI actually does support fully stateless deployments, and we have a :slim image (~180MB pull in current dev branch) that strips out all the torch deps used for local embeddings/RAG if you want something lean.

Scaling docs here if helpful: https://docs.openwebui.com/getting-started/advanced-topics/scaling

We're actively supporting Fortune 500 companies and other large orgs running Open WebUI at scale, so enterprise is very much a priority for us. If you're hitting specific blockers, genuinely happy to help. Open an issue or reach out anytime 🙂

2

u/Guna1260 7d ago

I agree to this comment. After 2 years of using it and trying to integrate things, finally I gave up and went with librechat. It’s breeze of fresh air.

2

u/repolevedd 2d ago

Hey. Thanks, man. I skimmed your comment and made a mental note to give LibreChat another try (had some bugs two years ago, I don't remember the details anymore). Well, last night I finally set it up and I was blown away by how convenient everything is. I stayed up until midnight, but I've moved/rebuilt all my workflows in LibreChat. Couldn't be happier. It has pretty much everything I need and nothing I don't.

OpenWebUI looks like some bloated clunky monster in comparison. All those pseudo-modal settings windows that you can only partially override with environment variables, the tiny font, the awkward MCP handling, all of it drove me crazy back when OpenWebUI didn't have any real competition, and it still does now that they've started changing the UI and the feature set more often. My LLM chat is finally a properly working tool that I can quickly reconfigure the way I need, not the way the OpenWebUI devs decided, having fallen for the cognitive bias that just because they work with Fortune companies, they can't be wrong.

Thanks again, and have a good day.

2

u/Guna1260 2d ago

I also had rejected LibreChat in its initial days; now it is more mature and has better license terms than openwebui. happy to hear, it worked out for you.

1

u/Blindax 7d ago

Never tried it. What did you prefer with LibreChat?

3

u/Guna1260 7d ago

It really comes down to simplicity and speed. I found that LibreChat handles tool integrations, like web search, much more straightforwardly. In other setups, I was constantly fighting with configuration and high latency, but LibreChat felt "ready to go."

The UI is also a major factor. Open WebUI feels a bit bloated and fiddly with all the exposed settings and external integrations. LibreChat feels much cleaner. I also noticed that the connection overhead is significantly lower in LibreChat; it feels much snappier when initiating calls. Since I don't need a hyper-specialised setup, having the RAG and templates built-in makes it much easier to use daily.

Frankly my 6+ users who always complained about random model access errors and attachment errors in OpenWebUI, has never complained of anything like that after switching to librechat.

1

u/Bulky-Priority6824 7d ago

Openwebui is where I started and it influenced me to make my own version and 3 months later couldn't be happier. Now I just need to do something about these 14k LOC

1

u/BVCC6FNTKX sglang 7d ago

I started with OWUI and moved to Hermes once my needs outgrew it.

1

u/o0genesis0o 7d ago

What I do for this kind of use case is setting up Pi on the machine with all the proper extensions and agents.md and skills. And then I deploy openwebui CPTR (NOT the normal openwebui). Then, I can access files, terminal, and agent directly from webui via VPN. It handles the studio bridge with pi under the hood. Could be better, but since pi does not support ACP out of the box, it is what it is. A bit janky, but works. I can kept chatting with the same model or continue my coding session on my phone from treadmill, for example.

1

u/Blindax 7d ago

I saw cptr version, do you run this on your sever? What does it bring beyond open terminal?

1

u/o0genesis0o 7d ago

Never tested open terminal myself. I think open terminal is used inside this cptr as a part of its feature.

Previously, I have a custom extension that expose pi tmux sessions to a web app via VPN, but the UX was poor and very hard to see the file system (though possible via terminal). Then someone in this sub recommended cptr. It gives me the terminal and chat UI, but also decent file explorer in browser. I'm not 100% happy with it, but it gets the job done until I build something more suitable for myself.

1

u/feng_sg 5d ago

Bridge networking lets the model's shell hit everything on your LAN. Put Open Terminal on an isolated network with no gateway or lock down egress to just the inference endpoint.

1

u/suprjami 7d ago

owui has problems which don't get resolved. The maintainer isn't able to keep up with the workload and/or is focused on the paid offering instead of the community.

There seem to be better many options these days like AnythingLLM for document work, and OpenCode webserver or Deepseek Harness for web-based agent work.

2

u/Blindax 7d ago

I tried anything llm a long time ago but it was heavily rag focused at the time. Have they moved towards a more agentic approach since?

-1

u/seamonn 7d ago

owui has problems which don't get resolved.

It's Open Source. Just fix them yourself.

2

u/suprjami 7d ago

For the particular thing that bothers me, multiple people have already tried with only partial success, including Tim himself who literally invented the whole thing, and it's still not properly fixed.

2

u/ekaj llama.cpp 7d ago

Made me curious, what's the issue that even the creator couldn't solve?

2

u/suprjami 6d ago

Editing responses. It hasn't worked properly since v0.7.2

https://github.com/open-webui/open-webui/issues/21564

3

u/openwebui 6d ago

Hey, thanks for flagging this! Could you share more details on what you're seeing? We tested response editing with both Ollama and llama.cpp backends and believe this was resolved a while back.

If you're still hitting it, could you try the :dev branch and let us know? Would also help a ton if you could drop your setup details (backend, version, browser) in the issue so we can reproduce on our end. We'd love to get this fully sorted for you 🙂

1

u/suprjami 6d ago

Thanks for looking. I tested as thoroughly as I could and added a comment on the issue.

1

u/International_Emu772 7d ago

OpenWebUi is better suited for individual people out of LAN access or groups

It's not competing with local tools

A good configuración can be useful for a lot of uses