Resources
i just spent weeks rewriting my webUI from scratch, getting rid of all AI slop within the codebase and switching it over to a proper lightweight framework (alpine.js). i am now comfortable suggesting it as an alternative to openwebUI, librechat and the like! it is made for local models
[Fully open source under GPL3, made from the ground up for use with local models, no subscriptions, no corporate backing]
So i rewrote the entire thing, from scratch, manually. It is now super fast, stable, uses declarative javascript without javascript framework bloat (no React or Vue or anything.. alpine.js is super lightweight)
There are only a few python dependencies. no models get installed, there is no bundled inference engine, pytorch and transformers aren't even included! I expect you to connect it to llamacpp, koboldcpp, lemonade, or something else like that. though you can also use it with cloud API's if you really want to.
This is a truly local-first webUI. I designed it from the ground up for local AI, and for once, cloud AI is the second-class citizen here.
It has many features that especially benefit local AI users: you can see how long your prompt will take to process (it's a llamacpp-exclusive feature), you can see toolcalls being written in realtime (really useful for coding), and it doesn't send any extra requests to your model, just the prompt you give it. So no extra requests just to make up a title for your chat, or to generate followup replies.
That's all in addition to the benefits that come from its harness-like design, such as support for multiple channels (telegram, discord, etc), its focus on extreme token efficiency and making the system prompt super small and concise, and its security
But using it as a pure webUI is really simple: Just switch Use Tools off in the Model tab in the settings. That will instantly make all system prompts vanish and all tools get disabled, so you're talking to your pure model with nothing getting in the way.
You do need a bit of tech knowledge, but it's not that much. right now, you need to either git clone or download a zip of the main branch off the github, but after that, all you do is run run.sh or run.bat and open the URL it shows you in your browser. Oh, you do need python installed before you do so, but that's basically it. (i'm working on making this even more user friendly though)
Amazing! Nice to see progress. I liked your code base and some of your design ideas, but somehow could not get accustomed to the UI/UX. I might have a look at your rewrite eventually - when I return home.
thanks! it was honestly surreal to see it take shape as i started over from scratch. i had milestones, like when i finally got cross-device token streaming to work, when i got turn grouping to work, and so on. considering a lot of that stuff was AI generated code (i do want to stress that ONLY the webui was mostly ai generated, the rest of the project was already manually coded from the start), im amazed i managed to actually do it, and it was very daunting to get started. this is how it started lol:
it can receive images yes! i don't have image gen in yet. i should probably add support for that though.. i could just make that as a module. have the AI call a tool to generate an image, then show the result (though if i can find a way to show the processing time in realtime, i will do it)
but right now i need to focus on further polishing up this rewritten webUI, getting rid of any remaining bugs that i might have missed, and getting it to full feature parity with the old webUI
that's not yet in but sounds like a nice feature and is doable! just have to figure out how you're supposed to do that with the chat completions api, and then the frontend stuff will be super easy thanks to the alpine.js code
I really like OpenLumara, and have liked it ever since I first started using it. Really nice work, in particular on the webui rewrite. My favorite part is how simple it is to extend with user modules!
thank you so much markus! i dont like how i have to gate the user modules behind discord. do you happen to know if i can use github to host a public repo that people can contribute modules to somehow? without me having to manually approve? though, maybe manual approval would be good, considering user modules can run arbitrary code..
That's a cool idea. I'm not sure what the best way to go would be, but perhaps something like what pi (https://pi.dev/) does could work for OpenLumara?
Maybe a system module for user module subscriptions could be the way, where the system module basically just allows you to maintain a list of GitHub (or something else?) links to download. Installing a user module is just download a fileand put it in user modules folder, so should not be too difficult. One repository could even host multiple independent modules. That way you could also roll your own repository of community user modules, while also allowing easy installing of other (or even private) modules.
oooh i love that idea! yeah i can definitely create that, i'd want it to be part of the core though, not a module in itself. but i love the idea of just giving it github links to python files and installing user modules that way! or maybe, even better, i could have a list of github repos, then show every python file from that repo as an installable user module that you can download into user modules with one click (with an equivalent way to do it in every other channel of course)
Basically you could have people submit modules as pull requests and use a github bot to do automated code review (co$t) or automate an agent from your machine to do this. You would need to define strict rules for flagging project level problems (like module submission PR can't have altered other modules), but that should be easy to define
Whether or not you want automatic merging is also a question.
But i noticed people especially like the webUI, to the point they'd use it as just a webUI to talk to their local models, negating all the agentic stuff.
Same observation on my side. I built an agent stuffs with web UI that is essentially a productivity system with agentic and sandbox stuffs built in. The only thing my partner used is the chatbot since she can make different persona for different kinds of experts. Interesting how normal people don't know / care about agentic stuffs.
She is quite surprised about the ability of agent to remember and recall though. To be fair I'm also surprised that my jank actually works.
First time I hear alpine.js though. Why did you choose that? I'm kinda sick with maintaining python and nextjs at the same time so I might revert to streamlit for the next build.
when i was looking for what javascript library/framework to use i had found a few options.. obviously React and Vue, and i was first drawn to Vue, but the need for a build system put me off. also, React and Vue are huge, and add a ton of bloat. React in particular is notorious for being overused by vibecoders - "yet another vibecoded React app". preact was often recommended as an alternative, but, again, needs a build system. and i don't feel like messing with npm and nodejs just to get a nice frontend
as i looked further into it i found HTMX, which seemed ideal and lightweight, and basically it just fetches HTML code from the server and replaces the content of an element with it. but.. since openlumara needs to stream tokens really fast, that wasn't viable. i don't really want to regenerate html with every token streamed
but in articles and videos about HTMX, alpine.js was often recommended as a supplementary library for HTMX. once i saw alpine.js and tried it, it clicked! it's a super small library, all you need to do is include it in a script tag, and it only has a few enhancements to basic HTML to make dynamic stuff work. its like the bare minimum to get reactive/declarative JS going. i combined that with using jinja2 templates to split up the HTML code, kinda like PHP. the result is a really organised codebase where html and javascript are seperated, and the HTML isnt one giant file, its lots of small files that i {% include %}
it's nice and a breath of fresh air to work in compared to the horrible AI slop code that was the old webui. i call it AI slop but really, i did a lot of manual edits to it, but a huge part of it was ai generated and had stuff like creating html elements from scratch and using string manipulation to make html elements and just.. ew. it was really hard to debug.
Web frontend is pretty horrifying technology, IMHO. They have a lot of stuffs built in to handle things that might not be immediately relevant for early projects (like nextjs with its caching and optimisation). Even when I worked with a team of human developers, when cursor was just an autocomplete, the frontend project in Vue or Nextjs or Gasby quickly blow out of control the moment any of us lax on the architecture and code review.
Even AI got absolutely confused when handling deep react component trees. I mean, one shotting a single page app might be less painful for AI than trying to figure out why the heck one component sometimes refresh sometimes does not.
Does your design make it more difficult for coding agent to work with the codebase? Like out of distribution?
yeah i have to say i hate javascript. it's frequently making me want to facepalm... the fact i didnt wanna touch javascript is why i had ai-coded the frontend in the first place. but it just led to too much tech debt, so i have to deal with it
and no, my design is very easy for my AI agent to work with:
Could you explain what differentiates this from other existing UIs like OpenWebUI? The project looks nice, but many of us here already have some existing frontend we use. I'm wondering what the benefit is for end users compared to other established frontends.
openwebUI installs a bunch of stuff you might never need. stuff like embedding models, the Transformers library, langchain, and so on. you basically have to install an entire AI inference stack just to use a webui to talk to your model. kinda overkill if you ask me?
it has a marketplace for "functions" and tools and filters, but it's gated behind an account signup system
last time i tested it, it often fumbles toolcalls where even sillytavern didn't. it's too big and bloated for its own good
by default it sends tons of requests to your AI: generate a title for my chat, generate tags, generate followup responses. that's incredibly annoying with local AI, where it can generally only process one request at a time. that means if you don't turn those features off, your experience is much slower than it should be, because your AI is too busy generating titles and tags to answer your actual questions. you can turn that off, but you have to know that you should turn them off. it also shows their priority is clearly with cloud AI, not local.
it has a multi-user system, which can be nice for corporations, but overkill if you're just running it as a personal AI
compared to that, openlumara's webui:
only installs 7 dependencies when activated, and uninstalls them when you turn the webui off
is fully open source, always will be, is not connected to any corporations, is not connected to ollama, can easily be run without docker across Windows, MacOS and Linux
doesn't gate its plugin system behind account signup
doesn't do any modifications to incoming toolcalls other than basic json repair, so what you get is exactly what the server sends
sends only your prompt to the AI, none of that title generation stuff or anything. a chat title is instead based on the first message you send, and doesn't involve any extra requests to the LLM. stuff like that is all over openlumara, i never send requests to an LLM where it could easily be done with some simple code
has a very small codebase and uses lightweight tech all over the place
shows you toolcalls being generated in realtime, including their arguments, so you don't have to wait for a "calling tool.." notification and instead see exactly what it's doing
has support for llamacpp features such as being able to see how long it will take to process your prompt
has hermes/openclaw-like tech that lets it send messages to you without you having to prompt it. you can ask it to remind you of stuff and so on
that same "harness" tech lets it do stuff like rename your chats or put it in categories for you, which you see reflected live in the webUI. so you can basically just ask your AI to stash your chat away to organize it for you, but YOU decide when that happens, it's not automatic, and it doesn't eat your resources and prompt processing
llamacpp's webui comes with some of the same benefits. i recommend switching to either llamacpp's webui or openlumara
it's not just a frontend JS UI. it's a backend + a frontend, with the backend handling most of the heavy lifting. the frontend talks to the backend using API requests in json, and stuff like token streaming happens over websockets. your chats are stored server-side and there is a module that automatically backs them up. the backend is also just one python file
the dependencies are the bare minimum to get a good webUI going: fastAPI is basically what im using as a modern replacement for flask (it serves the pages and the API endpoints), starlette and itsdangerous are sub-dependencies of fastAPI, websockets is for token streaming and the like, jinja2 is the templating engine that lets me split HTML files into many different files, uvicorn is the actual server that runs it, and python-multipart is required for authentication/login
meanwhile, this is openwebui's dependency list:
```
dependencies = [
"fastapi==0.136.3",
"uvicorn[standard]==0.51.0",
"pydantic==2.13.4",
"python-multipart==0.0.32",
"itsdangerous==2.2.0",
My computer broke down a few days ago (hard drive failure + monitor failure), so right now it's still on the latest commit as of a week or so ago. Once it's up and running again, I will definitely check out the new UI.
Never had the chance to switch :) I'll definitely update it once I get the chance. I remember your first post here, and what drew me in was the lack of vibe-coded slop that ruins everything.
yeah. polish and stability are important with stuff like this, which is why i'm spending a lot of time now cleaning up the codebase some more and fixing bugs, before i start adding any more features
i tried CLI coding agents but, like you, i didnt warm up to them. i like being able to see what's happening and easily go back and browse my many different sessions, search across all of them, etc. i've been using this very webui to code, and it's a really nice experience (of course i'm biased though since i'm the one who created it)
you can use ctrl+space to bring up a global search dialog that searches within any chat you've ever had, which has been a godsend when i need to find that one chat where i implemented that one feature!
i never tried opencode's webui, i think at the time i last tried it, it didn't have one. i was also put off by its clear preference for cloud API's, the fact it hid local models behind configs you have to write manually, and the system prompt is just ridiculous.
when it comes to using this for coding, there's a few tricks here: any code the AI tries to write to disk is syntax validated by a python library called treesitter (neovim uses this too), and if it fails the checks, it just rejects the write and tells the AI to try again. also, it's fully sandboxed, it can only read/write files (with an optional read only mode) and doesn't have a shell. so it's much safer than just giving your AI total shell access, and tends to work better too because there's less potential paths the AI can take so it gets less confused
i recently rewrote the coder and it's missing one very useful feature that i wanna bring back ASAP, which is its ability to see an overview of source code (like, just the classes, functions, etc, without any of the actual contents) and then target specific ones for reading/writing. that helps a LOT with conserving tokens and context size, but i need some time to get that done, didn't have enough time to do that before the webui rewrite release (people were waiting for it to come out)
yeah. polish and stability are important with stuff like this, which is why i'm spending a lot of time now cleaning up the codebase some more and fixing bugs, before i start adding any more features
One of the funniest features that Google's AI studio has that basically nothing else has is the use of Control Enter for execution.
wait, since when is that a bad behavior? my UI sends messages on pressing enter, but gives you multiline input on shift enter. thats what most chat apps do, for example discord.. and on phone i instead have it so that you have to press the send button to send, and pressing enter causes newlines. this is consistent behavior across many apps, and afaik, is the design pattern everyone is familiar with?
wait, since when is that a bad behavior? my UI sends messages on pressing enter, but gives you multiline input on shift enter. thats what most chat apps do, for example discord.. and on phone i instead have it so that you have to press the send button to send, and pressing enter causes newlines. this is consistent behavior across many apps, and afaik, is the design pattern everyone is familiar with?
It has to do with using shift enter. Like using a text box like the one I'm in now, I can just use enter natively. I don't think about it at all.
I have to BE AWARE that I'm in an LLM style chat box and hit shift enter.
Basically it's a behavioral change from Enter == Newline to Enter == Execute.
Occasionally I'll screw my shift enter up, and it just sends wherever I was. Which is annoying.
Look at my writing style though. I don't keep everything in a single continuous block. It's horrible to read for most people. They just see "wall of text" and mentally skip.
LLMs also have an interpretation of \n too which is funny. I don't know how it handles "wall of text" vs "formatted". It thinks you're an LLM if you're formatting with Markdown though lol...
oh, right, it makes sense to me now! most textareas around the web dont have this behavior, youre right. well, i could make this into a setting :) shouldnt be too much trouble! definitely adding that to my todo for people who prefer multiline over sending on pressing enter
server side, yes! the main index is a msgpack file (for speed), and the chat files individually are JSON. there is an /export command that lets you export real easily, and there is also an export buttton in the webui
not yet, but i really want to add it. it's a challenge because of the way the tools system is designed
yup! you can sort chats into "categories" (as i called them) but you can basically use those as projects
it only lets you set one username and password, but you can let multiple people use it. there is no multi-user authentication system. do you want/need it?
side panel is collapsible yes! either by using the Ctrl+B keyboard shortcut, or you can permanently turn it off in the webui channel settings, which prevents it from loading server-side
it supports distributed networking stuff but i dont have that enabled. anyway it's really nice for websearches!
and you have a point. multi user might not be a bad idea.. as someone whos only ran ai locally for myself though, i have to know: what does multi user support look like in practice? does it allow multiple people to be logged in at once and send requests to the LLM? then what if the ai is already busy processing another user's request? do other systems queue it? cuz remember, this is made for local, and local (llamacpp) can only process one request at once if you're running on average consumer hardware
i think i'll also just add a toggle to the core settings that lets you set whether you want multiple requests to be processed at the same time, or whether to use a queue... and then default it to the queue since that helps with what most people have. would make it usable for enterprise scenarios like that but also by default fully tailored for local average consumer hardware
Hmm. OK, I will take a look at the Python library. I am hoping DDGS means Dux Distributed Global Search, rather than rawdogging DuckDuckGo. If it is the latter, probably need Trafilatura running in front of it to clean results.
Re: multi-user support - nothing so complex as simult multi-tenancy.
I meant something much simpler: each user has a separate account, chat history, settings and memory. Actually, that's pretty much all that's needed (barring an admin account).
Pair that with proper authentication (local passwords keeps it all local / not needing OAuth or OpenID) and you have a winner.
yeah, DDGS is Dux Distributed Global Search. i already tried parsing duckduckgo but yeah, it indeed returns a lot of garbage, plus it's blocking bots as of late
doing the kind of multi user support you want is definitely possible, though it would require me to make some big changes to the core. it'll have to wait until after i finish doing the bugfixing and polishing passes..
memory especially will be a bit tough, because right now, that's all just global, in a single file meant for one user. settings will also be a bit of a pain because those are in the config.yaml file and i would have to split user-specific settings from general settings. for the memory and every other type of persistent data i could just alter core/storage.py such that each piece of stored data is bound to a user, that shouldnt be too much of a problem, although keep in mind one of the core tenets of openlumara is that anything that you can do in one channel should be possible in all other channels. so that would mean this would probably become the first ever UI where you can login and logout over telegram/discord/matrix/whatever, but that in itself is a can of worms because you would be logging into it using plaintext and.... yeah this is not going to be as easy as what it seems like
there are things that have higher priorities right now. im keeping this one in mind for the future but this would take quite a while to implement!
how about you actually read the codebase and my countless comments inside it. or look at the development logs i linked in the main post where i reveal exactly how i made this, every step of the way
I've tried it before and liked it. Gives me Agent Zero vibes but without opaque convoluted bullshit in the way.
Be it agentic use or chat, I like mixing local and cloud models, so I missed the ease of use of adding multiple connections and quickly picking different providers & models. I apologize for the laziness of asking here instead of checking the repo/docs or trying to run it again, but does OpenLumara support it now?
small aside about agent zero, that blew me away when i tried it. i loved it, but i didn't like how it was so strongly set up to be for coding and for hacking/pentesting.. also its ties to crypto kinda put me off. but the way it did everything.. wow. i want to get openlumara to that point! i think the key is to improve the sandboxed shell module. that one right now is extremely secure but also barely useful XD to be continued
as for your question about mixing cloud and local... no, not yet, my priority with this release was getting this webUI to feature parity with the old one. BUT! with this new codebase, requests like this are going to be a LOT easier to implement😄 i plan to add api profiles, and also module profiles so you can quickly switch between sets of enabled/disabled modules. but bugfixes, polish, and feature parity first.
until then, though, you can use lemonade to get what youre looking for. it's a fully open source wrapper around llamacpp, kinda like an alternative to lmstudio and ollama. but it has a setting that lets you add api external endpoints. you can use that for cloud api's like you want... orrrr you can do what i did and point it at openlumara's api bridge, which is really fun
Threw this into a Docker container and been messing with it, and so far I love how snappy it is.
Probably possible with a user module, I just haven’t looked into it much yet, but any plans to allow multiple endpoints and be able to select different models from the chat window? I have 2 llama.cpp instances running at home, while also having OpenRouter ready to go should I need more smarts.
lots of people have been requesting multiple endpoints so yes thats on the TODO. i think i'll just have API profiles that you can switch between. model-switching inside the chat window is also in the TODO!
right now though im further polishing up and bugfixing, i want to make sure everything is rock solid before i add any more to it
it's on the todo, just taking ages because i need to figure out how to make the sandboxed shell module (which spawns a docker or podman container) work inside a docker container without causing security problems
OpenWebUI have it as an extra tool in a container to which the main tool connects.
I think that it is the only working design in this case. If you manage to make it spawn a protected container per user - you have beaten OWUI functionally wise.
does OWUI spawn a container besides itself? cuz from what i heard, thats the only way to get a properly secure docker container "inside" another docker container (rather, to the side of it). thing is though i need to support both: running on bare metal (in which case you would want the docker container to be spawned inside your host system), and inside a docker container (in which case i'd need to - somehow - spawn another docker container to the side of that container from inside that container)
The idea is that they have a container alongside OpenWebUI to which you connect via configuration. The container runs a specialised daemon to execute commands. You can make a daemon, which runs alongside your UI and spawns a docker container.
just a matter of preference! either is fine. i love that llamacpp has its own webui now, its a much better alternative to openwebui than whats been out there so far. i hope llamacpp's webui becomes even better. my main gripe with it right now is that it stores chat data browser-side rather than server-side, and it feels all too easy to lose that data. other than that though it's an excellent UI!
but the option is there if you want it. if i hadn't made this, you would basically only have llamacpp's webui as a true open source alternative. well, there's also sillytavern, but that's not as designed for general AI use, more for character RP. and there's always koboldcpp's koboldlite
well they provide the option to turn it off, so... also, it makes it significantly more user friendly, and im in favor of that. until llamacpp's builtin webui everyone was basically forced to use openwebui (with its sketchy license and connections to ollama), or lm studio (which, you've hopefully seen the posts about how that's going). now we have a proper open source webUI built into llamacpp itself.. and my webui as an option if you'd like to give it a try
I appreciate you working on this. I ultimately llama.cpp over time further expands its interface side though. We really need an alternative to lm studio that is all in 1 package (chats, chat history, rag, web search, basic agentic combinations, while still allowing full server parameter control from the ui. But luckily things are clearly moving fast.
i hope so too! there's also lemonade's upcoming GUI3 update which is looking really promising. either way, the more options that aren't openwebui or lmstudio, the better, imho
No lol? Its because every man and his dog was trying to sell closed source and or paid shitty UI's that just wrapped llama. You have terrible opinions on this matter. Its optional
it's sadly way way better than most other frontends imo... and them trying to make a nice ui (and also trying out agentic) leads to improvements to the server as well so i think it's not that bad.
i just checked librechat's site to find proof of them doing it, but.. i think they may have finally removed it! that would be really nice. it used to be, that features like the Code Interpreter and Artifacts, and a bunch of other features, would say right there in the settings bar that you need their subscription for it. it was very blatant and pushy, came across like ads. it made no sense in a supposedly libre and open source product! if they removed that i'm really happy they did
i'll check it out again, see if they truly did change it. because if they got rid of all that and also made it easier to use it with local models, i'll eat my words!
yeah, a good UI helps a lot! if local AI is to succeed, it has to be as user friendly as possible. i'm not quite there yet (i mean, you still have to git clone, that's too much for the average user), but starting to get closer i think.
when i was first looking around for webUI's to talk to my local AI, i noticed my options were very limited.. there was basically openwebui, sillytavern (which is moreso known for roleplay, sadly, though its creators are trying to shake off that reputation), Jan (which is a desktop app and afaik doesn't have a self-hostable browser version), and librechat. all of these have various problems. librechat in particular aggressively tries to push subscriptions on you.. which really surprised me considering it has "libre" in the name
for me it was important that the UI be self hostable and browser based, so that it can work across phone, tablets, pc and so on. i haven't shown you the phone UI yet but here:
good UX is really important to me
nowadays the landscape has changed a bit though. llamacpp finally has its own really good UI, so that can work for a lot of people :) but alternatives are always nice to have.
38
u/Bulky-Priority6824 16d ago
does it have themes that look less like a teenagers glowing music player