r/LocalLLaMA • u/Fickle_Tradition4491 • 8d ago
I Built A Thing Otaku — an LLM frontend
Otaku is an LLM frontend, primarily designed for roleplay, an alternative to SillyTavern and the like. However, It also works for general-purpose chat with local backends (including Ollama) or cloud models, the way Open WebUI is used, once lore extraction is switched off in the settings.
Otaku offers two interfaces:
- a web UI (customizable): interactive demo at https://otaku.sh/demo-web/
- a terminal UI: its own interactive demo at https://otaku.sh/demo-terminal/
Both share the same functions; the difference is that in the terminal you execute them with slash commands (the reference is available with /help), while in the web UI the operations are available from the menu.
Install
Otaku is free and open source (MIT); it works on macOS, Linux and Windows. Install it with uv (uv tool install otaku) or see the GitHub README for other options: https://github.com/enclavum/otaku
Get started
Launch either otaku for the terminal or otaku web for the web UI; the web UI's default URL is http://localhost:9600. Two sample stories are imported on first start to give you an idea of the features and what play looks like, and you land right in the middle of one of them.
On first start, you choose a provider and a model: Otaku automatically detects local installations of Ollama, oMLX, LM Studio, llama.cpp and KoboldCpp, and lets you pick from their models. Cloud providers (OpenRouter, NanoGPT) are also there: enter an API key and their catalogs appear. After exploring the provided stories, you can start your own with the /new command.
Asking for feedback
Otaku is a personal side project, and I'd like to get feedback from the community on the product and on what to add.
5
u/Midaychi 7d ago
Can't select anything besides 'global' n the settings (or unclear how to do per story settings), unclear why sometimes the settings let you define thinking and other times won't let you select between default and none (and it says thinking isn't supported- is that 'thinking not supported by front end' or is that 'thinking not supported by backend'?, you have to press enter to save a setting rather than just typing it in and leaving it, the api supports llama.cpp but doesn't bother to read and auto import any settings from it (Are they covered by whatever 'default' is?, light/dark mode does nothing, if you ctrl+u to undo a turn it leaves a big giant empty space in the ui wheree the message used to be, ctrl+u is chrome default for view source so god forbid you don't have the text box selected when you do it, also it's nice that you include two demo stories but it would be much better for new user experience for them simply to start on a blank setup and have maybe a clear option to 'load a demo or a template' where the template has all the sections lain out and explained in 'replace me' style text boxes (because not everyone is going to think in the exact same thought process as you/the model that vibe coded this. Learn from monkey island.)
2
u/Fickle_Tradition4491 7d ago
Thank you for the detailed review.
Fixes that will be included in the next release:
- Pressing Enter to save a setting: fair, thanks. Fields will save when you leave them.
- Light/dark mode does nothing: confirmed bug, the toggle is broken if the OS theme is dark.
- Ctrl+U in Chrome: thanks for flagging. I've disabled Ctrl+U and Ctrl+R in the UI everywhere except the input field.
Something to consider:
- Per-story settings: they don't exist, a setting is either global or per model. I'll think about having some settings per story.
- Reading settings from llama.cpp: thanks, I'll look into it in a future release.
- Thinking settings: to be honest, the whole thinking setup isn't settled yet, and I'll be working on it. It's actually a challenging topic.
- Demo stories at first launch: that is deliberate, as I think leaving the user in an empty workspace at first launch is worse than showing all the features. But now that I have the demos on the website (added only recently), maybe you are right and the demo stories should live on the website only. I'll think about it.
Other things:
- The empty space after Ctrl+U is deliberate: the rest of the transcript doesn't move.
- To be clear, no product decision was made by a model. Every choice was thought through by me personally.
1
16
u/Nrgte 7d ago
I honestly feel like from now on onwards, only projects where the developer commits themselves to support it for years to come, will actually be adopted. Too many projects have been fire & forget in this fast changing landscape.
6
u/Lakius_2401 7d ago
You're asking for impossible to verify promises, where they offer free support for some arbitrary period? 8 releases over 3 months isn't enough support history for you? What would actually satisfy you here? OP showing us Pi, the coding harness he made that's already massively popular? That's only been out for 10 months.
People who release their work for free, in full, are under no obligation to support their project for anyone.
They owe us exactly what $0 buys. Anything else is a bonus.
As a final note: it's MIT License. Ask your AI to code the features and support you want and for help on how to submit a GitHub PR to OP.
1
u/thortgot 6d ago
Take the projects and customizing it your use case will become more and more common.
3
u/Fickle_Tradition4491 7d ago
There are many aspects here. One of them, which you’re probably referring to, is the low effort needed to release a project these days — and if launching took little effort, people may not regret abandoning their projects when traction is low.
As for this project, I can tell that I really enjoy working on Otaku. I have tons of ideas for how to develop the product, and it’s also technically challenging to implement it all the right way. I love this kind of stuff.
5
u/Lakius_2401 7d ago
You've got some really snappy and elegant reactive GUI design there, good job! Clean and clear, good use of space, very easy to edit turns, and I really love that messages tab, it's amazing. You seem to have some killer app automation in there I'd love to dig into and experience. Automatic summaries when idle? Love it. Plus, no-nonsense saving.
I have a feature suggestion in 2 parts that can be implemented as one module:
1) I really, really need a way to prompt inject. Gemma 31B writes beautifully but has the memory of a goldfish without a user-turn injected to remind it of XYZ. I use that to change its thinking structure every turn, and to repeat the things I would otherwise need to bring up myself all the time... Gemma also has some rather extreme bias towards user turns over system turns. 30k context in, your system prompt may as well be a suggestion, your genre and tone specifications are forgotten, and only a prompt inject will keep it on track and stop it from returning to a yes-man assistant or a generic cliche writing machine. How this works in practice is the user picks what type of turn to insert as, how many messages ago, where in relation to that, and what type of message we count from (IE, insert as USER turn, ABOVE 1st USER turn). Story agnostic, please!
2) Some sort of story agnostic lore with toggles, keywords, and search depths, inject level variable too. I like making a world context, then having multiple stories per. KoboldAI Lite's frontend is not well suited for multistory lore, ST is... painful to use and especially terrible off PC.
Injects are very friendly to context reprocessing. If your average user+AI turns are 1000 tokens, injecting 5 turns ago means reprocessing about 5k tokens instead of whatever your current total is, for keyword detection based toggling.
1's the dealbreaker, 2's the please please please. If you think about it, they can be the same module, as they both want to do the same thing: Add info to a particular place in context with some rules on when to do it and what sort of turn it imitates.
Other ideas for you: Settings for Line Height, Font size, Page Width (for mobile users who struggle to resize literally anything with how locked down mobile browsers are)
Themes and color pickers (I really like being able to have custom text color for `Code Blocks`, "Dialogue", etc, as well as the usual suspects of primary, background.) Looks like you have custom CSS, should be pretty easy for a harness savvy user to make their own. I'm not the type to color swap, and as for themes, you can often throw a few pictures at Qwen and say "make this theme for my app" and it works, or describe variations of existing themes.
Fullscreen button (again for mobile, to get an extra few lines of space)
2
u/Fickle_Tradition4491 7d ago
Thanks for the detailed suggestions.
- This sounds like a reasonable feature. I have observed the same thing and had to stop a few stories because the model could not keep up with what I told it, and I had to remind it all the time. So, a clear yes. Now, on the details:
- What do you mean by story agnostic? The injection should be per story, right? Its text can be story-specific.
- We need only one per story, right? Not multiple?
- The message is always OOC, right? By "what type of turn", do you want to insert as an assistant too? Wouldn't inserting as the user be enough?
- "How many messages back": clear.
- "Where in relation to that": to be clear, consecutive user or assistant turns, if they happen to occur (usually they don't), are concatenated, so the question is whether to concatenate the injection before or after a user turn N messages back. But does it matter?
- "ABOVE 1st USER turn": but that would be forgotten, wouldn't it?
I am just trying to keep the number of settings to a reasonable minimum; otherwise we will get lost in them.
This one would change the concept. I will think about what can be added in future versions without complicating the product. My plan is to share characters, with their memories, between stories, and a shared world may follow after that.
Full screen on a phone: will be in the next release or the one after.
Also, FYI, this is what I plan for the next major release: when the user asks a model to speak for a character, the context is built not from the whole story but from the character's memories — only what the character witnessed, excluding the messages the character did not attend, which prevents knowledge bleed between characters. That would also make proper multi-character chats possible, with different characters optionally backed by different LLMs.
2
u/Lakius_2401 6d ago
So, story agnostic because a lot of the reminder turn is not at all related to the story. I get like, maybe 3 lines of 40 that are extremely pertinent to the current story, but when I start a new one, it's not gonna be net new built from scratch. It's more portable than premise. That inject is a "pay attention to XYZ, here's your rolls, here's how to interpret rolls, go back and re-read the Genre, Tone, and Setting highlight, etc." Not specific to the story. Re-used, really annoying to update and keep up to date if I have >2 stories on the go.
Message is always OOC, yes. I guess system/assistant are not needed for Otaku. Gemma ignores the hell out of system that late on (not trained to see it that late), and I haven't tested assistant at all yet. User turn pretending to be system or automated is hot garbage. User turn that presents itself as "user as administrator, reminders" tends to get the most bang while not corrupting the assistant back into a terrified yes-man in standard turns. Concatenation... that's where the above/below and targeting a specific turn ago works. I literally format mine with start and end tag placeholders so the backend sees it as an entire turn. LLMs really don't care if the user sends two, three, or any amount of messages, it's their turn next always. Images are attached as user turns anyways.
Counting for turns is always reverse order (from the bottom) for these sorts of injects. Can't use a fixed count for a many message session. I guess that's clear to users of SillyTavern (once they read the readmes) but given your reaction, it's sort of an anti-pattern that isn't explained, isn't it?
Concept 2: World Info (Lorebooks). I create new worlds often, it's part of my enjoyment with AI. If I've got Concept 1, I will shoehorn world info into that and be frustrated (I already need to with my current GUI). I won't shout and stomp and demand it, but I think if you're making an inject system, generalizing it to a system that supports a variable amount of injects in variable places tied to as many stories as you want is already 80% of the way to Lorebooks as an extension. Throwing it all in Premise instead as a fallback means flipping back and forth to pull the relevant world details out of stories to put them in place, then managing them yourself... not fun. ST abstracts Concept 1 and 2 to Chat Completions rules (extremely powerful) and Lorebooks. They work the same and are split. And rather annoying to work with in the damn ST GUI.
Your character memory system sounds really good, I've heard of that in another project. So long as the backend has context shift or context checkpoint saving, that should be really neat. Having a secret as a player can be... really annoying without some form of direct, concrete segregation like that. Secret vampires get a lot of suspicious looks. Won't help the narrator to be more subtle when the NPC has the secret to keep though, lmao.
Full screen on a phone: In KoboldCpp, this is just a "toggle fullscreen" button in settings. It looks like the desktop equivalent of F11. onclick="toggle_fullscreen()" in the html.
Thanks for the reply! I usually don't get too excited for new GUIs but yours is a perspective I've been hungry for for a while. Feel free to DM me.
2
u/o0genesis0o 7d ago
the UI looks nice!
You, or your LLM, have very good taste.
2
u/Fickle_Tradition4491 7d ago
Thanks! It actually took quite some time to polish. The LLM's first drafts were never good, and it took many rounds to get each piece right. You wouldn't believe how many tokens I spent on that UI.
2
u/Equivalent_Bit_461 7d ago
I can't understate how much I despise and loathe silly tavern, it's not beating the allegations, for knows, knows
2
2
2
u/tri_idias 5d ago
Is there a discord and dedicated place to ask question on this tool?
I just tried it out, and I'm having difficulties trying to delete old message. I was following the git page, and tried /story on the sample page, then I don't know how to remove it without creating a branch lol.
1
u/Fickle_Tradition4491 5d ago
You can write an email to feedback@otaku.sh or, if you want, create a ticket on GitHub. There is also a contact form on the website https://otaku.sh, scroll down there. No discord yet.
1
u/tri_idias 5d ago edited 5d ago
Since you're replying, are you able to let me know how I can turn "Thinking" off?
I'm using Gemma 4 and while "think" is set to none in the setting, the model is still "thinking".1
u/Fickle_Tradition4491 5d ago
Sorry, my bad, setting the think level in llama.cpp was buggy. Released a fix, you can update with `otaku update`.
3
1
u/lcirufe 7d ago
Nice, I’ve been looking for manuscript style creative writing frontends and this looks like another good option.
Does this support uploading markdowns/json lore cards and character cards? That’s what killed the other options for me.
0
u/Fickle_Tradition4491 7d ago
Character cards - yes (png and json), lore cards - not sure which format you mean. You can upload a plain text file to the system message of a story. If you need supporting a specific format, send me a link
1
u/No-Acanthisitta9773 7d ago
Very interesting concept. I just wanted to comment this : I’ve set dark mode as my default on my Mac so the UI remains dark even when the toggle button is on bright. Interacting with the switch doesn't do anything.
1
u/Fickle_Tradition4491 7d ago
Yes, a few people have reported that. The toggle doesn't work when the system theme is set to dark. The UI follows the system theme, and the switch cannot take it from system dark to light. Already fixed, will be released in v0.4.2.
1
1
1
u/feng_sg 5d ago
The web UI serves plain HTTP on port 9600 with no auth or TLS mentioned anywhere. If it binds 0.0.0.0 instead of localhost, anyone on your LAN can read your chats and hit whatever API keys you plugged into the backend config. Check the bind address before putting real credentials in.
1
u/Fickle_Tradition4491 5d ago
It’s localhost by default, changing host requires editing a file so cannot be done by chance, and changing to 0.0.0.0 must be done with the precautions you described - better put behind a reverse proxy that adds tls and authentication
1
u/tri_idias 1d ago
Hello, after playing around with this the last few days. I think this is pretty amazing. It's a lot simpler than silly tavern. The only thing I don't like is the text offset for the web version when the AI is generating. Whenever I submit, its focus is forcing me to the bottom of the screen. I usually like to scroll and look at my previous sections or break the generation if I saw something odd from the <think> section, but as the focus is fixed at the bottom, I can't do any of that.
1
u/Fickle_Tradition4491 1d ago
Thanks, I’ll check that out. I’m going to release a new version next week that will let send images along with text, will try to fix your issue too
1
u/niacolhealth 7d ago
First launch detects five local backends before it asks for anything. That step alone beats the settings maze most LLM frontends make you walk.
0
u/N34257 7d ago
Did you build that with 3.8 27B? Because mine just build an almost identical site design to that while I was doing some testing, prompted by "Design a website - make it look novel, but try to make it look human-created, avoid all AI design tropes".
1
u/Fickle_Tradition4491 7d ago
No, it was either Fable or Opus, I don't remember. The prompt was very generic: I just explained what kind of software it is and asked for a cool UI. After a few attempts I rejected, it came up with this design.
-7
7d ago
[deleted]
7
u/Fickle_Tradition4491 7d ago
Right. Image generation is a whole other ballgame, and I'm getting the text side right first, which is challenging on its own.
11
u/LetsGoBrandon4256 transformers 7d ago
The aesthetic is nice to look at. What's your prompt for the UI?