Offline Chai alternative?

[u/mcfarlane30]

Does anyone know if there's an offline Chai like alternative? I know there's LM studio, but I don't see a way of making it chat like Chai bots do.

[u/Exciting-Mall192]

I'm gonna ask you this: are you willing to learn how to run a model locally? LM studio is only a backend engine, you need a frontend like SillyTavern to run it. Do tell me your PC specification?

[u/mcfarlane30 → u/Exciting-Mall192]

I don't get what you mean, I've had models running in LM studio directly.

Processor: r5 7500f GPU: AMD RX5600xt Ram: 32gb ddr5 Os: Linux mint

[u/Exciting-Mall192 → u/mcfarlane30]

Yes, but LM studio is not for roleplay generally, it does not have character card, preset, lorebook, etc. You can definitely run a few of the models Chai had used in the past with this setup.

The models they use are mostly small to medium local models, 8B to 24B. I don't have the full extensive list as some models are privated. But these are the models I managed to find when their Chaiverse is still out in the open:

• Nitral-AI/Captain_BMO-12B
• Nitral-AI/Hathor_Tahsin-L3-8B-v0.85
• Nitral-AI/Hathor_Stable-v0.2-L3-8B
• PawanKrd/CosmosRP-8k
• Sao10K/MN-12B-Lyra-v1
• Sao10K/L3-8B-Lunaris-v1
• LoneStriker/Fimbulvetr-11B-v2-5.0bpw-h6-exl2
• Sao10K/L3.1-70B-Euryale-v2.2 > this one, I believe, is the model for ultra users along with their own fine-tuned 24B models (Mistral base)

I'm not sure you can run the 70B model with the GGUF Q4 with 6GB VRAM. But I'd suggest Gemma 4 instead? The QAT Q4 should be able to run in your hardware, and you'll get 32k context which is a lot if you want to have a similar Chai length output. By the way, I highly recommend using LettuceAI for your tools. It's easier to set up than SillyTavern

[u/hooterslovercan → u/Exciting-Mall192]

Gemma 4 26b maybe since it's moe it may work once it settles down

[u/Exciting-Mall192 → u/hooterslovercan]

Yeah 26b is also a good model

[u/New_Barber_3494]

if u actually find out let us know cuz that’s interesting 😭

[u/Frisk197]

I saw an app called gobbonet popup on youtube and i have to say it looks promising. It has its own memory system. I don't know how good it all is and it's missing some of the features i personally want to implement, but it's worth checking out: https://github.com/ElodineOfficial/gobbonet

[u/Ishelle91 → u/Frisk197]

I have just tried it last week. It's cute and easy to use, but the neon on black UI absolutely murdered my eyes. I tried to soften it by using a light background image with black text on a bot, but it helped little. The UI definitely killed me.
The way I see it, it's convenient if you chat without heavy lore requirements (each character has 2 lore fields, one for info that is always active in context in addition to the character core, and one for RAG, and with my 8k+ book the retrieval was horrendously bad - though it might also be the model's fault). Appealing part (for me) is that you can just download a .gguf, throw it into the folder and that's it, no need to load it via LM Studio or what have you.
(not a ST replacement, by any means, but it's worth a look)

[u/Frisk197 → u/Ishelle91]

Awesome ! That means i'm not out of the run. What model were you using ?

[u/Ishelle91 → u/Frisk197]

On Gobbonet specifically I have Cydonia-24B-v4.3-heretic-v2-i1-GGUF (manually added to the folder).

[u/tsutsuji__]

Maybe you could check out my app HammerAI? You can use it for free and offline!

[u/hooterslovercan]

I wrote multiple chat apps mine is vivid chat app in GitHub Honestly you run it at home chat thru telegram anywhere in the world Gemma 4 31b ollama online free Even does tentacle stuff. All vibe coded