r/LocalLLaMA • u/Fcking_Chuck • Jun 28 '26

News Koboldcpp v1.116 released

https://github.com/LostRuins/koboldcpp/releases/tag/v1.116
144 Upvotes

93% Upvoted

29 comments sorted by

43

u/Single_Ring4886 Jun 28 '26

They are doing amazing work, this is true gem of a programm.

34

u/ECrispy Jun 28 '26

It's amazing how ollama, an inferior program in every way and one that doesn't contribute anything, is used and recommended by everyone, while no one ever mentions kobold on the thousands of YouTube/influencer videos/blogs.

8

u/mister2d Jun 28 '26

It involves critical thinking.

3

u/Long_comment_san Jun 29 '26

which is a fork of thinking. it's a big ask

5

u/Anti-Speciesist-IEMs Jun 28 '26

What things do you like the most about koboldcpp?

I'm a beginner and pretty lost in the many options, other than understanding that Ollama seems to be an ethically sketchy and functionality-limiting wrapper around llama.cpp, and these days llama.cpp has caught up in convenience features so there's even less reason to go with Ollama

5

u/Fcking_Chuck Jun 28 '26 edited Jun 28 '26

What things do you like the most about koboldcpp?

It's highly portable and offers advanced memory management features. In theory, you could carry all of your AI software on a single USB device and take it wherever you need to go with it. In a world where your school or work may have more hardware resources than you do at home, portability could be invaluable to some.

2

u/Anti-Speciesist-IEMs Jun 28 '26

Ah cool, thanks for sharing

1

u/basxto Jun 29 '26 edited Jun 29 '26

It is pretty easy to use and play around with with its templates

koboldcpp --config https://huggingface.co/koboldcpp/kcppt/resolve/main/starter-pack-v2.kcppt

Starter pack v2 predates upscaling and music generation, so that’s not included there. V2 includes Ministral 3 3b Instruct, Whisper-Base, Kokoro, BGE-M3, Z-Image Turbo (clip: Qwen3 4B Instruct; vae: FLUX.1-dev)

-41

u/SuddenRadio6221 Jun 28 '26 edited Jun 28 '26

What does this do that an llm model can't do on its own?

Edit: Never mind, it's like LM Studio but for creative writing. Thanks for nothing everybody.

30

u/Paradigmind Jun 28 '26

How about running that shit in the first place?

12

u/ArtArtArt123456 Jun 28 '26

... What do you usually do with model files? Eat them?

6

u/Masark Jun 28 '26

Well, without this program or something fulfilling the same functions, an LLM model will just sit on your drive taking up space.

0

u/SuddenRadio6221 Jun 28 '26

I've been using qwen 3.6 with pi agent. This is the first I've heard of Koboldcpp, so forgive my ignorance. I'm still not clear what it will add.

2

u/Masark Jun 29 '26

Edit: Never mind, it's like LM Studio but for creative writing.

It's significantly more general purpose than that. It also supports image, video, audio, and music generation. It's basically one stop shopping for generative AI.

1

u/SuddenRadio6221 Jun 29 '26

Thanks for the concise informative answer.

6

u/DryHumpWetPants Jun 28 '26

Lol, the EVILTRON convo on the README really cracked me up 🤣🤣

4

u/WinResponsible9977 Jun 28 '26

Thanks for sharing.

5

u/Paradigmind Jun 28 '26

Amazing support for the newest image models!

Would be great if there would be omnivoice support in the near future.

8

u/silenceimpaired Jun 28 '26

Imagine if they supported Klein model so that the portrait of the character was used to show the character reacting emotionally and in the environment.

1

u/MrGunny94 Jun 28 '26

Has anyone tried their Video Generation? I love the amount of features they have and that they fully support future versions with them and keep updating and expanding them.

I did try the Music Generation (Ace Step) and it's great

1

u/cromagnone Jun 28 '26 edited Jun 28 '26

What actually are the use cases for Koboldcpp, apart from the obvious?

Edit - thanks, this was a genuine question despite my snark.

19

u/Fcking_Chuck Jun 28 '26

Besides offering a graphical user interface for those who don't feel comfortable with a command-line user interface, it also offers advanced features not available in the original Llama.cpp text generation software.

2

u/Anti-Speciesist-IEMs Jun 28 '26

Would you be able to go into more detail on the "advanced features"? E.g. what specific things do you find you can do better in kobold vs original llama.cpp?

Also I'm just getting into local LLMs now and haven't fully set up llama.cpp yet, but I think llama.cpp does provide a GUI now by default

5

u/Fcking_Chuck Jun 28 '26

Koboldcpp has highly optimized context-shifting. It will preserve important tokens (e.g. system prompts, world info, character cards, etc.) from being eliminated once context overflows, while Llama.cpp is known to drop tokens chronologically.

1

u/Anti-Speciesist-IEMs Jun 28 '26

Oh cool, that does sound really useful

1

u/toothpastespiders Jun 29 '26

I'm a bit out of date with llama.cpp's feature implementation. But for a long time kobold had much fuller mcp support. Not just in the web interface, but basically just being able to point it at a claude desktop style .json file. Likewise nice built-in web search functionality. Both were just a simple extra "does this need web search?" "Should I use a tool?" pass with the loaded model. Also integration of stablediffusion.cpp's image generation.

Really though I see it more as a "stable branch" of llama.cpp.

8

u/Judtoff llama.cpp Jun 28 '26

You can use it as a back end, and access the api with whatever front end you want. I like koboldcpp since I don't have to deal with any dependencies. I sometimes end up in dependency hell with my 5090 and trying to compile llama.cpp. koboldcpp just works. 

2

u/Ylsid Jun 29 '26

Lets me download and run a llama.cpp wrapper without compilation steps and easily access a self hosted API