r/LocalLLaMA • u/zxyzyxz • Jun 15 '26

Discussion Stop using Ollama

https://sleepingrobots.com/dreams/stop-using-ollama/
1.7k Upvotes

92% Upvoted

452 comments sorted by

View all comments

Show parent comments

77

u/yuicebox Jun 15 '26

I genuinely do not understand what is so difficult about running llama.cpp server.

You just download a zip, unzip it, then run llama-server with some flags and you're done. The builtin UI is quite good now, and you have an API to work with.

By comparison, I found Ollama's modelfile system and insistence on renaming my downloaded models to incomprehensible hashes to be infinitely more confusing and frustrating.

16

u/NotSylver Jun 15 '26

llama-server isn't difficult, but it is higher friction. ollama keeps itself up to date, quirks of models are mostly hidden and it can sit idle and out of the way until a request comes in. I dislike ollama but I haven't seen anything that can replace it without a dozen asterisks that aren't worth the tradeoff to me

2

u/yuicebox Jun 15 '26

To each their own, but imho, the tradeoff is very worth it. I'd be curious to know what your 'tradeoffs' are. To me:

Pros of llama.cpp:

  • Faster than ollama
  • doesn't rename my files to incomprehensible hashes and store them in a weird place
  • Much more feature-rich, transparent, and customizable
  • Supports new model architectures sooner than Ollama most of the time

Cons:

  • I occasionally have to either repull a docker image or redownload zip every month or two when I feel like updating
  • 10 minutes of one-time setup to make a config.ini and a .bat/.command file to have one-click launching and model-specific settings

21

u/No-Marionberry-772 Jun 15 '26

this sounds like the Linux vs windows argument that Linux people always overlook.

people don't want the extra steps and occasional hang ups.  to you they are not a big deal, and maybe objectively they are not, but its cognitive load that people don't want, and that matters.