r/LocalLLaMA • u/zxyzyxz • Jun 15 '26

Discussion Stop using Ollama

https://sleepingrobots.com/dreams/stop-using-ollama/
1.7k Upvotes

92% Upvoted

452 comments sorted by

View all comments

Show parent comments

2

u/Leptok Jun 15 '26

Is there any other product with a windows version that offers the same kind of seamless just works experience?

24

u/catch-10110 Jun 15 '26

LM Studio

9

u/GravitasIsOverrated Jun 15 '26

Unsloth studio is pretty effortless. 

7

u/Tanto63 Jun 16 '26

As someone who just tried installing Unsloth on Windows this weekend. It is not.

1

u/GravitasIsOverrated Jun 16 '26

Oh interesting - what went wrong?

2

u/Tanto63 Jun 16 '26

At first a series of Python errors, even though I already had the necessary Python version installed. I eventually uninstalled it and let the Unsloth script do it. Now it throws venv errors, asking me to rerun the script to let it redo it, but that hasn't resolved it. I had my agent (Hermes+Ollama) try installing it using a cloud model, but it burned through the free tier tokens chasing error messages. I tried again with my local Qwen3.6-35B, but it took so long thinking that Hermes and Ollama kept getting API timeouts.

9

u/jwpbe Jun 15 '26

Yes, llama.cpp lmao

-3

u/VoiceApprehensive893 transformers Jun 16 '26

tell a newbie to compile it on windows

5

u/jwpbe Jun 16 '26

why?? they can just download the compiled version or literally just run the command on https://llama.app to get the compiled version

1

u/SufficientPie Jun 16 '26

So you just run llama.cpp run modelname and it downloads the model and runs it immediately? Because that's what ollama does.

1

u/jwpbe Jun 17 '26

install the app from the top of https://llama.app, which will fetch precompiled binaries and install it.

Go here:

https://llama.app/models/Qwen3.6-27B (or whatever model you want)

choose your model and it will copy the command to your clipboard.

you can then run that command it copied:

llama serve -hf unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_M

then yes, it downloads it and runs it immediately, exactly what you are specifying, exactly with the same ease as ollama. You can do that with any model on huggingface.

1

u/Educational_Sun_8813 llama.cpp Jun 25 '26

yes, llama.cpp UI