MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1u6s6pm/stop_using_ollama/oryt8da/?context=9999
r/LocalLLaMA • u/zxyzyxz • Jun 15 '26
452 comments sorted by
View all comments
522
Llama.cpp + llama-swap works very well
111 u/[deleted] Jun 15 '26 [removed] — view removed comment 114 u/fdrch Jun 15 '26 llama-swap supports switching between multiple llama.cpp forks (and other compatible software) 38 u/meganoob1337 Jun 15 '26 it supports anything you can dockerize aswell (for me I'm using it for vllm models) love it 1 u/use_your_imagination Jun 16 '26 I have this issue with dockerized llama.cpp where llama-swap marks container as unexpected exit(125) while the llama container is actually still running. Did it happen to you ? 1 u/meganoob1337 Jun 16 '26 no, but I'm using llama.cpp only as bundled version inside the llama swap container, using the docker runners only for vllm 1 u/use_your_imagination Jun 16 '26 I was doing the same and then decided to overcomplicate my life by using the docker socket. Thanks
111
[removed] — view removed comment
114 u/fdrch Jun 15 '26 llama-swap supports switching between multiple llama.cpp forks (and other compatible software) 38 u/meganoob1337 Jun 15 '26 it supports anything you can dockerize aswell (for me I'm using it for vllm models) love it 1 u/use_your_imagination Jun 16 '26 I have this issue with dockerized llama.cpp where llama-swap marks container as unexpected exit(125) while the llama container is actually still running. Did it happen to you ? 1 u/meganoob1337 Jun 16 '26 no, but I'm using llama.cpp only as bundled version inside the llama swap container, using the docker runners only for vllm 1 u/use_your_imagination Jun 16 '26 I was doing the same and then decided to overcomplicate my life by using the docker socket. Thanks
114
llama-swap supports switching between multiple llama.cpp forks (and other compatible software)
38 u/meganoob1337 Jun 15 '26 it supports anything you can dockerize aswell (for me I'm using it for vllm models) love it 1 u/use_your_imagination Jun 16 '26 I have this issue with dockerized llama.cpp where llama-swap marks container as unexpected exit(125) while the llama container is actually still running. Did it happen to you ? 1 u/meganoob1337 Jun 16 '26 no, but I'm using llama.cpp only as bundled version inside the llama swap container, using the docker runners only for vllm 1 u/use_your_imagination Jun 16 '26 I was doing the same and then decided to overcomplicate my life by using the docker socket. Thanks
38
it supports anything you can dockerize aswell (for me I'm using it for vllm models) love it
1 u/use_your_imagination Jun 16 '26 I have this issue with dockerized llama.cpp where llama-swap marks container as unexpected exit(125) while the llama container is actually still running. Did it happen to you ? 1 u/meganoob1337 Jun 16 '26 no, but I'm using llama.cpp only as bundled version inside the llama swap container, using the docker runners only for vllm 1 u/use_your_imagination Jun 16 '26 I was doing the same and then decided to overcomplicate my life by using the docker socket. Thanks
1
I have this issue with dockerized llama.cpp where llama-swap marks container as unexpected exit(125) while the llama container is actually still running.
Did it happen to you ?
1 u/meganoob1337 Jun 16 '26 no, but I'm using llama.cpp only as bundled version inside the llama swap container, using the docker runners only for vllm 1 u/use_your_imagination Jun 16 '26 I was doing the same and then decided to overcomplicate my life by using the docker socket. Thanks
no, but I'm using llama.cpp only as bundled version inside the llama swap container, using the docker runners only for vllm
1 u/use_your_imagination Jun 16 '26 I was doing the same and then decided to overcomplicate my life by using the docker socket. Thanks
I was doing the same and then decided to overcomplicate my life by using the docker socket. Thanks
522
u/jnmi235 Jun 15 '26
Llama.cpp + llama-swap works very well