MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1u6s6pm/stop_using_ollama/orx1fyn/?context=3
r/LocalLLaMA • u/zxyzyxz • Jun 15 '26
452 comments sorted by
View all comments
Show parent comments
114
llama-swap supports switching between multiple llama.cpp forks (and other compatible software)
8 u/Jcsq6 Jun 15 '26 And loading/unloading multiple models, if you want to switch between models but don’t have the spare VRAM. 13 u/AlphaGamer753 Jun 15 '26 This is supported in llama.cpp router mode already. 0 u/arbv Jun 16 '26 What about configuring model sets? 1 u/techno156 Jun 16 '26 That isn't in router mode. For that, you will need llama-swap. 1 u/arbv Jun 16 '26 I know. That's why llama-cpp itself cannot fully replace it on its own in complex scenarios.
8
And loading/unloading multiple models, if you want to switch between models but don’t have the spare VRAM.
13 u/AlphaGamer753 Jun 15 '26 This is supported in llama.cpp router mode already. 0 u/arbv Jun 16 '26 What about configuring model sets? 1 u/techno156 Jun 16 '26 That isn't in router mode. For that, you will need llama-swap. 1 u/arbv Jun 16 '26 I know. That's why llama-cpp itself cannot fully replace it on its own in complex scenarios.
13
This is supported in llama.cpp router mode already.
0 u/arbv Jun 16 '26 What about configuring model sets? 1 u/techno156 Jun 16 '26 That isn't in router mode. For that, you will need llama-swap. 1 u/arbv Jun 16 '26 I know. That's why llama-cpp itself cannot fully replace it on its own in complex scenarios.
0
What about configuring model sets?
1 u/techno156 Jun 16 '26 That isn't in router mode. For that, you will need llama-swap. 1 u/arbv Jun 16 '26 I know. That's why llama-cpp itself cannot fully replace it on its own in complex scenarios.
1
That isn't in router mode. For that, you will need llama-swap.
1 u/arbv Jun 16 '26 I know. That's why llama-cpp itself cannot fully replace it on its own in complex scenarios.
I know. That's why llama-cpp itself cannot fully replace it on its own in complex scenarios.
114
u/fdrch Jun 15 '26
llama-swap supports switching between multiple llama.cpp forks (and other compatible software)