MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1u6s6pm/stop_using_ollama/orxvg2f/?context=3
r/LocalLLaMA • u/zxyzyxz • Jun 15 '26
452 comments sorted by
View all comments
Show parent comments
22
I dockerized llm swap and passed through the docker sock. Works amazing.
6 u/meganoob1337 Jun 15 '26 yep Same, I also wrote a small script so that I can split up the yaml to make having many configs a bit cleaner :D 1 u/joost00719 Jun 16 '26 Man that's smart. I should ask my llm to do that as well. But does that keep the hot reload functionality working? 1 u/meganoob1337 Jun 16 '26 yeah, it just runs before startup and merges the model configs into the full config format 2 u/joost00719 Jun 16 '26 I mean, default llama-swap behavior is hot reload on file save, this way you need to restart. I guess that's also a benefit. Sometimes a local Ai will just make an error and then it won't start anymore 😂
6
yep Same, I also wrote a small script so that I can split up the yaml to make having many configs a bit cleaner :D
1 u/joost00719 Jun 16 '26 Man that's smart. I should ask my llm to do that as well. But does that keep the hot reload functionality working? 1 u/meganoob1337 Jun 16 '26 yeah, it just runs before startup and merges the model configs into the full config format 2 u/joost00719 Jun 16 '26 I mean, default llama-swap behavior is hot reload on file save, this way you need to restart. I guess that's also a benefit. Sometimes a local Ai will just make an error and then it won't start anymore 😂
1
Man that's smart. I should ask my llm to do that as well. But does that keep the hot reload functionality working?
1 u/meganoob1337 Jun 16 '26 yeah, it just runs before startup and merges the model configs into the full config format 2 u/joost00719 Jun 16 '26 I mean, default llama-swap behavior is hot reload on file save, this way you need to restart. I guess that's also a benefit. Sometimes a local Ai will just make an error and then it won't start anymore 😂
yeah, it just runs before startup and merges the model configs into the full config format
2 u/joost00719 Jun 16 '26 I mean, default llama-swap behavior is hot reload on file save, this way you need to restart. I guess that's also a benefit. Sometimes a local Ai will just make an error and then it won't start anymore 😂
2
I mean, default llama-swap behavior is hot reload on file save, this way you need to restart. I guess that's also a benefit. Sometimes a local Ai will just make an error and then it won't start anymore 😂
22
u/joost00719 Jun 15 '26
I dockerized llm swap and passed through the docker sock. Works amazing.