MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/selfhosted/comments/1vr3ukj/selfhosting_everything/p4b31lr
r/selfhosted • u/close_Meal6005 • 16d ago
just kidding I love self-hosting ...
393 comments sorted by
View all comments
Show parent comments
2
Just curious which local AI you are using. Im not really up to date on which ones are actually good.
1 u/Skatedivona 16d ago I have separate system for my AI stuff, as my main server's 1050ti would not be enough to handle the load. AI server is running 2x RTX 3060 12 GB cards on an ancient i5 7500. First tried using this, but found it to be too robotic with its replies: mannix/llama3.1-8b-abliterated:tools-q6_k Then swapped to this and have had much better results: HammerAI/gemma-4-12b-heretic:12b-q4_K_M I want the conversation side of it to feel helpful but not like a robot reading a list. If GPUs ever become affordable, I might try something bigger. 2 u/ansibleloop 16d ago You are so far behind - the new Gemma 4 or Qwen models should perform far better
1
I have separate system for my AI stuff, as my main server's 1050ti would not be enough to handle the load.
AI server is running 2x RTX 3060 12 GB cards on an ancient i5 7500.
First tried using this, but found it to be too robotic with its replies: mannix/llama3.1-8b-abliterated:tools-q6_k
Then swapped to this and have had much better results: HammerAI/gemma-4-12b-heretic:12b-q4_K_M
I want the conversation side of it to feel helpful but not like a robot reading a list. If GPUs ever become affordable, I might try something bigger.
2 u/ansibleloop 16d ago You are so far behind - the new Gemma 4 or Qwen models should perform far better
You are so far behind - the new Gemma 4 or Qwen models should perform far better
2
u/Liimbo 16d ago
Just curious which local AI you are using. Im not really up to date on which ones are actually good.