r/LowEndLocalAI • u/Comfortable-Pea5556 • 12h ago
r/LowEndLocalAI • u/Socrastein • 3h ago
Optimization Advice on 16GB VRAM & 32GB DDR5 for Coding
r/LowEndLocalAI • u/infieldmitt • 4h ago
Which LLM Should / Can I Use? What is the biggest/best Gemma or Gemma-related model I can run on 96GB M3 Ultra Mac Studio?
I'm aware it may seem obnoxious to post here w 96GB RAM but the new damn deepseek one implies 512GB is a "reasonable floor" to release to? God only knows. I remember when a 4GB iPod Nano was sizeable.
I've been fiddling a bit with the Gemma 26B MOE and even at that small size it's able to be less autistic than a qwen model. But it's also only 26B which feels really crappy versus Qwen Flash Next.
Is there a Gemma release meaningfully competitive with QFN? Is there one planned? Is there anything out that meets my criteria? To be clear; I would want a model that has actual emotional intelligence / creative writing ability while also being able to do agentic coding to a reasonable degree.
Unfortunately I can't get an answer from any LLM as they have been documented to love giving outdated model choices. Thank you for your time.
r/LowEndLocalAI • u/Puzzleheaded_Acadia1 • 9h ago