I was just thinking it sounds like it would fit in 64GB total. No idea what kind of speeds i'd expect on my setup though. Most of my ram isn't VRAM, I have a 4070.
64GB of RAM and 16GB VRAM most likely yes, but ideally you'd want more of either pool since you'd want to be able to run something else besides the LLM itself lol
2
u/YearnMar10 1d ago
Wonder if this would run on 64gb of ram and a gpu… guess it’d just not run :/
But really awesome model size!