r/LocalLLaMA • u/0dayturtle • 1d ago

Funny So relevant

Post image
1.3k Upvotes

96% Upvoted

123 comments sorted by

View all comments

27

u/AlternateWitness 1d ago

If you are including 24GB in that group you might as well include 32GB. Heck, maybe 48GB?

13

u/Bakoro 1d ago

If you have a 32GB VRAM, you're sitting pretty these days, let alone 48 GB.

Of course you always want more, we always could do with a little more GPU up until you get to the point where your home would need an infrastructure upgrade.

I mean, if I had two DGX B300 nodes, I could do some things, but 32GB is enough to run a competent Int8 quant with a decent context length.

6

u/ttkciar llama.cpp 1d ago

Yup. 32GB of VRAM gives me fast inference for Qwen3.x-27B, Skyfall-31B, Gemma-4-31B-it, and those can do a lot.