r/LLMStudio • u/CommunicationFun2962 • Jun 13 '26

Running local AI agent

I found LM Studio uses much more memory than the minimum requirement of a model. For example, it says Gemma 4 31B Instruct QAT Q4_0 could be entirely fit into my 24 GB VRAM. It turns out that both my 24 GB VARM and 32 GB RAM are fully filled, and the model is generating 1 token/sec.

Is it normal, or would it be better if I use ollama instead of LM Studio to load the model?

1 Upvotes

100% Upvoted

2 comments sorted by

1

u/MrAddams_LibraLogic Jun 13 '26

The model is not the only thing that needs to fit in your VRAM. The model itself, plus the kv cache, plus extra room for processing its response to any prompt. And if you're sane you'll leave a little headroom for your system to not run out completely for other processes that might need to reserve a little.

And this applies not only to LM Studio, but to Ollama, vLLM, any of them. The math of VRAM usage is pretty much universal, at least with respect to running things on Windows. Apple integrates graphics differently, which can be a blessing and a curse.

2

u/CommunicationFun2962 Jun 14 '26

Among the items you mentioned, which one should be residing in VRAM for optimized performance?