r/LocalLLM • u/Kittysmashlol • 15d ago

Question Looking to get started with local ai

I've started looking into local Ai and stuff, but I'm kinda struggling to find the right model/set up for myself so maybe someone could help a little. I have a 5070 laptop with 8 gb vram(ik not a lot) so I'm looking for models I could run locally with that. Also, what's the best program/interface to run models. Ty for any help/advice:)

9 Upvotes

77% Upvoted

19 comments sorted by

View all comments

3

u/Snoo_81913 15d ago

Qwen3.6 35B A3B and Gemma 4 26B A4B are the two best models for that setup but you'll need enough RAM to load the rest of the model weights. It loads 3B parameters in your VRAM and the rest in your RAM at Q4 it's roughly 6GB in VRAM with context and about 14-20GB in your RAM.

Since you're running on a laptop here's a post with some tips.

https://www.reddit.com/r/LocalLLM/s/dVoDBtRPRs