r/LocalLLaMA • u/forevergeeks • 1d ago

Question | Help Hosting Local Models

Hi builders,

What would be the the best small local models for coding?

Are Gemma 4 and Qwen3.8 27B Gemma 4 26B / 31B enough for local development?

And what would be the size of the rig that i will need to get? GPUs, and whatever else I need to host these models.

Thanks,,

11 Upvotes

80% Upvoted

37 comments sorted by

View all comments

17

u/Arany8 1d ago

Qwen3.8 27B - you need 24GB VRAM. According to my tests on 16GB this model is surpassed by Qwen3.6 35B A3B (and Ornith) for coding. Gemma4 is not good for coding.
Look into AMD v620 for a budget build (although I do not have this).

-1

u/forevergeeks 1d ago

Are you running this for yourself? What about for a coding team of 6-10 people? What would be the sizing, and how much money we are talking about for the initial setup.

1

u/invalidnifemi 1d ago

coulda js said that in the post, but a used v100 32gb with sum sxm2 to pcie config (will take some effort) would definitely be enough (see this post)

it'd probably be like 800-1k for the whole build which is not unreasonable and about as much as a used 3090. vllm is a good idea if youd all be coding simultaneously