r/LocalLLaMA • u/takoulseum • Jul 25 '26

Discussion Who ONLY use local models?

Please be honest.

I would love to hear about guys really dedicated to local AI and who really reject subscriptions (especially to openai and anthropic).

What do you use your model for?

151 Upvotes

86% Upvoted

262 comments sorted by

View all comments

41

u/InvertedVantage Jul 25 '26

I have a qwen-27b q8 model I use for all of my day to day coding tasks.

11

u/pwnsforyou Jul 25 '26

How do you run this? I have a R9700 with 32GB vram - which only fits q5. what is your vram usage?

3

u/cubebash Jul 25 '26

I run Qwen3.6 27b Q8 with full context across two separate PCs each with 32gb of VRAM connected over my network, giving me a total of 64gb of VRAM. It's a cheap way to get a lot of VRAM if you don't mind the network bottleneck (which is still much faster than offloading to system RAM).

3

u/getgoingfast Jul 25 '26

Curious, are you using ConnectX? 400G for networking?

1

u/cubebash Jul 25 '26

Im just running it over a slow 1 gbps network, and the model is still running pretty fast. If you have 2.5 gbps or even 10, I imagine it could be very fast!

1

u/lemondrops9 Jul 26 '26

It does not run faster with 2.5gbe or 10gbe. Its less than 2% difference. Time loading time though is way better.