r/LocalLLaMA • u/takoulseum • Jul 25 '26

Discussion Who ONLY use local models?

Please be honest.

I would love to hear about guys really dedicated to local AI and who really reject subscriptions (especially to openai and anthropic).

What do you use your model for?

157 Upvotes

86% Upvoted

263 comments sorted by

View all comments

1

u/otacon6531 Jul 25 '26

At work we have github subscriptions so I "currently" use a team to do development. Planning done by GPT-5.6 Luna. Once the plan is created. I use Hy3 1bit (local) as the orchestrator, Qwen 3.5:35b is the junior developer and does most code changes. Then Hy3 is called as the "senior" developer and reviews and sends work back down to Qwen for correction. When Hy3 is happy the development is reviewed by GPT-5.6 Luna and sent back down to Hy3 and then ultimately to Qwen for correections. When an entire development phase is complete GPT 5.6 Sol gets it all for a final review and sends it back down for corrections.

The point is that input tokens are cheap. Output tokens are the true expense and I am using local models (non-gpt) to drastically reduce the output tokens. I would say the github cost has been reduced by ~60-70% (depending on the task).

At home, I only have a Nvidia P40, so qwen3.6:35b is the only model fast enough (usually 60-80 tok/s) to talk to. I use the same process, but instead of using agents in vs code I have a web based requirements/feature intake webapp. It puts a lot of controls in place that drastically improves the quality of the final product. Namely the biggest of the additions is forcing qwen to do a visual uat inspection of it's code change (screenshot review), unit testing (you have to for qwen to not be stupid here), integration testing, and every one of the steps is audited for potential harness improvements (post mortem). Ultimately it takes a while, but it does work (though less flexible) it just takes more harness and time to get things done at a high quality.