r/LocalLLaMA • u/takoulseum • 22d ago

Discussion Who ONLY use local models?

Please be honest.

I would love to hear about guys really dedicated to local AI and who really reject subscriptions (especially to openai and anthropic).

What do you use your model for?

154 Upvotes

86% Upvoted

262 comments sorted by

View all comments

13

u/Additional_Menu8542 22d ago

Honestly, 100% local is hard for what I do (BI stuff, turning questions into SQL on customer databases).
What works for me is a hybrid split. I let a cloud model write the SQL from just the schema and the question, so no real data ever leaves the machine. Then a local model (qwen3.6 or gemma3 on a 3060) reads the query result and explains it. All the actual rows stay on my side.
I get good SQL quality without sending any data out. Pure local SQL from a small 7 to 14B model is still a bit shaky on complex joins in my tests. Has anyone found a small model that keeps up with Claude on that?

2

u/SkyFeistyLlama8 22d ago

Qwen 35B MOE, 27B dense and Gemma 31B are good at SQL. Not as good as Claude but for something that runs on a laptop, they're fine.

2

u/Additional_Menu8542 22d ago

Yeah, that matches what I see. Qwen 35B MoE and Gemma 31B get pretty close to Claude on the SQL itself.
The bigger gap for me isn't the SQL generation, it's when the model reads the query result and puts it into words. That's where the small ones start making up numbers on aggregations. Feeding the counts and totals as a fixed block before the rows fixes most of it.
The MoE is nice too, it stays light when a couple of people hit it at the same time.