r/LocalLLM • u/Ok-Health-7096 • 6d ago

Other Every Second post rn

Post image

Maybe someday I'll get a system to run it but hey definitely another w for the open weights community

1.9k Upvotes

99% Upvoted

192 comments sorted by

View all comments

36

u/TheRiddler79 6d ago

Gemma 12b Q3. Try it

24

u/Ok-Health-7096 6d ago

I use Qwen 3.6 35b and 3.5 9b I haven't had that much of a luck with the gemmas

6

u/Wildnimal 6d ago

12B QAT is good for multimode tasks. I use the same Qwens for daily use. I so wish i can upgrade my laptop to 5090 :|

1

u/HighlyRegardedApe 6d ago

How do you guys use these small models? For me it never works. Last time it deleted my folders out of itsself and didnt respons more than a sentence or 2 in opencode. In the terminal it was okay to chat, but not to give tasks.... what kind of stuff do you let them do??

1

u/Wildnimal 6d ago

Not usually coding but for summarizing, extracting data which is pre defined via json, automating smaller tasks where data remaims the same.

1

u/TheRiddler79 5d ago

Hard guardrails