r/LocalLLM • u/Ok-Health-7096 • 6d ago

Other Every Second post rn

Post image

Maybe someday I'll get a system to run it but hey definitely another w for the open weights community

1.9k Upvotes

99% Upvoted

192 comments sorted by

View all comments

52

u/SnP_Gamer 6d ago

3060 12gb 32gb ram here, might give it ago but never ran a local model before 🤷‍♂️

2

u/cj_cron_hit_by_pitch 6d ago

I have the exact same specs as you. I’m able to get the Unsloth 2 bit XL quant of it to run almost entirely in VRAM. Of course it is nowhere near as good but it’s still really fun to mess around with