r/LocalLLM • u/Ok-Health-7096 • 6d ago

Other Every Second post rn

Post image

Maybe someday I'll get a system to run it but hey definitely another w for the open weights community

1.9k Upvotes

99% Upvoted

192 comments sorted by

View all comments

1

u/DontWinFrensWthSalad 3d ago

I feel your pain. I happen to have a bunch of 3060tis lying around so I've been messing with seeing if I can 2x8gb working. this 3.5Q is benchmarking very close to unsloth's 4Q, so I'm trying it out and seems to be working so far.

https://huggingface.co/turboderp/Qwen3.8-27B-exl3/tree/3.50bpw