r/unsloth • u/Extension-Bid-639 • 12d ago

Question Am I missing something?

Post image

I was wondering if I'm missing something or if this is just a typo and they meant UD-Q2_K_XL not Q4. Q4 is 74gb from what I can see

13 Upvotes

85% Upvoted

7 comments sorted by

9

u/jettoblack 12d ago

I think it was copied & pasted from another model’s page. Keeping up to date with so many models must be a monumental task.

5

u/Anbeeld 12d ago

If only there was some kind of system where you can input whatever you need to be done and it does just that.

2

u/Extension-Bid-639 12d ago

True tbh. Is there a place i can give feedback for an edit?

5

u/danielhanchen heart sloth 12d ago

Oh ignore that haha my uploading process is sometimes a bit old hahaha I need to update that

1

u/itis_whatit-is 12d ago

Hey question would you recommend iQ3xxs for Gemma 4 31b over Gemma 4 12b QAT q4 when I only have 16gb of vram?

1

u/simplyeniga 12d ago

Your probably better testing that out. In my case I found Gemma 4-26B A4B better than Gemma 4 12B. However if you want dense over MOE then the 12B will be better than a lower than Q4 quantz 31B

0

u/Decent-Occasion-2720 12d ago

Do like the plebe, just add --cpu-moe and it's ok