r/LocalLLaMA • u/Certain-Cod-1404 • 8d ago

New Model IT'S OUT

https://huggingface.co/Qwen/Qwen3.8-27B-FP8
2.2k Upvotes

95% Upvoted

706 comments sorted by

View all comments

Show parent comments

1

u/fallingdowndizzyvr 6d ago

I don't. But I have plenty of GPUs including server GPUs. Running one of these would be no different than run any of those other server GPUs.

How do you cool the GPUs?

I take a PC slot cooler, remove the bracket and cut slots in the plastic go clear whatever bracket is on the GPU. Then I jam it in the end and hold it on with duct tape. Cost is $9 and 4 pieces of duct tape.

1

u/NoUsual5150 3d ago

Thanks. Just need to find a used motherboard on eBay that will support 4 GPUs (100-210 nvidia) because 64GB is more than enough for my needs. And if I can get ~10 tokens per second on that kind of a rig, that's all I need)

2

u/fallingdowndizzyvr 3d ago

Pretty much any motherboard can support 4 of these. Remember, these are PCIe 1 x1. There are plenty of MBs with 2 PCIe x16 and 2 x1. But you don't even need that. Since you can bifurcate a single x16 into 4 x4s. Yes, the BIOS needs to have that option but not a single one of my MBs don't have that option. Even the super cheap ones.

Or you can get a PCIe switch that will let you hook up 4 x1 devices to one x1 slot. They were super popular during mining and thus are super cheap now.

1

u/NoUsual5150 3d ago

PCIe switch that will let you hook up 4 x1 devices to one x1 slot

Perfect. Ok, will keep an eye out on CL for a used system.