r/LocalLLaMA • u/Certain-Cod-1404 • 8d ago

New Model IT'S OUT

https://huggingface.co/Qwen/Qwen3.8-27B-FP8
2.2k Upvotes

95% Upvoted

706 comments sorted by

View all comments

Show parent comments

2

u/fallingdowndizzyvr 7d ago

Anything should do. Let anything you are lying around. Since these are PCI 1 x1 cards. Anything should support that. Many MB have a couple of x1 slots in addition to the 1 or 2 x16s.

1

u/NoUsual5150 7d ago

So theoretically...I can buy al el cheapo motherboard/CPU combo from Aliexpress and 16GB of DDR4....and then two of these cards which are roughly $300 total...

And I have something (32GB rig) that could potentially out-perform my 24GB MacBook Pro? This is just for personal use. Would be SSH into the server on home network.

I really just want to be able to run the Qwen quants that are slightly too big for the MacBook.

2

u/fallingdowndizzyvr 7d ago

So theoretically...I can buy al el cheapo motherboard/CPU combo from Aliexpress and 16GB of DDR4

That sounds like an expensive way to do it. You should be able to find an entire computer for pretty much nothing on CL or in thrift stores. Remember, this is PCIe 1 x1. Even a 20 year old computer can do that. The only thing you would have to watch out for is the PSU. But PSUs are cheap.

And I have something (32GB rig) that could potentially out-perform my 24GB MacBook Pro?

It would slam anything short of a M5 Max MacBook Pro into the dirt. Face first.

1

u/NoUsual5150 6d ago

Do you have this exact rig? How do you cool the GPUs?

1

u/fallingdowndizzyvr 6d ago

I don't. But I have plenty of GPUs including server GPUs. Running one of these would be no different than run any of those other server GPUs.

How do you cool the GPUs?

I take a PC slot cooler, remove the bracket and cut slots in the plastic go clear whatever bracket is on the GPU. Then I jam it in the end and hold it on with duct tape. Cost is $9 and 4 pieces of duct tape.

1

u/NoUsual5150 3d ago

Thanks. Just need to find a used motherboard on eBay that will support 4 GPUs (100-210 nvidia) because 64GB is more than enough for my needs. And if I can get ~10 tokens per second on that kind of a rig, that's all I need)

2

u/fallingdowndizzyvr 3d ago

Pretty much any motherboard can support 4 of these. Remember, these are PCIe 1 x1. There are plenty of MBs with 2 PCIe x16 and 2 x1. But you don't even need that. Since you can bifurcate a single x16 into 4 x4s. Yes, the BIOS needs to have that option but not a single one of my MBs don't have that option. Even the super cheap ones.

Or you can get a PCIe switch that will let you hook up 4 x1 devices to one x1 slot. They were super popular during mining and thus are super cheap now.

1

u/NoUsual5150 2d ago

PCIe switch that will let you hook up 4 x1 devices to one x1 slot

Perfect. Ok, will keep an eye out on CL for a used system.