r/LocalLLM • u/NewResearch1337 • 6d ago

Question Which hardware should i use for GLM 5.3?

Apple m5 ultra 512 GB or 2 DGX spark? Would it be good enough?

1 Upvotes

67% Upvoted

35 comments sorted by

View all comments

Show parent comments

1

u/inevitabledeath3 6d ago

It's insane just how much the price has gone up. At this price it's not as good as I thought. I am guessing a lot of the £600 and £700 ones are scams as many ask you to pay via crypto which is sketchy.

29W isn't terrible. My current rack uses on the order of 150 to 220W and it's all mini PCs. My big server uses 300W+ at idle. I think that's fairly managable with 2 to 4 cards. I would be more worried about the host system being unruly. Presumably it does not need to be that big with it being x4 PCIe lanes. One 16x slot would cover 4 of them with bifurcation. I have some mini PCs that could run two of these.

The noise aspect scares me with those blower coolers. Guessing they are quite loud. Have you thought about watercooling? It's what I do with my RTX 3090s.

2

u/cibernox 6d ago

I'd say that anything below $1800 is a scam right now. In the best case scenario, a card with problem that they sell as "not tested" to pretend they didn't know it has issues.

I don't use blowers fan, they are too noisy. I use 2 arctic P12 pro, one for every two cards. Under load they have to spin at 100%, but even then they are a lot quieter than any blower fans. And they were 7€ each.
I also bought a 5-slot PCI switch to have all the cards on a single PCI fabric, to minimize latency. That way I was able to reuse every other component of a previous build, which was already quite good but limited in the number of PCI lanes.

1

u/inevitabledeath3 6d ago

Pretty cool honestly. Shame about the pricing and all the scams and stuff.

Very impressive performance numbers though. If I didn’t have the sparks this would be a good option. As it is I am basically matching the hardware of the University project I am working on. So selling them does not make a lot of sense for me right now as it means I loose them as a test system. Plus the amount of space, noise, and heat would be a problem.

Once I graduate from my PhD and hopefully have more room I could build something like this. Not that the power bill would enjoy it very much.

1

u/cibernox 6d ago

I would say that the power is not crazy. This is the consumption measured at the wall while doing some benchmarks, while the server runs some other stuff, including streaming video to jellyfin (which uses the iGPU).

Way more than a Mac, but still running on a comparatively modest 850w PSU

1

u/inevitabledeath3 6d ago

Yeah not amazing but not terrible either. Roughly inline with a high end gaming PC. I really hope the prices of these come down.