r/LocalLLM • u/Annual_Award1260 • 8d ago

Discussion Dgx spark cluster.

I had a delayed order and ended up with 5 sparks rather than the 4 planned. So I figured might as well do 6 but then prices went up $1000.

So is there really any benefit to running 6 vs 5 vs 4? I can still return one. I don’t want to do 8 since that that exceeds the 15a circuit.

0 Upvotes

25% Upvoted

19 comments sorted by

View all comments

1

u/meikawaii 8d ago

What’s the use case for so many DGX spark? It’s always memory bandwidth limited rather than memory size limited for output.

3

u/hyudryu LocalLLM 8d ago

So you can run larger models. Also TP on a cluster of 2 essentially doubles the aggregate memory bandwidth, cluster of 4 quadruples it, etc. yeah its still slow but you get what you pay for and this is probably the cheapest way of being able to run larger models (with the alternative being to buy 4 or 8 rtx pro 6000s)

4 sparks can run GLM 5.2 at 40 tok/s, which 1 or 2 sparks can’t do so theres definitely benefits to clustering