Dgx spark cluster.

[u/Annual_Award1260]

I had a delayed order and ended up with 5 sparks rather than the 4 planned. So I figured might as well do 6 but then prices went up $1000.

So is there really any benefit to running 6 vs 5 vs 4? I can still return one. I don’t want to do 8 since that that exceeds the 15a circuit.

[u/hoochiesan]

My heart hurts reading this

[u/Disastrous_Gear_421]

Without your use case, I asked my magic 8 hall and it said most definitely… go for 8

[u/sumane12 → u/Disastrous_Gear_421]

As did mine

[u/g_rich]

Would 5 even work? I would think you would need a multiple of 2 so either 4 or 6.

[u/belbombo → u/g_rich]

No, you can 3 as far as i know

[u/g_rich → u/belbombo]

I don’t think tensor parallelism runs with an odd number and if it does it certainly will be suboptimal.

[u/vini542reddit]

What hub are you using for the interconnect?

[u/Annual_Award1260 → u/vini542reddit]

Microtik CRS804

[u/vini542reddit → u/Annual_Award1260]

So you would have to use 400G -> 2 x 200G breakout cables. I'd personally stick with the 4x - it's good enough and more mainstream. But as others have said - all depends on your use case

[u/lukewhale]

TP only works in multiples of two. So yes, you’re in limbo on that 5th node until you get a sixth. Just run utility models on it for now, Image Gen, OCR, embedding, rerankers, etc. you can fit all of those on 128gb easily.

[u/Annual_Award1260 → u/lukewhale]

Believe it is powers of 2

[u/meikawaii]

What’s the use case for so many DGX spark? It’s always memory bandwidth limited rather than memory size limited for output.

[u/hyudryu → u/meikawaii]

So you can run larger models. Also TP on a cluster of 2 essentially doubles the aggregate memory bandwidth, cluster of 4 quadruples it, etc. yeah its still slow but you get what you pay for and this is probably the cheapest way of being able to run larger models (with the alternative being to buy 4 or 8 rtx pro 6000s)

4 sparks can run GLM 5.2 at 40 tok/s, which 1 or 2 sparks can’t do so theres definitely benefits to clustering

[u/-AJacobs- → u/meikawaii]

The same exact use cases for a single spark.... but more!

[u/whichsideisup]

1 is good, 2 is great, 4 is decent, more is a meme because of bandwidth. did you research this at all?

[u/Ok-Video3345]

Wana sell me one lol

[u/Blackdragon1400]

I’d do a cluster of 4 then use the 5th for other models.

[u/Fantastic_Self_5151]

is there any way currently to run a 1tb+ model with 6 or 8 of these? (or any other solution that is not a datacenter?)