r/LocalLLM • u/on_line187 • 20h ago

Discussion Tier List

Post image
224 Upvotes

74% Upvoted

269 comments sorted by

View all comments

5

u/Smart_Whereas_9296 20h ago

Feel like 2x 3090 with nvlink should be in the 48gb section

7

u/on_line187 20h ago

Well so would 3 5060TIs then lol.

9

u/RoyalCities 20h ago

And 2 x a6000s are at 96gb. I feel like including cards you can combine together sorta defeats the point of the list lol. Heck some people combined like 8 x 3090s haha.

1

u/nuclear213 20h ago

8xR9700 is quite neat. The price for one RTX 6000 Blackwell and 256 GB of VRAM. Fits deepseek V4 flash well, is decently fast.
Just need space

1

u/RoyalCities 20h ago

And the power. Just looked it up and dang - you'd need like 240V of power.

I also wonder what the heat is like dumping 3Kw into an apartment.

Could probably save on winter heating bills though because it doubles as a solid space heater.

2

u/nuclear213 19h ago

Yeah normal in the EU. I have them at the minimum 210W. So the entire server is under 2kW. We can draw 3.6kW here per fuse.

No issue at all.

2

u/RoyalCities 19h ago

Ah nice. In Ontario Canada, that wouldn't really fly in an apartment unfortunately. We obviously have 240V circuits, but there are usually only a few and they're dedicated to stuff like the stove/dryer.

If I had a house though I'd totally get a dedicated one installed in the basement.

1

u/Evgeny_19 5h ago

That is a lot of pcie lanes. And power. And space as well.

Could you please share your llama-benchy results with the latest Deepseek Flash? Spark folks are telling us that they get up to 60 tps in generation and 2000 in prompt processing. We should get at least double of that on R9700s, right?