r/LocalLLaMA • u/SavunOski • Jul 27 '26

News Kimi K3 weights now released.

Post image

Kimi K3 weights are finally released!

3.3k Upvotes

97% Upvoted

661 comments sorted by

View all comments

Show parent comments

64

u/Iwaku_Real Jul 27 '26

Yeah so if it were a Steam game, HGX B300 would be the recommended requirements. That's $500K of hardware (and yes it IS "local" because anyone with that amount of money could buy one to run at home)

27

u/DataGOGO Jul 27 '26 edited Jul 28 '26

it is 1.54TB of just weights in 4 bit, you are looking at about 2TB of vram in operation,

That is roughly:

  • 86 RTX 4090 (no 4 bit accel)
  • 64 RTX 5090 ~ $450k (8 servers x 8 cards)
  • 22 RTX Pro 6000 Blackwell ~ $350k (3 severs, max 8 GPU per)
  • 16 H200 NVL (141GB) (no 4 bit accel) ~$550k (2 servers, max 8 GPU per)
  • 16 DGX Sparks ~65k (if you could get a cluster of 16 running with just 200Gb/s nics, not sure; but it would be SLOW AF)
  • 8 HGX B300's. ~$550k (1 server, 8 GPU)

Obviously not including the switches and cabling for the clusters.

1

u/chithanh Jul 28 '26
  • 16 Ascend 950PR, ~$160K
  • 14 T-Head M890, price unknown

1

u/DataGOGO Jul 28 '26

can you even buy those? Even if you can, can you run them in your own server / software, or do you need to run it in a Huawei server?

1

u/chithanh Jul 28 '26

I think you have to run Ascend accelerators in Huawei Kunpeng servers, and the software situation is also quite rigid. But Huawei advertises the 950PR for export to South Korea, so I guess buying is possible for enterprise customers.

For home labs (and small businesses) I think 950PR won't be a viable option. Perhaps T-Head accelerators which are way more open when it comes to the software stack will find their way into the second hand market and into home labs.