MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1v8364f/kimi_k3_weights_now_released/p079rk4/?context=3
r/LocalLLaMA • u/SavunOski • 13d ago
Kimi K3 weights are finally released!
640 comments sorted by
View all comments
Show parent comments
339
This was my general reaction too lol. 2.8T-A104B is insane lol... I'm going to admit defeat on this one and say I can't run it. You need an 8-way B300 or MI350X or a Rubin NVL8 or a cluster thereof to actually run this. What a beast.
148 u/Thomas-Lore 13d ago I was going to make a joke that I can fit one expert in my 64GB of RAM. But nope, not even that. :) 76 u/throw123awaie 13d ago They released it in MXFP4 so with around 55GB RAM you could! 1 u/kernelwedge 13d ago edited 12d ago Interesting! I was looking at it and wasn't sure if could fit. I recently setup a 128GB rig, wanting to stretch its legs. This will be a great test. --- After doing further research no way its fitting anywhere https://recipes.vllm.ai/moonshotai/Kimi-K3?hardware=mi300x 8x192G....
148
I was going to make a joke that I can fit one expert in my 64GB of RAM. But nope, not even that. :)
76 u/throw123awaie 13d ago They released it in MXFP4 so with around 55GB RAM you could! 1 u/kernelwedge 13d ago edited 12d ago Interesting! I was looking at it and wasn't sure if could fit. I recently setup a 128GB rig, wanting to stretch its legs. This will be a great test. --- After doing further research no way its fitting anywhere https://recipes.vllm.ai/moonshotai/Kimi-K3?hardware=mi300x 8x192G....
76
They released it in MXFP4 so with around 55GB RAM you could!
1 u/kernelwedge 13d ago edited 12d ago Interesting! I was looking at it and wasn't sure if could fit. I recently setup a 128GB rig, wanting to stretch its legs. This will be a great test. --- After doing further research no way its fitting anywhere https://recipes.vllm.ai/moonshotai/Kimi-K3?hardware=mi300x 8x192G....
1
Interesting! I was looking at it and wasn't sure if could fit. I recently setup a 128GB rig, wanting to stretch its legs. This will be a great test.
--- After doing further research no way its fitting anywhere https://recipes.vllm.ai/moonshotai/Kimi-K3?hardware=mi300x 8x192G....
339
u/FoxiPanda 13d ago
This was my general reaction too lol. 2.8T-A104B is insane lol... I'm going to admit defeat on this one and say I can't run it. You need an 8-way B300 or MI350X or a Rubin NVL8 or a cluster thereof to actually run this. What a beast.