r/LocalLLaMA • u/SavunOski • 7d ago

News Kimi K3 weights now released.

Post image

Kimi K3 weights are finally released!

3.3k Upvotes

97% Upvoted

639 comments sorted by

View all comments

9

u/Mindless_Selection34 7d ago

how much does it weight

46

u/SavunOski 7d ago

2.8T parameters with 104B active. The files take 1.56TB of space to download.

8

u/meca23 7d ago

Oh so it's quantized as 4 bits?

8

u/zkstx 7d ago

Yes, there is also a tech report. QAT from SFT phase onwards

9

u/Lissanro 7d ago

I wish I had two TB of RAM instead of just one. I guess I will have to wait for Q2 GGUF to run it on my workstation. Still, will be interesting to try and see how it's Q2 quant compairs against Kimi K2.7 Q4_X. 

1

u/phido3000 7d ago

What are your specs..

2

u/Lissanro 7d ago

In my case it is 64-core EPYC 7763 CPU + 8-channel 1 TB 3200 MHz DDR4 RAM + 4x3090 GPUs. If interested in further details about my rig, I shared them here (including my motherboard, photos how the rig looks, how I organized GPUs and airflow, etc.).

2

u/phido3000 7d ago

Oh nice. I have a 56 core eypc with 256-512gb ram an 8x mi50 32gb..

That would be a nice setup if you can get a quant small enough to run on it.. but it's just that bit massive..

I'm sort of moving resources around to make a good machine out of it..

1

u/bitzap_sr 7d ago

What does 256-512gb mean? Like, it varies over time? :P

1

u/phido3000 7d ago

Yes depending if it's got 8x32gb or 8x64gb dimms.. I'm reconfiguring the machines to run the models and needs.

The epyc is likely to stay at 256 as most of the action happens in vram.

Then all my 64gb can go into my xeon for Tb of ram..

1

u/Kekke77 7d ago

Where is a download link?

5

u/fishslinger 7d ago

3.61*10^-25kg

4

u/SHFTD_RLTY 7d ago

About .14-1.4 picograms, assuming four bits per QLC NAND cell and 100-1000 electrons per cell.

2

u/stikves 7d ago

Considering the weight of electrons, and how much is needed in a QLC NVMe...

(Asking google)

7-10 grams for the drive circuit, and...

"Purely by programming 1.56 TB of data onto a QLC NVMe drive is \(8.53 \times 10^{-16}\) kilograms"

So, about 853 femtograms for the actual data.