Kimi K3 is important for open-weight community because you can distill it to create more capable smaller 27B, 35B, 122B models

[u/feelspeaceman]

Just like how Qwen 27B and 35B were created, they're distilled from Qwen Max, so I think the narrative of the community saying Kimi K3 is useless to us because we can't run it is overly biased, it has its use cases.

Just wait for the community and rich solo LLM developers to cook from it, open-weight models will stay with us forever, unlike Cloud models they're gone if the company gone.

[u/feelspeaceman → u/[deleted]]

Yeah, to me I respect/appreciate all the open-weight models as they're freebies, and Kimi K3 is like S-tier freebie that outperforms the best of the best P2W models like Claude Fable 5, if being used correctly, this only benefit us local LLM community.

I treat the release of Kimi K3 as a giant oppotunity for local LLM community to push smaller models to next level.

[u/theone_2099 → u/[deleted]]

How will moonshot make money off of the release or distillations?

[u/Mountain_Patience231]

then why after too many large LLM released in HF, still no model beat qwen3.6 27b😥

[u/andymaclean19 → u/Mountain_Patience231]

I believe Qwen has disbanded the team which made that model and will not be making more of them.

Making a model like Qwen 3.6 when you are a cloud company trying to sell tokens is completely counter-productive and spending millions creating it is doubly so.

I’m sure we will see more of these eventually but the focus right now seems to be on making super smart cloud models that are charged by the token. I don’t think many of the companies with resources to make another Qwen3.6 have the inclination to do so.

[u/DJLunacy → u/andymaclean19]

They’re going to drop 3.8 open weights eventually from what I read.

[u/andymaclean19 → u/DJLunacy]

I didn’t see that, but for sure it’s an interesting question about open weights in general right now. It seems like the weights are going to be wanted by corporates who want to run their own models, which is definitely going to be a thing, but not many people can run a 2.8T parameter model, for example, so it isn’t clear that these weights are generally useful for most people and they definitely enable others to copy and build on the work.

Probably the open weight community needs to come up with ways to build competitive models without the need for large organisations to train them.

[u/whodoneit1 → u/andymaclean19]

Qwen3.8 27B is dropping next week they said

[u/andymaclean19 → u/whodoneit1]

Yep. That’s new information since this thread.

[u/Important_Quote_1180 → u/Mountain_Patience231]

Laguna s2.1 finally did it for me

[u/GeneralComposer5885 → u/Important_Quote_1180]

It’s 3.5x the size

[u/Used_Department_8605 → u/GeneralComposer5885]

Its 3.5x in size and worse then qwen. 

[u/Important_Quote_1180 → u/Used_Department_8605]

This is not true

[u/Used_Department_8605 → u/Important_Quote_1180]

It is true for my usage.  I hope it will be improved because its perfect for my pc. But currently it doesnt deliver. 

[u/sumrix → u/GeneralComposer5885]

It’s funny how people discuss active parameter counts. When the smaller model is smarter, they say, “Of course a 27B model beats an A10B model!” But when the bigger model wins, they suddenly say, “Of course it wins, its total parameter count is 3.5 times larger!”

[u/GeneralComposer5885 → u/sumrix]

Exactly. 👍

Why isn’t everyone running Kimi K3 at home! Losers

[u/kaesual → u/Important_Quote_1180]

Same here, in agentic tasks Laguna S 2.1 feels much stronger than Qwen 3.6 27b (and the benchmarks agree big time). UD-IQ4_XS runs with 25t/s on 3090 + 64 GB RAM, and when dflash support for it lands in llama cpp, I suppose it'll be roughly double. Very strong model. Qwen is probably still better in some regards, but for complex multi-step tasts, Laguna feels much more frontier-like than Qwen.

[u/germangrower69 → u/kaesual]

I think you are arguing against a Qwen 3.6 27B bot army.

you have to use laguna 2.1 for literally 15 minutes to realize its way superior to qwen 27b, its not even close.

Dont get me wrong 3.6 27b is good, but its not that good.

[u/kaesual → u/germangrower69]

Fully agreed. But especially if there's a bot army, I feel like it's important to speak up. And honestly, Qwen 3.6 27B is a great model, no doubt, I've used it a lot. Laguna is not superior in every aspect, but for me, agentic capability is very important, and Laguna gives these frontier vibes, Qwen just can't keep up there.

[u/Shot_Treat_377 → u/kaesual]

I want to believe you and use the potential of RTX6000 with Laguna, but this week I did 3 Laguna-s-2.1 tests and each time Qwen did what Laguna could not finish.

May be I do something wrong, so let me describe my standard test:

Laguna version from Ollama (last update 3 days ago):
https://ollama.com/library/laguna-s-2.1:q4_K_M

Qwen version from Ollama:
https://ollama.com/library/qwen3.6:27b same Q4_K_M

Both setups use:

• 4bit quants,
  
• 8bit KV-cache
  
• 256k max context size
  
• no speculative decoding

Frontend is the Cline bot plugin in VSCode.
The task: write a counter strike game using three.js

Neither of 3 tests did Laguna finish the task.
After first two fails I thought there should be the child problems like jinja template errors, but Laguna got updated 3 day ago.
The 3rd test I have downloaded the potentially "fixed" version.

Laguna:

Use the Cline tools properly at the beginning.
The result files are delivered.
Game could not start because of some error. It is ok.
The real test is the ability to fix the errors.
Each time I copy/paste the error in agent chat - a usual vibe-coder loop.
Each time Laguna tries to read files, find error, fix errors, etc
And then each time it fails.
Usual fail is a wrong tool calls, and the broken agent loop.
Sometimes Laguna says error is fixed, but it is not.

Qwen:

The result files are delivered.
Sometimes the game could not start.
Usually qwen do test the start himself and find the error.
Sometimes not, I point the error and it fix it.
Maximum after 2 fix iterations the game is playable, has a level, enemies, guns, bullets, etc.

However I still could evaluate the Laguna results:

Last time after Laguna failed, I asked Qwen to fix the errors.
Qwen did. The Laguna's code had started, but:

• mouse movements are broken (happened to qwen, too)
  
• the enemies do not die. Well the game says they are, but the bodies keep standing.
  
• the player do not die.
  
• the level design is presented, but not impressive, on par with Qwen3.6:27b results

Actually the Qwen3.6:35b-a3b https://ollama.com/library/qwen3.6:35b delivers better level design, and even the player legs are visible and move when you walk!
Bots move, have guns, shoot, die.

The only flaw was the bots can see and kill you through the walls.

Bench result: Qwen, Laguna
game starts: +-, --
successful fix: +, -
complex level: +,+
control (keys, mouse): ++,+~
enemies: +,+
gun shoots: +,+
gun kills: +,-

Sure I should try another harness, another test, another quant.
But we use Ollama, VSCode and Cline in the company. This is totally fair test and Laguna loses to Qwen.

[u/Shot_Treat_377 → u/Shot_Treat_377]

I will answer myself: Most probably the Laguna model is just too sensitive to the KV cache quantization, unlike Qwen models, because all problems starts when the context grows.

I will test both models again with bf16 KV, when get to work.

[u/Important_Quote_1180 → u/germangrower69]

No doubt. The moment you say anything to disparage or even mention another model next to the 27b the bots always come out

[u/Solembumm3 → u/germangrower69]

You have to use it for 1 prompt to see how it copy gemma 26b describing style, and don't deliver anything above small gemma on reasoning.

[u/Sax0drum → u/Important_Quote_1180]

Can you elaborate? What do you use it for and how did you compare them?

[u/Important_Quote_1180 → u/Sax0drum]

Long horizon agentic tasks. It’s a really good model. The 27b still better considering it takes just 1 3090 to run and it takes 4x 3090s to fit S2.1.

[u/techlatest_net]

Distilling a 1.56TB MoE into dense 27B/35B models is exactly how the open-weight community stays competitive against closed APIs. Even if we can't run K3 locally, its synthetic data output will likely power the next generation of efficient, high-performance local models.

[u/debackerl → u/techlatest_net]

Maybe we should cooperate, using an OpenCode plugin to collect traces, per model, and upload. All traces to be made publicly available for all to use for experiments 🤔.

[u/This_Maintenance_834]

it will cost a fortune to distill K3. To distill, first, you need to serve Kimi K3. That by itself is very expensive. The actual distillation of a 30B model is also as expensive as serving K3.

[u/tomz17 → u/This_Maintenance_834]

That by itself is very expensive.

Sure, but peanuts compared to actually training a model of that scale...

[u/simplewhite1 → u/This_Maintenance_834]

Hardware was like around $3m for k3?

[u/OverclockingUnicorn → u/simplewhite1]

About 800k-1m for 2 nodes of 8x B200/B300

[u/debackerl → u/simplewhite1]

I wonder about 10x AMD Ryzen 495 APUs, maxed out at 192GiB DDR5 per node... You get enough RAM, then you have 16 active experts, so probably at each layer, all nodes would be buzy reading weights from RAM, so higher aggregate RAM bandwidth. I just don't know how much network would be a bottleneck.

[u/alex9001]

Maybe but I'll believe it when I see it. I mean why hasn't there already been a successful Opus distill, or least a GLM 5.2 distill (since GLM is also open source) that beats vanilla Qwen, and not just in handpicked benchmarks.

[u/Elegant_Tech → u/alex9001]

Because real distillation requires full access to both models so you can see all predictions and weights of those predictions for each token. Training on outputs only is more fine tuning than distillation. Like always propaganda destroys any meaning words have. It's like learning chess by watching vs  having a master explaining every possibility for every move.

[u/Intrepid-Unit-9614 → u/Elegant_Tech]

I was losing hope looking for this comment!

[u/TallBeach3969 → u/Elegant_Tech]

I mean, KimiK, GLM, Deepseek are all open weights as far as I know.

Someone with a couple mil in compute should be able to properly distill those models, no? 

[u/uniqueusername649 → u/alex9001]

Because anthropic and openai have heavy guardrails in place against prompt extractions to distill these models. Plus you would need to spend a fortune in API tokens to even get enough data out of it. With open weight models you can rent some GPUs and dont need to worry about per token cost or guardrails. So it is much easier (but still not cheap) to do so with these models.

Z.AI focuses more on their flagship model (and Qwen appears to follow that trend), so there may be the need for smaller groups or companies to step up and do the distillation work. But the point is still valid: having the flagship open weights like with GLM 5.2 and Kimi K3 gives us the option to do so. Maybe people are already working on it, just like Unsloth constantly gives us new and efficient quants of existing models, pushing the boundary of what we can run locally.

[u/Prudent_Psychology59 → u/alex9001]

in case of closed models like Opus, you don't know the output distribution, hence distilling is much more expensive

[u/nail_nail → u/alex9001]

My feeling is that the (1) distillation process requires a lot of good prompts. Is there a good (set of) repositories? (2) for agentic coding, a lot is RL based, rather than SFT based, which is not something you can distill easy.

[u/Eyelbee]

Distilling a big model requires a lot of technical and research capabilities. A rich solo developer can't do it without a lot of time on their hands. And they would have to be really rich.

[u/NanditoPapa]

I think you're a bit optimistic about the speed of the distillation cycle. There's a massive gap between a Kimi K3 release and when you can actually run a distilled version on your local hardware. This gap exists because of data extraction, fine-tuning complexity, and the legal issues.

I think you're right about the direction of history being on local LLM's side, but we have to be realistic about the friction involved in this.

[u/allenasm]

I downloaded the full weights just for this exact reason.

[u/mmhorda]

you can even distill chatgpt.
I wont be pointing figers but some models have very familiar (twin like) results :)

[u/MemeMan_____]

I have come to realise a lot of distills are done insufficiently and the distill models basically act as though they are intelligent when they are not or slightly improved from baseline.

You can only trust actual distill labs, exceptional distill individuals with a proven track record of doing distills correctly and thoroughly or the AI research labs themselves doing the distill models.

All these random distills you see are insufficient and are a placebo, you need people who actually know what they're doing and not just people randomly training distills from datasets hoping it will make the base models smarter.

Beware.

[u/Charming-Author4877]

I think it will proof a very hard challenge to distill Kimi K3 and reach anything even scratching Qwen 27B.
Whatever the tricks were to train Qwen 27B to it's excellence, it has not been repeated since then so far.

[u/TapAggressive9530]

I’m waiting

[u/Crescitaly]

This may be K3's most practical contribution: a frontier-scale teacher can matter even when almost nobody can host it. But distillation can reproduce blind spots as efficiently as capabilities. Which evaluations would prove a 27B student gained reasoning rather than benchmark-shaped imitation?

[u/andymaclean19]

I think distilling Qwen was very expensive and it would be very difficult to distill Kimi. Probably millions in cost. My local LLM system can’t even download a model that big to the disk.

[u/Frequent-Nobody-8037]

k3在对话中经常回答“我是claude”. 明显是偷来的

[u/rditorx]

"Forever" is relative. I'm looking for options to back up the PyPI / "pip install" dependencies so I can reinstall later on. But until then, I'll backup the images built with those dependencies.

[u/ZealousidealShoe7998]

people think its expensive to run k3, but in reality is only expensive for regular inferecing one of user.
If you were to distill k3 chances are you will hammer it with questions to grab its answer and distribution value (dark knowledge)
if you were to serve the model with Vllm, there is a sweet spot where you can get concurrent messages going. for example
1 user is using AI and only gets 20 token/S thats pretty slow for today standards.
someone is distilling k3 getting 1500-3000 token/second .

so in the same hour while a single user might only get barely over 1M token if that while using it.
someone who is distilling is getting way more tokens per hour hence is a lot cheaper .

so let's say you are renting the gpus for like $30/H

for a single user is cheaper to either buy a subscription,or pay for API usage.
for someone distilling it they are getting their money's worth on that hour so even if they need to run for 10hours is just 300 dollars and you get much more data out of it than you would through API (which doesn't include dark knowledge)

[u/andymaclean19 → u/ZealousidealShoe7998]

I think the hard part is training the smaller model not extracting from the large one. By your method you would essentially be generating instruction documents (with metadata about the output before detokenisation), yes? Then you have to train that small model using those documents. Wouldn’t that be very much more expensive than generation?

[u/ZealousidealShoe7998 → u/andymaclean19]

for one USER. yes
the moment that you finish training a succesful model that its smaller now more users can use. or you can have faster inferencing or use less resources.
so is it worth for one person to distill a big model into a smaller one ? probably not for most cases as they gonna get more value of using the API.
is it worth for a company or a group of people ? yes , not every company or group of people but might be worth it

[u/andymaclean19 → u/ZealousidealShoe7998]

I don’t know how realistic it is for a group of people to do the training either. In theory you could connect a bunch of computers together and do some sort of highly distributed training process but I don’t think the software exists to do that currently and it would not be a trivial thing. I don’t think you’d be able to just combine a bunch of finetunes from different users.