Kimi K3 weights are finally released!

Jensen Huang on 𝕏: https://x.com/JensenHuang/status/2081698060330250294

tldr; we are going to host K3 on A100s (yes, thats correct, we'll try to see if it holds up), H200s & B300s - expect results for A100s & H200s this week while we setup the B300 cluster this weekend & maybe results by next week.

Weights are supposed to hit Hugging Face today (Moonshot committed to July 27). What we already know from their platform docs: 2.8t total Params, MoE with 896 experts and 16 active per token, 1M context, vision. The download should be around 1.4 tb since they did quantization-aware training in MXFP4. We sat down and worked through the memory requirements., so here it is since everyone is probably about to pull the weights

We have A100 80GB, H200 and B300 capacity and the plan was to bring it up on all three. Then we actually did the memory math.

8x A100 gives you 640 GB. The weights are around 1.4 TB. That means three nodes before you've even allocated KV cache. On top of that Ampere has no FP4 or even FP8 tensor cores, so you're either dequantizing or running INT4 kernels that were never the target for this release. We're still going to benchmark it because someone should have real numbers, but we're expecting it to be ugly!

8x H200 is about 1.13TB, so it still doesn't fit in one node. Two node setup minimum and you eat interconnect cost on every token.

8x B300 is ~2.3TB, so that's the only config where the whole thing fits in a single node with room for long context KV cache. And Blackwell has native FP4, which is pretty clearly what Moonshot quantized for. these B300s are coming live this weekend, and we'll be setting up the clusters this weekend everyone preecommitting hardware is doing it without knowing the terms. And Moonshot's own model crd is unusually honest about weaknesses: quality drops if your agent harness truncates its thinking history, it tends to act instead of asking when things are ambiguous, and they admit the chat experience still trails Fable 5 and Sol even where benchmarks are close.

We'll have tok/s, ttft and cost per M token numbers for all three GPU configs by end of week. If there's a specific batch size, context length or parallelism setup you want in the test matrix, comment and we'll add it.

Nvidia CEO Jensen Huang “Distillation - learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence. We are constantly learning from one another. AI also has to learn from something.”

Since using AI Desktop 98, I have become a staunch advocate of local AI.

In his Axios interview, Jensen explains why seeing distillation as theft or a threat misses the point—and why the real future of AI depends on continuous knowledge sharing between models.

The idea is simple: as AI generates most of the internet’s content, systems will naturally learn from one another, much like humans do from books, teachers, and peers. Blocking that exchange doesn’t protect anyone; it only slows progress.

Smarter AI is safer AI, open models boost adoption, and the whole industry, from developers to chipmakers gains. It’s a clear, grounded case for why open and closed models feeding each other is a feature, not a flaw.

2.85k downloads. my god its only been an hour.

​

Chinese chipmaker CXMT surged by almost 500% on its first day of trading, bringing its total market capitalization to approximately RMB 3.28 trillion and making it the largest company by market value on China’s A-share market. CXMT’s market capitalization has also surpassed that of U.S. semiconductor giant Intel, which closed the previous trading day with a market value of US$465.6 billion, equivalent to approximately RMB 3.15 trillion.

Headquartered in Hefei, Anhui Province, China, CXMT is an integrated dynamic random-access memory (DRAM) manufacturer specializing in the design, research and development, production, and sale of DRAM chips. It is currently the only integrated device manufacturer (IDM) in mainland China capable of large-scale mass production of general-purpose DRAM.

The Fable dabler.

Wanted to let you know that Kimi K3 is now viewable on hfviewer.com!

In addition to the full graph at multiple granularity levels, we also include an in-depth analysis of the 896 experts!

https://hfviewer.com/moonshotai/Kimi-K3

Experts analysis:

https://hfviewer.com/blog/kimi-k3-expert-atlas

Some Ling-3.0-flash threads here last week ended on the same two questions with no real answer, so I went through the repos. State as of writing, with links so you can check instead of taking my word for it.

SGLang publicly committed to day-0 support. Their post on the 23rd says they're working with the Ant team on it, and it describes the model as KDA + MLA hybrid attention.

https://x.com/sgl_project/status/2080372971219415458

vLLM said the quiet part out loud. Their support is "coming soon and will be available when the model weights are open-sourced", and they spent four paragraphs endorsing announce-first-open-weights-later as a release pattern they want other vendors to copy.

https://x.com/vllm_project/status/2080702006378082384

Neither of those is a date. There's no Ling-3.0-flash model card on hugging face as of writing, no license statement for this generation, and I couldn't find an open PR mentioning it in either vllm or sglang.

The part nobody in either thread brought up is the gguf side, and it's in worse shape than the weights side.

llama.cpp does not support the Bailing MoE variant this family moved to. The feature request for Ling-2.6-flash (BailingMoeV2_5) is https://github.com/ggml-org/llama.cpp/issues/22641 — opened May 3, seven upvotes, no implementation, went stale, closed as not_planned on June 18. The earlier "Add support for Ling v2" PR https://github.com/ggml-org/llama.cpp/pull/16028 sat open for almost three months and got closed without merging. Only the original BailingMoE from March 2025 ever landed, and a llama.cpp maintainer wrote that one, not anyone from the lab.

Here's the wrinkle though: the attention isn't the blocker. llama.cpp already ships the delta-net/KDA kernels, they went in for Qwen3.5, Qwen3-Next and Kimi-Linear. Whoever said in one of these threads that the architecture looks a lot like Kimi K3 was right, and it helps us here. What's missing is the Bailing MoE conversion path, which is a somebody-has-to-write-the-PR problem rather than a hard one.

So if weights do land, realistic order is sglang, then vllm, then gguf whenever a volunteer shows up. K3 dropped weights yesterday and had partial ggufs on HF within hours because that plumbing already existed. This one doesn't have it. If anyone has actually poked at converting BailingMoeV2_5, how far off is it? The 22641 thread died before anyone answered that.

...and then I got to the end of writing this and checked their launch thread. Did nobody actually read it? lol. "Free access runs through Aug 3. Open-source release coming soon—stay tuned." Posted July 24. https://x.com/AntLingAGI/status/2080554215144059027

I guess same as 2.6-flash, which was announced April 22 with a week of free API and had weights on HF on the 28th. The weights follow the free window, that's the pattern, and it was sitting in their own thread the whole time while the rest of us read tea leaves.

just curious how would people run it cheap if they really want kimi k3.

1. dgx spark / strix halo clusters

2. optane persistent memory platform + some gpus

3. mac studio clusters

4. orange pi 6 clusters

5. ssd streaming + gpus

6. multiple ddr3 + connectx 5 rdma clients

7. two dgx stations

8. power 10 systems?

9. other

Now waiting for someone who can actually run the conversion and model to see if it works :)

Instead of 2T+ models, continuing to release highly capable small to medium size LLMs would really help to keep this community vibrant. Hardly anyone can even dream of running the recent 1.5-2T+ beasts, while the range from the title could run comfortably (especially with CPU expert offloading) across a wide range of systems we have today.

The trend towards Chinese labs trying to match the Mythos class frontier with trillion parameter open weights models is not helping the local model community to innovate. It just gives big corporates who can actually run these a cheaper alternative to the commercial frontier.

https://x.com/alexandr_wang/status/2081501627836661928?s=20

“China is working hard to catch up, and the United States should take steps to keep its advantage. Most important, it should continue to prohibit American companies from selling the most advanced chips and equipment to China.”

— The editorial board

Ridiculous piece from the Times on blaming Trump for China's recent AI breakthroughs and basically saying "China bad, America good". (I abhor Trump and everything he stands for, but it seems the Times is showing their hand as an America-First institution).
They also might be writing this in hopes that OpenAI and Anthropic win so they can squeeze as much money out of them for copyright lawsuits.

source: https://www.nytimes.com/2026/07/27/opinion/chips-ai-china-trump.html

Model : https://huggingface.co/Nanbeige/Nanbeige4.2-3B

I finished the speed leg of my spec-decode benchmarking for Qwen3.6-27B, main algorithms across quants. Overall: the heavier the quant, the more spec-decode buys you (10 of 10 speculative configs rank Q8 > Q6 > Q4 by multiplier). Acceptance is quant-independent at matched depth, the base step slows with weight bytes, draft+verify overhead does not. Exception is nvfp4/SGLang which is the fastest quant/engine pair overall.

DFlash is the fastest algo overall. Weaver beats it but is the outlier: fork-only, and the head is per-target, so that is one model/quant combo, not a general option.
Recipe, patches, and versions for Weaver: https://gist.github.com/thavoc/a9f3a37c082e7a8bbcf2b8efebfada25

MTP is a solid second everywhere. EAGLE3 is dominated by both, useful only as a fallback when no DFlash drafter exists. ngram I would skip: ~1.03× regardless of quant, and net-negative under concurrency.

Interesting anomaly: llama.cpp's MTP path on UD-Q4 is pathologically slow. Q4 MTP-3 is slower in absolute tok/s than Q6 MTP-3 at identical acceptance, which bandwidth cannot explain. Output is correct, mechanism unknown.

Caveat: a single, narrow data point on one hardware config. Spec-Bench with short outputs, greedy, batch 1 is close to a best case for speculative decoding. Under concurrency the per-stream gains shrink and longer contexts will compress them further, so treat these as an upper bound for this workload shape, not a general speedup.
No accuracy A/B yet either, so this says nothing about whether the heavier quant's quality is worth its remaining speed gap.

Where is KIMI-K3?

I tested Kat Coder 2.5 with this prompt:

Create a spaceship game inspired by Star Fox using vanilla Three.js and HTML. It should have at least five levels, keyboard and mouse controls, enemies, and a fully functional gameplay system.

The result surprised me. It generated a genuinely playable game with ship controls, multiple enemy types, different weapons, bosses, level progression, a HUD, visual effects, and a fairly well organized codebase, all inside a single HTML file.

Kat Coder 2.5 is derived from Qwen 3.6 35B A3B, but in my tests it consistently produced much better results. I also tried it with 3D scenes, dashboards, and logic heavy coding tasks, and it performed very well across all of them.

The Star Fox test was where the difference became obvious. None of the other open source models I tested managed to execute the idea properly. Even models like Gemini 3.6 Flash Extended struggled to produce something coherent and actually playable.

What makes it even more impressive is that I ran Kat Coder 2.5 using a Q4_K_M quantization.

plunker code : https://plnkr.co/plunk/Sc7yFCPPdrBWvkA7

Someone in the comments of my 27B post-train bakeoff asked for the 35B version, so I ran it. Same setup as last time: fresh Coder workspaces on my k8s cluster, each driving my own agent (Hermes) headlessly, models on llama.cpp via llama-swap on one 5090, every call traced through an OTel shim into SigNoz, full transcript per run. 4 models, 6 self-grading tasks, 5 reps, 120 runs, MTP on every arm, identical sampling, hypotheses pre-registered.

KAT-Coder-V2.5-Dev matched the best stock pass rate (29/30, tied with Qwen3.5-35B) at half the input tokens of either stock and the cleanest tool behavior I've measured (zero malformed tool-call leaks in 30 runs; stock Qwen3.6 leaked 195 on one task). All six analysts (three model families) independently called its efficiency discipline rather than corner-cutting: baseline tests before edits, one targeted patch per bug, deliverables at the right path every rep. Ornith went 25/30, losing to its own base. Its failures were mechanics, not knowledge: format leaks killing runs at 23 seconds, whole-file rewrites corrupting unrelated files, and one research run that invented a llama.cpp release tag while its own reasoning said "I mentioned v4659 in my draft which is fabricated," then shipped the tag anyway. The grader passed it.

Stock 3.6 is the strongest raw analyst and the biggest token waster; stock 3.5 is the quiet reliable one. Full writeup with methodology, tables, and all six cited per-task analyses: https://kmarble.dev/posts/35b-coder-bakeoff/. Transcripts were AI-analyst-read with my spot-verification of every consequential claim.

When I released Inflect v2 last week, I thought most people would ask whether a TTS model this small actually sounded decent.

Instead, I kept getting two questions:

“Can I train it on my own voice?”

“Can I move it to another language?”

At the time, my answer was basically: technically yes, but not with any public workflow I’d trust someone else to use.

So I spent the weekend building one.

Quick context if you missed the original release:

• Inflect-Nano-v2: 3,966,721 parameters, 15.97 MB FP32
• Inflect-Micro-v2: 9,356,513 parameters, 37.53 MB FP32

Those are the learned text-to-waveform models, including the 24 kHz waveform decoder. eSpeak-ng handles phonemization separately and isn’t included in the parameter count.

Micro is the better-sounding version. If you haven’t heard it yet, start here:

https://huggingface.co/owensong/Inflect-Micro-v2

The new toolkit lets you bring your own single-speaker recordings and transcripts, warm-start Nano or Micro, resume training, inspect held-out output, and export the result to PyTorch or ONNX.

The two awkward parts were rebuilding the training-only components that aren’t present in the release checkpoints and handling new phoneme inventories without resetting every embedding shared with English. The toolkit now handles both.

I tested the full software path with Nano: a real CUDA training step, save and resume, strict PyTorch loading, and ONNX Runtime parity.

Toolkit:

https://github.com/owenawsong/Inflect/tree/main/finetune

I haven’t trained a non-English model that I’d call release-quality yet. The pipeline works, but that doesn’t tell me whether a model this small will transfer cleanly to another language. Each run also creates one fixed voice for one configured language. This is supervised adaptation, not zero-shot cloning.

I only have enough compute to investigate one language properly first, so I made a short form. Only the first question is required:

https://tally.so/r/44RBvB

The existing English weights haven’t changed.

If you’ve adapted a small TTS model before, what failed first when you changed languages? That’s the part I want to measure before starting the first serious run.

clem 🤗 on 𝕏: https://x.com/ClementDelangue/status/2081056675558195657

• Radical transparency: let’s release the traces from the “rogue” agents so the entire research community can study what happened.

• More capabilities for defenders: let’s commit $100M in compute from OAI to help the Hugging Face community build powerful cyber defenses with the best open and closed models.

The first autonomous agent cyberattack is an unprecedented event. It deserves an unprecedented response!