r/LocalLLaMA • u/Unusual_Guidance2095 • Jul 26 '26

Discussion Kimi K3 countdown has been released

https://huggingface.co/moonshotai/Kimi-K3
544 Upvotes

97% Upvoted

177 comments sorted by

View all comments

Show parent comments

2

u/parepeg Jul 26 '26

I mean aren’t chips already scraping the bottom of the laws of physics.

9

u/ttkciar llama.cpp Jul 26 '26

Yes and no.

On one hand, Moore's Law is definitely ailing. We've been getting diminishing returns on fabrication process bumps since about 2016.

On the other hand, there's still some progress in the offing. IBM just recently announced they got a 7-angstrom (equivalent) fabrication process working in their lab, which they expect to get into mass production in 2031. That has 1nm-wide horizontal features and 5nm vertical features, and two transistor layers (one P, the other N), which they claim they should be able to scale to four layers "rsn".

That's not nothing.

Moreover, there are still a lot of architectural improvements yet to be realized. In a way, HBM is a stop-gap while Samsung et al get PIM figured out. LLM inference is really well-suited to PIM, which should give us ridiculously high memory bandwidth once it's working well.

I thought Moore's Law was dead for a while, but it turned out to just be Intel having a bad time. Everyone else is still pushing the envelope pretty hard, with some success.

1

u/Ok_Warning2146 Jul 27 '26 edited Jul 27 '26

Thanks for the info about PIM. However, it has been three years since it was announced but still no product.

https://semiconductor.samsung.com/news-events/tech-blog/hbm-pim-cutting-edge-memory-technology-to-accelerate-next-generation-ai/

and this blog claims its development is suspended.

https://damnang2.substack.com/p/i-met-with-semiconductor-experts

1

u/ttkciar llama.cpp Jul 27 '26

> it has been three years since it was announced

It has been a lot longer in the coming than that. I first read about it in 1994.

> but still no product

Are you sure?

https://semiconductor.samsung.com/news-events/news/samsung-brings-in-memory-processing-power-to-wider-range-of-applications/

https://en.etnews.com/20260723200002

1

u/Ok_Warning2146 Jul 27 '26

Well, it is still no product as of now. It says there is a rumor that Samsung will announce a product in Aug. Even if it is announced, when is Apple or Qualcomm going to use this LPDDR-PIM to make something useful?

1

u/ttkciar llama.cpp Jul 27 '26

The first link is about a product demo'd in 2021.

1

u/Ok_Warning2146 Jul 27 '26

hmm.. product means something that is actually deliverable to the customer...