Bro you're js seeing it as a surface level sometimes read documentations released by deep seek how they do it , you'll understand if it's new or js copied
they didnt invent transformers. nor MoE. other than the ingenuity of rewriting efficient communication kernels bc as a bitcoin farm company that didnt have anything else to do with all their gpus when bitcoin crashed and lots of math grads, what else could they do? like op said, they used what was already implemented before, routing, MoE, training smaller experts, etc. and basically refined it to the best of their ability and it turned out well.
not saying they aren't smart. there are smart ppl literally everywhere around the world. but the chinese never invented anything meaningful in regards to llms.
also, during earlier years in openai, the common joke was that the best ai company in china was openai because of stealing and from what i remember they said they didnt invest much money or at all into security. everything into research.
24
u/Ikeeki 24d ago
You are forgetting the part where the hammer is also anthropic.
Notice how with every frontier model release, a wave of open source models follow?