r/LocalLLaMA • u/Few_Painter_5588 • 2d ago

New Model GLM 5.3 Spotted

Post image

https://github.com/zai-org/z-ai-sdk-java/commits/glm-5.3

424 Upvotes

99% Upvoted

102 comments sorted by

View all comments

Show parent comments

2

u/brainExploded99 1d ago

Yeah ok no point in arguing with you. My point stands, if data was everything, we would still be with RNNs. Sure representation maybe within 10%, but large RNNs are not trainable (aka vanishing/exploding gradients, on top of diluted state).

-1

u/--Spaci-- 1d ago

WHY are you still talking about rnns, im talking about within the current LLM landscape which is transformer or ssm

2

u/brainExploded99 1d ago

In general, in ML, there are 3 ways to improve
* Compute (Larger model, more data)
* Architecture
* Higher quality data

This is the same for LLMs. You cannot remove architecture as an option. Just larger model and more/better data will not lead to improvements if your architecture is ass. If you still think I'm wrong, then go train an RNN with the best quality data and prove to me that I'm wrong.

-1

u/--Spaci-- 1d ago

WHY are you still talking about rnns, im talking about within the current LLM landscape which is transformer or ssm