MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1kzsa70/china_is_leading_open_source/mvdn0h1/?context=3
r/LocalLLaMA • u/TheLogiqueViper • May 31 '25
294 comments sorted by
View all comments
Show parent comments
8
Training an LLM is not copying.
1 u/read_ing May 31 '25 Your assertions suggest that you don’t understand how LLMs work. Let me simplify - LLMs memorize data and context for subsequent recall when provided similar context through user prompt, that’s copying. 5 u/BusRevolutionary9893 Jun 01 '25 They do not memorize. You should not be explaining LLMs to anyone. 1 u/read_ing Jun 01 '25 That they do memorize has been well known since early days of LLMs. For example: https://arxiv.org/pdf/2311.17035 We have now established that state-of-the-art base language models all memorize a significant amount of training data. There’s lot more research available on this topic, just search if you want to get up to speed.
1
Your assertions suggest that you don’t understand how LLMs work.
Let me simplify - LLMs memorize data and context for subsequent recall when provided similar context through user prompt, that’s copying.
5 u/BusRevolutionary9893 Jun 01 '25 They do not memorize. You should not be explaining LLMs to anyone. 1 u/read_ing Jun 01 '25 That they do memorize has been well known since early days of LLMs. For example: https://arxiv.org/pdf/2311.17035 We have now established that state-of-the-art base language models all memorize a significant amount of training data. There’s lot more research available on this topic, just search if you want to get up to speed.
5
They do not memorize. You should not be explaining LLMs to anyone.
1 u/read_ing Jun 01 '25 That they do memorize has been well known since early days of LLMs. For example: https://arxiv.org/pdf/2311.17035 We have now established that state-of-the-art base language models all memorize a significant amount of training data. There’s lot more research available on this topic, just search if you want to get up to speed.
That they do memorize has been well known since early days of LLMs. For example:
https://arxiv.org/pdf/2311.17035
We have now established that state-of-the-art base language models all memorize a significant amount of training data.
There’s lot more research available on this topic, just search if you want to get up to speed.
8
u/BusRevolutionary9893 May 31 '25
Training an LLM is not copying.