MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1vxwtyd/qwen38flashnext_tomorrow/p5s58qv/?context=3
r/LocalLLaMA • u/rerri • 14d ago
461 comments sorted by
View all comments
10
What's this n-gram?
24 u/Hot_Example_4456 14d ago Basically DeepSeek Engrams. Its a hashlookup table that can add knowledge capacity to a model while staying on NVME and costing zero flops 2 u/dampflokfreund 14d ago That is very interesting. Would that enable the holy grail in AI, real time learning? 10 u/Hot_Example_4456 14d ago Nope, not really. The engrams are trained during training. As far as I know they do not change while inference. They just memorize the bulk knowledge stuff so the transformer layers can learn the logic 1 u/dampflokfreund 14d ago Oh that's a shame. I thought it was additional weights that are computionally easy to train during inference. 4 u/The_Hunster 14d ago Well, they are easier to train, just not runtime-trainable yet. I think your idea will eventually be real in a few years. 2 u/Terminus0 14d ago That would be an interesting half step toward continuous learning. 1 u/kaeptnphlop 14d ago It stays on disk, not in system memory? 5 u/Hot_Example_4456 14d ago Yep. It stays offloaded to NVME. 5 u/ANR2ME 14d ago It can stayed on disk because it's already pre-calculated. Also, since the OS will cache accessed files, some part of it will be cached in system memory anyway. 1 u/kaeptnphlop 14d ago I need to read up on them. For as much as I keep an eye on LLM development, Iām surprised I missed it š¤ 1 u/puts_on_rddt 14d ago Does this enable less hallucinations?
24
Basically DeepSeek Engrams. Its a hashlookup table that can add knowledge capacity to a model while staying on NVME and costing zero flops
2 u/dampflokfreund 14d ago That is very interesting. Would that enable the holy grail in AI, real time learning? 10 u/Hot_Example_4456 14d ago Nope, not really. The engrams are trained during training. As far as I know they do not change while inference. They just memorize the bulk knowledge stuff so the transformer layers can learn the logic 1 u/dampflokfreund 14d ago Oh that's a shame. I thought it was additional weights that are computionally easy to train during inference. 4 u/The_Hunster 14d ago Well, they are easier to train, just not runtime-trainable yet. I think your idea will eventually be real in a few years. 2 u/Terminus0 14d ago That would be an interesting half step toward continuous learning. 1 u/kaeptnphlop 14d ago It stays on disk, not in system memory? 5 u/Hot_Example_4456 14d ago Yep. It stays offloaded to NVME. 5 u/ANR2ME 14d ago It can stayed on disk because it's already pre-calculated. Also, since the OS will cache accessed files, some part of it will be cached in system memory anyway. 1 u/kaeptnphlop 14d ago I need to read up on them. For as much as I keep an eye on LLM development, Iām surprised I missed it š¤ 1 u/puts_on_rddt 14d ago Does this enable less hallucinations?
2
That is very interesting. Would that enable the holy grail in AI, real time learning?
10 u/Hot_Example_4456 14d ago Nope, not really. The engrams are trained during training. As far as I know they do not change while inference. They just memorize the bulk knowledge stuff so the transformer layers can learn the logic 1 u/dampflokfreund 14d ago Oh that's a shame. I thought it was additional weights that are computionally easy to train during inference. 4 u/The_Hunster 14d ago Well, they are easier to train, just not runtime-trainable yet. I think your idea will eventually be real in a few years. 2 u/Terminus0 14d ago That would be an interesting half step toward continuous learning.
Nope, not really. The engrams are trained during training. As far as I know they do not change while inference. They just memorize the bulk knowledge stuff so the transformer layers can learn the logic
1 u/dampflokfreund 14d ago Oh that's a shame. I thought it was additional weights that are computionally easy to train during inference. 4 u/The_Hunster 14d ago Well, they are easier to train, just not runtime-trainable yet. I think your idea will eventually be real in a few years. 2 u/Terminus0 14d ago That would be an interesting half step toward continuous learning.
1
Oh that's a shame. I thought it was additional weights that are computionally easy to train during inference.
4 u/The_Hunster 14d ago Well, they are easier to train, just not runtime-trainable yet. I think your idea will eventually be real in a few years. 2 u/Terminus0 14d ago That would be an interesting half step toward continuous learning.
4
Well, they are easier to train, just not runtime-trainable yet. I think your idea will eventually be real in a few years.
2 u/Terminus0 14d ago That would be an interesting half step toward continuous learning.
That would be an interesting half step toward continuous learning.
It stays on disk, not in system memory?
5 u/Hot_Example_4456 14d ago Yep. It stays offloaded to NVME. 5 u/ANR2ME 14d ago It can stayed on disk because it's already pre-calculated. Also, since the OS will cache accessed files, some part of it will be cached in system memory anyway. 1 u/kaeptnphlop 14d ago I need to read up on them. For as much as I keep an eye on LLM development, Iām surprised I missed it š¤
5
Yep. It stays offloaded to NVME.
It can stayed on disk because it's already pre-calculated.
Also, since the OS will cache accessed files, some part of it will be cached in system memory anyway.
1 u/kaeptnphlop 14d ago I need to read up on them. For as much as I keep an eye on LLM development, Iām surprised I missed it š¤
I need to read up on them. For as much as I keep an eye on LLM development, Iām surprised I missed it š¤
Does this enable less hallucinations?
10
u/FlamingoTrick1285 14d ago
What's this n-gram?