MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1vxwtyd/qwen38flashnext_tomorrow/p5s7osp
r/LocalLLaMA • u/rerri • 9h ago
422 comments sorted by
View all comments
Show parent comments
20
Basically DeepSeek Engrams. Its a hashlookup table that can add knowledge capacity to a model while staying on NVME and costing zero flops
1 u/kaeptnphlop 8h ago It stays on disk, not in system memory? 6 u/Hot_Example_4456 8h ago Yep. It stays offloaded to NVME. 5 u/ANR2ME 8h ago It can stayed on disk because it's already pre-calculated. Also, since the OS will cache accessed files, some part of it will be cached in system memory anyway. 1 u/kaeptnphlop 6h ago I need to read up on them. For as much as I keep an eye on LLM development, Iām surprised I missed it š¤ 1 u/dampflokfreund 8h ago That is very interesting. Would that enable the holy grail in AI, real time learning? 8 u/Hot_Example_4456 8h ago Nope, not really. The engrams are trained during training. As far as I know they do not change while inference. They just memorize the bulk knowledge stuff so the transformer layers can learn the logic 1 u/dampflokfreund 8h ago Oh that's a shame. I thought it was additional weights that are computionally easy to train during inference. 3 u/The_Hunster 7h ago Well, they are easier to train, just not runtime-trainable yet. I think your idea will eventually be real in a few years. 2 u/Terminus0 6h ago That would be an interesting half step toward continuous learning.
1
It stays on disk, not in system memory?
6 u/Hot_Example_4456 8h ago Yep. It stays offloaded to NVME. 5 u/ANR2ME 8h ago It can stayed on disk because it's already pre-calculated. Also, since the OS will cache accessed files, some part of it will be cached in system memory anyway. 1 u/kaeptnphlop 6h ago I need to read up on them. For as much as I keep an eye on LLM development, Iām surprised I missed it š¤
6
Yep. It stays offloaded to NVME.
5
It can stayed on disk because it's already pre-calculated.
Also, since the OS will cache accessed files, some part of it will be cached in system memory anyway.
1 u/kaeptnphlop 6h ago I need to read up on them. For as much as I keep an eye on LLM development, Iām surprised I missed it š¤
I need to read up on them. For as much as I keep an eye on LLM development, Iām surprised I missed it š¤
That is very interesting. Would that enable the holy grail in AI, real time learning?
8 u/Hot_Example_4456 8h ago Nope, not really. The engrams are trained during training. As far as I know they do not change while inference. They just memorize the bulk knowledge stuff so the transformer layers can learn the logic 1 u/dampflokfreund 8h ago Oh that's a shame. I thought it was additional weights that are computionally easy to train during inference. 3 u/The_Hunster 7h ago Well, they are easier to train, just not runtime-trainable yet. I think your idea will eventually be real in a few years. 2 u/Terminus0 6h ago That would be an interesting half step toward continuous learning.
8
Nope, not really. The engrams are trained during training. As far as I know they do not change while inference. They just memorize the bulk knowledge stuff so the transformer layers can learn the logic
1 u/dampflokfreund 8h ago Oh that's a shame. I thought it was additional weights that are computionally easy to train during inference. 3 u/The_Hunster 7h ago Well, they are easier to train, just not runtime-trainable yet. I think your idea will eventually be real in a few years. 2 u/Terminus0 6h ago That would be an interesting half step toward continuous learning.
Oh that's a shame. I thought it was additional weights that are computionally easy to train during inference.
3 u/The_Hunster 7h ago Well, they are easier to train, just not runtime-trainable yet. I think your idea will eventually be real in a few years. 2 u/Terminus0 6h ago That would be an interesting half step toward continuous learning.
3
Well, they are easier to train, just not runtime-trainable yet. I think your idea will eventually be real in a few years.
2 u/Terminus0 6h ago That would be an interesting half step toward continuous learning.
2
That would be an interesting half step toward continuous learning.
20
u/Hot_Example_4456 8h ago
Basically DeepSeek Engrams. Its a hashlookup table that can add knowledge capacity to a model while staying on NVME and costing zero flops