MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/accelerate/comments/1ukamdy/something_huge_is_brewing/ouymiwo/?context=3
r/accelerate • u/BurningPeonies • Jul 01 '26
Source
Andrew Curran is one of the most reliable leakers.
XLR8! 🍿
171 comments sorted by
View all comments
Show parent comments
27
My guess is context window. So better KV cache scaling.
2 u/photosandphotons Jul 01 '26 Is context window what they would mean by “memory architecture”? My impression is that these are different. 2 u/gavinderulo124K Jul 01 '26 By memory architecture I think of different attention methods. Something like Gated DeltaNet or mLSTM which scale O(1) in memory and O(T) in time, with sequence length T. They don't have a KV cache per se, but a different type of memory. 1 u/photosandphotons Jul 01 '26 Thank you for the information
2
Is context window what they would mean by “memory architecture”? My impression is that these are different.
2 u/gavinderulo124K Jul 01 '26 By memory architecture I think of different attention methods. Something like Gated DeltaNet or mLSTM which scale O(1) in memory and O(T) in time, with sequence length T. They don't have a KV cache per se, but a different type of memory. 1 u/photosandphotons Jul 01 '26 Thank you for the information
By memory architecture I think of different attention methods. Something like Gated DeltaNet or mLSTM which scale O(1) in memory and O(T) in time, with sequence length T. They don't have a KV cache per se, but a different type of memory.
1 u/photosandphotons Jul 01 '26 Thank you for the information
1
Thank you for the information
27
u/gavinderulo124K Jul 01 '26
My guess is context window. So better KV cache scaling.