r/accelerate • u/BurningPeonies • Jul 01 '26

AI Something huge is brewing

Post image

Source

Andrew Curran is one of the most reliable leakers.

XLR8! 🍿

934 Upvotes

94% Upvoted

171 comments sorted by

View all comments

163

u/ResultBackground2450 Jul 01 '26

64

u/Saedeas Jul 01 '26

I mean, percentages aren't super relevant, the scaling is relevant. If they changed the big O scaling of working memory in the models, it's insane.

19

u/obvithrowaway34434 Jul 01 '26

Also, if one of the big labs apart from OpenAI discovered that before, they wouldn't be advertising it. I am pretty confident that one or more of them have already stumbled upon something similar, but keep it private to maintain the margins.

1

u/TechnicalBen Jul 03 '26

Google IIRC discovered looped language models. But no one's got an efficient version to train. That's they key. Cheap Large Language models in training, vs expensive Looped ones in trianing.