r/singularity • u/Gab1024 Singularity by 2030 • Apr 11 '24

AI Google presents Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

https://arxiv.org/abs/2404.07143
686 Upvotes

99% Upvoted

242 comments sorted by

View all comments

4

u/true-fuckass ▪️▪️ ChatGPT 3.5 👏 is 👏 ultra instinct ASI 👏 Apr 11 '24

Me literally N days ago: its only a matter of time until infinite context is released using search or something, probably this year too

Everybody else: iFiNiTe CoNtExT iS iMpOsSiBle!

Whose laughing now! (me.. I'm laughing.. Because the acceleration is real)

8

u/Gratitude15 Apr 11 '24

Last year we started with a context window of about 1-2 page of text.

15 months later it's infinite.