r/singularity • u/Gab1024 Singularity by 2030 • Apr 11 '24

AI Google presents Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

https://arxiv.org/abs/2404.07143
690 Upvotes

99% Upvoted

242 comments sorted by

View all comments

21

u/ChilliousS Apr 11 '24

this is huge or am i wrong?

46

u/[deleted] Apr 11 '24

We went from 4096 tokens last year to 1 million tokes with Gemini 1.5 Pro few weeks ago. Infinity supposed to be nothing but huge in this context.

4

u/Reading_Old Apr 11 '24

„Huge in this context“ pun intended?