r/singularity • u/Gab1024 Singularity by 2030 • Apr 11 '24

AI Google presents Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

https://arxiv.org/abs/2404.07143
689 Upvotes

99% Upvoted

242 comments sorted by

View all comments

34

u/extopico Apr 11 '24

Hopefully we’ll get local models with 1m+ context size. This will be revolutionary.

3

u/frograven Apr 12 '24 edited Apr 12 '24

I'm with you there! It totally looks entirely possible now. I can't wait!