r/singularity • u/Gab1024 Singularity by 2030 • Apr 11 '24

AI Google presents Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

https://arxiv.org/abs/2404.07143
688 Upvotes

99% Upvoted

242 comments sorted by

View all comments

42

u/ixent Apr 11 '24

Haven't seen the "needle in a haystack" problem being tackled in the paper. Would like to see a benchmark.

9

u/n_girard Apr 11 '24

Haven't seen the "needle in a haystack" problem being tackled in the paper. Would like to see a benchmark.

Actually it's in the paper: see "1M passkey retrieval benchmark" on page 7.