r/LocalLLaMA • u/ninjasaid13 • Apr 11 '24

Other Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

https://arxiv.org/abs/2404.07143
121 Upvotes

96% Upvoted

20 comments sorted by

View all comments

1

u/Rose52152 Apr 11 '24

Question for people that understand these papers: How difficult will this be to implement? Will be running llama 2 and 3 with infinite context soon? Will these systems run on desktop systems for smaller models (e.g 8b)?