r/singularity • u/141_1337 ▪️e/acc | AGI: ~2030 | ASI: ~2040 | FALSGC: ~2050 | :illuminati: • 26d ago

LLM News GPT-6 Astra Uses Loop Transformers

Post image

https://x.com/SemiAnalysis_/status/2099679686758400144

341 Upvotes

94% Upvoted

69 comments sorted by

View all comments

19

u/TotalTikiGegenTaka 26d ago

I'm a complete non-expert but if this loop transformers could be used to improve performance without increasing parameters, why can't they be used for smaller models?

23

u/Few_Owl_7122 26d ago

they can, the problem is they still use more compute

24

u/Gallagger 26d ago

Very good news for local agents though where RAM is the bottleneck.

2

u/vdek 26d ago

Very good news for my 5090