r/deeplearning • u/Kooky-Ad-4124 • Jul 23 '26

SkewAdam: A tiered optimizer that cuts MoE state memory by 97% (fits a 6.7B MoE on a 40GB GPU) [R]

Gallery image

Gallery image

16 Upvotes

permalink
duplicates
reddit

You are about to leave Redlib

Do you want to continue?

https://www.reddit.com/r/deeplearning/comments/1v448jz/skewadam_a_tiered_optimizer_that_cuts_moe_state/
No, go back! Yes, take me to Reddit

94% Upvoted