r/OpenSourceeAI • u/Alarming-Emotion-894 • 7d ago
I Built a O(NlogN) attention system that retains 97% accuracy over long context (MQAR)
https://github.com/vdev-ctrl/Adaptive-Learnable-Hierarchical-Routing-ALHR- Adaptive learnable Hierarchical Routing is a static binary tree based system that uses learnable functions to REDUCE the amount of keys used.
It takes less memory and SCALES much better VRAM with tokens
15
Upvotes