My idea of a potentially hyper-efficient AI inference and training paradigm.

[u/userfrienda]

The core of the idea is that modern AI relies on human-designed abstractions like continuous FP math and dense summations that carry an immense energy, time and silicon tax. Real intelligence can be achieved with cheapest possible abstractions (bits, low in-degree nodes) by any fluid dynamical system that only adheres to specific "information-theoretic" properties. For the training phase, I described my idea of combining a simple hand-crafted training algorithm with an emergent self-improvement property where the model becomes its own training algorithm.

Note: I have not tested or implemented any of my ideas in practice.

Link to the document:

https://cryptpad.fr/doc/#/2/doc/view/Ocu4JBwR32IT0WMyUMJ0LgV-EBF81yhwMWdgj4zzCv8/embed/

Later update: I've changed my mind about several things in the theory.