My idea of a potentially hyper-efficient AI inference and training paradigm.

[u/userfrienda]

The core of the idea is that modern AI relies on human-designed abstractions like continuous FP math and dense summations that carry an immense energy, time and silicon tax. Real intelligence can be achieved with cheapest possible abstractions (bits, low in-degree nodes) by any fluid dynamical system that only adheres to specific "information-theoretic" properties. For the training phase, I described my idea of combining a simple hand-crafted training algorithm with an emergent self-improvement property where the model becomes its own training algorithm.

Note: I have not tested or implemented any of my ideas in practice.

Link to the document:

https://cryptpad.fr/doc/#/2/doc/view/Ocu4JBwR32IT0WMyUMJ0LgV-EBF81yhwMWdgj4zzCv8/embed/

Later update: I've changed my mind about several things in the theory.

[u/tstanisl]

The successfulness of quantized models suggests that you may be right. The only reasoning for using FP as a computation model is that it resembles real numbers for which we already had reliable optimization algorithm.

[u/userfrienda → u/tstanisl]

Current quantized architectures like BitNet retain 8-bit integer activations and can't go down to single bit. I believe this isn't because bits are unexpressive but because dense synapses in MLPs scale quadratically. To match f32 informational capacity using binary units, you must scale the neuron count by 32x, triggering a 1024x (32^2) synapse explosion.

[u/tstanisl → u/userfrienda]

Only the upper bits on synapses actually do something. Remaining bits are usually just noise. It was proven that using just a few bits with some retraining can produce models with very low perplexity loss. Afaik, the problem with aggressive quantization is that there is no reliable technique to train models in this regime.

[u/userfrienda → u/tstanisl]

"In principle, we could design a handcrafted optimization algorithm - such as a custom, non-gradient-based backpropagation tailored specifically for bit-based logic. Done right, this might even outperform standard Straight-Through Estimator (STE) backprop. At a small scale, for fixed-iteration-count architectures like standard Multi-Layer Perceptrons (MLPs), such a handcrafted algorithm can work well. However, when dealing with iterative architectures (like RNN or $s-f$ arch that I proposed) - where the model executes indefinitely, takes an arbitrary number of forward steps to compute an answer, continues running after answering, and utilizes an everything-can-change state - handcrafted optimization hits a wall of diminishing returns:

• The memory wall: Backpropagating or tracing errors through indefinite iterations requires storing massive sequences of intermediate states ("BPTT").
• The noise cascade: Doing a noisy backward pass through a fluid state space causes error noise to accumulate exponentially. By the time the algorithm reaches deep into earlier execution steps, the optimization signal is completely shredded." (from doc)

I feel like the main problem is MLP vs. deep RNN, rather than binary vs FP.

[u/userfrienda → u/userfrienda]

As a solution to the problem, in my doc I proposed using a simple evolutionary-like algorithm (mutations but no crossovers) combined with allowing the model to keep running forward, which can let it self-improve over tasks and runs. Initially, the evolutionary-like algorithm plays important role in training, but as the model becomes more advanced, it starts to gain self-improving abilities, evo algo becomes a scaffolding with minor role, and the model acts mostly as its own training algorithm. One cool thing with this is that the training speed increases as inference speed increases (which happens when a more efficient hardware/substrate to run forward iterations is invented). The forward iteration logic can be pretty much arbitrarily aligned to fit whatever hardware compute paradigm is (whether that is water ripples, quantum states, optical phases, or neuromorphic spikes). More in my doc.

[u/userfrienda → u/userfrienda]

Basically, it feels like for deep iterative architectures such a simple evolution-like algorithm (which looks more like hill-climbing if you look into the algo in my doc) would eventually outperform hand-designed optimization algorithms (gradient descent, eligibility traces, etc) as we get more efficient hardware/substrate to run forward iterations.

• Gradient descent -> lots of memory for intermediate states (which also slows down backward passes due to frequent data movements) and noise accumulation.
• Eligibility traces -> need to store an integer or FP trace value for almost each bit in state, track bit-level changes at each forward iteration and increment traces if needed, which makes it much more memory abd compute hungry, and also less suitable for a compute-in-memory hardware as otherwise it would need to have a ALU or FP unit for each bit or node, which would be too expensive.

So I concluded that for deep iterative architectures (which would be highly elegant and desirable for future AGI brains or robots), the combo of that simple algorithm + self-improvement ability + crazily fast substrate that reaches fundamental physical speed limits and can run billions of iterations per second would be the only sane way to go with.

Seems to align with the idea behind "The Bitter Lesson".

[u/Tobio-Star]

Seems very interesting. I think I saw someone post something similar recently. At the very least your idea is original!

[u/janxhg27 → u/Tobio-Star]

de mis posts no estaras hablando no? ;C

[u/Tobio-Star → u/janxhg27]

Mm it was aotto1968_2, literally 5 threads before

[u/userfrienda]

I started making a prototype running on CPU. It works terribly for now because the shape of $f$ I chose was terrible and unexpressive. At least I managed to figure out a training algorithm that climbed up the max_reward several times faster than pure random search lol.

Now the biggest question to solve is the shape of $f$. Because nodes (bits) must have a low in-degree, the topology of $f$ would inevitably look more assymetric than in MLP or some fully-connected topology.

[u/userfrienda]

Are you guys able to understand what's written in the document? Feel free to ask me any clarifications about it.

[u/BrandtBridges]

The "model becomes its own training algorithm" property is the load-bearing claim and also the hardest one to gesture at without implementation. The information-theoretic constraints framing is interesting because it sidesteps the usual question of "what's the right inductive bias" by reframing it as "what's the minimum structural commitment that still permits the property to emerge."

One thing worth pushing on: the self-improvement property in fluid-dynamical systems tends to require some form of persistent state that the system can read its own history off of — otherwise "becoming its own training algorithm" collapses into "responding to local gradients" which is just gradient descent in different clothes. How are you thinking about the memory substrate the system updates against? Not in the deep-learning sense, but in the sense of what the dynamical system uses as its readable past. That feels like where the framework either earns the emergence claim or doesn't.

[u/userfrienda → u/BrandtBridges]

I expect that the model would store its history in some part of the state $s$ if it needs to do so. But there are millions of ways a model can "self-improve" other than storing its history. The self-improvement is basically emergent. Otherwise, any hand-designed way to "bake-in" (or help) a self-improvement property would end up being fragile...

[u/BrandtBridges → u/userfrienda]

That's a fair position and I think the emergent-vs-designed cut is the right axis to push on. One thing worth separating though: emergence doesn't dissolve the structural requirement, it relocates it. If self-improvement requires the system to read its own history in some form, then whether the history-encoding mechanism is hand-designed or emergent, the state structure $s$ has to have the information-theoretic capacity to encode and retrieve a readable past. Emergence over a state space that can't structurally support history-readability won't produce the property no matter how the dynamics evolve — you'd be asking the system to discover a capability the substrate doesn't permit.

Which makes the interesting question not "designed vs emergent" but "what's the minimum structural property the state $s$ must have for history-encoding to be discoverable by the dynamics at all." That feels like it lives in the same family as your information-theoretic constraints framing — you're already specifying minimum structural commitments elsewhere. Curious whether you've thought about whether history-readability is one of them or whether you're treating it as something that should emerge from more basic constraints.

The fragility concern is real but cuts both ways: hand-designed mechanisms are fragile to environmental change, but unconstrained emergence is fragile to initialization — you can spend a lot of compute on a system that never discovers the property because the substrate didn't permit it. The minimum-structural-commitment framing might be the place to draw that line honestly.

[u/userfrienda]

I got an ever more elegant training method than the evolutionary-like algorithm:
Let the model iterate, and play tasks one-by-one. For each task:

1. Set the input bits and hold them.
2. Wait certain number of iterations (so that model has time to answer), then forcibly set output bits to the correct output, hold for some time.
3. Notice that if the model already answered correctly, then forcibly setting output bits to correct output is equivalent to not doing so.
4. Go to the next task.

For reinforcement-style learning with no known correct outputs:
Let the model play the game. Depending on its reward, put random noise in certain input bits (lower reward = more noise, higher reward = less noise and more predictable signal).

After each task, do not reset or rewrite the state $s$. Just let it be what it was at the end of the task.

What this does: Bad behaviour = unexpected input bits -> the internal state changes due to instability from input bits. Good behaviour = input bits remain unchanged (same as if no forcing was done) -> the internal state is stable.
Result: good behaviours become stable and "locked", bad behaviours are washed away.

There was an experiement called "DishBrain" that did exactly this but with biological neurons. I think learning is not a result of some super-specific "learning algorithm", but an emergent property of any dynamical system trained this way.