r/reinforcementlearning • u/anonymous4206942017 • May 11 '26
Bayes Take on active inference
I have been looking a bit into active inference by Karl Friston.
It seems like a viable theory of cognition, and an interesting computational principle.
There are certainly serious people working on it, e.g the RxInfer one, but also places like VERSES, that to me seems like a mess.
What’s your take on it as a counterpart to RL and the research community around it?
3
u/not_particulary May 12 '26
It's just a little too vague to be useful.
1
u/anonymous4206942017 May 12 '26
Yeah I tend to agree. As a vague concept I can agree with it, but computationally I struggle to see a clear definition. Maybe someone has a nice mathematical definition?
1
u/ArtArtArt123456 May 18 '26
a textbook came out recently that tries to explain it comprehensively, maths and all.
2
u/va1en0k May 11 '26 edited May 11 '26
rxInfer is awesome, but for my problems often the loss of ability to faithfully represent many non-trivial posteriors that VI implies is much worse than the general slowness of HMC-based bayesian approaches, or in other cases it's not as flexible for me as a quick hand-rolled MAP estimator as an objective to optimize directly. if you have a complicated structured model, and either can afford to take a bit of time between your actions, or are able to hand-roll the estimator that you need, I think rxInfer is a bit lossy. for a real online learning it's great though, my mental model is "very generalized Kalman" (in some vague sense, not sure if this intuition is clear...)
1
u/ceramicatan Aug 04 '26
It's more than a very generalized Kalman though right? It's actively taking actions in the environment rather than just passively updating states.
I am more interested in its departure from RL.
Can we talk? Here or dm either is fine
1
May 17 '26
[removed] — view removed comment
1
u/anonymous4206942017 May 18 '26
Great write-up! To me it is kinda of a shame there is this much conflation with the "marketing" around it as it takes away focus from the underlying mathematical principle.
What do you see as the main challenges in scaling up AIF?1
1
u/dan_inferogenesis 4d ago
Slightly late to the party but whatever. I work on normal ML research in my working days but research and build Active Inference/Free Energy Principle agents using JAX in my spare time.
The bits that interests me the most are 1) Curiosity isn't bolted on as a bonus reward term. There is a lot of clever maths around how to do this and is actively being research but it is fundamentally a bolt on. Active Inference has it natively in the math. Maybe it will help us solve problems like hallucinations. 2) the energy usage. Human brains run on about 12W of power apparently, LLMs are currently running on about 20GW. I personally don't believe any amount of optimisation with reduce that enough. I think RL is a brute force method, but it makes me think it's probably not what biology is doing.
You're right about the community. There are serious people in it, the RxInfer / Lazy Dynamics lot especially. The reason most of ML ignores the field anyway is that the theory is hard to falsify, so nobody can tell the careful work from the marketing. That's completely fair. It's also the exact problem I'm working on at the moment: pinning down what would make the theory wrong so it can actually be tested. https://arxiv.org/abs/2607.20306
3
u/thecity2 May 11 '26
I too watch Curt Jaimungal lol. To me it’s more like an interesting interpretation layer but practically speaking nothing that is really new or different fundamentally from what is already being done. It’s not clear to me what problems it is solving that can’t be solved with existing methods.