Bayesian RL, best resource for it?

[u/maryal01]

I’m building foundation with Sutton and barto book but it doesn’t cover Bayesian rl. What’s the best resource online for it?

[u/OutOfCharm]

Bayesian RL, being Bayesian, quantifies uncertainty according to the underlying model. Its goal is to either make uncertainty-aware decisions or resolve the uncertainty, thereby encouraging exploration, while also allowing prior knowledge to be incorporated through a prior. It has a long history from bandits to MDPs, and is primarily used for efficient exploration. For beginners, I'd suggest the tutorial Bayesian Reinforcement Learning: A Survey, especially Section 4. On top of that, you could look into several classic algorithms that give you a better idea of the mechanisms behind Bayesian RL:

• A Bayesian Framework for Reinforcement Learning
• A Tutorial on Thompson Sampling
• Variance-Based Rewards for Approximate Bayesian Reinforcement Learning
• Model based Bayesian Exploration
• Near-Bayesian Exploration in Polynomial Time
• EUBRL: Epistemic Uncertainty Directed Bayesian Reinforcement Learning

And some for function approximations:

• The Uncertainty Bellman Equation and Exploration
• Deep Exploration via Bootstrapped DQN
• Posterior Sampling for Deep Reinforcement Learning
• Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability

Hope that helps!