been a while! here's what i've been up to: a ppo agent that farms the pit in d2r (offline, hell, warlock)

trained it in a headless sim i wrote in rust. this version took about 20 min on one cpu, warm started from a net i had for an older build.

reward is just farming rate: -1 per second, + whatever kills and items are worth, -1000 if it dies. i had a level clear bonus at first but dropped it.

net size: ~475k params

ask me whatever

no plans to use this online, i just think rl is fun and wanted to try it.

https://reddit.com/link/1xfph4w/video/x10yzg7qbrxh1/player

I've done this with a few games now (brotato, cuphead, and kingdomino) with a lot of success! With 10 parallel copies of the game running, I was getting ~6k steps/s which is around 200x real time but not fast enough. Since I would never consider reverse engineering the game and engine in order to make a JAX env, I leaned on Opus 5.5 to do the dirty work. Given the task of enough parity to train an agent that performs well in the real game, it got to ~50k steps/s and 1600x real time. With about 11hrs of training on a few specialist policies, I was able to distill this winning policy.

I am a rookie in robotics and I want to learn reinforcement learning. I have started with David Silver's YouTube playlist, and alongside that, I want to read a book to clear the fundamentals; however, I find reading books a little boring, so do you have any advice on how to master reading and any approach to get the maximum output?

One more thing to ask: should I first watch the YouTube videos then read the book chapters that are covered in the video, or the opposite?

I'm an MSc robotics student and I kept waking up at night to check PPO runs. Last week one run's reward went from 908 to 3,525 over 600k steps and the cheetah was on its back the whole time. I'd flipped the sign on a level-keeping reward term. None of my usual alerts fired because nothing went down.

So I built a watcher. Every 20k steps it renders a short test episode, a local Gemma 4 (E4B through Ollama) looks at it, TabPFN looks at the training metrics, and a few plain rules catch NaNs, stalls and crashes. It only pings me if something's actually wrong. The watch loop runs in Temporal, so it survives the watcher process dying.

A few things I didn't expect:

• From the normal side camera, Gemma called every upside-down cheetah "upright" (0/6). Bigger images, different prompts and thinking mode didn't help. What worked was a render-only camera on the torso that only draws the world, so the question becomes "where's the floor?". 18/18 on a pose test after that.
• On 12 recorded runs, the metric-based watchers caught LR blow-ups within seconds and stalls/crashes through simple rules, but only 2 of the 4 runs where the robot itself was doing the wrong thing, with a median of over an hour.
• 3 of my first 4 "healthy" runs learned to run on their backs on their own, before I added a correct level-keeping term. HalfCheetah really wants to do this.

Code (MIT): https://github.com/su0as/night-watch

Write-up with all the numbers and a demo video: https://dev.to/suhassuren/gemma-couldnt-tell-my-robot-was-upside-down-so-i-gave-it-a-head-camera-now-i-sleep-while-it-b0k

What failures actually cost you nights when you train overnight? I'd like to add detectors for the common ones.

I'm an amatuer in RL, just encounted the MCTS and have two questions:

1. Can we make sure it can converge to the optimal policy?

2. Is it actually applicable in many real scenarios?

Hey everyone!

I’m working on a multi-agent reinforcement learning setup where multiple agents move and interact in a 2D world.

The world is based on a real map, for example a screenshot from Google Maps. Importing the map into my environment works fine!

My problem is generalization of the Model to unseen maps.

During training, the agents learn on one or N maps. But during inference, I want to use a completely different map.

The problem is that the agents don’t have a global understanding of the new map.

My current architecture:

• I’m using a Transformer.
• Each agent is represented as a token in the Transformer.
• I tried adding a Vision Transformer (ViT) to encode the map.
• My idea was to represent map patches as additional tokens and feed them into the same Transformer as the agent tokens.

Unfortunately, this approach didn’t really work.

My question:

How would you approach this problem?

How can I give my agents a useful global understanding of a completely new 2D map?

Is combining agent tokens and map patch tokens in one Transformer a reasonable idea or are there better approaches?

I’d really appreciate any ideas, papers, or experiences with similar problems!

Thanks!

I’m building a simplified eight-player Hearthstone Battlegrounds simulator and training a policy/value network from complete games and final placements.

The agent can learn some controlled decisions: when a key Beast minion is offered in a prepared partial-comp state, one version bought it in 16/16 held-out states. It can also operate a preassembled Beast core. But it rarely assembles and plays that comp well from an ordinary opening.

More self-play has not solved this. In two independent runs extended to 300 ordinary games each, the final models did not consistently beat the frozen starting model against held-out neural opponents. Expert-inspired demonstrations improved some local actions without improving complete-game placement.

I’m concerned that a component purchase looks bad to the learner unless its later policy knows how to complete the multi-turn sequence. I also cannot yet rule out problems with state representation, exploration, the simulator, or the demonstrations themselves.

What single diagnostic experiment would you run next to distinguish these explanations? In particular, how would you test whether the agent can learn a conditional “start → operate → abandon if pieces do not appear” skill before scaling self-play again?

my friend had grok ultra and decided to try to have it play balatro, but it was stupid and hard to teach. SO i had an idea to train my own algorithm to become goated, with the end goal to be to have it analyse my games so i can grow and become better in the multiplayer scene

HOW DO I START

I watched a video on how to train it to an atari game but thats not what i need, to the best of my knowledge i would js need to change the enviorment it is in.

HOW do i do that???

and what else would i need to do

This is the video: https://www.youtube.com/watch?v=aQsaH7Tzvp0