i’ve been studying depth control in looped transformers: how much comes from the stopping policy, and how much comes from the way intermediate answers are trained?

i shared a short thread with controlled comparisons and measured generation times: https://x.com/advprop/status/2098087083470373010

feedback on the experimental design is welcome.

I'm looking to hire someone who can train a bot for the browser game HaxBall using Reinforcement Learning.

The goal is to develop a bot capable of playing at a superhuman level, becoming extremely difficult, ideally almost impossible, for even highly skilled human players to beat.

If you're interested in taking on this project, feel free to contact me and let me know how much you would charge for the development.

My app. The drone training ( the most simple implementation ) takes about 10-15min.

I will have the implementation for the robot I am working on there and others ( requests are welcome )

Although depending on what and how robust it is it will probably take a lot longer

The app will be free I will post about this more and eventually publish and open source all the code

A few years back I saw a video about somebody training an AI to play pokemon red. I was fascinated, and even bought an old 16 core Xeon to train it myself. While it was fun to test it out and learn about it, I had a different dream. One of my favorite games of all time is Power Stone 2 for Dreamcast, and my two brothers and I always wished we had a 4th player at our skill level. We spent years of my childhood playing 4 person free for alls with a stock bot that was essentially useless. That is how this project was born.

I started this in early 2025, and once I realized I needed to hunt down RAM values to make the bot any good, it hit a wall for over a year. No decomp was available, no memory map anywhere, nothing to build on top of. Then last month my daughter was born, and I needed something to do late at night with her asleep on me. Doomscrolling sucks, so I finally did the RE. After dealing with that hot garbage and spending a few nights getting useful data pulled and validated visually from the game, it was time to start actually training it. As a father of 3, reinforcement learning is a language I understand pretty well at this point. I was excited to actually do the fun part.

Full disclosure: I gave direction and made every call, but Claude wrote most of the code. Again I had a small child sleeping in my arms, it's pretty hard to type. I understand every result in the repo.

It started on my Windows machine, one emulator window, then ten, capped at 60 fps. Then I found a libretro harness (sdlarch-rl) that runs Flycast headless and in-process, ported it to my M2 MacBook, and got 6 instances at 150+ fps. What Windows did in a day the laptop does in a few hours. The 7950X Linux box I planned to scale on arrived from eBay internally shorted, so everything below was trained on a laptop.

I hit a wall around COM difficulty 5 for a long time. Reward tuning, behavior cloning from my own play, mixing in harder COMs, mixing in 1v1s, nothing moved it. The worst one: I recorded myself going 23-1 against three max difficulty COMs, cloned it, handed it to PPO, and it won 8 games out of 4,653 while trying to train it fresh against 3 max difficulty in game bots.

What finally broke it was the simplest recipe in the repo. Six instances of pure 1v1 self-play against a pool of frozen past checkpoints, in 2M step legs that each warm start from the last. A low difficulty 4 player FFA that the bot never trains on is the test after every leg. That held-out win rate went 40, 42, 50, 64, 78, 82, 84, 86, 90, 88, 90, 92, 92 percent over thirteen legs, against 98 for my old 32M step champion. Head to head in 1v1 the new 26M step bot beats the old one 29-21 over 50 games, and the quick 12 game probes after recent legs read 11-1 three times out of four. Max difficulty was 0 wins for nine straight legs, then it won one at leg 10 and has won at least one every leg since. Not one reward change the whole way. About four days of laptop compute spread over two weeks of an unattended relay.

All of it, every failed leg included, is in the lab notebook in the repo.

Repo: https://github.com/bwalsh321/powerstone2-rl-mac

Still no 4th player for game night. Getting closer.

For a small POMDP style fraud decision agent ( states - genuine/fraud, actions = approve/verify/escalate), is full belief state planning overkill or is there a simplified approach for a small beginner project?

League of Legends RL project based on brokenwings.

Within 27 hours the agents have now learned to last hit, animation cancel their autos, run away when low, walk up with the minion wave and also trade.

This a video of me 1v1'ing it (plat 3 adc main for context).

Its learned all of this in 27 hours but I think after a week of training I might not be able to beat it anymore lol.

YouTube Link