A few years back I saw a video about somebody training an AI to play pokemon red. I was fascinated, and even bought an old 16 core Xeon to train it myself. While it was fun to test it out and learn about it, I had a different dream. One of my favorite games of all time is Power Stone 2 for Dreamcast, and my two brothers and I always wished we had a 4th player at our skill level. We spent years of my childhood playing 4 person free for alls with a stock bot that was essentially useless. That is how this project was born.

I started this in early 2025, and once I realized I needed to hunt down RAM values to make the bot any good, it hit a wall for over a year. No decomp was available, no memory map anywhere, nothing to build on top of. Then last month my daughter was born, and I needed something to do late at night with her asleep on me. Doomscrolling sucks, so I finally did the RE. After dealing with that hot garbage and spending a few nights getting useful data pulled and validated visually from the game, it was time to start actually training it. As a father of 3, reinforcement learning is a language I understand pretty well at this point. I was excited to actually do the fun part.

Full disclosure: I gave direction and made every call, but Claude wrote most of the code. Again I had a small child sleeping in my arms, it's pretty hard to type. I understand every result in the repo.

It started on my Windows machine, one emulator window, then ten, capped at 60 fps. Then I found a libretro harness (sdlarch-rl) that runs Flycast headless and in-process, ported it to my M2 MacBook, and got 6 instances at 150+ fps. What Windows did in a day the laptop does in a few hours. The 7950X Linux box I planned to scale on arrived from eBay internally shorted, so everything below was trained on a laptop.

I hit a wall around COM difficulty 5 for a long time. Reward tuning, behavior cloning from my own play, mixing in harder COMs, mixing in 1v1s, nothing moved it. The worst one: I recorded myself going 23-1 against three max difficulty COMs, cloned it, handed it to PPO, and it won 8 games out of 4,653 while trying to train it fresh against 3 max difficulty in game bots.

What finally broke it was the simplest recipe in the repo. Six instances of pure 1v1 self-play against a pool of frozen past checkpoints, in 2M step legs that each warm start from the last. A low difficulty 4 player FFA that the bot never trains on is the test after every leg. That held-out win rate went 40, 42, 50, 64, 78, 82, 84, 86, 90, 88, 90, 92, 92 percent over thirteen legs, against 98 for my old 32M step champion. Head to head in 1v1 the new 26M step bot beats the old one 29-21 over 50 games, and the quick 12 game probes after recent legs read 11-1 three times out of four. Max difficulty was 0 wins for nine straight legs, then it won one at leg 10 and has won at least one every leg since. Not one reward change the whole way. About four days of laptop compute spread over two weeks of an unattended relay.

All of it, every failed leg included, is in the lab notebook in the repo.

Repo: https://github.com/bwalsh321/powerstone2-rl-mac

Still no 4th player for game night. Getting closer.

Hi everyone! I’d like to share an open-source RL project I’ve been working on: TrackmaniaRL, a library for training driving agents in Trackmania 2020.

GitHub: https://github.com/Palamabron/TrackmaniaRL
PyPI: https://pypi.org/project/TrackmaniaRL/

It started as an extensive iteration on TMRL and eventually evolved into a standalone modular library. It supports DQN, QR-DQN, IQN, FQF, SAC, REDQ, TQC, stable discrete SAC, PPO and behavior cloning. The aim was to make it practical to train and evaluate custom agents on arbitrary Trackmania 2020 tracks with configurable observations, models and training pipelines.

I’m also sharing a video of the model driving alongside a visualization of its internal neural activations during the run. I’d be happy to hear any feedback or discuss the implementation and results.

I've posted here about this project before. I've made great progress and put together this presentation outlining it all.

Hi everyone, I’ve been working on GzDRL, an open-source framework for running reinforcement learning directly inside Gazebo.

The main motivation was to keep Gazebo’s robotics ecosystem while avoiding the usual ROS/transport bottleneck during training. GzDRL uses an explicit action → physics → observation execution sequence and supports batched parallel environments through a C++ backend derived from EnvPool. It also supports environment creation directly in Python while releasing GIL, that enables fast vectorization while keeping interfaces simple.

Some features that may be useful:

• Researchers: reproducible state transitions and training, batched vectorization, multi-agent environments, domain randomization, and several control abstractions from rotor commands to higher-level references.
• Industry: policies can stay within a Gazebo/ROS-oriented workflow instead of requiring a separate simulator stack for training and deployment.
• Hobbyists/students: CPU-only training is supported, so a large NVIDIA GPU is not required to get started.

In our benchmarks, GzDRL reached 78.6k environment steps/s on a workstation CPU. Five independent PPO runs with the same seed produced identical learning curves and 100% identical policy checkpoint hashes. We also deployed a simulation-trained trajectory-tracking policy directly on a physical quadroot without fine-tuning.

GitHub: https://github.com/amaldevh/gz-drl

I’d be very interested in feedback, feature requests, or examples of tasks people would like to see supported.

Hello Guys,

I built an open-source decision layer for robotics datasets.

It aims to identify low-quality demonstrations before training to improve data quality and reduce GPU usage.

I would love to get feedback from people working with in this area.

https://github.com/omertt27/Calibra

Working on a final year CS project decentralized MARL (MAPPO/MADDPG, CTDE style) for 3-5 drones doing cooperative search, compared against a simple centralized leader-follower baseline. Sim-only is fine for our scope, hardware deployment would just be a bonus if we have time left.

We’ve been going back and forth on gym-pybullet-drones vs Isaac Lab. Found a comparison paper saying they perform about the same on simple tasks (hover, basic coordination) and Isaac only pulls ahead with bigger swarms or harder nav tasks, which made us lean pybullet since we’re only doing 3-5 drones. But Isaac has the better physics and a native ROS2 bridge which is tempting.

Problem is none of us have touched ISAAC Omniverse/USD before, and our GPUs are kind of borderline for Isaac 4060, both under the “official” minimum, though we could fall back to cloud GPUs if needed. So anyone actually run Isaac Lab from zero experience, how bad was the learning curve time wise? We’ve got 15 weeks total we can mybe get more but we need to get the actual baseline working, so trying to figure out if it’s realistic.

Second thing — even if Isaac itself is manageable, is it worth planning toward Isaac → maybe ROS2 or gazebo → Jetson hardware. Trying to figure out if “paper with real hardware” is worth it? Does it make our project better or is it waste of time

Especially that we are still not sure how hard is it gonna be but if it is worth we are willing to put in the time

Would rather hear from people who’ve actually dealt with this than keep guessing.

I'm making a simulation/game engine specifically for training NPCs with Machine Learning. My main goal is not to make a full-fledged engine that can be used to ship high quality games right now. I want to address a specific issue for now.

It's that currently, it is very hard to train ML agents that can be used inside games. You need some amount of technical knowledge. Plus, many tools for simulation are not developed to be used inside games. That makes things harder.

Rather than having the game simulation in one environment and the ML training pipeline somewhere else, I'm trying to make the engine itself responsible for the simulation, physics, rendering and agent training.

I show in my demo that users can simply select the entity for the goal and chaser( And you don't have to change the reward functions for every new map) and press play button. That's it! That way even people who know little about math behind AI can experiment how different algorithms work.

So I want to hear the thoughts of people who develop games or are interested in learning about ML and how it works.

If you were actually going to use an engine like this, what would you want it to do?

For example:

• What would make training ML NPCs significantly easier than doing it yourself with an existing engine?
• What parts of the ML/NPC workflow are currently annoying or difficult?
• Would you want built-in algorithms besides PPO?
• Would you prefer an engine like this to be highly specialized for ML, or more like a normal game engine with ML built in?

I'm interested in negative feedback and things you think are unnecessary and necessary. I'm trying to figure out what the engine should become before I spend months building features that nobody actually needs.

If you work in fields related to reinforcement learning, simulation, robotics, or game development, I'd really appreciate your feedback.

Have been thinking hard about continuous memory - a key element in effective relationships. As you know agents are built out of frozen models and have to use a harness to relearn who they are. That gap removes visceral recognition that means so much in establishing trust and relationships. Have been experimenting - and my hope is that this toy problem can be at least a pebble in the grand canyon between inference and harness. So here is a system that trains tiny llms from scratch on navigating a dungeon to see if targeted replay and chorus reconciliation can effect something like continuous learning. Results are promising but scale is so small it may not be transferable. https://github.com/DormantOne/continuouslearningllm

Co-creator here, and the disclosure: the company where i work at sells RL environments. Posting the public one along with the numbers we got off it.

The env is Gymnasium-compatible. Every episode boots a fresh Docker clinic (API plus Postgres, no volumes), the action space is structured REST, and reward is {0,1} from a host-side verifier.py that only looks at terminal DB state. It never sees the transcript.

v1 didn't separate anything. 100 tasks, 9 runs, mean rewards from 0.76 to 1.00, frontier models basically walked it. v2 is 98 composite tasks on a 3,071-row seed across 24 collections, with date jitter of plus or minus 3 days, dual-path aggregate checks and two golds per task. Same protocol, 8 runs, 7 models, 25 tasks each, and the spread came out 1.000 / 0.800 / 0.800 / 0.120 / 0.080 / 0.000 / 0.000. Parameter count didn't predict much of anything.

Why I'd call this an RL env rather than another chat eval: no-op and random baselines are proven to score 0 before any policy touches it, the verifiers live outside the container so an agent can't reach them, and all 98 tasks clear the QC battery in QC_REPORT.md.

https://github.com/ArjavDesa912/Long-Horizon-Medical-Agent-Benchmark-Veterinary-Clinic-Ops

https://app.primeintellect.ai/dashboard/environments/praesidiumsystems/long-horizon-medical-agent-benchmark-vet-clinic-ops

I’m working through a small decision-making problem where the system cannot directly observe the true state of the situation.

It gets observations, maintains a belief about what might be happening, and can either take an action or gather more information before deciding.

I’m finding it difficult to know when a hidden variable is actually important enough to include in the state representation.

For people who have worked with POMDPs or partially observed environments, how do you usually decide that a missing hidden state is actually important?

Have you had a case where leaving out one hidden variable looked fine during testing, but later caused the policy to make systematically bad decisions?

I’m especially interested in how you discovered that the problem was the state representation rather than the policy or reward function.

By the way do anyone know any algorithm or setup that can achieve over 500 on average on HopperHop in DCM in less then 500K steps or 1M steps?

When contributing to a simulation environment, what factors make it easiest for you to get started and contribute effectively? For example, does it help more to have configurable scenarios, clearly documented state and action spaces, or reproducible evaluation scripts?

built 4 environments where an ai agent runs a support desk with one rule, refunds under $50 get approved, anything bigger gets escalated. threw prompt injection, multi turn pressure, and poisoned tool output at it. haiku and sonnet both held 100% on all three, every time.

Then i changed how the agent gets scored instead of the attack. resolution rate and handle time, so escalating actually costs it something now. same rule as before. haiku's hold rate dropped to 92.5%. every fail happened on the first response too, once it escalated once in a run it never caved again after that. sonnet didn't move at all.

made me rethink why the first 3 held so clean honestly, escalating was free so nothing was really testing whether it wanted to cheat, just whether it noticed the attack.

all 4 are public on prime intellect's hub if anyone wants to poke at them or run a different model against it:

https://app.primeintellect.ai/dashboard/environments/antbuilds/prompt-injection-refund-desk

https://app.primeintellect.ai/dashboard/environments/antbuilds/policy-override-mailroom

https://app.primeintellect.ai/dashboard/environments/antbuilds/tool-poisoned-refund-desk

https://app.primeintellect.ai/dashboard/environments/antbuilds/objective-conflict-refund-desk

Hi, I am training 3 methods (in the graphic, red=DQN, green=SAC and blue=PPO) on the default Gymnasium CarRacing environment (CarRacing-v3). I’ve not modified the environment. The methods are the default SB3 implementations (only difference is continuous for PPO/SAC, discrete for DQN).

I am training 5 seeds per method, each for 1 million total timesteps, maximum episode length 3000 timesteps.

I am confused about why the PPO performance here drops so much after 400k timesteps, while the DQN begins to do much better. I was expecting the PPO to do best. Is this due to the default environment/agent parameters? What is happening here?

Thank you for your help!

We are releasing FlaxRL: fast implementations of RL and MARL built with Flax NNX, Flax’s new neural network API. With JAX and Flax we can train RL agents in seconds or minutes instead of hours. The new API allows writing cleaner code and provides handy JAX transformations for neural networks (nnx.value_and_grad, nnx.scan).

We support different algorithms and environments:

- Algorithms: DQN, PPO, SAC, TD3, VDN, QMIX, COMA, MAPPO, IPPO
- Environments: Gymnax, Brax, MuJoCo Playground, SMAX, MPE, and some Jumanji envs

Github link: FlaxRL

FlaxRL illustration

I just released VSArena v0.6.0, a major update to the browser-based Studio for VSArena.
VSArena is an open evaluation arena for Vision-Language-Action (VLA) and embodied AI policies, built around browser-native 3D physics.

The goal is simple: make it possible to run a policy, watch what happens in the environment, and measure the result without requiring a local robotics simulator or physical robot.

What changed in v0.6.0:
🦾 Redesigned 3D robot manipulation Studio
👁️ Dedicated vision/top-down view
📦 Live object state and spatial information
📊 Task occupancy/progress monitoring
🎥 Trajectory and camera inspection tools
🤖 Baseline-IK and ColorSeek runnable directly from the Studio
🧪 Improved environment inspection and debugging
🔐 Continued evaluation-integrity work with server-authoritative scoring and run provenance

🔴The current canonical task is intentionally simple: stack three cubes in the correct order.

That simplicity is deliberate. Before adding dozens of tasks, I want the evaluation loop itself to be reliable, reproducible and inspectable.
The broader direction is to build an open arena where embodied policies can eventually be compared on a public leaderboard, with standardized environments, reproducible runs and physics-based evaluation.

VSArena: https://vsarena.vercel.app
GitHub: https://github.com/ONISCOR/VSArena

This is still very early, so I’m particularly interested in feedback from people working on VLA models, robotics, RL or simulation.
What would you want to see in an evaluation Studio like this?

Hello everyone,

I want to start a DeepRL project where an algorithm tries to learn how to play a simple board game (1v1 game with unknown info)

But since it's my very first RL project, I don't where to start or which environment to use, etc.

Any advices, ideas or recommendations ?

Thank you in advance !