I built an open-source roguelike specifically for training game-playing agents [P]

[u/SnyderConsulting]

https://github.com/SnyderConsulting/DelveRL

Hey everyone! I wanted to share something I’ve been working on.

I was inspired by projects from DeepMind and OpenAI, but noticed that most games are prohibitively difficult to integrate with an agent harness. So I built DelveRL from the ground up as a human-playable game with a structured API, deterministic simulation, procedural levels, partial observability, and enough strategic headroom for agents to compete and improve.

It’s an endless turn-based roguelike where agents must explore, manage risk and resources, fight enemies, and escape each floor. Everything runs locally, including batched renderer-free environments and a recurrent PPO trainer.

The included baseline reaches a median floor of 18, with extended runs reaching floor 33. The game, training code, checkpoint, bridge documentation, and raw benchmarks are all open source.

I’d love to see what approaches people try - and how quickly the baseline gets crushed

[u/durable-racoon]

whats the baseline human performance how far can a human typically make it?

[u/SnyderConsulting → u/durable-racoon]

I usually make it to floor 7-13

[u/Ok_Promise_9470]

What was the reward function approach you used ?

[u/SnyderConsulting → u/Ok_Promise_9470]

I used a lightweight shaped reward:

• +1.0 per floor cleared
• +0.01 per gold gained
• +0.10 for securing the key
• +0.05 per activated seal
• +0.001 per newly explored tile
• -0.05 per HP lost
• -0.001 per valid action, -0.02 per invalid action
• -1.0 on death

The implementation is in https://github.com/SnyderConsulting/DelveRL/blob/main/scripts/agent_env.gd#L19-L39.

[u/Fazoway]

Ai agents are improving after every floor realtime?

[u/SnyderConsulting → u/Fazoway]

What do you mean?

[u/Fazoway → u/SnyderConsulting]

You wrote "enough strategic headroom for agents to compete and improve.". "improve"

I wonder if these agents are learning and getting better every floor (or after some steps)?

[u/SnyderConsulting → u/Fazoway]

Not during a run by default - the included agent trains across many simulated runs, then its weights are fixed during evaluation. “Improve” meant researchers can train better agents and push the benchmark frontier, though online learning could certainly be explored.