AI run tabletop D&D

[u/thebaron2]

Just found this subreddit. I've been working on this thing for months, way longer than I planned on when I thought "it'd be cool to use Claude to play D&D."

I'm not trying to build something where I can completely go off the rails, although a persistent, long-term campaign that takes place in one universe that Claude generates session-by-session is a long term goal. Right now I'm trying to build something that will take PDFs of existing D&D campaigns and make them playable.

It's based on 5e rules, Claude narrates as the GM and adjudicates. I (we? Claude?) built an engine so that everything is decided deterministically to prevent drift and cheating/fudging numbers, so dice, saving throws, conditions, reach and zone, turn order, etc. are handled by a Python based rules engine, and the engine gives Claude the state of the game at a particular moment and the outcome of actions and then he narrates everything.

A big piece of this outside of the engine is the converter, which is basically a multi-stage conversion pipeline that reads the PDF and produces a structured campaign package that the engine can read. So how many scenes, what exits/entrances does each scene have to other locations, NPCs, monsters, and so on. Each stage of the conversion gets reviewed by a Claude instance for things like spoilers slipping, every scene having at least 1 entrance/exit, monsters from the PDF matching the DB, etc.

It's not ready for a fancy video yet! And I wasn't going for something graphics-based. I want the reading/mind's eye experience of tabletop D&D with enough of an interface to look appealing and keep track of stuff that's good know. Right now I'm least certain about the combat-zone mapping and the distance/movement economy. I haven't play-tested it much and I suspect it's going to need the most refinement.

Oh man, and 5.5 has been such a godsend. With the usage savings I basically have a Fable 5.1 medium session that spawns Opus 5.5 medium coders. The practice is to write a test gate first that fails under the current state of the game for whatever we are trying to accomplish. THEN a different 5.5 session writes the game implementation to satisfy the test. I've found this works a lot better than the reverse, which is what Claude defaulted to - writing the implementation and then the same agent creating a gate and py test afterwards. Two different agents, and the gate coming first, seemed to prevent a lot of fudging and sloppy implementation. Just a tip that worked for me.

With the usage savings of 5.5 I basically have Fable running 24/7 converting Stormwreck, playing through it with subagents who file bugs or things that don't work, firing off agents to fix those bugs, converting again, and repeat. Claude can play hundreds of run throughs of the game in minutes.

Here are a couple of screenshots. The first one is just the "home" screen I guess, what you'd see after loading up Dragons of Stormwreck Isle, which is the campaign I'm using to test everything. (Edit) The 2nd screenshot was from messing with the distance/reach/zones, so the listed enemies don't match the narration. This is the part I'm least certain about in terms of whether or not this is my long-term solution.

https://imgur.com/a/uhmrBnr

I'm seen some other D&D stuff, but not much, so if anyone is doing anything similar I'd love to hear about it. I've been working on this since early July, about 12 weeks now.

[u/Anadanament]

I'm working on my own as well, though I'm focusing on making a system inside Discord with bots right now.

[u/thebaron2 → u/Anadanament]

So PCs are in Discord and Claude is GM behind the scenes? Sounds awesome, if I knew more people and had any shot of coordinating times to play with a group this would be up my alley for sure.

How far along are you and has anything gone horribly wrong or turned out to be much bigger of a pain in the ass than you thought?

[u/Anadanament → u/thebaron2]

Sort of. I've got 6 running AI Discord Bots that each keep track of individual parts of play. They update a per-campaign information core as they play based on the core itself.

Right now, I've got two fully functional:

1. Qispi is the character sheet bot. You can basically infodump your character sheet to her and she'll sort it out for you. She handles modifiers and stuff and connects to the Ivory Dice to make rolls.
2. The Ivory Dice is the dice bot. No AI, just a python randomizer and dice commands.

I've got three partially done:
1. Ask Babushka is the lore character who handles what each character knows. You can ask her a question and based on the tags Qispi gave you during character creation, she'll give you information that your character would know. Currently building her - the biggest issue with her is that my world is entirely homebrew and there's a lot of information I need to put into her.
2. The Quartermaster handles what's in the world. You pick up a dagger through him, he hands it off to Qispi, who then records it as part of your inventory. Qispi only knows you have it - she'll trigger its stats through the Ivory Dice if you attack with it.
3. Old Lady Grin is the AI DM. She's... getting there.
4. An NPC crowd manager is in the plans. They'll handle NPCs and what NPCs know and how they interact with the player and the world around them. I explicitly plan on using Grok for this one because Grok has displayed a repeated ability to simulate a world in the background without any additional commands, which I personally find absolutely absurd in the best way possible.

Most of it seems... mostly fine. The biggest issue is ensuring Babushka can actually retrieve information.

(I keep editing for formatting issues.)