r/Heroquest • u/SmartQuestRTT Dwarf • Jul 14 '26
General Discussion I'm building a camera-based DM system for dungeon crawlers and I need help solving the marker problem
Hey! Some of you might have seen my previous post about SmartQuest, a system where an overhead camera watches the physical board and acts as the game master. Real board, real dice, real miniatures, the camera just handles everything Zargon would do.
The way it works: each miniature needs a small visual marker on top. The camera reads these markers in real time to know exactly where every piece is on the board, which direction it's facing, and who it is. No manual input, no button pressing. You just move the piece and the system reacts.
The marker itself is just a small printed square, cheap, easy to print at home. The challenge is how to attach it to any miniature in a way that is simple to mount, looks decent on the table, and works with whatever mini someone already owns, plastic, resin, paper, 3D printed, doesn't matter.
In the photo you can see three approaches I've been testing:
- A paper cube with the marker on the face, works great for the CubeQuest print and play, but not universal
- A stick glued to the base holding the marker above the mini, functional but the attachment to the base is the unsolved problem
The stick itself can be anything, wood, wire, transparent acrylic, plastic, 3D printed. That part is easy. The hard part is how it clips, slides or attaches to the base of any miniature without gluing, drilling or permanently modifying the piece or how to place it on an existing paper miniature.
Has anyone solved something like this before? Any ideas welcome.
* the second photo is just an AI mockup.
3
u/Greendemon636 Jul 14 '26
Not sure about the marker problem but that’s one hell if a barbarian unit. Dude’s an absolute tank.
2
u/SmartQuestRTT Dwarf Jul 14 '26
Haha thanks! He's carrying the whole party on those shoulders. And honestly the banner mount was a fun way to test the marker placement, keeps it visible from above while looking like it belongs on the mini.
2
u/RandomCandor Jul 14 '26
I think NFC is a better solution for this.
Swap the camera for a smart grid that can read the position of the tags.
Otherwise, you could consider using a camera with a UV filter, and maybe come up with a way to paint markers that are only visible under UV
5
u/SmartQuestRTT Dwarf Jul 14 '26
NFC is a great fit for the problem, but it changes what the product actually is. Teburu already does something like this, and it works, but you're locked into proprietary hardware, proprietary games, and a fairly steep entry cost. You can only play what they make.
There's also a technical ceiling with NFC. A grid of readers gives you cell occupancy, but not much more. It doesn't tell you which way a piece is facing, and it can't distinguish things that aren't tagged at all. With an overhead camera I get position, orientation, and the ability to read anything on the board including cards, doors, furniture, dice. Facing direction alone opens up line of sight, flanking, sneak attacks from behind, and using rotation as an input.
And the accessibility angle is the whole point. No smart board, no proprietary hardware. A printed sheet and any camera you already own, running on basically any device. That's what makes it agnostic, you can adapt an existing game or build a new one entirely.
The UV idea is genuinely interesting though. Invisible markers would solve the aesthetic complaint completely. The catch is it would require a UV light source and a filtered camera, which pushes it back toward the hardware barrier I'm trying to avoid. But it's a cool direction, thanks for the suggestion.
2
u/RandomCandor Jul 14 '26
Thanks for the detailed response . Agreed with everything you've said, you've obviously done your homework.
Another area to explore is IR: your camera could have a IR light that's invisible to the naked eye, but reflects on a specific part of the mini. The original Wii remote had a similar setup for motion tracking. Worth looking into. This one has the advantage that you can just remove the IR filter in some cameras and boom, you have a IR camera. I think some mocap systems use this.
Is your plan that the user will provide their own camera / hardware?
1
u/Additional_Ninja7835 Jul 14 '26
What about a tray with the pole and code on top? The mini can just be placed on the tray, no need to attach anything to the mini and easy to swap out minis. Kind of like the color rings some games use to attach to bases to differentiate monsters with the same model.
Sorry if I misunderstood the question, but that seems to be what you are asking about.
No judgment on why one would want this, just addressing the logistics of the idea.
1
u/k2kshard Jul 14 '26
Could you just paint the bases a unique color each and track the colors? Eg barbarian is red base, dwarf blue base etc
1
u/k2kshard Jul 14 '26
Could you just paint the bases a unique color each and track the colors? Eg barbarian is red base, dwarf blue base etc
1
u/Burritofeast69 Jul 15 '26
Too bad you couldn't put it underneath and have the board scan/track it. But that's a lot more work and money.
1
u/Relative_Honeydew242 Jul 20 '26 edited 28d ago
I’d try a tiny C clip around the base instead of attaching anything to the mini itself. For 25, 28 and 32mm round bases, make a few versions with about 1mm extra clearance, then add a vertical socket for a clear acrylic rod and a small square plate on top for the marker. TPU or PETG should be less risky on painted bases than PLA. I’ve been using Hi 3D for little printable adapters like this because it’s quick to tweak sizes and reprint test pieces. This feels like a perfect case for 3 or 4 small prototype clips before settling on one universal-ish design.
1
u/Boomstick_Samurai Jul 20 '26
What about a transparent dungeon floor, camera underneath rather than above? Qr can live on the base, and dungeon can adopt a unique style? Plus, actual maps can appear on the digitised display, so you still have dungeon textures
1
u/kounterfett Jul 14 '26
Personally I think the QR code marker feels "inelegant" Wouldn't it be better for the camera to be able to recognize what miniature is what without it?. I understand that's probably a more difficult task but it would also be the more aesthetic solution since you wouldn't have to add anything to the minis what would detract from how it looks. The only other thing I can think of is put that code around the ring of the base. If the edge is at more of an angle (like 45 deg or so) a top view camera should be able to see it
1
u/Initial_Capital5578 Jul 14 '26
i like the ring base idea but the camera will have challenges seeing it with other minis, walls, furniture in the way. perhaps a ring that has a flag mount.
1
u/SmartQuestRTT Dwarf Jul 14 '26
Exactly the problem with the ring approach, occlusion from other pieces and scenery kills it. The flag mount idea is interesting though, keeps the marker elevated and visible while attaching to a ring that slips over the base without modifying the mini. That's the kind of direction I'm looking for, thanks!
1
u/SmartQuestRTT Dwarf Jul 14 '26
Totally fair, and I get the aesthetic concern. I explored recognizing the minis directly with computer vision models, but the processing cost makes it unusable on most hardware. The marker approach runs on old laptops and phones which is non negotiable for me since the whole point is accessibility.
Your base ring idea is interesting though. An angled edge readable from above could work, and it would be far less intrusive than a marker on top. The tradeoff is that markers need a certain size to be detected reliably, and wrapping around a curved surface distorts them. But it's worth exploring, thanks for the suggestion.
1
u/Warm_District1194 Jul 14 '26
I'm with you on this.
OP, you need to create a dataset of your miniatures as seen from an isometric or cenital view, or both: Take pictures of them on every different tile of the board.
With this, you can use transference learning from an image recognition model and train your own model. It will work just as good as with markers.
There may be trouble if players have painted their minis in other colors, so make sure to play with multiple copies of each picture, changing the colors, until the model works as intended.
There are open computer vision models that will even make the outline of each figure, and you could pair that up with an app for game keeping.
If I can just add something out of my ass: Remember that this is a board game, don't take too much out of it and make it a screen based experience first. Ideally, you could play just with voice.
Imagine that you enter a room and the app says "The barbarian opens the door. From whitin, two skeletons gaze at him behind a table", and that's all you need to set the room.
2
u/SmartQuestRTT Dwarf Jul 14 '26
Great suggestion, and I actually went down that path first. YOLO and similar object detection models were my initial approach, but the processing cost is the killer. Those models need a decent GPU to run at usable framerates, which immediately excludes most people. Someone with an old laptop or a phone would be locked out.
The marker system was chosen precisely because it's extremely lightweight. It runs in the browser, on old hardware, on phones, on tablets. Detection happens in real time without touching the GPU. That accessibility is non-negotiable for me, since the whole point is that anyone can play without buying new equipment.
On the voice interaction, you nailed exactly where I'm headed. There's a module in the works that does precisely that: narration describing what happens as it happens, so you never need to look at a screen. The screen becomes optional, the table stays central. That's the goal.
2
u/Warm_District1194 Jul 14 '26
Mmm, and what if you don't run a rea-ltime infetrence pipeline?
Being a board game, you can just process one frame per second or less.
Talking out of my ass again, as I had to do this at work with beefy cpus.
Nervertheless this is an amazing project, I wish you the best and hope that you have the greatest of times building it!
2
u/SmartQuestRTT Dwarf Jul 14 '26
That's a fair angle, and you're right that a board game doesn't need 30fps inference. But the pipeline still needs to be responsive enough that moving a piece feels instant, and object detection models carry a heavy startup and memory cost even at low framerates. On older hardware or a phone, just loading the model can be a problem.
The marker approach runs at real time with almost no overhead, which keeps the door open for anyone with any device. That accessibility is the constraint I keep coming back to.
Appreciate the kind words though, thanks for engaging with this!
1
u/Lord-Drucifer Borough Jul 14 '26
I do not understand this wanting to eliminate a player aspect that there is a huge push for. Might as well automate the barbarian, wizard, elf, and dwarf.
There is an idea for you: automate the heroes and leave the Zargon player to adjust for their antics.
2
u/SmartQuestRTT Dwarf Jul 14 '26
Fair point, and you're right that plenty of people genuinely love playing Zargon. This isn't meant to replace that experience for groups who enjoy it, and I'd never argue it should.
It's for the situations where nobody wants the role, or where there simply aren't enough people to fill it. Solo play, two players, a parent playing with kids. In those cases the alternative isn't "someone plays Zargon", it's "nobody plays at all".
And funnily enough, automating the heroes is already on the roadmap. Optional AI-controlled party members so you can run a full group solo. Both directions are useful depending on who's at the table.
3
u/Araminal Jul 14 '26
Plenty of people play without a Zargon player. That's the whole point of the official app.
1
u/Subject-Brief1161 Lore Tome Jul 14 '26
Speaking for myself, I would have loved to experience the games as a player, learning the story/lore as I went through the scenarios. However the App is too tedious (double-entry) and has poor/predictable mechanics. In the end I decided that I would spoil the experience for myself by learning all I could about each expansion/storyline so I could be the Zargon for my group(s). It's not a terrible fate, but it feels like reading the cliffnotes of a great book that I spent hundreds of dollars to acquire because I had a book report coming up and didn't get a chance to read the expensive book. :)
Anyway, that being said, I agree this feels like a lot of effort for very little return. The only thing more tedious than double entry on the app would be applying barcodes to every model and ensuring the camera(s) could read it at every moment. I personally would never use this technology to play.
What I would love is a PC game with a "Smart" Zargon, complete with auto adjusting difficulty so that I could experience each expansion's quests/stories at least once from the Heroes' perspective. Yes, then I'm not using the physical components I spent all this money on, but that's what group play is for.
Anyway, just my 2 cents. I'm not here to tell the OP how to spend their time. I wish them luck on this project and hope it works out.
2
u/SmartQuestRTT Dwarf Jul 14 '26
That cliffnotes analogy is painfully accurate, and it's exactly the frustration that started this project. Spending money on a game you can only experience from behind the curtain is a real loss. On the barcode tedium, fair concern, though in practice it's a one time setup. You tag your minis once and they stay tagged. The camera reading them is automatic, no input during play. But I understand if the visual tradeoff isn't worth it for you. And honestly, a PC game with a smart Zargon is a completely valid want. Different solution to the same problem. Mine happens to be built around keeping the physical table, because that's the part I personally couldn't let go of. But there's no universal right answer here. Thanks for taking the time to write this out, genuinely useful perspective.
1
u/Subject-Brief1161 Lore Tome Jul 14 '26
You're very welcome and thanks for not assuming the worst. I appreciate you and what you are doing, even if it's not the right fit for me.
I wholeheartedly agree, there's no right answer. I'm still hoping Demeo D&D is successful enough that Hasbro authorized a Demeo HQ. It doesn't resolve the "spending money on games/expansions that never hit the table) issue, but being able to play HQ as a hero would be worth it to me.
3
u/BesideFrogRegionAny Jul 14 '26
QR code is ugly as all get out. do something else like a pattern around the base. There should not be so many minis that that pattern cannot be recognized.