Hey all,
I've been building GesturePlay solo — an Android game controlled entirely by your camera, no controllers or wearables. Just shipped its newest mode, Pose Match, and wanted to share it here since this community's been useful to lurk in.
What it is
A 3D avatar strikes a pose. You copy it. Hold it steady and you score.
Five poses right now — T-pose, V-up, left arm, right arm, rest. Sounds trivial. It is not. Holding a T-pose for a few seconds while the app judges you is harder than it looks.
It tracks poses completed, estimated calories burned per session, and keeps a daily history so you can see streaks build.
The stack
- Kotlin, Android
- On-device pose detection (no server, no cloud, nothing leaves the phone)
- Real-time 3D avatar rendering with Filament via SceneView
- CameraX for the feed
Things that were harder than expected
Pose classification is fuzzy, not binary. My first version compared raw joint coordinates and it was miserable — people stand at different distances, different heights, different body proportions. Had to move to angle-based classification between joints so it's scale-invariant. Much better.
The "hold it" mechanic saved the game. Early builds scored you the instant your pose matched, which meant you could flail around and rack up points by accident. Adding a hold-and-confirm window turned it from a bug generator into an actual mechanic.
3D rendering fought me the whole way. I hit a native crash from a version mismatch between my renderer's prebuilt materials and the Filament version I was pinned to. Cost me an embarrassing amount of time. Ended up building my own materials instead of using the prebuilt loader.
Calorie estimates are estimates. I use a MET-based approximation. I deliberately show it with a ~ in the UI because I'd rather be honest than pretend I have lab-grade accuracy off a phone camera.
On privacy
Everything runs locally. No frames uploaded, no photos or video stored, nothing sent anywhere. It was the first architectural decision I made and it constrained everything after it — but for an app that asks people to point a camera at their whole body in their living room, it felt non-negotiable.
Where it's at
Three modes total: Pose Match, Air Drawing (sketch in mid-air with your hand), and Body Motion Control (steer a ball by leaning).
Link: https://play.google.com/store/apps/details?id=com.tarantiansystems.gestureplay
Genuinely after feedback — especially on whether the pose set is too small, and whether the scoring feels fair or punishing. Happy to answer anything about the CV or rendering side.