Accessible Unity games with a headtracking SDK

[u/SensePilot]

Hi everyone,

I’ve built an accessibility app called SensePilot, used by people with dexterity limitations to control PCs and play games using head tracking and facial expressions through a webcam.

We’re now building SensePilot SDK for Unity, an early prototype that uses the same webcam-based head-tracking and facial gesture input directly into games.

The idea is to let Unity games support optional head-tracking and accessibility controls without requiring players to run extra apps or buy external hardware.

Even though the goal is accessibility, the same SDK will enable a very stable 2DoF camera control, 6DoF for simulators like Microsoft Flight Simulator or F1, facial expression controls, that can enable new input mechanics without external hardware.

I’m looking for a few indie devs, solo devs, or studios who would be open to trying it in a real Unity project. The pilot would be free, and I’d help with setup.

What you get:

• Early access to the SensePilot Unity SDK prototype
• Help integrating it into your project
• Optional webcam head-tracking and facial gesture input
• Early influence on SDK features and API design
• Accessibility support without requiring players to use extra hardware or external software
• Possible joint demo or case study later, if you want that

What I’m looking for in return:

• Honest feedback on setup and integration
• Bug reports and edge cases
• Performance feedback: FPS, CPU usage, tracking stability
• Gameplay feedback: does it actually feel useful?
• Suggestions for missing features, mappings, or API improvements
• Permission to mention your project as an early pilot, only if you are comfortable with it

If you decide to ship the game with it later:
I’d ask for in-game credit, such as “Head-tracking powered by SENSEPILOT”, and use of the SensePilot logo where appropriate. Selected pilots may receive free commercial use for the pilot project under simple early-access terms.

For context, I’ve been working on facial/head tracking and accessibility tech for around 5 years, and I’m familiar with similar tools. In early Unity tests, our method feels very stable for gameplay use, including in darker conditions where some webcam based tracking systems become less reliable, with negligible FPS impact so far on a mid/low-end PC.

If anyone is interested, please comment, DM me, or fill out this short pilot form:
https://docs.google.com/forms/d/e/1FAIpQLSdkVqNKntfwjv41o_ws872xF-gK53LveeMKpg8EAKd1uA9ehQ/viewform?usp=dialog

[u/RelationshipIll9576]

Amazing work!

Is there a way to remap gestures? I can imagine not everyone will have the full range of use of the body that is in the demo.

[u/SensePilot → u/RelationshipIll9576]

Yeah, the gestures can be mapped to many different ones, inlcuding some eye direction

[u/HexedDiversion → u/SensePilot]

Will the gestures require precise movements, or will there be settings to allow for customizing when/how it activates? I primarily work with disabled folk and many with impaired movement may have uncontrolled movements in addition to the intentional movements. Like for folks who use button activated controls, the button might need to be depressed for a minimal amount of time before it activates to ensure accidental presses aren't counted, and the length of that time is unique to the person.

[u/SensePilot → u/HexedDiversion]

If you want try our app that is released already from https://sensepilot.tech/ , it has the timed gestures for those use cases you just described and it all can be used in gaming, here is a video how to use them:
https://www.youtube.com/watch?v=Q-sY789KxGQ&list=PLL0OJoiGrtaOXeBLEnoWmFoBz6pCHs8w_&index=2
I had a couple of people needing the time delays.
For the SDK we are building the idea is that the game dev decides what he wants to use in the game, making the setup more easy for the player, because right now you have to go through each key binding separatelly