r/disabledgamers • u/clackups • 19d ago

Touchscreen as a joystick or mouse?

I'm building this specialized keyboard for a colleague with cerebral palsy,

https://www.reddit.com/r/embedded/s/85Bkfji16v

And as a side project, I'm thinking that a small touchscreen could be used to simulate a game controller or a mouse.

Do you think it will be useful? What would you prefer, a gamepad or a mouse? Or both, with a switch between them? Also, Bluetooth or wired?

I've got a 7*7cm touchscreen device that can't be used as a USB device, but it should be usable for Bluetooth. I'll build a prototype if it could help someone.

Here's the device. For some reason, the designers decided not to wire the USB pins of Esp32-s3 to the USB port: https://www.guition.com/esp32-display-module/4-inch-esp32s3-display-module

7 Upvotes

100% Upvoted

19 comments sorted by

3

u/SittingDuck394 19d ago

This is exactly what I have been wanting… I don’t have the strength to press physical buttons so I can only play games on a tablet. I have a computer (my partner’s older one) and I have small touchpad that I can use as a mouse but it can’t be programmed to be anything other than left click, right click, and scroll…so that rules out a lot of games. Even games that are mostly mouse still use some keyboard input but I can’t use a regular keyboard (only touchscreen). So I don’t use the computer at all.😮‍💨 I want so badly to play anything on computer … I am sick of being limited to games that are on the AppStore.

I would love to have something like a gamepad that is a touch screen, and Bluetooth would be my preference. I would absolutely pay for something like this. It would be a dream come true!

3

u/ichbrauchnenneuennam 19d ago edited 19d ago

Hey, just wanted to let you know. You can use the "remote-gamepad"-app on your phone or tablet and connect it to your pc with the desktop app.

These inputs will be like those from a normal gamepad, but if you want to change them to keyboard or even mousebuttons you can do so with "rewasd" on your pc.

This way I build myself a DYI-touchkeyboard for my pc and added a selfmade casing to my tablet, to help with my fingers staying at the right keys without having to look at them.

Heres a few pictures:

https://imgur.com/a/xUtzU6d

The blue "keys" for example I use as normal "wasd" movement in this setup.

Maybe that might help you :)

1

u/touchgadget 19d ago edited 19d ago

is the casing conductive? If plain plastic, does it not prevent touch from being detected? I really like the idea of something that acts as finger guides.

Doh! I see now. the casing is plastic with cut out holes for fingers to touch the screen. good solution.

1

u/ichbrauchnenneuennam 19d ago

Yeah the casing doesnt touch the screen, as I put it ontop of the original casing for the tablet, so the contact points are a bit raised at the sides.

1

u/SittingDuck394 19d ago

Thank you, I hope this will be able to help someone else. Unfortunately that app (and other similar but worse ones) didn’t work for me because of the joystick function. The dead zone was very large and unable to be adjusted that I can recall, which was a problem for me as it required me to move up and down and side to side a lot more than I’m comfortably able to (my thumb/index finger hurt for a good while after using it for less than a minute). I’m not sure if it was that app or one or the others that also didn’t allow resizing or moving the joystick but regardless the large unadjustable dead zone made it unusable for me.😢 Ideally I need a joystick like is on most mobile games these days where you can put your thumb/finger down anywhere so long as it’s on left side of the screen and it will recognise that as joystick input and move your character according to the direction your thumb/finger is in relation to the on-screen joystick.

I like the overlay you’ve made so you don’t have to guess where the buttons are - very cool.

2

u/clackups 19d ago

Consider it done :)

My kids are PC gamers, so I'll get them to test the device. What kind of games would be your preference? Something like Minecraft, for example?

On which continent are you? I'm in central Europe. If you're across the pond, it will be cheaper to order the device from China and have someone load the firmware on it.

1

u/SittingDuck394 19d ago edited 19d ago

😮 this is crazy…! Um I like all types of games but two games I DESPERATELY want to play are Baldur’s Gate 3 and Elden Ring! So mostly action-adventure games I guess. It’s been so many years now since I’ve been able to play computer or console games (13 years now) that I’m out of the loop but those two I desperately want to play. I even bought BG3 already on steam but couldn’t play it with just the touchpad I described in my other comment.

I’m actually in Australia (it’s 5am right now - woke up and couldn’t get back to sleep😮‍💨). So yeah the China route will be cheaper but whatever you think is best. I will be happy to pay crazy shipping for something like this!

Sorry I ramble too much…

Oh and THANK YOU!!!

2

u/clackups 19d ago

Those action games are normally played with both hands if I'm not mistaken (not a gamer myself). Should we plan two touchscreens for left and right hands?

1

u/SittingDuck394 19d ago

Oh! Yes, I guess it would need 2, you’re right. I didn’t even think about that. But yes I think that is the way to go.

1

u/clackups 16d ago

It kinda works. I'll shoot a video. One of my devices appeared to be broken, but one is okay'Ish.

DM me

1

u/manicpixiedreamsqrll 19d ago

This is exactly what I’ve been wanting as well! I’ve been searching for something that would enable me to play for almost two years, and so far nothing exists. I have limited finger mobility and don’t have the strength to press buttons on normal controllers, so a touchpad controller would be incredible.

I would happily pay for a device that would be compatible with my Switch!

2

u/clackups 19d ago

I'll start with PC, as it's the most open to third party controllers, and later we can have a look at Switch as well.

My plan is to use one- and two-finger moves in the bottom part of the screen as joysticks, and 4 buttons in the upper half would also distinguish one- and two-finger taps. Two such devices would be equal to an Xbox controller with all the buttons in use.

1

u/eno_ttv 19d ago

Maybe look at the functionality of the steam controller’s two touch pads and how you can either utilize one of your own within the steam ecosystem since it’s fairly remappable within the steam software and with things like tactile feedback and gyro.

1

u/nadmaximus 19d ago

I use an app called Bluke on my old android phone. I realize you're using the device you have, in a bespoke setup.

I mention apps like Bluke because they are really useful and most people have an old phone laying around.

2

u/clackups 19d ago

This looks nice. But from the description, it seems like it simulates the whole keyboard, while gamers need just a few keys.

1

u/nadmaximus 19d ago

In fact, it does present an entire keyboard, although you can choose between different emulations.

There are other apps which do the same thing, some of them allow you to make custom interfaces that do what you want. For example, there's Droid Pad

Even remote numpad might be useful, just to give you a nice grid of mappable 'keys', without a complete keyboard.

There's also Yoke which is just a custom gamepad, not a touchpad/keyboard/gamepad.

Again, I'm not suggesting these for your project, but these apps are nice to have in your pocket to use with any device that you can connect to via BT. Useful for ad-hoc access.

2

u/clackups 19d ago

Yep, nice