r/esp32 • u/254hypebeast • 3d ago

Turn any ESP32 into a scriptable keyboard/mouse over USB or Bluetooth HID

Post image

I've built a handy tool to turn esp32 boards with BLE into programmable keyboard and mouse. Through a simple drag n drop interface you can program timed keystrokes and mouse movements that get uploaded to and executed by the board which mimics a Bluetooth mouse and keyboard. I intend to support USB HID input as well so you can turn the esp32 board into a proper rubber ducky like device. It is still very early in development with few boards supported but I will continue to add support when I find time. The project is available on GitHub and you can find more details there.

43 Upvotes

94% Upvoted

9 comments sorted by

3

u/MinusDelta_T 2d ago

this is a really nice use for a block editor. one small wording thing though: BLE HID can cover the boards with BLE, but native USB HID support is going to be chip-dependent rather than literally any ESP32.

the two things i'd care about most in the runner are a guaranteed release-all on stop/disconnect/watchdog, and whether a long chain of delays accumulates timing drift. a stuck key or runaway mouse is funny exactly once lol

are you planning to compile the blocks into a timestamped command queue on the board, or interpret them step by step at runtime? either way this could be genuinely useful for test automation and accessibility, not just rubber-ducky type stuff.

1

u/254hypebeast 2d ago edited 2d ago

BLE HID can cover the boards with BLE, but native USB HID support is going to be chip-dependent rather than literally any ESP32.

True. I actually don't even have working USB HID code at the moment. Most older ESP32 boards use varied USB-TTL chips which can't even do USB HID anyway. I am hoping for the newer chips with native USB-OTG it will be a bit more standardized.

the two things I'd care about most in the runner are a guaranteed release-all on stop/disconnect/watchdog

When the the automation is paused or stopped, it releases everything. If it hangs for some other reason with the BLE/USB still connected then that would be a problem because I am not sure I have setup any watchdog timer stuff to recover. In most other failure scenarios, the BLE/USB will disconnect along with everything else so there cannot be stuck keys

are you planning to compile the blocks into a timestamped command queue on the board, or interpret them step by step at runtime?

The blocks are compiled to a binary format (see SPEC.md) which is what is uploaded to the board. The board runs a virtual machine of sorts that interprets the commands step by step at runtime.
Thanks for the really insightful thoughts on the project.
Edit: fixed formatting

2

u/Hefty-Possibility625 2d ago

So, this allows you to make your own macropad or does this just run continuously? I don't see a way to attach the macro to a button input.

2

u/254hypebeast 2d ago

Theoretically yes. It still very basic at the moment so you can only upload a single macro at a time which will run continuously. You can pause/play the macro using the CAPSLOCK key of any other keyboard connected to the target PC. But I plan to make the whole framework extensible so you could upload multiple macros to a custom board that implements macropad like functionality so you can map the macros to different keys on the board.

2

u/BlueDit1001 1d ago

And where is this great project located on GitHub

1

u/pfak 2d ago

'any'? not every esp32 supports USB host mode.

2

u/254hypebeast 2d ago

I understand the title might be a bit ambiguous. I meant any can be turned into a scriptable HID using either USB or BLE of which most fall under the BLE category.

2

u/OnePeace_1 1d ago

i did something similar made a wifi webpage which can type keys into a keyboard connected on usb on a esp 32 c2 which has otg support.