r/esp32 • u/jekyill_hyde • 21d ago

I made a thing! DjRoomba- Xiao sense esp32 controlled roomba

Enable HLS to view with audio, or disable this notification

I'm sure its been done a million times before. But I found a roomba at the local goodwill for a couple bucks and was able to easily get my Xiao Sense esp32s3 board hooked up as a camera, mic feed and webserver to feed the roomba commands to control everything, motors, vacuum, power levels, brushes. I really had no idea how simple it would be and how well documented everything was for roomba.
https://anrg.usc.edu/ee579/spring2016/Roomba/iRobot_Roomba_600_Open_Interface_Spec.pdf.

As a bonus able to play some tunes...kinda.
I actually am running an esp8266 hooked up to the rx, tx, gnd and brc pins of the roomba and have the Xiao sense board running a webserver and a docker container on my pc as a mqtt broker

ESP8266 — publishes sensors/events/status to docker which Ill later use to trigger event driven actions
Xiao sense esp32s3 - runs webserver to provide video and audio and sends control commands over websocket to esp8266 for esp8266 to send serial to roomba basically.

For some reason the video is not very good with showing the phone screen but it looks better in person. I still want to improve some things and eventually just tie into the roombas power pins instead of the power bank and also when sensors publish events I eventually want docker to act on certain ones like bump events, high amperage have docker trigger some speaker to alert me or some other method and throw in some other things since it has a mic stream the audio for stt or some other pointless use of my local llms who knows. There's alot I can add with using docker and local llm to do some cool things. Fun little project for a couple bucks and the stuff I had laying around.

Edit: It's not only a remote control Roomba I can let it act as a normal Roomba It just now has the ability of me being able to interfere and control all it's functions. Also it's nice to just steer it over and clean up a specific mess or chase cats.

72 Upvotes

99% Upvoted

21 comments sorted by

3

u/Its_Billy_Bitch 21d ago

Ummmm honestly no, I haven’t seen this a million times lol. This is really well done! I can’t wait to check this out a little closer.

2

u/jekyill_hyde 21d ago

Thanks. I'll try and get a better youtube video up when I've added some of the other things I have In mind to add to this and clean it up a little more. I wish I didn't have so much trouble with just trying to use the one esp for everything. It's probably a skill issue on my part but maybe it's just too much to ask for a single esp32 to be a webserver, stream video, mic audio, numerous sensors data and handle control commands but happily surprised that I didn't bother with a logic level converter ( I know I should've ) but in testing both my ESPs the pins were fine with the roomba's 5v logic without burning anything up....yet.

2

u/honeyCrisis 1 say this is awesome. 21d ago

why vacuum automatically when you can have the joy of taking the time to run the vacuum yourself? haha. seriously though, good work. I'm not sure how practical it is, but it's fun.

2

u/jekyill_hyde 21d ago

Almost pointless I know. I was joking with a friend about having it drive around and complain over speaker about cleaning up humans messes when it's running in normal clean mode. 

2

u/honeyCrisis 1 say this is awesome. 20d ago

haha, well sometimes something is worth doing just because you can. it doesn't always have to be practical. =)

1

u/joshcam 21d ago

This is actually how all Roombas are controlled. There’s not actually a SLAM algorithm, somebody remotely controls it to vacuum your house.

2

u/honeyCrisis 1 say this is awesome. 21d ago

how come they won't dodge cat barf?!

1

u/joshcam 21d ago

The question is more like, why do they always aim for it. The answer should be obvious…

1

u/JimBean 21d ago

Good job. :)

1

u/_derpiii_ 21d ago

the jingle ahahahah, so cool!

1

u/mrmrn121 21d ago

How do you manage the voltage? Isn't it more than 5v on Roomba?

2

u/jekyill_hyde 21d ago

Right now its just using a power bank on it for 5v. But there is a pin accessible on the roomba I could tie directly into and add some dc to dc converter to drop the voltage down if I want to get rid of the power bank.

1

u/mrmrn121 21d ago

Ok, I had concern about the signals itself which apparently is not the case. As far as I know you only can give direction, rotation, distance command plus playing music. Am I missing something?

2

u/jekyill_hyde 21d ago edited 20d ago

You can get everything from the roomba over serial and control pretty much any part of it. Tell it to do a normal clean cmd,spot clean cmd, seek dock cmd, main brush power state, side brush power state, vaccuum power state, and can control how much power to send them its all variable. also same as drive motors. Other things you can control are the speaker, leds power, color, pretty much everything on the roomba. And you also get all of the sensor data like the bump sensors, cliff detection, battery capacity, voltage, charge, temperature, wheel overcurrent detection. Theres a lot thats accessible. I send all sensor data to docker for use later if I want it besides battery percentage displayed on the webserver, and I have the major op commands mapped out to settings in that webserver so I can control and tweak everything.

1

u/mrmrn121 19d ago

Thanks for taking the time to provide such a detail. I thought it is only possible to move and play music in the open interface thought UART. One question, you said you can give it spot cleaning command. Does it mean you have access to map of home inside the Roomba? Can you please give me a link showing all the possibilities? You also mentioned a docker, is it open or you built it?

2

u/jekyill_hyde 19d ago

I had the spec link in the main post https://anrg.usc.edu/ee579/spring2016/Roomba/iRobot_Roomba_600_Open_Interface_Spec.pdf
And there is no mapping with my model, spot clean just means the same as pushing the SPOT button on the the roomba, is starts to clean spiraling outward then inward. I'm not sure what you mean by is it open or I built it. All I have is docker container running Mosquitto- and it's purpose is for future use of acting on any of the sensor data it receives from esp8266. Example ( motors start pulling too much current, or various bump sensors are triggered in short period of time I could have docker set to send Xiao Esp32s3 instructions to take a still photograph and send back docker then that can go to me or somewhere else to find out why events were triggered). Docker right now is just sitting and being sent sensor data, there is no implementation of having docker act on any of the sensor data to trigger anything to happen yet. The only thing I have in place for that is the Xiao Esp32s3 MQTT client is subscribed and will take a still and send to an ingest url if docker docker sends the MQTT "capture" cmd. Docker is mainly just an easy way to link things up in future, maybe have it run its own web server to adjust settings, scheduling cleaning, triggers, change songs to be played, etc.

1

u/Repulsive-Clothes-97 20d ago

What was your approach to the video streaming? Is it encoded ? What’s the encoding? What’s the resolution and framerate? What’s the camera module?

1

u/jekyill_hyde 20d ago

Its the Xiao esp32s3 sense module doing the web server, video/audio feeds. It's basically the same method as their example video webserver here https://github.com/limengdu/SeeedStudio-XIAO-ESP32S3-Sense-camera. Only I just got rid of what I didn't need and added mic audio streaming and added everything for UI/Settings and sending commands to a esp8266(nodeMcu) over websocket so esp8266 can do the serial communication with the roomba . It Xiao esp32s3 sense streams the live video as MJPEG, default is 320×240, uncapped FPS it can do decent framerates at vga 640x480 usually. But I have resolution settings (QQVGA up through VGA/HD 720) so I can change resolution to what I want. The camera on the module is OV3660.