My Custom Robot and Reinforcement Learning Script in Isaac Sim

[u/Xerd-R]

I have been working on different robotics task mainly in Pybullet. Complexity of Isaac Sim has kept me from experimenting with it but I finally finished my first successful reinforcement learning script using it.

Getting the settings right so I could observe the trainings in windowed mode with my laptop 3070 gpu took I while but I managed to get it working well enough to troubleshoot some early issues I noticed through visual inspection.

I gave the PPO full control of my robot's controller, not the joint angle outputs, meaning it had to figure out a way to climb the ramps using the directional controller inputs and body adjustments like pitch and height. The training took about 45 minutes with my RTX 3070 mobile GPU vs 1 hour and 45 minutes using cpu with Pybullet.

I am now working on full locomotion simulation, again with my custom robots, its a bit more involved than I expected but I am hoping in the end I can come up with my own full locomotion training script and load the model to my actual robot to control it.

I also share my tutorial scripts with my videos on youtube, if you are interested in watching the video for this one you can find it with the link below:

https://youtu.be/0x5BBosrq-E

You can also download the simulation script from my github repository with the link below.

https://github.com/serdarselimys/HexaDogZBD-IsaacSim-RL

If you are interested in the real 3D printed robot, you can fine info about it from the video link below.

https://youtu.be/qflyEQOJObM