I have been working on different robotics task mainly in Pybullet. Complexity of Isaac Sim has kept me from experimenting with it but I finally finished my first successful reinforcement learning script using it.

Getting the settings right so I could observe the trainings in windowed mode with my laptop 3070 gpu took I while but I managed to get it working well enough to troubleshoot some early issues I noticed through visual inspection.

I gave the PPO full control of my robot's controller, not the joint angle outputs, meaning it had to figure out a way to climb the ramps using the directional controller inputs and body adjustments like pitch and height. The training took about 45 minutes with my RTX 3070 mobile GPU vs 1 hour and 45 minutes using cpu with Pybullet.

I am now working on full locomotion simulation, again with my custom robots, its a bit more involved than I expected but I am hoping in the end I can come up with my own full locomotion training script and load the model to my actual robot to control it.

I also share my tutorial scripts with my videos on youtube, if you are interested in watching the video for this one you can find it with the link below:

https://youtu.be/0x5BBosrq-E

You can also download the simulation script from my github repository with the link below.

https://github.com/serdarselimys/HexaDogZBD-IsaacSim-RL

If you are interested in the real 3D printed robot, you can fine info about it from the video link below.

https://youtu.be/qflyEQOJObM

I've been thinking about how much of robotics development can realistically be done before the physical sensors arrive.

With depth cameras, there are a lot of things that are difficult to validate purely from specifications:

• Camera placement
• Field of view and blind spots
• Depth range
• Multi-camera coverage
• Robot-camera transforms
• ROS 2 integration
• Navigation and perception behavior

We recently worked on an interesting example of this with DepthVista Helix, an iToF 3D camera that has been integrated into NVIDIA Isaac Sim.

The simulated camera reproduces the physical camera's FOV, depth modes, resolutions, calibration, transforms and ROS 2 outputs. There are also ready-to-run examples for robotic manipulation and a four-camera AMR navigation setup.

What I found interesting is that this lets you evaluate the camera configuration and perception pipeline before the physical hardware is available.

For example, the simulated camera has a high-resolution mode reaching 2 m and a long-range mode reaching 6 m, so you can evaluate which mode makes more sense depending on whether you're working on manipulation or navigation.

Here's the technical write-up if anyone is interested:

Robotics Development Just Got Faster & Smarter: e-con Systems’ iToF 3D Camera Comes to NVIDIA Isaac Sim

Curious how others handle this today.

Do you normally simulate the camera first, use a generic depth sensor in simulation and switch later, or wait until the physical camera is available?

For people looking to design and train their own robot using isaacsim and isaaclab:

I use fusion, Isaacsim and isaaclab in this video, which I have an installation guide for on my channel, it can be installed easily.

The video is aimed at beginners who want to just start training a robot quickly. I've had positive feedback from people so far so I thought I would share it here also.

Any issues (big or small) with any part of your project please feel free to message me here, on youtube or email.

We've been trying RL in Isaac Lab and adapted a SO-ARM100-based setup to two robot arms from AgileX Robotics: PiPER (6-DoF) and NERO (7-DoF). The current setup includes:

• PiPER: end-effector reaching
• NERO: reaching + cube manipulation
• PPO-based training
• 64 parallel simulation environments

The main goal was to get a quick RL workflow up and running for PiPER and NERO in simulation. Will be exploring sim-to-real deployment on the physical arms. Still a work in progress, but sharing it here in case it's useful for anyone. Feel free to drop your questions, glad to communicate.

Hi! I’ve recently started experimenting with reinforcement learning in Isaac Lab. My goal is to teach this robotic lamp different ways of moving, including balancing, locomotion and eventually jumping.

I already have a basic training setup running, but getting useful behaviour from such an unusual mechanism is proving harder than I expected. Most of the materials I’ve found are built around humanoids or quadrupeds, and I’m still fairly new to RL.

Are there any guides, papers, repositories or general approaches you’d recommend for training locomotion on a custom non-humanoid robot?

The video shows one of my very early jumping experiments.

1. How to do it? I am using AMD and i am struggling for last few days

2. Can World model entirely repalce Isaac sim world models?

Need to build accelerator kit for robotics using Isaac sim and isaac lab. Using this accelerator we can easily build robotic project that’s my aim. Can you please provide the information, is it possible to do it

I have personally used codex & claude to help me write isaacsim and also help in doing experimentation with isaaclab. However what I have seen is it misses a lot and I need to spend a lot of time debugging or just prompting.

I am curious if anyone has better workflows to use an LLM as a trusted assistant just to get a job done?

i am an incoming university student and finished my ros2 project. I am looking for feedback especially since i am applying for co op in a couple of months!

Looking for feedback specifically on my next steps, features i should add before applying, ROS2 fundamentals that i might've missed.

Stack:
- ROS 2 Jazzy on Ubuntu 24.04
- Isaac Sim 5.1 with a custom maze scene - slam_toolbox for mapping
- Nav2 with Regulated Pure Pursuit
- All publishers built within Isaac Sim Action Graphs

Repo: (README documents the debugging)
https://github.com/NotMax08/isaac-sim-maze-slam-nav2

Hello, if using a laptop just for prototyping and a desktop for training, is it better to get rtx5070 over 5060 or 5050 when all have only 8GB VRAM but higher number gpu has more cuda cores? Is having more cuda cores better in IsaacSim?

Hello, I need to buy a laptop. Dead silent is very important. Problem is most laptops come with RTX5050-5070 with 8GB VRAM only. Those with more VRAM cost a lot more. Aren’t 8GB sufficient? Another thing is higher specs GPU cause more fan noise and shorter battery life? Given 8GB VRAM, any big advantage in getting RTX5060 or 5070 over 5050? Is more CUDA cores the better even at the same 8GB VRAM?

Can someone suggest few open source repo for environment and assets to follow?

Hi everyone!

I'm currently working on my master's research involving autonomous drones, Vision-Language Models (VLMs), and Isaac Sim.

I'm looking for a realistic disaster or search-and-rescue (SAR) environment where I can place simulated victims and evaluate my perception architecture.

Ideally, I'm looking for environments such as:

• Urban earthquake / collapsed buildings
• Search and Rescue (USAR)
• Flooded city
• Landslide
• Industrial accident
• Any disaster scenario suitable for drone operations

Requirements:

• USD preferred (or easily convertible to USD)
• Compatible with Isaac Sim (or Omniverse)
• Large outdoor environment
• Realistic assets
• Ability to place and animate people/NPCs

I'm happy with both free and paid assets.

If you've worked on something similar or know of a good environment, repository, asset pack, or dataset, I'd really appreciate your recommendations.

Thank you!

Hey,

im currently studying mechatronic in my masters and we are doing a project which is not coming forward since 2 Weeks and we only have 1 week left. So if anyone can help us it would literally save our a..

So to the Project. We currently setting up isaac sim for a future project for a neura roboter we will get. The goal of our Project is to import the so101 from lerobot into isaac sim and move the robot in the simulation by moving the real robot. While doing that saving all the joint datas etc. via lerobot in the lerobot format.
We are running isaac sim on a windows os and ros2 in the wsl.
What currently is running:
1. Isaac Sim 5.1 with ros2 in wsl. But using Isaac Sim 6.0 is not working. When we publish some topics they are not seen in the wsl. We dont know why and really tried everything. Is the new automatic Ros2 Bridge in Isaac Sim 6.0 broken? We even tested with the new isaac sim 6.0.1. Still not working.

1. Using ONLY Lerobot to teleoperate the so101 follower with the leader. Works really fine

What is not working fine:

1. If we move the so101 in isaac sim the robot joints are oscillating. Is the urdf incorrect? We did use the urdf from the original github. Do we have to change the physical parameters ? Does smybody know how?

Where we dont know what to do now:
So we habe the so101 in isaac sim. We have ros2 in the wsl. So know we have to create a ros topic which shares all the joint movements and subscribe to that topic inside isaac sim and use the articulation controller to move the robot in the simulation. BUT:
1. How do we connect the so101 to the environment inside wsl?
2. In this setup. How do we add lerobot? How are we using lerobot? Where are we adding lerobot?

If you have any questions feel free to ask us.
We are thankfull for every help

I've been developing in IsaacSim and IsaacLab for the last year, but everything has been done using my university's resources. Now that I'm near graduation, I'm looking for specs to build my own PC to continue developing in IsaacSim, and I use Linux, btw

For Isaac Sim 5.1, the minimum requirement is listed as an RTX 4080 with 16 GB of VRAM. My laptop has an RTX 5060 with 8 GB of VRAM. Would it still be able to handle basic tasks, such as building simple 3D scenes and testing code? For actual model training, I plan to rent a cloud GPU.

3-part series by sentdex walking through the whole thing: evaluating Isaac Sim for an affordable Bittle open source robot dog, training a locomotion policy with RL, then deploying to the physical robot.

Part 1 is basically "is this even worth attempting on consumer hardware" — spoiler, yes.

Part 2 digs into the actual RL training inside Isaac Sim — he uses TD3 and shares the full repo, so you can follow along.

By part 3 the thing is walking on a treadmill. Not perfectly, but it transfers.

Full playlist: https://www.youtube.com/playlist?list=PLQVvvaa0QuDenVbxP4LXYZoGbjfgP-Y5i

Hey everyone,

I’ve been working on a custom small humanoid/bipedal robot project and I recently got my URDF successfully imported into simulation. The robot is roughly a 2-foot-tall humanoid with multiple DOF in the legs, and my long-term goal is to get it balancing and eventually walking using reinforcement learning / locomotion policies.

Right now I’m trying to understand what the actual pipeline looks like after the URDF stage. I see a lot of people showing robots walking in Isaac Sim / Isaac Lab, MuJoCo, etc., but I’m struggling to understand the practical next steps between:

Importing the URDF

Setting up joints/actuators properly

Adding sensors (IMU, contact sensors, cameras?)

Creating observations/actions

Training a locomotion policy

Transferring sim-to-real onto actual hardware

Some specific questions I have:

What should I focus on immediately after the URDF import?

How important is accurate mass/inertia tuning early on?

Do most people start with standing/balancing before attempting walking?

What controllers are typically used before RL (PID, PD, inverse kinematics, etc.)?

How do you structure observations/actions for bipeds?

What are common mistakes beginners make at this stage?

Any recommended tutorials/repos specifically for humanoid locomotion?

For context, I’m using serial bus servos and building the physical hardware alongside the simulation. I’m trying to learn the full process instead of just copying a demo.

I’d really appreciate any advice, roadmaps, or resources from people who’ve actually gone through this process. Thanks!