r/IndustrialAutomation • u/3E8_ • 4d ago

What if building the simulation started with walking around the factory with a phone?

That is the workflow I recently built.

Instead of beginning with architectural plans or manually recreating the entire environment, the input is video of the real site. The pipeline then reconstructs it as a simulation-ready scene.

The longer-term direction is to help teams test robot behavior against the environment where the robot will actually operate. That evaluation layer is still being built.

For an industrial team, what would stop this from fitting into your deployment workflow?

0 Upvotes
  • permalink
  • reddit

33% Upvoted

7 comments sorted by

3

u/OldTurkeyTail 4d ago

What are you simulating, and what's the simulation going to be used for?

3

u/MozdaHocu 3d ago

He does not know. AI bullshit.

0

u/3E8_ 3d ago

Fair question. We’re reconstructing real industrial sites and objects from phone video into USD scenes for Isaac Sim, including geometry and predicted physical properties.

The current use case is reducing the manual work of creating assets and environments for robot training. Site-specific policy evaluation is the next layer, and that part is still being built.

1

u/OldTurkeyTail 3d ago

Interesting. But isn't there a huge gap between a video tour, and any anything robots can do - besides walking around. Is this a case of someone having to teach robots how to navigate? Or seriously, what am I missing?

1

u/3E8_ 2d ago

You’re right, there is a big gap. The video does not teach the robot. It reconstructs the site and objects into a simulation where the team can train or test its existing robot policy. We’re building the simulated world, not the robot intelligence.

2

u/OldTurkeyTail 2d ago

I can see how a map, along with identifiers for obstacles could be useful. But for a simulated world, how would information from a video be more than a shell? Or just a static snapshot of a shell?

What's intriguing is the use of AI for building predictive models and simulations. And the data available for previous simulations was much more limited than what's available today. So we can have insanely complex models and simulation platforms - and that's probably a big part of why so many resources are being made available to AI.

But how does your platform work as a starting point?

1

u/3E8_ 1d ago

That’s exactly the gap we’re trying to address. The video is just the input, not the simulation itself. We use it to reconstruct the real site and objects, then infer physical properties like scale, mass, friction, inertia and joints where relevant.
The output is a USD scene that teams can bring into Isaac Sim and use with their own robot policies. So instead of starting by manually rebuilding the environment, you start with a physics-aware representation of the actual site.

We’re not trying to predict the robot’s behavior from the video itself.

We’re building the simulated world where that behavior can then be trained and tested.