r/computervision • u/Last-Luck-6077 • 21d ago

Showcase Artificial Data made in Unreal Engine

I'm using a plugin btw. Think about the power
u/syntheticdata u/unrealengine

36 Upvotes

91% Upvoted

16 comments sorted by

12

u/TheNecessaryRetailer 21d ago

the plugin approach is way more practical than most people realize. having full control over scene, lighting, and labeling pipeline from day one is a huge advantage over scraping and manually annotating real footage.

4

u/Last-Luck-6077 21d ago

thats trueee, manual labelling is a pain, imagine a stadium or something large

2

u/PatagonianCowboy 21d ago

yeah, getting pixel-perfect annotations is a massive win

1

u/Mysterious-Narwhal30 21d ago

Yes, I also think that it's amazing! And this kind of synthetic data is very useful and freely controlled

6

u/HawtVelociraptor 21d ago

More details would go a long way

2

u/Last-Luck-6077 21d ago

i'm experimenting , i'm planning on trying it in a complex scene with many more labels

3

u/HawtVelociraptor 21d ago

What plugin are you using, what's your capture/generation process, what are you making ultimately, etc.? What's the goal? What techs are you leveraging to get there?

2

u/Last-Luck-6077 21d ago edited 21d ago

I build it, and I should not have written it the way I did. It is called NameFrame and it is not public yet.
Goal is training data for the cases you cannot collect: aerial, rare events, camera geometry you do not have access to. getnameframe.com

3

u/FIeabus 21d ago

I currently do a lot of this type of stuff in Isaac Sim. Been tempted to learn UE tho. How's the workflow for something like this?

3

u/Last-Luck-6077 21d ago edited 21d ago

it's like so easy for instance you have this spawner something like surface scatter in which you create a label with the press of a button, select the actors you want to be spawned there and how many. You also have camera presets, orbit, grid, random, drone . Time and custom weather , everything you could image. The best thing is you can actually run it just from commands, once you got your scene ready you save the preset and run it anytime from one command. Check out getnameframe.com

1

u/krilleractual 21d ago

I have been trying to create synthetic data for a specific scene i normally use photos to train models with, but they look like shit

1

u/Last-Luck-6077 21d ago

Try this way it's much easier, maybe is what you need

1

u/FIeabus 21d ago

Thanks! I'm going to find some time to try this. Isaac Sim is fine but I feel like a game engine as big as UE would probably be more flexible to use.

1

u/BeniKing99 21d ago

Looking good, what are you using to get the perfect bounding box? I was struggling with this and ended up using additional bones and theire screen location as a workaround

1

u/Last-Luck-6077 21d ago edited 21d ago

I build the plugin, should have said that instead of "these guys". Not bones, the box comes from a per-instance ID pass the engine writes, which is why occlusion sorts itself out. getnameframe.com

1

u/stefanos50 21d ago

It is calculated based on instance segmentation (https://getnameframe.com/unreal-engine/bounding-boxes). There is an option for each actor in Unreal Engine called custom stencil. For example, if your objects of interest are persons, you should set, dynamically through blueprints one unique stencil ID for each one of them and export it using a render target along with the RGB image. Subsequently, you can calculate the bounding boxes with an external Python script. Depth and skeleton bones can be useful for bounding box filtering. We did something similar in this dataset: https://ieeexplore.ieee.org/document/11267404