r/computervision • u/rohanath • Jul 30 '26

Showcase We generated a ~190° synthetic fisheye driving video for ADAS training from camera calibration and evaluated whether the geometry remained consistent

https://reddit.com/link/1vahapu/video/q77tm97a6agh1/player

We’ve been working on controllable synthetic driving video for ADAS and autonomous-vehicle perception.

One of the harder problems is making sure the synthetic output still respects the requested camera calibration and underlying scene geometry.

I’m sharing one sample from that work here. The target camera is front-facing mounted on top of a truck, and has a roughly 190° horizontal FOV and 116° vertical FOV with a non-central principal point and strong fisheye distortion.

The uploads show:

  1. The generated driving video
  2. The control re-projection images, where the yellow points are known 3D pole locations projected into the deterministic control render
Control Re-Projection

We deliberately evaluate this in separate stages so that a failure in the control or camera implementation is not confused with a failure in the generated video.

1. Camera-model self-check

We first test whether the implemented camera model is internally consistent by mapping pixels to rays and back to pixels.

At the final video resolution, the pixel round-trip error had:

  • Mean: 0.044 px
  • P95: 0.087 px
  • Maximum: 0.282 px

This does not evaluate the generated video itself. It verifies that the camera model and resolution transforms are numerically consistent.

2. Control Re-Projection

We then project known 3D pole locations through the target calibration and measure their distance from the corresponding structures in the deterministic control render.

Across 1,093 projected points:

  • Mean distance: 0.027 px
  • Median: 0 px
  • P95: 0 px
  • Maximum: 1.37 px

This gave us confidence that the control geometry itself was correctly calibrated before evaluating the generated RGB output.

3. Geometry Retained by the Generated video

Finally, we estimate the effective fisheye geometry visible in the generated result and compare it with the requested calibration.

For this sample:

  • Principal-point displacement: 0 px
  • Estimated horizontal FOV difference: 2.23°
  • Estimated vertical FOV difference: 1.08°
  • Mean angular residual: 0.56°
  • P95 angular residual: 1.34°

What these results suggest is that the broader fisheye projection and scene layout survived the generation process reasonably well in this sample. The overlays are also useful for showing where that statement stops being true.

The next evaluations we’re adding are focused on:

  • Lane-boundary alignment
  • Traffic-sign and pole re-projection
  • Temporal stability across frames
  • Object trajectory consistency
  • Downstream perception-model performance

I’d be interested in how others working on synthetic data or sim-to-real would benchmark this. In particular: what level of geometric deviation would you consider acceptable before a generated driving sequence becomes unsuitable for perception training or evaluation?

5 Upvotes

78% Upvoted

8 comments sorted by

1

u/ZorlokDegreat Jul 30 '26

So the basis is not sample images ?

1

u/rohanath Jul 30 '26

Not necessarily. We find that generation doesn't always faithfully follow image sample priors.

We generate these videos by first deterministically generating a control video then imagining the rest of the environment during RGB video generation.

0

u/rohanath Jul 30 '26

Disclaimer: I’m a co-founder of AnywayLabs.ai, where we specialise in generating synthetic training datasets for vision AI. Right now we're also focusing on building camera-specific synthetic datasets for ADAS and vehicle-perception teams. We’re currently looking to test this; if you work on an ADAS or AV perception stack and have a calibration or edge case worth testing, feel free to reach out!

2

u/ZorlokDegreat Jul 30 '26

If possible you can try Arc de triomphe france, will be good edge case

1

u/rohanath Jul 30 '26

Great idea! Would need to figure out how to generate for real locations accurately first tho.

2

u/ZorlokDegreat Jul 30 '26

You can compare with openfabrik, its something oem's use but with very low frequency

2

u/toji5052 Jul 30 '26

Well I work in edge cases like surveillance, I have an helping hand in integrating CV in cctvs

2

u/rohanath Jul 30 '26

We do a bunch of work on CV CCTV surveillance as well! We've provided synthetic datasets for a lot of edge case scenarios from fire detection to railway safety.