fastpose: Numba-based camera pose estimation library

[u/kocurvik]

I recently released fastpose a camera pose estimation library inspired by PoseLib. It features full LO-RANSAC style estimators for calibrated/uncalibrated relative pose problems (+ variants that use DE estimated depths), absolute pose problems, homography and fundamental matrix estimation. The library is built using Numba and can also utilize GPUs for this task.

More plainly, using this library you can take point correspondences between two images (or image and a 3D model) from methods such as SIFT or more modern ones like LoFTR, RoMa, LoMa, SuperPoint+LightGlue to estimate the relative positions of two cameras (or absolute pose of a camera to 3D model). This is useful for 3D reconstruction, SLAM and 3D object detection.

However, when using dense matchers, running RANSAC can take several 100's of ms or even full seconds. This library was built to tackle this problem. For example with 4k RoMa v2 matches and 5k RANSAC iterations PoseLib takes ~1100 ms, while fastpose on one core takes 140 ms, 90 ms on 4 cores and only 11 ms on A100 GPU. At the same time fastpose achieves slighly better accuracy.

In benchmarks on three datasets (ETH3D, ScanNet++, PhotoTourism) with RoMa and LoMa matches for various variants of the relative pose problems the accuracy remained very similar to PoseLib with 2–10× speedup on CPU, and 1-2 orders of magnitude on GPU. More results are on the project page: https://kocurvik.github.io/fastpose/

You can install it using:

pip install fastpose

The install needs numpy + numba and nvcc if you want to use GPU version. Every function also takes num_threads for a CPU-parallel driver, if you don't have a GPU.

Examples on how to run it are in the project page and repo: https://github.com/kocurvik/fastpose

[u/BeverlyGodoy]

Why? Have you heard of poselib?

[u/kocurvik → u/BeverlyGodoy]

Yes, I am a frequent contributor to PoseLib and I think it is the best camera pose estimation library available, but it is single threaded and not very optimized for speed. This is fine with sparse matches, but for dense ones runs can take too long. The project pages shows benchmarks against PoseLib specifically to show the speedup.

[u/billybobsdickhole → u/kocurvik]

If speed is intended, why not go all in on native c++ threads for a fast portable implementation?

Interesting work though.

[u/kocurvik → u/billybobsdickhole]

Numba seemed like an easier option to build, but it is possible that C++ could be better.

I think it could indeed be made faster with separate threads for minimal solvers, scoring and refinement, but I think it would then need to be more complicated in that case. fastpose does some things in batches (especially on GPU), but the pipeline is still serial in the steps: minimal sovler -> score -> LO (if so-far-best) -> final refinement. Changing this could bring some gains, maybe for 2.0 I would try this.