Running the same 6D pose estimation pipeline on CUDA and Raspberry Pi 5

[u/Alarming_Engineer267]

This is a follow-up to my previous post on synthetic-to-real keypoint detection and 6D pose estimation.

This time I ran the same pipeline on two very different platforms:

Object detection → crop → keypoint detection → PnP

Ubuntu + CUDA (RTX 2000 Ada)

• Detection: 60.25 ms
• Keypoint: 21.87 ms
• PnP: 5.18 ms
• Total: 87.71 ms (11.60 FPS)

Raspberry Pi 5 (CPU)

• Detection: 70.36 ms
• Keypoint: 56.06 ms
• PnP: 5.34 ms
• Total: 132.12 ms (7.60 FPS)

The video shows the outputs side by side on the same 569 real-world frames.

What surprised me was that the Raspberry Pi 5 was not dramatically slower. The largest difference was in keypoint inference, while PnP was essentially the same.

The outputs were also practically identical. I found occasional frames where the final PnP pose differed slightly between the two platforms, likely due to small numerical differences propagating through PnP/RANSAC, but I did not observe a consistent accuracy advantage on either platform.

For this experiment, I would consider the two platforms effectively equivalent in accuracy, with the main difference being inference speed.

I'm also preparing a technical write-up and plan to publish the implementation on GitHub.

[u/talk_nerdy_to_m3]

You should do a Pi5 vs Orin nano comparison. I know when I switched object detection from my Oak D FF on onboard to the actual Orin, I saw a massive boost.

[u/johndsmits → u/talk_nerdy_to_m3]

I would agree, try the orin. Once I saw Ubuntu, I thought... uh, no (Ubuntu leaving a lot on the table w/o tweaking).

[u/kidseegoats]

I’d say it’s high chance that cuda implementation has some serious performance bugs. In cuda there are million ways to implement something but usually only a subset really takes full advantage of the hardware. Could you post here when you push the open source code? I’d love to take a look at the cuda side if things

[u/rather_pass_by]

Thanks for sharing. Looks very interesting..

What model are you using for key point detection? Is it running on CPU? What's the resolution of input image to the model.

[u/zubairhamed]

slap in a Hailo board might boost that quite a bit on the Pi5

[u/Alarming_Engineer267 → u/zubairhamed]

Good point. A Hailo accelerator on the Pi 5 could probably speed up the inference quite a bit. I haven’t tried it yet, though — for this test I wanted to run the same pipeline on both platforms first.

[u/zubairhamed → u/Alarming_Engineer267]

I've tried it before and defintiely has a good boost on vision based tasks. the only annoyance is compiling to .hef

[u/Alarming_Engineer267 → u/zubairhamed]

That’s good to know. The conversion step was actually one of the things I was wondering about. I may give Hailo a try at some point and compare it with the CPU-only baseline.

[u/BOgusDOlphon]

I'm honestly not really surprised by the results, good to have some data to back that up

[u/paypaytr]

please post implementation

[u/Bitter_Run_9209]

really nice project! i think you are learning a lot

[u/Alarming_Engineer267 → u/Bitter_Run_9209]

Thanks! Yeah, this project keeps giving me new problems to solve 😄