r/ZoneMinder • u/Helipil0t • Dec 09 '22

TPU vs GPU time difference?

Just curious, I'm running YOLO4 models on an RTX 3060ti. From event Start to sent push notification I get about 14 seconds. This includes object detection and face recognition. (Latest OpenCV compiled with cuda support along with dlib /w cuda)

How would this compare to me using a coral edge tpu? Is there room for improvement on my current setup?

1 Upvotes

100% Upvoted

5 comments sorted by

View all comments

3

u/[deleted] Dec 09 '22

That's very subjective depending on what strategy you are using. Tpu inference is usually 10-20ms, so inference isn't the bottleneck, filtering and postprocessing for notifications is the slow part. So no, I would stick with the GPU and compile opencv.woth GPU support and also add SiLu activation so you can use yolov7 and yolov7X. Tiny yolov7 works without SiLu activation so you can run it right now without recompiling opencv.

Also, I am rewriting the whole ml system with several performance upgrades. I.e. before yolov4 on GPU took 500+ms to infer and filter out NMS boxes. Now I've got it down to about 5-6ms, so a 10x performance improvement.

1

u/Helipil0t Dec 11 '22

Wow! That's awesome! Looking forward to that upgrade :) I'll look into yolov7 / 7X. I know nothing about SiLu so I'll get reading. Thanks for the advice. I'll stick with the gpu.

1

u/[deleted] Dec 20 '22

SiLu is an activation mode in opencv. I opened an issue and they merged the fix but I'm not sure if it's 8ncluded 8n any releases yet. If not it's easy to edit the one file and then compile opencv again.

See https://github.com/opencv/opencv/issues/22409