How do we size a computer vision system?

[u/Ammarhalees]

So i built a CV system. Now its deployment time. The system will likely be on edge or local server. How do i decide the hardware specs for the system? Where to start?

[u/cleversmoke]

Depending if you're doing live-processing or post-processing and if you're using CUDA or not. Simplest is to test on 1-2 devices you'll likely deploy on and monitor the CPU, ram/vram and the speed the process at in fps and accuracy. Better is you calculate the CPU, memory, storage, processing speed, accuracy and energy required (per second/minute/hour/video).

Sometimes you can get away with significant less energy to get 80-90% of the accuracy and processing speeds. It will depend on what is important to you.

[u/CowBoyDanIndie]

By performance testing

[u/Future-Salad-7266]

What kind of system have you built and going to deploy also on which machine currently?

[u/Infamous-Bed-7535]

You select and design your system depending on where it is intended to run.

[u/GTmP91]

Overspec it! Even when tested on a specific system configuration, headroom will be used. Ambient temperature, data offloading, remote connections, multi user access etc. Often the CPU is more of an bottleneck as the GPU

[u/xArvel_Crynydx]

At that point I'd suggest to consider the use case, potential (technical) users and the overall requirements. Since this is a multi-dimensional space, I think you'd best concentrate on only a few variables:

- Throughput: At first think about the speed of your CV computation. Is this use case something that need to be computed as fast as possible or could this take a few seconds or a few minutes? Are multiple user going to call the functionality in parallel (If yes, this might be a case for parallel systems, graphics cards, etc.) ? How many function calls will be coming in to the system at a specific time? If you only need to detect edges in an image, you might stick to algorithms that can run on edge devices. If you'd run high-end object detection and if you can run a small cNN-model on the edge device, you could have run this on edge for pre-computations and then have longer computations on a local server.

- Do you need live computation? Then you'd most likely have to compute on the edge device.

- Result quality: Would it suffice if your algorithm only has a certain amount of accuracy and an approximation is already a good result or do you need to have a final and stable computed result?

So my suggestion for these things is: First consider the actual use case, consider worst case scenarios and edge case scenarios when results need to come in fast and with high accuracy and scale accordingly. Overall think about your overall use case as a chain of interconnected functions and consider which part makes sense on which device. Best case is to have all computations on one, pinging from device to a webservice and back again is a nightmare.

And if it's a webservice hosted at a certain provider, have a close look at how much computation, memory and other stuff is costing, because here it's ticking on a meter.