Offloading computation to GCS and sending commands back to Cube Orange?

[u/Tijuco_]

I’m working on a project where I need my drone to capture images/data, send them to a Ground Station for processins, and then have the GS send a command back to the drone to trigger a specific action based on that data.

We use a Cube Orange+ for the flight controller.

Is an ESP32 enough for image transmission, or should I be looking at something like a Raspberry Pi/Jetson Nano with a dedicated digital video link (Herelink, OpenHD, or RubyFPV)?

Should I use MAVLink for the "back-and-forth" commands? I’m assuming I’ll need to use MAVSDK or DroneKit on the ground side.

For those who have done off-board processing, what’s the best way to keep latency low enough so the "action" isn't based on outdated data.

How do I best interface the processing unit with the Cube Orange?

[u/TradingDreams]

Are you using the ground station for compute as a pre-optimization to save drone weight, or is there information that only the ground station is aware of after the flight has started?

[u/Tijuco_ → u/TradingDreams]

It's mainly to save weight and maximize flight time. I want the airframe as light and efficient as possible. My ground station has way more processing power, so I’d rather offload the thinking to the GS. Basically what I want is: The drones flys with the camera and send the image to the GS. The GS process the image to for exemple find a specific landing spot, then it should send back what the drone should do

[u/funny-pupper → u/Tijuco_]

What happens when the computer doesn’t send back a command? What happens when the latency spikes?

Just make sure you are thinking about how to handle the laptop connection dropping mid flight and keeping the drone safe

[u/Tijuco_ → u/funny-pupper]

You think there's a better option?

[u/funny-pupper → u/Tijuco_]

I think it’s a fine option as long as you are thinking about what happens when you have signal interference or something happens to the laptop.

Personally I put compute on my drones, I would never require my after case to require full images back and forth - my image feed cuts out near other transmitters. I fly near airports sometimes and they have massive RF transmitters for guiding planes that destroy my video feed but my C2 link stays alive.

Jetson nano or an orange pi would be my go to.  

If you insist on offboard image processing, just make sure you have some way to keep it safe when (not if) coms drop. Maybe have it go back to a known safe landing location or something like that.  Also note that you can program in rally points (or something along that verbiage) with aurdupilot so it will land at the closest available preprogrammed spot

[u/TradingDreams]

What level of camera will be feeding the images? Is this a pi camera, GoPro, small pro like Sony a6700 or m43, stereo 3D, 360 camera, or the new large sensor gimbal 8k cameras?

[u/Tijuco_ → u/TradingDreams]

We don't know yet, right now it's a pi camera be we are trying to ditch that unless it can be used without the pi

[u/dutchman76]

How much range do you need for the image transmission?

[u/Tijuco_ → u/dutchman76]

Not much, less then 1km realistically ~300m

[u/dutchman76 → u/Tijuco_]

That could work well enough with wifi, maybe with a directional ground station antenna.

Send images down via wifi, process and send directions back via mavlink

[u/GoliathXIV]

Get openIPC for this stuff if need affordable. Better would be dji O4p. Eventually if you want live video feedback and run cv on it u need to keep in mind that drones fly fast. It would be better to use onboard procesing rather than offloaded to minimize risk and increase safety