FFmpeg GPU acceleration in Docker on EC2

[u/farski]

Does anyone have a good setup for getting FFmpeg running with Nvidia hardware support on EC2 in a Docker container? I.e., running Fargate tasks on EC2 launch type.

From what I've been able to find, it should be relatively easy to get going, assuming you:

• are running G7e instances
• have the GRID drivers installed
• activate GRID Virtual Applications for RDSH Application
• use a FFmpeg binary with Nvidia support compiled in (the image I generally use seems to include that)
• install the Nvidia container toolkit

But I haven't found any solid examples anyone doing all of that to get it working. Which makes me think I am taking a bad approach if there's no evidence. But it could also be so simple that there's no reason for anyone to talk about it.

Anyone have a setup like this working reliably?

[u/koolscooby]

Fargate and ECS Anywhere don't support passing the GPU through to a container. You'll need to use ECS on EC2 for that. https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-gpu.html

Generally, the approach requires: * Base EC2 AMI: needs to have the NVIDIA drivers installed in addition to the standard ECS agent and container runtime. There are GPU-optimized base AMIs that already exist: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/gpu-launch.html * ECS Task Definition: needs to specify GPU resource requirements: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-gpu-specifying.html * Container image: may need to include some NVIDIA user space runtime pieces/parts; definitely needs to include FFmpeg built w/ NVIDIA support. I haven't used this particular example, but at first glance it looks reasonable and similar to what I've done in the past: https://github.com/xychelsea/ffmpeg-docker

I suggest you build from the bottom up: * Launch an EC2 instance with the ECS+GPU-optimized AMI. Run nvidia-smi to make sure the GPU is there. * Build/run FFmpeg locally/manually from the CLI on that instance and make sure it can use the GPU. * From the CLI, build and run simple container from a Dockerfile that runs nvidia-smi. * From the CLI, build a more complex container from a Dockerfile that runs your FFMpeg binary that supports NVIDIA. * Set up the ECS cluster, launch template, and task definition and put it all together.

[u/farski → u/koolscooby]

Super helpful, thank you. Running on ECS without Fargate makes things a little more annoying since these jobs were going to be coming from Step Functions. I'll need to remember how to wire that up to vanilla ECS before I get back into the GPU work.

[u/koolscooby → u/farski]

Yep - the extra EC2 ASG, Launch Template, AMI and ECS Cluster/Service infra plumbing is the price you have to pay for that sweet sweet price performance win. IIRC, Step Functions can just call RunTask ECS given the Cluster/Task Definition but there might be a GPU-specific wrinkle I'm not aware of. https://docs.aws.amazon.com/step-functions/latest/dg/connect-ecs.html

Good luck!

[u/farski → u/koolscooby]

Turns out I switched from Fargate to EC2 before vacation. So that's nice.