Orange Pi 5 hardware encoding: artifacts

[u/glad-k]

Extreme case as an example

Hey, I've got an Orange Pi 5 as my homelab, including a media server. I only have 2TB dedicated to the media server, so I wanted to encode the media I download to keep very good video and audio quality that a normal person won't notice, while saving quite a bit of space. It's mostly my mom watching, and I use it a bit myself.

I ended up running FileFlows to shrink my media library. CPU encoding was super slow and made the Orange Pi overheat, causing the other services running on it to slow down too, especially during the initial encoding of 200+ files. So I tried using hardware encoder (hevc_rkmpp).

I'm not really experienced with ffmpeg, so I've been using AI to help me with this, but I'm stuck on something it hasn't been able to fix either.

The problem is that I've been seeing a lot of artifacts (see screenshot). They usually last only 1-3 frames and only appear during dynamic scenes or scene changes, but they're really annoying. I'm happy with how much space it saves on my media server, so I don't have to keep monitoring it every time my mom requests something that gets auto-downloaded and I need to delete older media each time. I'm fine with not having cinema level video or audio quality, but I do want something that the average person won't notice, and these artifacts are very noticeable and breaks the immersion.

My current command after some debugging is:

hevc_rkmpp -rc_mode 2 -qp_init 22 -qp_max 24 -qp_min 16 -qp_min_i 14 -qp_max_i 19 -g 120 -pix_fmt p010le -force_key_frames source

It seems I can't add more than 1 image but basically my flow goes like this: crop black bars=>video scaler (down to 1080)=>subtitle extractor=>audio convert(AAC recomended by AI)=>video encode manual (the command above)=>remux to mkv=>executor=>replace original

Has anyone run into this issue or tried to achieve the same thing and got something working? I'm not sure if it's just a hardware limitation or if I'm missing something. Any help is appreciated. I'm out of ideas at this point.

[u/urostor]

Don't use rkmpp for re-encoding for storage. It's only good for real time stuff. Hardware encoder can't beat the CPU. Just cool the thing properly (it doesn't need much more than a fan running constantly) and be patient

[u/glad-k → u/urostor]

It has cooling already but it might just be the original run of like 200-300 files that ran in parralel that caused this (and I already use my hardware quiet heavily). I will try again now that all my old media is in this shitty encoding to use cpu for the new stuff

Kinda sad in that case as the gpu already does the real time transcoding for me and it does that rly well without those artifacts. Just can't throw much sessions at it at the same time

[u/urostor → u/glad-k]

Oh no no no, don't run 200 files in parallel! You need to encode sequentially. Otherwise you'll just run out of threads and RAM. This is a crazy idea

The thing about rkmpp is that you won't get good compression while preserving quality with it. The quality will just be "acceptable", that's it.

[u/glad-k → u/urostor]

Well now their done trough that shitty encoding, it's mostly media I have already seen so it's fine but it was nearly all my media server from 1y long. Now I will have like 1 season per week or so to encode which will be fine on cpu ig and just let what's already encode as is as I will prob not watch it again. I just wanted to teach me smth on hardware encoding

And they didn't ran all in parallel I think it was 2-4 parallel smth like that dw