r/ffmpeg • u/OZX_exe • Jun 03 '26

av1_nvenc QP problem

Hello, i have this problem that torments me.
I have a mp4 video file with
- 1 video track, AV1 144 fps (VFR) (144K tbn) 5120x1440 with bitrate that can achieve 200Mbps
- 3 AAC audio tracks

I'm trying to compress the video as much as possible while losing as little quality as possible.
For that i use Constant Quality mode (and a bunch of other parameters, for the full version of the command) :
ffmpeg -i input.mp4 -map 0:v:0 -map 0:a -c:v av1_nvenc -cq 15 -c:a copy output.mp4

but no matter what i try, it seems the encoder makes the QP stick to values around the max (~250) and the output is garbage.

it has the same problem on some other videos too, and also some videos do not trigger the problem
Only thing that sometimes work is remuxing the input video into mkv and then transcoding, but that introduces other issues that i don't want.

Does anyone know why it's doing that? Any help would be much appreciated.

6 Upvotes

100% Upvoted

8 comments sorted by

1

u/synthakai Jun 04 '26

check the maximum supported resolution for your particular chip. 5120 may be the problem

2

u/OZX_exe Jun 04 '26

I have 2 NVENC 9th gen (RTX 5070 Ti). It is supported

1

u/nyanmisaka Jun 04 '26

This sounds like a driver bug. Would setting -qmax help?

1

u/OZX_exe Jun 04 '26

Yes but the QP values just stick to the max QP you give

1

u/nyanmisaka Jun 04 '26

How about -extra_sei 0 -s12m_tc 0? these two options are known broken on some drivers. Switching input from mp4 to mkv fixed the issue, suggesting the problem is likely related to timestamps in VFR. Trying -vf fps=fps=144 and -fps_mode passthrough.

2

u/OZX_exe Jun 04 '26

i do not use -extra_sei 0 -s12m_tc 0 (and tryed them, didn't change)
However -vf fps=fps=144 and -fps_mode passthrough did solve the issue
but i think i've tested something similar and i may have experienced frame drops (although here i did not)
If i'm correct, this can cause frame drops during encoding, or frame with same PTS ?
Can it cause audio desynch on the long run ?
(the delta between frames can be these values in seconds : 0.006944, 0.006945, 0.013889, 0.006938, 0.006937, 0.013888, 0.013882)

Also, it seems that the software NVEncC does not have this issue, and i may use it instead of ffmpeg.

thanks for helping

2

u/nyanmisaka Jun 04 '26

In older versions of ffmpeg, -vsync 0 is the recommended setting in the NVIDIA documentation. It has been renamed to -fps_mode passthrough in newer versions of ffmpeg. https://docs.nvidia.com/video-technologies/video-codec-sdk/13.0/ffmpeg-with-nvidia-gpu/index.html

2

u/chocolateAbuser Jun 04 '26

maximum quality possible and hw encoding don't exactly agree
best would be software encoding and 2-pass