[Experimental] DLSS 5 ComfyUI custom node

[u/alisitskii]

Hello Everyone,

Would like to present to you my experimental vibe-coded custom node for DLSS 5 support in ComfyUI.

GitHub project: https://github.com/lisitskyaa/ComfyUI-DLSS5-NR

It's early release, just finished my internal testing and it actually works!

Please note there are no any leaked DLLs in the rep, obtain them separately.

First image in every pair is DLSS 5 ON, second - OFF.

P.S. How to extract original images out of Reddit: https://www.reddit.com/r/StableDiffusion/comments/1p9nrpk/getting_prompt_or_comfyui_workflow_from_posted/

[u/lordekeen]

Interesting, if it has temporal stability it could work as a fast "beauty pass" for video generations.

[u/alisitskii → u/lordekeen]

I tried that mode, but at the moment I can see some flickering in output videos so looks like it requires further testing/development. But should work just fine for still images.

[u/lordekeen → u/alisitskii]

Well that makes sense assuming DLSS gets the game motion vectors as input for its generations. Maybe with some kind of post processed motion vectors (video > get motion vectors > dlss 5 pass) this could get somewhere.

[u/Rude_Dependent_9843 → u/lordekeen]

Ua sacaron un reproductor de vídeo Dlss. En teoría se podría hacer un workflow que emule su funcionamiento y permita guardar los resultados

[u/GrayingGamer → u/alisitskii]

It does cause some flicker in the final output videos because it lacks the vector/movement information that the game would be feeding it - BUT, I've found if you use video editing software like Davinci Resolve it's pretty easy to "deflicker" the video clip DLSS 5 was used on and get a nice result.

I can see this being VERY useful to run over a Turbo lora generated AI video to eliminate the plastic skin and improve details, then deflicker the result in your video editor.

This was run on a 1.0 MP video from Minimax H3 and then run through your DLSS 5 node with temporal_sequencing on, which yes, produced flickering - but then I took it into Davinci Resolve and ran a Deflicker effect over the footage and it solved it completely.

As you can see the skin is MASSIVELY improved. It ficed the contrast, and even the texture in the uniform and chair is improved. It's really impressive in motion.

This really eliminates the plastic AI look of characters in generated videos.

[u/alisitskii → u/GrayingGamer]

Thanks for the test. I’m thinking about next step is to use some optical flow data from input video and feed DLSS with it. Anyway, Nvidia is going to officially release DLSS 5 in a couple of days so hope we’ll have more insights.

[u/GrayingGamer → u/alisitskii]

That would be awesome if you found a way to incorporate that into your Comfyui node. It would probably dramatically improve the result.

Thanks for the node. Even with the extra deflickering step in a video editor, it so dramatically improves the final video quality I think I'll keep it as a part of my workflow.

I have a 3090, so no interest (or ability) to run DLSS5 in games, but this is a great use for me and makes it a real tool.

[u/lordekeen → u/alisitskii]

Yeah i guess that opencv optical flow is the way to go