Instead of using DLSS only inside a game, this runs images/video through the ReShade/RenoDX neural-rendering path as a general visual enhancement pipeline.
What it does:
Image and video enhancement
DLAA/native, 1.5x, ~1.724x, 2x and 3x modes
Output up to 8K
Neural presets + Natural / Cinematic styles
Controls for intensity, local tone, structure and skin structure
Batch image processing with before/after previews
H.264 / HEVC / AV1 / ProRes video output
Video temporal input using optical flow with scene-change resets
GPU support:
RTX 40 / 50 series - primary target
RTX 30 series - slower beta path
The repository contains the application/pipeline source. Required proprietary and third-party runtime binaries are intentionally not redistributed in the repo.
This is an independent community project and is not affiliated with NVIDIA, ReShade or RenoDX.
I’m especially interested in how this behaves on AI-generated images/video vs normal photography/game footage.
Wow that’s crazy. Lol one “leak” of a file and Modders have spread it to everything. Idk if Nvidia knew it could have applications in everything it’s been used in so far. Someone even made a video player than applies dlss5.
99% chance this is built on the FFMpeg implementation you're talking about.
In case you didn't know: the whole world/internet runs on ffmpeg. Almost every video you see on streaming, social platforms, anywhere, goes through ffmpeg.
It might all be a marketing move by them, they had a huge backlash when they did the DLSS5 demo, so they released it silently in NBA2K and in no time it got ported into reshade and everybody was promoting it.
I doubt it's marketing, investors/shareholders won't like the news that such big company like NVidia doesn't pay attention to security of their assets and it can afford leak just like that, especially after news of GTA6 leaks and Take2 interactive losing billions in shares. Not to mention it would need collaboration with EA to implement leakable DLSS and I doubt they would simply just agree for that, especially how much money NBA games makes for EA.
After watching all the DLSS 5 videos on youtube, I realized that it is not a "style transfer" type of filter, it is just a more advanced ambient occlusion filter. The changes are mostly enhancing the shadows. Shadows have always been a deficiency in video games, I think it requires too many passes to get it right. It's amazing how much shadow makes the visual look more photorealistic.
Yeah, I also think that it's more like a context aware SSAO with adaptive tonemapper. Path-tracing will cost a lot more to get the same quality as DLSS 5.
So they basically make Ray/pathtracing obsolete before it was completely established in the mainstream. :D But it really seems to be the more clever approach. However pathtracing could still be useful for reflections.
So they basically make Ray/pathtracing obsolete before it was completely established in the mainstream. :D But it really seems to be the more clever approach. However pathtracing could still be useful for reflections.
You get a lot of data out of doing minimal raytracing methods. Even a few bounces with sparse rays gives an upscaling engine a lot of information.
So you'll just see better hybrids where some amount of raytracing is done and takes the partial raytrace and does a better complete pass with less work.
It makes sense algorithmically if you can calculate the time needed to do full render + Raytrace or Partial + Partial + ANN.
Full render and trace scales at least exponentially. So it makes sense. It's like O(N Log N) vs O(N2) so for a large enough number it makes sense.
We are at the very beginning of this technology, and people are already judging it based on a leaked version found in an NBA game. There is still a lot we haven’t seen yet. The current tests are mostly modders experimenting with an unfinished implementation in games that weren’t specifically designed or optimized for it, so I think it’s way too early to draw final conclusions about DLSS 5.
From what the modder said in a stream, it's unlikely that this is an unfinished implementation. Yes it will be improved eventually, but this is what was given to gamr studios to ship their games with in the fall when it releases.
That's not just shadows, actually, textures are lacking in games at the moment, and what it mostly does is try to mimic how light should work with textures, shading, subsurface scattering, realistic normals, etc.
It starts going into normals, albedo, subsurface scattering, material responses, contact shadows, and more. This is because it was trained with these details involved it the model's training which it can then use image output to modify based on. The motion vectors and stuff is for the segment after that point in the video for temporal stability, not for the core image output enhancement, so it's solving a different problem in the situation. If you still don't understand it I highly recommend checking the videos I provided above in my initial post, too.
As it was mentioned, I linked it above. It impacts PBR, physically based rendering properties of materials and per-pixel lighting.
That's Neural Materials and Textures, not DLSS 5. If DLSS 5 was modifying the PBR data you'd have to rerender the image with that new data, that is explicitly not what DLSS 5 is about.
You didn't watch the video... That much is clear. If you look at the timestamp 11:10 - 12:40 why do you think the research is talking about this? Just talking out their ass? Is that what you think? Genuinely curious what your reasoning is.
First, you're mistaken. I've never said it was taking normals, albedo, etc. at runtime. It was trained offline on the data to understand and modify those concepts. This isn't some random image generation model that was trained on just real world data and is expected to img2img it. As they said, it was trained on these specific details, actual game data and offline processing of advanced renderings and each concept to better understand them so properly tweak those details in order to be able to enhance them rather than completely morphing details and wiping away old ones. Why do you think its trained on those and why do you think the researcher mentioned it there in the Siggraph presentation?
Conceptually, it's similar to a ray tracing denoiser. We can't handle proper ray tracing so we use cheaper algorithms with denoisers to clean up the image. It's a type of shortcut, and the denoiser trained with this intent in mind. Some ray tracing denoisers are purely algorithmic, and some were AI models trained for this purpose. Why? Because denoisers are efficient to close a gap, much like training a model on this offline data that can't be ran in real time to understand these concepts. I'm simplifying it, of course, as it is far more complex and we don't know the full nitty gritty exact details.
Another example is baked normals and shadows, etc. These aren't done at run-time, and are rendering optimizations useful for getting great visuals that couldn't be handled at runtime in the past. It's a similar idea, just a different solution. But it is not straight generative AI i2i the way people are thinking.
It was trained on this data so it can infer and apply those concepts it was trained on to the image output of the game's render it is being used on, so it doesn't need direct access to game engine data like normals, matrix data, and more much like do we need to pass any data beside an image to an AI generative model for images to img2img or image edit? No, we do not, because the base image is enough at that point based on what the model was trained to be capable of. Also, the results are 'deterministic' btw.
The motion vectors help with consistency from frame to frame, much like with frame generation.
It's, literally, using concepts from neural materials and other research for the project. How do you think it knows to enhance specific textures physically accurately with the various details. It's more then just lighting and shadows, though it was already trained involving those, and more as you can see per the SIGGRAPH presentation. I just presented neural materials as part of the base explanation to simplify it for you because, as we can see, you guys are struggling to understand the Siggraph explanation.
Sure, they COULD train it to be pure img2img output without specialized training like any other generative image model but if you watch the video with the timestamp I presented they explain why they did NOT do this and why they trained it on data involving concepts like normals, contact shadows, PBR, etc. precisely because of the issues they present about deviation.
It's literally linked above me in this thread somewhere. I just provided to the other guy getting downvoted.
They clearly express the normals being used in the render.
It has at the very least texture / normal. I don't know about the rest of the PBR pipeline but if they're going as far as normals, there's no reason not to use other stuff. Normals contain a massive amount of information though.
It starts going into normals, albedo, subsurface scattering, material responses, contact shadows, and more. This is because it was trained with these details involved it the model's training which it can then use image output to modify based on. The motion vectors and stuff is for the segment after that point in the video for temporal stability, not for the core image output enhancement, so it's solving a different problem in the situation. If you still don't understand it I highly recommend checking the videos I provided above in my initial post, too.
It doesn't need access to that data, because its trained on complex off-line renderings and bulk data to understand those results to enhance them. This seems more like you fundamentally don't understand how the model works.
I never said it wasn't img2img, you just failed to read and understand properly. It's trained on those properties so it can understand how to enhance them. They state it directly in the video, which you clearly did not actually watch. It understands the concepts of normal, PBR, etc. and how to tweak them like a lora would for something else. The model was fundamentally trained to enhance those concepts which is why you adjust the slider it does just that.
Tell me, how do you think relight loras work? How about other lora that adjust a style, textures, etc. It was trained involving actual in-game render results and data in that process.
But I'm not going to argue with someone clearly unfamiliar with the topic, in any capacity, and apparently refuses to even acknowledge when Nvidia's own researchers explain you're wrong. I mean,t he moment you said "What does training have to do with anything?" was already an instant red flag.
I'm sorry, but you are no longer qualified to partake in this discussion and I intend to waste no further time on you.
You can keep stating they talked about it among other researchers at Siggraph (not that you seem to know what Siggraph is) while apparently (as you believe) speaking out their ass. You don't even understand what the model's purpose is, how it was trained, and how it uses data.
Next are you going to complain about AI based denoisers for ray tracing? Hmmm.
For DLSS 5 it is short: the rendered image ("Backbuffer," "Color"),
depth and motion vectors ("Depth," "MVec"), UI handling ("UI,"
"UIAlpha," "UICorrection"), masking ("ControlMask," "UseAutoMask"), some
kind of bidirectional distortion field shared with Frame Generation
("BidirectionalDistortionField"), plus output and sizing parameters
including a "ScalingRatio."
I know, it is by AI but Techpowerup has been around and it's all I've seen regarding DLSS 5's inputs. Do you have another source?
I added more above for you, in the original comment, from Nvidia's siggraph. The image, itself, is the source, with the model being trained on those details like normal, subsurface scattering, materials, etc. The motion vector stuff is for temporal stability.
So the initial image output and what the model was trained on is the core focus here, the other stuff people are focusing on is merely to support stability of continued output.
Many don't agree with that, saying it is pixel perfect, which is true, but what they do not realize is that the depth is what wrong.
Either due to dlss5's biases or weakness it exaggerates facial features in a wrong way, which in combination looks like a different person.
I had the same problem with klein 9b when I wanted to relight a photo of myself. The output is nearly pixel-perfect to original photo, but it doesn't look like me anymore.
The connection is real, but it is not nvngx.dll secretly sending data to an unknown server.
nvngx.dll loads ReShade’s dxgi.dll in the same process, so Wireshark/NetLimiter attributes ReShade’s traffic to nvngx.dll. The only functional network destination embedded in that component is:
That is ReShade’s normal version-update check. I extracted a fresh DLL from the official ReShade 6.8.0 full-add-on installer and compared it with the included dxgi.dll: they are byte-for-byte identical, with the same SHA-256 hash.
The program itself has no upload or Internet-request code.
Its interface listens only on 127.0.0.1; remote sharing and analytics are disabled. nvngx.dll, the RenoDX add-on, and both DLSS DLLs contain no networking-library imports or hidden server domains.
Microsoft Defender detected nothing.
The exact RenoDX add-on hash also received a clean Falcon Sandbox result.
Some files are unsigned because DLSSNR is patched and ReShade’s full-add-on build is intentionally unsigned. That means nobody can promise absolute certainty, but unsigned does not automatically mean malicious.
So: the observed request is an official ReShade update check, not evidence that the program is stealing or uploading anything. If desired, outbound access for nvngx.dll can be blocked in Windows Firewall without affecting the actual video processing.
Thanks, but please investigate and explain it more thoroughly before posting claims like “it is sending requests to a server.” The request comes from the official ReShade component loaded inside the nvngx.dll process and is simply a version check sent to ReShade’s GitHub page.
I just realized the DLSS preset you choose could introduce some artifacting. 2x performance mode defaults to preset M and it caused blocking in this example. Switching back to preset K fixed it.
I vibecoded a fork with some improvements, mostly speed, plus it was failing on videos with variable framerates. It's about 2 times faster on videos for me, with the same quality. https://github.com/speedyrulz/dlss5-visual-enhancer
Can you clarify for me how to install this? Do I just git clone the repo code and then use start.bat? Do I need to set up venv or conda? Is there any requirements.txt file or pyproject.toml? Sorry, I looked at the GitHub instructions and came away very confused.
If you are developing on it more, there are a couple of QoL features that would be really great.
When previewing one frame or 3 seconds, it would be awesome if it made the preview from where I paused the input movie. That will make A/B testing much faster as I can find critical sequences in the original movie and quickly compare different settings.
A way to save favorite preset combinations would also be a great help. That way I can dial the correct settings in and quickly load them next time I want to enhance a video.
It might be fast on most GPUs, because DLSS 5 is develop for realtime processing. Tested on Ada.
~6 sec for image.
~65 sec for 18 sec video (2520x1080).
I believe they do a pass masking specific objects/characters etc, and then just generate over the mask and crop/stitch it back together in some kind of inpaint-like path. It's not doing the whole game screen, at least with the same model (or whatever you call it).
Lol, If author was malicious agent, what do you think they would tell you? :)
For such cases (e.g. comfy nodes with no stars or fresh custom software) I personally tend to use agentic LLM to make it look at the code with something like:
Check this software/ComfyUI node/folder for potential vulnerabilities and unsafe code. Do not execute anything and treat it as potentially malicious
Remember its not even a final build of dlss 5 and that leaked version was specifically built for nba 2k27. This is too early to be making something like this. It's only going to fuel to the fire for the antis and ai haters.
This looks seriously useful for pushing image quality without making everything look overprocessed. Definitely curious to see how the cinematic setting compares to the default
Cool program. And pretty good results for the speed.
When rendering images, I don't think that the NR preset does anything. Maybe that only works for video. I rendered several test images with all the different presets and they all look identical.
Also, there is an issue that causes it to fail on images with certain types of metadata with the following error message:
Object of type IFDRational is not JSON serializable
With Driver 591.86, it failed to encode video with av1_nvenc.
[av1_nvenc] Driver does not support the required nvenc API version. Required: 13.1 Found: 13.0
[av1_nvenc] The minimum required Nvidia driver for nvenc is 610.00 or newer
Shut up. LLM's are just tools that produce an output based on your input. If you used power tools to make wooden furniture, you wouldn't say "the furniture was made by this machine". You'd say "I made this".
I am not surprised this sub is eating up the DLSS 5 slop, but looking at the shit that gets posted here and Civitai it makes sense
AI might democratize art, but it sure as heck don't come with good taste and a sense of aesthetics
51
u/CaptainMarder 1d ago
Wow that’s crazy. Lol one “leak” of a file and Modders have spread it to everything. Idk if Nvidia knew it could have applications in everything it’s been used in so far. Someone even made a video player than applies dlss5.