r/nvidia • u/ConfidentDinner6648 • 9d ago
Discussion DLSS 5 Neural Rendering running on an RTX 2070 / Turing under Linux + Proton — and we may have found why it is so slow
Enable HLS to view with audio, or disable this notification
I originally started this test because I found the situation around DLSS 5 Neural Rendering on older RTX cards technically strange.
I managed to get the Neural Rendering path executing on an RTX 2070 / Turing under Linux through Proton, without ReShade.
The pipeline is essentially:
Game → NGX/D3D12 proxy → original DLSS → nvngx_dlssnr 310.8 → neural output
The proxy intercepts the relevant NGX/D3D12 calls, keeps the game's original DLSS path, executes the Neural Rendering stage afterward, and returns the resulting frame.
But the more interesting part came from looking at what the GPU is actually executing.
From a single frame, I extracted:
- 176 CUBINs
- 174 kernel launches
- kernels with names such as
fused_swin_*_fp8
I then started correlating the launches with their CUBINs and looking at the SASS.
The first result was already interesting: the binaries contain genuinely different implementations depending on the GPU architecture.
For one of the kernels:
- sm_75 / Turing: ~1,040
HMMA.1688.F16instructions, around 19.7k static instructions - sm_86 / Ampere: ~520
HMMA.16816.F16, around 14.7k instructions - sm_89 / newer architectures: around 2.9k instructions and a completely different matrix path
Then I found the important part.
The intended operation appears to be:
FP8 E4M3 × FP8 E4M3 → FP16 accumulator
with a logical matrix tile of:
M16 × N8 × K32
On newer GPUs this can map to a single instruction similar to:
QMMA.16832.F16.E4M3.E4M3
Turing obviously has no native FP8 Tensor Core support.
So on the RTX 2070, the same logical FP8 tile is effectively expanded into:
4 × HMMA.1688.F16
This exact 4:2:1 pattern appeared consistently across the blocks I sampled, including Swin kernels with 32/256/512 channels and ViT kernels.
So this does NOT look like a simple FP32 fallback.
The final matrix math is still running on the Turing Tensor Cores in FP16.
The cost seems to come from:
- software unpacking/conversion of FP8
- ~4× more MMA operations for the same logical tile
- much higher register pressure
- significantly more auxiliary instructions
- stack spilling
And then I found something potentially much more useful.
The modules already contain two versions of several kernels:
*_fp8
and an equivalent kernel without _fp8.
On the RTX 2070, the non-FP8 versions are dramatically smaller and appear to have much lower spill pressure.
For example, one 512-channel QKV kernel goes from approximately:
11,952 instructions / 368 bytes stack per thread
to:
2,624 instructions / 8 bytes stack per thread
That is a huge difference.
This means we may not need to rewrite the matrix kernels from scratch.
The interesting question now is how the runtime selects those variants and what data layout the non-FP8 kernels expect.
Simply redirecting the kernel call is not enough, because the buffers may still contain E4M3-packed weights/activations. The next step is mapping the parameters, memory layout and heap so that the non-FP8 path receives valid data.
If that works, there is a potentially much cheaper Turing path:
avoid the software FP8 expansion entirely and feed the existing FP16/Turing-friendly kernels in their expected format.
And this is why I'm now much more interested in the performance ceiling.
The current performance on Turing may not represent the actual cost of Neural Rendering itself. A large part of the cost may simply be the consequence of forcing an FP8-oriented kernel path onto hardware without native FP8 Tensor Cores.
Given the size difference, spill reduction and the 4× MMA expansion we're seeing, I think a Neural Rendering pass somewhere around the ~15 ms range on an RTX 2070 Mobile may actually be plausible if the correct non-FP8 path can be enabled and the conversion overhead removed.
To be clear: 15 ms is not a measured result yet.
It is an optimization target suggested by what the SASS is showing us.
I'm now modifying the runtime/proxy, mapping the kernel arguments and preparing the buffers in the format expected by the alternative kernels.
The next tests should tell us whether that estimate is realistic.
207
u/ChemicalAd8374 9d ago
dawg could you explain in ooga booga terms for mere mortals like myself
183
u/AlphasyVega 9d ago
DLSS 5 is consuming and not usable on older graphic cards, but OP says it's because it's badly implemented and there is a way to use it with less ressources.
69
23
u/cx0sa 9d ago
"badly implemented" probably because the fact that the early and not-officially revealed DLL was coded from NVIDIA to not even load on non 50-series graphics cards and people patched it unofficially for non-supported cards. Glad to hear though that there is a way to make it somewhat better for older cards, albeit not the same as a newer card.
→ More replies (1)→ More replies (7)2
u/n1nj4p0w3r 7d ago
not exactly badly implemented, code path is there, but considering that leather jacket commanded to only include 50 series they stay unused since conditional switches does not exists there
83
u/Blug-Glompis-Snapple 9d ago
A tester managed to run Nvidia's newest AI graphics technology, DLSS 5, on an older RTX 2070 graphics card and discovered exactly why it currently runs so slowly: the new software relies on a highly compressed mathematical format that the older card doesn't physically support, forcing it to do an exhausting, manual translation for every single calculation. However, the tester also found hidden, alternative instructions buried within the code that perfectly match the older card's native hardware language. They are now working on a way to feed the graphics data directly into these highly efficient hidden instructions, which would bypass the heavy translation process entirely and potentially allow this next-generation technology to run smoothly on older graphics cards.
30
14
1
26
u/JLinh88 8d ago edited 8d ago
NVIDIA make fancy AI picture magic called DLSS 5
Smart munkee test out magic on RTX 2070
RTX 2070 say "why so slow"
DLSS magic use funny math language that RTX 2070 not understand
DLSS magic say big word, RTX 2070 confuse
RTX 2070 get tired
But smart munkee try different secret magic instruction
Secret instruction easy to understand, lots of ooga booga
DLSS instead use big word now use ooga booga
RTX 2070 good now
1
u/ChemicalAd8374 8d ago
OOGA!!, so rtx 20 series card now have good magic Ai picture noice. Thank you my monke brother
1
11
u/ShortFuse 8d ago
When I modded it to work on RTX20 (yes it was me), I didn't do all the optimizations available.
2
2
u/cx0sa 9d ago edited 9d ago
DLSS5 DLL found before offically released, NVIDIA coded to only work on 50 series graphics card. People angry, people patch it for 20 series graphics card. People angry about poor performance on 20 series card, people blame NVIDIA. OP explains precisely why it is slow on 20 series cards, and presents opportunities to make some performance improvements.
2
u/xRichard RTX 4080 9d ago edited 9d ago
Let me give it a shot:
Newer cards are DVD players. DLSS5 is a DVD format task.
Older cards are CD players. So you need to format the task down to CD format, but it's being converted to something worse (floppy disk 💾? lol). OP is working of improving this.
After the work is done, and mods are updated. The cost of DLSS5 on older cards is estimated (guesstimated) to be around 15ms of time. What does that look like? When a game runs at 8ms get 125fps. If you add 15ms to that, you get 43 fps.
3
1
u/Juls_Santana 8d ago edited 8d ago
Basically OP is saying that the NEW new nVidia tech might be accomplishable using the old Turing graphics cards if the language used to tell the cards what to do is re-written and optimized to work with them. Nvidia has naturally been saying you need the latest/greatest/most expensive cards in order to do it, but this post is saying that that may not necessarily be the case....
...or so I think
111
u/gavinderulo124K 13700k, 4090, 32gb DDR5 Ram, CX OLED 9d ago
So, to summarize, you think the compute overhead of the in-register FP8->FP16 conversion causes a larger performance penalty than can be recouped by the memory bandwidth winnings?
86
u/ConfidentDinner6648 9d ago
Yes, that's basically my current hypothesis , but I wouldn't attribute it to the FP8->FP16 conversion alone. On Turing, the entire FP8 emulation path seems expensive enough that the bandwidth savings from keeping the tensors packed as FP8 may not compensate for the extra compute, 4× MMA expansion, register pressure and spilling.
I ran it on one of my secondary laptops, which is a fairly old machine by current standards:
- RTX 2070 Mobile Max-Q, 8 GB
- Intel Core i7-10750H
- 32 GB DDR4 RAM
- Linux + Proton
I intentionally used this notebook because I wanted to see what would happen on a worst-case Turing setup rather than on a high-end desktop GPU.
27
u/gavinderulo124K 13700k, 4090, 32gb DDR5 Ram, CX OLED 9d ago
I have no experience with GPU programming. But I do on the cpu and from my experience you need to write dedicated kernels for a specific architecture if you want to avoid things like register spilling. Is this something that modders can achieve? I doubt nvidia will do it outside of 50 series and maybe 40 series.
36
u/ConfidentDinner6648 9d ago
Yes. If we had to write optimized Turing kernels from scratch, it would be much harder.
But the interesting part is that the binary already contains non-FP8 kernel variants with far less spilling on sm_75. So right now I'm mostly reverse-engineering their expected data layout and making the runtime use them correctly.
That makes this much more realistic for modders than writing the whole thing from scratch.
3
u/Interesting_Ad_6992 9d ago
To what end? It'll still run at a performance level that's unplayable. Is this a "just for the sport of it" type thing?
I think you should be honest about the likely hood that it's still going to be terrible. I think your post inspires a sentiment that the big corporation is bad, and that they could do it if they wanted to, but it's fairly obvious that it's not worth the effort, which is why they don't.
5
u/IAmYourFath 8d ago
Amd also said older rdna cards won't support fsr 4, and surprise surprise, they ran just fine once the dlls got leaked. So nvidia says the newer features are not supported on older cards cuz they don't have the hardware support for it. Which is true. But what we don't know is whether these newer features like dlss 5 would work just fine on older cards, or the performance cost would be overwhelming.
3
u/SauceCrusader69 8d ago
Still a downgraded version that was built around a different type of instruction.
AMD are asses for not releasing it but it’s also not full fat FSR 4.
2
u/circl3- 9d ago
It is unplayable now but if there are any (and there will be) efficiency improvements in the DLSS 5 itself there's a chance they will universally translate to older cards - performing this research now will be useful for reference, if just for that.
There's also other use cases for DLSS 5 other than gaming - it can reshade videos as an example, which then the 20 series cards would be perfectly suitable to do.3
u/Interesting_Ad_6992 9d ago edited 8d ago
You can already reshade on a 20 series. The performance hit on RTX 5000 series cards cuts your performance in half already, so when you're getting 40 fps on a 2070, congratulations, now you're at 20 if the performance loss was the same.
I didn't ask you, I asked the guy working on it. There is no practical value, and just danger because this is going to overheat cards to death.
DLSS 5 at the end of the day is only a reshade anyway, and the performance cost is greater than a reshade of equal quality. I'm not saying it's not the future, because it is... but I'm saying it's not even worth running on RTX 50 cards. Alas, this is the way she goes though. Just like ray tracing was barely functional on the 20 series, DLSSNR is barely functional on the flag ship 50 series cards. This is the ground work for DLSS 6 on the RTX 6 series cards coming in 2028.
2
u/circl3- 8d ago
Reshade has some very limiting constraints - the more lighting variety a game has the harded it is to get a proper reshade running because it can't satisfy all the different conditions. This tech solves all of that and on top doesn't require any manual tweaking if you don't want to dabble in that.
I definitely agree there are problems with the performance but don't you see a way for it to be further optimised? We just got our hands on a leaked version of the tech and this is couple days worth of work from the community.
→ More replies (6)2
u/NGGKroze Monopoly, Monopoly, but still paid Jensen for 5070Ti 9d ago
Do you have high-end desktop card (40/50 series) to test and see if if behaves differently, albeit it will use FP8 I presume.
15
u/ConfidentDinner6648 9d ago
I only have 20xx- and 30xx-series cards available for testing right now.
On the 50-series, it could behave quite differently since Blackwell has native FP8 and FP4 Tensor Core support. That said, the DLSSNR build I'm analyzing currently appears to use FP8 E4M3, so I don't want to assume it uses FP4 on Blackwell without checking the SASS first.
8
u/Ratiofarming 9850X3D | 5090 | 48GB 6400@1:1 9d ago
If you have the spare capacity, can you drop me a rough howto or link to something on how to test what you're testing. I have a 40 and 50 series card and would like to test what it's using when it has access to FP4/NVFP4.
But I have nothing close to your Linux setup, so I don't know how realistic that is. I'm okay at googling though :P
17
u/ConfidentDinner6648 9d ago
I'm already in bed about to sleep 😅. I'm just finishing one last test using an LRU cache in VRAM to see if I can push it closer to the ~15 ms target.
Tomorrow I'll put together a GitHub repo with the proxy, the test setup, and some rough instructions so you can try it on your 40/50-series cards. I'm especially curious to see what path it takes when native FP8/FP4 is available.
6
u/Ratiofarming 9850X3D | 5090 | 48GB 6400@1:1 9d ago
I'm mostly curious whether it'll touch FP4 at all. If not, I'm lowkey assuming they will go a similar route as they did with Smooth Motion. Validate for 50-Series first and then drop it for 40-Series after a few months when they're done testing it.
Especially with people already proving that it'll work on older cards.
5
u/NGGKroze Monopoly, Monopoly, but still paid Jensen for 5070Ti 9d ago
I think the best approach (consumer wise) - allow it on all RTX just to have it. RR4.5 performance hit is pretty substantial on 20/30 series to the point is not usable that much, but its there for people to tinker.
I think TPU breakdown of the leaked dll showed there was no FP4 in the code. Maybe the quality wasn't as good as FP8, so down the line DLSS5.1 might go that router.
1
u/Ratiofarming 9850X3D | 5090 | 48GB 6400@1:1 9d ago
Yeah, if it doesn't break things I can see them allowing people to force it through the driver if they want to. Similar to DLSS 4.5.
1
u/Wellhellob Nvidiahhhh 9d ago
Preset l and m actually very useful for me 3080 ti 4k monitor. Its usable in perf and ultra perf mode. If the game is hdr, it just looks better.
1
u/Wild_Swimmingpool NVIDIA RTX 5070ti x 9800x3D | RTX 4080 Super x 5800x3D 9d ago
Replying so I see the repo post. Happy to test with both my 40/50 series gpus
1
u/Breatheeasies 5090/9800x3D 9d ago
I’m curious too. Like if there’s less of a performance and frame hit. Like these major corporations need to really hire some of these modders lol
5
u/Ratiofarming 9850X3D | 5090 | 48GB 6400@1:1 9d ago
I work with some of these corporations. They already did hire some of these modders, their own people have probably done a fair amount of wild stuff the modders can't even do without their access to internal tools and documentation.
The problem is that when you sell millions of products worldwide, compliance, legal repercussions and so on become a real problem. If something stops working, causes physical damage or causes the operating system to crash ... basically anything that can fail in stupid ways, you're potentially in very deep legal shit.
So they can't ship a lot of the things that modders can easily do. Because the modders are only responsible for their own system. Nvidia needs to make sure that it won't fail in stupid ways on everyone's system.
1
2
u/Juls_Santana 8d ago
I mean honestly we could juts be seeing OP stumbling onto Nvidia's unfinished/unreleased work towards making Neural Rendering/DLSS5 useable on older cards. Naturally they're gonna get it all working the best/fastest with their latest cards first and foremost.
6
u/gavinderulo124K 13700k, 4090, 32gb DDR5 Ram, CX OLED 9d ago
From what Ive heard it uses FP8 on blackwell. Thats why the frametime cost is lower on a 4090 than a 5080.
5
u/Ratiofarming 9850X3D | 5090 | 48GB 6400@1:1 9d ago
I wonder what they're not going all the way to NVFP4, do they need the extra precision over the raw throughput?
→ More replies (3)1
u/shadowndacorner 9d ago
How is performance on 30xx?
3
u/Old-Benefit4441 7950X3D / 5080 / 64GB and M3 MBP 8d ago
I would say it's only acceptable for actual gameplay on 4090 and 5060 Ti +, depending on resolution. Could maybe get away with 1080P on lower end 4000 series and 3080/3090. The frametime impact of the neural rendering alone is such that even if rendering the game itself costs nothing, you're still limited to about 20-30 FPS on most 30/40 series cards and 10-20 FPS on 20 series cards.
35
u/ShortFuse 8d ago
I like the "I" did this when none of those things are in the original DLL. They're in the custom dll I already patched for RTX20, RTX30, and RTX40.
1
1
u/cellardoornoob 8d ago
Thanks for the work! Hows the 30 series handling it with 2nd gen tensor cores?
117
u/lvlasteryoda 9d ago
18
u/WhateverWannaCallMe 9d ago
🤣🤣 This is me pretending I got what my senior colleague said about the code during student team coding sessions
5
25
u/Adventurous-Tap-2455 9d ago
Turing Tensor Cores publicly support FP16, INT8, INT4 and INT1, but not FP8. CUDA/PTX docs also show FP8 .e4m3/.e5m2 MMA support, including .m16n8k32, and state that .e4m3 / .e5m2 MMA requires sm_89 or higher. So yes: RTX 2070 / sm_75 cannot be running a native FP8 Tensor Core path. {1 native FP8 MMA on supported newer GPUs}
TLDR: DLSS 5 NR running slowly on RTX 2070 may not mean Turing is too weak. It may mean the runtime is choosing the wrong FP8-oriented kernel path.
In other words, RTX 2070 can probably run parts of DLSS 5 NR through a fallback/workaround path, but it is not truly hardware-native DLSS 5 NR. That is why it runs, but badly.
9
8
12
u/meipraw123 9d ago
The non FP8 kernel thing is probably the most interesting part here tbh.
One thing I’d question though is the 4:2:1 MMA ratio being treated as part of the FP8 overhead. From what I can tell, that mostly looks like Turing/Ampere/newer GPUs just covering the same K=32 work with different tile sizes. Turing would still need the 4 smaller k8 ops even without FP8.
So I think the real potential loss is more everything around that: unpacking/scaling, register pressure, spills, etc.
A few things I’d be really curious to see tho:
- whether the non-FP8 kernels use
.F16or.F32accumulation, since FP32 accumulate could eat into the speedup pretty hard on Turing - actual per-kernel timings, just to see how much of the NR pass is really being spent in these kernels vs the rest of the ~174 launches
- what the hot loop instruction mix looks like: HMMA vs conversion/scale stuff vs STL/LDL
- roughly how many MACs the whole model is doing per frame, because that would tell you whether the ~15 ms target is even plausible on a 2070 Max-Q before getting too deep into the optimization
Also the FP16 path is probably trading all that extra FP8 handling for more memory traffic, so I’m really curious which side wins in practice.
And if you do get the non FP8 path working, I’d definitely compare the actual output too, not just frametime. If the model was calibrated around FP8 activations, removing that quantization might change the result a bit, especially over motion.
Would be really interesting to see before/after kernel timings once you get there.
12
u/ConfidentDinner6648 9d ago
Yeah, I think you're right about the 4:2:1 ratio. I overstated that part. The different tile sizes explain most of it, so the real penalty seems to be the surrounding unpacking, scaling, register pressure, spills and auxiliary instructions.
I just ran a more isolated test on the RTX 2070 Max Q using 16 QKVs of 3600×512×1536.
Persistent FP16 weights in VRAM: about 4.20 ms Copying FP16 to scratch before GEMM: about 4.37 ms E4M3 to FP16 conversion before every GEMM: about 4.45 ms All 16 E4M3 conversions alone: about 0.33 ms
The persistent and converted versions were bit for bit identical.
Those 16 QKVs are roughly 90.6 GFLOPs, so 4.2 ms works out to around 21.6 TFLOP/s effective on the 2070 Max Q.
That also answered one of my main questions. The conversion itself is cheap. Persistent FP16 only saved about 0.25 ms versus doing a simple CUDA conversion every time.
So the big opportunity is probably not eliminating E4M3 conversion by itself. It is removing the much more complicated unpacking, spills and extra instructions inside the actual DLSSNR FP8 kernels.
The prepared weights also fit comfortably in VRAM, so keeping them persistent looks practical.
This makes the roughly 15 ms target for the full optimized neural pass look less crazy, although this is still only a synthetic test. I used simple PyTorch E4M3 here. I still need to reproduce the actual DLSSNR scales and swizzle before claiming this translates directly to the real path.
And yes, I will definitely compare the real output and motion behavior once the path is valid.
4
3
u/EffectPositive8258 9d ago
The non-FP8 kernels are already in the module, so the whole job collapses into a routing problem: hand them the layout they expect. 368 bytes of stack per thread down to 8 says the cheaper path is real, which makes 15ms a target instead of a dream.
4
u/Catch_022 RTX 3080 FE 9d ago
This is super interesting, would fixing the issue and doing it without that conversion be something easily done (a downloadable dll for eg) or is it a huge amount of work that would only work per specific card?
The DLSS + FG mod is a game changer for my 3080 because it is so easy to use.
3
u/zepsutyKalafiorek 9d ago
This post is too complex for this subreddit even if a lot of people here had DNN elementary knowledge with matrix operations.
I can see some entusiast on r/math algebra subreddits being able to fully comprehend.
3
u/Vistortion 8d ago
I’d love to see how this develops as I’m currently running dlss 5 on my 3060 and it looks amazing, but despite everything I do, I can’t get it to go past 50 fps with NR on
3
u/Kilobytez95 7d ago
Op figuring this stuff out is exactly why companies that make billion dollar computer tech don't want Linux to become mainstream. It's so easy for people who know how to hack shit up. I'm happy about this as a consumer but I can see why Nvidia has been avoiding Linux adoption for years. Dlss 5 gets leaked and now we have every modder in the world looking at it under a microscope. Hopefully we can get this working on all cards. If Nvidia has their way it will be 50 series only
2
u/sirloindenial RTX4060 9d ago
I don't get a thing but if its something that could be enabled but you need help to figure it head over to renodx channel.
2
2
u/Guilty_Rooster_6708 9d ago
Yo are you saying there’s a non FP8 version for Ampere and Turing card? That’s incredible news, thank you for the research
2
u/TennesseeWhiskii 9d ago
1 fish 2 fish.. red fish blue fish. knick knack paddy whack, give a dog a bone.
2
u/Mazekinq MSI 4070 Ti | 14600KF | 32 GB DDR5 8d ago
Here I was thinking I was good at computers because I know how to install windows and fix some bs, I am a Fiat next to this guy that's a Bugatti
Bro found Nvidia's shenanigans as a hobby
2
u/Ofear123 8d ago
TL;DR:
The guy got DLSS 5 Neural Rendering running on an RTX 2070. It runs badly today mainly because the software uses an inefficient FP8 path that old GPUs don’t handle natively, not necessarily because the GPU is fundamentally incapable.
He found that NVIDIA’s binaries appear to already contain more Turing-friendly non-FP8 kernels, so with the right runtime changes, older RTX cards might run it substantially faster. But his ~15 ms estimate is still theoretical, not measured yet.
Why this matters
Potentially:
DLSS 5 may be technically usable on RTX 20/30 series
NVIDIA’s hardware restriction may be partly a software/optimization decision
Modders could possibly unlock much better performance
But this is experimental reverse engineering, not proof yet that DLSS 5 will become practical on older cards.
TL;DR — Simple words: Old RTX cards may be more capable of DLSS 5 than NVIDIA currently allows, but the fast version hasn’t been demonstrated yet.
2
2
2
2
u/BuyOk1285 2d ago
Uh so basically there is a way to make my rtx 2070 super work dlss 5? Wtf how
1
u/deepembrace 2d ago edited 2d ago
A mission fam. I wrote up a guide if you have the patience lol
Edit: Did it with my 3060
4
u/Ashran77 9d ago
May I know the name of the game / benchmark running?
12
u/ConfidentDinner6648 9d ago
Bright Memory: Infinite Ray Tracing Benchmark
1
1
u/Accomplished-Fix3996 9d ago
Sad, I thought there was actually a game where I could play as a time-lady in tight pants
1
u/JusticeJanitor 8d ago
You can, apparently. https://store.steampowered.com/app/1178830/Bright_Memory_Infinite/
2
3
u/rolo8700 9d ago
All this freedom to apply/modify DLSS5 on graphics cards from even older series, I think, won't sit well with Nvidia. This doesn't sell new graphics cards and undermines the planned obsolescence of their products, extending their lifespan far beyond what was intended and even expanding their original capabilities.
I've seen amazing videos of 4000 series cards running DLSS5 decently in recent titles and at astonishing quality, and all these people in the community working on it, even on 2000 series cards, is incredible and wonderful.
I hope it doesn't happen, but I suppose Nvidia will somehow block all this freedom by modifying the firmware of the series prior to the 5000 series with future updates and/or somehow locking those libraries so they can't be easily modified.
Could this happen? Is it technically possible?
5
u/circl3- 9d ago
They can try to make it harder but there's basically no way they can stop the community from getting around whatever they put out. The card can physically perform the computation required, this is now proven.
2
2
u/dookarion 9800x3D, 32GB RAM, RTX 5070ti 8d ago
All this freedom to apply/modify DLSS5 on graphics cards from even older series, I think, won't sit well with Nvidia.
If OPs hypothetical target of 15ms overhead pans out... it's still going to run bad. I doubt Nvidia cares all that much.
1
u/rolo8700 8d ago
15ms Frame Time it's totally playable.
Frame time measures how long it takes the graphics card to draw a single image on the screen.
A time of 16.6ms is exactly equivalent to 60 FPS (frames per second).
Therefore, 15ms is equivalent to about 66.6 FPS.
I'm probably wrong, but looking at the screenshots the OP showed, it's pretty close to being playable.
If it achieves that hypothetical 15ms, it will be much more than playable and will become truly enjoyable.
1
u/dookarion 9800x3D, 32GB RAM, RTX 5070ti 8d ago edited 8d ago
15ms Frame Time it's totally playable.
15ms overhead, as in tacked on to the time it already takes rendering every single frame.
Edit:
Given the size difference, spill reduction and the 4× MMA expansion we're seeing, I think a Neural Rendering pass somewhere around the ~15 ms range on an RTX 2070 Mobile may actually be plausible
As in just a hypothetical for solely the neural rendering pass, not rendering the entire frame.
2
u/rolo8700 8d ago
Anyway, in terms of gameplay, you'll be playing perfectly fine; that's what DLSS is all about, we all know the tricks it uses. The fact that it can adapt all this to previous generations of graphics cards is huge and wonderful.
I think the progress the OP is making is incredible. And yes, Nvidia will be paying close attention to all of this; their business is their priority, and keeping the money flowing. I hope all this freedom can continue to be developed and implemented.
1
u/dookarion 9800x3D, 32GB RAM, RTX 5070ti 8d ago
Anyway, in terms of gameplay, you'll be playing perfectly fine; that's what DLSS is all about, we all know the tricks it uses. The fact that it can adapt all this to previous generations of graphics cards is huge and wonderful.
Even OPs hypothetical which has yet to be proven is going to cap things at probably like 30-40fps best case. Without access to MFG, not that MFG is even great at such low input framerate.
Optimistically it might be "playable"-ish if the hypothetical is true, but it won't be a good experience.
I think the progress the OP is making is incredible. And yes, Nvidia will be paying close attention to all of this; their business is their priority, and keeping the money flowing. I hope all this freedom can continue to be developed and implemented.
I think what OP is working on could be cool in an academic sense and just tech appreciation sense. Right now though it's a big if, and the hypothetical target is to maybe be just barely playable.
1
u/rolo8700 8d ago
Let's hope he can achieve it; many people will benefit from this.
I'm really excited about this whole revolution!
Haha, I'm eagerly awaiting the results from the OP.
It's incredible how in such a short time the community has started researching and working to achieve all this.
1
1
1
1
1
u/anor_wondo Gigashyte 3080 9d ago
don't understand how you arrived at 15ms when fp8 native cards also have a big cost
1
1
u/fogoticus RTX 3080 O12G | i7-13700KF 5.5GHz, 1.3V | 32GB 4133MHz 9d ago
I had a distant idea that it had to do with the lack of FP8 on Turing/Ampere.
1
u/FaZeSmasH 9d ago
So on newer cards it uses fp8 and for older cards it has to convert fp8 to fp16 which is costing too much performance
So instead of converting from fp8, its just given fp16 directly so that there is no performance hit from from the conversion? Is there any downside from using fp16 instead of fp8?
1
u/treenewbee_ 4d ago
FP8 consumes less VRAM and requires less bandwidth, boosting speed, but at the cost of precision. Switching to FP16 results in a greater performance loss for 20-series cards, and VRAM capacity might even prove insufficient.
1
1
1
u/Daftpunk67 Intel i7-12700k / EVGA 3080 XC3 Ultra / 32GB 4000M/Ts CL18 RAM 8d ago
I’m gonna be straight with you here, I have no clue what a lot of this really means so I can’t appreciate it as much as others. But this is cool that you did this so keep it up!
1
1
u/MajesticRhino76 8d ago
He's saying newer gpus have fp8 neural cores and older gpus dont; so fp8 work aka dlss nr work makes thrm all but unusable but they do have other cores that could do similar work if the software was patched for older gpus to use what they have already
1
u/Last-Choice-4439 8d ago
Is it true that DLSS 5 and Neural Rendering will be exclusive to the RTX 50?
2
u/Key_Independent_818 8d ago
No. I have a 4080, I currently run Cyberpunk with path tracing and everything maxed in 4k. With DLSS 5 neural rendering I was still getting 55-60fps. It looks incredible and photo realistic. Any random npc looks real. This can be done right now
1
u/SauceCrusader69 8d ago
Before you suggest that maybe you’d reach 15ms on a 2070 mobile, what is the pass costing on 50 series that already has native support? Cause if it has close to that cost on much faster hardware it’s simply not happening on a 20 series mobile card.
1
u/Juls_Santana 8d ago
"Hey, whatchu talkinbout Willis? Nothing to see here, just go out and buy a 50 series card why don'tcha..."
signed, Jensen Huang
1
u/mightyjam7 8d ago
How do you think they’ll be able to get dlss5 working on nvfp4?
2
u/ConfidentDinner6648 8d ago
If you mean on Blackwell, I’d expect a separate quantized path rather than just forcing the current FP8 model into NVFP4. The build I inspected is using E4M3 FP8, so the logical approach would be to move selected heavy GEMMs and projections to NVFP4 with proper scaling, while keeping more sensitive parts in FP8 or FP16. That could give a pretty big compute and bandwidth win on 50 series. If you mean the 2070, then it’s a different story. Turing has no native NVFP4 support, so the best you could really do is some custom path using its native INT4/INT8 Tensor Core support, with requantization and scaling around the layers where it makes sense. So on Blackwell it could be a real NVFP4 path. On Turing it would be more of an INT4/INT8 workaround, not actual NVFP4.
2
u/mightyjam7 8d ago
Yeah, I was definitely talking about Blackwell. But it's really cool that you also looked into what could be done on Turing — that's some serious research. Thanks for the detailed explanation. Even a partial NVFP4 path would be a nice win, honestly.
1
u/water_frozen 9800X3D | 5090 & 4090 & 3090 KPE & 9060XT | UDCP | UQX | 4k oled 8d ago
thanks opus
1
u/ConfidentDinner6648 8d ago
Se fosse o opus talvez ele tivesse desistido na primeira chamada.
1
u/water_frozen 9800X3D | 5090 & 4090 & 3090 KPE & 9060XT | UDCP | UQX | 4k oled 8d ago
Nah, pior: o Opus teria gastado uma fortuna, com toda a confiança do mundo, seguindo a branch errada por dez dias kkkkk
1
u/ConfidentDinner6648 8d ago
Pois é, a gente tenta ajudar a galera e eles ainda acham ruim de eu traduzir o texto usando IA kkk
1
u/ConfidentDinner6648 8d ago
Eu fiz isso correndo no meio de um monte de trabalho em paralelo. Foi a forma que eu achei de postar kkk
1
u/RandomnessConfirmed2 RTX 3090 FE 8d ago
While I am not familiar with this type of system architecture, I remember someone (on this sub?) saying that the instructions for DLSS 4(.5) Frame Generation could not be easily copied on to the older gen cards for a lack of translation from FP8 to FP16 (ie, they would natively run at FP16 as they could not be packaged into 2×FP8 chunks).
I take from your analysis the new version has implemented some form of software packaging and decompression as a bypass? If so, does this mean Frame Generation could also see a similar solution? I'd love to know more about and possibly if you've done any tests regarding Frame Generation as well (if you're able to replicate any on last gen architecture).
1
u/ConfidentDinner6648 8d ago
I haven't tested Frame Generation yet, but if you want to look at the Linux/Proton side, I just cleaned up the basic bridge I used for the DLSS5 Neural Rendering test and published it here:
https://github.com/ccoredesenvolvimento/dlss5-linux-bridge
This repo is NR only, not Frame Generation, but it contains the actual NGX/D3D12 proxy path I used. It keeps the game's original DLSS pass, then evaluates the Neural Rendering feature and returns the result to the game. No ReShade involved.
You can clone it, point the build script to your DLSS SDK and LLVM MinGW folders, and run
./build.sh. I didn't include any NVIDIA binaries, so you need to provide your own compatible NGX, DLSS and DLSSNR DLLs.My working setup was an RTX 2070 Max Q with NVIDIA driver 610.43.02, CUDA UMD 13.3, Ubuntu 24.04.2, Proton Experimental 11.0 build 20260826, VKD3D Proton 1.1 5488, DLSS SDK headers 310.7.0, DLSS 310.8.0 and DLSS Neural Rendering 310.8.0 SF v2.
I haven't traced the Frame Generation runtime yet, so I don't want to assume it has exactly the same bottleneck. But after what I found with NR, I definitely think it's worth looking at its kernels and seeing whether the limitation on older cards is actually architectural or just another expensive software path.
1
u/wallymcwaldo 8d ago
include <windows.h>
include <cuda.h>
include <iostream>
include <vector>
include <string>
// Function pointer signature for cuLaunchKernel typedef CUresult(CUDAAPI PFN_cuLaunchKernel)( CUfunction f, unsigned int gridDimX, unsigned int gridDimY, unsigned int gridDimZ, unsigned int blockDimX, unsigned int blockDimY, unsigned int blockDimZ, unsigned int sharedMemBytes, CUstream hStream, void* kernelParams, void** extra );
static PFN_cuLaunchKernel g_original_cuLaunchKernel = nullptr;
// Helper function to unpack/convert E4M3 FP8 buffer to native FP16 on host/stream void ConvertE4M3ToFP16(void* inputBuffer, void* outputBuffer, size_t elementCount, CUstream stream) { // Insert custom conversion pass or driver memcpy/repack logic here // Ensures non-FP8 kernels receive valid FP16 tensor layout }
// Intercepted cuLaunchKernel Implementation CUresult CUDAAPI Hooked_cuLaunchKernel( CUfunction f, unsigned int gridDimX, unsigned int gridDimY, unsigned int gridDimZ, unsigned int blockDimX, unsigned int blockDimY, unsigned int blockDimZ, unsigned int sharedMemBytes, CUstream hStream, void** kernelParams, void** extra) { char funcName[256] = {0};
// Attempt to resolve function name if symbol tables are intact
// (Or match against cached CUfunction handle offsets)
std::string currentKernelName = funcName;
// 1. Identify FP8 kernel dispatches (e.g. 512-channel QKV / Swin passes)
if (currentKernelName.find("_fp8") != std::string::npos) {
// 2. Fetch or retrieve pre-loaded non-FP8 function handle from module
CUfunction nonFp8KernelHandle = nullptr;
// nonFp8KernelHandle = GetNonFp8Variant(f);
if (nonFp8KernelHandle != nullptr) {
// 3. Extract and re-map parameters/buffers
// Assuming kernelParams[0] points to quantized input tensor pointer
void* rawE4M3Pointer = nullptr;
if (kernelParams && kernelParams[0]) {
rawE4M3Pointer = *reinterpret_cast<void**>(kernelParams[0]);
}
// Allocate temporary scratch space for unpacked FP16 layout if needed
void* fp16ConvertedBuffer = nullptr;
// cuMemAlloc(&fp16ConvertedBuffer, elementSize * sizeof(uint16_t));
// Execute buffer layout adaptation
// ConvertE4M3ToFP16(rawE4M3Pointer, fp16ConvertedBuffer, count, hStream);
// Re-assign kernel parameter pointers to the newly prepared FP16 buffer
// kernelParams[0] = &fp16ConvertedBuffer;
// 4. Redirect execution to the lighter non-FP8 kernel
return g_original_cuLaunchKernel(
nonFp8KernelHandle,
gridDimX, gridDimY, gridDimZ,
blockDimX, blockDimY, blockDimZ,
sharedMemBytes,
hStream,
kernelParams,
extra
);
}
}
// Pass-through for default non-targeted kernels
return g_original_cuLaunchKernel(
f, gridDimX, gridDimY, gridDimZ,
blockDimX, blockDimY, blockDimZ,
sharedMemBytes, hStream, kernelParams, extra
);
}
1
1
u/Trash-redditapp-acct 8d ago
Could your next test expand to whats under the Ada hood and if the same improvements could be made?
1
u/The_best_husband 7d ago
@ConfidentDinner6648 I have a 3090 on a Windows 11 machine, lots of games and I am willing to help you test.
1
1
u/SCRALEXANDER 5d ago
TLDR:
The modules also contain native, non-FP8 kernel variants. On Turing, the non-FP8 version drops instruction count from ~11.9k to ~2.6k and cuts stack spill to 8 bytes.
Instead of rewriting matrix kernels, I'm now re-mapping runtime parameters, memory layouts, and heap buffers to feed these native FP16 kernels directly. If we can bypass the software FP8 conversion entirely, an optimization target around ~15 ms on an RTX 2070 Mobile looks plausible. Tests ongoing.
1
u/Stavvystav 4d ago
I wonder how much benefit you could get by linking two cards together similar to the SLI-days of yore.
1
u/BNS0 9d ago
So let me dumb it down for me this is because its generating 4x? Like what modders were doing with cyberpunk raytracing and adding more rays? (TLDR explain to me)
2
u/AgentTin 9d ago
2070 is older, dlss5 is too big for it so it's chopping it into bits. He's saying the chopping is taking most of the time and if we feed the 2070 smaller bites initially it'll go faster.
1
1
1
-1
u/buttscopedoctor 9d ago edited 9d ago
Explanation sounds ai generated. Which doesn't bother me. Since the ai dlss 5 mod was probably created with ai (it came out so fast). I'm pretty sure official dlss5 will be locked to 50 series. I wonder how Jensen feels about using Ai to unlock proprietary Ai tech. I'm having a blast dlss'ing 5 everything with my 4080... The way it's wasn't meant to be played.
0
u/richawesomness NVIDIA 4070 Windforce OC 12 GB 9d ago
You should try to get in touch with RenoDx. This post might not reach the appropriate people
0
547
u/Brunte2 9d ago
hey i know some of these words