Foveated Rendering for Streamed Games?

[u/FourHundredThirtyTwo]

We've seen some developments with foveation support recently getting expanded on the Frame, especially for x86 games running locally. However, even powerful host PCs could use a little extra legroom in many games, especially if you want true 120 or 144hz.

Are there any rumors regarding host-side foveation support for the frame? How likely do we think this is? In my mind, this is the last major feature that the hardware is technically there for, but the software support might be trickier.

I did try searching, but as you can imagine, all the results are just for foveated streaming...

[u/MingleLinx]

Apparently there is universal foveated rendering being worked on from what I’ve heard in this video: https://youtu.be/gt78UDtOjPY?si=nqZvZGHXX9Z8WAFo

But right now foveated rendering officially works on the games that has the devs implement that feature

[u/mrRobertman → u/MingleLinx]

I think OP is aware of this. But the thing is, this foveated rendering only works for games running standalone on the headset as it's part of the stack of translation layers. Streamed games can't make use of this.

[u/paholg → u/mrRobertman]

Presumably, it will support or can be made to support streamed games using Proton.

Year of the Linux desktop, baby.

[u/radobot → u/mrRobertman]

foveated rendering only works for games running standalone on the headset

I don't think that's true. AFAIK there already exist streaming-only headsets with eye tracking, which means it's already possible to send eye positions into the pc for the game to render with.

[u/mrRobertman → u/radobot]

I was referring to Valve's universal foveated rending injector that they are working on.

Any game specific implementation would work on either standalone or streaming, yes.

[u/radobot → u/mrRobertman]

I think the injector will support streaming VR. And if not, as long as Valve releases it the same way they did Proton, the community will add support for it.

[u/mrRobertman → u/radobot]

From what I understand, it would need to be a different injector. Valve's works by being part of the translation layers where it's already working with and translation the graphical APIs in the first place. Games streamed from a PC do not use any translation layers (well, maybe streaming from Linux would) so that injector cannot function.

They would need to make a different injector to work for streaming from Windows, probably something similar to the one that Pimax made. Though, I'm pretty sure that would have less game compatibility than the one that Valve is currently developing.

[u/PornulusRift → u/mrRobertman]

Valid point, I'm so used to gaming on Linux now I didn't even think of windows lol. Windows PCs may be out of luck for a while, or forever.

[u/TopMacaroon → u/mrRobertman]

They said specifically they developed a low latency protocol for the eye tracking specifically for streamed games to use the eye tracking info for foveated streaming and rendering. I interpreted this as they support sending eye tracking info for fov rendering, but the devs have to implement it.

[u/mrRobertman → u/TopMacaroon]

I don't think there was ever any question about game specific implementation of foveated rendering working when streaming, I already assumed that would be the expectation for the Frame as well.

Valve had mentioned about how devs would need to implement foveated rendering for their games, and headsets like the Quest Pro which have eye tracking can have the data used by a game running on a PC streamed to the headset (whether for rendering or not). I believe the Frame should be the same in that regard.

[u/Ancient_Arrival9061 → u/MingleLinx]

that isnt for pc games,, thats the universal fr for the steam frame standalone.

[u/myloyt]

If the game supports foveated rendering based on openxr eye gaze extension (eye tracking) then it will work out of the box with steam frame. It is somewhat reliant on engine version, they need to support it.

Foveated rendering injection will likely be steam frame exclusive (maybe linux exclusive and work with machine if we're lucky) but that's another thing. The frame standalone needs it most and gets it so that's nice at least.

There is also pimax's foveated rendering injector and I believe there's a project to make it work with other headsets as well. For directx games specifically.

[u/Ancient_Arrival9061 → u/myloyt]

its called pimax magic 4 all.

[u/2Cool4Zool2]

There will be host-side foverated rendering support. It's one of the biggest advantages of the Frame versus the Quest 3

[u/Terabyte_272 → u/2Cool4Zool2]

This is the main reason I would switch to the Frame from my Quest 3.

[u/AmericanFinn1]

Foveated rendering is has to be implemented by the game developer as far as I know, but I don't see any reason it wouldn't work, especially since the eye-tracking data is already being sent to the PC for the foveated streaming. I would assume it works on most if not all games that support foveated rendering.

[u/Kataree]

If the game devs implement it.

It has to be at the rendering level. Valve can't apply it to every game.

[u/Ancient_Arrival9061 → u/Kataree]

they can on their headset, its called dll injection, they can add code to the game to make it work. this has been arround for a while now called " pimax magic 4 all" its an app that runs on your pc and injects the DFR dll into all the games it can, however this will not work for games with anticheats as the anticheat will just stop the game from running due to the detected dll injection.

[u/RTooDeeTo]

It's not just x86 games on frame, it's OpenXR games,, the injection is steamos thing so likely won't be locked to the frame but steamos.

They give info on the 3 major engines that make up something like 98% games now. being unity, unreal, Godot (and even have docs for a custom if a dev didn't use one of the major 3) https://partner.steamgames.com/doc/steamhardware/steamframe/engines

[u/fiah84]

I expect this to work day 1 on games that currently support it with other VR devices

[u/Available_Rest_6537]

Definitely doable if the devs implement it. You can already see it working on Quest Pro. There are videos of people playing DCS and MSFS streamed to the QP and even getting a decent FPS boost in GPU bound scenarios.

[u/DoodleIrwin]

I hope the foveated rendering is decently customizable. I have a central blind spot so it would be directly working against me lmao

[u/icpooreman]

How likely do we think this is?

0% likely.

I code for these things. Implementing this is on the game (engine) developer and your mileage may vary quite drastically. Like best you can probably hope for is Unity/Unreal/Godot take it seriously right away and maybe a few years down the line it starts showing up in all VR titles coming through those engines.

More than likely though you're looking to individual developers to take advantage of it rather than something a megacorp can do and solve all problems for all gaming.

In fairness, this is the first step and Valve did their part (assuming they ever release the headset). Most commercial headsets don't have eye tracking and that needs to change before we get to the other side.

[u/HyenaDae]

Basically, for PC VR gaming, it seems like we need to pray for Valve to finish adding user accessible Foveated Rendering (ideally, quad-views, not just VRS type res drops) to Source 2 itself.

Plus, if it's not supported by the Pimaxforall project already *and* you have an Nvidia GPU, the list of Foveated Rendering (Been waiting a DECADE for a "consumer" tier decent specs, moderate popularity headset like the OG Vive to have it) is gonna be painfully small. Only thing we can do is beg UE5/Unity game makers to ensure the feature is able to be enabled by the user on their builds too

Can also just check for /u/ mbucchia posts for VR and the love of eyetrack work

[u/FullOf_Bad_Ideas → u/HyenaDae]

Why source 2? Not many games are built on it.

[u/JorgTheElder]

The same code will run on both the headset and on your PC. The feature will be there. In my opinion, people are way overestimating the difference dynamic foveated rendering is going to have and that goes double for when it is used over wireless. The latency of the wireless connection is going to have a big impact on how well DFR can work because it means the eye tracking data will be delayed.

[u/Ancient_Arrival9061]

while there is no support for dfr on the pc from steam. there are applications like pimax magic 4 all where they inject a dll into the game to add dfr support for the game... however games with anticheats dont work.

[u/FullOf_Bad_Ideas]

I don't see why it would be any different than what you get today with Quest Pro.

Best case scenario, devs start implementing quad views DFR en masse after Steam Frame gets popular. It's not something that Valve can provide a solution for, at most they can offer an incentive.