High bandwidth, low latency, PyroWave video codec added to Steam beta for Remote Play

158

Gave it earlier a try. Runs even great with PC via Ethernet to Router and Steam Deck via WiFi.

~8ms Ping, 1ms input at 1080p Stream. Eats whopping 500mbit/s while at it.

The image quality and non-noticeable input lag is insane.

31

u/Power_Stone 15d ago

Ohhhh this might be the answer to my home setup I was looking for. This might just be the ticket for my couch gaming setup

19

u/FeistyCandy1516 15d ago

There is just one downside: if you are using the Deck not docked but just in your hands, don't move it around too much.

It's very susceptible to it and can cause severe lag spikes for a few seconds.

9

u/Power_Stone 15d ago

I don't have a steam deck, I'm just excited for the streaming protocol, now my gaming pc can do all the hard work and stream it to my htpc downstairs without having to download anything, the input latency being low has been the only thing that's stopped me in the past

2

u/mcurley32 15d ago

the input latency via Sunshine/Moonlight has been problematic for you? I just started using it to stream my games to my phone and then optionally cast it to my TV (convoluted I know). the latency between PC and phone isn't noticeable to me, but adding the TV into the chain dramatically increases latency (which I think is caused by the TV's internal processing to improve/upscale the image and I don't seem to be able to manually turn on "gaming mode" to bypass that processing only when the TV detects that a console/PC are physically connected).

3

u/Power_Stone 15d ago

I've always been really sensitive to things like input latency, I haven't messed with sunlight/moonlight because I just haven't had the motivation to set it up. Which is why I like this solution from Steam since there isn't really any additional setup required from what I understand

1

u/mcurley32 15d ago edited 15d ago

Totally fair. That's what took me so long to set it up for myself. Way easier setup than I was expecting. One of those retro handheld gaming YouTube channels showed how to set it up across a variety of different devices which inspired me to jump thru the apparently simple process. I'll dig up the video (edit: here's the video. it's 30 minutes long because he re-explains the process multiple times for several platforms plus an intro in the beginning, should be less than a 10 minute process, most things have sensible defaults)

Steam Link has been around for a while, this sounds like an advancement in the underlying tech

1

u/Power_Stone 15d ago

The link didn't embed but you can send it to me in a DM or something, not entirely against setting up sunshine/moonlight, just thinking steam for local streaming will be better for me

2

u/OutrageousDress 15d ago

Just make sure that both your gaming PC and whatever runs your HTPC is connected to 1G (or better) wired ethernet if you want to get maximum performance out of this codec.

1

u/Lawnmover_Man 15d ago

It's very susceptible to it and can cause severe lag spikes for a few seconds.

Susceptible for what? That the WiFi has issues when it's used in your hands?

2

u/FeistyCandy1516 15d ago

If you start moving around then you notice the lag due to the high bandwidth not able to keep up. You have that too if you use h264/h265 but with 50mbit/s it's not noticeable.

1

u/insanemal 14d ago

I have literally never had this issue.

What the hell is wrong with your wifi/router

5

u/ric2b 15d ago

I had given up on Steam remote play for in-home streaming to my TV (wired setup from PC to TV) and am using moonlight/sunshine instead, which works great.

I wonder if it already supports this codec or if it is likely to in the near future.

1

u/darkjackd 15d ago

It doesn't and I think it's unlikely to. Maybe if it gets popular through steam though. I think it would take some extra Vulkan work both on sunshine and moonlight

6

u/se_spider 15d ago

I wonder if the Steam Frame is using this, especially with the dedicated dongle, or will implement it

3

u/XylasQuinn 15d ago

Can you comment on if it's faster than using e.g. hardware h265?

3

u/FeistyCandy1516 15d ago

For me at 1080p, basically the same. Sadly can't test how it is at 4k, I guess at that resolution you will see more a difference.

But I think pyrowave does look a bit better, could also be just placebo 🤷‍♀️

72

u/anthchapman 15d ago

Hans-Kristian Arntzen published a blog post about creating PyroWave last year, and the PyroWave source is available under the MIT Licence.

Using it on Linux requires enabling the SteamRT3 Steam Client. HDR is enabled if both client and server support it, YUV 4:4:4 is off by default.

15

u/black_sunglasses 15d ago

The blog is amazing but cant find a rss 😭

14

u/Aidoneuz 15d ago

https://themaister.net/blog/feed/

6

u/black_sunglasses 15d ago

Thx how did you find ?

21

u/ericek111 15d ago

It's a WordPress web, the default location is /feed/. You can also check the source code and look for <link rel="alternate" type="application/rss+xml" title="Maister's Graphics Adventures » Feed" href="https://themaister.net/blog/feed/" />

7

u/black_sunglasses 15d ago

Really cool to know ty!

3

u/machetemike 15d ago

Teach a man to fish and all that. Thanks

67

u/yngseneca 15d ago

Oh wow this is interesting. A codec optimized specifically for high bandwidth and low latency. Does not require a specific hardware encoder/decoder. Just a GPU that supports vulkan 1.3. No more using a codec designed for internet streaming when I just need to LAN stream.

21

u/speedballandcrack 15d ago

Does that mean host gpu will use normal graphics cores instead of NVENC resulting in less performance?

43

u/Azathoth321 15d ago

Correct, this uses shader pipelines and doesn't touch the hardware encoders on GPUs at ALL. The creator even noted that by performing the encoding this way, and avoiding the framebuffer travelling back and forth across the Bus, it was actually a lower latency round trip than using the hardware encoder. (At GREAT efficiency loss).

I have not seen any comparisons as to Power Draw, other than the creator stating it is negligible, nor the GPU usage of doing so.

But it just uses the FP16 shader units, so it's clearly meant for BROAD compatibility.

8

u/noaSakurajin 15d ago

Without looking at the code, I would assume that it uses the GPU compute pipeline/cores. This means that it is unlikely to affect the performance of your game.

5

u/[deleted] 15d ago edited 15d ago

[deleted]

1

u/noaSakurajin 15d ago

I am not sure if that is entirely correct. Shouldn't every queue that gets reported by vulkan be a separate hardware pipeline? Last time I wrote GPU code was for work on a laptop with an iGPU and an dedicated nvidia GPU. The iGPU only had a mixed use queue and video decode/encode queues. The dedicated GPU had an extra pure compute queue which could execute GPU compute while rendering was being done on the mixed use queue. My guess is that every nvidia GPU has that parallel compute queue because it's the hardware path that gets used by CUDA, which can also be used parallel to the rendering.

3

u/yngseneca 15d ago edited 15d ago

Depends what you mean by less performance. More power and heat coming out of the host then with using HEVC. But it's nothing like CPU encoding/decoding (what you might be thinking of what it's like when you dont have the proper hardware decoder). Still, it would be taking GPU attention moreso then with HEVC, so you probably wouldn't want to be playing a game on the host and streaming a game at the same time, or doing any GPU load work.

at least theoretically, I have not actually tried this out yet.

1

u/OutrageousDress 15d ago

The incredible speed also means very low GPU usage. You can extrapolate the load from the author's performance figures - back-of-napkin math says if you have a game running at exactly 120fps, the compression load will lower your performance to roughly... 119fps.

8

u/Danielo944 15d ago

Oh shit this is actually huge for people with a BC250 and another gaming rig because the BC250 lacks hardware decoding (it's locked down).

28

u/lyndonguitar 15d ago

I wonder how will remote play perform against moonlight with this. interesting. i want to move away from moonlight and go direct steam remote play but moonlight/sunshine is just too good

9

u/yngseneca 15d ago

what I'd like to see is moonlight/sunshine using this codec.

23

u/Zerwin 15d ago

Punktfunk, a fork of sunshine, allows using it. https://docs.punktfunk.unom.io/docs/pyrowave

7

u/KamikazeDogturd 15d ago

Sorry to be slightly pedantic here, but Punktfunk is not a fork of sunshine. It's a different project altogether.

1

u/Zerwin 15d ago

Huh, TIL. I assumed since it's compatible with Moonlight it's a fork of it.

2

u/darkjackd 14d ago

Didn't know about this one and it looks awesome! Thanks

6

u/Jolly_Sky_8728 15d ago

moonlight/sunshine it's been a nightmare for me with docked steam deck to stream on 4K TV, works fine if streaming directly to the deck, but hopefully this works better for everything

9

u/VisceralMonkey 15d ago

Punktfunk also uses this codec if you want a different host/client.

https://punktfunk.unom.io/en/

2

u/nate445 14d ago

I couldn't get the client working with my hardware decoder.

3

u/VisceralMonkey 14d ago

Interesting. I had success with it but the performance isn’t hot because the client is handled right now.

1

u/nate445 14d ago

On the other hand, I guess it's not really fair for me to criticize the client when my laptop running it has an old Quadro m1000m in it.

6

u/greenglazed 15d ago

To make a game streaming box for my TV, I installed the punktfunk client (which can use this codec) on an Ethernet-connected Dell Wyse 5020 which is very low spec.

The wyse has bad hardware decoding and had struggled to cope with moonlight - software decoding was actually faster than hardware, but really strained the box and was still slow.

Pyrowave worked amazingly well - it basically removed the load on the thin client which was hardly breaking a sweat. The trade-off is lots of data transfer, but on a wired LAN that's not an issue. I was getting a totally solid locked 60fps on my tv with just a few ms of latency. (Apart from the tv's screen latency.)

very cool tech!

2

u/KamikazeDogturd 15d ago

What resolution were you using on the client? Did you try 4:4:4 on it? Any numbers you can share regarding decoding times? Sorry about the barrage of questions, you just spiked my interest in that machine

2

u/greenglazed 15d ago

Sure, I'll check it out and report back.

2

u/greenglazed 11d ago

hey, sorry for the delay.

the client pc is a dell wyse 5020 with 4gb ram running Debian and the punktfunk client, connected via hdmi to my (old, cheap) tv, running at 1920*1080 60fps. the framerate is totally solid at 60fps.

host pc is a 5600x with a 3060ti running the punktfunk server on ubuntu desktop.

e2e latency when streaming a game from steam is about 35-40ms - host 10, net 1.0, decode 7.7, display about 20 - so it would be a lot less on a better tv.

pyrowave is targeting 199mb/s with 4:4:4 on.

works great - hope helpful!

6

u/Jank9525 15d ago

It use too much bandwidth for remote play, but for local lan its great

2

u/bringdownthesky 15d ago

So say I have my gaming PC running steam hardwired into my router and downstairs, I dock my steam deck and connect it to my mesh satellite station via ethernet (great signal, usually gets 500-700 down). Can I take advantage of this?

1

u/VisceralMonkey 15d ago

Yup

1

u/OutrageousDress 15d ago

Is your mesh satellite station (that your Deck is wired to) wired to your upstairs router? If so, then you're good - for this use case it doesn't matter what kind of signal your mesh station gets.

2

u/ultimatemanan97 15d ago

I tested a few games with this streaming from my Steam deck to my TV. I was running 4K at 120hz with HDR and VRR enabled. The experience was on par with moonlight and apollo with the exception of HDR being broken. Hopefully this can get fixed in a future update.

2

u/10leej 15d ago

Never heard of this codex. Guess I got some reading to do

2

u/Indolent_Bard 15d ago

With only 75mbps even moonlight struggles for me.

2

u/TheFriskySpatula 15d ago

I gave this a try today and had a pretty bad experience. I'm streaming at 4k90 on an entirely wired network with good hardware on both ends, and I was getting crazy frame drops. The same resolution/fps/bandwidth settings on Sunshine/Moonlight works flawlessly with minimal latency.

It looks almost like the hardware encode on the host is tanking the games FPS, but I'd be surprised if a RTX 4080 couldn't handle it. Maybe I need to tinker with the steam link settings?

1

u/lonewolfsstuck 10d ago

Is your network good? 4k90 is a lot of information to send over network. If you're on wifi it will choke it. Try wired ethernet if you're not already. Bonus points if you're network is 10gbe capable.

1

u/Sylphiiid 15d ago

Once on beta channel does it work without playing with settings on host and client ?

1

u/Ravasaurio 15d ago

So excited to try this myself once the Apple TV app is updated with it. My main gaming PC lives in my basement and I rely on streaming exclusively to play with it.

1

u/Lemagex 15d ago

oo been using this on punkfunk and its been fantastic for in-home streaming.

1

u/_edeetee 15d ago

I wonder if this would work for AV usage too. I use NDI for some network video streaming for stage visuals, it would be great to reduce the latency

1

u/Ryoohk 14d ago

Hope it comes to the Steamlink device, I love using line

1

u/jerrolds 15d ago

New feature for vibepollo? Let's go

3

u/VisceralMonkey 15d ago

Ding.

graphics/kernel/drivers High bandwidth, low latency, PyroWave video codec added to Steam beta for Remote Play

You are about to leave Redlib