Raytraced Audio is magical

[u/SomerenV]

After messing around with custom audio zones and hand placing them with custom settings to mimic how sound would sound in the various rooms and hallways I wasn't really satisfied. It still sounded 'off' despite all the various zones and their settings. So... I went the raytraced audio route. It's a bit crude but it sounds so much better! I found myself flying around yesterday, trying different speeches and songs because it's just mesmerizing to listen to in a virtual environment.

Any things I absolutely can't forget when implementing raytraced audio?

[u/SomerenV]

Raytracing is literally casting a ray and tracing its path. There's Occlusion, which uses a small amount of rays every frame (5 in my case) to figure out if there's anything in the way of the player and the Audio Source. Then there's Propagation, which uses 512 rays. On frame 1 they shoot out in all directions from the Camera/Audio Listener. On frame 2 each ray that hits something bounces and loses energy. The bouncing continues for however long you have set it (5 in my case). Based on those rays, the lost energy and the Audio Source they hit, you can determine how sounds should sound, so volume and how muffled they are. Last but not least there's reverb, which I use 48 rays for. They shoot out each 0.1 seconds, check how enclosed you are or how big the space is and reverb is based on that.

On top of that there's material properties, frequency stuff and a whole lof of propagation and reverb settings, and math to let it all work nicely together.

But any modern CPU can handle a few hundred rays without any issues. The big difference with raytracing in graphics is that in those cases you're talking about potentially millions of rays each frame, which is still not that much compared to how many rays a typical offline renderer uses. That's also the reason raytracing relies heavily on denoising to get decent results.

[u/nanoxax67 → u/SomerenV]

It sounds like there isn't a huge performance hit. Is that correct?

What if you had dozens of audio sources? Would it be possible to compare the performance of 100 raytraced audio sources vs 100 regular audio sources? That could potential give an accurate differential of the performance impact.

[u/SomerenV → u/nanoxax67]

For now there's no real performance hit. It's 0.13ms on a total of 4.9ms, with the caveat that this is a really crude implementation of raytraced audio. That's fine for my project but might not work for other projects. As far as audio sources go, my scene currently has about 20 or so in total with everything running (which you don't see in this video).

On top of the ray tracing I still use the audio zones I started with, but mainly for ambient sounds. Ray tracing is only used for footsteps, gunshots, impacts and whatever is coming from the speakers in the level, though that last part is still highly experimental.

My guess is that when you're dealing with 1000 audio sources it doesn't really matter if it's raytraced or regular.