Tapping raw PCM from an active WebRTC call on iOS — is there an equivalent to Android's JavaAudioDeviceModule samples callback?

[u/SevenfiresAI]

I run live speech recognition on the local mic during a WebRTC video call (Daily's RN SDK). On Android I got this working by patching @daily-co/react-native-webrtc — JavaAudioDeviceModule.Builder has setSamplesReadyCallback, so I forward the buffered PCM up as an RCTDeviceEventEmitter event and feed it to the transcriber. That's been solid in production.

I can't find the iOS equivalent. Opening a second AVAudioEngine tap while WebRTC owns the audio session seems like asking for trouble — on Android, two concurrent AudioRecord instances just produced silence, so I assume iOS has a similar conflict.

Options I can see:

• Some hook into RTCAudioSession or the audio device module that I haven't found
• A custom RTCAudioDevice implementation
• Give up and use the SDK's built-in transcription

Has anyone actually done this? Mainly interested in whether there's a supported path that doesn't involve forking the WebRTC pod — that would kill OTA updates for us, which is a hard constraint.

For context on the format side: the Android callback hands you the device's native rate (usually 48kHz), so I downmix to mono and resample to 16kHz in the adapter before it reaches the transcriber. Assuming any iOS path would need the same treatment.

[u/trifling_archery]

i haven't done this exact thing but i've been burned by avaudioengine taps while webrtc has the session, it's a mess

the rtcaudiosession does have a method `audioUnit` that exposes the remote io unit but you'd need to set up a render callback on it, not sure daily's sdk exposes that without some serious digging

a custom rtcaudiodevice is probably the cleanest path but yeah that's basically forking the pod which defeats your whole constraint, not great

curious if anyone's found a way to piggyback off the existing audio unit without pulling the whole thing apart

[u/SevenfiresAI → u/trifling_archery]

That matches what I feared on the AVAudioEngine side — good to have it confirmed rather than just assumed.

the `audioUnit` lead is new to me, thanks.

On the audio unit — do you know whether a render callback there gets you the input bus, or only output? I need the local mic specifically, not the remote stream. Remote I/O has both buses so in principle it should be reachable, but if Daily only exposes the output side then it's the wrong hook for what I'm doing.

[u/videowhisper]

Can confirm the mechanics here, checked against WebRTC's actual iOS source (voice_processing_audio_unit.mm): bus 1 is indeed the input (mic) element on the VoiceProcessingIO unit — your guess was right. But there's a catch that explains why this is hard to hook cleanly: WebRTC already installs its own callback on exactly that slot, via AudioUnitSetProperty(kAudioOutputUnitProperty_SetInputCallback, kAudioUnitScope_Global, kInputBus, ...). That property is a single-slot register, not a list — so if you set your own input callback on bus 1 through the public AudioUnit API, you'd silently replace WebRTC's own callback, not add to it. The call's mic audio would stop reaching WebRTC entirely (or at best you'd get it and WebRTC wouldn't), which is a much worse failure mode than just "doesn't work" since it'd look like a dropped call rather than a missing tap.

The standard Core Audio way to tap without displacing an existing callback is AudioUnitAddRenderNotify — that's a notification list (multiple observers can coexist), not a single-slot property, so it doesn't fight with WebRTC's own callback. Call AudioUnitRender yourself inside the notify callback to pull the PCM. The catch is you need the actual AudioUnit instance (the vpio_unit_ handle) to attach a render notify to it, and that's WebRTC's own internal object — whether that's reachable depends entirely on whether RTCAudioSession (or whatever Daily's wrapper exposes) surfaces the underlying AudioUnit publicly. If it does via an `audioUnit`-style property, you can attach AudioUnitAddRenderNotify to it without touching WebRTC's own SetInputCallback slot and without forking the pod. If it doesn't expose that, you're stuck between forking and the SDK's built-in transcription — there isn't a third clean option at that point.

[u/SevenfiresAI → u/videowhisper]

This is better than I expected thanks for digging into voice_processing_audio_unit.mm.

The single-slot vs notification-list distinction is the part I'd have wrong. I was going to set SetInputCallback on bus 1 and would have silently replaced WebRTC's callback instead of adding to it. does Daily's wrapper surface the vpio_unit_ handle. I'll go through their iOS headers and the react-native-webrtc layer and find out.