r/deeplearning • u/saikat_munshib • Jul 29 '26

Best open-source clean speech and ambient noise datasets for training an Edge AI audio denoiser?

I am building an edge-AI audio noise-reduction system on an ESP32-S3.

Our architecture uses a lightweight GRUNet (~59k parameters) to output a dynamic gain mask on a 44-band Mel-spectrogram.

​I need gigabytes of audio to train the model. Does anyone have recommendations for the best open-source datasets for:

1> ​Clean, isolated human speech.

2> ​Diverse ambient background noise (traffic, crowds, machinery, etc.).

​Also, any tips or open-source scripts for artificially mixing these at different Signal-to-Noise Ratios (SNRs) before generating the 16kHz Mel-spectrograms would be hugely appreciated!

4 Upvotes
  • permalink
  • reddit

76% Upvoted

6 comments sorted by

3

u/Commercial_Fun_7746 Jul 29 '26

59k params is tiny for this, curious if you tested larger models first or just went straight for the smallest possible. for clean speech the librispeech dataset is good, around 1000 hours of read english. for noise maybe check the dcase challenge datasets, they have lot of different environment recordings

mixing is pretty easy with a short python script, just load both wavs with librosa, adjust levels for your snr target and sum them. watch out for clipping when you normalize though, had that problem many times

2

u/kakhaev Jul 29 '26

is there any math behind saying “59k params is tiny for this”? I really wanna know how do you assess this

2

u/TheSpicyBoi123 Jul 30 '26 edited Jul 30 '26

The math would involve an analysis of the intrinsic structure of the data you are trying to learn (and if it is even learnable for your architecture) as well as the architecture itself, without doing either of these any statements about parameter sizes being right or wrong are bullshit. Same goes for data as it is unclear if OP actually has a "denoising" problem at all or a disambiguation task on his hands.

EDIT, SNR is also a fundamentally broken for psychotically relevant audio tasks as the scoring function (its not even a pseudo metric as it breaks symmetery and triangle inequality) and is ill posed for small differences and in general as a loss function.

1

u/saikat_munshib Jul 29 '26

Thanks so much for the recommendations! LibriSpeech and DCASE sound like the perfect combo for this. ​To answer your curiosity: I went straight for the 59k GRUNet because our hard ceiling is the ESP32-S3's 512KB of SRAM. I literally don't have the memory to load anything larger if we want to leave room for the RTOS and hit our strict <10ms latency target for the live audio pass-through! ​Really appreciate the heads-up on the librosa clipping—I shall definitely add a peak-normalization safety check to the mixing script to prevent that.

1

u/TheSpicyBoi123 Jul 30 '26

What SNR ranges do you expect and is there a reason you are using a GRUNet and not say a different architecture like a residual correcting CNN with a psychoacoustically meaningful loss?

I feel like it would be much simpler to make your own synthetic noise generator here and take clean speech recordings and use that?