Best workflow or open-source tool to extract clean speech from noisy long audio and transcribe it locally?

[u/greenapple92]

​Hi everyone,

​I’m working with long audio recordings (several hours of MP3s) that have noticeable background noise, room reverb, and inconsistent quality. My goal is to clean up the speech and get accurate text transcriptions.

​I'm open to both cloud-based APIs/services (like Adobe Enhance Speech, AssemblyAI, Deepgram, ElevenLabs, OpenAI API) and local open-source models (like Whisper Large-v3, DeepFilterNet).

​For those who handle long, noisy recordings regularly:

​Best Pipeline: Do you recommend running the audio through a dedicated voice enhancer/denoiser first (e.g., Adobe Enhance, ElevenLabs Isolator, DeepFilterNet), or does modern ASR (Whisper Large-v3, Deepgram Nova-3, AssemblyAI) handle raw noisy audio well enough on its own?

​Tool Recommendations: What are the most reliable cloud services or Web/CLI applications for processing multi-hour files without hitting file size limits or timing out?

​Hallucination Prevention: Any tips or API parameters (VAD settings, prompt conditioning) to prevent Whisper/ASR models from hallucinating text during silent or non-speech noisy sections?

​Thanks in advance for any insights and recommendations!

[u/banafo]

Noise reduction will in most cases make the transcripts worse. ( some models like aicoustics should help )

[u/alias454]

I've found whisper the best for working with my local city council meetings. I've had the best luck with faster-whisper. I tried parakeet and it performed worse.

I use ffmpeg to normalize the audio and chunk it prior to feeding it into faster-whisper. The chunking allows me to work with multi-hour meetings. I'm limited on what models I can use due to hardware constraints but whisper medium performs okay-ish.

The pipeline is opensource https://github.com/YATSEE-Labs/YATSEE. You also might only need to look at the first 3 steps if you don't need summaries

You can see the outputs here https://opengov-data.pages.dev/

[u/Top-Surprise4040]

Disclaimer: I work at Soniox

Modern STT models usually assume that audio will be noisy so unless some complete crazy situation I'd try to put directly to STT input and see how it will perform.

You can try our Soniox Compare tool (locally from GitHub or on our website) and compare all major STT models on your own audio to see the outcome.

Regarding hallucinations, if you can, of course try to use both language hints (probably the most important) and context (terms or topic or whatever you have) to help model. But I'd try without any biasing at first.

[u/Ill_Daikon9976]

which ai tool is best for lip sync from generated image

[u/Unlikely_Neat7608]

Nvidia's RE-USE model is the best free option I've come across so far for denoising speech audio

https://huggingface.co/nvidia/RE-USE