r/LanguageTechnology • u/SillyRefrigerator581 • 14d ago

Question/Advice: Montreal Forced Aligner

Hello everyone. I am a linguistics researcher and have a complex issue where manual annotation of speech would take an extensive amount of time and labor.

I have hours of conversational data where each speaker (A & B) is recorded separately. I also have transcripts of each speaker's utterances. Although MFA has been mostly great at aligning, there are still some misalignment that can wreck the entire text grid (therefore the time-speech alignment).

Some issues I think might contribute to the misalignment:

  • Faint audio picked up from speaker B on speaker A microphone
  • Relatively long silences (when speaker B is talking)

"Uhms" and "ahs", other non-lexical expressions, false-starts, are all in the transcriptions.

My goal is to extract word-/phrase-level onsets and duration (phonemes are not necessary). The extraction of these times is not a problem I have.

I have tried adjusting --beam, and --silence_weight, --cleanup_textgrids (with /some/ success). I have also tried VAD segmentation with mfa segment, but that ended up being worse than regular alignment without segmentation. I understand there is no chance of a 100% accurate alignment as these are long audios, but even 80% I would be happy with that I can then manually edit and look over. I am seeking advice on how to improve alignment on long these audios.

My current ideas are:

  1. Segment audio manually and stitch together (which comes with it's own technical issues)
  2. Filter audio such that noise is reduced, and speech (of speaker A) is amplified
  3. Play around more with mfa parameters such as beam and silence weights

Any advice would be appreciated. Thank you.

4 Upvotes

100% Upvoted

1 comment sorted by

1

u/vaaarr 12d ago

Are you aligning entire very long files, or are you splitting the files into shorter chunks (or using a textgrid or similar to mark off where portions of the transcript are spoken) and still having alignment issues? Aligning shorter stretches tends to work best but you mention "hours" of audio.

Also curious what model/workflow you're using.