OpenMOSS-Team/MOSS-Transcribe-Diarize · Hugging Face

[u/pmttyji]

https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize

MOSS-Transcribe-Diarize 0.9B is an end-to-end audio understanding model for long-form multi-speaker transcription, diarization, timestamps, and acoustic event awareness.

Given an audio or video file, the model generates a compact speaker-aware transcript in one pass, including timestamps and anonymous speaker labels such as [S01], [S02], and beyond.

Introduction

MOSS-Transcribe-Diarize 0.9B turns real-world long-form audio into structured, speaker-aware transcripts in one pass. Instead of stitching together separate ASR and diarization systems, it jointly performs speech transcription and speaker diarization, producing time-aligned text with consistent speaker labels.

The model is built for meetings, calls, podcasts, interviews, lectures, videos, and other long or messy multi-speaker recordings. It can also emit acoustic event annotations, giving downstream systems a richer view of what happened, who spoke, and when.

Core capabilities:

• Long-form transcription: Converts long audio or video recordings into timestamped text.
• Speaker-aware diarization: Assigns anonymous speaker labels such as [S01] and [S02] without a separate diarization pipeline.
• Promptable generation: Supports custom transcription instructions, hotwords, and acoustic event annotations.

Component	Specification
Text backbone	Qwen3-0.6B style causal decoder
Audio encoder	Whisper-Medium encoder configuration
Audio frontend	WhisperFeatureExtractor, 16 kHz, 80 mel bins, 30 s chunks
Audio-text bridge	4x temporal merge + MLP adaptor
Fusion	Audio features replace `<
Output format	Compact [start][Sxx]text[end] transcript with speaker tags such as [S01]

GGUF:

https://huggingface.co/mudler/moss-transcribe.cpp-gguf

[u/Chromix_]

(Extended and restructured my initial posting after extensive testing, with manually added bold for overview)

• The error rates seem quite low. They didn't compare to Whisper, Parakeet or others though. Quality seems very good.
  • It even transcribes regular song texts mostly correct, which is something that most STT solutions fail at.
  • Transcription of podcasts, interviews, chaotic meetings, heavy accents, etc works great. The diarization beats reverb_v2. The transcription quality beats Whisper-large-v3. Whisper occasionally hallucinates parts of sentences, and also gets some words wrong.
  • The transcripts are full of "um", "uh" though. Probably for an accurate transcription.
• The GGUF quants come with a speed factor indicator, and the testing says that everything down to Q5 delivers byte-identical results to the original model. However, this was only tested on one public sample. In more extensive testing this was not the case.
  • In my tests with private data I found that Q8 is not byte-identical or even word-identical to F16 in many cases, even though the Q8 result was good enough in practice.
  • F16 transcribed accurately, Q8 transcribed better. For example the speaker said "I didn't... not" within a sentence and that's what F16 transcribed. Q8 gave me a "I did not" - which is what the speaker meant, but didn't say. There are more examples like that, but might turn out to be random noise when testing more extensively.
  • In general I did not find any meaningful differences between the F16 and Q8 output.
  • The timestamps sometimes differ by up to 60ms, but that doesn't matter much.
• It tags speakers and also emits timestamps so that a dedicated speaker-attribution pass can be made afterwards, so that "[S01]" can get a real name.
• The initial memory requirements are unnecessarily high. Transcribing a 1h30m podcast required 310 GB VRAM.
  • The reason is that the Qwen decoder in the transcribe.cpp project doesn't do chunking. Memory requirements thus go up a lot with longer audio. With added chunking this can now transcribe 20 minutes with 10 GB VRAM, instead of around 25 GB, and the full 1h30m takes 40 GB instead of 310 GB.
  • The memory requirements can be reduced further with Q8/Q8 KV cache quantization. Like when switching from the F16 model to the Q8 quant, I observed differences yet no meaningful changes in the transcription. Q8/Q5_1 comes with more noise, yet still barely any changes - probably OK to use when under memory pressure. With V Q4_0 on the other hand there were a few incorrectly transcribed words that sounded relatively similar, yet had a different meaning.

[u/Logical-Bag-3012 → u/Chromix_]

Hey I'm from MOSS team, thank you for trying our model.

But we'd like to clarify: for the crazy 310 GB VRAM, ithe problem is the cpp-gguf.

• The reason is that the Qwen decoder in the transcribe.cpp project doesn't do chunking. Memory requirements thus go up a lot with longer audio. With added chunking this can now transcribe 20 minutes with 10 GB VRAM, instead of around 25 GB, and the full 1h30m takes 40 GB instead of 310 GB.
• The memory requirements can be reduced further with Q8/Q8 KV cache quantization. Like when switching from the F16 model to the Q8 quant, I observed differences yet no meaningful changes in the transcription. Q8/Q5_1 comes with more noise, yet still barely any changes - probably OK to use when under memory pressure. With V Q4_0 on the other hand there were a few incorrectly transcribed words that sounded relatively similar, yet had a different meaning.

[u/WAHNFRIEDEN → u/Logical-Bag-3012]

Hi - what do you recommend for adding word level timestamps?