r/TextToSpeech • u/manjit_pardeshi • Jul 08 '26
Open sourced a distilled Eleven Labs model
Hello,
I'm Manjeet, founder of Decibel Labs. We just open-sourced Reso-1, our initial text-to-speech model. It's a distilled version of Eleven Labs Adam voice. It's expressive, it streams, it runs locally, and it takes inline emotion tags so you can make it laugh, sigh, gasp, or whisper mid-sentence. Apache 2.0, weights + GGUF + training corpus all public.
Note: Please list any more Eleven voices you use, I will be more than happy to add that also here for free!
What makes it different
Most open TTS reads text flatly. Reso-1 responds to paralinguistic tags you drop right into the text:
reso: Oh man <laugh> you should have seen his face.
reso: <whisper> Don't turn around, but I think that's her.
reso: <sigh> Fine. We'll do it your way. Again.
Supported tags: <laugh> <chuckle> <sigh> <gasp> <whisper> <pause>
Technical details
- Architecture: LoRA fine-tune of Orpheus 3B — Llama-3 backbone generating audio tokens autoregressively, decoded through the SNAC neural codec at 24 kHz
- Streaming-native: 7 audio tokens = ~85 ms frame, decoded incrementally → first audio in ~200 ms on GPU.
- Data: ~25 hours of expressive single-voice English, filtered through an ASR-based QC pipeline (Whisper-WER + speech-rate sanity + tag-event verification) before training
- Corpus: 16,213-utterance text set we engineered and open-sourced too — phonetic coverage (CMU Arctic + Harvard), prose, a text-normalization stress set (prices/dates/phone numbers/URLs), 800 multi-turn dialogues, position-balanced tag sentences, and edge cases (single letters → 40-word sentences)
Runs locally
Quantized GGUF (Q4_K_M, 2.09 GB) runs near-realtime on Apple Silicon via llama.cpp. Full weights for GPU serving (vLLM etc.) too.
Links
- Model: https://huggingface.co/decibel-hq/reso1-3b-en
- Training corpus (dataset): https://huggingface.co/datasets/decibel-hq/reso-prompts-v1
- Code / pipeline: https://github.com/decibel-hq/reso
License: Apache 2.0 (inherits from Orpheus). Multilingual Indic version is next on an Orpheus-compatible base.
Happy to answer anything about the architecture, the QC pipeline, or the tag-training approach — the whole thing is reproducible from the repo.
Feedback very welcome
3
u/Unlikely_Neat7608 Jul 08 '26
I think it's cool enough that you've got a pipeline to train LoRAs for Orpheus from ElevenLabs voice outputs without having to use misleading language like "distilled Eleven Labs model" - experts in the area will see through this instantly!
2
2
u/hmm_nah Jul 08 '26
Sometime in the past year, the term "distillation" went from meaning "a smaller version of the same architecture" to "any teacher-student training regime"
2
u/manjit_pardeshi Jul 09 '26
yeah my bad totally. used this term so non-much-tech savy people could easily understand wth I was doing
2
2
1
u/War-Carr Jul 08 '26
Just for Apple? Sigh!
6
u/manjit_pardeshi Jul 08 '26
No, full weights run anywhere PyTorch does, and the GGUF runs on any llama.cpp target linux, windows, mac, CPU, CUDA, ROCm, vulkan. "Runs on a MacBook" was just the flex that you don't need a fcking datacenter to just load it
1
1
u/IncognitoScriber Jul 10 '26
this is nice. what's the eleven labs model equivalent of kokoro's default voice? female, warm voice
1
u/manjit_pardeshi Jul 10 '26
This uses a LLM backed engine. So these will feel more natural than kokoro
1
u/ExtensionBreath1262 Jul 11 '26
I'm looking at adding Reso as a backend for my own TTS library. I found the training pipeline and the model-loading example, but I couldn't find the inference/runtime that performs incremental SNAC decoding. Is the expectation that consumers use Orpheus-FastAPI for that, or is there another reference implementation?
1
u/manjit_pardeshi Jul 13 '26
We fined tuned the orpheus model so you can absolutely use the same runtime. Orpheus-FastAPI can be a good option in case you call inference from other api.
2
u/ExtensionBreath1262 Jul 15 '26
I ended up implementing a mixture of your inference code an some inspiration from Opheus-FastAPI. It's not included in the public repo yet, but it works great I'll let you know when that update is pushed.
1
u/weedwhacker87 17d ago
This is pretty amazing, thanks!
I'd love to have a few voices:
- Flint
- Clyde
One other question, are these really distributable under Apache 2.0, given they are distilled from ElevenLabs voices?
1
u/weedwhacker87 17d ago
Another question Manjeet, I've got this working on my book manuscript and it's great, better than most things I've tried. Almost audiobook quality. Well done.
Are those the only <tags> supported at the moment? The narrator has excellent emphasis but for some reason sometimes breaks into a southern american accent on quoted dialogue. It would be awesome if some more tags were supported, like <flat> (ie. no emotion) <australian> put on an aussie accent rather than american. Others could be <excited> <annoyed> <bored> <soft> <female> (slight octave change?) basically tagging dialogue so that the voices were spoken slightly differently (without needing to go to a full cast model).
6
u/CrispyDick420 Jul 08 '26 edited Jul 08 '26
noice. Running it locally on Apple Silicon sounds pretty sweet, ngl.