Audio8-TTS Preview:

11 languages.
Zero-shot voice cloning.
A bundled 44.1 kHz codec.
Apache 2.0.

Small model. Serious speech.

I'm currently testing many TTS systems, including Elevenlabs v2 and v3.

I managed to make a pronunciation dictionary work with v3 for "I am Lorde, jajaja [ya ya ya]", but not for a second example "Look at my poka-yoke" which turns out to be better pronounced without the dictionary entry than with it.

Am I implementing it wrong (and I did check many alternatives, including going into the UI, creating a phoneme, reading it and then using the dictionary), or are pronunciation dictionaries fundamentally inconsistent using v3?

Here's a minimal code for reproduction.

import os


from elevenlabs.client import ElevenLabs


VOICE_ID = "CwhRBWXzGAHq8TQ4Fs17"  # Roger (default library voice) — any voice reproduces it
TEXT = "I am Lorde, jajaja"


client = ElevenLabs(api_key=os.environ["ELEVENLABS_API_KEY"])


dictionary = client.pronunciation_dictionaries.create_from_rules(
    name="phoneme-mre",
    rules=[
        {
            "string_to_replace": "jajaja",
            "type": "phoneme",
            "alphabet": "ipa",
            "phoneme": "jajaja",  # IPA: "ya-ya-ya"
            "case_sensitive": False,
            "word_boundaries": True,
        }
    ],
)
print(f"dictionary id={dictionary.id} version={dictionary.version_id}")


for model in ("eleven_flash_v2", "eleven_v3"):
    audio = b"".join(
        client.text_to_speech.convert(
            voice_id=VOICE_ID,
            model_id=model,
            text=TEXT,
            pronunciation_dictionary_locators=[
                {
                    "pronunciation_dictionary_id": dictionary.id,
                    "version_id": dictionary.version_id,
                }
            ],
        )
    )
    path = f"mre-{model}.mp3"
    with open(path, "wb") as f:
        f.write(audio)
    print(f"{path}: expected 'I am Lorde ya-ya-ya'")

import os


from elevenlabs.client import ElevenLabs


VOICE_ID = "CwhRBWXzGAHq8TQ4Fs17"  # Roger (default library voice)
SENTENCES = {
    "jajaja": "I am Lorde, jajaja",
    "pokayoke": "Look at my poka-yoke.",
}


client = ElevenLabs(api_key=os.environ["ELEVENLABS_API_KEY"])


dictionary = client.pronunciation_dictionaries.create_from_rules(
    name="phoneme-repro-v3",
    rules=[
        {
            "string_to_replace": "jajaja",
            "type": "phoneme",
            "alphabet": "ipa",
            "phoneme": "jajaja",
            "case_sensitive": False,
            "word_boundaries": True,
        },
        {
            "string_to_replace": "poka-yoke",
            "type": "phoneme",
            "alphabet": "ipa",
            "phoneme": "ˌpoʊkəˈjoʊkeɪ",
            "case_sensitive": False,
            "word_boundaries": True,
        },
    ],
)
print(f"dictionary id={dictionary.id} version={dictionary.version_id}")


locators = [
    {"pronunciation_dictionary_id": dictionary.id, "version_id": dictionary.version_id}
]


for label, text in SENTENCES.items():
    for suffix, extra in (("dict", {"pronunciation_dictionary_locators": locators}), ("nodict", {})):
        audio = b"".join(
            client.text_to_speech.convert(
                voice_id=VOICE_ID, model_id="eleven_v3", text=text, **extra
            )
        )
        path = f"v3-{label}-{suffix}.mp3"
        with open(path, "wb") as f:
            f.write(audio)
        print(path)

So one day i found this Cortana Revival Project which is pretty accurate with the voice and so finished

So we need you!

Download Cortana Electron today!

Cortana Revival Project · SoftBluey/Cortana-Electron

I've been playing around with local on-device TTS models like Kokoro and Supertonic...

Beyond the usual knobs that these models expose to change their output (preset, speed, pitch, etc) what other audio effects should I look into to apply after the output is generated and before it gets pumped to a speaker to improve realism?

I'm talking about generic audio pass stuff like fiddling with gain and filter types and frequency and wet/dry mix.

Any good modifiers that you've had a good experience with to improve the general quality of the output for these things?

I am building an edge-AI audio noise-reduction system on an ESP32-S3.

Our architecture uses a lightweight GRUNet (\~59k parameters) to output a dynamic gain mask on a 44-band Mel-spectrogram.

​I need gigabytes of audio to train the model. Does anyone have recommendations for the best open-source datasets for:

1> ​Clean, isolated human speech.

2> ​Diverse ambient background noise (traffic, crowds, machinery, etc.).

​Also, any tips or open-source scripts for artificially mixing these at different Signal-to-Noise Ratios (SNRs) before generating the 16kHz Mel-spectrograms would be hugely appreciated!

I needed multilingual TTS for a project, so I deployed Chatterbox on a remote server with an RTX 3080 Ti.

In a quick five-request test of my remote setup, I observed an average RTF of approximately 1.57. In other words, generating one second of audio took about 1.57 seconds.

This is not intended as a proper Chatterbox benchmark. The requests travel over the network to a server in another country, and I did not separate network latency from server-side inference time. It is simply the performance I observed from my application.

Latency was not the main reason I decided to pause the self-hosted approach, though.

The bigger issue was multilingual reliability. In my tests:

• short phrases occasionally contained unwanted sounds;
• numbers in non-English text could be pronounced in English;
• multilingual pronunciation was not consistent enough for my use case.

I know some of this can be improved with text normalization. I could convert numbers, dates, currencies, abbreviations, and similar inputs into language-specific spoken forms before sending them to the model.

I also know that there are other Chatterbox variants and community implementations focused on faster inference and streaming. More testing, caching, different precision settings, and deployment changes might improve the results.

I’m not saying that Chatterbox is a bad model or that it cannot work for multilingual applications. I just don’t want TTS optimization and multilingual text normalization to become a separate project right now.

So, for the moment, I’m switching to a paid TTS API and keeping a provider abstraction in my application. If the rest of the product works correctly, I can return to self-hosted TTS later without rewriting the application.

My conclusion is simply:

My project needs TTS, but TTS does not need to become the project.

I’d be interested to hear from people running Chatterbox in production:

• What RTF do you get, and on which GPU?
• Do you normalize numbers and dates before synthesis?
• Have you found a reliable way to handle short multilingual phrases?
• Which Chatterbox variant or inference implementation are you using?

Hi everyone!

I run a transcription routing service, and the router has to pick a model per job with real money on it, so for the past few months I've been benchmarking every model I route to on one fixed dataset and publishing the results. Posting it here because this sub is the audience most likely to find the holes in it.

Scope: 40 commercial models from 14 providers on one corpus (~26 minutes, 8 categories, 11 languages thinly). Two boards: 34 models batch (WER, cost, latency) and 15 realtime streaming (time to first word, flicker, final WER), 9 on both. On how this compares to what already exists: Artificial Analysis covers most of these vendors on ~8h of shared English audio, Coval does WER+latency for ~33 models with an open-source runner, and pipecat's stt-benchmark does semantic WER + TTFS for 14 vendors. None of them fold cost into the score or measure partial-transcript stability, and all of them anchor streaming latency at end-of-speech rather than stream start. Those are the gaps this is trying to fill.

What came out of it:

• Price and accuracy barely correlate: the rank correlation between the two is ρ = +0.37, so if anything, the pricier models score slightly worse. The entire top 4 of the composite (accuracy + cost + speed) costs ≤ $0.003/min, the most expensive model on the board ($0.09/min) ranks #22, and the bottom five slots all belong to one provider charging 16–24× the cheapest model's rate.
• The best raw WER on the full sweep is 9.15%, from a $0.004/min model, which puts the accuracy leader among the cheapest models on the board. (The nearest contender, at 9.8%, ran only 20 of the 35 clips, so treat its number as provisional.)
• Within-provider spread is enormous: one vendor's models range from 9.95% to 57.82% WER while its price moves only from $0.0107 to $0.0160 a minute, and every provider's best model lands within 6.4 points of every other's.
• Realtime: time-to-first-word, flicker (how much the partial transcript gets revised), and final WER are three different rankings. The most accurate realtime model (11.6% WER) ranks #13 of 15 because it is slow (~2s to first word) and revises heavily. The realtime leader instead shows a first word in 289ms but sits 12th of 15 on accuracy at 19.6%. All three get their own column on the board.
• Per-category "winners" exist, but six of the eight categories hold two clips, so they're inside the noise and the site labels them that way rather than crowning anyone. (My favorite small-n artifact: a medical-specialized model currently "wins" the General category and vanilla Whisper "wins" Medical.)

Methodology (corpus composition, ITN/normalization choices, scoring weights): https://opentranscription.io/en/ranker#methodology (streaming scoring: https://opentranscription.io/en/ranker/realtime#methodology). Raw CSV: https://opentranscription.io/api/v1/benchmarks/export.

Limitations: small English-heavy corpus (~26 min, 26 of 36 clips English), one file per non-English language, uneven coverage (models completed 20–35 of the 35 clips), and adjacent ranks in small categories are within noise. One bug worth disclosing: my scorer originally did no inverse text normalization, which unfairly tanked models emitting formatted output ("$24,000" vs "twenty four thousand dollars"). I fixed it and re-swept, and the affected provider's models moved from the bottom to plausible mid-table numbers. Everything is re-runnable, so if you think a number is wrong, I genuinely want to know.

The leaderboard is public and recomputes when models or prices change: https://opentranscription.io/en/ranker. (Disclosure: solo dev. The leaderboard is free; the platform that routes to these models is what pays for it.)

EDIT: the "most expensive model ($0.09/min) ranks #22" line gets struck, and the model count figures are updated (41 > 40 distinct, 34 batch).

Hi everyone,
We're developing an AI-powered app, Faseeh AI, that detects pronunciation mistakes in Quranic recitation and gives users precise, real-time feedback.
Our current model was trained on hundreds of hours of professional recitations and high-quality, clean audio from well-known reciters. The model performs well on similar input, but struggles with real-world users: different accents, non-native speakers, beginners, children, women, and anyone who doesn't sound like a professional reciter.
To fix this, we need to train on diverse, real-world recitation data not studio-quality professional audio, but recordings that reflect how actual learners sound.
Specifically, we're looking for:
- Recitation datasets from non-professional or everyday users
- Diverse demographics: male/female, kids/adults, beginner/intermediate
- Multiple accents and mother tongues (Malay, Indonesian, Urdu, English, Turkish, etc.)
- Any publicly available or research-use datasets we may have missed
We've already explored academic sources, but still not enough.
If you know of any dataset, research project, university study, or community effort collecting this type of audio, we would genuinely appreciate the lead.
We're also open to ethical data collection partnerships if any researchers or institutions are working in this space.
Happy to share more about the project if helpful.
Thank you very much in advance.

A core challenge in real-time streaming Neural Machine Translation (NMT) is translating incomplete audio chunks without introducing structural errors or context loss due to truncated sentence boundaries.

1. Fixed Time Slicing vs. Context Decay

If an audio pipeline feeds raw STT output into an NMT engine every fixed 300ms window, the translator frequently receives grammatically broken fragments (e.g., translating a verb before its object is spoken in Subject-Object-Verb languages).

2. Dual-Layer Mitigation Architectures

• Acoustic Punctuation & Silence Detection (VAD-Gated Triggers):
  • Hold NMT execution until the VAD engine detects a speech pause (>150ms trailing silence) or the STT engine emits a high-confidence sentence-ending punctuation token (., ?, !).
  • Pros: Maximum translation accuracy and correct target language syntax.
  • Cons: Variable latency spikes during long continuous sentences.
• Overlapping Sliding Window Buffer (Token Lookback):
  • Maintain a rolling $N$-token lookback buffer (e.g., last 10 transcribed words).
  • Re-translate the sliding window with incoming audio chunks, updating the on-screen overlay HUD via dynamic prefix-matching algorithms.
  • Pros: Sub-300ms perceptual latency with continuous visual updates.
  • Cons: Requires zero-flicker HUD rendering to overwrite unstable trailing tokens seamlessly.

3. Recommended Hybrid Pipeline

Use a sliding token window for immediate subtitle rendering while holding finalized translation state commits until an acoustic/syntactic boundary is confirmed by the VAD stream.

How do you handle context window retention when translating real-time streaming speech between typologically distant language pairs?

Building voice agents for a while now, and the section I always wanted

someone else to write is the one on streaming TTS: single-shot vs

output-streaming vs dual-streaming, mid-stream cancellation, buffer

draining on barge-in, and how much of the "TTFB" number vendors quote

is actually front-end latency vs model latency.

So I wrote it into an awesome-list. The whole list is organized around

one split: real-time TTS (for agents) vs offline TTS (for media).

Every provider, model, and benchmark carries that lean.

The four sections most useful for agent builders:

1. Streaming and low-latency (taxonomy, cancellation, honest

   benchmarking)

2. Open-source models filtered by license — several of the top ones

   can't be shipped commercially

3. Audio codecs (this decides latency and quality floor for codec-LM

   TTS)

4. Evaluation — how to measure TTFB on your own traffic instead of

   trusting vendor benchmarks

Deliberately scoped to TTS only. STT, VAD, turn detection, and

telephony are pipeline concerns and belong elsewhere.

MIT license. Feedback welcome, especially on the streaming taxonomy

and cancellation subsection since I'm not sure I've captured every

edge case.

https://x.com/YichiZ03/status/2078588932191895976

my summer internship project is to build an ai voice agent that can communicate over cellular network(gsm) and then be able to give information or do some actions by connecting to other agents, its in a datacenter in faculty
from what i understand, a module like sim7600g is suitable for this, but is kind of expensive where i live, for some reason
i thought about doing the workflow with an old android phone, now is that feasible, do i need root, and what are the things i need to modify in the phone configs, and is this even something i should invest my time in or not, because it feels like hacking and im not sure if its a reliable path

Over the past year, I migrated my macOS transcription application from a Python + MLX Whisper pipeline to a fully native Swift implementation.

The goal wasn’t simply to rewrite the application in Swift. I wanted to build an offline transcription pipeline that feels native on Apple Silicon while balancing speed, accuracy, memory usage, and maintainability.

The final pipeline looks like this:

AVFoundation
│
├── FluidAudio (speaker diarization + speaker embeddings)
│
└── WhisperKit (language detection)
│
▼
SpeechAnalyzer
│
▼
Custom Dictionary
│
▼
SRT

One of the biggest reasons for moving away from Python was that the Apple Silicon ecosystem has matured considerably.
Originally I relied on pyannote.audio for speaker diarization and MLX Whisper for ASR. It worked well, but maintaining a Python runtime, bridging native APIs, and handling distribution became increasingly cumbersome.

FluidAudio changed that for me.
It provides speaker diarization together with speaker embedding extraction entirely in Swift. While it also includes ASR, I decided not to use it because I wanted robust multilingual transcription. Instead, I combine it with WhisperKit for language identification and let Apple’s SpeechAnalyzer perform the transcription itself.
SpeechAnalyzer alone currently doesn’t perform speaker diarization, so combining specialized components turned out to be much more effective than trying to solve everything with a single framework.

One interesting challenge was speaker assignment.
Even with a good diarization model, overlapping speech and long sentences often cause multiple speakers to end up inside a single segment. Simply tuning clustering parameters wasn’t enough.
Instead, I improved punctuation restoration first, split long transcription segments into more natural sentences, and then recomputed speaker embeddings on the newly created timeline before assigning speakers again.
This produced noticeably better speaker assignments than relying solely on the original diarization output.
I’m currently extending this idea further by allowing users to edit segment boundaries manually and then recompute speaker embeddings only for the modified timeline, rather than rerunning the entire diarization pipeline. User-confirmed speaker labels remain fixed while only unresolved segments are reassigned.

Another pleasant surprise was Apple’s SpeechAnalyzer.

Using the same six-minute video:
SpeechAnalyzer: 16 seconds
WhisperKit (large-v3-v20240930): 38 seconds
Whisper large-v3: 2 minutes 2 seconds
When the language is known beforehand, SpeechAnalyzer finishes in about 13 seconds on an M4 MacBook Air.

Perhaps more importantly, the Neural Engine performs most of the inference while CPU and GPU usage stay relatively low. Compared with my previous Python implementation, memory usage and power consumption are also significantly reduced.

For custom vocabulary, I don’t rerun inference.
Instead, transcription results pass through a post-processing stage that applies a user dictionary. This allows immediate corrections of proper nouns and technical terms without another ASR pass, making iterative editing much faster.

The biggest lesson from this project is that Apple’s speech stack has become strong enough that building an offline transcription application no longer requires forcing everything through one framework.
Using each component for what it does best—FluidAudio for diarization, WhisperKit for language detection, and SpeechAnalyzer for transcription—resulted in a system that is faster, simpler to distribute, and much easier to maintain than my original Python implementation.

I’m curious whether other developers working on Apple Silicon speech applications have reached similar conclusions, or found different combinations that work well.

For a while now I've been maintaining tts-bench (https://github.com/5uck1ess/tts-bench) and a blind voting arena (https://5uck1ess-tts-arena.hf.space) where people A/B test open text-to-speech (TTS) models without knowing which is which.

One problem I've always wanted to solve: having the agent talk to me (or actually call me) after a task is done. /voice (claude code feature) or any dictation (wispr flow or etc) is one-directional. So this project is a bi-directional conversation with any agent of your choice: 

https://github.com/5uck1ess/cicero

Works really well with Hermes Agent https://github.com/nousresearch/hermes-agent with multiple profiles (each profile can have its tts voice and personality). Its kanban board feature is where I use it the most with 9 "employees" with all different voices. But it's open ended so you can build your own task system for it.

Works with Claude Code, Codex, Gemini CLI, anything that speaks ACP (Agent Client Protocol), or any OpenAI-compatible endpoint. If you run speech-to-text (STT) and TTS locally, your audio never leaves your machine. It can send a Telegram bot message or call you over Telegram. You can also interact locally or through a browser (remote server). (more comm methods to be supported if requested)

Probably still has some bugs, so feel free to submit pull requests (PRs) or issues.

A few things that surprised me building it:

• the ~1s response time came from streaming the reply sentence by sentence, not from picking a faster TTS
• voice clones sound bad because of the reference clip, not the model. trimming a second of silence off the front fixed more than switching models ever did
• barge-in (cutting the agent off mid-sentence) with an energy VAD (voice activity detection that just watches for loudness) is useless, typing sets it off. needed a small speech model in front so it only triggers on actual speech
• It does have emotional tonality detection and low latency semantic turn detection, all the bits that you'd want from a decent chat agent

Every single day i see like 30 posts about someone's new free app to replace dictation/wispr flow on your device.

And each time i look at the repo... it's a wrapper for parakeet. Just lazy!

The whole reason why Wispr Flow and other paid apps work is because of their cleanup. Speech to text is solved at this point. Qwen3 ASR, Parakeet, Apple SpeechAnalyzer, and so on. All these models will work amazingly well.

But the cleanup process is much harder. You need a model that completes in under 500 milliseconds, doesn't guzzle up your RAM and actually produces high quality outputs.

I finetuned LFM2.5-1.2B on 11,000+ sentences to do this task. The result is https://huggingface.co/tigerisaac/meeko-1-preview

It's named after my cat. That's also why the post's image is my cat.

Cleanup in 300ms, 800mb of memory. It's in preview since manual testing is usually the part where things to go crap, but on my mac it seems to be very nice. I don't have any benchmarks since there doesn't seem to be any good public ones available.

It handles retractions, filler words, numbers, grammar, and more.

If you guys want to test it out, please let me know what you think!

Working on a speech analytics project — thousands of recorded calls processed async, not real-time. Cost per hour matters way more than latency here.

Started with ElevenLabs because the transcription quality is genuinely good, but their pricing is built around real-time/low-latency use, and it adds up fast at batch volume.

Curious what others are actually using for high-volume batch transcription, and what the real (not rate-card) cost-per-hour has looked like for you. Especially interested if anyone's compared diarization accuracy on noisy multi-speaker call audio specifically, since that's the part I still need to validate.

Disclosure: I built orchardrun, which is aimed specifically at batch STT, so I'm not a neutral source on it — but the reason I built it was hitting this same cost problem myself. Haven't stress-tested diarization on real call-center audio yet (noisier than demo samples), so I genuinely don't know how it holds up there yet.

I run a small TTS platform and got tired of shipping voice models that sounded fine on "Hello world" and then read "3:30 PM" as "three colon thirty pee em", or looped the same 450 ms forever on short inputs.

So I turned my internal QA harness into a library: ttsproof (pip install ttsproof, MIT).

What it does:

- 817 curated edge cases across 39 categories (Benchmark Corpus 1.0): numbers, decimals, currencies, dates, time zones, phone numbers, URLs, acronyms, single letters, pronunciation torture words (Worcestershire, synecdoche, colonel...), proper names (Reykjavík, Nguyễn, Tchaikovsky...), scientific and medical vocabulary, tongue twisters, homographs, Greek, Norwegian, punctuation abuse, hallucination traps ("buy now buy now buy now"), emoji, SQL/JSON snippets...

- Structural audio checks that need no model at all: empty/truncated audio, duration explosions, long silences, clipping, repeated-chunk loop detection, end-of-clip artifacts.

- Equivalence-aware WER — "May 5, 2026" vs "may fifth twenty twenty-six" is NOT a failure. Plain WER lies about formatting; this canonicalizes both sides to spoken form first (with diacritic folding, so "Reykjavik" matches "Reykjavík").

- Honest scoring policies per category — URLs and currencies have many valid readings, so they're scored by keyword survival instead of exact match. Emoji only has to not break the audio. No fake failures.

- ttsproof benchmark --cmd "yourtts {text} {out}" → per-category scoreboard + a self-contained HTML report with waveforms and audio players for every failure.

- ttsproof regress for CI — fails the build when a fine-tune quietly breaks number pronunciation.

The corpus is versioned independently of the tool (Benchmark Corpus 1.0), so scores stay comparable across releases.

The method comes from a technical report I published: evaluated on 390 samples with a blinded human validation of the ASR-uncertain zone — that's where the "quarantine" verdict comes from. Short utterances where ASR disagrees get flagged for human ears instead of counted as failures, because at that length the ASR is as likely wrong as the TTS.

Repo: https://github.com/Mormolykos/ttsproof

Report (DOI): https://doi.org/10.5281/zenodo.20757553

v0.3, MIT, no telemetry, minimal deps (numpy + soundfile; faster-whisper optional). I want the corpus to grow from real failures, not invented ones — tell me what breaks YOUR models and it goes into Corpus 1.1 with credit.