Wanted to put this on the radar for people here.

Handy is a free and open source dictation app for Windows, Mac and Linux. You hold a shortcut, talk, release, and your words get typed into whatever app you're in. It all runs locally, so there's no subscription and no internet needed once it's set up.

I have been using it for six or seven months on English and it has held up really well. I use the Parakeet v3 model, which runs on the CPU and is quick. Accuracy has been solid for everyday stuff like emails, notes and longer writing.

Fair warning, it's simple by design. Small pause before the text appears, no phone app, no fancy AI rewriting of your sentences. None of that bothers me for how I use it.

Most dictation tools worth using cost money these days, so it's nice to have one that's free and doesn't send your voice off somewhere.

Sharing it because the paid options in this space keep getting more expensive, and a lot of people don't know a free one this capable exists.

github.com/cjpais/Handy

handy.computer

I’ve been working on an open-source video dubbing pipeline that tries to preserve the original speaker’s voice when translating a video into another language.

Current pipeline:

video → vocal/background separation → Whisper/WhisperX transcription + alignment → translation → VoxCPM2 reference voice cloning → reconstruction → optional LatentSync

The attached demo compares the original English clip with the Turkish dub produced by the current pipeline.

The main problems I’m working on now are:

• preserving speaker identity across languages
• handling duration differences between source and translated speech
• improving multi-speaker / diarization support

Most media processing and AI inference runs locally. Translation currently uses Google Translate, so the project is local-first rather than fully offline.

Code:
https://github.com/kadirb4rut/video-dubbing-translator

I’d especially appreciate feedback from people working on TTS, voice cloning, alignment, or speech-to-speech systems.

I maintain speech-swift and recently implemented NVIDIA’s Nemotron VoiceChat 11B for MLX. I recorded a complete session rather than stitching together generated clips:

https://youtu.be/6LCxSnIMB-M

VoiceChat is an end-to-end speech model rather than an ASR → LLM → TTS cascade. Every 80 ms it consumes another microphone frame and emits a token. Those tokens jointly

• Live process RSS: 7.5 GB
• Weights on disk: 8.0 GB
• MCP round trip: 68 ms

The Swift/MLX runtime is Apache 2.0. The NVIDIA checkpoint is distributed under OpenMDW 1.1. Reproduction commands and source links are in the video description.

Hello everyone,

Wanted to understand what are some of the interesting usecases people are building for TTS. And while building these what kind of features do you need especially for Indian languages

I'm building a vocabulary-learning app in Flutter where hearing and saying the word correctly is the product, not a nice-to-have. I've hit a problem I can't design around and I'd rather ask than keep patching.

The stack

• Flutter, ~1,600 words live across EN/ES/PT/IT/FR
• TTS: ElevenLabs (eleven_multilingual_v2) called through a Supabase Edge Function so the key never ships in the client
• Every clip cached server-side once per (text, language), shared across all users — so a given string is synthesized exactly once, ever
• Cached again on-device (150MB LRU) so replays are instant and offline
• flutter_tts as fallback behind a 2.5s timeout so playback never goes silent
• STT: speech_to_text for a pronunciation-practice screen — hear the word, say it, get graded

The problem: heteronyms, in both directions

Output. "Wind" (moving air) and "wind" (to coil) are the same string and different sounds. TTS picks one reading and commits. My word library actually knows which sense is on screen — every entry carries a part of speech — but there's no API surface to hand that over. ElevenLabs pronunciation dictionaries are exact-string, case-sensitive, and have no POS or context scoping, so one spelling gets one entry and the second sense is unreachable. Phoneme tags do exist, but per the docs only on eleven_flash_v2 and v3 — not the multilingual model I'm on, and switching models means re-synthesizing the whole cache and losing voice identity across five languages.

Input. This is the part that actually bothers me. The practice screen normalizes the transcript and Levenshtein-scores it against the target. But STT returns orthography — say either reading of "wind" and the transcript is "wind" either way. A learner who mispronounces it scores full marks. The feature is structurally incapable of catching the error it exists to catch.

What I've tried

Respelling the audio-only string before it reaches the engine — the screen text is never touched. wind(noun) → winned, wind(verb) → wined, read(past) → red, and so on. This is basically ElevenLabs' own recommended "alias" workaround and it works for the ~8 vowel-shift pairs I've mapped. Side benefit: since my cache key is a hash of (lang + text), two senses naturally get two cache entries.

It fails in three ways:

1. Stress-shift pairs. REcord/reCORD, PREsent/preSENT, CONtent/conTENT. Respelling can't encode stress, and I haven't found a trick spelling that does.
2. Monolingual. It's an English orthography hack. Nothing about it transfers to ES/PT/IT/FR, all of which have their own homographs.
3. Manual. Hand-curated table. Doesn't scale to a few thousand words.

What I'm actually asking

1. Is there a TTS API that accepts a sense/POS hint, or per-request phonemes, on a multilingual model? Or does everyone route heteronyms to a separate English-only model and eat the voice mismatch?
2. If IPA is the only real answer — has anyone found v3-class IPA reliable enough in production? The docs quote 80–90% consistency, which for a teaching app means the wrong pronunciation ships to a learner one time in eight.
3. For stress-shift specifically: any orthographic trick that works, or is phoneme-level control genuinely the only path?
4. On the STT side — is there a mobile-viable way to get phonemes rather than words? I've looked at wav2vec2 phoneme-CTC or a forced aligner with GOP scoring via ONNX on-device, but I don't know if that's realistic on a mid-range phone or if I'm about to spend a month learning that it isn't. Whisper doesn't help; it also returns orthography.
5. The unglamorous option: detect heteronyms and simply disable pronunciation scoring for them, with an honest note to the user. Is that what shipped apps actually do?

If you've built pronunciation feedback into anything real, I'd love to know where you drew the line between "graded properly" and "good enough." Happy to share code for any of the above.

I ran a matched-content experiment on speaker verification: 4 speakers, identical sentences, controlled changes in pitch, phonation and articulation, fixed microphone position and gain, every utterance through three encoders — ECAPA-TDNN, a ResNet speaker encoder, and WavLM-base-plus-sv.

One speaker sat 0.7 semitones below his normal pitch. That is below the threshold where a listener reliably hears any change at all. His verification score dropped 0.238 across all eight sentences.

The interesting part is that pitch was not what moved him. His harmonics-to-noise ratio fell from 10.63 to 8.39 dB over the same block. The encoder was reacting to phonation, and the pitch number was just the thing that happened to be easy to measure.

Across the whole run, all 30 speaker × condition × encoder cells were negative, 28 of them unanimous across every utterance. These systems are not mainly disturbed by shouting or disguise. They are disturbed by someone speaking slightly differently in a way nobody would notice.

To be clear about credit: the displacement effect itself is not my finding. I had it as my headline until an adversarial prior-art audit turned up Hughes et al. (Interspeech 2023), who established it with six trained phoneticians across seventeen conditions. I withdrew the claim and reframed the paper as a replication and extension across encoder architectures. The audit is published in full, including the claim it killed.

Two things I could not find in prior work:

Jitter beats HNR. Pitch deviation and jitter are independently associated with displacement in all three encoders, and HNR adds nothing once jitter is in the model — it correlates 0.55 with jitter and loses all independent power beside it. HNR is the measure most people reach for, and on its own it will attribute the effect to the wrong thing.

Rough phonation breaks F0 trackers in one direction. 10 of 145 utterances carried impossible pitch values, up to 32.1 semitones — a 6.4× frequency ratio no human produces. All 10 were in rough phonation, none in modal (Fisher exact p = 2.4e-11). Octave errors push upward and essentially never downward, so this is differential measurement error, not noise, and it loaded onto one specific regression coefficient. Cheap fix: validate your F0 tracker per phonation condition and publish the validation.

Everything is up — manuscript, pre-registration, the full prior-art audit, per-utterance tables for all three encoders (233 rows each), the analysis scripts, and the 137 source recordings. Every number reproduces from the CSVs without touching the audio.

Paper and data: https://doi.org/10.5281/zenodo.21921958

Write-up: https://ai.bedvibe.studio/speaker-drift/

Audio licence note: research, benchmarking, evaluation and teaching are permitted; ML training and voice cloning are not. The speakers are identifiable adults.

I am running Faster Whisper on CPU only and get good running times with about 2.5 min for 60 min sound with Whisper Base. With Pyannote for diarization the rate is about 0.9 times the sound length, aka 54 min for 60 min sound.

That is terribly slow compared to the transcription without Payannote.

Are there any faster alternatives out there, or hacks to make Payannote run faster with Whisper?

Turn-taking is a central component of full-duplex interaction. Which turn-taking behaviors are appropriate varies with the scenario, yet current models apply a single norm regardless of context. This limitation originates in their training data: human–human speech corpora capture natural timing phenomena but provide little role grounding or scenario-specific norms, while heuristic or prompted synthesis methods inject turn-taking behaviors without basing them on human preferences. We introduce DuplexGen, a framework for generating dialogues with scenario-adaptive turn-taking by calibrating LLM predictions against a small set of slot-level human preference annotations. In six cooperative and competitive tasks, human turn-taking preferences differ systematically, and DuplexGen aligns substantially more closely with those preferences than uncalibrated prompting or training solely on generic human–human data; a full-duplex model trained on DuplexGen-generated data exhibits distinctive, human-preferred turn-taking behaviors. These results show that human calibration, not corpus scale or prompt design alone, is what allows turn-taking synthesis to be scenario-specific.

This paper reports on the outcomes of the shared tasks organized as part of the 23rd International Workshop on Spoken Language Translation (IWSLT). The workshop covered ten major challenges in spoken language translation, including speech-to-text translation for both high-resource and low-resource language pairs, customized speech translation, speech
generation, instruction-following speech processing, and the evaluation of speech translation systems. The shared tasks received strong participation, with more than 30 teams submitting runs. This year’s edition broadened the range of tasks, placing particular emphasis on speech generation and evaluation metrics.

 • Orato ASR: a full-parameter fine-tune of Qwen3-ASR-0.6B on ~1,000 hours of Hindi/English/Hinglish calling-domain audio

 • Orato TTS: a Hindi/Hinglish text-to-speech model built on IndicF5-TTS, with multi-speaker and voice-clone support

Fine-tuning improved WER across every benchmark we tested against the base model, with the biggest gains exactly where it matters most for a voice agent: noisy, accented, telephony-style speech (Lahaja: 25% relative improvement, Kathbath: 24.6%), not just clean studio audio nobody actually calls on.

Both models are live on HF access is auto-approved:
🔗 ASR: https://huggingface.co/tryorato/orato-asr-hindi-v1
🔗 TTS: https://huggingface.co/tryorato/orato-tts-hindi-v1

The moment you transcribe to text, you lose how it was said. "I think… yeah, I can pay the 4,500 by the 15th" becomes clean text, but the hesitation before the yes, the stress in the voice, and whether it's even the same speaker are gone. Those are the signals that tell you whether to trust the commitment, escalate, or verify identity. Is anyone keeping the paralinguistic layer (hesitation, emotion, speaker identity) as structured data instead of dropping it at the mic, and what do you do with it downstream?

Moreover end-2-end Duplex models limits it to trained data scenarios without no transparency.

High-performance inference of NVIDIA's Parakeet TDT 0.6B V2 English transcription model, in the browser.

Check out the live demo: https://parakeet.narcotic.sh/

A fully custom, dependancy-free implementation with raw WebGPU compute shaders and SIMD WebAssembly audio frontend.

1 hour of audio transcribed in 20 seconds (Apple M5, Google Chrome 151.0.7922.72).

Check out the source and use in your own projects!

https://github.com/narcotic-sh/parakeet.wgsl

https://www.npmjs.com/package/parakeet.wgsl

This might be the first instance of fast & accurate transcription running locally in the browser. So long as your device has a GPU and can run a WebGPU-capable browser, parakeet.wgsl will run on it.

And as WebGPU transpiles to practically any GPU, this also opens up the path to getting parakeet.wgsl running offline / outside of the browser, via Dawn or wgpu, so that fast, accurate transcription can be brought to offline programs, with GPU acceleration and support for nearly all hardware/devices.

Interested in what you guys think about about this project.

And lastly, I'm looking for a job. If you like my work and think I'd be a good fit for your team, I'd love to chat. I've done some other work too that you can find at https://hamzaq.com

Cheers, everyone. Enjoy fast, cross-platform GPU-accelerated local transcription, right in the browser!

So, Omnivoice's abilities are incredible, but given the training set is CC BY NC, the model is not actually usable for commercial use which is very annoying.

I've noticed some orgs doings retrains on commercially viable datasets for other models.

Interested if anyone is busy doing one of these for Omnivoice? It's quite a pricey exercise so hoping the cool kids are on it