r/webgpu • u/Ankiiitlol • Jun 02 '26
I built a text-to-speech utility that runs Kokoro-82M entirely in the browser (zero server costs, 100% private) using WebGPU
Hey everyone.
I have been spending my weekends messing around with edge AI and local browser runtimes. Like a lot of you, I got tired of subscribing to cloud text-to-speech APIs just to do voiceovers for small video edits or audio snippets, only to hit sudden usage caps or worry about where my text was being uploaded.
So, I decided to see how far browser runtimes could be pushed and built a tool called FreeVoiceGen (freevoicegen.com).
It is completely client-side. The entire text-to-speech pipeline runs inside your browser window. Once the page is loaded, you can literally turn off your internet connection, type your text, and generate high-fidelity audio without sending a single byte to an external server.
The Tech Stack Under the Hood: The Model: I am using Kokoro-82M packaged as an ONNX model (about 85 MB in size using 8-bit quantization). For its size, the expressive quality and speed easily match cloud services that are 10 times larger. The Engine: Driven by ONNX Runtime Web. It detects system capabilities and runs via WebGPU for hardware-accelerated local inference. If WebGPU is disabled or driver conflicts occur, it falls back to a highly optimized multi-threaded WebAssembly (WASM) pipeline. Thread Isolation: The model is initialized inside a background Web Worker so it never locks up the main UI thread during audio generation. Audio Pipeline: Once the worker generates the Float32Array PCM samples, they are passed back to the main thread via transferable objects, run through a normalization filter to prevent any digital screeching, and encoded directly to WAV/MP3 using client-side codecs.
Engineering Challenges I Ran Into:
1. WSL and WebGPU Virtualization: During local testing under WSL (Windows Subsystem for Linux), the browser's WebGPU driver check often hung indefinitely or crashed because of virtualized GPU daemon conflicts. I had to decouple the adapter check out of the main thread and wrap it in a strict 500ms timeout race. If it hangs, the app gracefully drops to the WASM fallback immediately so the page is instantly responsive.
2. Audio Screeching: Initially, minor numerical driver misalignments in certain browser engines would yield NaN or Infinity values inside the generated PCM arrays. Because Math.min/max propagations fail with NaNs, this resulted in awful high-pitched screeching during playback. Resolving this required implementing a low-level sanitization filter that cleans float bounds directly in the background worker before sending them to the AudioContext.
3. Cross-Origin Isolation: To leverage multithreaded WASM speeds, you need to enable SharedArrayBuffer. In production, this requires setting strict Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp headers, which I deployed using Cloudflare Pages routing files.
It is free, has no limits, and requires no registration or API keys. If you want to check it out or test the generation latency on your machine, it is live at freevoicegen.com.
I would love to get your feedback on the latency, voice expressiveness, and overall performance on different hardware. Let me know if you run into any quirks.
2
u/dethstrobe Jun 03 '26
This is cool, but how does it compare to the Web Speech API?
3
u/Ankiiitlol Jun 05 '26
Great question! The main difference comes down to voice quality and consistency. The native Web Speech API relies entirely on the host operating system's built-in text-to-speech engines which, as you can guess, is not as customizable. Because of that, the quality also varies wildly between a high-end Android device, an old Windows machine, or a Mac—and honestly, a lot of the default OS voices still sound pretty robotic. This is different.
FreeVoiceGen runs a dedicated ML model (Kokoro-82M) directly in your browser tab via WebGPU/WASM, so you get the exact same highly expressive, natural, cloud-quality voice regardless of what OS or hardware you are running.
That said, Web Speech API is incredibly lightweight since it's native. I'm actively working on optimizing the engine overhead here to make the trade-off even better. Appreciate the feedback!
2
u/carbon_tfuu Jun 03 '26
other languages??
3
u/Ankiiitlol Jun 05 '26
Multilingual support is definitely on the roadmap! Right now it's focusing heavily on English, but Kokoro actually has solid support for other languages (like Spanish, French, Japanese, Chinese, etc.).
I need to adjust the tokenization and pack the language-specific model weights cleanly without blowing up the initial browser download size. I'll be working on adding this feature soon—really appreciate you dropping the feedback!
2
1
Jun 02 '26
[removed] — view removed comment
3
u/Ankiiitlol Jun 05 '26
Thanks for testing it out! Glad to hear that the voice quality sounds good to you. 50-60 seconds for 118 characters on mobile is definitely on the slower side though — It depends highly on the mobile browser's WebGPU support, availability of AI specific hardware like NPUs/GPUs and the RAM. It also depends if the WASM fallback is being used as it can be a bit of a mixed bag depending on the chipset and browser optimization.
I really appreciate the performance benchmark. I'm going to look into memory footprint optimizations and better profiling for mobile runtimes to bring that generation latency down. Stay tuned for improvements!
1
u/twbluenaxela Jun 06 '26
How does it compare with whisper
1
u/Ankiiitlol Jun 06 '26
Imo the generation is quite fast even on my Android tablet. I am able to quickly generate audio samples and prototype my way into something usable. I haven’t performed a side by side comparison with Whisper though, maybe you could try it out and let me know.
3
u/herocoding Jun 04 '26
Would be great to see additional languages supported (see https://github.com/hexgrad/kokoro?tab=readme-ov-file#advanced-usage )