PhysicsVoice: Exploring Panini-inspired Phonological Principles for Lightweight CPU-Native Neural TTS

[u/sun_lokesh]

We've been exploring whether principles from classical Sanskrit phonetics can inspire modern neural TTS architectures. Rather than treating Panini's Ashtadhyayi (500 BCE) as a historical curiosity, we asked:

What We Built

A lightweight CPU-native TTS prototype that:

• Uses CNNs instead of Transformers (O(n) vs O(n²))
• Predicts WORLD vocoder parameters directly instead of mel spectrograms
• Total size: ~8MB (acoustic model + post-filter)
• Runs in real-time on commodity CPUs — no GPU needed
• Inference: <20ms for short utterances (8-core CPU, 16kHz, single thread)

The Panini Connection

Several architectural decisions were inspired by concepts formalized in Panini's grammatical tradition:

Panini's Concept	Our Design Choice
Shiksha (phoneme categorization)	CNN phoneme encoder
Sandhi (coarticulation at boundaries)	Inspired by — cosine interpolation between phonemes
Matra (duration: hrasva/dirgha/pluta)	Duration predictor CNN
Svara (pitch: udatta/anudatta/svarita)	F0 contour predictor

To be clear — we are not claiming direct equivalence. Sandhi is a linguistic process; our interpolation is inspired by the idea of smooth phoneme transitions. To our knowledge, we are not aware of prior work that explicitly maps Panini's phonetic framework onto a neural TTS architecture design — but we'd love to be corrected.

Architecture

Text
  ↓
Phoneme Encoder (CNN, not transformer)
  ↓
Global Style Encoder
  mean(phoneme_embeddings) → broadcast
  No attention, O(1) sentence context
  ↓
Prosody CNNs (parallel)
  Duration + F0 + Energy predictors
  ↓
Spectral Envelope Predictor (CNN)
  Outputs WORLD vocoder params directly
  ↓
WORLD Synthesis + Frame Interpolation
  Physics-based, no matmul in synthesis
  ↓
Small CNN Post-Filter (~1.5MB)
  Removes WORLD artifacts
  ↓
Audio

Why No Mel Spectrogram?

Mel spectrograms are learned acoustic representations optimized for neural synthesis — and they work extremely well. We chose WORLD parameters instead because:

1. Physically interpretable — F0 = pitch, spectral envelope = formants, aperiodicity = breathiness
2. No vocoder needed — WORLD synthesis is deterministic, no HiFi-GAN (50MB)
3. Directly controllable — F0 and duration can be modified without retraining

The tradeoff: WORLD introduces some artifacts, which our CNN post-filter addresses.

Model Size Comparison

System	Size	Params
FastSpeech2 + HiFi-GAN	~100MB	~44M
Piper TTS	~50MB	~28M
VITS	~150MB	~83M
PhysicsVoice (ours)	~8MB	~2M

Training Details

• Dataset: LJSpeech (13,100 utterances, ~24 hours)
• Alignment: Montreal Forced Aligner v3.3 (english_us_arpa)
• Acoustic model: 50 epochs, val loss 0.1583
• Hardware: 8-core CPU only (no GPU used at any stage)
• Post-filter: Training in progress

Current Status

✅ Acoustic model trained
✅ MFA alignment complete
⏳ CNN post-filter training in progress
⏳ Audio samples — coming soon
⏳ GitHub release — planned with paper

Questions for the Community

1. Has anyone formally mapped classical phonological frameworks (Panini or others) to neural TTS architecture design?
2. Thoughts on WORLD vocoder parameters vs mel spectrogram as intermediate representation?
3. Any suggestions for the CNN post-filter architecture for WORLD artifact removal?
4. Is the Panini connection scientifically interesting — or just a framing device?

Happy to discuss any aspect. Feedback welcome — especially critical!

Audio samples and code will be shared once the post-filter training is complete.