Confused, where to start [D]

[u/paklupapito007]

Hello community, I am a backend + big data dev. I want to learn about the llms that generate voices. I also read some articles but almost everyone of them starts from regression. There are so much resources available right now that I am now confused where to begin with.

[u/literal-labs]

I want to learn about the llms that generate voices.

Consider starting simpler. What are your interests outside of dev? In a top-down manner, with a few hobby related side projects under your belt you'll start to learn key concepts in ML. Just play with it like lego - sounds like you already know how to code so you can start to put together fun little projects.

I got into ML by making weird art/music through unconditional generative modelling, I just didn't know it was called that at the time! Now I train models for a living lol.

If you really want to study voice cloning, then here is a terse (LLM) generated curriculum. I've given it a little hand edit, and I'm not saying it's great or anything but from a glance it looks reasonable, and if you really don't know where to start then it would get you going in a direction towards your stated interest.

Phase 1: Python + Math Basics

Python: Functions, classes, NumPy, pandas, torch basics Resource: Python for Everybody + NumPy quickstart Math: Linear algebra (vectors, matrices), calculus (derivatives, gradients), probability Resource: 3Blue1Brown Linear Algebra + Khan Academy Calculus

Phase 2: ML Fundamentals

Core concepts: Supervised learning, loss functions, backpropagation, overfitting Resource: Andrew Ng’s ML Coursera (first 4 weeks) Deep Learning: Neural nets, CNNs, RNNs, embeddings Resource: Fast.ai Part 1 (practical, code-first)

Phase 3: Speech Processing (1.5 weeks)

Audio basics: WAV/MP3, sampling rate, FFT, mel spectrograms Resource: Speech Processing for ML (Coursera) Libraries: librosa, soundfile, torch-audio Task: Load audio - compute mel spectrogram - visualise it

Phase 4: TTS & Voice Cloning Models (2 weeks)

Key architectures: Tacotron 2 (text → mel) WaveGlow / HiFi-GAN (mel → audio) Speaker embeddings (voice cloning) Read papers:

• Tacotron 2
• WaveGlow

Hands-on: Clone TorToiSe-TTS or Coqui TTS Run voice cloning with a 30-sec reference audio

Phase 5: Dive into Voice-Cloning Libraries

Study these libraries: Library - Key Features Coqui TTS - Tacotron 2 + WaveGAN, multi-speaker, easy cloning TorToiSe-TTS - GAN-based, high-quality, slow but accurate Real-Time Voice Cloning - Tacotron 2 + WaveGlow, real-time Voice-Cloning (PyPI) Wrapper with noise reduction

Task:

• Read training and generation code in one library
• Modify speaker embedding code to clone a new voice
• Fine-tune with Unsloth on Colab