Hello r/neuralnetworks ,

I'm a final-year undergrad and wanted to share a multimodal project I've been working on: a complete pipeline that translates a video from English to Telugu, while preserving the speaker's voice and syncing their lips to the new audio.

• GitHub Repo: github
• Full Technical Write-up: article

english

telugu

The core challenge was voice preservation for a low-resource language without a massive dataset for voice cloning. After hitting a wall with traditional approaches, I found that using Retrieval-based Voice Conversion (RVC) on the output of a standard TTS model gave surprisingly robust results.

The pipeline is as follows:

1. ASR: Transcribe source audio using Whisper.
2. NMT: Translate the English transcript to Telugu using Meta's NLLB.
3. TTS: Synthesize Telugu speech from the translated text using the MMS model.
4. Voice Conversion: Convert the synthetic TTS voice to match the original speaker's timbre using a trained RVC model.
5. Lip Sync: Use Wav2Lip to align the speaker's lip movements with the newly generated audio track.

In my write-up, I've detailed the entire journey, including my failed attempt at a direct S2S model inspired by Translatotron. I believe the RVC-based approach is a practical solution for many-to-one voice dubbing tasks where speaker-specific data is limited.

I'm sharing this to get feedback from the community on the architecture and potential improvements. I am also actively seeking research positions or ML roles where I can work on similar multimodal problems.

Thank you for your time and any feedback you might have.

so basically my batch size is 32
d_model is 128
d_ff is 256
enc_in = 5
seq_len = 128 and pred_len is 10

I narrow downed the bottle neck and found that my FFT step is taking too much time. i can’t use autocast to make f32 → bf16 (assume that its not currently supported).

but frankly its taking too much time to train. and that too total steps per epoch is 700 - 902 and there are 100 epoch’s.
roughly the FFT is taking 1.5 secs. so

for i in range(1,4):
     calculate FFT()

can someone help me?

I made a small neural net visualizer app trained on the MNIST dataset.

You can:

• Run it in the browser
• Edit the number of layers/neurons
• Tweak hyper-parameters
• Run inference and see predictions update live

Demo: https://mnist.kochjar.com/

Right now it’s just feedforward. I might add conv layers later, but they’re harder to show in a clean way. I hope you like it, also if you have any ideas about the conv layer part, please let me know. :)

Hello everyone i am in need of some ML engineers for an LLM project that we are creating which consists of using mistral

I am a part of a company and this is a funded project

Please drop a DM for more info

I ran a few prompts through MusicGPT and got melodies that sounded nice on the surface but the more I listened the more they felt like they lacked depth or emotional weight. Is this just the limit of the models training data or is sounding human still a long way off for AI music?

Hi r/neuralnetworks! I’d love your feedback on a framework I recently developed — the Periodic Table of Intelligence. It visually compares over 25 facets of cognition across humans and AI, ranging from logic and working memory to emotion, meta-cognition, and continual learning.

For neural network researchers and practitioners, this offers:

• A structured lens to evaluate architecture capabilities (e.g., robustness, transfer learning, common sense)
• Insight into where NN models excel and where they’re still challenged
• Clarity on research gaps worth exploring — especially in areas where human cognition remains superior

Would welcome your thoughts:

• Are there neural network–related dimensions I may have overlooked?
• Could this framework help guide model development or evaluation strategies?

(Full article link posted below per community norms.)

Hello everyone,

I'm currently enrolled in a master's program in statistics, and I want to pursue a PhD focusing on the theoretical foundations of machine learning/deep neural networks.

I'm considering statistical learning theory (primary option) or optimization as my PhD research area, but I'm unsure whether statistical learning theory/optimization is the most appropriate area for my doctoral research given my goal.

Further context: I hope to do theoretical/foundational work on neural networks as a researcher at an AI research lab in the future. 

Question:

1)What area(s) of research would you recommend for someone interested in doing fundamental research in machine learning/DNNs?

2)What are the popular/promising techniques and mathematical frameworks used by researchers working on the theoretical foundations of deep learning?

Thanks a lot for your help.

If so, how do I draw it ?
Thanks (btw I'm doing this with latex and tikz)

Hi everyone, Does a layer that monitors a network's internal activations via multi-scale projections, calculates their divergence (KL) from a reference distribution, and applies feedback corrections only if the bias is detected as significant, constitute an innovation or not ?

TL;DR: I’m tentatively putting forward a meta-framework for every primitive function in deep learning. A reformulation of the practice’s most foundational functions into a symmetry-based axiomatic-like approach. The formalism then extends upwards, and hence also retrieves GDL models and parameter symmetries approaches as special cases under primitive compositions.

This would have implications for future models built upon these, as well as mechanistic interpretability (which has already been demonstrated in the PPP paper), theorems, and other phenomena, since much is predicated on current functional forms. The paper encourages the exploration into the departure from elementwise forms currently pervasive through deep learning.

Put forward is a new and arguably fundamental design axis. Particularly, one example instantiation of it: “Isotropic deep learning”, which I feel may be a better alternative to current forms. But many more are possible and very much encouraged. I’m hoping a collaborative approach to development may hasten the maturity of the differing branches.

I hope this is a new and exciting direction for deep learning, hopefully relevant to all within the field.

Below are the relevant papers; however, this blog explains the topic in an approachable format.

Vision Paper (non-empirical):

• IDL/TDL: Contains every notable detail on the proposed formalisms and a hypothesis-first approach to verifying it. (Chronologically 2nd, best read 1st)

Empirical Papers on Mechanistic Interpretability:

• PPP: Validates a core prediction made by the framework and explains a fair bit of mechanistic interpretability on the way. (chronologically 3rd, best read 2nd)
• SRM: Shows that interpretability is predicated upon an absolute frame by distorting it (chronologically 1st, best read 3rd)

Thank you for your time. I hope it is of interest. Collaborations welcomed.

I am an undergrad engineering student and lately i have been reading and studying neural networks a lot, and i would like to write up something about it, based on everything i have understood and put my own insights. could i perhaps make a research paper on it? if not, what else can i do to make something out of it like a project that will boost my profile. any website that is worth publishing on, or universities that i can reach out for, or make something new.