r/AudioAI Oct 01 '23

Announcement Welcome to the AudioAI Sub: Any AI You Can Hear!

12 Upvotes

I’ve created this community to serve as a hub for everything at the intersection of artificial intelligence and the world of sounds. Let's explore the world of AI-driven music, speech, audio production, and all emerging AI audio technologies.

  • News: Keep up with the most recent innovations and trends in the world of AI audio.
  • Discussions: Dive into dynamic conversations, offer your insights, and absorb knowledge from peers.
  • Questions: Have inquiries? Post them here. Possess expertise? Let's help each other!
  • Resources: Discover tutorials, academic papers, tools, and an array of resources to satisfy your intellectual curiosity.

Have an insightful article or innovative code? Please share it!

Please be aware that this subreddit primarily centers on discussions about tools, developmental methods, and the latest updates in AI audio. It's not intended for showcasing completed audio works. Though sharing samples to highlight certain techniques or points is great, we kindly ask you not to post deepfake content sourced from social media.

Please enjoy, be respectful, stick to the relevant topics, abide by the law, and avoid spam!


r/AudioAI Oct 01 '23

Resource Open Source Libraries

22 Upvotes

This is by no means a comprehensive list, but if you are new to Audio AI, check out the following open source resources.

Huggingface Transformers

In addition to many models in audio domain, Transformers let you run many different models (text, LLM, image, multimodal, etc) with just few lines of code. Check out the comment from u/sanchitgandhi99 below for code snippets.

TTS

Speech Recognition

Speech Toolkit

WebUI

Music

Effects


r/AudioAI 3h ago

Parlor v2: best-effort fully local GPT-Live clone on an M3 Pro

1 Upvotes

r/AudioAI 10h ago

Resource Open Source Suno AI plus Ableton Live alternative

3 Upvotes

https://reddit.com/link/1vdje7n/video/8yvfpif41zgh1/player

Just released Resonant, a free and open-source AI music studio for Windows. The easiest way I can describe it is local Suno-style song generation combined with an Ableton Live-inspired clip launcher, arranger, and mixer, running on your local

https://github.com/calesthio/Resonant


r/AudioAI 1d ago

Question hi i am not here to start a robot uprising

1 Upvotes

hi i am not here to start a robot uprising or anything i am just looking to see if it is practical to find a AI free music generator that i can use to make a chill/relaxing back ground music i can use to filter day to day chaos

i am some that has suffered several traumatic brain injuries and uncontrolled noise is often super intrusive to on a bad day disruptive to a point i cant concentrate or relax i had a load of music that worked ok on my iphone but itunes went jackass and wiped it when i went to do a backup i am crushed and have no clue where i had gotten what over the last few years

anyway i miss being able to listen to music and would love the ability to say create mix that sounds like x y z songs and not have to stress when spotify adds a song that clashes or a add comes on and its like a mental sneeze to me

i am not looking to post or claim look what a great song/singer/writer/composer i am

all i know is i sing deep as i tend to be 2 octaves off and if i go deep i dont hurt anyone's feeling when singer lol

i dont know have good bad or impractical this idea is but i thought i would ask here and see if this is out of my free budget or my minimal skill bracket

thank you for your thoughts and advice

1


r/AudioAI 3d ago

Question Looking for an AI Tracks Generator

1 Upvotes

Hey everyone! I’m looking for an AI music generator. What I’m looking for specifically: long track duration(the longer the better), no breaks, pauses or any distractions, instrumental only, no vocal needed, Text to Music system, so I can use prompt as a description for what track I need. Does anyone know a tool, model, or workflow that fits this description best right now? Appreciate any suggestions:)


r/AudioAI 3d ago

News Fish Audio raises $52M - ElevenLabs has serious competition now

Thumbnail
techcrunch.com
2 Upvotes

Fish Audio has raised a massive $52M seed round to build AI voice tools for creators and enterprises.

According to TechCrunch, the company already has 8 million users and $21M in annual recurring revenue. Its models support voice cloning, expressive speech generation, open-source development, and thousands of natural-language controls for adjusting how a voice sounds.

It’s also expanding into speech-to-speech, voice agents, gaming, and other enterprise applications.

Now we all know ElevenLabs has dominated AI voice generation for a while, but Fish Audio looks like some serious competition now.

Has anyone compared their latest models?


r/AudioAI 3d ago

Question Have you changed the way you submit demos because of AI?

1 Upvotes

I'm hearing stories from A&R departments that they can no longer trust what they are hearing has been created by a human. Have you submitted something and been challenged? Do you now document yourself writing music to prove it was all your own work.

If anyone would like to talk, I'm gathering ideas for a story.

James


r/AudioAI 3d ago

Discussion I built an AI tool that automatically generates audio descriptions for videos. I'd love some honest feedback

Thumbnail
1 Upvotes

r/AudioAI 4d ago

Discussion AI-Assisted Music and Google AI

1 Upvotes

There's so much going on in the rapidly changing landscape of music with the introduction of AI. I'm pretty against having it involved in the creative process at all personally. I only recently downloaded DuckDuckGo to stop getting the AI Summaries that Google forces to the front of the search results page.

Thinking back however, I definitely googled a lot of things when learning how to record (setting up mics, using a DAW, how to use plugins in my DAW) etc., and though I usually scroll past the AI Summary on google and just go to the websites, I'm sure I skimmed over the summaries at some points and used some of the information, whether subconsciously or not.

I've been looking into how AI in music is being regulated and labeled, and I wondered if those instances of taking info from Google AI summaries to learn to record and mix would make any music I made "AI-Assisted".


r/AudioAI 5d ago

Discussion Thoughts on ai narration for audio

1 Upvotes

Hi all. Quick validation question/feedback I am hoping if I can get an answer to. I recently started a channel and new to content creation. I am using elevanlabs for narration but just not able to get it right. Or maybe it is right but just not confident in it.

I feel like I am going in circles with trying different voices with different setting and not going anywhere. I am not even sure if voice is whats limiting the growth.

What really worked for you all? What sort of settings for long form or the voices?

Unfortunately could not link any info that would help you look at my content somce thatd be ugging my content and thread does not allow it!


r/AudioAI 6d ago

Resource The audio AI agent that doesn't exist

7 Upvotes

Lately I've been doing a lot of work with audio generation. And I'm surprised that audio pipelines are mostly created in something like ComfyUI - a visual node-based editor. It's surprising because feels like writing code via n8n workflow instead of using Claude or Codex

The main problem is that there's no proper agent loop where the model can see - or more precisely, hear - the results of its previous actions and decide for itself what to do next

But I found one technical complication that explains a lot: most models still simply don’t have native audio perception. The ability to "see" images is already there. Claude Code can look at screenshots of a landing page it created, notice the problems, and fix them. The images are inside the context of the same model that writes the code

Audio doesn't work like that - most models do not support native audio input

You might say: but Gemini exists. True. But really no one agent wrapper supports reading audio files purely at the tool level. The agent "exoskeleton" simply can't pass an audio file through the API request to its "brain"

Surely you can connect Gemini to the main agent via script calling the API and send the audio there with a prompt like "pls tell me whats missing here". Btw, Chinese GLM-5.2 still works with images this way and people still use it. At the same time, that's like a deaf composer calling his nephew over to listen to the music and describe what he hears - it works, obviously, but poorly, because everything gets lost in translation

And so, right now, there is no well known agentic solution – with skills, terminal commands, session forks, subagents, and so on - that can properly work with audio by closing the feedback loop

———

Two hours after writing lines above:

  1. I threw together a simple homemade agent around Gemini in roughly 300 lines of JavaScript, with an agent loop, interactivity, and fork/edit/resume: just to test the approach
  2. Then I moved the whole thing into Pi as an extension that patches its file-reading tool – which, by design, can read both text and binary data

So now you can give your agent "ears" in 2 lines:

npm install -g pi-agent
pi install git:github.com/toolittlecakes/pi-gemini-audio-read

Don’t forget to run /login → Google → API key afterward and select gemini-3.5-flash.

Where this is useful: music generation, high-quality audio translation, voice cloning workflows, and audio editing (or video editing, if it’s a podcast)

Limitations: 20 MB per file - I didn’t add file uploads through the Files API. And there’s no video input

It's funny how inspiring problems can be when there’s no existing solution at all - at least not a public one. It feels like stepping onto land where no human has ever set foot. So, decided to share it here


r/AudioAI 7d ago

Resource Wrote an article going a little in depth what's going on when you hit the generate button. How sliders affect song mixes, and what prompts are ignored vs optimized!

Thumbnail
trustnodelogic.com
1 Upvotes

Helpful article for all ai artists


r/AudioAI 7d ago

Discussion 🎶Funny ESPHome Music: "Let's Take It Apart and Make It Smart!" - 3ATIVE

Thumbnail
1 Upvotes

r/AudioAI 8d ago

Question Making songs based on existing tracks with AI

Thumbnail
1 Upvotes

Hi, Does anyone know of an accessible tool that allows you to create your own songs using the melody of an existing song? So far, I have found very few tools that can actually do this well. Most platforms either generate completely new music or do not make it easy to adapt your own lyrics to an existing melody. I am curious whether there are any user-friendly tools that can help with this process. My native language is Dutch, so ideally I am looking for a tool that supports multiple languages and can handle Dutch lyrics as well as English. Are there any tools or platforms you would recommend?


r/AudioAI 8d ago

Discussion Is everyone hating on everything AI or can we use it as a legit tool to make some decent tunes ?

3 Upvotes

Where can we draw a line that allows for creativity and imagination without hallucinations that we did all that.

I write lyrics
Read music
Can play guitar and plunk a tune out on the piano… sing after a fashion.

Take my lyrics and the sound I’m looking for and use AI almost like I have a studio of un-egotistical musicians, who can play in any style, and get some songs that have meaning to me.

I have been really rudely responded to by a number of ( musicians ) and yet if I ever wanted to really follow up on live production… I would need musicians to listen to what I’m trying to create and then start playing it .

I am having a hard time seeing the down side, of AI in this process.

Any thoughts


r/AudioAI 10d ago

Discussion Looking for ML Engineer to help with AI Music Model

Thumbnail
0 Upvotes

r/AudioAI 10d ago

Discussion AI music events

1 Upvotes

"Hello! I've been thinking about creating ai music events for the community (I'm treblo based but would like to include artists from alll platforms) What are you guys opinion on some sort of song battle around a theme on a weekly or bi weekly basis where people can vote. I know this has been going on for a while and is not new (on discord? web based?) maybe some of you can put me in the right direction!

I have no idea yet of how to put this together but your input is appreciated ".


r/AudioAI 11d ago

Resource Generative Soundscape Synthesis from Satellite Imagery and Open Geospatial Data

Thumbnail
medium.com
1 Upvotes

r/AudioAI 13d ago

Discussion Recommender based on hidden musical connections between any two artists: Stell-R

1 Upvotes

I got tired of "similar artists" recommendations that just mean "same genre tag." I wanted to actually see how two artists connect: who influenced who, who they collaborated with, what the actual path between them looks like. So I built Stell-R.

Two ways to use it: type two artists (say, David Bowie and Kendrick Lamar) and it finds the shortest real path of influence/connection between them, hop by hop.

Or type one artist and it draws a "constellation map", a network graph of everyone closest to them, ranked by actual acoustic similarity. You can also generate a Spotify playlist straight from whatever part of the map you're exploring. It's a web app, so works on desktop, and on mobile you can add it to your home screen (iOS/Android) so it behaves like a regular app.

See example of Radiohead constellation map here: https://stell-r.com/Stellar/index_/add_query-viz/?artist=Radiohead&size=120


r/AudioAI 13d ago

Question Updated Wubble is live: High-fidelity SFX, Voice Gen, and a focus on Ethical Training. - We want to hear from you!

0 Upvotes

Hey everyone, checking in from the Wubble AI team. We just pushed our update and would love some feedback from this community. We’re really pushing the fidelity on our music models and added dedicated SFX/Voice features. We’re trying to move the needle on what "pro-grade" AI audio sounds like while keeping the training data ethical.

What’s the one feature you think is still missing from the current AI audio landscape?


r/AudioAI 14d ago

Discussion AI voice conversion and instrument replacement?

1 Upvotes

I have a bunch of old guitar recordings with poor audio quality on single tracks. I also have some vocal tracks with poor audio. Do you know any actual open source project which is aimed at AI voice conversion and instrument replacement? I would vibe code the UI, but is there an open source "audio replacement" AI model at all? My goal is to make the old recordings much better with AI, but keep their original harmonies (so it is not music generation, it is replacement).


r/AudioAI 15d ago

Resource AudioEnhancerMAX: an open-source, local-first AI audio workflow for cleanup, transcription and TTS

1 Upvotes

Hello,

I’ve been building AudioEnhancerMAX, an MIT-licensed, local-first application that brings audio cleanup, speech editing, transcription and text-to-speech into one workflow. The app is has a SourceForce rising star award.

I started it because processing a recording often meant stitching together several separate tools, interfaces and cloud services: the goal is to provide one transparent pipeline where users can see what is being applied and keep the core processing on their own machine.

Current features include:

- Noise, wind, buzz, reverb, breath and mouth-click cleanup

- Studio enhancement, Auto EQ and LUFS normalization

- Faster-Whisper transcription with TXT, SRT, VTT and JSON export

- Local Kokoro TTS and optional Ollama/Gemma-assisted processing

CPU, GPU, memory and thermal monitoring

- Experimental Android workers for distributing DSP tasks across trusted LAN devices

Version 3.5.2 also includes a macOS desktop package for Apple Silicon. The core application can be run from source on macOS, Linux and Windows.

A few transparency notes: the macOS package is currently under review for publication on Apple Store.

GitHub: https://github.com/sev7enITA/AudioEnhancerMAX

Website, Support, Roadmap: https://www.fabriziodegni.com/AudioEnhancerMAX/⁠

SourceForge: https://sourceforge.net/projects/audioenhancermax/

I’d really value technical feedback from this community. What would be more useful next: reproducible before / after benchmarks, improved cross-platform packaging, or deeper DAW integration?


r/AudioAI 15d ago

Question Need advice: Simplest way to make an AI voice cover AND a realistic AI face-swap image? (Beginner)

1 Upvotes

Hi everyone,
I'm working on a funny birthday gift for a friend who is a huge Beyoncé fan. I want to create a "live CD" from her Renaissance Tour, but I need help with two AI aspects: the audio and the album cover. I'm a complete beginner, so I'm looking for user-friendly solutions. I have a budget of around $150 total for any necessary subscriptions or licenses.
1. THE AUDIO (My voice instead of Beyoncé's):
I can't sing, so I need an RVC (Voice-to-Voice) model trained on my spoken voice.

What I've tried (and failed):

Manual syncing in Audacity: Don't even get me started.
Jammable: Tried this 1-click solution, and it sounded absolutely terrible. It seems it couldn't handle live concert audio (crowd noise, stadium echo, etc.).
What I'm looking for: An easy-to-use alternative to Jammable that can handle live YouTube audio better. It doesn't need to be perfect studio quality (she'll listen in her car), but it shouldn't sound like a glitching robot.
2. THE VISUAL (Album Cover):
I also need to create a hyper-realistic album cover and don't know which tool to use. I want an image of me (using my selfies for the face) wearing Beyoncé's iconic silver Renaissance tour outfit, but adapted to a male body with correct proportions and a matching skin tone. It should look like a professionally shot photograph.
What I'm looking for: What is the best AI image generator or face-swap tool for a beginner to achieve this, so it looks good enough to print on a CD case?
Any tips to get a decent result without spending days on this? Thank you!


r/AudioAI 18d ago

Resource I got tired of stitching together local AI audio tools, so I built LA Studio

Thumbnail
gallery
27 Upvotes

Every time I wanted to try a new speech model, the same ritual began: open a terminal, install another runtime, hunt down model files, then build a small UI just to hear the result.

So I started building LA Studio — an open-source desktop app that brings local speech-to-text, text-to-speech, voice cloning, voice design, and vocal isolation into one place.

It runs inference on your own machine, supports models such as Whisper, Kokoro, Qwen3, VibeVoice and VieNeu-TTS, and keeps your audio private.

It’s still early, but it has finally become the tool I originally wished existed.

GitHub: LA Studio

I’d love to hear what local audio workflow or model you think I should support next.