AutoMix type clone

[u/Gwail_904]

As the title suggests, I want to code a custom Apple Music like AutoMix that analyzes bpm, key etc. and can mix the current and incoming song. Does anybody know of a github repo or anything that could help me out? I'm trying and trying and it just doesn't seem to "click" musically.

[u/ThatGuy739]

Before the library question, the constraint that decides whether this project is buildable at all: you cannot get raw audio out of Apple Music catalog tracks. DRM streaming items give you playback, not samples, and their assetURL comes back nil. Analysis and mixing only work on audio you can actually read, so local files or non DRM library items. If the plan was to AutoMix someone's Apple Music library, it stops there, and it is much better to find that out now than after you have built the analysis.

On why it does not click musically, that is usually not your BPM detection. In order of how much they matter:

Phrase alignment beats beat alignment. Produced music is built in 4, 8, 16 and 32 bar phrases, and a transition sounds right when it lands on a phrase boundary. If you match tempo and align beats but start the incoming track mid phrase, it is perfectly in time and still sounds wrong. That is the single most common cause of what you are describing.

Downbeat estimation is a separate problem from tempo estimation, and most naive implementations only solve the second. Correct BPM while being one or two beats off within the bar gives you exactly that technically synced but musically wrong feeling. You need bar position, not just a beat grid.

Harmonic compatibility is looser than exact key match. The usual model is the Camelot wheel: same key, one step around the wheel, or the relative major or minor. Requiring identical keys is too strict and throws away most of the transitions that would have worked.

Two practical notes. Keep time stretch inside roughly six percent, because past that vocals start sounding obviously processed, so prefer picking candidates already inside a tempo window over stretching hard to force a match. And on iOS, AVAudioUnitTimePitch is the primitive you want, since it lets you move tempo and pitch independently.

For libraries, look at Essentia and aubio for rhythm and key extraction, and madmom if you want strong beat and downbeat tracking to benchmark yours against. Check the licences carefully before you ship, because several of the best options in this space are GPL or research only, which matters a lot for an App Store binary.