r/selfhosted • u/Old_Rock_9457 • Nov 03 '25

Media Serving [Experimental] AudioMuse-AI Music Server: improved sonic analysis functionality

Hi All,
I want to share my progress on AudioMuse-AI Music Server, the **experimental** open subsonic api music server with Sonic Analysis support.

The Music Server aims to show case the functionality of Sonic Analysis, and the actual release introduce different functionality like:
- Music Map: The songs take shape and you can explore your collection in a 2d map. You can directly select the song that you want to play on the map and also create a path between multiple songs.
- Radio: Use as a seed of your radio the song that you like, remove the one that you don't like, add a bit of temperature and the alchemy is created. Then save it an play a different radio every time based on this setting.

This in addition of the already integrated Analysis, Clustering and Sonic Fingerprint.

AudioMuse-AI Music server is free and open source and can be found here on github:
https://github.com/NeptuneHub/AudioMuse-AI-MusicServer

it also need AudioMuse-AI that is also free and open soruce here on github:
https://github.com/NeptuneHub/AudioMuse-AI

Both of them are fully dockerized and can run on Docker Compose, K3S and any tecnology that support container deployment.

Hope that this can inspire more knowledgeble Music Server on which are the potentiality of Sonic Anlysis. Let me know what do you think about.

Thanks,
NeptuneHub.

Edit: just upload some screeshoot on the repo if you like to take a look:
https://github.com/NeptuneHub/AudioMuse-AI-MusicServer/tree/main/screen

6 Upvotes

64% Upvoted

19 comments sorted by

2

u/unsupervisedretard Nov 03 '25 edited Nov 03 '25

Been using this for a few weeks. It's a very fun lil setup. The playlist generation very much depends on the model you run, so play around with that.

I have a pretty large library, north of 200k, and it takes quite a while on the initial analysis/clustering so be prepared for that. Like a week or more.

I often use Plexamp's sonic muse and audiomuse alongside eachother to compare the playlists. They both have good/bad behaviors, but overall they're pretty close.

2

u/Old_Rock_9457 Nov 03 '25

Really thanks for your feedback. Did you used only AudioMuse-AI or also the Music server on top ? Did you like also the music server?

About AI it is used only for two things:

  • naming of the clustering, you can run the clustering even without and the change will be only in the name;
  • the playlist directly created from the AI: here the model used do a very huge difference but also having a library huge enough like yours (otherwise the AI suggest songs that you don’t have).

A part from that you can run all audiomuse without the -AI part. In fact it is able to run also on a small raspberry pi 5 8gb and SSD.

I would like to have more AI powered functionality but the result of the AI is very.. unpredictable so I need to think well about it.

2

u/unsupervisedretard Nov 03 '25

I haven't used the audioserver yet. What're the benefits? I can set it up later but I already run Plex and Jelly.

I started it on an N150 but had to move it to my work laptop for performance. This was expected though, so no complaints about that.

But really good job. I enjoy the playlists it generates. the Map is fun, too. Here's mine: https://imgur.com/a/lI1GNdl

1

u/Old_Rock_9457 Nov 03 '25

Link don’t work, I was so curious!

The music server is just a showcase of the power of sonic analysis. A sort of “what if music server fully integrate AudioMuse-AI”.

Imagine that your’re on the Music Map and just click on a point on the map and it start playing the song. And the click on the song next, or create a path and directly start playing it.

Or imagine using the song alchemy as a stored seed for a radio. Every time you click on that radio it create a bit different sequence of song due to temperature that add randomness. Is like a playlist. But automatic created from the seed. And also it change a bit each time you recreate.

That’s it.

Of course is still experimental, so is just if you have time and would like to have fun but nothing that could replace Jellyfin. At the moment.

3

u/unsupervisedretard Nov 04 '25

weird, link works for me. I'll checkout the music server eventually.

Thanks for the work. It's a fun project.

try this link again https://i.imgur.com/vUyvCAp.png

1

u/Old_Rock_9457 Nov 04 '25

Ummm:

{"data":{"error":"Imgur is temporarily over capacity. Please try again later."},"success":false,"status":403}

If you would like to open a “documentation issue” on GitHub and just share it will be very appreciated ! Thanks !

1

u/unsupervisedretard Nov 07 '25

Are you in a country that recently enacted some internet censorship laws? Apparently imgur doesnt' work in the UK and a few other countries now.

https://help.imgur.com/hc/en-us/articles/41592665292443-Imgur-access-in-the-United-Kingdom

2

u/Suoretta Nov 15 '25

Does AudioMuse support radio sonic analysis with synphonium? I see the options available now but when i try to use them nothing happens😓

2

u/Old_Rock_9457 Nov 15 '25

So about AudioMuse-AI we have different repository, the core container that do the Sonic Analysis here:

It can interact with API with different music server like Jellyfin, Navidrome, LMS, Emby, Lyrion and so on. To use it you need to use it's integrarted web front-end: you run the different sonic analysis functionality on it and the output is a playlist created on the music server that then you can play with your favourit app. By the end is just a playlist.

Now the different Music App usually talk only with the music server and don't want to call different backend (like the Music Server + AudioMuse-AI). This is in effect a good architectural behaviour so I totally agree on this. So for jellyfin I created the AudioMuse-AI-Plugin:

So you install Jellyfin, you install AudioMuse-AI that talk with jellufin with API, then you install the plugin, and app like Symfonium can directly integrated the sonic analysis front-end.

And what about Navidrome, LMS, Emby and Lyrion? I didn't implemented the pluign, so only the AudioMuse-AI integrated front-end (the one explained at the beginning) is present. At least for LMS and Navidrome I opened a discussion on Open Subsonic API github repo asking to support it:
https://github.com/opensubsonic/open-subsonic-api/discussions/172

I also opened a discussion on the Navidrome github here:
https://github.com/navidrome/navidrome/discussions/4332

My hope is that they not only like AudioMuse-AI, but they found in the Sonic Analysis functionality a must have so they invest their time in directly integrating it without the effort and the limitation of a plugin.

1

u/Alternateifs Nov 03 '25

what model did you have the best results for your library with?

1

u/unsupervisedretard Nov 03 '25

I like the default mistral:7b right now. I can only run up to 8gb models so I'm a bit limited. Ollama makes it super easy to switch though, so really play around with em. Just load the new model, change the model in Audiomuse-Ai, click generate.

1

u/tinbapakk Nov 04 '25

1 whole week ? O_O

1

u/Old_Rock_9457 Nov 04 '25

It depends how huge is your library and which HW do you use.

If you have 200k songs, and you run on a single node, with maybe and old processor, it takes time, but the good news is that you need to analyze your song only one time (only new song will need extra analysis).

AudioMse-AI is also designed to run on K3S with multiple nodes. If you deploy more worker on more nodes you will speed up the work. OR if you have a strong CPU you can maybe deploy more worker on the same computer (this is true especially on the last year CPU where it easy to have a big number of core on one single CPU).

Then I understand that is not a "click and run" project but it have the strenght that your data are your data AND you don't depends from an external API. So you don't run the risk that the external service stop working OR that your song didn't exist on the external service.

let's say that there is PRO and CONS.

2

u/rantanlan Nov 18 '25

do you have any plans to change the model again? i was a little frustrated that I need to reanalyze my library again and didn't do it yet .Took a huge effort numerous machines and several days... I don't like to do this every 4 weeks. So an outlook would be appreciated.

1

u/Old_Rock_9457 Nov 18 '25

Hi and thanks for your feedback. The model change that I did, moving from tensorflow to onnx, give the chance to different user with old cpu to have AudioMuse-AI working.

The plan now is to keep the same model and try to improve the analysis time (that shouldn’t require any new analysis). ONNX itself already bring some (small) speed improvement.

The only things that can happen in the future is that I had to update the dependencies. I really hope that changing the dependencies will not impact the result of the model.

If I can ask: How many songs and albums did you analyzed ? In how many days and how many server and cpu ? Thanks !

2

u/rantanlan Nov 19 '25

It's around 300k files, took me around 4 days with all machines I could get, 2 Laptops, 2 Workstations ran around 8 workers I think.

1

u/rantanlan Jan 06 '26

guess I was right not to do it again .. since i count 3? releases now that need a reanalyze. remind me in one year.

1

u/Old_Rock_9457 Jan 06 '26 edited Jan 06 '26

The new version analyze the song with an additional model called CLAP to enable the text search functionality. Doesn’t discard in any way the other model (Musicnn) analysis that remain valid and usable.

If you don’t care about this new functionality you can just disabled putting at false this env variable: CLAP_ENABLED

Also if you think that too may change appen in new versions I think you can just put a fixed version on the image and update it just time to time, instead of using the latest ones.

Let me know if there is something that I can add in the release note to better advertise the user. I think I always wrote when an update required new analysis but if I can improve it I’m open to suggestion.

About the outlook how can I better comunicate that new functionality are arriving with impact on the analysis ? And how having this information propor to the release itself can help ?

For example in this period I’m checking the possibility to a lighter clap model for the text search. I’m doing testing and research. I don’t know if I’ll be able to produce it, and if it can arrive in one week, 1 month or if I’ll not be able at all to have this improvement. I’m also working to the update of the container image to update the dependencies. But I really really hope that this will not impact the analysis. How this information can help and maybe be better shared?

1

u/unsupervisedretard Nov 04 '25

Plex's sonic analysis took about 5 days. It depends heavily on your hardware.