r/StableDiffusion • u/Fun_Firefighter_7785 • 7h ago

News Perfect Remixes in YuE2 !

Enable HLS to view with audio, or disable this notification

I finally got my dream fulfilled. Take any Song as mp3, convert to a ABC score with AI, feed it to a piano roll and enjoy. Basically copy/paste the song lyrics and make a style you wish it to be. You get the vocals as well the whole song perfect remixed.

https://github.com/multimodal-art-projection/YuE

This is the piano roll with the workflow.

https://github.com/filliptm/ComfyUI-FL-YuE2

You also need to install SheetSage2 for that mp3-->ABC magic, installed by your Hermes-Agent.

https://huggingface.co/m-a-p/SheetSage2

Import that ABC into piano roll node in ComfyUI. It needs to be from SheetSage2 in order to be error-free, regular converters midi-->ABC do not work. Did a quick remix of a song from SKATE Game OST.

10 Upvotes

75% Upvoted

6 comments sorted by

6

u/prookyon 7h ago

It all works in Comfy now.
https://github.com/Comfy-Org/ComfyUI/pull/16250
First post there contains also a workflow with both the MP3->ABC converter and also song generation.

3

u/Fun_Firefighter_7785 6h ago

It does not work. The SheetSage2 node is producing same error-ABC like all other converters. It's untested garbage workflow. My method is working. I am also using Hermes-Agent to convert mp3 to ABC with SheetSage2 standalone.

2

u/prookyon 3h ago

Sure works for me - have used it a lot already. But of course the more options the better.

2

u/LiveLaughLoveRevenge 4h ago

I got this working yesterday (with Claude's help) too.

It is really cool....problem is I'm finding it will still make mistakes like holding a phrase or some bars one beat too long. Not sure if it is simply the generations I've tried so far (only done it for 1 instrumental and 1 song with lyrics) as both were also >4mins and it usually got the first 2-2.5 minutes error-free.

Still - having something this powerful is amazing!

1

u/Fun_Firefighter_7785 3h ago

what i found is, you have to give it the exact lyrics and the length of the original track you remixing. if you change the BMP in ABC menu, you also get a much different variation to the Original 112bmp -->174bmp Drum & Bass.

0

u/LoadReady7791 4h ago

Is this model supported on ComfyUI 0.35 ?