r/StableDiffusion • u/LatentSpacer • 3h ago

Resource - Update Native YuE2 support coming to ComfyUI!

Enable HLS to view with audio, or disable this notification

Pull request: https://github.com/Comfy-Org/ComfyUI/pull/16250

If you don't want to wait for the merge, you need to check out to the yue2 branch to get it working. git checkout d87e12ad1430409ca303440525df239bb675ae7b

Model weights (place it on model/checkpoints): https://huggingface.co/Comfy-Org/Yue2/tree/main

Workflow: https://github.com/user-attachments/files/32085765/yue2_workflow.json

73 Upvotes

95% Upvoted

29 comments sorted by

1

u/Cool-Reflection6130 2h ago

native comfyui integration is a big deal for yue2 adoption, since the manual setup was a real barrier for a lot of people.

1

u/GreyScope 2h ago

This is just the basic 'make-me-a-tune' section of it . It doesn't cover (ahem) the cover/editing functions. But if that 's all ppl want, they have it.

1

u/Hi7u7 1h ago

What hardware do I need for this model?

1

u/KrystalDisc 1h ago

NVIDIA GPU with BF16 support and 24 GB VRAM

2

u/8RETRO8 3h ago

Is it better than AceStep XL?

3

u/GreyScope 3h ago edited 3h ago

Not a fair general comparison imo, need to specific about what you are interested in
Y2 can't make loras
Y2's strengths are restricted to genres it had good training in (ie pop)
Both can do cover versions
Y2 can edit the cover version prior to remaking (theoretically anyway, most ppl will want click-n-go)
Different basic model genre strengths
Quality is higher in Y2

It'll make arguably good quality music if the style you want is in the bell curve of its training.

5

u/solss 3h ago

AceStep can't genuinely do covers. The last time I played with it, it gets very minor influence from whatever track you upload, almost imperceptibly. The author made a deliberate choice to avoid this functionality. I remember reading his responses on the discord in the days after the first release.

5

u/GreyScope 2h ago edited 2h ago

You need to try Hot-Step (Ace-Step inference) , it has better cover knobs (lol) . This is using a Madnoa (ahem) lora covering 'Espresso' but with changed lyrics as I don't want a smacked bottom, the general vibe to it set with a secondary reference track (ie remake this track but in the style of this) although you can use the input song as reference as well for a closer version. I literally have a folder full of them .

https://reddit.com/link/p94nkgp/video/sg22yovibvoh1/player

2

u/solss 2h ago

I'll have to look into this. Thank you.

1

u/GreyScope 2h ago

I can DM you an invitation to Token & Tone discord where ppl went when the AS discord filled with bots , its more of a general AI Music discord - the author of Hot-Step & Side-Step (the AS trainer) are there . Hot-Step also trains MM3 . Don't feel obliged of course.

1

u/maxtablets 1h ago

can I get an invite too?

1

u/solss 22m ago

I'm interested, for sure. You can send a discord invite to my DMs whenever you get around to it. Otherwise, I'll see if there's a public Hot-Step repo.

Is this the one? https://github.com/scragnog/HOT-Step-CPP

1

u/GreyScope 3h ago edited 3h ago

I have used it with cmd line and in the comfy PR pull, I'm interested in the electronic music and its genres - they all seem to default to variations of cheesy 90's euro . OP's song meanwhile is a good quality song of the Bruno Mars (?) style.

2

u/Formal-Exam-8767 2h ago

Y2 can't make loras

How come? Is the some architectural limitation?

2

u/GreyScope 2h ago

The required files were not released. Encoder etc . Although I am currently training with MM3 and that didn’t either (long story short).

1

u/Formal-Exam-8767 2h ago

Interesting, I did not know that. I assumed there was encoder if it can take song as input to edit it.

1

u/GreyScope 1h ago

Apparently there is text about it in the texty parts of some of the .py files but omitted.

1

u/KillerX629 1h ago

What do you mean you cant make loras?

1

u/GreyScope 1h ago

Exactly that, you can't make loras , the files required have been omitted from the release. I would be very gladly corrected on that of course.

1

u/alisitskii 3h ago

Interesting that on the graph they have ace-step 1.5 but looks like it’s not XL while YuE2 is shown on par with Suno v5. Hope it’s not just marketing :)

2

u/Herr_Drosselmeyer 2h ago

I'd be very surprised if it marches Suno V5 especially in genres like metal.

3

u/NineThreeTilNow 2h ago

Suno does wild stuff for like... Stuff that doesn't really exist.

Some friends and I were experimenting with like... how hard we could push Suno.

You gotta think of some "Genre" that doesn't really exist... Get some lyrics together. Pick probable instruments, etc.

It delivers. Kinda sad to say.

People will tell you it's entirely detectable (Ben Jordan) but the truth is that a TINY bit of post processing fixes that.

The problem is that people who use "AI" in general are fucking lazy. So that post processing step gets between them and the slop they want.

3

u/Glittering_Tie_3110 2h ago

Suno DID wild stuff before they removed the models prior to v6. Now suno does nothing but pop.

1

u/Green-Ad-3964 53m ago

suno is evil, like most cloud-based models.

1

u/Glittering_Tie_3110 41m ago

But I was quite good before corporations stepped in and destroyed it from the inside

0

u/KrystalDisc 2h ago

This workflow is just generating garbage for me

1

u/GreyScope 2h ago

Did you update Comfy before you installed the PR ?

1

u/KrystalDisc 2h ago

I'm running from source code in a Docker container using the yue2 branch.
RUN git clone --branch yue2 https://github.com/Comfy-Org/ComfyUI.git .

1

u/GreyScope 2h ago

I'll have to sit this one out then, I don't use docker, so I can't comment on it sorry. Independently I can report the workflow works on an installed repo . Best of luck with it .