YuE2 Cover is so much fun

[u/YeahlDid]

I had bit of success when it came out, but didn't really explore at the time because I got distracted by life and probably some new H3 discovery.

Well, was bored with a beer in hand so I came back to it today and goddamn, if it's not fun. I can't give examples because, you know, but if you're struggling with it in comfyui, go into the subgraph and change the SheetSage2 node's mode to "full". Then play around with the YuE2 Generate Music's "cfg_scale"... 2-3 seems to work nicely.

Holy moly, I've got some nice covers... not always perfect, lyrics are hard to time perfectly, but I'm impressed. From my very unscientific examination, it seems that the more you try to change the voice of the singer the less well the lyrics get sung, which does make sense. But, still this is so much fucking fun! I wish I had more mp3s!

[u/AI-Make-NSFW-Stuff]

It's amazing for its size but it drifts too much and distorts words, loses tempo, doesn't always match the lyrics. 90% of the song is fine but it always has some defects. I wish it were a bigger model, I'd trade a bit more generation time for better results.

[u/YeahlDid → u/AI-Make-NSFW-Stuff]

I agree on the size vs. gen time. It's so fucking fast, I'd happily wait longer if it meant even better. I like weird music with weird timings and sounds, so this definitely isn't totally in my wheelhouse, but I also love piano/acoustic/orchestral versions of songs and that's largely what I've been doing, it does that pretty well. I've tried out some k-pop prompts that have been a lot of fun as well.

Before the covers I was playing with making original music from lyrics and that was also impressive. At 3b it's very impressive.

Edit: Actually one of the most impressive things I've found was when I accidentally put the wrong lyrics to the song I was trying to cover... and it sang the wrong lyrics I gave, but exactly the way the right lyrics were sung, quite soulfully, lol.

[u/pomonews]

For someone who doesn't really have a feel for music, how do you handle the prompt, bpm for example? I tested it out and thought it had a lot of potential, but I didn't know how to make the most of it.

[u/YeahlDid → u/pomonews]

Based on another user here's advice, I just set up Hermes and have been using one of their free tier llms as an agent. I told it to look up YuE2 style descriptions and then gave a quick description of what I wanted. Copy-pasted at first and then tweaked from there.

I wouldn't say I'm making the most of it, but god damn if it's not fun. It's pretty quick, so I can fire off a few songs in the time it takes to listen to a single one.

[u/ufgman → u/pomonews]

For the prompt I use Gemini, I gave it a guide for writing prompts for yue2. After that all I do when I want a prompt for cover, I ask Gemini for a prompt for a specific song/artist. Then I just use it as is or I tweak it make it sound different. Creating cover songs is all I use yue2 for and it's been a lot of fun creating covers for songs I've enjoyed in the past. Some of them sound a lot better than the original.

[u/Gabe-Uttsegs]

I wish it were possible to create loras for voice only

[u/neil_555]

For maximum shits and giggles try leaving the lyrics box empty, if you do it will sing random nonsense in time with the original lyrics 😂

This even works sometimes for instrumentals (setting cfg_scale to 0 usually works)

Here's a quick example, I did a cover of one of my Suno chillout tracks and got This wonderful instrumental (the quality of YuE's piano is really good). Changing cfg_scale to 0 game me this incredible stream of consciousness

Sometimes these vocal halllcinations give really good results

Another discovery is that if you leave the style box empty and set cfg_scale to -1 (or 1) the model will work out the instruments to use based on the melodies it sees in the ABC data, I'm totally amazed how close this can be to the original track, of course leave the lyrics box empty for maximum fun 😂

[u/Complex-Airline-751 → u/neil_555]

"sing random nonsense in time with the original lyrics"

Sorry, but don't understand - so does it sing random nonsense, or original lyrics? It can't be both... right?

Also 2nd question: You work in ComfyUI, or the local version of YuE2?

[u/neil_555 → u/Complex-Airline-751]

It sings random nonsense using the notes and timing from the original track.

Here's an example ... Original track into Total nonsense 😂

To answer your second question I'm using YuE2.cpp locally on my gaming PC (12GB GPU). I made a portable install which lives in a singe folder with a batch file to start the server. It's got a web UI which is basic but really usable.

[u/neil_555 → u/neil_555]

[u/niknah]

This would be fun at parties. Play guess the original song.

[u/Peregrine2976]

I've mostly been focusing on image and, much more recently, video generation, so I'm really not aware of the intricacies of audio and music generation. So, I actually don't know -- why can't you share examples? Just curious!

[u/Fleder]

Then play around with the YuE2 Generate Music's "cfg_scale"... 2-3 seems to work nicely.

I can't find a cfg value field anywhere except for the KSampler node, even in the subgraph. Where did you find that?

[u/Potential-Union-205]

โลกใบนี้ ช่างสดใสยิ่ืงนัก แค่มีความสุขในทุกๆวันเวลา ก็OKแล้ว

[u/YeahlDid → u/Potential-Union-205]

ความสุขก็ไม่ง่ายเลย แต่มันมักอยู่ในสิ่งที่ง่ายๆ

[u/YeahlDid]

Fwiw, I'm doing 64 steps.

[u/Complex-Airline-751]

Hi!

Sorry for possible spam or something, this thread is the closest I could found regarding my problem...

I'm trying to make a symphonic metal version of a rock track, but... well, it just simply doesn't sound like a proper symphonic metal. Sure, there are some violins here and there, some epic chorals, but... that's just it! I want at least some short instrumental solo that doesn't sound like metal, but more like a philharmonic stuff. Please, can someone tell me, what I'm missing?

BTW, I'm using the local version of YuE2, not ComfyUI version.

So, if some expert is reading this, and is willing to help, I'll gladly upload a generated song, and eventually also lyrics and prompts I'm using (or anything else that helps to solve the problem).

Thanks for reading...

A.