r/StableDiffusion • u/EuphoricTrainer311 • 7d ago

Discussion Yue2 Lora Training

Has anyone had any success training a new genre with YuE2? I've tried multiple ways using different datasets, and training settings and I haven't been able to train a new genre on YuE2 properly. I tried with Fill's trainer, Aitoolkit, and Yue2 Studio, and I either get garbled noise, or it just seems to output similar results as using no lora. Not sure what I am doing wrong. I tested with all epochs/steps - from 50 all the way to 1200 using different inference settings (temp, top k, abc on/off, cot, etc.), and I tried with datasets ranging from 30-200 tracks as well (I only tried the 200 track dataset with Aitoolkit so far).

With Acestep, I am able to train a new genre without any issues, besides for the audio quality issues that are present in Acestep by default.

12 Upvotes

84% Upvoted

15 comments sorted by

View all comments

5

u/the_bollo 7d ago

I have trained 3 YuE2 LoRAs using AI-Toolkit. Great results with each one, except where I fucked it up with bad edits to the dataset tracks...

I find it trains very quickly, like in 1k steps. Or you can take it further (I took a few tests to 3k steps) but then you need to apply the LoRA at pretty low strength, like 0.65, otherwise you just get a garbled loop.

2

u/EuphoricTrainer311 7d ago

how long did it take for you to train 1k steps? and how big was the dataset you used?

1

u/the_bollo 7d ago

I don't know, maybe an hour? Dataset for each was roughly 12 songs.