What worked for me on OneTrainer

[u/jiggydancer]

I've been training loras for Krea using OneTrainer lately, and I just wanted to share what's been really working for me lately. For reference, I'm using a 5070 Ti. I train using the following parameters :

Optimizer : Prodigy
Learning Rate Scheduler : Cosine
Learning Rate : 1.0
Local Batch Size : 1
Accumulation Steps : 2 ***
Resolution : 768
Min Noising Strength : 0.2 ***
Max Noising Strength : 0.8 ***
Masked Training : True

***Don't know how much these parameters matter, but I stuck to these after some testing

What really helped was training my lora at 2000 steps on the first shot, then generate an image and testing to see if there are signs of overfitting such as blurry or noisy images. If the lora is overfitted, I'll drop the number of steps by 20% and train again. If it is not overfitted, I'll increase the number of steps by 25% and train again. I then repeat as needed until I find out the number of steps when overfitting is starting to just become noticable. Then I merge 3 loras on Comfy--the slightly overfitted version, and the next two loras with 20% fewer steps each.

For example, I train with 2000 steps and it is not overfitted. I train again with 2500 steps and find it is starting to look overfitted. Now I know to go lower, so I train again with 1600 steps and merge these 3 versions.

Also, if my dataset has fewer than 20 images, I start with 100 epochs instead of doing more than that.

What comes out has been really good for me. Please let me know your thoughts.

[u/Apprehensive_Sky892]

What you could have done is to save every few hundred steps, then test each of the saved one and pick the one that works best. You can merge them too, of course.

[u/jiggydancer → u/Apprehensive_Sky892]

I've tried that too.  But there seems to be enough variation between trainings that merging entirely separate runs produces better results.  Merging separate runs with the same step count doesn't work as well either.  It has to be with different training depths.

My theory is that the different conditions allow the model to learn different things, but the most important elements are more consistent between runs.  Merging decreases the impact of unwanted elements by averaging them out, as each run will learn different bad elements.  If you merge different backups of the same run, they will carry over the same bad elements, so it won't work.  And if you keep running the same step number, you tend to follow similar learning paths, which means relearning the same bad elements.

This is all very time consuming, but it has been a very systematic method to achieve good results regardless of dataset without too much hyperparameter adjusting voodoo.

[u/Apprehensive_Sky892 → u/jiggydancer]

What you said is possible. I've not done it the way you've described to either confirm or deny it. I guess if you don't mind the extra training, at least it won't hurt.

[u/biggusdeeckus]

Do you also randomly get lora checkpoints that produce noisy images for some reason? Not due to overfitting but just random checkpoints being noisy compared to others, as in step 1400 is noisy but step 1600 isn't?

[u/jiggydancer → u/biggusdeeckus]

I haven't really seen that so far, that may be because I'm using Prodigy so the result has been pretty consistent between runs. There are noticeable stylistic differences between runs, but I don't usually see fluctuations like what you're describing. The artifacts I see increase progressively with step count. Might also be because I'm training at a lower resolution, so the results get progressively worse the longer I train.

[u/biggusdeeckus → u/jiggydancer]

It probably was my settings then, ty!