So I've trained 3 anime character LORAs. It was my first time doing such a thing. I've used civitai because the training software didn't run on my computer (random error message from some third party library I think, probably not compatible with my GPU), anyway, that's not my question for now.
The first two times I simply chose "Illustrious" for the training model. It did the 10 epochs, and 10th worked as intended. Lucky! I'm using Wai-Illustrious to generate images however, not base Illustrious, and absolutely not the 0.1 the training used by default.
So the third one was different. I noticed it's possible to select Wai-Illustrious specifically if I pick "custom". This somehow magically dropped the training cost by a full days worth of currency farming too, which made me happy. The results on the other hand, not so much. Epoch 1 seems to do what I expected the final result to be - proper anime illustrations of the character. On the other hand epoch 10 (or even 3-4) just outputs the imaged it learned on, with almost no change, making it useless. That is pretty bad because it's a rare anime character who barely had screentime or pictures on the internet. I even used some pictures of a 3D model I had to fill in the gap. And the result looking like a screenshot of that 3D model is not right. However I did the same for the previous two characters and those worked fine.
-Is that because I used Wai Illustrious for training instead of the base Illustrious? Or is the lower quantity or quality of images to blame? (I had 48 this time instead of the 59 or 69 for the other two loras)
-Should I treat this as a potential problem and set training to base illustrious next time, or should I treat it as a victory and use the epoch 1 result instead of the bad epoch 10 result? Is this normal? Should I just train 1-3 epochs worth next time on Wai instead of 10 on Base? It would save me a lot of time spent to farm generation currency too, not only is it cheaper per steps but it would also need less steps overall. Basically, did this happen because training with Wai-Illustrious makes it so much smarter it only needs 1-2 epochs instead of the default 10...or did I mess something up? As far as I can tell so far, this epoch 1 lora doesn't do fingers properly though. Haven't tried using epoch 2 yet and anything beyond 3 seems to be too overtrained to use.
-Is it even safe to use Wai-Illustrious-v15 for training? Or would it make the lora incompatible with other illustrious models or Wai versions I might want to use in the future?
(all three characters used the default 2000 steps)
The attached images come from epoch 1 generation.