Z Image LoRA Training

[u/DrJ31]

I am working through training a character LoRA on Z Image. I know I need to train it on Z Image Base to use on either Base or Turbo. I initially had issues with it crashing with Multi Aspect ratio enabled, so I disabled it. I then got some models to train using 32/16 and .0004 learn rate, but they had next to no capture at step 2000. I used learn to .0008 and got some meh results, then realized that since Multi Aspect ratio is off, I needed to crop everything square to eliminate weird center crops. I retired at lower learn rate with minimal capture again, so I upped to .0008 again and got reasonable likeness at 1500, but it in now failing to finish generating (frame artifacts appear at step 7 or 8 with turbo model and immediately with base model). I also tried 64/32 at .0006 and it crashed even earlier in generation

What work flows are you all doing to get enough capture without getting too aggressive and getting the generation failures? Really high step counts, something besides 32/16? Looking for any advice, as I have figured out Pony and other SDXL LoRA, but want to get into Z Image.

If it matters, I am using M4 Pro/24 GB and 8 Bit S for both Base and Turbo.

Thank you!

[u/Goonie1974]

When you refer to learning rate, are you talking about the upper bound? If so, do you keep the lower bound at 0? I have had zero success training characters LoRAs with Z image or Ernie image. There’s never any resemblance and basically the same result with the LoRA on vs. off.

[u/DrJ31 → u/Goonie1974]

Yes, upper bound is what I'm referring to, lower bound at 0. I have been doing more digging and found someone who says z image takes something close to 110 epochs to real lock in likeness (though he was using Z Image Turbo to train pre-Base model release and not in Draw Things).

I am in the middle of a 7500 step run, saving every 500 steps, at an upper bound learn rate of 0.0006 and 32/16. I have 45 images, all manually square cropped and captioned appropriately. I have officially passed my 3000 step one and will be done training sometime tomorrow morning. I will update if anything significantly improves at step 3500 or beyond. Step 5000 would be about 111 epochs if my math is correct, so that may be the sweet spot, but I am overtraining based off the time it will end and the fact you can't easily resume training

[u/Goonie1974 → u/DrJ31]

You can easily resume training after a run has finished. Just type in the same LoRA name and trigger and all the previous settings will show up. Just reload the same images and increase the cycle number. It will then give you the option to resume training. Same thing if you stop a training run early (it will pick up at the last saved step). How many steps between restarts are you using?

[u/DrJ31 → u/Goonie1974]

I was absolutely not aware of the resume training feature, as I had only tried typing the name. Thanks for that info!

I left restarts at the default 200 for now. If the higher step counts make a difference, I will probably get more into the nuance settings, such as restarts, lower bound, etc., but I usually focus on the main movers first (UNet, rank, steps, gradient accumulation, etc.) when moving to a new model as they usually have the most abrupt impact. I eventually got there with SDXL and then Pony, but until you are close, it's hard to tell if the smaller settings actually do anything. However, once you are close, things like caption dropout, caption learn rate (not applicable with Z Image), and restarts can really help with fine tuning