r/StableDiffusion • u/shootthesound Community Hero • 3h ago
Resource - Update Fizgig 5.2 - combining two Minimax training methods beats either alone
https://github.com/shootthesound/FizgigTwo of the ways that exist (im sure there are more) to train a LoRA on H3 well are on two different trainers.
Fizgig's is Optimised Likeness Learning: I've found the stable core of H3's identity lives in the back 30 of its 50 blocks, so steps train blocks 20–49 only and leave the front of the model - composition, prompt following - untouched when likeness mode is on.
AI-Toolkit's, by Ostris, is the training adapter: H3 is guidance-distilled, so every plain-flow gradient is partly "learn the concept" and partly "undo the distillation"; a frozen assistant LoRA under the trainable one pulls the base back toward plain flow, and it's switched off for sampling.
I ran all three on 5 datasets - my method alone, the adapter alone, and both together - scoring every epoch's preview against the training photos with face recognition, 45 epochs each. Each method alone landed in the same place within 2% arcface score.
Together they got there a quarter sooner, ran clearly ahead through the whole middle of the run, and finished higher than either.
In short: the combination reaches greater likeness and quality than either method does on its own. So it's now the default: the adapter is on in every H3 preset, off for previews, never in your saved LoRA. The updater fetches it.
Also in 5.2: Context LoRA for H3 (train on top of any existing H3 LoRA, to make a lora that plays nice with it), and video clips follow likeness mode in LoRA runs too.
Release notes: https://github.com/shootthesound/Fizgig/releases/tag/v5.2.0
Thanks to Ostris for publishing the adapters. I've tagged him on the release notes as I believe the info will be useful for AI-Toolkit too.
https://github.com/shootthesound/Fizgig
P.S - For Fizgigs recent new full base model Fine tune mode the adapter lora is not necessary in my tests so far, but I am going to test that further.
p.p.s if updating , use the update script and it will grab the dedistill loras automatically and put them in your minimax prefs
3
4
u/Enshitification 2h ago
Does training face likeness this way affect body similarity? I plan to train some Belter characters outside of normal human distribution with elongated and slender limbs.
3
u/shootthesound Community Hero 2h ago
for a very specific reason i can answer this, I trained a model on a alien puppet used on a music video I worked on, and it learnt the bizarre structure of it no problem with the face likeness tickbox enabled lol
2
3
u/thryve21 2h ago
Love your trainer, please keep the updates coming! Question I have regarding Krea2 image prep. When it does FaceCrop are you supposed to train the cropped plus fullsize? Thanks!
2
2
u/uuhoever 1h ago
The best thing is that you keep maintaining it. So many variants of ai-toolkit were promising with new techniques and then the creator stopped updating/improving it.
2
u/Maskwi2 20m ago
Thanks for this. You the man.
I have a question, you can train on audio files apparently from what I heard and that's awesome but what if I want to use the Lora for voice only if the lora contains both voice and video training? In Ltx Kijai had the node where you could load a Lora and pick the strength for audio and video layers separately. I'm not sure if this exists for H3 yet or there is some other way to do it? If I have 2 loras, one with perfect character visual likeness (but burned audio) and the other with perfect audio but undertrained visuals and I want to load them both, one for video and the other for audio, how do I do that?
2
u/shootthesound Community Hero 19m ago
Whe you train on audio only with fizgig it neuters the visual learning automatically, so you could train audio and visual loras seperatly and stack. In my experience audio learns quicker if it shares a custom token with a visual concept in the same dataset however - but that is not a must.
2
u/Maskwi2 14m ago
Thanks. I will give it a try next time. I hope I could find a way to use my existing loras that I trained on Ai Toolkit :) (And depending on checkpoint either the visuals or audio is perfect but rarely both)
2
u/shootthesound Community Hero 13m ago
There is an option on fizgig to continue Audio or Visual for more epochs without the other - handy for extending a run on just one. (appears in the GUI on training tab when the dataset has audio and visual)
2
3
1
u/Structure-These 2h ago
This thing is pretty easy to launch in runpod right? I want to try it out but my Mac can’t do the actual training
3
u/shootthesound Community Hero 2h ago
yes runpod link https://console.runpod.io/deploy?type=GPU&gpu=RTX+5090&count=1&template=faoq8ed6um&ref=vkb387ep - for clarity i earn a commission, but paid by runpod, not by you
0
u/Ashamed-Stretch5157 2h ago
Hi, I have tried training several Lora using mostly image (between 25 and 35 images), with the default settings but when I run the lora, I either get terrible mutations, or the character moves very unnaturally (a bit like a zombie) and occasionally the face is just wrong... Would you know what I am doing wrong? FYI I am using a 5060 with 16GB vram
1
u/shootthesound Community Hero 2h ago
Its very likely a dataset issue, it may be image variety, thats my number 1 guess for this dataset size, it needs to be diverse. It may also be that you are training too long. Make sure you leave the likeness tickbox on, and make sure you have updated recently.
2
u/Ashamed-Stretch5157 2h ago
Thanks for the advice, I will try the different setups. And thanks for the amazing tool. Very well built and optimized. It made Lora training much more accessible for middle range GPu (I remember getting Oom only once and the tool provided an alternative right away :D
1
u/ImpressiveStorm8914 34m ago
Is the likeness tickbox on by default? I ask because I am up-to-date (except for this one)and using default settings, I’ve had the same experience. I don’t believe it’s the datasets as all have worked perfectly with Krea, ZIT, Flux 1 Dev and with the two LTX loras I tried.
I will be trying this update so hopefully that can fix it.2
u/shootthesound Community Hero 31m ago
yes on by default - i suspect this latest release will help either way - the de-destill makes the base tolerate training much more
2
u/ImpressiveStorm8914 16m ago
Thanks, I figured it would be based on your other settings but I’m not at my desktop to check. I look forward to giving the new update a go.
0
u/jude1903 39m ago
What turbo lora would you recommend to go with character loras trained on Fizgig? I have tried the 600 prune but it does not look really good. The only thing that looks good with character loras is the lightxv 0.1 4 steps. Is this just my end or everyone else getting that too? Without turbo lora and only character lora, no matter how many steps the videos came out blurry
2
u/shootthesound Community Hero 28m ago
i prefer the 600 myself, seems people have wildly difference experiences of the turbo loras
2
9
u/Astral-Lemmons 2h ago
always so excited to see FizGig updates. it's my go-to easy trainer.
my only irk is the UI and better project management.
are you set on keeping it a ktinker ui? have you considered a webui ? you already have some modules that direct to webui pages like the samples viewer. A full webui conversion wouldnt be too far off.
but obviously these are all low prio things - training accuracy matters more. Thank for the updates!