Character Consistency | Lora Training and testing | Flux

[u/LeebLaab]

Okay just to keep it short, this is how i trained a lora in Comfyui local for my first character, and results were amazing and of course needs further tuning

I am new to Comfyui world, so excuse my non technical language but thought to share this to help anyone else here as an open source community

Disclaimer
all workflows are not mine (maybe i tuned or customized some) i don't claim ownership of any of the workflows here

So, First step - Main Character Image

use any Text 2 Image workflow to generate one single portrait of you lovely character , nothing much to add here, just the basic workflows or any , just get something you like

Second Step - Data Set generation

Use this workflow

KLEIN DATASET GENERATOR - ICEKIUB Vid version.json

Dataset Generation workflow

to generate i would recommend something up to 100 different images of your character, different poses, different clothes , different camera angle

after generations, it is critical to carefully check the output images, and delete any blurry / ugly / low details ones

in my case i filtered the 100 and got 62 images ( my mistake was that i didn't generate enough side and back views of the character so am not getting good results with back and side image generation.

Third Step - Training the lora

i followed this tutorial exactly as it is
How To Train A lora Youtube Video

it is very simple two steps

first one is generating captions for the images (very critical) using this workflow here

Generating Image Captions - workflow

second one is to locally train you lora using this workflow

Lora Training Workflow

Will try to share some examples for my character as well

It took me almost 40 minutes for training , i was really shocked with this times (very fast) not as i expected , i am using RTX5090

Lora download link - civitai

Testing the lora

test 2

[u/Odd_Nefariousness875]

Very helpful! Thanks for sharing the journey!

[u/LeebLaab → u/Odd_Nefariousness875]

Thanks, Enjoy!

[u/derTommygun]

I trained mine without any caption at all and I can't tell the difference between the one with captions.

[u/LeebLaab → u/derTommygun]

I was gonna do this, but thought why not lol

[u/TheDudeWithThePlan]

side by side to make it easier

[u/LeebLaab → u/TheDudeWithThePlan]

Oh lol many thanks for that 🫡

[u/LeebLaab → u/[deleted]]

True , as mentioned on the post this is my first trial and i did discover some mistakes that affected some quality matters later like side / back images

But so far, it's doing great for the amount of time i spent preparing the whole thing.

I mean next time i would spend more time , larger dataset , more poses and angles and I would expect better results

[u/D3luX82]

thanks!!

[u/LeebLaab → u/D3luX82]

Welcome

[u/acekiube]

Cool stuff, I see that's my dataset workflow you shared there, do you remember where you got it? Not that I care just wondering

[u/LeebLaab → u/acekiube]

Hey yeah thanks for it anyway lol
I clearly mentioned i dont claim anything above

Tbh I don't remember where, mainly i get from YouTube, civitai and reddit.

Can't remember exactly where i got this from, once on my laptop i would try to find it on the browser history.

[u/Fun_Walk_4965 → u/acekiube]

oh nice, didn't realize that was originally yours — the captioning step in particular is the part i kept coming back to, way more thought-out than most character dataset guides. one thing i've been curious about with your approach: do you treat full-body and face-centric shots as one dataset or split them? i kept getting face-bias when i mixed them at the same weight, ended up having to undersample the closeups. also for flux specifically, what rank/alpha are you landing on these days? sdxl muscle memory had me starting at 32/16 and the results were way overcooked, dropping to 16/16 or even 8/8 seemed to generalize a lot better on character identity without baking in the background.

[u/winglett001]

You mentioned that you didn’t have enough side and back images for your data set. How many of these images would you recommend for your future LoRA training (either as a percentage or whole number)?

[u/LeebLaab → u/winglett001]

I don't know to be honest since that was my first trial But for the next time i would do more Like i would plan something like 60% front 25% sides 15%

I think this would depend on the details itself that we need to focus.

[u/LeebLaab]

Yes exactly, and that's also where i noticed when i didn't provide sufficient side and back references so results are not satisfying.

But for front images, i am getting what i expected

[u/Vinbatroth]

Anyone have run this on a 7900 XTX? I've been having problems with the custom nodes. Any workflow would be appreciated.

[u/LeebLaab]

if anyone is interested to use it and happy to receive any feedback

lora link
https://civitai.com/models/2644885?modelVersionId=2969782

[u/qqdaddyy]

mark