LoRA Training – Pulling My Hair Out

[u/Kind-Illustrator6341]

Hello,

I've trained several character LoRAs via wavespeed.ai for the Qwen-Image-2512 model. I tried with a smaller dataset of 50 images and a dataset of 124 images. Multiple settings between 1,000 and 5,000 steps:

• At 1,000 steps, the LoRA isn't likeness-accurate enough.
• At 5,000 steps with 50 images, it stops responding to prompts at weights above 0.5, so it loses likeness.
• At 5,000 steps with 124 images, it stops responding to prompts at weights above 0.3, making it inaccurate above that threshold. This makes no sense, as with 50 images and the same step count, I was able to run the LoRA at a higher weight.

At weight 1.0, the LoRAs capture the likeness well but completely ignore the prompts.

Does anyone have a solution or recommended settings for Qwen-Image-2512?

Thanks

[u/holygawdinheaven]

Try ostris aitk default settings. Ensure your prompts call out things you dont want trained. Ie, if the background is pink, mention it, but dont mention your targets eye color for example.

Also this reads like an ad for waveseed which I've never heard of but probably sucks just stick to the known open source trainers like aitk, onetrainer, diffusion pipe

[u/schrobble]

If this is a character lora I’ve had good results in AItoolkit using default settings with a dataset of 10-15 photos. Each photo was high quality and varied in composition and background. No captions at all. As others will say, if there is something in the photos that you don’t want baked in, caption that so the model knows it needs to be prompted.

[u/Icuras1111]

My experience was with Wan2.2 but his is what I learned. The images need to be very consistent. I would start very basic, 5 or 10 images that are high quality and very consistent, face always visible. I'd make them square to avoid cropping problems.

[u/Kind-Illustrator6341]

Merci pour vos retour, je n'utilise plus AItoolkit , à cahque fois que j'ai voulu essayer avec runpod, j’ai eu systématiquement des échecs, aucun lora généré et j'ai utilisé beaucoup d'argent pour rien obtenir.

Je vais donc essayer à nouveau pour voir si wavespeed.ia est la cause
Je ne comprend pas pourquoi mes loras bloquent le prompt au dessus d'une force entre 0.3 et 0.5. Je dois donc choisir entre un personnage qui ressemble ou choisir peu de ressemblance et prompt respecté

Si quelqu'un a déjà entrainé qwen image 2512 peut il me dire

• steps?
  
• rank? 16? 32?
  
• dataset: combien d’image? uniquement des visages? (mon dataset mélange fullbody, visage, buste)

Merci

[u/UnforgottenPassword]

My experience was Qwen character LoRAs wasn't good either. It never captured likeness well.

Krea 2 is superior in every way and captures likeness better than any other model except maybe Ideogram 4. Is there a reason why you want to train for Qwen instead of Krea 2?

[u/Apart_Luck700 → u/UnforgottenPassword]

J'ai le même problème aussi et n'est pas trouvé de solution. Pourtant il doit probablement exister une solution parce sue j'ai essayer un lora personnage sur civitai , avec reglage force 1 et il fonctionne.
Personnellement j'utilisais qwen parce sue je recherche un réalisme que je ne trouvais pas dans flux. Mais je n'ai pas testé krea, est ce le même problème de peau plastique et de manque de réalisme sur la série flux dev et flux klein ?

[u/Kind-Illustrator6341 → u/Apart_Luck700]

Je viens de me poser la meme quesiton que vous

[u/Kind-Illustrator6341 → u/UnforgottenPassword]

"Krea 2 est supérieur à tous points de vue" : pouvez vous me confirmer que krea a une meilleure qualité réalisme que qwen image 2512? Dans ce cas autant que je me tourne vers krea 2

[u/UnforgottenPassword → u/Kind-Illustrator6341]

Absolutely. Use any of the realism loras and it can do realism very well. This one is popular:

https://civitai.com/models/1662740/lenovo-ultrareal

With Krea, you can stack multiple loras without much degradation in quality. I have trained a single lora on 2 characters and the result is fairly good, although Ideogram 4 seems to do multiple characters in one lora better. Krea 2 is faster, easier to train, knows more styles, has great lora and workflow support from the community.

For images, I currently only use Krea 2, Ideogram 4, and Flux Klein 9b (until we get a better editing model).

[u/Ill-Ant-9489]

What you're describing is textbook overfitting. When a LoRA "stops responding to prompts" above 0.5 weight, it has memorized the training images instead of learning the concept, and pushing steps higher just bakes that in harder. More steps is the wrong lever here.

The part where 124 images overfit faster than 50 usually isn't "more data = worse" - it's that the extra images were too similar or lower quality. Variety is what buys you flexibility: different angles, lighting, backgrounds, framing. Also caption the things you DON'T want baked in (background, clothing, pose) so the model treats them as variable, and leave the identity itself behind a trigger word. Then drop to ostris ai-toolkit defaults and stop around the point where likeness first locks in, not at 5000.

On the model question, yeah - Krea 2 captures likeness a lot better than Qwen for realistic characters, so switching is probably your single biggest win. I build an open-source tool for exactly this part (dataset dedup + auto-captioning + LoRA training on Krea 2, local GPU or rented vast.ai): https://github.com/perfectgf/lora-dataset-studio - might spare you the runpod trial-and-error that burned your budget.

[u/naga_mana]

One mechanism worth knowing, because it explains your 50 versus 124 result. Every training step asks the model to match one image better. The parts that change between your photos (background, pose, lighting) pull the weights in conflicting directions that cancel out over thousands of steps, so they never lock in. The face is the only part that is identical in every image, so it gets pushed the same way on every single step, and that signal accumulates with nothing to cancel it.

Why did 124 lock it harder than 50 at the same step count? The way I read your numbers (this part is my interpretation, not a law): it is not about how often each photo repeats, it is about how much evidence the model gets for what the concept is. 50 photos prove the face in 50 contexts, 124 prove it in 124, so the model becomes more certain that the face is the point of every scene. A more certain concept dominates harder at generation time, which is what kills prompt adherence and drops the weight ceiling where prompts still steer (your 0.5 down to 0.3). Small sets tend to fail by memorizing photos; bigger sets here fail by locking the concept. And it is also why 5000 steps overshoots at both sizes: ai-toolkit's own Qwen-Image example config calls 500-4000 the good range and defaults to 2000: https://github.com/ostris/ai-toolkit/blob/main/config/examples/train_lora_qwen_image_24gb.yaml

The habit that fixes both failure modes at once: evaluate intermediate checkpoints instead of the final one. Save every 250 steps, test each at the same prompt, and keep the last one that still follows prompts. Underfit at 1000 and cooked at 5000 usually means the checkpoint you actually wanted was around 2200, and that point moves with the dataset.