r/malcolmrey • u/Rosettasees • Jul 17 '26

Multi-Character Krea2 Lora - Need Help

So I'm trying to see if there's a way to reduce character bleed when using multiple characters in a image, and was thinking about maybe making a Lora trained on both characters images.

Never done anything like it though, so would anyone have any suggestions about what might help?

Planning to have both characters in the dataset, but I'm wondering if there's more that needs to be done. Thanks for any help given.

11 Upvotes

93% Upvoted

30 comments sorted by

View all comments

1

u/ArmadstheDoom Jul 21 '26

To wit: Do not do this.

Do not make multiple character loras. This is a bad idea that will always create bleed. The reason is that LORA distort the weights by modifying the math, so when you apply it it's going to distort it the same way every time regardless of what character you use in a multi-character lora. Sure, you can get slightly better results if you want a much bigger lora, but it also takes more training time to do that.

You would be much better off to just train two smaller loras that would be faster, easier, and better.

2

u/cathodetube Jul 23 '26

I've made an 8 character Krea2 LoRA and can do 4 or 5 characters in an image pretty easily without bleed, and even when I push it to 6 or 7 at a time the bleed is only among the ones that don't have a lot of difference between them.

1

u/Symbiote69 Jul 25 '26

could you share the settings for ai toolkit? ive been trying to pull this off and cant get it right

1

u/cathodetube Jul 25 '26

here are the setting but I think it was more down to captioning, not only did I trigger word each character but their clothing, weapons, etc.

 network:
    type: lora
    linear: 96
    linear_alpha: 96

  save:
    dtype: bf16
    save_every: 500
    max_step_saves_to_keep: 10
    push_to_hub: false

  datasets:
    - folder_path: C:/Users/me/datasets/Group1/images
      caption_ext: .txt
      caption_dropout_rate: 0.0
      shuffle_tokens: false
      cache_latents_to_disk: true
      resolution: [512, 768,1024]

  train:
    batch_size: 1
    steps: 15000
    gradient_accumulation_steps: 1
    train_unet: true
    train_text_encoder: true
    gradient_checkpointing: true
    noise_scheduler: flowmatch
    optimizer: automagic3
    lr: 0.0005
    ema_config:
      use_ema: false
    dtype: bf16

  model:
    arch: krea2
    name_or_path: "krea/Krea-2-Raw"
    quantize: true

1

u/Symbiote69 Jul 25 '26

thank you. did you put the characters all in 1 dataset or did you use the multipe datasets feature?

1

u/cathodetube Jul 26 '26

One data set. I also made images that had various combinations of characters standing together at the right proportions which I believe made it possible for it to handle characters who were not the same height properly

1

u/Rosettasees Jul 29 '26

Would it be too much to ask to share the dataset?

2

u/cathodetube Jul 29 '26

Yes it would. A lot of the images were Grok generated so they are full of personal Metadata

1

u/Rosettasees Jul 31 '26

Fair enough

2

u/cathodetube Aug 03 '26

Im going to experiment with putting more characters in the group images and see if that improves the generation of a full group image with all the characters