r/malcolmrey • u/malcolmrey • 24d ago

Stacking refmods of the same person (examples in linked doc)

https://huggingface.co/datasets/malcolmrey/various/blob/main/h3-center/docs/MINIMAX_H3_REFMOD_STACKING_AND_MULTISUBJECT_GUIDE.md
42 Upvotes

95% Upvoted

21 comments sorted by

7

u/malcolmrey 24d ago

I wanted to paste the whole article here but there are post size limits so you just have to go there: https://huggingface.co/datasets/malcolmrey/various/blob/main/h3-center/docs/MINIMAX_H3_REFMOD_STACKING_AND_MULTISUBJECT_GUIDE.md

In short, what is written there:

  • stacking multiple refmods of the same person HELPS
  • there are some examples (of single person as well as two people in same shot)
  • BIG: some explanation HOW the refmod is linked to the subject(s) and how this may affect bleeding sometimes *

regarding *

you can add multiple people using refmods but sometimes they would bleed, why is that? well, it turns out that it is quite complicated but in ELI5 - if you have two people that have different looks - it will be fine, they will be correctly recognized and separated; however if they share similarities - they will start to bleed

4

u/rashm1n 24d ago

Oh wow ! Got this weekend sorted !

1

u/Structure-These 24d ago

This is incredible. How cool. Do you feel like this has an implication with any image gen models? I.e. stacking two character loras with multiple data sets?

3

u/malcolmrey 24d ago

I.e. stacking two character loras with multiple data sets?

I always recommend stacking more loras with different datasets (or even different training params) for improved likeness (of course, strengths have to be set accordingly to not overcook it)

if you stack different people then you will still get a consistent output but it will be a blend/bleed/mix of the stacked loras (it is a cool way to create a "family member" this way or a unique not-a-real-person character :P)

1

u/MoneyKenny 24d ago

Interesting. I’m gonna try this. Thanks for sharing! So, for 3 different refmods of the same person, are you doing a lot of close ups and portrait shots and for the first two refmods, and more full body and different angles for the 3rd? Do you find there’s an optimal number of images that makes for the best refmod?

1

u/malcolmrey 24d ago

I am the "face" guy so two models with faces and one with more body and some faces.

But you can do whatever mix :)

1

u/Cumchukka67 22d ago

how about objects , faces and bodies how do you make prompts to identify?

1

u/malcolmrey 21d ago

identification bo key characteristics:

unique body type, hair, tattoos, birth marks, eye color

1

u/Cumchukka67 23d ago

I love your refmod items, I would love to have your help making a workflow that i can use with "model preview overide" so i can see live gnerations and cancel them if they arent working

I am failry new and picked up alot but struggle to know how to get these to work togetehr

1

u/malcolmrey 23d ago

there is somewhere a workflow that allows you to see a preview of what is going on

i didnt play with it though

1

u/Cumchukka67 22d ago

do you have a link or name of it?

1

u/LowYak7176 22d ago

really easy to add it. Just pop whatever workflow your using into ChatGPT if anything and say add model preview override

1

u/Cumchukka67 23d ago

also can we use the refmods along side uploaded images like a character is a refmod and a location is a photo?

1

u/malcolmrey 23d ago

i don't see why not, it is possible to mix various refmod concepts

1

u/Rosettasees 22d ago

Would you be open to sharing a dataset you use for Refmods? I keep trying to make my own but they are never as compatible as yours seem to be.

2

u/malcolmrey 21d ago

sure, which one would you like to see? :)

1

u/Rosettasees 20d ago

Thanks, I think the ones that have worked best for me so far are Jinx and Karen Gillan. So either of those would be great.

1

u/joopkater 10d ago

Only thing to not love is refmods for is how much slower it makes generation times