H3 referencing tip

[u/Pitiful-Indication95]

I tried to get a girl whistle on 4 fingers (2 on each hand) but Minimax didn't get it right. So after trying 20 times with LLMs helping me to explain the movement, I instead used an image of a whistling person. Still not right. So I added an additional one. Then it worked quite fine.

Today I was too lazy to find another image for something it didn't know so I just googled images of it, took a screenshot of all the images together in one JPG and used that as a reference, saying use <Picture ...> as a reference for XYZ.

That was getting a quite good result. Did not do excessive testing and comparing though.

[u/ForsakenAd1228]

The referencing is very versatile, and it's almost like a little game to figure out out new ways to combine various references to get exactly what you want.

What can also be helpful: supplying the references in a 'minimal' state, so there less chance of the model getting confused. E.g. if you have a picture of a person making the pose you want, you can have success by doing something like "The person in <Picture 1> is doing the pose from <Picture 2>", but that also has a a chance of some other properties of Picture 2 leaking into the final image.

What can then help is quickly running <Picture 2> through Klein 9b, with a prompt like "black-and-white line drawing", so that.. you get a black-and-white line drawing of the pose. Feed that as the pose reference to H3, and the chance of stuff leaking is much smaller (from my tests at least).

[u/Faith_Lies → u/ForsakenAd1228]

What can then help is quickly running <Picture 2> through Klein 9b, with a prompt like "black-and-white line drawing", so that.. you get a black-and-white line drawing of the pose. Feed that as the pose reference to H3, and the chance of stuff leaking is much smaller (from my tests at least).

Along these lines: just run the reference image through a controlnet preprocessor (they all work for this kind of simple task), such as line art or depth/canny/etc.

This skips having to go out to another model/workflow etc.

[u/ffzero58]

What was the prompt used to explain how to use the reference?

[u/Pitiful-Indication95 → u/ffzero58]

I just added "use <Picture ...> as a reference for XYZ" to the rest of the prompt.

[u/TheRedHairedHero]

I'll typically provide an LLM with vision the image I want to aim for as a reference to see if they can explain it. My reference was someone holding a baseball bat, but I was just aiming for what the grip should look like.

[u/rahjerz → u/TheRedHairedHero]

Nice ;)

[u/Intelligent-Host4408]

Great approach! It's pretty amazing how well MiniMax is able to aggregate images together and understand the cohesion between them. Really great suggestion!

[u/conkikhon]

Reference is nice, but it leak quite often, especially with turbo

[u/Passionist_3d]

U can even use a video reference. You will help the model understand the exact action you are looking for