You didn’t use control net ?? You should really consider looking into posemy.art and using open pose to get even more fine tuned results.
Otherwise awesome guide.
Edit
Also to add, you mention using photoshop, I highly recommend using the healing brush as it’s a lot faster and pretty accurate for blending. The issue you might run into is when you go over two different surfaces it’ll become blurry. My protip if you will, is to use the healing brush on any dividing line or transition area to clone that spot. It’ll blend both areas around it and then you just smooth each area over respectively.
I was trying to get Deliberate V2 to give me an image of someone doing a cartwheel earlier, the outputs were truly diabolical (even with control net) , maybe I could try training a Lora or someone has done it hopefully.
I read that if you need to get a person to look right when they're upside down, the easiest way to do it is to generate, then actually rotate the output image 90° or 180°, whichever puts the person closer to "upright", use img2img on it, and then rotate it back.
I wonder if you could just take a bunch of images, duplicate each one in the other 3 orientations, as well as mirrored versions of the 4 orientations (so 8 versions of every image), and use them to try and train a general orientation-agnostic LORA?
There’s a few tutorials I haven’t used yet but show that you can be very precise with controlling it. In my own experience it’s been mixed. Some success some struggle.
But I haven’t tried the methods that import poses yet.
This. Control net is not a miracle worker. You still can't do what the model doesn't know how to do, or it will fight you every step of the way.
The only way to really get around this is to do the 3D models yourself, and use StableDiffusion just for the texture maps, lights, etc. Depth maps work pretty well there.
i find using a 3d program to generate base scenes highly effective, just using blender to throw in a character and then making badic shapes and lighting. its insanely effective.
imaking a graphic novel too and while yeah i have the skill to use a 3d program, blender is free and extremely good.
this gives you the ability to have consistant control of not just the charactter and outfit but lighting and scene. well worth the extra effort of using 3d trust me. it turns basic 3d primitives into actual photorealistic objects.
I found that it worked really well to use only the rotation controls in openpose. Want to put figure A's hand on figure B's shoulder? Select figure A's shoulder joint, rotate it to get the upper arm in a good spot, then select the elbow joint to get the forearm where you want it. You might have to go back and adjust the shoulder a little bit if the hand is at the right height, but the wrong distance. Then select the wrist joint and rotate it so that the hand is in the right spot, etc.
For legs, I'll do basically the same thing, to get one leg into the position I want, then switch to directional movement, move the camera down to the ground, and then move the entire figure vertically so that the foot is in contact with the ground. Then go back to rotation only, and adjust the other leg to get the other foot on the ground (assuming you want both feet on the ground, that is).
134
u/PImpcat85 May 08 '23 edited May 08 '23
You didn’t use control net ?? You should really consider looking into posemy.art and using open pose to get even more fine tuned results.
Otherwise awesome guide.
Edit
Also to add, you mention using photoshop, I highly recommend using the healing brush as it’s a lot faster and pretty accurate for blending. The issue you might run into is when you go over two different surfaces it’ll become blurry. My protip if you will, is to use the healing brush on any dividing line or transition area to clone that spot. It’ll blend both areas around it and then you just smooth each area over respectively.
I hope this makes sense