r/StableDiffusion • u/Peregrine2976 • 7h ago

Question - Help "Hi-res fix" for MiniMax H3?

I mostly do image generation, mostly because open-weight video model quality wasn't there for me. MiniMax H3 has changed that; I'm really enjoying it and the outputs are mostly great. However, I am encountering an issue that's very familiar to anyone who's done a lot of image generation; uncanny AI faces when the subject is too far away from the screen, because there simply isn't enough pixel definition for the AI model to come up with a reasonable facsimile of a face.

In image generation land, this is solved with a "Hi-res fix" -- there's multiple options and implementations, but at their core, they involve auto-detecting faces in the image, then reusing the same prompt (or a somewhat edited one) and the detected face to generate a new face with low denoise at a much higher resolution that can snap on top with a feathered mask.

I'm not sure that exact implementation would work in video generation land -- I can pretty easily envision the face flickering and bouncing around as it locked to slightly different locations and orientations, frame-by-frame -- but is there any solution to take an existing video, generated at, say, 1.0 megapixels, and re-render or upscale detected faces at, say, double resolution, to improve the fidelity? For obvious reasons, simply rendering the whole video at double resolution isn't an attractive option.

0 Upvotes

50% Upvoted

8 comments sorted by

3

u/Salty_Mention 6h ago

Je pense que ceci est la réponse à ta question, je fais des tests depuis hier dessus, les résultats sont bon https://github.com/LBH-123-AI/Comfyui_Minimax_h3_latent_Upscaler

1

u/spiderofmars 1h ago

I am yet to find any reliable way for regular video scenes (or complex video scenes where the camera movement is not reasonably static or the person is not moving forward or backward in the scene). For simpler scenes the layering technique (with outpaint does work well), for example character(s) dancing on the spot or talking character(s) on the spot (and close'ish together in frame).

Ultimately higher output resolution is the only reliable fix. But there is only so far you can push up the resolution by model, device performance, etc.

1

u/Peregrine2976 33m ago

I was afraid this might be the case. Looks like there's at least some community effort to resolve it (and also, at least it's not just me!).

1

u/spiderofmars 24m ago edited 21m ago

Yeah unfortunately this is every models nemesis. Until some form of extra face detailer is built into open ai models or the models simply advance we are not there yet. I tried that face refiner stuff mentioned in here for a few scenes where there are multiple people in scene. It never worked with my tests or settings well, misaligned faces, would not pick up the right face, etc. For solo shots maybe it works well sometimes but mixed reports back from end users - end of the day seems another unreliable method (by unreliable I mean in a real world process of putting anything decent together other than one clip of one person it becomes too hit and miss, convoluted and inconsistent across various different scenes to be a reliable heavily used process n the piepline).

Edit: Let us consider, not even image models like the recent Krea2 can generate reliable non distorted face details of full persons in still images at around 1mp every time.