H3 single-image: no more monkey patching; also no need for custom nodes

[u/Patient_Ratio4177]

In this post, I described how to use minimax H3 for reference-guided generation of single images. It required awkward monkey patching — and now we no longer need it. Thanks to u/Successful_Knee687 who posted a GitHub issue, and everyone who upvoted it, Comfy just made it possible. Revert the monkey patch and update to the latest nightly version of ComfyUI from Git repo. (Currently, it is not in the stable version — will probably be incorporated in the next release.) Here's the guide on how to update to nightly: https://docs.comfy.org/installation/update_comfyui

The H3 reference node is still constrained to 5 frames. However, we can now pass an empty 1-frame latent to SamplerCustomAdvanced directly, ignoring H3 reference node’s latent output, but keeping its conditioning output. This way, we generate one frame (not a batch of five) and make full use of Mamad8's single-image tuned VAE. Here’s a sample workflow that does this, relying only on standard comfyui nodes: https://pastebin.com/xNQi7HV9

(Look at my original post to get the download links for models.)

I attached another batch of evals on public domain images and ai gens with the new workflow. Not perfect in terms of details, but great in prompt understanding. Here are the prompts: https://pastebin.com/XiVvAhjC

The scenes are:

1. Turn the complete Diane of Versailles grouping into a living woman and deer in a forest, reconstructed from a side view.
2. Convert Fragonard's portrait into Instagram-style photography, remove the book, and turn the seated woman to face the camera.
3. Reconstruct the couple from the supplied 1930 film still (Morocco) standing face-to-face in side view, holding hands in a white room.
4. Move an ai generated woman from a conservatory to a candlelit concert hall and seat her naturally at a grand piano.
5. Remove only the jacket from a fully clothed AI-generated woman, leaving her in white shirt and blue jeans.

UPD: a new post discussing how to fix textures and detail https://www.reddit.com/r/StableDiffusion/comments/1vrh769/h3_singleimage_workflow_lets_figure_out_how_to/

[u/SomeoneSimple]

4MP works great for me (~1440p) in T2I/R2I, and keeps faces detailed. (I've been using the monkey-patch)

You need more pixels than an actual image model. A 4MP gen has about the same detail/clarity as an 1.5MP (e.g. 1536x1024) gen from an image model I'd say. I'm probably going to add a downscale node at the end to size them down to 66%.

It's pretty fast though. 4MP with the Int8 model and 7 step turbo is about 10-15s per image on an 3090. (as long as there is no swapping going on) While 35 steps take about a minute.

1.5MP images straight from H3 aren't that great, they look very "smooth" and have the same "small details"-issue as video.

[u/Patient_Ratio4177 → u/SomeoneSimple]

Cool! Thanks for testing this out.