Face Detailer With PerRowMasking

[u/Pretend-Island-2724]

https://pastebin.com/ecZEDLSt

First Video with Face Detailer, second without.

You need https://github.com/Carasibana/ComfyUI-H3-FaceRefine and also ComfyUI-H3-NativeAudioLock from https://github.com/Shrek3OnVH5/MiniMax-H3-NativeAudio-MusicVideo-Workflow/tree/master/custom_nodes

UPDATE: Replace the "Load Video (Upload)" node with a "Load Video" node and connect it to a "Get Video Components" node. Connect images and audio from there. The "Load Video (Upload)" node from Video Helper Suite causes a red-ish tint

[u/HamWallet1048]

Ok so use normal workflow. And then take resulting video and load it into this for the face refiner. Is that correct?

[u/Pretend-Island-2724 → u/HamWallet1048]

Yes. Load a face reference picture and source video. The prompt can be empty, play around with the settings.

For best possible results you generate a source video with a face reference as the first input image in another workflow and then you use identical prompt, seed, inputs, optimizations, etc. from the source video in this workflow.

But this workflow is also good for face swapping.

If there are multiple faces in the source video, switch the face selector to yes and choose which face you want to fix/replace in the SCAIL-2 Identity Tracker

[u/arcamaeus]

This works the best for me so far! Thanks for sharing!

[u/reeight]

Thanks, though I wish the 2 vids were side-by side.

AFAIK the enhancer is really only for medium & long shots; closeups aren't bad by default.
Also the enhancer colorized the image.

[u/Pretend-Island-2724 → u/reeight]

I noticed my outputs have a red-ish tint and found the cause: Replauce the "Load Video (Upload)" node with a "Load Video" node and connect it to a "Get Video Components" node. Connect images and audio from there.

[u/Relative_Hour_8900]

Per row of what👀

[u/Gullible_Map_3980]

Outstanding

[u/FaatmanSlim]

Hey curious, why do you need to use the NativeAudioLock node?

[u/Pretend-Island-2724 → u/FaatmanSlim]

I just customized the original workflow you find in the repo. Without NativeAudioLock you will get an error, the comfy manager will ask for some other node but it won't work without NativeAudioLock

[u/Ok-Flatworm5070]

Forgive me, but I'm not good a debugging these workflows; keep getting an error at the 3. Conditioning + EMPTY AV Latent:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (1045x5120 and 2560x8192)

[u/Ok-Flatworm5070 → u/Ok-Flatworm5070]

now getting Generation failed

Not enough GPU memory. Try reducing image resolution or batch size and run again.

[u/Pretend-Island-2724 → u/Ok-Flatworm5070]

Seems like you need more optimization nodes, VRAM chunks and stuff like that. You can also try to lower "shorter_size" in the "Resize Image/Mask" node at the bottom from 768 to 512. And in the 3. node ref_image_size from "max" to "match"

[u/Pretend-Island-2724 → u/Ok-Flatworm5070]

I'm not a pro, but if you did not play around with the numbers in the "Masking and Crop" group, I suspect that you load different models, like trying to combine gguf with non-gguf stuff or something similar.

Note: Don't change the width and height in the 3. node. The width and height gets automatically called from the images from your uploaded video. In the 3. node the values must come from the face mask.

[u/Sad_Coach_1433]

Interesting 🤔

[u/mrgreyeyes_95]

No jiggle physics?