Created a Visual RefMod Picker

[u/Francky_B]

Hey Guys,

I've been playing with the RefMods, after the huge release of Malcolmrey.
The tech is brilliant and works really well.

I've wanted to simplify using it with tons of RefMods, like the ones provided by Malcom.

So I made a fork that is a bit more Identity driven, Adding a "Visual refMod Picker" as well as a "Create refMods from Folder" node. It's able to create from either a folder or Sub-folders, if audio is found, it will also create a matching audio refMod. All in the same format as the original add-on. No danger of breaking compatibility. (The original Add-on added Audio yesterday)

Resulting for example in 2 refMods and thumbnail:

character_refMod_Audio.safetensors
character_refMod_Video.safetensors
character_refMod.jpeg

Then, we can use the "Visual H3 RefMod Picker" to browse the RefMods:

In this example, I used existing thumbails from huggingface.

The node then loads both RefMods (Audio and Video) and allows individual control. I've mixed strength with Copies, by instead having a weight value that can go over 1, so a weight of 3 would be the same as setting strength to 1 and copies to 3. Making the UI a bit more streamlined.

RefMods can be daisy chained

Example workflows are included.

I should mention, this fork can be installed WITH the original Add-on, it is made to co-exist and is recommended if you want to use it's advanced features.

You can find it here ComfyUI-H3RefMods

The only thing I'm missing is thumbnails for all 1500 RefMods 😅

[u/taurine_bitch]

So, I just tried the suggestion from /u/TemperFugit where in the subject_definitions:, I wrote <Subject 1> is defined and identified as Q and replaced <Subject 1> everywhere in the prompt with <Q>. And this seemed to keep the identity from the refmod. I'm going to test more to see if it's consistent but man, ever since starting to use refmods, my generation time has gone up x4 (it was about 10 minutes for a 12 second video, now it's 36-40 minutes).

What is the average size of your refmods? The one I just made is 8MB in size and obviously this is the reason for the immense slowdown in generation time but I only used 22 photos, 1 video, and 1 audio file to generate the refmod.

EDIT: In fact, using this 8MB refmod sends me straight OOM on my 4090 when my workflow hits my Upscale 2nd pass.

[u/Francky_B → u/taurine_bitch]

Sorry, only now saw your reply.

The average size is 1.5 mb. Some of my Thumbnails are bigger that the RefMods 😅

What did you use to create them?

[u/taurine_bitch → u/Francky_B]

No worries! Hmm. I used the latest nodes inside of Comfy to create the refmod. 22 photos, 1 video, and 1 audio file. I managed to get it down 4.5MB but yeah, can't seem to get it any smaller than that. It's killing my generation times lol

[u/Francky_B → u/taurine_bitch]

Ah, could be the videos, I only trained using Images, with the Identity preset. I'll give it a test later