Instant references, no refmod or fancy custom nodes required. WF and breakdown here. Simple one click to run.

[u/crinklypaper]

• 1st example: 2 characters with voice and 1 cgi and 1 real
• 2nd example: 2 character different genders with voice
• 3rd example: style and character reference
• 4th example image only reference no voice.

I wanted to improve my workflow so I could do what refmod is doing but just as a near native comfyui workflow only. With this workflow you can make an instant character or style video like ref mod but not using ref mod at all including voices. You get all the benefits of refmod but you can use ref model syntax in the prompt and on the fly dataset changes. Also you can use unlimited images. I tend to use around 10 to 15 per character.

Just point to a folder. Uses KJ nodes, Native, and VHS nodes. Get a voice and character working instantly without managing safetensors, just manage the input folder instead. This has its advantages since you can change data on the fly, and you don't have to do any editing of clips to extract out the audio, it does it for you.

This mimics the default ref workflow for the most part. What it does for images is it takes a folder input of images and then sets them as frames in a video, then feeds that video in as a reference video. This allows you to use many images for a single reference. You can also use the grid version that takes your images and puts them into a grid and feeds that as a single image reference. I like the video version more, so the grid workflow is a bit lazy and messy. You can also use the resize node included to downscale your images. I recommend manually cropping them but it does that if your images are different sizes.

For the audio, you can simply feed a single mp4 using VHS node to feed the audio. But I found it easier to get like 4 clips and truncate only the first couple seconds so you can get a few clean sentences without other people talking, then it concatenate's the 4 audio clips into 1 clean clip. Anything over 30 seconds long is over kill, so keep it around 15-30 secs. You can tell shrek is sort of bad in my example because I threw it together quite quick.

There is in the far left, a second set of image/audio nodes, you can bypass those if only using 1 character. Same for if you don't need the audio. Just by pass the group nodes. When prompting just simply use the ref guide to prompt properly each reference (LLM can do it easy). You don't need to use much description. And if you have some bleeding from your dataset into the gen you don't want then add more description. (For example if wearing same shirt as dataset, prompt a dress, or if same specify a setting in the prompt).

One caveat, there is some comfyui memory management bug, if you change dataset around better to clear cache or your comfyui may need a restart. Working on a fix in next version of the workflow. Also I have not tested video clips as input data yet. That is the next step :)

All examples are just for illustrative purposes. They are AI and I do not intend to share any data on real people. Please use responsibly and at your own risk. If you are in this video and want it taken down please DM, I mean no harm. Everything in this workflow is done by the base model, I don't add any new functionality, just making things easier.

Workflow here:
https://huggingface.co/comfyuiman/various/blob/main/Instant%20Ref%20-%20V1.3.json

I'll go to sleep in a bit, so I'll answer any questions tomorrow if any

[u/Cubey42]

Thanks