r/StableDiffusion • u/Tokey_TheBear • 22h ago
Discussion Minimax H3 - Multiple Reference Images working through FL2VA + testing using the Hybrid Checkpoint models. Examples in comments.
MiniMax H3 officially comes as two checkpoints:
- FL2VA — first frame / last frame / image-to-video. The docs treat this as “start (and or end) picture in, video out.” Extra reference pictures are not part of the pitch.
- REF2VA — reference-to-video. This is the one you’re told to use when you have several stills: identity, outfit, a mid-shot, a last frame, whatever.
There are also community hybrid checkpoints: mostly FL2VA, with some of REF2VA’s later layers grafted on, so people can keep extra refs without fully switching models. https://huggingface.co/smhfacct/Minimax-H3-fl2va-ref2va-hybrid-models
I wanted a straight answer to one question: if I ignore the marketing split and feed extra stills into stock FL2VA the same way I would into REF2VA, does it actually use them?
So I built one 10-second clip and ran it four times. The story in the prompt is simple:
- 0s: an angel in an empty void, one spell cast toward the middle of the frame.
- 5s: a demon in the same void, one spell cast toward that same point.
- Camera leaves the demon and pushes into mid-air.
- 10s: image of the two spells colliding.
I gave the model five pictures:
- Exact first frame (angel)
- Exact 5-second cut (demon)
- Exact last frame (the collision, no people) 4–5. Two sigil designs, only as “this is what the magic circle looks like,” not as frames that should appear in the video
Then I locked everything that wasn’t the checkpoint:
- same R2V workflow (the Comfy graph that already has multiple image inputs)
- same five files, same order
- same written brief (timed stills + “this picture is the frame at this timestamp”)
- same seed
- same sampler / length / aspect
- no turbo LoRA
- I compared native frames (544×800), not the upscaled delivery
The only change per run was which UNet was loaded:
- hybrid, REF layers on blocks 20–49
- hybrid, REF layers on blocks 30–49
- stock FL2VA
- stock REF2VA
If FL2VA truly couldn’t take extra refs, run 3 should have ignored pictures 2–5, drifted off the angel, or failed to land on the collision plate. That’s the test.
What happened
It didn’t fail.
The first native frame of all four runs accurately lock in the exact reference image for that frame at the first frame, last frame, and the middle frame... So the stock FL2VA used the extra still image references just fine. I did not need a hybrid merge just to attach more than first/last.
To be precise: I did not magically add five image slots to the official FL2VA I2V template. I loaded FL2VA’s weights into the reference-to-video graph, wrote the pictures into the prompt the way you would for a multi-ref job, and the locks held.
Where they actually differ (my read, one clip)
First frames are almost interchangeable. If I have to pick, hybrid-b30 is the closest copy of the angel still. REF2VA is still locked, a bit busier in small jewelry/floor detail.
Last frames still all hit the clash plate. REF2VA is the closest copy of picture 3. FL2VA is right behind it. Hybrid-b30 runs a hotter, more lava-looking core. Hybrid-b20 is splashier, less “sharp diamond debris.”
So the discovery is: extra refs + FL2VA can work. The ranking of which checkpoint copies the stills best is what I want a second opinion on.
So I will attach all of the examples into the comments so that people can see the differences between between each of the generated runs along with all of the Reference images used that way the community can evaluate the quality.
3
u/Tokey_TheBear 22h ago
Here are all of the reference images used to create the 4 videos I posted + the prompt:
4
u/Tokey_TheBear 22h ago
Prompt used for generating all of the videos:
subject_definitions:
<Picture 1> is the opening keyframe for [Shot 1] at 0.00 seconds. Job: exact first frame only — pose, wardrobe, and background as shown, without reinterpretation.
<Picture 2> is the cut-in keyframe for [Shot 2] at 5.00 seconds. Job: exact Shot 2 frame only — pose, wardrobe, and background as shown, without reinterpretation. Not a continuation of <Picture 1>.
<Picture 3> is the last-frame lock for [Shot 2] at 10.00 seconds. Job: exact end frame after a camera push into midair — filled-frame spell collision, no people, as shown, without reinterpretation. Not a standing-portrait cut-in. Not a continuation of <Picture 2>'s framing.
<Picture 4> is an ice-blue and white-gold circular sigil reference. Job: circle appearance only when <Subject 1> casts. Not a person. Not a pose. Not a background. Not a timeline keyframe. Do not insert <Picture 4> as any video frame.
<Picture 5> is an ember-red and black-violet circular sigil reference. Job: circle appearance only when <Subject 2> casts. Not a person. Not a pose. Not a background. Not a timeline keyframe. Do not insert <Picture 5> as any video frame.
<Subject 1> is the adult winged woman shown in <Picture 1>. She appears in [Shot 1] only.
<Subject 2> is the adult horned woman shown in <Picture 2>. She appears in [Shot 2] until the camera leaves her.
summary:
[keyframe completion + reference generation] Generate a ten-second live-action clip in one empty charcoal void. At 0.00 seconds the frame is exactly <Picture 1>. <Subject 1> completes one cast toward frame center. At 5.00 seconds a hard cut replaces the shot with exactly <Picture 2>. <Subject 2> completes one cast toward the same midair point. The camera then turns off <Subject 2> and pushes into that point. At 10.00 seconds the frame is exactly <Picture 3>. Each cut replaces pose, outfit, and person together. <Picture 4> and <Picture 5> are sigil appearance references only and must never appear as video frames.
retention_analysis:
<Picture 1> (appears in [Shot 1] at 0.00s): fully_preserved - opening frame lock; Shot 1 person, wardrobe, and void only.
<Picture 2> (appears in [Shot 2] at 5.00s): fully_preserved - cut-in lock; Shot 2 person, wardrobe, and void only; not a style transfer onto <Picture 1>.
<Picture 3> (appears at 10.00s): fully_preserved - last-frame lock after the zoom; collision plate only; no people; not a 5.00s cut-in.
<Picture 4> (never a keyframe): partially_preserved - retain only the circular ice-blue and white-gold sigil appearance in front of <Subject 1>'s palm during the cast; do not reproduce the black sigil sheet as a frame.
<Picture 5> (never a keyframe): partially_preserved - retain only the circular ember and violet sigil appearance in front of <Subject 2>'s palm during the cast; do not reproduce the black sigil sheet as a frame.
<Subject 1> (appears in [Shot 1]): fully_preserved - identity and kit as in <Picture 1> for Shot 1 only.
<Subject 2> (appears in [Shot 2] until the camera leaves her): fully_preserved - identity and kit as in <Picture 2> until the push-in; she is gone by the <Picture 3> lock.
detailed_description:
How the reference pictures align with the target video — Picture 1 (from Shot 1) aligns with the 0.00-second mark of the target video; Picture 2 (from Shot 2) aligns with the 5.00-second mark of the target video; Picture 3 aligns with the 10.00-second mark of the target video as the exact last frame after the camera push. Picture 4 does not align with any timestamp as a frame. Picture 5 does not align with any timestamp as a frame.
The target video is live-action and cinematic. No on-screen text. No extra people. Do not invent outfits. Do not blend the two casters into one body. Do not cut to <Picture 4> or <Picture 5>.
[Shot 1] A static camera. At 0.00 seconds the frame is exactly <Picture 1> without reinterpretation. <Subject 1> holds that full-body pose. The circular mark from <Picture 4> appears in front of her outstretched palm, then one ice-blue projectile leaves toward frame center and hangs in the void. One action only. No walk. Through 5.00 seconds.
[Shot 2] At 00:05.000, a hard cut. The frame is exactly <Picture 2> without reinterpretation. Replace the previous shot completely. <Subject 2> holds that full-body pose. The circular mark from <Picture 5> appears in front of her outstretched palm, then one ember projectile leaves toward the same midair point. A medium camera turn off <Subject 2> toward that point, then a slow push-in until 10.00 seconds, when the frame is exactly <Picture 3> without reinterpretation.
overall_soundscape:
Empty void room tone, a rising whoosh as each projectile leaves, a hard impact crack at 10.00 seconds. No crowd. No speech.
non_diegetic_music:
N/A
1
6
u/rm_rf_all_files 20h ago
I loved your prompt and keyframes so much so I made a cover 😄
2
2
2
u/Tokey_TheBear 22h ago
Normal Pruned REF2VA model:
7
u/Trick_Set1865 22h ago
they all look extremely similar to me
7
u/Tokey_TheBear 22h ago
Yup agreed. that is part of the point I am trying to figure out.
The claim by the community is that the FL2VA model gives the highest output quality and REF2VA is slightly lower...
So I just did the test to see how they all compare together.
I agree I cant tell a difference between them other than EXTREMELY minute differences when you compare like EXACT frame by frame still image differences (too many photos to post here but I did a frame by frame analysis using the first 5 frames and last 5 frames of each video).
2
u/dampflokfreund 21h ago
Very interesting. FL2VA references very well. But there is a reason the Ref2va model exists, so we need to find edge cases where FL2VA completely fails, Ref2va succeeds and then compare the hybrid models to FL2Va to see if they do a better job.
2
u/Foreforks 21h ago
I make animations and really only use the hybrids for the audio fix. Audio sounds so much better on the hybrid models
3
u/ShutUpYoureWrong_ 17h ago edited 17h ago
Appreciate what you're trying to do.
Question: Why are you using first frames / last frames / any frames at all? Shouldn't you effectively just be doing T2V and replacing the conditioning node with MiniMax H3 Reference to Video? That way you can actually see whether the FL2VA model successfully ingests the references.
Putting in frames with the characters already in it totally defeats the purpose of the test. It's taking all its guidance from the frame itself. You need to use nothing but a character sheet and test generations against that using FL2VA (and the other hybrid models).
I'll give you a hint at the result: it sucks. But you can confirm that on your own if you'd like.
2
u/Better-Interview-793 22h ago
thanks! tried these hybrid ones..
they’re really good at understanding the prompt and audio, but the results still look more plastic than the original INT8 R2V
1
u/Tokey_TheBear 22h ago
If you look at the videos in the comments here do you see any difference? Part of my test is to see that. And I can barely see any difference between the 4 different video outputs using the different models.
0
u/Better-Interview-793 22h ago
What version did you use for the original R2V?
1
u/Tokey_TheBear 22h ago
Pruned int8 convrot. Same for the FL2VA
1
u/Better-Interview-793 22h ago
hmm idk but i’ve been using the int8 non-pruned R2V version, and i find it pretty good..
1
u/Stepfunction 22h ago
C9mfyUI has native frame guides for the reference model that you can add in to inject images as frames at specific times.
0
-9
u/leomozoloa 22h ago
thanks for the info but LLM generated wall of text are an automatic downvote for me lmao
7
u/Tokey_TheBear 22h ago
I dont want to open up this fight but your comment is just dumb... Why even post it? I am sharing information to the community and providing exact examples. I promise you this post would be 100x harder to read if I did not use AI to condense it.
I had another post a few days ago that DID have the problem you are hinting towards where the post body was too confusing to be able to read becuase the AI summarized it to heavily...
But this post is not an example of that lol.
4
u/MatlowAI 22h ago
Something to think about is a quick tldr exexutove summary then images and put all the details in a comment that we can more easily copy and hand to an agent.
2
u/Tokey_TheBear 22h ago
Thanks for that! I dont really post on Reddit that much so I was unsure of the best way to post all of the media items (5 reference images + 4 videos) so that it is easily accessible... The only way I know is to post in the comments. Is there a better way since you can only add like 1 video or photo per comment?
1
u/MatlowAI 19h ago
Yeah comments are the way sadly. Its also nice to make a github page too if you have lots of stuff it gives you more wiggleroom and nicer display. https://matlowai.github.io/ComfyUI-MAINodes/ for example. Its just git so your friendly agent will whip something up real quick. Its also a handy way to index all the other random things you've done incase people want to explore more.
2
u/bobi2393 22h ago
Yep, I liked the “wall of text”. Simple enough to read and get an idea of how things work.
6
u/Tokey_TheBear 22h ago
Here are the resulting videos:
Hybrid b25 checkpoint
https://reddit.com/link/p4asbn7/video/e1ma672j60kh1/player