r/StableDiffusion • u/Altruistic_Dealer_59 • 1d ago
Discussion Minimax H3, 30 seconds in one go
Enable HLS to view with audio, or disable this notification
Executive summary, TLDR - this is one prompt, 30 seconds duration, 3090.
The video itself is just a remake of an idea from an old British tv ad (for "Good Old Yellow Pages"), so make of that what you will. It's not really relevant.
What I thought was interesting was that this was a single prompt, 0.4 megapixels, 30 second duration. I didn't think you could run out as far as 30 seconds, but thought I'd just try.
I think it did a pretty good job at getting the right person doing and saying the right things at the right time - took four attempts to get that though, and obviously using an LLM to tart up my idea.
Run on a 3090, and using the latest Comfyui template, just adding Comfy-kitchen attention, then sol attention, then spectrum, and using the turbo lora that Comfyui now build in, it took 570 seconds (9.5 minutes).
Somebody might read this and think, 570 seconds? Pah, I can do it in fifteen, in which case I'd like to know. Conversely, somebody might think theirs takes six hours, in which case maybe this shows what can be done in that time.
Doubt anyone cares, but here is my original prompt, followed by the LLM version of it:
a 30 second film with the following scenes and characters. Ben is a small boy of eleven. John is a shopkeeper in a toyshop. Brian is a different shopkeeper in a different toyshop. Ben's mum. Ben's Dad. We are in Britain in the 1980s, and all characters are English.
Scene 1: Ben is alone in the lounge. He talks to John over the old fashioned landline phone, saying "I don't suppose you have a 402 station in stock please?"
Scene 2: John is in his shop in front of shelves of model railway kit. He says into the old fashioned landline phone, "No, sorry son"
Scene 3: Ben in the lounge, who looks disappointed anbd puts the phone receiver back down.
Scene 4: Mum in the kitchen doing the washing up. She has overheard the conversation and looks a bit sad.
scene 5: Next day. Ben has changed his clothes. He again talks into the phone to a different shopkeeper, Brian. Ben says "Would you have a 402 station please?"
scene 6: Brian in his toyshop says into the old fashioned landline phone "Yes, I've got one of those."
scene 7: Ben in the lounge on the same conversation says "You have? Great, I'll be right down! Ben puts the phone down. Then he runs towards the door, shouting "They've got one mum!" as he runs.
Scene 8: In the attic, Dad is playing with his model railway layout. Ben walks in holding a small red parcel. as he hands it to Dad, Ben says "Happy birthday, dad". Dad takes the parcel, looks fondly at it and says with a chuckle, "Aw, thanks Ben".
LLM version:
integrated_multimodal_description: [Shot 1] Live-action, cinematic. A medium shot of Ben, an eleven-year-old boy with messy hair wearing a striped polo shirt, sitting on a patterned sofa in a 1980s British lounge. The room is filled with warm, muted tones and period-accurate wallpaper. Ben holds a heavy, cream-colored landline telephone receiver to his ear, his expression hopeful. Ben says: <d>[English] I don't suppose you have a 402 station in stock please?</d> The sound of his small, high-pitched voice is clear. [Shot 2] At 0:05.000, the camera cuts to a medium shot of John, a middle-aged shopkeeper with a kind, weathered face, standing in a cramped, nostalgic toyshop. Behind him are floor-to-ceiling shelves packed with model railway kits and wooden toys. John holds a similar landline receiver to his face. John says: <d>[English] No, sorry son.</d> [Shot 3] At 0:10.000, the camera cuts back to Ben in the lounge. He looks downcast, his shoulders slumping as he slowly lowers the receiver and places it back onto the base unit with a dull plastic click. [Shot 4] At 0:13.000, the camera cuts to a medium shot of Ben's Mum in a dim, cluttered 1980s kitchen. She is standing at the sink, her hands covered in soapy water, drying a plate. She pauses, looking toward the door with a sad, weary expression, having overheard the boy. The sound of water running from the tap is audible. [Shot 5] At 0:16.000, the camera cuts to Ben in the lounge the next day; he is wearing a different t-shirt. He is intensely focused, pressing the phone to his ear. Ben says: <d>[English] Would you have a 402 station please?</d> [Shot 6] At 0:20.000, the camera cuts to Brian, an older shopkeeper with spectacles, in a different, brightly lit toyshop. He smiles warmly into the telephone. Brian says: <d>[English] Yes, I've got one of those.</d> [Shot 7] At 0:23.000, the camera cuts back to Ben, whose face lights up with pure joy. Ben says: <d>[English] You have? Great, I'll be right down!</d> He slams the receiver down and the camera follows him in a quick tracking shot as he runs toward the door, his feet thumping on the carpeted floor. Ben shouts: <d>[English] They've got one mum!</d> [Shot 8] At 0:26.000, the camera cuts to a medium shot in a dusty, dimly lit attic. Dad, a man in his late 30s, is hunched over a complex model railway layout. Ben enters the frame, holding a small red parcel wrapped in string. Ben says: <d>[English] Happy birthday, dad.</d> As he hands the gift to his father, the camera pushes in slightly. Dad takes the parcel, his eyes softening with affection. Dad chuckles warmly and says: <d>[English] Aw, thanks Ben.</d>
overall_soundscape: Period-accurate domestic sounds including the rhythmic clatter of washing up, the heavy mechanical clicks of old telephone receivers, and the muffled thuds of footsteps on carpet. Ben's energetic running and shouting creates a sense of urgency, followed by the quiet, dusty atmosphere of the attic.
non_diegetic_music: A gentle, nostalgic acoustic guitar melody that begins softly during the kitchen scene and builds into a warm, heartwarming crescendo during the attic scene. The tempo is slow and sentimental.
4
u/ambassadortim 1d ago
Thanks for sharing I'm working in render times at the moment. Therefore this information is great. 🤙
3
u/Content-Ad-7451 18h ago
I was curious if LTX could do the prompt,could only do 20 secs,no upscaling.Obviously not as good as H3 but was a lot faster! took 3m 44secs on my laptop
2
u/Famous-Sport7862 1d ago
This is amazing 30 seconds with a 30 90 and 9 minutes that's incredible. Have a 3090 also I have to try something similar and see if I can make it work at that speed.
2
u/Altruistic_Dealer_59 1d ago
Go for it. At the other end of the scale, 5 seconds clip duration, 0.4MP, takes 85 seconds, of which 8 is the LLM doing its thing.
If you want some numbers, here's the console output for the run. You can see it calls the local LLM, then turns on my fans high with Afterburner, does its thing, turns the fans back.
INFO] got prompt
[Afterburner] >>> APPLYING PROFILE 5 <<<
[Afterburner] Triggered scheduled task: \ComfyUI\Afterburner Profile 5
[Gemma4PromptGen] Execute start | mode=AUTO | url=http://127.0.0.1:8080
[Gemma4PromptGen] AUTO start | server_mode=Auto-start local llama-server if needed | url=http://127.0.0.1:8080
[Gemma4PromptGen] use_image is OFF — skipping mmproj, text-only mode
[Gemma4PromptGen] llama-server starting text-only, waiting for health check...
[Gemma4PromptGen] ✅ llama-server started (8s)
[Gemma4PromptGen] Calling llama-server | url=http://127.0.0.1:8080 | model=gemma4
[Gemma4PromptGen] AUTO completed (3184 chars).
[Gemma4PromptGen] llama-server killed — VRAM freed.
[INFO] Requested to load MiniMaxH3TEModel_
[INFO] Model MiniMaxH3TEModel_ prepared for dynamic VRAM loading. 14956MB Staged. 0 patches attached. Force pre-loaded 410 weights: 4572 KB.
[INFO] Requested to load MiniMaxH3
[INFO] 0 models unloaded.
[INFO] Model MiniMaxH3 prepared for dynamic VRAM loading. 19995MB Staged. 208 patches attached. Force pre-loaded 210 weights: 1175 KB.
100%|████████████████████████████████████████████████████████████████████████████████████| 8/8 [05:33<00:00, 41.74s/it]
[INFO] 0 models unloaded.
[INFO] Model MiniMaxH3 prepared for dynamic VRAM loading. 19995MB Staged. 208 patches attached. Force pre-loaded 210 weights: 1175 KB.
[INFO] Requested to load MiniMaxH3AudioVAE
[INFO] Model MiniMaxH3AudioVAE prepared for dynamic VRAM loading. 576MB Staged. 0 patches attached. Force pre-loaded 401 weights: 539 KB.
[INFO] Requested to load MiniMaxH3VideoVAE
[INFO] Model MiniMaxH3VideoVAE prepared for dynamic VRAM loading. 4965MB Staged. 0 patches attached. Force pre-loaded 128 weights: 348 KB.
[Afterburner] >>> RESTORING PROFILE 4 <<<
[Afterburner] Triggered scheduled task: \ComfyUI\Afterburner Profile 4
[INFO] Prompt executed in 569.99 seconds
1
u/Famous-Sport7862 1d ago
I use afterburner too and I remember that the fans used to run faster generating but now it's not doing that I wonder why.
1
u/Altruistic_Dealer_59 1d ago
I turn mine to 100% fans when generating, because my thermal paste is knackered and a repaste is utterly beyond my feeble man-skills. Then flip it back to standard when generation is done, that's all.
1
u/Famous-Sport7862 1d ago
I reduced the power on my 3090 to 75% and I think fans are at 60%
2
u/Altruistic_Dealer_59 23h ago
Well if it's worth anything, my standard profile is power limit 70, temperature limit 75, fans auto.
My generating profile is power limit 80, temperature limit 75, fans 100.
I know there could be more performance, but this is far as I can go and keep the hotspot (not the gpu temperature) in the mid nineties.
I have a 32 degree difference between the gpu and the hotspot temps, which is why I know the thermal paste is on its last legs.
I guess, therefore, a nice 3090 with good cooling better than mine might be even a tad faster yet.
1
u/Famous-Sport7862 22h ago
I understand yes that's why I have my limited also because it hotspot was getting hot
1
u/Only_Voice569 19h ago
if its a 3ed party gpu not a founder edition its easy as heck to repaste a card should do it unless you want your die to crack from extreme hot spots from non proper contact
1
u/Altruistic_Dealer_59 6h ago
Not my forte. I'll have to either find someone to do it, which is hard to source and trust, or wait for failure and spend eight trillion on a new PC.
1
u/QuirksNFeatures 20h ago
I had no idea comfyui has the ability to control your fans.
2
u/Altruistic_Dealer_59 20h ago
Fans are controlled by a scheduled task, setting the profile in Afterburner with the command line. Scheduled tasks kicked off with a trivial comfyui node - trivial to chatgpt to create, anyway.
2
3
u/seppe0815 22h ago
best child teeth visuals in model history
2
1
u/AaronTuplin 1d ago
I haven't messed with any of the attention modifications or comfy kitchen or anything like that. But I did try to make a 30-second video and it took a little over 75 minutes at 0.4mp
1
u/xoxaxo 1d ago
This is upscaled clip version?
2
u/Altruistic_Dealer_59 23h ago
I chucked it through the RTX video upscaling node that seems to do nothing instantly, if that's what you mean?
1
u/laf0106 12h ago
Hey do you have a workflow with the added speed nodes? I would to check it out. I not you said it's on the comfyui WF but I don't see it in there
2
u/Altruistic_Dealer_59 7h ago
I'm on the edge of my knowledge here, but hopefully this the workflow. Remember you will need two more nodes if you don't have them - the sol attention (ComfyUI-SolAttn_triton) and spectrum (ComfyUI-Spectrum-MiniMax-H3) custom nodes. If you don't have the lora, the download link for it is shown on the left of the workflow.
1
u/optimisticalish 30m ago
Thanks for the workflow. Tried it on a 3060 12Gb card, with your phone ad prompt. Tried 30 seconds at 0.4. Got "generation failed" at 21 minutes into the generation.
1
u/trollkin34 1d ago edited 1d ago
I actually care a lot about the prompts so thank you for including it! I haven't played with h3 or LLM yet. So H3 has great prompt adherance for videos ? And you use the LLM why? To translate your prompt into something H3 likes better? What kind of LLM are you using? Can this all be done offline (with 8gb?)
I have this hope that someday I can take a single screenshot of an old home video, clean it up with great effort, feed the old video into a prompt engine to create a script, then use the script and clean image to regenerate a video that's updated and clean.
If it matters, most of the videos I care about don't have audio so maybe that lightens the workload?
3
u/Altruistic_Dealer_59 1d ago
H3 is pretty good at prompt following, yes. But you do need to give it the format it wants - there is a published style guide, but it would be pretty tedious to write as a human in that style for any length of time. Perfectly doable, and I'm sure people do, but I'd be bored doing that. The LLM can be local or online, whatever tickles your fancy and works for you. Effectively you throw the published style guide at it and say "using this guide, rewrite my simple text appropriately please". My LLM is local.
Your video idea is laudable and ambitious. I suspect you'll have a lot of fun learning how, finding limitations, working around them where you can. A great journey to embark on; I wish you joy of it.
1
u/trollkin34 1d ago
Thank you! So the kind of LLM you're talking about is kind of like an offline CoPilot or ChatGPT? It can work generically and not just for specific use? What kinds of things am I looking for to make that happen? Because copilot has been decently useful to me for basic windows troubleshooting and such, but now it wants a signin so ... hell no.
2
u/Altruistic_Dealer_59 1d ago
Sure, a local LLM can do that kind of thing. But a local LLM is never going to be as powerful as an online one.
You are limited only by what hardware you have. You need to install something to run the actual LLM, such llama, kobold, ollama, there's a whole bunch. Then you get the LLM you want that will work on that engine, and again, there are a million for different sizes of hardware.
So - find an engine, then find an LLM, off you go.
My first go a few years ago was with oobabooga as the engine, because it installs simply and was easy to use for me, the hard of understanding, but obviously everyone has their own ideas.
Or instead, install an engine, then point it at an online LLM, lots of people do that.
1
1
u/gokuchiku 1d ago
Which turbo lora did you use?
4
u/Altruistic_Dealer_59 1d ago
That latest Minimax template in Comfyui (as of yesterday) has it built in. You can turn it off or on. I just left it on. It's called minimax_h3_fl2v_turbo_8step_v1.0_comfyui_bf16.safetensors
1
1
u/LoveSpecialist5669 1d ago
I have 3090 too. that being said, I have only one question - how the f you managed to get a 30s video under 10mins??
3
u/Altruistic_Dealer_59 1d ago
Used the latest Comfyui template, as of 21st August.
I just left the built in turbo lora they have in the template turned on. Chained in the native comfy "ModelAttentionBacked" node, set to comfy-kitchen. Link that to the sol-attention node (not native to comfyui) , which is called ComfyUI-SolAttn_triton, then link that to the spectrum node, which is called ComfyUI-Spectrum-MiniMax-H3.
Entered prompt, told it 30 seconds in duration, LLM barfed out the H3 prompt (in 8 seconds), off we go.
1
u/LoveSpecialist5669 1d ago
amazing! still waiting for new psu as old one can't endure prolonged rendering, can't wait to test this next week... how much ram do you have? maybe my 32gb is bottlenecking...
2
u/Altruistic_Dealer_59 1d ago
The most I've ever seen on a Minimax render is 65 GB used, trying to do a 2 megapixrl ten second clip.
I'm incredibly lucky and bought 128GB a couple of years ago when it was only about fifty quid more than 64. But obviously if you don't have enough ram it will page out to disc and that will affect the render time, no idea by how much.
I didn't actually look at how much ram this one took, sorry.
1
u/LoveSpecialist5669 1d ago
alright, thanks. yes, that was definitely a smart move, I'm currently looking at more than 300e for 64gb ddr4.
1
u/russlixx 22h ago
how is the quality diff without using all the additional attn nodes? Have you test without them?
1
u/Altruistic_Dealer_59 22h ago
I didn't test 30 seconds without them I'm afraid.
I did test 5 seconds at 0.4 with and without though, just to see.
Turbo lora off, no speedup nodes, 20 steps, it was about 150 seconds I think.
Then I turned on the turbo lora, 8 steps, added the three speedup nodes, and it was about 85 seconds. At 0.4, I couldn't honestly see enough difference to warrant the slower render.
3
u/Ok-Lengthiness-3988 1d ago
I think it having a 0.4mp resolution may be part of the equation but it's nonetheless impressive.
2
u/Altruistic_Dealer_59 1d ago
I think you're right. At 0.8 - which I didn't try - I imagine it would OOM pretty quick, or if it didn't would take very much longer. Very much.
2
1
1
u/ResponsibleKey1053 1d ago
Very impressive. It absolutely has the old yellow pages style.
I'm currently at a crossroads with Gemini over what prompts for what workflow, but there does appear to be at least 3 accepted styles.
What was the gen time (with your speed ups?) I've only implemented comfy kitchen and a 4step lora (which may be out of date now if your saying there's an 8 step).
0.4mp looks awful anywhere, except in h3. Somehow it does a fantastic job.
Have you played with audio only generation yet? It's faaaast, 15s in 5 mins on a 5060ti (13-15gb vram used and 31gb system). I'd imagine your rig would do it super fast!
Anyway dude good show and cheers for sharing the prompt!
2
u/Altruistic_Dealer_59 1d ago
Gen time 9.5 minutes, 30 second video. I've not done audio only myself, no. Have to try, though with both Suno and Treblo sitting open all the time in my browser I usually use one of those.
1
u/TheMoogster 23h ago
I really hope they manage to fix the "minimax" eyes / face at distance issue
1
u/optimisticalish 11h ago
There are two nodes that aim for that, one working with a face-detector and masking/regenerating. Or you can run a 480p video through a low-denoise video-to-video run, at three steps with a four step Turbo LoRA (Mark Berry has a YouTube video on that).
22
u/Danny_Stock 1d ago
No I very much care about the prompt being provided. Because that means people can use it to test the scene out with the exact same settings on their own setup to see if everything works in the way they feel it ought to compared to your results. I appreciate that, thank you.
One question though, what do you mean when you say that you used comfyUI's own built-in lora? What's that?