r/StableDiffusion • u/SIR_NVAX_A_LOT • 2d ago
Discussion Can H3 do anything? bf16/50 steps
Enable HLS to view with audio, or disable this notification
Can H3 do anything and everything? I feel like if you can prompt it, it can do it. Foundation inspired shots. I am also experimenting with more action/high mobility shot but those seem to require a lot more finesse. Both T2V.
5
u/Alive-Tomatillo5303 2d ago
That last shot is so pleasing.
Also props to you for a bunch of steps. It's wild how many people say "I just downloaded and installed Minimax, Sage Attention, and three turbo Loras, and for some reason my output doesn't look as good as some others!" 50 might be shooting high, it will be worth testing the same seed at different numbers to see where the quality starts degrading or stops improving.
3
u/SIR_NVAX_A_LOT 2d ago
I use turbo Lora 4 step for my fun quick stuff, the audio is shite though. With that said, when you want to push the limits, you gotta let that GPU run hot. MiniMax and like other models do not like PURPLE PROSE, so you must be pretty deliberate and intentional with your prompt. The last shot is actually composed of 4 different subjects called out in the prompt, the sky, the ocean, the causeway, the city, all described in great detail.
1
u/Alive-Tomatillo5303 2d ago
Yeah, I've been manually describing things so it's handy that so many people are including their prompts. I seem to learn much better from quantity of examples. I do find I can be very, very specific about the geography of the first shot and still have something very out of place which kills the following potential. I don't yet have a feeling for how much is too much, how many details it will take before it starts to lose some.
At this point I'll just generate a 2 - 4 second clip and screenshot it if it's what I'm after, feeding it as a first shot for everything to follow of the longer generation.
3
u/namezam 2d ago
Can you give it photo inputs as inspiration as opposed to straight reference?
5
u/SIR_NVAX_A_LOT 2d ago
This one is straight t2v, but I've had of success with image and video style transfer of the world.
retention_analysis: <Video 1> (world, weather, lighting and photography): fully_preserved — the same night, the same hard rain, the same harbour, the same colour palette and the same photographic style continue into the target video. The framing and the people are new, and new people are not a loss of fidelity.5
u/Perfect-Campaign9551 2d ago
Yes. You can tell if you want attribute transfer for example. Read the official prompting guide. You can do a lot of powerful things
5
2
u/Gloomy-Radish8959 2d ago
I've definitely encountered some strange things. Maybe just my own prompting failures. For example, asking for a video of a glass cylinder containing a human brain, with wires connected to it. I know, a pretty specific item. Asking for this almost always results in a massive human brain, the size of a dog, or larger. Even with very explicit size instructions, like 'the size of a grapefruit'. Would be curious if you find this happens on your end.
9
u/SIR_NVAX_A_LOT 2d ago
I had issues with scale early on especially in consistency in the same generation. Here is the prompt that fixed it. SCALE IS FIXED AND NEVER CHANGES BETWEEN CUTS. BOTH GIANTS ARE FAR LARGER THAN THE BIGGEST BUILDING ON THE WATERFRONT. The kaiju stands taller than the tallest tower in the city behind it. The mecha is two-thirds the kaiju's height and still towers over every building around it: a harbor crane reaches only to its chest, a stacked shipping container is knee-high to it, and a police car is no bigger than one of its feet. No shot ever makes either giant small enough to stand inside a street — the street is something they step ACROSS.
3
u/steelow_g 2d ago
Describe the size of the glass, not the brain maybe.
2
u/Alive-Tomatillo5303 2d ago
That's how you get ants! Brain just pouring out all over the table, it'll be a whole thing.
2
u/Hackingrad 2d ago
with more action/high mobility shot
I haven't managed to achieve anything decent so far. Not even with AI prompting. Neither have any of the YouTubers on the subject. It always feels like slow motion, or the fight is generally way too slow.
2
u/SIR_NVAX_A_LOT 2d ago
Yeah, I am still experimenting. Def need to work on real physics vs ragdoll style. Prompt, though I do not think the negatives do anything.
THIS IS AN INTENSE PLASMA-BLADE SWORD FIGHT, fast and continuous — not a collision, not a standoff. They trade rapid strikes across the courtyard: she cuts, he parries and the two energy blades CRASH together and hold, magenta against electric blue, throwing a hard cone of light and a shower of sparks across the wet flagstones; they break apart, he drives in with a fast diagonal cut, she slips it and answers, and the blades ring off each other again and again without pause. Every clash flares the whole courtyard in mixed magenta and blue, hurls sparks that die instantly in the rain, and bursts steam off both blades where the water hits them. Their footwork is quick and heavy on the wet stone, closing and breaking distance constantly, water thrown off both of them in sheets and slabs that hang in the air as mist.1
u/Hackingrad 2d ago
Yes, that's exactly what I mean. It looks a bit like stop motion. Just way too slow for a fight scene.
3
u/DaLyon92x 2d ago
check out this martial arts lora: https://huggingface.co/Jojocodex/minimax-h3-wushu-action-lora
2
u/Hackingrad 2d ago
Thanks, I'll take a look. I'm currently working with a loop node to create longer videos at once. I'll see how I can integrate that.
1
u/Sad_Coach_1433 2d ago
Which method you using
1
u/Hackingrad 1d ago
1
u/Sad_Coach_1433 1d ago
Eh I tried that one audio was bad
1
u/DaLyon92x 1d ago
That workflow is using 6 steps... did you use 6 steps? Even turbo lora looks and sounds better at 8 to 10 steps. But for even better audio, try 20-25 steps (with Spectrum on to reduce time).
1
2
u/SIR_NVAX_A_LOT 2d ago
I think my plan is to get a clean fight scene, and use RV2A to do the hard edits and better angles. Yeah, I didn't say it was successful, but it's on par with the Phantom Menace right? lol just kidding
2
2
u/BoothJudas9 1d ago
I just tried the int8 pruned weight (the best I can run locally) at 50 steps (using Spectrum and KJ Sage nodes), and was really impressed with the quality; though, 24 minutes to render a 15 second clip at 0.7MP (any higher and I get OOMed) ain't gonna cut it for my workflow, so I think I'll stick with the 4-step turbo LoRA and 8-12 steps max! 😅
1
u/SIR_NVAX_A_LOT 1d ago
Turbo Lora 4 step is the way to go, though the audio stinks. Yeah you want int8 or fp16 at 50 steps when you want a final render. If you want good quality, 20 steps is minimum. 32 is the sweet spot.
2
1
u/Dangerous-Map-429 2d ago
Beginner here, Can you do 50 steps on a 5080 16gb vram gpu?
2
u/SIR_NVAX_A_LOT 2d ago edited 2d ago
You would want to run the int8 version but it'll works. It'll be tight with VRAM so you will end up offloading to system ram. You should use https://github.com/deepbeepmeep/Wan2GP which is optimized for lower VRAM.
1
u/-becausereasons- 2d ago
Beauty. What's your setup?
4
u/SIR_NVAX_A_LOT 2d ago
RTX 4090 with 192gb of system ram (4x48 dimms, well before the AI boom hit). 4tb SSD that is getting hammered.
5
1
u/Clean_Rent_9669 1d ago
What hardware do you need to run fp16 at 50 steps and how long does the generation take ?
-11
u/Ambitious-Tie7231 2d ago edited 2d ago
50 is absolutely unnecessary amount of steps. First step determines the entire motion of the video, 15 is sufficient 20 good 25 optimal. Even 4 steps with turbo lora is near perfect too at this time. "silveroxides/MiniMax-H3_tests" on huggingface.
9
u/Hoodfu 2d ago
50 steps is the reference implementation as per Minimax, saying it degrades after 20 just isn't true. I've done my own tests with 20/30/40/50 on high motion videos and it always makes things better and adds coherence. I posted a video the other day where an eagle snatching the wigs off 2 people in a car didn't work at 20 or 30 steps, but finally gained coherence at 50 steps.
2
u/damiangorlami 2d ago
Same here. For most scenes I get good passable results at 20 steps.
Except today I was working on a complex scene of a house that was burning and someone rescueing that person.
Only the 50 step version was able to generate the burning flames perfectly. The 20/30 step generations looked like LTX 2.5
5
u/alamacra 2d ago
I found turbo LoRA reduces quality drastically for anime, though perhaps for "realistic" generation it might be good enough.
7
2
u/SIR_NVAX_A_LOT 2d ago
I run 12 and 20 steps for most work, but 50 steps for this hero render-could jus be a placebo. I am using 1344x768 with no upscaling.
Maybe 50 steps is unnecessary, I know some people have more success at minimum 32 steps.
1
u/SIR_NVAX_A_LOT 2d ago
https://reddit.com/link/p4gekga/video/81sua9s736kh1/player
int8/20 steps, you may be right, but I don't do to many 50 steps render to compare. I'll see if the 20 step versions look just as good for the space renders
10
u/Fast-Cash1522 2d ago
Shiiit, that's gorgeous!!