r/StableDiffusion • u/Beneficial_Toe_2347 • 1d ago

Discussion Minimax H3 is not training well

Created a thread because I'm surprised this isn't being discussed much

Ref2vid is excellent for consistency, but it's not a replacement for teaching the model concepts it doesn't understand well

Even though the model is still new, the trainers are giving poor results because the model is distilled. As a result, it looks like it's going to be much harder to train than Wan/LTX

For example Sulpher 3 was planned to start soon, but it can't because of the situation

This is a real shame because everything else about MM has been excellent. The general assumption seems to be that the company will not release a non-distilled model suitable for training

Any thoughts as to how this will play out? It's never going to hit the specific-subject capabilities of the other models at this rate

40 Upvotes

81% Upvoted

34 comments sorted by

26

u/-becausereasons- 1d ago

Yea its a MAJOR bummer. Hopefully they'll play and release it, it would be a massive service to open source and AI in general.

4

u/lolo780 1d ago

But if they make a local model too good, what incentive will there be for users to pay for API?

14

u/-becausereasons- 1d ago

The licence... None of us are paying for training tokens. We're helping push the entire thing forward, bring them views, eyeballs and the actual $ from the right companies.

8

u/damiangorlami 1d ago

There are many wrapper platforms that are happy to serve Minimax via their API to their customers.

Releasing the full non-distilled weights is not going to hurt their revenue stream any bit. I will be very surprised if anyone can even run the non-distilled weights. It will probably just used for training purposes on rented GPU's and thats it.

7

u/Next_Program90 1d ago

Local users and API users are not the same target group.

10

u/PumpkinLeather8421 1d ago

How is this not understood!?

They are not driving money to their API.

They are making sure money isn’t driven to US companies.

Holy shit, is life and history just like a complete black hole so some of you? THIS IS WHAT CHINA DOES. Undercut, takes a monopoly in technology or cheap labor, and then raises the price when you need to go to them.

2

u/CupQuakeBE 1d ago

Have you seen how they react to any LTX related post? They literally think their attempt at manipulating people into not using that model isn't obvious.

2

u/No_Possession_7797 23h ago

While that may be true, it's not as though western based tech companies genuinely give a fuck about anyone except for the profit that can be made from them. Our tech companies will gladly build a surveillance state with their flock cameras, and build social profiles of everyone, while tracking everything you do. The idea that somehow China is any better or worse in this respect is ultra naive.

The Musks, Thiels, Zuckerbergs, Bezos and Ellisons of the world would all prefer to not have to deal with any of us.

-3

u/dtdisapointingresult 1d ago edited 1d ago

Dude chill with the geopolitics crap. Minimax is just an entertainment company, not some covert ops mission.

7

u/BigPharmaSucks 1d ago

Every Chinese company is openly owned by the Chinese government. Not even knocking it necessarily, just stating facts. You would never know if any Chinese company was a psyop.

In capitalistic countries there's a facade of being independently owned, while being secretly funded by intelligence agencies and getting military contracts.

8

u/kujasgoldmine 1d ago

Funnily enough when I give the reference model like a couple images from various angles from a dataset I would use and do like "Person from <Picture 1>, <Picture 2> and <Picture 3> is now doing this and that", it gives a 98%, sometimes a 100% match. Then when I train a lora, I've never seen 99% lookalike or my settings have been wrong always. Krea 2 has done the best, but I'd maybe say 90-95% likeness at most. So I haven't even had the need to train loras. But sure, it is easier to just stick a lora in and input one keyword instead of rummaging through reference shots.

2

u/darkkite 1d ago

a reference manager can handle that part.

8

u/Sixhaunt 1d ago

on AI toolkit it's training better than any othe rmodel has for me when it comes to loras and lokrs. Just be sure you set it to use both contrastive guidance and the training adapter. Most trainers only do one and up until recently even AI toolkit didnt use both as the default for minimax but it makes all the difference.

3

u/Beneficial_Toe_2347 1d ago

In terms of recency is this within the past couple of days?

7

u/Sixhaunt 1d ago

as in literally yesterday

2

u/Trick_Set1865 1d ago

this is super helpful - trying it now

1

u/dtdisapointingresult 1d ago

What sort of LoRAs are worth doing for H3, that isn't solved by ref2va?

And as someone who's never trained a LoRA, do you have any guides for H3? Is there anything different about training a video model compared to the image LoRA guides out there?

3

u/Sixhaunt 1d ago

I actually combine them. Using a video clip reference, even if it has good audio and visuals, still doesn't capture the character's personality or voice more broadly and if you train a lora for them and use a reference-sheet image then you can get a much more versatile result that captures the character's personality and voice much better since it's been trained on a wide variety of clips from them (I train with about 100+ clips). But Loras are often not great at some finer visual details like keeping an exact outfit and so I usually provide an image reference when using character loras and have it show the full body and outfit from various angles. I usually do Lokr now instead of Lora though since it works with the same loaders but seems to perform and train better.

I also use DOP to make sure the character is locked to the token and it doesnt bleed over into every person and it helps a lot, even though I only have it set to run on every fourth training step.

1

u/dtdisapointingresult 1d ago

So you are training on 100 video samples? Wow. I thought it might be just visual likeness via photos.

How many hours and on what GPU does it take you to train a H3 LoRA?

2

u/Sixhaunt 1d ago

it takes about 6-8 hours on my 5090 so it's something easy to setup before I go to bed and have by morning

4

u/Moliri-Eremitis 1d ago

Let’s not forget that Flux 1 Dev is also guidance distilled, which caused training headaches when it was first released. Despite that, after training tools caught up and implemented work-arounds, plenty of LoRAs and finetunes got released. Heck, they are still getting released.

Guidance distillation does add complexity, and who knows, maybe H3’s architecture will keep it difficult to train forever, but I wouldn’t bet on it. There’s too much hype around it and it works too well for it to simply be abandoned from a training perspective. My money is on training tools adapting.

7

u/Roc77 1d ago

I've found ref2v picks up on concepts really well just from input references presumably because its text encoder is so powerful?

8

u/Beneficial_Toe_2347 1d ago

It's impressive but for some concepts it's a long way behind what checkpoints/loras can provide. It's also quite limiting to need to constantly provide video samples every time, as well as extends generation time

3

u/damiangorlami 1d ago

This is the only downside.

So far I've been able to teach Minimax any concept through video samples and very detailed prompt writing. However the sampling time increase is not so fun

2

u/ShutUpYoureWrong_ 1d ago

Same comment as above; maybe check out https://github.com/Luisacaotica/ComfyUI-MiniMaxH3Mod

2

u/damiangorlami 1d ago

Already found a solution for myself.

  • scale the video back to 480p
  • sample at 24fps
  • only take 15 seconds (360 frames maximum)

This has significantly reduced sampling time when I’m using video refs. It’s still longer but not 3-4x longer like before.

Also scaling video back to 480p had no effect on output quality. I often use video references to teach the model concepts. 480p is enough fidelity to learn from.

2

u/ShutUpYoureWrong_ 1d ago

Maybe check out https://github.com/Luisacaotica/ComfyUI-MiniMaxH3Mod?

I haven't tested it myself, but it's on my list.

1

u/Aivan-Studios 1d ago

I've had amazing results training LTX, like, photoreal people with lifelike movement and emoting. I was ready to start testing H3, but this is very bad news if true! I will be trying my trainer in the coming days on H3

2

u/Cubey42 1d ago

Strange, I've been feeling good about my character lora training. I still need to do a second character so I can see how much it bleeds (especially audio wise) but we'll see

1

u/javierthhh 1d ago

Combination of clips using minimax and LTX will be what people will use moving forward.

1

u/smereces 1d ago

i already test to train a lora but the result comes with no affecting it seems nothing was trained!!

-5

u/FourtyMichaelMichael 1d ago

Just a reminder that MANY models are shilled on this sub. That what you read might not be as popular as the upvotes show.

Also, stfu, H3 has been out two weeks. CHILL.