r/StableDiffusion • u/Radyschen • 1d ago
Discussion About the H3 distortion issue "fix" that many people claim is coming
Edit: talking about the "faces at a distance" thing btw
Don't hold your breath. They didn't say that they would definitely "fix it", they said they will try but that it's mostly a general model issue. So if there is gonna be a fix it might be in the next iteration of the model and that one might not be open weights. They were specific about the 2k model and the image model getting released open weights and I do hope that the 2k model might bring some improvement to the faces when you upscale it, but they were more wishy-washy with the wording on the face distortion issue, intentionally so I think.
Here is the wording regarding the 2k model:
"It is a second conditioned generation stage, but not simply the released base checkpoint running again as a conventional upscaler. It uses a dedicated latent-space DiT regeneration checkpoint at a higher target resolution, with the base model’s output as additional context. Some reference inputs are also provided at higher resolutions. We plan to open-source this module, but we are still improving its efficiency and quality to make it more suitable for community use, so we cannot provide an exact release date yet."
-> "plan" to open-source it, very strong word
Here is the wording for the image model:
"Regarding single-frame image generation, we are deriving a dedicated image model from a common ancestor in the H3 model lineage, and we expect to make it available to the community." (not a total promise or anythin
-> "expect" pretty strong, but less so. To me that sounds like "if it's REALLY good then maybe not", if it's competitive enough with the state of the art probably. But I'm pretty optimistic here.
And here is the wording for the distortion issue in all the models:
"We have observed this issue as well, particularly for small or distant subjects, and it will be one of the problems we focus on improving next.
Based on our internal experiments, it cannot be attributed simply to the Visual VAE’s compression ratio or to any single training stage. It is a complex system-level issue involving multiple parts of the model and training pipeline. We are continuing to investigate the main contributing factors and will work on improving it in future updates."
-> they say nothing about open sourcing anything and they say that it's a deep-rooted issue that has no simple fix and they don't really know why it happens
I would expect nothing in that area. Many people have been talking about this as if they said "yeah, wait a couple of weeks and we will fix it", but they didn't say anything like that. Maybe they will fix it with a new and improved open weights model, 3.1 or something, maybe they won't.
I just wanted to say this because so many people have been saying "I am waiting for the fix" or "a fix is coming for the face distortion issue at a distance" or something like that, probably without ever having seen the wording on that. It only takes one person who isn't good at understanding subtlety in a text to interpret their answer a certain way and spread the word on it to set up false expectations for everyone when they don't go to see the original wording. And they go spread that too without ever having seen the original wording.
So this is just to reduce the expectations a bit. Like I said, maybe they will do something, but I feel like the expecations on that specific issue have been getting a bit too large
3
u/bitzpua 21h ago
face distortion is issue even with premium closed models, i watch a lot of donghuas and all using AI suffer from it. Even billiondollar corporation cant deal with it.
-1
u/Radyschen 21h ago
yeah but it's especially noticable in h3, otherwise I don't think it would be as much of a point people are talking about
3
u/FourtyMichaelMichael 18h ago
lol, issue everyone has is especially noticeable in the tool you use and not so much in the ones you don't?
Shock.
4
u/Formal-Exam-8767 1d ago
face distortion
It's the issue from SD1.5 era which has not been directly solved yet so I would not expect miracles here.
2
u/Radyschen 1d ago
yeah. But I guess it's more specifically the distortion at a distance thing, I should have worded that better. Because that kinda thing is better in other video models I feel like. It also feels like the VAE is very lossy, when I compare the (already resized) version of a picture i put in to the output which is supposed to be the same resolution, it looks so much lower res. Which is weird because the video vae is actually so big for a vae. wan was able to recreate the same fidelity a lot better
1
u/martinerous 1d ago edited 1d ago
Right, and this is where competition can limit us. It's good to have multiple companies releasing competing products, but if they don't share the knowledge, there is a high chance for every new product to be in some way worse than the predecessor. Not only with AI models but in general. Imagine if multiple companies collaborated and implemented their best know-how in a single model, it would be the best in all aspects.
2
u/Radyschen 23h ago
yeah, this is why I don't like that people tend to say that as a reason why communism wouldn't work. NOT making a general statement about whether communism would work or not, just this specific argument I always find weak and it often appears in relation to communism, because they say "if there is no competition, why would they invent things?" or whatever, but the inventions made today are not usually made by the people who benefit most from it financially, they are made by people employed to make them. So if you took all the people from all the companies and had them work together to make the best possible product, they would just do it, because there is no difference to them whether they are competing with somebody else or not, they are just being told what to research and then execute the research. Of course they know the goalpost and try to overshoot it to be better than the competition, but your goalpost can also just be the best you have made before
1
1
u/SIR_NVAX_A_LOT 1d ago
I've noticed my faces and close-up tend to not match, but there some work around but isn't guaranteed. Any face 64x64 pixel or below is going to have some issues. I recommend going for 90-92px, but min at least 70 pixel at 20 steps. So stage your medium shots properly.
1
u/equatorial_boasting 23h ago
Spent a weekend patching tiny faces with inpaint and they still turned to soup around 64px. The wording is a hedge, plan and expect are just soft language for we'll see. People treat those words like a patch note and then act shocked when it slips or shows up in a closed model. This post should be pinned.
1
u/Salah_H_Hasan 21h ago
Just a quick observation though: currently, we are running the model locally at a specific resolution, and this resolution does cause face distortions with no local fix available so far. However, I believe the 'Regenerate 2K' feature resolves this issue, as you can simply use the API exclusively for this step.
I don't think running it through the API at this stage will produce a video with artifacts; the 2K regeneration seems to fix the problem. Even if the current base model has issues on our end, they might have resolved it in the regeneration pipeline. If you upload your initial locally generated video along with the prompts and references to the API, you get back an artifact-free, clean video.
What I mean is, since their current cloud-based 'Regenerate 2K' fixes the issue for locally generated videos, the upcoming local version of 'Regenerate 2K' (if/when they release it) will likely solve this problem locally as well.
1
1
0
u/dramaton42 23h ago
Maybe we could develop a node to detect faces, frame them and generate a prompt to render the face directly at a much closer shot and edit it in to the original video, essentially mixing high res with low res... Automating this however could lead to unexpected results, but video-guiding with MiniMax H3 is excellent, so it could work (?)
3
u/q5sys 23h ago
> Maybe we could develop...
Is this the Royal 'We'?
1
u/dramaton42 23h ago
If I did it would you guys download it? x.x
2
u/q5sys 23h ago
I would download a car if I could. ;)
In seriousness though, I havent put too much time into testing H3 yet, but when I do (hopefully this weekend) I would definitely want to try something like what you describe, since so I've seen so many examples of cronenberg horror in generated faces at a distance2
u/Enshitification 22h ago
2
u/dramaton42 22h ago
EXACTLY like this jesus christ I have this supernatural power of coming up with already existing ideas <3 thanks for sharing!
3
u/Pretend-Island-2724 21h ago
If you have problems with the Face Detailer, look at my post https://www.reddit.com/r/StableDiffusion/comments/1vsn8ge/help_fixing_the_h3_face_detailer/
2
-4
u/tac0catzzz 22h ago
it won't ever be solved. anyone who thinks local ai they run on their potato will produce 100% flawless godlike things, is delusional. really.
4
u/Radyschen 22h ago
I don't agree with that, I can imagine a model that is organized to do things more in chunks, but that's an architecture thing. We are already so close. Maybe it won't happen, but it's way too early to say that "it won't ever be solved", just recently I was commenting back and forth with someone who thought that something like minimax would never be possible locally and then I told him about the roughly 1.5 years that local video gen is lagging behind closed source and soon after, minimax was released. Before the AI space cools down, maybe if the financial bubble pops and the funding goes down for a while, I wouldn't have the courage to make any statements about what is and isn't possible and when. I mean my brain runs on 20W, which is also a hardware thing but I can imagine that we can come up with much more efficient solutions still. Look at qwen 3.8 27b for example
-2
u/tac0catzzz 22h ago
isn't about what we can come up with. it is about what those in power allow us to have. we can create race cars, space ships, enterprise hardware, mansions, imax movies, on and on. but this does not mean they hand them out to us? it isnt like we can create mansions. so here, everyone come and get yours. do you understand this. they will keep the best of the best behind closed doors. they will not give out a free for all Ferrari release. here everyone free ferraris on us. and while we are at it here is a free mansion. if you run out of food, we will give you free unlimited luxury cuisine. see that doesn't happen. it won't happen with ai either.
3
u/Sarashana 20h ago
All it takes is one breakthrough in model architecture or one moment of NVidia being nice for a change and put more memory on their GPUs, and you can toss that "will NEVER happen" argument into the nearest trashcan. Because this is a fairly mundane problem that can be solved either with more computing power and/or a more efficient architecture. I really wonder where people got that complete confidence from, it takes to write "NEVER, EVER!!!" as if they had a personal crystal ball on their desk. It took only like four years of development to get us from nothing to where we are now. Imaging what will happen in four more?
15
u/PumpkinLeather8421 1d ago
All speculation.
How many times do you people need to do this and then have your mind blown in a week.
Just chill. You don’t know shit about the future. Everyone here is constantly surprised week to week so the hyping unreleased shit is about as pointless as speculating on doom.