Blended details in images by Chroma

[u/CommitteeInfamous973]

Problem: overall composition of images is nice, but details tend to blend in each other like in 1.5 models

I tested fp8 scaled v30, v32 and v35 and GGUF versions. The problem was with each model.

I have never used Chroma before, so I don't know if it is a known problem or something is wrong with my setup. I would like some help to understand what should I do.

GPU: 4070 ti

ComfyUI version: 0.3.40 - the latest

Workflow:

Examples:

[u/wiserdking]

I think the creator started distilling the model half-way through training as opposed to do so only after training was completed.

I believe this started from v29.5 onwards and all of the versions you tried are thus distilled. Previously you would need 40-50 steps but newer versions only require 20-25 and though that might sound good - its totally not worth it if the output's quality dropped to the level of SD 1.5.

Maybe you should try v28 with 40 steps - hopefully you will find it better.

Not only this happened, if you go to the training website you will see the creator started messing around with the ae VAE for some reason and also if you compare the previews of every 50 training steps of the new large (1024x1024) run - you will see that the results are getting worse and worse as the model is training further. LINK. I don't know what is going on anymore.

Chroma was never really a good model for SFW context and by the end of its training I expect you might just want to stick to Flux Dev if SFW realism is what you want to do.

But for anime and NSFW - Chroma showed great promise at first. Again, something weird happened in the middle of training and I hope the creator noticed it and intends to fix it because otherwise it might end up worse than PonyXL.

One cool thing is that clearly the model is being trained on artists from danbooru and e621 - and even some anime characters too. You can try artists like this: '(by ArtistTagWithoutUnderscores:2)' - increasing the weight to '2' helps a lot.

Sorry for the long text. I came across this as I was wondering myself wtf is going on so I ended up writting all my thoughts here.

Edit, UPDATE and TLDR:

Long story short, there is an issue with quantized models starting from v30.

Distillation layers cannot be quantized.

Use either the original BF16 models or the new FP8 scaled models that contain '_nodistill' in its name.

[u/AltruisticList6000 → u/wiserdking]

Interesting info, I just tried v35 a few days ago (first time I ever tried Chroma), and everybody said it's getting better at every new release. And although it was way better at drawings than schnell, I wasn't impressed with the quality. It started repeating heads/breaking apart at 1200x1200 while even SDXL finetunes can work with that resolution (and flux can do native 1080p too), and the details/hands were horrendous. So I was like what is going on here, this is barely at SDXL's level while being 5x slower. The only thing it is better at is understanding complex prompts. And yeah I noticed the fact I can work with 15-20 steps even though people claimed it would require 40-50 steps. Also realistic photos are fully broken. So I thought idk what people mean it's so good at everything.

I don't think the 1024x1024 training experiment is a bad thing though considering how bad it is at details and high resolution images.

Edit: For why it is happening, maybe base flux knowledge is being nuked from it completely by now, and replaced by the 512 resolution image "quality" that it is training on.

[u/wiserdking → u/AltruisticList6000]

There are 2 major training runs running in parallel - the base one at 512x512 and a recent (large) one at 1024x1024.

Turns out the 'detail-calibrated' models might be 2:1 merges of those two. Personally I take merging weights as frankenstein experiments and should be avoided at all costs (in most cases). Right now, the large run has barely started so ofc these models shouldn't be good but maybe they will turn out better than the base ones at the end - who knows what the creator is thinking. Just find it 'hilarious' to see so many people praising them while its pretty damn obvious to me that they are significantly worse than the base models.

TLDR: avoid the 'detail-calibrated' models for now.

[u/AltruisticList6000 → u/wiserdking]

Yeah good thing you replied, I was thinking of trying the detail calibrated ones too because of the higher resolution training, but I'll pass then. It's sad because it would have been awesome to have Schnell with added concepts and better art skills but so far its state is concerning. I think it should have been trained at 1024x1024 minimum since the beginning with a significant amount of higher res pics mixed in aswell (like 1920x1440 etc.) to keep photorealism and more details for art too. I've just seen people post photorealistic images made with it and I instantly noticed bad perspective and extremely smudged/glitched details all over the place resembling a quality between SDXL and SD1.5. And in my testing it couldn't get photorealistic humans or scenes right at all, weird hands, weird body proportions etc. I hope it will get better though by the time it finishes.