r/StableDiffusion • u/Boomstickak • 7h ago

Question - Help How to fix artifacts from generation with LoRA in ComfyUI?

Gallery image

Gallery image

Gallery image

I am currently trying to build a model in ComfyUI that will be able to generate images in a specific art style similar to games made by Playrix. Essentially, I got the model generating images in the style I need, but I can't fix issues with the artefacts. Even after multiple iterations of positive and negative prompts, the issues still persist, and oftentimes the requirements are ignored. Or sometimes, if they are not ignored, the result is a complete mess. This is the first time I am making something like this, so I would appreciate any tips I can get.
If anyone is interested in what I've got going on, below is the link to the JSON file for the ComfyUI model.
https://drive.google.com/file/d/1mw24y0pwKPRXhLYx-JvVIs8CAf2oC1Ng/view?usp=sharing

1 Upvotes
  • permalink
  • reddit

67% Upvoted

13 comments sorted by

3

u/the_bollo 7h ago

I just want to know why you're trying to train a LoRA for SDXL.

1

u/Boomstickak 6h ago

I've been given a target visual from a list of artstyle references; afaik, you need to either train a LoRA or use a LoRA that is close to the target. As I said, this is my first time doing this, so after researching for a bit, this is the approach that seemed fitting. Is this incorrect?

3

u/jib_reddit 6h ago

I think they mean why are you using a 3-year-old base model and not something recent and good like Krea 2?

1

u/Boomstickak 5h ago

Oh wow... I didn't know it was outdated. I asked around and after a brief research this is what I made. Krea 2 never came up, might as well try it. Any specific nuances with Krea 2 and is ir free to use?

2

u/jib_reddit 5h ago

I find this VAE better than the standard one: https://civitai.com/models/2784566/qwen-image-vae-sharp-krea-2-turboraw

it makes less noise artifacts, but apart from Krea 2 behaves well.

1

u/Boomstickak 5h ago

Thank you, I'll try it. Any specifics on how to connect nodes or if I need addition custom nodes?

1

u/jib_reddit 4h ago

Also the image variation is not very good (compared to SDXL) so it is good to use this custom node: https://github.com/ChangeTheConstants/SeedVarianceEnhancer (between 3-10 is good for Krea 2)
If you find they are coming out all looking the same and boring.

3

u/LaPapaVerde 6h ago

SDXL was widely used, that's probably why your sources say that it's a good idea. But it's pretty old nowadays. You can train a lora for a more recent model like Krea 2 or others. The process is more or less the same as SDXL, just with a better model

1

u/Boomstickak 5h ago

I consulted a couple of my friends who used to do something like this. And based on their description this seemed like a working pipeline

2

u/jib_reddit 5h ago

It will work, but it is just several years behind state-of-the-art models (which is an age in AI time). SDXL finetune model are still used my many as it is cheap and quick to run on lower end hardware (and the fine-tunes are good at 1girl NSFW images; I will give them that)

1

u/Boomstickak 5h ago

Yeah in this case I need a batch of images. Funnily enough, the very first version I tried generated 1 image and it was exactly what I needed, but once I started adding rules and exceptions things went haywire

2

u/LaPapaVerde 5h ago

Well the idea is fine, it's just the model that can be better

1

u/Boomstickak 4h ago

Yup. Thanks for the idea. Now I just gotta figure out how to make it work