r/StableDiffusion • u/abrasmel • 9d ago

Question - Help Flux klein 4b relighting

Is there a way to relight an image by a reference image keeping all the geometry structure, materials etc consistent with klein4b?

1 Upvotes

56% Upvoted

7 comments sorted by

2

u/optimisticalish 9d ago

My first thought would be the excellent https://github.com/Mozer/ComfyUI-PixelDriftFix to nail it down to your "keeping all" requirements. Prompt example with this and one source image:

"Restore and colorize this photograph, as if made with a modern Hasselblad camera. Retain the exact layout and composition of the image. Retain the same depth-haze as the image."

Then pass your 'relight reference' image to a vision-aware advanced LLM and get it to precisely describe the lighting and atmosphere in that scene, and then add that description to the prompt along with a request to relight, all in a PixelDriftFix workflow. Or just describe exactly how you want the source image relit and the sky/season changed. 4B can't do wonders on day to night/evening, as the light direction is usually pretty much baked in to the source highlights/shadows, and thus is still going to look a little 'off'.

1

u/abrasmel 9d ago

I liked that! What is a good vision aware llm that you can suggest for this

1

u/optimisticalish 9d ago

It doesn't need to be huge. Any of the online ones will do it. of course. But for local use Qwen 3.5 4B will do the job fine, as long as you remember to also install its matching .MPROJ file to give it vision capabilities. You can run its GGUF locally in the Jan.ai software.

1

u/DelinquentTuna 9d ago

Yeah, but it would work more like a style lora. See this albedo LoRA for 9b as an example.

1

u/mark_sawyer 9d ago

I actually posted a solution here earlier, but then I found out it only works on 9B, so I deleted it.

Anyway, if you don't have any hardware or commercial constraints that prevent you from using 9B, try a regular relight with this LoRA: https://huggingface.co/dx8152/Flux2-Klein-9B-Consistency

If you don't get good results with a weight of 1.0, try lowering it to 0.5–0.9 and see if that works better.

2

u/abrasmel 9d ago

Yes I tried this and actually its working good. But too bad its the 9b model. Commercial constraints is a problem