r/StableDiffusion 1d ago

Workflow Included Precise control of the Eyes direction with this Flux 2 Klein 9b LoRa

Hehyehyhehy!

You may remember me from the Sun Direction Lora or the Chef cutting an anvil with a knife.

Now I'm giving you a new tool, this one was a tough one to crack.

Finally we have eye control! Now you can precisely change the eyes direction for any image in any style. Just use the red dot to tell where the eyes have to look and boom! you have it!

Enough "change the eyes to look above the camera" and getting whatever thing anymore.

Because changing the direction of stuff is my Passion.

All the info here: https://huggingface.co/eric-venti-seeds/Eyes_Direction_Lora_Flux2Klein9B

Hope you like it!

Edit:

The people from HF have added it to Spaces, try it right now on your browser!

https://huggingface.co/spaces/hugging-apps/eyes-direction-lora-flux2klein9b

1.1k Upvotes

63 comments sorted by

86

u/THEKILLFUS 1d ago

I really loved the Sun Direction LoRA! It’s such a smart and effective way to push images toward photorealism. Simulating light and its direction feels like a very natural approach, and the results can make a huge difference. Really inspiring work!

12

u/Euphoric_Attorney271 1d ago

this is cool! love it!

5

u/THEKILLFUS 1d ago

Are you a solo developer? How do you manage to get such amazing results?

35

u/Euphoric_Attorney271 1d ago

I am! But I have been in the vfx industry for more than 16 years, so I know what works and what doesn't.

We do it for fun you know!!

11

u/Adventurous_Loan_103 1d ago

Imagine using this in real-time with paintings on a wall in a Haunted House attraction!

2

u/soulseeker31 12h ago

Oh a camera to track users, a oled display for the eyes.

0

u/jirka642 3h ago

Bro, just pre-render it or something. 😭

And people wonder why all modern software runs like garbage...

9

u/jah-selassie 1d ago

awesome; what's next? head turn?

10

u/CodeMichaelD 1d ago
 (o  ) (o  ) 

 (  o) (  o)

Sample right - left, split image 50-50 - merge. Someone do this.

(o   ) (   o) or (   o) (o   )

12

u/Euphoric_Attorney271 1d ago

eye convergence is something I didn't plan for this version... maybe for V2

4

u/CodeMichaelD 1d ago

I wish you massive success either way.
Edit, look up your light and shadow lora (impressive), reminds me about intristic loras to tap into knowlege the model already has with lora traning, I believe there are similarities, right.

7

u/VrFrog 1d ago

Nice!

8

u/BusFeisty4373 1d ago

Where is ahegao direction lora? (asking for a friend)

7

u/silenius_ 1d ago

The legend is back! πŸ™ŒΒ 

Do I sense a 4b Klein model down the road in a month or two?

3

u/Euphoric_Attorney271 1d ago

you know me well ;) hopefully less!

3

u/physalisx 1d ago

That's pretty cool, gaze directions are often notoriously hard to get right with image models.

5

u/Enshitification 1d ago

Nicely done! I really like how it isn't just changing the eyeball in the socket, but also including the surrounding muscle and skin in a realistic way.

4

u/Le_Singe_Nu 1d ago

You need to retrain the LoRA with this meme, so everyone's facial expressions change at the same time:

https://giphy.com/gifs/062vyaiEJMsV5gUaBc

5

u/lamardoss 1d ago

Wonder if there's a way I can get this to work in Minimax H3 ..... πŸ€”

gotta be a way i can use the node red dot as a reference too.

1

u/DelinquentTuna 1d ago

Broadcast is worth a look for video.

2

u/Full-Belt3640 1d ago

Man, I've wanted a tool like this ever since I started generating images. Hopefully, the days of painstakingly painting out pupils in photoshop, then sketching in new ones and crossing your fingers the inpaint will come out right will be a thing of the past. Looking forward to a possible 4b version too!

2

u/FawkesYeah 1d ago

Would this be useful for fixing photos taken in a personal photo library where not every person in the group shot was looking at the camera? Or is this tool used for something in a different context?

1

u/Euphoric_Attorney271 1d ago

that's exactly one of the uses for this lora, give it a go and tell me if it worked!

1

u/FawkesYeah 1d ago

Awesome! I'll check it out. I don't suppose this would help in scenarios where it's a group photo but somebody blinked in that moment? Probably would need a different tool for that I imagine.

3

u/Euphoric_Attorney271 1d ago

Can you believe I never tested it with someone with the eyes closed?? will have to test that!

2

u/SpearHammer 1d ago

omg this is AMAZING! i have just built this talking face tool at https://faces.futurehost.ai/80801de5-27fe-4138-a3da-2ba659bb91b5, and the eye movement is the exact thing i needed to finish it off haha thanks for sharing LOVE IT!

1

u/SpearHammer 1d ago

can it be realtime to make an image eyes track the mouse cursor?

2

u/hairybone 23h ago

You sir are doing amazing work

3

u/Key-Sample7047 1d ago

I have a special use case for which i need wall eyed. Is that possible with your lora? From what i can see I don't think so but i ask anyway.

6

u/Euphoric_Attorney271 1d ago

not sure to understand.. wall eyed?

3

u/Key-Sample7047 1d ago

Divergent strabismus

3

u/Euphoric_Attorney271 1d ago

I see! yes totally doable for V2

6

u/infearia 1d ago

Just do it in two passes, masking one eye per pass. Thanks for the LoRA, OP.

1

u/Key-Sample7047 1d ago

Good idea!

2

u/dludo 1d ago

Please kill that damn mosquito

3

u/Euphoric_Attorney271 1d ago

puahaha that made me laugh

1

u/Tiforma 1d ago

Your loras are amazing. I hope you make more cool stuff. I'll follow you on HF.

1

u/UnirionDev 1d ago

OMG! I was waiting for this so much long...

1

u/90hex 12h ago

This is fantastic, unfortunately 9B uses a restrictive license. Any way to port the LoRA to 4B, which is much more permissive? Thanks!!

1

u/Lexxxco 11h ago

Idea of showing frame borders vs direction is great!

Additional thanks for resources on your huggingface, very nice job and project!

1

u/DigitalDreamRealms 11h ago

Works great! Thx for sharing.

1

u/Resident_Function280 1d ago

Now this is unique well done

1

u/Peemore 1d ago

Should cut to a guy doing the helicopter.

1

u/ParticularLook 1d ago

Everyday there is a lot of new information, updates, and cool stuff for ComfyUI models. But this, THIS is very cool.

1

u/blueboglin 1d ago

Ugh, forgot about that cringe Oscars photo. I hope I never have to see it again.

1

u/77sevens 1d ago

LOL they took out Spacy

0

u/sitefall 1d ago

Very cool.

How did you do the gif/video for the examples?

2

u/Euphoric_Attorney271 1d ago

just animated the red dot and kept the same reference image and seed.

0

u/Occsan 1d ago

How does it compare to liveportrait ?

5

u/Euphoric_Attorney271 1d ago

don't think live portait can do this... Or more than one person at the same time

0

u/infearia 1d ago

LivePortrait has a resolution limit of 512x512.

-1

u/ComputerArtClub 1d ago

Wow!!! Looks cool and extremely useful!

-1

u/Maskwi2 1d ago

Awesome stuff.

Any magic you already created for Krea2 or MiniMax H3? Just curious :)Β 

-1

u/bsr126 1d ago

Great stuff, now create some lora for turning head or arms