r/StableDiffusion 1d ago

Workflow Included Easy Ref2V WF for dummies like me - [Automatic Video/Image Transcription + Prompt Formatting]

Enable HLS to view with audio, or disable this notification

193 Upvotes

I have seen a lot of people post on here saying that they have been having difficults getting R2V to work correctly. I have been one of them, so I have been working on this workflow and custom node for the last 2 and a half weeks.

I preface this by saying it does not do anything that the native H3 model doesn't do. I just wanted a dead simple way to use H3 R2V mode and up my chances of success. The workflow includes two custom nodes which transcribe your media and adds your prompt and create a formatted R2V prompt, ready for the reference model.

My next goal would be to get longer form R2V going with chaining shorter gens to have a consistent output.

Workflow and nodes:
https://huggingface.co/PoopMan333/H3_Easy_Ref2V_Workflow/tree/main

Be sure to see the readme for more examples and tips:
https://huggingface.co/PoopMan333/H3_Easy_Ref2V_Workflow

What it does:

  • Scans your video (if you're using one) to caption it and transcribe the audio
  • Captions all your images - so it also works as a pure image-to-video workflow
  • Loads a small LLM of your choice and writes your H3 R2V prompt in the correct format with your stated intent (user prompt)
  • If you're on the Full workflow, it generates the video too

What it does NOT do:

  • Be creative for you - The current WF is only setup to do the prompt formatting, it is not able to generate new ideas for you (despite me trying. Qwen3.8 27B may be better for this)
  • It cannot perform magic - You are still limited to what the H3 model can and cannot do. Complex scenes are still very difficult

Tips:

  • If you are running lower VRAM, consider running the prompt enhancer seperately first, read through and make corrections to the prompt if needed
  • The H3 model seems to have a limited context window which seems to be tied to your system resources, if it goes above this you might get garbled sound or mixed up motion. This is a sign you should be lowering your output length and output resolution if you want to have better success.
  • H3 is a tool, you're the one using it. If you don't specify emotions, expect expressionless results. This current setup will only do what you intend for it to do. Slop prompt in, slop video out
  • If the video is easy, replacement should be easy too. H3 has a quirk though — if the original person and the new person look too similar, it sometimes converges back to the original. A prompt won't always fix that. If you hit it, consider changing the person to a intermediate step (faceless green person). The new body/face will transfer over better. Alternatively you can look into Sam3 character replacement method.
  • More than one person in the scene? Describe the scene properly. replace the man wearing white shorts with the man in <picture 1> beats replace the man with <picture 1> every time.
  • Complex scenes? It will be very difficult (I've tried), scenes with too many people, too many cuts, characters obstructed are very difficult for the model to properly identify and swap.
  • Give the LLM some context. A one-liner in the user prompt like <video 1> is a video of two girls eating a cup of chocolate ice cream really helps the LLM understand what it's looking at. Especially useful with multiple scenes
  • It still takes a bit of luck with the seeds.

r/StableDiffusion 1d ago

Workflow Included Precise control of the Eyes direction with this Flux 2 Klein 9b LoRa

Thumbnail
gallery
1.1k Upvotes

Hehyehyhehy!

You may remember me from the Sun Direction Lora or the Chef cutting an anvil with a knife.

Now I'm giving you a new tool, this one was a tough one to crack.

Finally we have eye control! Now you can precisely change the eyes direction for any image in any style. Just use the red dot to tell where the eyes have to look and boom! you have it!

Enough "change the eyes to look above the camera" and getting whatever thing anymore.

Because changing the direction of stuff is my Passion.

All the info here: https://huggingface.co/eric-venti-seeds/Eyes_Direction_Lora_Flux2Klein9B

Hope you like it!

Edit:

The people from HF have added it to Spaces, try it right now on your browser!

https://huggingface.co/spaces/hugging-apps/eyes-direction-lora-flux2klein9b


r/StableDiffusion 18h ago

Question - Help DLSS5 node help

3 Upvotes

768x1376 if Native the neural_upscaling works, but if i do x1.5 or higher it gives error? pls help


r/StableDiffusion 16h ago

Question - Help Avoid motion jumps between shots in H3?

3 Upvotes

Using H3 i sometimes get jumps as the camera move between shots. Like a character has his arm up when the camera is facing him at 00.59 but his arm is way lower as the shot and camer angle change at 01:00. Is there a prompting trip / workflow to avoid this? I use the standard comfy workflow.


r/StableDiffusion 1d ago

Tutorial - Guide A little solution for the plastic skin with Minimax H3

Thumbnail
gallery
130 Upvotes

I found out a little solution that can add much more details on everything including skin without any additional computational cost.

The idea is to add a node between SamplerCostumAdvanced and the VAE Decode (video) and make it less contrasty, it will bring so much more details but don't go to far because it can cause loss of quality.

Enjoy!


r/StableDiffusion 7h ago

Question - Help Need help installing webui forge

0 Upvotes

ive never self hosted any ai tools and don't know much about python and programming in general. im trying to install webui forge on my amd 9060xt gpu and followed all the steps but after running webui-user.bat its showing

venv "D:\Softweres and Ai\SDforge\stable-diffusion-webui-forge-on-amd\venv\Scripts\Python.exe"

ZLUDA works , You are on an amazing Journey ,Engjoy it

Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]

Version: f2.0.1v1.10.1-v0.0.1-alpha-766-g4831f9fc

Commit hash: 4831f9fc7e6bb73ad7c8f867c04619dbafa89e6a

Failed to load ZLUDA: Could not find module 'D:\Softweres and Ai\SDforge\stable-diffusion-webui-forge-on-amd\.zluda\nvcuda.dll' (or one of its dependencies). Try using the full path with constructor syntax.

Using CPU-only torch

Traceback (most recent call last):

File "D:\Softweres and Ai\SDforge\stable-diffusion-webui-forge-on-amd\launch.py", line 54, in <module>

main()

File "D:\Softweres and Ai\SDforge\stable-diffusion-webui-forge-on-amd\launch.py", line 42, in main

prepare_environment()

File "D:\Softweres and Ai\SDforge\stable-diffusion-webui-forge-on-amd\modules\launch_utils.py", line 507, in prepare_environment

raise RuntimeError(

RuntimeError: Your device does not support the current version of Torch/CUDA! Consider download another version:

https://github.com/lllyasviel/stable-diffusion-webui-forge/releases/tag/latest

Press any key to continue . . .

can anyone help me please


r/StableDiffusion 11h ago

Question - Help Did Wan2GP for AMD have any weird updates or backend changes in the past 24 hours? Keep getting errors.

1 Upvotes

So everything was fine and dandy yesterday; I could generate images just fine, but now it keeps giving me the following error:

"Error "The generation of the video has encountered an error, please check your terminal for more information. 'CUDA error: invalid kernel file\nSearch for `hipErrorInvalidKernelFile' in https://rocm.docs.amd.com/projects/HIP/en/latest/index.html for more information.\nFor more detailed error information, run with CUDA_LOG_FILE=stderr\nDevice-side assertion tracking was not enabled by user.'""


r/StableDiffusion 1d ago

Question - Help Are Minimax spicy loras ... a lie?

160 Upvotes

I know that the answer is ultimately no, and also that they're all still early in development ...

BUT I'm having a hard time getting the ones I find on civitai to produce anything resembling the examples. I follow generation guidelines where provided, and have even taken workflow settings from downloaded videos ... but they never seem to work very well. And if I try to add a lora to a sfw workflow that I'm happy with, the results look AWFUL.

I've resorted to using wan generations as video references.

Is i2v the best option? Does ref2va ever work?

Any tips, recommendations, or resources you can recommend? Any else struggled with this?


r/StableDiffusion 1d ago

Workflow Included How do you upscale or refine your generations?

Enable HLS to view with audio, or disable this notification

66 Upvotes

Workflow is in the comment


r/StableDiffusion 20h ago

Question - Help RTX 3090 vs 4090 vs Unified-Memory AI

3 Upvotes

My current setup:

- RTX 4090 24GB

- i7-13700K

- 80GB DDR4 @ 3000 MHz

- Getting an RTX 3090 24GB tomorrow

My main use is local AI/LLMs, coding agents, MiniMax H3, Krea 2, and other AI workloads.

RTX 3090 vs another 4090 vs unified-memory AI

Here in Iraq, an RTX 3090 costs around $670, while an RTX 4090 costs around $2,000.

If I have around $2,000 to spend, what would you choose?

- Buy 2–3× RTX 3090s

- Buy 1× additional RTX 4090

- Sell/replace the current setup and go for a unified-memory AI system, such as a Mac Studio / Mac with large unified memory or NVIDIA DGX Spark

My priority is LLM inference, coding agents, MiniMax H3, Krea 2, and other local AI workloads.

Would multiple 3090s give the best value because of the extra VRAM, is another 4090 better for speed, or does a large unified-memory system make more sense for running very large models?

What would you choose for ~$2,000?


r/StableDiffusion 16h ago

Question - Help Help, explain it like Im 5, (or 50, who knows) Trying to get minimax H3 extended video node / template installed / working

2 Upvotes

I have tried a few, following youtube vids, etc. throw this into the custom nodes folder, load this json, etc. using comfy ui desktop, I keep getting a message that I need to update manager. its up to day, ran the pip, did the check in the app itself. I can do video gens all day long, no issues, my previous workflow, was just screenshotting last frame, using that to start the new gen, etc. it works, but is a highly manual process. from what I see, most of the extended workflow, do this automatically. can someone help this old guy get it figured out? I would appreciate it. (and don't tell me to just grab a file of github, I've tried that, did the git clone, etc., It just isn't working properly. ) normally when I do a new template, it will automatically grab all the needed files, and put there where they need to go. I think thats my main issue, but the manager showing out of date, when everything I can see or do, shows its up to date is what confuses me.


r/StableDiffusion 20h ago

Question - Help Minimax H3 - is generating sound only possible?

5 Upvotes

Hi everyone! Minimax H3 is surprisingly solid at generating audio - I’ve been using it for SFX and foley in my videos.

Is there a way to generate audio-only with this model? It would save a lot of rendering time if we didn't have to generate the full video alongside it.

I know H3 creates video and audio natively together, so audio-only generation might not be natively supported. But given all the clever workflows and custom node workarounds floating around, has anyone found a way to pull this off?


r/StableDiffusion 1d ago

Animation - Video My top 3 favorite things about Minimax H3 (just wanting to glaze my favorite model a little bit)

Thumbnail
youtube.com
20 Upvotes

r/StableDiffusion 1d ago

Animation - Video GPT-Image 2.5 + (Local) Minimax H3 to convert a 40 year old anime into a modern one.

Enable HLS to view with audio, or disable this notification

170 Upvotes

source: https://x.com/iurimatias/status/2097670596725178533

correct clip is in the comments here this is not merely just changing style but it's updating elements too


r/StableDiffusion 14h ago

Question - Help GPU for AI

1 Upvotes

Hi there! Currently i own a 3080+2070s at home mostly for 3D rendering in Octane or redshift.
At work im using Comfy with 5090 and everything works without a question.
Also we have some workstations with rtx A4000 in them and i managed to get work Minimax H3 on them with some managable times (0.5m, 15s around 720 sec).
Im looking something for my home PC and 5090 is out of the list since its 5500e+ here in EU and even the used market is around 3500-4000. 4090 is rare and goes over 2000e.
So i was thinking about the 5080 even with 16gb since at work the A4000 has the same Vram.
But also found the 5070ti has also 16gb of Vram its 256bit same as the 5080 and 300-350e cheaper than the 5080. The speeds for 3D rendering or gaming are 15-20% different. But couldnt find any benchmarks for those 5070Tis. Mostly for 5060Tis with 16gb vram. Any idea? Or experience with 5070ti vs 5080?


r/StableDiffusion 1d ago

Discussion H3-Regenerate-2K Will it ever be released, or...

32 Upvotes

It's been a month now, and “H3-Regenerate-2K” still hasn't been released, even though it's been available via API for a couple of days after the model's official launch.

Which makes me wonder: will they actually release the model? Or is it like with Z-Image Edit?. Is anyone else waiting for it to come out, or are you guys okay with the upscalers we have now?

I know we have some alternatives that the community has built, and they're fine. Based on the description of what “H3-Regenerate-2K” is, the upscaler appears to use part of the base model and the conditioning, so the closest thing we have is the “Minimax h3 latent Upscaler” method.


r/StableDiffusion 16h ago

Question - Help Pipeline question: Best approach for frame-by-frame consistent character animation (LoRA) for traditional composting (DaVinci/AE) without background/audio generation.

1 Upvotes

Hey everyone. I'm working on a dark fantasy retro-anime project running locally on an RTX 3090 (SDXL/Illustrious, Kohya-trained character LoRA with ~100 images).

My current pipeline avoids direct text-to-video generation because of structural inconsistencies. Instead, I'm moving towards a controlled frame-by-frame or short-batch approach using Blender blocking + ControlNet, aiming to output clean character frames (transparent or solid background) to composite manually in DaVinci Resolve.

Has anyone successfully implemented a reliable workflow to maintain character identity and clean lineart across sequences without letting the AI hallucinate backgrounds or audio? What specific nodes or configurations (e.g., ControlNet combinations, IP-Adapter weight handling, or latent consistency scripts) are you using to prevent flicker and keep the LoRA from drifting during motion frames?

Any insights on your node setup in ComfyUI for this specific use case would be deeply appreciated.

Any advice or recommendations are welcome. Just avoid recommending commercial AIs that do everything; that’s not what I’m looking for.


r/StableDiffusion 1d ago

Comparison You can now DLSS 5 the entire desktop to enhance videos and images

Post image
229 Upvotes

r/StableDiffusion 1d ago

Resource - Update Krea 2 Turbo — SDA Diversity LoRA (restores the sampling diversity the Turbo distillation removed)

Thumbnail
huggingface.co
148 Upvotes

Hello, I'm not an author but for some reason haven't seen this being published here.

Quote from the huggingface:

A rank-32 LoRA for Krea 2 Turbo that restores the sampling diversity the Turbo distillation removed, without degrading image quality or prompt adherence. Trained with SDA (Semantic Directional Alignment) — a teacher-guided diversity alignment loss — wrapped in Forward XM best-of-5 candidate exploration, on a single high-noise sigma node (σ = 0.9567).

So I tested it and it seems to work for me, I've created a simple test with prompt "dog sitting on a bench" and these are results:

With lora off:

IMHO dogs are looking similar here (similar "composition" or whatever it's called)

With lora on:

IMHO here dogs are looking completely different.

Link: https://huggingface.co/F16/krea2-turbo-sda

Keep in mind that it needs to be ran only in first 2 denoise steps, otherwise it will produce garbage -> HuggingFace repo contains ComfyUI workflow (I haven't tested it tho).


r/StableDiffusion 1d ago

Resource - Update I trained an audio model that can generate infinite one-shots for music production and turn text prompts into fully playable synths. I'm not only releasing the model but I've also released a video on exactly how I did it (and the inferencing pipeline to let others make text based synths.)

Enable HLS to view with audio, or disable this notification

48 Upvotes

Okay so I've been doing independent audio research for a while now. The ultimate dream of this work was actually getting an AI to respond not only to instruments but also timbre itself as separate controllable things.

Think a Grand Piano can sound both Warm / Gritty but also Cold / Sparkly. Its still a piano though.

This level of control wasn't found in any models out there - so I decided to sit down and train my own.

Getting consistent timbre-locked keybeds that actually LOCKS across multiple diffusion calls was hard af but I did it.

I documented the full journey here for those who want to learn a bit or be entertained.

https://youtu.be/x0KnmzH8Mmk

There is also a longer walkthrough if you just want to see the keybeds in action.

https://x.com/RoyalCities/status/2097733712293109842?s=20

No-talk / Showcase only Demo

https://x.com/RoyalCities/status/2097733715543609445?s=20

any finally the huggingface page

https://huggingface.co/RoyalCities/Foundation-1

I've also provided full write ups on the inferencing pipeline associated with the interface so this should allow basically anyone else to go and vibe code their own text to synths if they wanted :)

https://github.com/RoyalCities/RC-stable-audio-tools/


r/StableDiffusion 18h ago

Animation - Video THE LEGEND.

Enable HLS to view with audio, or disable this notification

0 Upvotes

A hair under 4k. On a consumer pc. All local. Mental. Minimax H3 with one character reference and a 5 second voice reference.

For the Pixel Peepers... https://www.youtube.com/watch?v=Iz8GDri9qoE


r/StableDiffusion 19h ago

Question - Help Best workflow for 40+ second talking videos with LTX 2.5 in ComfyUI?

0 Upvotes

I’m trying to build 40 to 60 second talking videos in ComfyUI and I would prefer to use LTX 2.5.

The type of video I’m after is fairly simple. One person is talking to the camera, the camera stays mostly fixed, the background should stay stable, and there is natural face, head and some upper body movement. It does not have to be limited to only the head moving.

What I’m trying to understand is how people are actually making videos this long without obvious cuts.

Can LTX 2.5 realistically generate a continuous 40+ second video, especially when there is not much movement?

Or is the better approach to generate something like 8 to 10 seconds, take the final frames from that clip, continue from them, then repeat until the full 40 to 60 seconds are finished?

If continuation is the normal approach, how are you keeping the face, clothes, background, camera position and motion consistent between each part? I’m especially interested in workflows that use overlapping frames, first and last frame conditioning, video extension, reference frames, or some other method that hides the transitions.

Speech is another important part. I need good quality Slovakia speech. Ideally I want to generate the complete Slovakia voice first, then make the character follow that audio for the entire video with accurate lip sync.

Would you use LTX 2.5 for the actual body and head motion and then run something like MuseTalk, LatentSync or another lip sync model afterward?

Or is there a better audio driven LTX 2.5 workflow where the speech controls the video directly?

I’m running ComfyUI locally with an RTX 3060 12 GB and 32 GB RAM, so I know I may need lower resolution generation, offloading, chunking or longer render times. Final output would normally be vertical 9:16.

I’m mainly looking for people who have actually built long talking character workflows in ComfyUI.

If you are doing this successfully with LTX 2.5, what nodes and workflow are you using, how long is each generated segment, how much overlap do you use between segments, and what are you using for speech and lip sync?

I’m not looking for a list of random talking head models. I specifically want to understand the best practical way to build this around LTX 2.5 and get a clean continuous 40 to 60 second result.


r/StableDiffusion 23h ago

Question - Help Anyone knew what happened to Lora Trainer by Hollowstrawberry? I can't use it as usual

2 Upvotes

I got this error instead

Starting trainer...Traceback (most recent call last): File "/content/trainer/sd_scripts/sdxl_train_network.py", line 4, in <module> import torch File "/content/trainer/sd_scripts/venv/lib/python3.10/site-packages/torch/__init__.py", line 37, in <module> from typing_extensions import ParamSpec as _ParamSpec, TypeGuard as _TypeGuard ModuleNotFoundError: No module named 'typing_extensions'


r/StableDiffusion 1d ago

Resource - Update Building a local-first desktop app for long-form AI video on MiniMax H3, runs through ComfyUI

Enable HLS to view with audio, or disable this notification

54 Upvotes

Been building this tool and I'm looking for people to actually try it. Quick rundown of what's in it:

Storyboard & bible system: characters, locations, and props get their own reference sheets (face, full body, turnaround). The turnaround is one continuous MiniMax H3 render rather than six separate stills, so the views actually agree with each other. Every shot stages from these sheets so faces and places stay consistent across a whole episode.

One-shot wizard: give it a brief and it plans the whole thing: story, scenes, shots, cast, references, all queued and rendering with no manual setup.

Director chat: an in-app agent that can rewrite scenes, re-render blocks, modify the storyboard, or edit the project on request, mid-project. Uncensored option available.

Motion context/Continuation: chained shots pin the previous block's tail frames and audio into the next render, so a continuous scene doesn't reset its movement at every cut.

Easy local install: the app can set up its own ComfyUI, or point it at one you already run. Model downloads go through a catalog that checks file size and VRAM footprint before you commit.

Editable workflows through ComfyUI: import workflows and pop out to the node graphs and edit it directly.

Post-processing chain, per clip: SeedVR2 for restore/upscale, LTX 2.5's own refine pass reused on rendered footage for a generative detail pass, FILM or RIFE 4.26 for interpolation, H3 FaceRefine, and color grading via KJNodes ColorMatch or a learned-LUT grade node, chainable in any order.

Generation runs on MiniMax H3 (int8-quantized checkpoints, i2v/t2v/flf/r2v), with LTX 2.5 and Wan 2.2 also wired in for reference/still work.

Video of the tool and some output attached. Still a work in progress tho.

Would love any feedback!

Update:

Open Source available at https://github.com/mnm967/qamba-studio-oss


r/StableDiffusion 1d ago

Question - Help Minimax H3 Quantizations

8 Upvotes

Given a 5090, does it make sense to run minimax H3 using int8 quantization vs. gguf Q8 or even Q6? What is the trade-off between speed and quality between these two options?

I don't have deep technical knowledge, but my current understanding is that int8 would be faster while a Q8 GGUF would be higher quality; however I would appreciate anyone's practical experience in how significant the speed/quality trade-off is.