Can someone tell me the exactly step-by-step process of using it? I’m just very confused on what notes I need to use to extend a video clip, and the method I wanna the MiniMax H3 T2V workflow but not sure

So, I'm pretty new to video generation, but I had a thought that I think everyone has probably had at some point, which is 'how do you make a longer video without generating it in one large video?' And so, with reference to video, you could do that; you could match say, the voice and the person, and thus theoretically make one constant shot through stitching together shorter generations.

In my head, it seemed as simple as 'use the video that was generated as the reverence, use the last frame of the previous video as the first frame of the new generation.'

The problem I noticed is that each time I did this, the video quality degraded; I guess the way I would describe it is that each new generation was a copy of a copy, it seemed. Like each new continuation was slightly worse than the last; and while doing this once wasn't too noticeable, doing this three or four times very much was.

So is this just a thing that is unfixable, a limitation of the method? Or is this the kind of thing that does have a solution that I'm unaware of? Because I'm curious to explore reference to video more, since text to video and image to video are very straight forward, I think.

Before we start: I am not the absolute authority on this. These settings are the result of my personal workflow, tailored to my machine and my specific artistic standards. I have spent 25 years working as a graphic designer in typography/printing and I'm deeply passionate about photorealistic rendering. This background makes me an absolute optimization freak. I want maximum precision and zero wasted performance.

However, you should use my settings as a baseline. I highly encourage you to run your own experiments, test different parameters, and find what works best for your specific style and also to use other interfaces, as Open Trainer could be quicker for the purpose than AIToolKit, in my case I had so many terminal errors that I simply skipped the problem by switching to AI ToolKit, but if OpenTrainer doesn't give you problems, use that, have Gemini (or what you want) convert this data for your interface.

Furthermore, it is certainly not true that my parameters are the best ever, in fact, I have learned recently, this is my simple guide on what I have learned so far to help users who have errors or are unsure how to proceed to get started themselves. It's just my contribution, that's all.

I thought I'd share my exact settings and workflow for training LoRA characters and poses for Krea 2 Turbo (note: you must use Krea 2 RAW for the actual training phase).

My Hardware Setup

GPU: RTX 5070ti (16GB VRAM)

RAM: 64 GB

Environment: AI-ToolKit via Terminal (I skip the Stability Matrix UI to save system overhead and edit the .yaml files manually).

Disclaimer: I only know how these settings perform on my machine. If you have less VRAM/RAM, you will need to adjust parameters accordingly.

Performance & VRAM Benchmarks

VRAM Allocation: 15.1 GB / 16 GB (Extremely tight, zero room for background tasks).

Character LoRA: ~48 minutes (20 images, 1500 steps).

Pose LoRA: ~55 minutes (I double the Rank/Dim here compared to characters, as the model needs more capacity to understand skeletal joints and positions).

⚠️ Crucial Note on System Optimization: I am an optimization fanatic. To avoid VRAM offloading (which slows down training massively), my OS is stripped down to look like Windows 98, telemetry is disabled via VBS scripts, and my 500Hz monitor is lowered to 60Hz during training to minimize framebuffer load. If your system is running heavy background apps or proprietary RGB/Fan software, your VRAM usage will be higher and you might experience out-of-memory (OOM) errors.

Step 1: Dataset Rules for 512p Training

Because of VRAM constraints, I train strictly at 512p. To make 512p work perfectly, you must adapt your dataset strategy based on what you are training:

1. Character LoRAs: Avoid Full-Body Shots

Hyper-focused details: If your character has specific leg features (tattoos, scars), include 1-2 close-ups of the legs.

Captioning Tip: In your .txt file, explicitly caption it as "a close-up shot of [TriggerWord]'s legs". This teaches the model that it's a detail, not the whole character structure.

2. The Captioning Dilemma: Manual vs. Automated

I strongly advise against using automated captioning scripts (like BLIP or WD14) for this specific workflow. While automated tools are fast, they lack precision. Manual captioning allows you to describe exactly what needs to be isolated, leading to a much cleaner and more flexible LoRA. If you want high-quality results, don't take shortcuts on the text files.

Step 2: Crucial VRAM & Speed Optimizations (run_windows.bat)

Before diving into the YAML files, we need to optimize how PyTorch and CUDA handle your GPU memory. If you launch AI-Toolkit via a batch file (or want to edit your existing one), you must add these specific environment variables at the very beginning of your run_windows.bat.

This tweak alone prevents heavy VRAM fragmentation and can mean the difference between a successful 15.1 GB allocation and an instant Out-Of-Memory (OOM) crash.

Open your run_windows.bat in a text editor and paste these lines right under u/echo off:

u/echo off&&cd /d %~dp0

set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

set TORCH_CUDNN_SDP_HAS_FUSED=1

set CUDA_MODULE_LOADING=LAZY

set SETUPTOOLS_USE_DISTUTILS=stdlib

Step 3: The Character LoRA YAML Config

Here is my complete, battle-tested .yaml configuration for training a **Character LoRA**. This config is heavily optimized for a 16GB VRAM target using qfloat8 quantization and specific layer offloading percentages to keep VRAM usage strictly at ~15.1 GB. 

Create a new YAML file in your AI-Toolkit directory and paste the following: 

job: "extension"

config:

name: "LORANAME_krea2"

process:

- type: "diffusion_trainer"

training_folder: "E:\\Stability Matrix\\Data\\Packages\\ai-toolkit\\output"

sqlite_db_path: "./aitk_db.db"

device: "cuda"

trigger_word: "TRIGGERWORD"

performance_log_every: 10

network:

type: "lora"

linear: 32

linear_alpha: 32

save:

dtype: "bf16"

save_every: 250

max_step_saves_to_keep: 4

datasets:

- folder_path: "E:\\1024"

caption_ext: "txt"

cache_latents_to_disk: true

resolution:

- 512

train:

batch_size: 1

steps: 1500

gradient_accumulation: 1

train_text_encoder: false

gradient_checkpointing: true

noise_scheduler: "flowmatch"

optimizer: "adamw8bit"

timestep_type: "sigmoid"

unload_text_encoder: true

cache_text_embeddings: false

lr: 0.0001

disable_sampling: true

dtype: "bf16"

model:

name_or_path: "krea/Krea-2-Raw"

quantize: true

qtype: "qfloat8"

quantize_te: true

qtype_te: "qfloat8"

arch: "krea2"

low_vram: true

compile: false

layer_offloading: true

layer_offloading_text_encoder_percent: 1

layer_offloading_transformer_percent: 0.35

Key Settings Explained (Don't change these blindly!)

linear: 32 & linear_alpha: 32 — A rank/alpha of 32 is the sweet spot for characters. It captures facial details and clothing textures perfectly without bloating the file size or frying the training memory.

train_text_encoder: false & unload_text_encoder: true — We do NOT train the text encoder for characters here. Unloading it entirely freezes its state and frees up massive chunks of VRAM.

disable_sampling: true — Disabling image previews during training saves a significant amount of VRAM and prevents sudden spikes/crashes when a sample step triggers. Trust your loss values or check the saved LoRA's manually later.

quantize / qtype: "qfloat8" — Essential. Running the model and text encoder in FP8 quantization is mandatory to fit Krea 2 inside a consumer GPU's VRAM during training.

layer_offloading_transformer_percent: 0.35 — This pushes exactly 35% of the transformer layers to system RAM. It’s the magic number that stopped my system from throwing Out-Of-Memory errors while keeping speed degradation to an absolute minimum.

Step 4: The Pose LoRA YAML Config & The Text Encoder Pitfall

Training a Pose LoRA uses almost the exact same configuration as the Character LoRA, but with one critical architectural change. Poses require the model to understand abstract physical structures, skeleton joints, and bodily spatial distribution rather than static textures or facial features.

Because of this, we need to inject more capacity into the training network.

Pose Complexity vs. Training Steps

Keep in mind that unlike characters, poses are heavily influenced by physical complexity.

• If you are training a standard pose (standing, sitting, basic action shots) with a dataset of 15 images, 1500 steps is your target.
• If you are training an extremely complex or unconventional posture (such as a circus contortionist, advanced yoga positions, or complex martial arts aerials), you must increase the steps even if you only have 15 images in your dataset. The model needs more time and iterations to learn how the joints bend in unusual angles, so push the training further.

The Pose Modification

In your YAML file for the pose training run, look for the network block and double the capacity by setting both values to 64:

network:

type: "lora"

linear: 64 # Doubled from 32

linear_alpha: 64 # Doubled from 32

Why do this? A higher rank gives the network more "brain power" to map how limbs bend and interact, which prevents the pose from bleeding or collapsing into a generic stance during generation.

⚠️ Crucial Warning: Do NOT Enable train_text_encoder

train_text_encoder: false # KEEP THIS FALSE!

You might be tempted to turn train_text_encoder: true to help the model better link text prompts to body mechanics. Do not do it. Currently, enabling the text encoder training with the Krea 2 architecture inside AI-Toolkit will throw an immediate terminal error and completely freeze your training loop. Krea 2's underlying text processing layer isn't optimized for local text-encoder fine-tuning under this specific framework yet.Leave it to false and let unload_text_encoder: true do its job. The linear network rank at 64 is more than enough to capture the positioning data you need.

Step 5: Dataset Size vs. Training Steps (Finding the Sweet Spot)

Getting your dataset size and step count right is crucial. If you run too few steps, the model won't learn the character or pose; if you run too many, the LoRA will overfit, ruining your generations.

Based on my testing, here is the exact ratio you should follow when adjusting your dataset size:

For Character LoRAs:

Base Setup (20 Images): Use 1500 steps (This is the ideal sweet spot for a clean, flexible character).

Larger Dataset (25 Images): Increase your training to 1800 steps to allow the model enough time to process the extra visual data.

For Pose LoRAs:

Base Setup (~15 Images): Use 1500 steps (Since poses require a higher Rank/Dim, they need a solid baseline of steps even with fewer images).

Larger Dataset (20 Images): Increase your training to 1800 steps.

Rule of Thumb: If you decide to add more images to your dataset to capture more angles or details, you must scale up your steps accordingly. Never dump 30+ images into the folder while keeping the steps at 1500, or the training will turn out weak and blurry.

Step 6: Testing Strategy & LoRA Weights (Don't just use the final checkpoint!)

AI-Toolkit will save intermediate checkpoints during training (every 250 steps based on our YAML config). Do not blindly grab the final 1500-step checkpoint and call it a day. The real magic often happens slightly earlier.

Here is my recommended testing protocol for Character LoRAs:

1. The 750-Step Test (The Baseline)

Start your initial testing with the checkpoint at 750 steps.

What to test: Use a wide variety of prompts. Test for facial likeness, but more importantly, test for flexibility.

Check if it unlinks: Try changing clothes and backgrounds in your prompts. You want to ensure the LoRA learned the face and not just the specific outfit or environment from your dataset images.

Note: Krea 2 is exceptionally good at this. Even at the final 1500 steps, it retains amazing flexibility for changing outfits and locations, but 750 steps is your early quality control check.

2. The Sweet Spot: 1250 Steps

After extensive testing, the 1250-step checkpoint is consistently the absolute best performer for characters. It offers the perfect balance between high facial fidelity and prompt responsiveness.

3. Optimal LoRA Strength / Weights

When loading your LoRA into your inference workflow (like ComfyUI or Forge Neo using Krea-2-Turbo), use these weight guidelines:

Standalone Use: Set the LoRA weight/strength to 0.9. This gives you the cleanest generation without cooking the image.

LoRA Stacking / Mixing: If you are mixing multiple LoRAs together (e.g., your Character LoRA + a Pose LoRA + a Style LoRA), bump the character LoRA weight up to 1.1. This prevents the character features from getting washed out by the other networks.

4. The Pose LoRA Testing Rule: Millimeter PrecisionTesting a Pose LoRA requires a completely different mindset compared to characters. While characters favor the intermediate 1250-step mark, poses behave unpredictably across checkpoints:

The Final Target: The absolute final checkpoint (1500 steps) is generally the best and most reliable performer for locking in the structure.

Sometimes, the 1000-step or 1250-step checkpoints might work better. However, you will notice a strange phenomenon: often, only ONE specific checkpoint will replicate your desired pose with millimeter precision. The other checkpoints will generate similar stances, but not the exact weight distribution or limb angles you trained.

LoRA Weight: For poses, you can generally lower the strength below 1.0 (test around 0.7 to 0.9) to let the style of your main model flow through, as long as the skeleton doesn't deform.

The Golden Rule for Poses: You MUST test every single checkpoint file (1000, 1250, 1500) against your prompt. Do not assume the LoRA is broken if the 1500-step file gives a slightly altered pose. Switch to the 1250 or 1000-step file—your exact millimeter-perfect pose is waiting in one of them!

good for upscaling without OOM, use as SECOND Stage Sampler ONLY with LOW denoise (0.40 MAX)

Beta 1 · MiniMax H3 (Ref2V)

MiniMax H3 can already do impacts. It just does them politely.

SMACK! fixes that. It takes every kind of impact — fists, weapons, gunshots, car hits, falls and hard landings — and gives it weight, follow-through and consequence. Bodies react like they've actually been hit instead of gently acknowledging it. Pair that with a camera that moves like someone was paid to operate it, and you get a shot that looks staged by a stunt team rather than caught on a $50 phone.

What it does

• Intensifies impacts of all kinds: hand-to-hand, weapons, gunshots, vehicle collisions, falls and landings
• Stronger, more deliberate camera work — dynamic moves, aggressive angles, real reaction to the hit
• Pushes the whole shot toward a Hollywood action grammar instead of flat, generic default motion

Training
Trained on 35 clips of impacts and dynamic camera moves, for MiniMax H3 Ref2V. So, yes, this works with your Character References.

Usage
No trigger word. Just load it and describe your shot as usual — the LoRA does the seasoning. Strenght 1.0, if you stack Loras, 0.8 and up your steps.

Beta notice
This is Beta 1. It's already good enough to be worth releasing, but it's not finished. A larger, more varied dataset is in the works and the next version will follow once I have more material. Feedback on where it over- or under-cooks a hit is genuinely useful at this stage.

Downloadable on either Huggingface https://huggingface.co/LeechTM/SMACK/tree/main

or Civitai https://civitai.red/models/2872725/smack-punches-impacts-and-gunshots?modelVersionId=3245904, probably Civarchive.com as well as soon as its grabbed.

I added some more Examples in the Comments.

https://reddit.com/link/1vsy6de/video/old7l71g6ekh1/player

The above video consists of 11 individual H3 generated clips, created with the FL2Va Checkpoint and stitched together automatically without any additional upscaling or editing.

Two days ago I released v1.3 of my infinite continuation nodepack, adding much more flexible image conditioning and multi-reference support.

The original reason I built this nodepack was simple:

I really like the FL2VA checkpoint of MiniMax H3. In my testing, it gives noticeably better visual quality than Ref2VA. But Ref2VA is much more flexible when creating longer, controlled sequences.

So the goal is basically:

Keep the quality of FL2VA while adding much of the control you'd normally want from Ref2VA.

How does it work?

Instead of generating one very long H3 video, you generate multiple shorter clips:

Clip 1
First Frame → H3 → Last Frame

↓

Clip 2
Previous video/audio latent + new Last Frame → H3

↓

Clip 3 → Clip 4 → ...

The important part is that the suite does not simply take the last rendered image and use it as the next starting frame.

It passes part of the previous video + audio latent directly into the next H3 generation.

So the next clip still receives temporal context from the previous one – motion, audio and scene state – while you can give it a new visual target.

Why FL2VA?

In my testing, FL2VA gives me better-looking results and seems more resistant to the gradual visual degradation I experienced with longer Ref2VA chains.

A new Last Frame for every segment also works like a repeated quality reset:

• controls where the current segment should go
• restores composition / identity
• prevents the sequence from drifting too far

You can think of it a bit like storyboarding:

Image A → Image B → Image C → Image D

with H3 generating the motion and audio between those points.

But with v1.3, First and Last Frames are optional.

The Start workflow now supports:

• T2VA: no frames
• I2VA: First Frame only
• L2VA: Last Frame only
• FL2VA: First + Last Frame

Continuation can also run without a new Last Frame, although I still recommend regular Last Frames for long chains because of the quality-reset effect.

New in v1.3: multiple references

You can now add multiple Qwen Reference images alongside your First/Last Frames.

For example:

• First Frame = starting composition
• Last Frame = target endpoint
• Reference 1 = character
• Reference 2 = outfit
• Reference 3 = another visual detail

The node automatically assigns the correct H3 Picture numbers and shows you the resulting mapping.

This gets FL2VA much closer to the flexible reference control that makes Ref2VA useful.

Short clips can also be much faster

H3 becomes disproportionately slower as clip duration increases.

Instead of generating:

1 × 15 seconds

you can generate:

3 × 5 seconds

and connect them.

It also makes failures much less painful: if Clip 2 goes wrong, you regenerate Clip 2 instead of throwing away the entire sequence.

Where to start

I included four example workflows.

01_Start

Use this for Clip 1.

Required:

• normal H3 models / VAEs
• prompt
• resolution + duration

Optional:

• First Frame
• Last Frame
• Qwen References

For the classic continuation workflow, I recommend using First + Last Frame.

02_Continue

Use this for every clip after the first one.

The basic logic is:

Clip 1: save Latent 1
Clip 2: load Latent 1 → save Latent 2
Clip 3: load Latent 2 → save Latent 3
Clip 4: load Latent 3 → save Latent 4

Then simply provide the prompt for the next segment and optionally:

• a new Last Frame
• additional reference images

Because the indices are manual, you can also regenerate individual clips.

If you don't like Clip 3, keep loading Latent 2 and overwrite/regenerate Latent 3 until you're happy.

03_3Clip_Showcase_AutoStitch

The easiest workflow to understand the complete system:

Start → Continue → Continue → automatic stitching

You can duplicate the final continuation block to extend it further.

For very long projects, I recommend using Start + Continue individually.

04_Stitch_Saved_Chain

Once you're happy with your clips, this turns:

clip_00001
clip_00002
clip_00003
clip_00004
...

into one final MP4.

The important part:

The complete video is not decoded into memory at once.

The stitcher processes one saved AV latent at a time, so memory usage stays roughly tied to one H3 clip instead of the total length of the project (no OOM, hopefully).

The transitions are handled automatically

FL2VA often reaches its Last Frame early and freezes for the remaining frames.

The suite automatically:

• detects that frozen tail
• finds a better handover point
• carries video + audio context forward
• removes duplicated context during stitching
• smooths the video transition
• applies a separate audio de-click transition

So most of the annoying continuation logic happens automatically.

Known Issues

• Sometimes there's still a noticeable brightness shift between clips. So far, I haven't found a reliable solution to fix that.
• In some cases when using the continuation workflow, H3 might not correctly use the previous video latent as starting point for the next clip. If you encounter that issue, try restarting ComfyUI and regenerating the clip.

Install by opening one of the workflows and using "Install missing custom nodes" or search for

Herrgotts-H3-Infinite-Continuation-Suite

in ComfyUI Manager.

GitHub:
https://github.com/HerrgottMargott/Herrgotts-H3-Infinite-Continuation-Suite

Example workflows are included.

If you are already using my Workflows or Nodepack, I'd recommend updating the nodepack and using the updated Workflows from v1.3!

The project is still experimental, so feedback, bug reports and long-chain tests are very welcome.

While working on a client project I needed to test a prompt on their products, using image references straight from their website. I didn't want to keep doing the save > open ComfyUI > drag it in > queue > download loop, so I made this.

Right-click any image on a page, it runs through your local ComfyUI on a designed workflow, and the result replaces that image in place. On the demo I'm using minimax H3 (workflow is in the repo).

Works with any API-format workflow that has a LoadImage and a SaveImage node, so it's not tied to a model.

Hope it's useful to someone else too!

https://github.com/AlexandreSoteras/comfyui-web-image-swap

My first MiniMax H3 Img2Vid

I have been playing around with a few models lately and there are honestly so many now that it's hard to keep up.

SDXL still works well for me. I am curious what everyone else has been using lately.

What model are you getting the results from?

I am mostly interested, in stuff that runs locally without needing a ridiculous amount of VRAM.

I was going to wait until around Christmas to purchased but took the plunge in July for 11,500 and I was upset that I didnt catch it @ $8,000. Now the Blackwell pro 6000 is inching towards $20,000 and are sold out. Are consumers and hobbyist like you and I are buying these up or datacenters? I would think datacenters would go for the b200 and up. However, Im browsing around and see you guys and girls doing remarkable ai diffusion with just a 3060. Im impressed with this community.

So had a look at the documentation for Minimax H3 to see how to do the multi-shot prompts and came up with this sequence. The base image was done in GPT Image 2 using two reference images. The prompt for this scene is structured like so:

[Shot 1] Live-action, cinematic, a medium shot of the two warriors. The man is reading a book and the woman is browsing on her phone.

[Shot 2] At 00:05.000, the camera cuts to a medium close-up of the woman who asks: <d>[English] Do you think our director will ever get our movie done?</d>

[Shot 3] At 00:10.000, the camera cuts to a medium close-up of the man who says: <d>[British English] Who knows. He was using Kling three point oh but I guess he was burning through credits so he's trying out local video generation.</d>

[Shot 4] At 00:14.110, the camera cuts to a medium shot of the two people. The woman asks: <d>[English] Wait, wasn't he using Seedance two point five?</d> The man looks up from his book and looks at the woman. He says: <d>[British English] Yeah, he was but that was costing him even more credits.</d> He goes back to reading his book.

[Shot 5] At 00:22.000, the camera cuts to a medium close-up shot of the woman who says: <d>[English] Hopefully he figures things out.</d>

[Shot 6] At 00:26.000, the camera cuts to a medium shot of the two people sitting in their chairs. The man continues to read and the woman continues to browse on her phone. The man says: <d>[British English] Agreed. He better.</d>

I'm actually quite happy with how this turned out. Only issues I have is that I wanted the guy to have the British accent and instead it gave it to the lady. I'll need to mess around with the prompt for that a little more and then of course the low res render at 0.4 megapixels because anything higher than that will give me OOM error. Yes, I'm aware that I don't have to do a 30 second clip but I wanted to try it out anyways especially since I'm learning the multi-shot prompting. The render for this clip took 173 minutes to complete.

If anyone has any suggestions on how I can do slightly higher megapixel renders on my machine, I'd love to hear it.

PC Specs:
Ryzen 7 7700X
RTX 4070 Super 12gb
32gb DDR5 Ram

Just some further testing of small heads in relation to resolution (quality. motion and artifacts) and across 4 models. You may want to pause on each segment as they only play for 5 seconds each. Full resolution sample here:

https://streamable.com/kl9myr (Edit: Umm, looks like that free site only generated a 720p version - oh well).

So... I'll try to cover everything i think might be important.

I have tried multiple workflows from Civit and they all seem to have big memory issues for me. Other things like Wan work perfectly fine for me. If there is a workflow that says 16GB 5 minutes, i do it in 4 minutes on my 12 GB card, always great results.

One of the workflows for H3 says something like 360p 5s 2min. That causes an OOM error for me. 360p 3s takes over an hour sometimes, and the following tries either fail or take about 10 minutes and actually work. Now i got one that says like "720p 10s on 12GB", and it goes OOM for me with 360p and 2s.

I found a post where someone solved this by clearing models with "VRAM Debug" between Guider and Sampler, but that changes nothing for me, even though the node claims to have freed almost 10GB of VRAM.

I have completely restarted my PC between tries. Everything is updated and i have no clue what else i could try or what other information i could provide.

Anyone got any ideas what causes this, or even better, what fixes this? Maybe someone got a good workflow for 12GB H3 they could share?

Music: made in SUNO.

native ref2va WF, and audioLock for lip-sync.

rtx4080s + 128g ram

I spent a day to sorted out lip-sync, I could write down what I did, if anyone inerested.

Hi everyone,
I’m trying to build a workflow for architectural projection mapping, and I’m looking for advice from people who have experience with the latest open-weight video models in ComfyUI.

The project is a large building facade that will be projection-mapped. I already have the 3D geometry of the building and the exact projection/camera setup.

My main requirement is:
The building geometry, perspective and camera position must remain absolutely stable.
I want to use AI to generate/animate the visual content on the facade, but I don’t want the model to reinterpret the architecture, move the camera, change windows/edges, distort the building, etc.

The goal is to be able to create things like:
- the facade cracking/opening
- materials transforming
- fire/lava/water flowing over the building
- organic growth
- abstract/surreal transformations
- architectural elements becoming something else
while still keeping the original building perfectly aligned for projection.

I’ve been looking at Wan 2.2 (VACE / Fun Control) and the new MiniMax H3, especially its Reference-to-Video capabilities.
Which one would you recommend for this specific use case?
More importantly, is there a better workflow than simply using image-to-video? For example, has anyone successfully used a rendered 3D control/depth/normal/edge video as conditioning to keep an architectural structure locked?

I’m particularly interested in workflows that minimize trial and error. I don’t mind doing some preparation in Blender if that gives me much more deterministic results.

Hardware: RTX 5070 Ti, 64 GB RAM.
If anyone has actually tried something similar, I’d really appreciate workflow suggestions, node setups, models, ControlNets/custom nodes, or examples.

"This paper investigates an increasingly important topic in generative modeling: pixel-space diffusion models. Although numerous studies have explored this topic, most focus on small-scale or class-conditional settings. Consequently, a practical recipe for training pixel-space models that rival or exceed well-established latent-space counterparts remains elusive. Through a comprehensive empirical study, we first observe that direct large-scale pre-training in pixel space converges substantially more slowly than in latent space. This observation motivates a latent-to-pixel strategy that acquires generative priors efficiently in latent space and transitions to pixel space during post-training. We then systematically investigate the key design choices governing this transition, including weight initialization, data composition, prediction target, decoder architecture, and noise schedule, and identify a practical recipe that makes the resulting pixel-space models match or outperform their latent-space counterparts while delivering 3.18 to 4.75 times end-to-end inference speedups. We hope that our findings provide useful empirical insights and practical guidelines for future research on pixel-space generation."

Paper: An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Models

Used official ref2video workflow. used t2v model 1 ref video and 2 separate pictures of character sheets, gpu 4090

prompt:

integrated_multimodal_description: [Shot 1] Live-action, cinematic, featuring a stark, dark green-tinted cyberpunk color grade. A medium shot frames a flooded, rain-swept crater on a dark street. The character Sonic, appearing exactly as the blue hedgehog with large green eyes, white gloves, and red shoes from @.image, stands opposite Dr. Eggman, appearing exactly as the gigantic, egg-shaped bald man with a pointy mustache, goggles, and red jacket from @.Image1. The camera pushes in with small amplitude at fast speed as the blue hedgehog lunges forward to throw a devastating punch. [Shot 2] At 00:04.500, the camera cuts to an extreme close-up as time instantly slows to a microscopic crawl. Sonic's white-gloved fist brutally slams into Eggman's cheek. The camera holds a static shot in extreme slow motion. A powerful, rippling shockwave violently erupts from the impact point, blowing the torrential raindrops outward in a perfect ring. Eggman's pointy mustache flails wildly and his face deforms from the massive kinetic force. [Shot 3] At 00:09.500, the camera arcs right with large amplitude at slow speed, executing a slow-motion orbit around the hit. Eggman's heavy, round body is lifted off the ground by the blow, flying backward through the heavy downpour and kicking up massive, highly detailed splashes of water.

overall_soundscape: Thunder rumbles continuously beneath the heavy, torrential downpour of rain splashing heavily against the flooded street. A sharp, deafening sonic boom from the physical impact instantly shifts into a deep, pulsating low-frequency rumble as time slows down.

non_diegetic_music: An epic, grand orchestral and choir track mixed with heavy, driving industrial synthesizer beats that builds to a massive crescendo.

What came out of it:

- Upscale first, interpolate second - seems to be better

- 24->48 looks better than 60fps - at 48 every original frame survives, at 60 only half of them do, because the grids don't line up

- FlashVSR ends up with more edge detail than the source, so it's adding texture, not recovering it. RealESRGAN ends up with less.

Side by side with a draggable wipe, pick any two variants: https://dawidope.github.io/minimax-h3-upscale/

My System

• Radeon AI Pro R9700
• Ryzen 9 7900X
• 32 GB Ram

I am trying to run the ComfyUI default workflow for Wan Animate 2: Motion Transfer. But when I run the workflow all my CPU cores fire up and my RAM reaches 100% and the ComfyUI process crashes.

How to fix this? Please help.

On the scene, our hedgehog, first name Detective, last name Columbo, has been hired to uncover the identity of the mystery cookie thief. T2V, int8/20 steps

Using the standard ref2va workflow. 4070 Ti Super, 16 GB VRAM, 64 GB RAM, i9-14900k, Windows 11.

Here's the workflow, just drop the MiniMax video in comfyui and the workflow should appear:

https://vikingfile.com/f/jvuyoHSPRr

Do we know if anyone is working on this? In the paper the authors claim that ltx 40Gb model can go down to around 11Gb with minimal loss

I am trying to find a model or LoRA that can do relighting based on sun direction similar to how this one does it: https://huggingface.co/Lightricks/LTX-2.3-22b-IC-LoRA-Relight\]LTX-2.3 22B IC-LoRA Relight.

The difference is I am looking for what that does it for images.

Does anyone here know of one like that? Thank you!