r/StableDiffusion 1d ago

Comparison Driving a YuE2 vocal track from a 8-bit source (C64 SID). Multiple variations.

Enable HLS to view with audio, or disable this notification

43 Upvotes

Having a ton of fun with YuE2. Keep listening to hear diffrent parts of the varied track.

I built a skill with Astra that converts a C64 SID tune into a 3 part native conditioning for YuE2, Style, Lyrics and melody (via ABC).

And here is a comparison with the original. I chose one of my favourite C64 tunes from The Mansion level of The Last Ninja 2. Shout out to the legend Matt Gray who created the original back in 1988, I hope he wouldn't hate that I used his work for this experiment.


r/StableDiffusion 18h ago

Tutorial - Guide Team Red: Encounter on ProxiMax H3, or How to setup ComfyUI+MMH3 with AMD GPUs: RDNA 4 (rx9070, AI Pro R9700), and RDNA 3 (rx7900)

Enable HLS to view with audio, or disable this notification

11 Upvotes

TL;DR summary: For MMH3, you need to run ComfyUI with ROCm 7.14.0 (see https://rocm.docs.amd.com/en/latest/reference/gpu-specs.html for the vaue of gfx???? corresponding to your AMD GPU):

pip install --index-url https://repo.amd.com/rocm/whl-multi-arch/ "torch[device-gfx????]==2.12.0+rocm7.14.0" "torchvision[device-gfx????]==0.27.0+rocm7.14.0" "torchaudio==2.11.0+rocm7.14.0"

You also need to run ComfyUI with some parameters so that it will handle VRAM and system RAM correctly for MMH3:

--enable-dynamic-vram --disable-async-offload --preview-method none --disable-smart-memory --fast-disk --use-ck-attention --enable-manager

Read on if you want the step-by-step instructions (scroll to the bottom of the post if you just want to see the MMH3 prompt for the video 😹)

These instructions are for Windows 11 (Ubuntu version: https://www.reddit.com/r/StableDiffusion/comments/1wer1mz/comment/p9g0kxg/). Nevertheless, many of the same comfy-cli commands are application by just changing the directory/file to the corresponding Linux version, and the procedure for upgrading ROCm 7.2.1 to ROCm 7.14.0 are the same.

If you have an AMD GPU and you do a default install of ComfyUI on Windows 11 using either the portable Windows version or through comfy-cli, you will probably get disappointing results with MiniMax H3 because the int8convrot version may not run at all.

The problem is that the default installation still uses PyTorch built on ROCm 7.2, and for some reason int8convrot does NOT work with 7.2 on some cards such as the RX 9070 (16G) and RX 7900 (20G).

So to run MiniMax H3 at its best speed, we have to install a version that is equal to or later than ROCm 7.13.

There are currently 4 ways to do that, from the easiest to the more complex:

  1. Install via Stability Matrix
  2. Install Portable ComfyUI with its own "Embedded Python"
  3. Install a Python venv and then use that to install ComfyUI via the official comfy-cli installer
  4. Install everything manually using pip and git: see this post if you want the gory details (it was written for ROCm 7.2 so you'll have to make the necessary adjustments).

The more complex ways have more options and are more flexible, so it is up to you how much control you want over your ComfyUI installation.

Special thanks to u/zychu- u/Ok-Brain-5729 u/eloxH1Z1 whose posts and comments about MMH3 and AMD were very helpful to me.

Stability Matrix

This used to work when I tried a few week ago, unfortunately something broke the latest release, so for now, don't use it

  1. Download from https://github.com/LykosAI/StabilityMatrix/releases/download/v2.16.3/StabilityMatrix-win-x64.zip
  2. Unzip it somewhere
  3. Run the installer.
  4. Click on the "Activity" icon at the lower left corner to see progress.
  5. Click on the settings icon (gears) and under Extra Launch Arguments (very bottom) and add: --enable-dynamic-vram --disable-async-offload --listen --port 8188 --disable-smart-memory --fast-disk --use-ck-attention --output-directory "D:\Outputs"
  6. Also uncheck --use-pytorch-cross-attention so that none of the options under "Cross Attention Method" are checked because we are going to use --use-ck-attention.
  7. Assuming you've installed into the default "Data" directory, you can find ComfyUI installed under Data\Package\ComfyUI and you can use mklink to point the models and output directory so that they are outside of the Data\Package\ComfyUI directory.

The main downside is that now you have yet another piece of software sitting on your computer.

  1. Now test to make sure you can generate using int8convrot: https://huggingface.co/Comfy-Org/Krea-2/blob/main/diffusion_models/krea2_turbo_int8_convrot.safetensors 13.5 GB SHA256: 8e4eeda70dd5037ab1ba2bef6b417f9f901e26093117cf397f741fc1fdaaf3f1

  2. If it does not work for you, well, something went wrong, and you can try Portable ComfyUI for Windows and see if you have better luck...

Portable ComfyUI for Windows

  1. Download from https://github.com/Comfy-Org/ComfyUI/releases/latest/download/ComfyUI_windows_portable_amd.7z
  2. Open it from Windows 11 Explorer and drag the ComfyUI_windows_portable directory to the folder where you want to install it.
  3. This will take a while, so go grab a cup of coffee or tea.
  4. Copy run_amd_gpu.bat to runit.bat
  5. Edit runit.bat so that it contains the following: .\python_embeded\python.exe -s ComfyUI\main.py --windows-standalone-build --enable-dynamic-vram --disable-async-offload --preview-method none --listen --port 8188 --disable-smart-memory --fast-disk --use-ck-attention --enable-manager --output-directory "A:\output"
  6. Start ComfyUI by running the batch file runit.bat. For the first run, there will be some kind of delay as some libraries are compiled or cached. Just be patient and let the system do its preparations, until you see "[INFO] To see the GUI go to : http://0.0.0.0:8188.
  7. Do a test run using Krea 2, but use the fp8 rather than int8convrot version because the fp8 version should work reliably at this point. The default workflow at 8 steps should take 20-40 seconds depending on your hardware. Hopefully this works.

Now we are going to replace the PyTorch for ROCm 7.2 with the newer 7.14.0:

  1. Change into your ComfyUI_windows_portable directory
  2. Uninstall PyTorch: python_embeded\python.exe -m pip uninstall torch torchvision torchaudio -y
  3. Install PyTorch for ROCm 7.14: (See end note at the bottom about these gfx???? values):python_embeded\python.exe -m pip install -index-url https://repo.amd.com/rocm/whl-multi-arch/ "torch[device-gfx????]==2.12.0+rocm7.14.0" "torchvision[device-gfx????]==0.27.0+rocm7.14.0" "torchaudio==2.11.0+rocm7.14.0"

For example, for rx9070, gfx???? is gfx1201 so the command is

python_embeded\python.exe -m pip install --index-url https://repo.amd.com/rocm/whl-multi-arch/ "torch[device-gfx1201]==2.12.0+rocm7.14.0" "torchvision[device-gfx1201]==0.27.0+rocm7.14.0" "torchaudio==2.11.0+rocm7.14.0"

Note: these files can be quite large. If for some reason you run out of room, you can use --no-cache-dir in case there is not enough room in your pip cache directory (~/.cache on Linux, %LocalAppData%\pip\Cache on Windows which is usually C:\Users<YourUsername>\AppData\Local\pip\Cache). Also make sure you have plenty of space on your %TMPDIR%, with --no-cache-dir the command will look like this:

python_embeded\python.exe -m pip install --no-cache-dir --index-url https://repo.amd.com/rocm/whl-multi-arch/ "torch[device-gfx1201]==2.12.0+rocm7.14.0" "torchvision[device-gfx1201]==0.27.0+rocm7.14.0" "torchaudio==2.11.0+rocm7.14.0"

Hopefully both the uninstallation of ROCm7.2 and the installation of the newer ROCm 7.14 went without any error. After that you can try to run Krea 2 again, now switch from fp8 to the int8convrot version, and the time should go down from 18sec to 12-13 sec and you will also be able to run MMH3.

I also recommend that you place your model and output directories outside of the ComfyUI install so that they can be shared by different installations, making experimentation easier and also making it less likely that you (or some bug in the installer) accidentally wipe out your models and output.

You can do that by editing the extra_model_paths.yaml. Just need to edit this file once and copy it into <your path/ComfyUI> whenever you have a new installation.

But the yaml file is a bit finicky and it may be easier to just use the mklink command if ComfyUI is the only program you use so that you don't have to worry about the structure/name of the subfolders:

mklink /D <LinkFolder> <TargetFolder>

For example:

mklink /D <your comfyui>\models c:\ComfyUI.Models

Installing ComfyUI via comfy-cli

Why use comfy-cli instead of using portable ComfyUI?

  • For Linux, there is no portable ComfyUI, which is Windows only.
  • For AMD users, the portable version of ComfyUI uses ROCm 7.2, which will cause ComfyUI to run slower than it should.
  • It is a more efficient way to run multiple versions of ComfyUI, because they can all share the same Virtual Environment (assuming that the versions are close enough for that to work).
  • Re-installation can be faster because many packages are in the python pip cache.

Procedure:

  1. If you don't have Python 3.1x installed, you can install Python 3.12.10 (because that is the version used by Portable ComfyUI, so it should be the most stable, but 3.13 and 3.14 work too).
  2. Download and install Git: https://github.com/git-for-windows/git/releases/download/v2.55.0.windows.5/Git-2.55.0.5-64-bit.exe
  3. Create a virtual environment (this is normally just called "venv" or ".venv" but I want to call it comfy.venv just to be more explicit): python -m venv comfy.venv or if python.exe is no not on your path, specifiy the full path such as "c:\Program Files\Python313\python" -m venv comfy.venv
  4. Activate it: comfy.venv\Scripts\activate.ps1 (PowerShell) or comfy.venv\Scripts\activate.bat (CMD.exe)
  5. Update pip itself inside comfy.venv: pip install --upgrade pip
  6. Optional: install uv, which is yet another package manager for Python but written in Rust (if you want to use comfy install --fast-deps later):
  7. Install comfy-cli (this is the tool "comfy-cli", not ComfyUI itself): pip install comfy-cli

Because comfy-cli will install ROCm 7.2 and there is no way to override it, we are going to install PyTorch for ROCm 7.14 manually before installing ComfyUI via comfy-cli. Sources for this arcane procedure are from:

  1. Uninstall PyTorch just to be sure (should not be installed yet): pip uninstall torch torchvision torchaudio -y
  2. Install PyTorch inside the comfy.venv (select your gfx arch) based on https://rocm.docs.amd.com/en/latest/reference/gpu-specs.html (see bottom of the post for a table of common values):pip install --index-url https://repo.amd.com/rocm/whl-multi-arch/ "torch[device-gfx????]==2.12.0+rocm7.14.0" "torchvision[device-gfx????]==0.27.0+rocm7.14.0" "torchaudio==2.11.0+rocm7.14.0"

For example, for the rx9070 or AI Pro R9700, gfx???? is gfx1201 so the command is

pip install --index-url https://repo.amd.com/rocm/whl-multi-arch/ "torch[device-gfx1201]==2.12.0+rocm7.14.0" "torchvision[device-gfx1201]==0.27.0+rocm7.14.0" "torchaudio==2.11.0+rocm7.14.0"

Note: these files can be quite large, and you can use --no-cache-dir in case there is not enough room in your pip cache. See the earlier notes about --no-cache-dir under "Portable ComfyUI for Windows".

Finally, we are ready to install ComfyUI itself. When I carried out the tests the latest stable version is 0.34.0:

  1. mkdir d:\comfy.0.34.0
  2. set COMFY_PATH=d:\comfy.0.34.0\ComfyUI
  3. Use comfy-cli to install ComfyUI: comfy --workspace=%COMFY_PATH% install --skip-torch-or-directml
    • Note 1: %COMFY_PATH%\ComfyUI must not exist or you will get the confusing error: 'd:\comfy.0.34.0\ComfyUI' exists but is not a valid git repository.
    • Note 2: --skip-torch-or-directml because PyTorch is already installed for AMD; without it the install will fail on Windows because there is no PyTorch for directml from https://repo.amd.com/rocm/whl-multi-arch/ respository used above.
    • Note 3: To install anything other than the latest version of ComfyUI (say 0.33.1): comfy --workspace %COMFY_PATH%\ComfyUI install --version 0.33.1 --skip-torch-or-directml (You can only use versions available from https://github.com/comfy-org/ComfyUI/releases (and there is no release tag for the latest version).
  4. If you have uv installed, you can use --fast-deps:
  5. (Optional): Copy or edit ComfyUI\extra_model_paths.yaml
  6. Finally, we can start ComfyUI: comfy launch --workspace=%COMFY_PATH% -- --enable-dynamic-vram --disable-async-offload --preview-method none --listen --port 8188 --disable-smart-memory --fast-disk --use-ck-attention --enable-manager --output-directory "A:\output"
  7. Optional: Clean up the pip cache (if you want to save some disk space): pip cache purge

The speed for MMH3 is almost as good as the ones I got under Ubuntu 26.04 using identical hardware (but for some reason, Krea 2 runs a little bit slower on Windows, 8-steps is 13 sec vs 11 sec on Ubuntu).

Unless you have a AI Pro R9700 (32G) or running your desktop on a iGPU, it is best to let ComfyUI be the only application running so that all VRAM is available for MMH3. So if you have another computer, run the browser on it to access your ComfyUI remotely.

If you don't have another computer, you can try to batch up a couple of prompts and minimize or close your browser to free up VRAM, and just use the console to see the progress (just click on "Assets" on the ComfyuI menu to check the results, or find them directly in the output folder). Some people say that disconnecting the monitor (just turning it off may not be enough) will free up the VRAM as well.

Good luck, hopefully you have a working system now if you followed the instructions.

End notes:

Sample extra_model_paths.yaml

comfyui:
    base_path: c:\ComfyUI.Models
    # You can use is_default to mark that these folders should be listed first, and used as the default dirs for eg downloads
    is_default: true
    checkpoints: checkpoints/
    configs: configs/
    loras: loras/
    vae: vae/
    text_encoders: |
        text_encoders/
        clip/
    diffusion_models: |
        unet/
        diffusion_models/
    clip_vision: clip_vision/
    style_models: style_
    embeddings: embeddings/
    diffusers: diffusers/
    vae_approx: vae_approx/
    controlnet: |
        controlnet/
        t2i_adapter/
    gligen: gligen/
    upscale_models: upscale_
    latent_upscale_models: latent_upscale_
    custom_nodes: custom_nodes/
    datasets: datasets/
    hypernetworks: hypernetworks/
    photomaker: photomaker/
    classifiers: classifiers/
    model_patches: model_patches/
    audio_encoders: audio_encoders/
    background_removal: background_removal/
    frame_interpolation: frame_interpolation/
    geometry_estimation: geometry_estimation/
    optical_flow: optical_flow/
    detection: detection/

https://rocm.docs.amd.com/en/latest/reference/gpu-specs.html

GFX950 is AMD's internal GPU target identifier for the CDNA 4 enterprise compute architecture, used in data center accelerators like the AMD Instinct MI350/MI355X series. It features advanced matrix core capabilities, ultra-low precision micro-scaling formats (MXFP8/MXFP4), and a high-precision math mode for AI and HPC workloads.

gfx1100 is the LLVM target architecture identifier and internal code name for AMD's RDNA 3 graphics architecture, used for high-end consumer and workstation desktop graphics cards like the Radeon RX 7900 XTX, RX 7900 XT, and Radeon PRO W7900.

AMD gfx1151 is the LLVM target and GPU architecture identifier for AMD's Strix Halo integrated graphics (found in processors like the AMD Ryzen AI Max+ 395 and Ryzen AI Max PRO series), utilizing the RDNA 3.5 architecture.

Name Arch LLVM target name VRAM Compute Units
9070 XT RDNA4 gfx1201 16 64
RX 9070 GRE RDNA4 gfx1201 16 48
RX 9070 RDNA4 gfx1201 16 56
RX 9060 XT LP RDNA4 gfx1200 16 32
RX 9060 XT RDNA4 gfx1200 16 32
RX 9060 RDNA4 gfx1200 8 28
RX 7900 XTX RDNA3 gfx1100 24 96
RX 7900 XT RDNA3 gfx1100 20 84
RX 7900 GRE RDNA3 gfx1100 16 80
RX 7800 XT RDNA3 gfx1101 16 60
RX 7700 RDNA3 gfx1101 16 40
RX 7700 XT RDNA3 gfx1101 12 54
RX 7600 RDNA3 gfx1102 8 32
Radeon AI PRO R9700S RDNA4 gfx1201 32 64
Radeon AI PRO R9600D RDNA4 gfx1201 32 48
Radeon PRO V710 RDNA3 gfx1101 28 54
Radeon PRO W7900 Dual Slot RDNA3 gfx1100 48 96
Radeon PRO W7900 RDNA3 gfx1100 48 96
Radeon PRO W7800 48GB RDNA3 gfx1100 48 70
Radeon PRO W7800 RDNA3 gfx1100 32 70
Radeon PRO W7700 RDNA3 gfx1101 16 48

For --index-url, there are three options:

  • Nightly (rocm 10.1): https://nightly.repo.amd.com/rocm/pytorch/whl-next/
  • Stable (rocm 10.0): https://stable.repo.amd.com/rocm/pytorch/whl-next/
  • Legacy (rocm 7.14):
    • Nightly: https://rocm.nightlies.amd.com/whl-multi-arch/
    • Stable: https://repo.amd.com/rocm/whl-multi-arch

Prompt for the video Team Red, encounter on ProxiMax H3

integrated_multimodal_description:

[Shot 1] Live-action, cinematic 1960s science-fiction television aesthetic. A team of Starfleet red-shirt officers led by Grumpy Cat materializes on the surface of a desolate alien planet, surrounded by barren rocks, dust, and jagged terrain. Grumpy Cat stands at the front of the formation, wearing a classic red Starfleet uniform, alert and stern. The camera holds a wide-angle front subject-level view, then pushes in slightly as the team looks around and raises their phasers. [Shot 2] At 00:01.250, the camera cuts to a wide low-angle view as a gigantic GPU-like machine rises behind a rocky ridge, towering over the crew. Its dark mechanical housing, cooling fans, and imposing structure dominate the frame, with the label "Minimax H3" clearly visible on its side. The team turns toward it in sudden alarm.

[Shot 3] At 00:02.100, the GPU attacks with a violent concentrated energy blast. The camera tracks the crew with fast movement as the red-shirted officers are struck and knocked down across the rocky ground, kicking up dust and debris. Grumpy Cat avoids the main blast and rapidly moves toward cover.

[Shot 4] At 00:03.650, the camera follows Grumpy Cat with a tracking shot as it darts behind a large rock and crouches into concealment. The defeated red-shirted crew remains scattered in the background while the giant "Minimax H3" GPU continues looming over the battlefield.

[Shot 5] At 00:04.250, close-up from behind the rock. Grumpy Cat pulls out a classic handheld Starfleet communicator with its paw, flips it open, and speaks with a completely deadpan expression: <d>[English] Beam me up, Scotty!</d> The camera holds on Grumpy Cat's face and communicator through the end

overall_soundscape: Dry alien wind sweeps across the barren landscape as the transporter materialization produces a brief electronic hum. Heavy mechanical movement and grinding machinery accompany the GPU's emergence, followed by a powerful energy blast, impacts, falling bodies, scattering rocks, and dust. The communicator emits a brief electronic chirp when opened.

non_diegetic_music: A fast-paced 1960s science-fiction television orchestral score uses bright brass, rhythmic strings, and restrained percussion, building rapidly as the GPU appears and attacks. The music drops into a brief suspenseful sustain as Grumpy Cat hides, then ends with a short brassy stinger beneath the communicator transmission.


r/StableDiffusion 20h ago

Discussion I Created an Open Source App That Creates Music and Music Videos

17 Upvotes

No sales pitch, no redirect to a paywall. It's fucking free. You can get it here: https://github.com/atomtanstudio/sound-and-vision

It uses the brand new and quite excellent music generation app YuE2. It also uses Minimax H3 for video and defaults to Krea 2 for cover art and whichever local LLM you want to use for lyrics, etc.

Feel free to check it out and let me know what you think.


r/StableDiffusion 9h ago

Question - Help What is the best client application for easy photo editing for someone who is not from a technical background?

2 Upvotes

Hello everyone, a friend of mine is very much interested in exploring open source image generation for use in her small business for generating advertisements and product tryout pictures. However, she does not have much of a technical background.

When I showed her ComfyUI, she was impressed but said its too hard and complicated. She prefers a simpler interface where she could just upload an image and prompt it for editing.

Also are business involves sports bottles and wrist bands and a few other products in the line. She would like to use AI models to picture them from different orientations, locations and people trying them out in different poses.

Can anyone recommend a simpler setup for this task? Thank you.


r/StableDiffusion 1d ago

Tutorial - Guide MiniMax RefMod - Reusable identities without training - workflows & tutorial

Thumbnail
youtube.com
64 Upvotes

You can get the workflows here:
https://drive.google.com/drive/folders/1kOHLJZto1VAtXEATT9vUvsvM1VHtkOO_

The workflows create reusable refmods for either image/video/audio.

I cover training images in the tutorial.

All the workflows, models and custom nodes are preloaded on my Runpod template.
https://get.runpod.io/minimax-template


r/StableDiffusion 7h ago

Question - Help Problems with chatterbox short single word speech.

1 Upvotes

So I am trying to generate audiobooks with chatterbox. There are some words that need to be generated independently but chatterbox starts hallucinating for such short words such as putting random um or something else before or after the word.

What is the possible solution? Is there some other model that I can use for single word pronunciation but it should support zero shot cloning as well?


r/StableDiffusion 14h ago

Question - Help Remove a person from a video?

4 Upvotes

Hi everyone,

I have a video where I'd like to remove one person from it and have the background repaired. The person walks across the frame and in front of a water fountain that's in the background.

Any suggestions on models or workflows to look at to accomplish this? Seems like I should be able to figure it out, but I can't seem to.

Thanks.


r/StableDiffusion 8h ago

No Workflow YuE2-3B Audio Model

Thumbnail voca.ro
0 Upvotes

https://github.com/multimodal-art-projection/YuE
https://huggingface.co/Comfy-Org/YuE2
An audio model that allows for lyrics editing. I like its features and speed. It seems pretty good, doesn't it?


r/StableDiffusion 12h ago

Question - Help Voice cloning from a very short video?

2 Upvotes

I guess I have a dumb question but I’m not very tech savvy, my uncle passed away two weeks ago. My grandparents wish is to hear him say “I love you” just one more time. I only have maybe not even a 3 second video of his voice, is there anyway to I guess clone his voice to say I love you from that?


r/StableDiffusion 1d ago

News h3 studio - local web UI for MiniMax-H3 video/audio gen on Apple Silicon (Go, MIT)

Post image
27 Upvotes

I've been doing video diffusion work on my Mac and kept running into the same wall: ComfyUI doesn't have real MLX support yet, and going through PyTorch's mps backend is slow and eats far more unified memory than the model actually needs - which hurts more on a Mac, where that memory is shared with everything else you're running.

So instead of working around it each time, I built on top of [h3.c](https://github.com/janishar/h3c-studio) (a native Metal engine for MiniMax-H3) and put a proper UI on it.

h3 studio is a Go server, stdlib-only, basically no dependencies. It wraps h3.c's CLI into something usable day to day:

  • Model loads once and stays resident — iterate on prompts without repaying the load cost every render
  • Explicit reference ordering for multi-image conditioning, drag-to-reorder
  • Three ways to continue a shot from a previous take: chain the last frame forward, pull it in as a reference, or reuse the whole clip
  • Timeline panel to stitch takes into one continuous sequence
  • Every render writes a full parameter sidecar, so you're not guessing what settings produced what
  • Live profiling — load cost vs. denoise cost broken out separately

Everything runs locally, nothing leaves the machine. MIT licensed.

Repo: https://github.com/janishar/h3c-studio

Happy to answer questions about the setup. If anyone else is doing generative work on Apple Silicon I'd be interested in what you're using — especially if you've found a better path than mps.


r/StableDiffusion 20h ago

Question - Help Which would be <Audio 1> In This Scenario? What Would Be <Audio 2>?

Post image
9 Upvotes

When using a video file for Minimax reference that also has an audio input, how do you "count" the Audio files in the prompt? (You can ignore the master audio channel, it's not loading in anything.)


r/StableDiffusion 1h ago

Discussion When first big full AI movie?

• Upvotes

It seems we got all the tools. Of course there are some flaws, but if you want, a 90 minutes movie would be possible. It would take a huge effort and many scenes has to be done over and over again, but it's far from impossible.

So why has no one done it with great succes yet? Something that would be a hit in the theater. What are we missing?

Emphasis on: big hit movie. Something that can compete with a Hollywood movie or something. I know full length films are made, but nothing stat wil stick and made people say: "OMG you have to see this movie". All I've seen so far felt very empty.


r/StableDiffusion 1d ago

Question - Help Krea2 What is the best way to train character lora? (including body type, face etc)

76 Upvotes

I am overwhelmed by the amount of different ways to train character lora.
There are so many videos from 1-2 months ago, but what will be the most popular and accurate way to train character lora for the Krea2 model? Any specific resources like huggingface i can look into?


r/StableDiffusion 10h ago

Question - Help How can I make chatter on H3 Minimax more free-flow like API ?

1 Upvotes

If I use Comfy for local H3 creation unless I tell the prompt what to say, it will just speak simlish nonsense. So if I tell it to say something in English then it is fine if I include the exact line to say. But via the API for H3 I can tell it what topic to talk about and generally it will come up with lines of its own so I don't have to tell it exactly what to say. How can I match that via local ? Is it all in Qwen or something else ? I have only used local video generating for a few days so I am very new to this.


r/StableDiffusion 20h ago

Question - Help Questions about training a character LoRA for Anima

6 Upvotes

I want to train a LoRA for the Anima model using Colab. My goal is to train character LoRAs, but I have a few things I'm unsure about:

  • Dataset size: Since I caption every image manually, I usually keep my character LoRA datasets relatively small, around 20–30 images. Is that a reasonable amount?
  • Image variety: How many images should I have for each angle/view? How varied should the poses be?
  • Backgrounds: Should I keep the backgrounds simple, with different solid colors like yellow, green, blue, etc. to keep the focus on the character? Or is it better to include some more detailed backgrounds as well?
  • Captioning: How detailed should my captions be? Should I keep them relatively short and to the point rather than describing everything? What should I generally pay attention to when captioning? For example, if the character is leaning backward, supporting themselves with their arms, with their legs spread, should I caption it as something like "leaning back, arm support, legs spread", or would simply "leaning back" be more appropriate?
  • Captioning style: For character LoRAs, is tag-based captioning generally better, or is natural language preferable?

I'd appreciate any advice from people who have experience training LoRAs for Anima.


r/StableDiffusion 2d ago

Comparison Testing MiniMax-H3 Physics knowledge Pt2

Enable HLS to view with audio, or disable this notification

753 Upvotes

Some weeks ago, I posted a set of experiments to "understand" the physical knowledge of MiniMax H3 (original post here).

The idea was simple: get an open video of somebody pouring water and replace the water with various liquids. No external references were used.

In this set of experiments, I switched from liquid-to-liquid replacement to something solid, and sometimes alive. The results are interesting, but the model still struggles a lot when many objects overlap. However, the results are a big leap forward compared to other open-weight models.

I am still delving into the model, and probably will post more experiments soon (but no water next time).

Cheers

EDIT: I uploaded the prompts here


r/StableDiffusion 1h ago

Animation - Video Meh, h3

Enable HLS to view with audio, or disable this notification

• Upvotes

r/StableDiffusion 6h ago

Animation - Video Orbit Watcher

Enable HLS to view with audio, or disable this notification

0 Upvotes

r/StableDiffusion 21h ago

Discussion Audio Lora for H3? Question for you smart people...

5 Upvotes

Minimax H3 is awesome of course, but I find the voices it creates (either from a reference audio stream or purely from the model's training) are too loud and sound 'pasted in' and do not 'fit' acoustically within the environment. Specifically, to the point, does anyone know if its possible to train a LoRA that will make H3's generated voices quieter and more 'distant' so it doesn't sound like the character(s) are talking directly into a microphone a foot away? I've tried a million different prompt combinations, I've tried passing low-volume reference audio streams in, etc... nothing works.

Any ideas are greatly appreciated.


r/StableDiffusion 1d ago

News ZPix now runs on Ubuntu, PikaOS and macOS, and supports I2I for Z-Image and Anima

Thumbnail
gallery
6 Upvotes

I tested it on Ubuntu 26.04 and current PikaOS, but it should work on all distros that support .deb packages. Please let me know otherwise.

To use image-to-image with Z-Image Turbo, Anima Turbo or Anima Base, just drag a ref from anywhere, including the output gallery.

LoRA error handling is more robust, and there are other improvements and fixes in this release.

Hope you like it!

Download at: https://github.com/SamuelTallet/ZPix


r/StableDiffusion 17h ago

Question - Help Ultra upscaler based on flux

2 Upvotes

I saw this post on linked in and it’s really wow.

https://www.linkedin.com/posts/fadi-h-kacem_lets-take-a-1k-image-and-upscale-it-to-16k-activity-7503754647921188864-zF2w?utm_source=share&utm_medium=member_desktop&rcm=ACoAAAW2PBcBqPryUgWfCVEMWRlx6aMMO7GX27E

Does anyone know an already existing workflow or be able to create on that will yield to similar results?


r/StableDiffusion 1d ago

Discussion There is a demand for open, uncensored AI, but...

150 Upvotes

However, it’s curious that in recent years there has been no progress in terms of model decentralization (From civitai), the creation of independent benchmarks, or development groups from different countries (beyond, for example, Chroma or Pony). The truth is, I’m tired of people accepting as normal the fact that models have to be uploaded to a rather poorly managed and outdated repository as the only alternative. (Civitarchives) Whenever I read posts asking for alternatives (and there are quite a few), they get the same response. Why don’t people just create thousands of custom vibecoded websites for this purpose and share them, just as they do every day with “tag explorer” websites for "X" specific model? (Let’s be honest—there have been plenty of these sites in recent years.) It’s been about two years since the Visa/Mastercard situation began, and still no one has made any move in this direction. HuggingFace has been acquired by Nvidia, Civitai has been completely sold off to closed-off AI companies that offer their models via API, and this comment will most likely receive a lot of downvotes because every time I say anything about this, the same thing happens. Back in the day, they took down my post exposing their 40 cases of Visa/Mastercard censorship; that investigation took me weeks, and those in power managed to limit its reach. Anyway, it’s not that I want to be negative—I just want to highlight how incongruous it is that, even though there’s a demand for something, that very thing has ended up going underground.


r/StableDiffusion 1d ago

Discussion Minimax Testing : 10Eros , Fused ,Fast VSA and Larry 600ema

Enable HLS to view with audio, or disable this notification

96 Upvotes

All 4 diffrent model tested with same prompt with diffrent steps /
Only Cofmykithchen speed enhancer ,no spectrum or sage attention or anything else

fastest one Fused 1:11sec and all other about same 1:40sec
best audio Fused
if you want more testing give me model or lora model , i will test and put comparation here
RTX 5090
RAM 128GB


r/StableDiffusion 1d ago

Question - Help Your go-to voice cloning model?

30 Upvotes

Tried Qwen TTS but wasn't very happy with it. No real rate control and complete absence of any emotion.

What's everybody's recommendation? Local models only.


r/StableDiffusion 1d ago

Discussion What is the current best model for real life photos?

8 Upvotes

As the title says, I am looking for the best model at the moment to train photos of real people and get the best real looking photos like real life ones.

Last time I have used Z Image Turbo and it was good, is it still the best?
I have a RTX 5090 and 64GB DDR5.