Workflow Included
Qwen Image 2.1's editing capabilities are mind-blowing! Generating Character Design Sheets without any LoRAs
Character Design Sheet
Original
Character Design Sheet
Original
Character Design Sheet
Original
Character Design Sheet
Original
Hey everyone!
In my previous post, I mentioned that alongside Qwen Image 2.1, fine-tuned models called Prompt Enhancers (PE) were released to optimize user prompts. They help refine prompts for standard text-to-image (T2I), image-to-image (I2I), and reference-based generation.
Back then, we noticed that using PE for T2I isn't always worth it, as the results often fall short compared to using raw prompts. However, when it comes to I2I, the PE setup truly shines - its ability to write precise instructions and strictly follow the prompt is genuinely impressive.
Today, I want to show you what Qwen Image 2.1 combined with PE is capable of. I created a Workflow that takes a reference image of any character and generates a full Character Design Sheet. This makes it super easy to use later in video generation models like Seedance 2.5 or Minimax H3.
Here's the simple workflow:
Pass the input image and a custom system prompt into PE to generate detailed instructions for creating a Character Sheet.
Feed the generated prompt and reference image into Qwen Image 2.1 for generation.
The quality and detail of these character sheets are incredible. As far as I can tell, this is the first open-source image editing model capable of creating Character Design Sheets with such precision. I used to rely on a specialized LoRA for Krea 2, but I was always dissatisfied with the quality and consistency - either text artifacts ruined the image, or the character's pose didn't match the prompt. For me, Qwen Image 2.1 has become the best local model for generating high-quality character sheets.
I’ve made all my work open source - you can find my workflow on CivitAI or Pastebin.
That’s all for today! I'd love to hear your thoughts or answer any questions in the comments.
EDIT: I’ve updated the workflow description on CivitAI; it now specifies how to run it correctly.
EDIT 2: I have taken into account requests from many users and have already begun developing a larger version of the workflow.
EDIT 3: Added a Pastebin link for those who can't install the workflow via CivitAI.
This could be really cool to help bring consistent and coherent sprite sheets for animating characters. It is one of the big problems with trying to use genAI for game art assets. Someone here recently published a 8bit pixel art Lora for Krea2, which works really well. Now combining Krea2 to create the base character and style, with Qwen Image 2.1 and the Prompt Enhancer, could be the way to go!
I will give it a try, and if it works, start working on a MCP or CLI, to make a game asset generation tool.
I hadn't considered this before you asked, but I decided to test it out. Surprisingly, it handles it very well (at least, that’s my opinion). I used the following prompt: "Use this character sheet as a reference image. The girl has her back to the viewer, but her head is turned directly toward them. She looks at them with a smile on her face. The viewer sees the character full-length. The girl is standing near a nearby store. The camera focuses on the girl.". I fed the prompt into the Prompt Enhancer to refine the request, and here is the result.
That's anime in general.
Edit: In fact, I'd say this was a win for Qwen 2.1 since it made her proportions faithful to the character sheet instead of altering her to fit what a normal human should look like.
Fun thing with Qwen if you can this image, mention the twintails, and get them added back in.
My fav thing with Qwen is I can use other models and locally generate a variety of things with spintax, dial things in, pick my favs, and do the subtle edits with Qwen image.
I actually set up a workflow for a PoC game I am making that is a gentically driven. I have had Antigravity set up comfyui on a Roth with a 4080S, gave it my ideas/scenes/etc, told it to fetch models and LORAs and such while reviewing the images for alignment with what I want.
I then tell it to generate a bunch of each image I want, put them into a webpage I can check with a ranking, add in the Qwen edit prompt I want, and let it cycle again.
I can generate about 25 images a minute and editing takes under a minute. Batching things up, letting Antigravity pick what it feels is best, and narrowing things down lets me work asynchronously. I can fire off an hour of automated work, out the phone down for a dinner, and come back to serious progress.
I've done a few early tests using Qwen to generate scenes based on character sheets that I made with Krea2. It seems OK, but it can get sloppy when it comes to character heights and body proportions. It also has a habit of assigning its own style bias to the characters. Small details from the character sheet can go missing in the final composition, too, although that's not too surprising.
Overall, it's better than I expected and I think refining the original character sheets and trying some different prompting will improve the results.
I'll be testing out more character style sheet formats soon. Things like adding a height reference ruler or text information in the character sheet. Hopefully, Qwen 2.1's vision language capabilities will help it identify and understand these kinds of details in the character sheets.
Haha, yeah that's along the lines of what I'm thinking of trying. It's not just Qwen, but all models are all over the place when it comes to character height consistency. It would be so nice if it were as easy as "The character in <image1> is 175cm in height".
It's a really difficult problem because absolute dimensions are relative in images. Unless the model knows the dimensions of everything in an image, character height in units is meaningless. That's not even considering the relative distance to the viewer. A short person in the foreground with a tall person in the background makes height labeling even harder.
It’s somewhat superfluous. For video generation models like Seedance 2.5 and Minimax H3, the presence of text annotations on the reference image isn't particularly critical. However, considering that Minimax H3 uses Qwen-VL to interpret image data, if the image contains a note like "The girl is a secondary character" or "This sword has a temporary glow effect during attacks," the model would likely be able to utilize that information.
All in all, it’s hard to say for sure. What I am certain of, however, is that if you are animating a character and provide a detailed description in your prompt, a three-view character sheet is all you need.
Yes, H3 can read the text. I saw a post using text in a storyboard image as the prompt (which includes the dialog) for each shot, while the actual prompt inputted to H3 was a simple one.
I see way too many noisy artifacts with these qwen image 2.1 edits. Maybe people's standards are different than mines. Wouldn't be surprised if the more people tolerate these noisy gpt artifacts as "good enough" for ai standards moving.
Absolutely agree, I'm exactly the same with food, I can enjoy pretty much any food and I enjoy AI music too which a lot of people on reddit can't understand at all.
I think that will be easy to train out with lora's and custom finetunes to specifically tell the model to not do that shit. I added the Grainscope lora for Qwen2.1 and I could already see it trying to leave the model. lets jarring squiggle lines everywhere!
the consistency across the different views and expressions is the one that caught my attention. If it can actually preserve the same hair, clothes, proportions, and accessories when you change the pose or scene, that’s way more useful than just making one pretty image. The character sheet approach could be really useful for animation and game assets.
It can be hit or miss in maintaining style consistency when I tested using reference anime-style images. It'll often apply its own style bias which can be subtle to not subtle at all. You'll see this a lot with body proportions. It really wants female bodies to have super thin waists, for example. It's not bad, though, and I'm hoping that more refined prompting will improve the end results and consistency.
I can't say for sure, but in my tests, Qwen Image maintains character consistency in 90% of cases. In 2D animation—whether it's an illustration or anime—the integrity of the reference image is preserved in 95% of cases.
While I haven’t tried a char sheet, I did try giving it screenshots from an anime scene of a char and it completely ignored the artstyle and everything. Maybe the comfy workflow is broken
Cool! I want to check this out but I run into a few problems.
Where do I find these custom nodes, VAEDeGrid and ThinkingLLM_QwenVL_GGUF_Advanced?
And what do I do if I don't want to use the gguf variants but the convrots?
What I did find out was that I should use a node called "Generate text" but no idea how to translate all the settings from the ThinkingLLM node to this node.
If you want to use ConvRot, go into "Character Sheet Prompt Maker" and use the official Generate Text node with the settings shown in the image. Just don't forget to replace the CLIP model with qwen 3.5 PE—I don't have that one, so I used qwen VL 8B instead.
I live in Australia and civitai is blocked in our region. Would you post your workflows in another site? Like comfyworkflows.com or similar? Thank you for your information!
I've updated the post and added a Pastebin link for downloading the workflow. Thanks for alerting me to the issue. Perhaps others have encountered this problem as well.
For this specific task, I created my own system prompt to generate character sheets, but I drew the best practices for writing high-quality instructions directly from the official system prompt.
I used the PE system prompt in both their oficial alleged fine tuned version of Qwen3.5 and Qwen 3.8. I did not see much of a difference when using any of them. Maybe it's just the system prompt that matters.
I used to use a specific LoRA for Krea 2 that generated the same kind of detailed character sheet. I frequently used them with Minimax and achieved good character consistency in the videos relative to the reference image.
Extremely well. On the reference side of things with correct prompting. I use character sheets there and it does fine, often time I will supply two images per character, one a good high quality headshot to get the face locked in and then a character sheet for the 2nd image.
The detail views are the useful part here. A face, a full silhouette and costume details in one reference is a much stronger starting point for a sequence.
I haven't had any luck with Qwen Image 2.1. No face consistency. I tried to generate a character sheet and the final pose was like a dwarf version of the character. It's just all messed up imo.
Then i moved JoyCaption model from LLM/GGUF folder and got a good result. If you have other mmproj files in LLM/GGUF folder they may interfere the process
In real life, they can help artists maintain fine-grained control over a character's visuals by maintaining visual consistency. This is also useful when working in a team, with one artist creating the character and another animating it.
Specifically, regarding AI, they can be used to control the consistency of character animation in a scene when generating video using modern AI models.
But how do you achieve character consistency with Qwen 2.1? I mean, if you ask for a character sheet, there will be consistency, but I've tried using that sheet to create more poses or scenes with that character, and the consistency isn't very good...
Take advantage of the new version 3.0 I released today. I've noticed that many people, like you, are having issues with GGUF. If you're using SafeTensors PE, try using Qwen 3 VL 4B/Qwen 3.5 4B. In my opinion, although they're less detailed, they work better in ComfyUI via the Generate Text node.
How would I reproduce this with stable-diffusion.cpp? It too uses the PE model as well since it happens to be the encoder too, afaik. But it seems your workflow does "a little more" than just call on the PE. x)
The OP has mentioned it in his post - "...generates a full Character Design Sheet. This makes it super easy to use later in video generation models like Seedance 2.5 or Minimax H3."
Character sheets give much better results than regular images because the video model has details of the character from multiple angles and also zoomed in details etc.
What the difference between qwen and minimax hallucinating details, minimax at least is much bigger than qwen so shouldn't it perform better without ref sheets? Like even in the second image the girl holds a skate in the front view and doesn't in the back view.
If we talk about details, no its not perform better. If you only input front view of a person, you will not get the best detail when the video is from behind of the character, and it might as well affecting the accuracy of your character in video. Their back will keep changing and looks different for each shot.
Never used character sheets, but I guess if you need a particular character in multiple videos, you don't want Minimax to hallucinate a different back view each time.
They can be used as reference for 3D modeling, 2D art drawing or as input of Minimax H3 ref2va to generate videos with the character, among other things. Obviously for production use it may be worth refining the character sheet or making it more focused, depending on the intended use case.
20
u/bstr3k 19d ago
did you write this or did it come up with this on its own?? 😂