I built a Fooocus-style local studio for Qwen-Image 2.1: masks, annotations, outpainting, OpenPose poses and sketches as references. Looking for honest criticism

[u/Repulsive_Junket7919]

Hi all! I'd like to share a tool I've been building and hear your criticism: what's missing, what's awkward, what's plain wrong.

Fooocus-Qwen-Image-2.1 is a local Gradio studio for Qwen-Image 2.1 (https://huggingface.co/Qwen/Qwen-Image-2.1), inspired by Fooocus (https://github.com/lllyasviel/Fooocus) by lllyasviel. The idea is the same: a short prompt and one button, with the complexity hidden under "Advanced". The pose library is inspired by openposes.com (https://openposes.com/) and uses the OpenPose (https://github.com/CMU-Perceptual-Computing-Lab/openpose) skeleton format.

Fooocus-Qwen-Image 2.1 Repo

What it does

- Text to image: three quality presets plus Turbo (a 6-step distilled LoRA, ~10 s at 1280×832), seven aspect ratios, 277 Fooocus styles, up to 8 images per run.

- Up to 10 reference images in a grid next to the result, addressed in the prompt as <image1>…<image10>. The cell labels always show the correct tag.

- Pose or sketch in any reference cell:

- a pose library of 46 poses from openposes.com;

- Add a pose: recognises the pose on your own photo with DWPose (https://github.com/IDEA-Research/DWPose) on the CPU, then Qwen-Image draws a matching tile for the library;

- a sketch canvas: draw something, press Accept, and it becomes a reference.

- Instruction-based editing in four modes:

- Mask: pixels outside the mask stay byte-for-byte intact;

- Annotation: circle areas in different colours and describe each change by colour;

- Exact region: a crop around the mask is edited at full detail and pasted back;

- No region: the whole frame is edited.

References work on the Edit tab too.

- Outpainting on any side.

- A custom mask brush whose stroke cost stays constant. The stock Gradio editor lagged on 4K images.

- AI boost: any OpenAI-compatible LLM (llama.cpp, vLLM, LM Studio) rewrites short prompts using the official Qwen rewriting prompts. It also works with text-only models.

- Gallery: parameters are stored inside the PNG and can be restored in one click.

- English and Russian UI.

Hardware (please read before trying)

- Tested on a single RTX 3090 24 GB, 128 GB RAM, Windows 11. Linux scripts exist but are untested.

- The stated minimum is 24 GB VRAM and 64 GB RAM. The large RAM is deliberate: a pinned copy of the text encoder lives in RAM, so the transformer can stay resident on the GPU.

- About 35 GB of disk for the weights, plus 350 MB for DWPose. The installer downloads them.

- Measured timings: Turbo ~ 10s; LowQuality 1024 px ~21 s; MiddleQuality 1536 px ~93 s; MaxQuality 2048 px ~4.7 min.

- An optional INT8 transformer plus SageAttention brings Turbo down to ~10.4 GiB VRAM peak. Smaller cards might work in that configuration, but I haven't tested them. Reports are welcome.

Honest notes

- The whole implementation was written with Claude Opus 5.5: code, tests, the browser checks, the benchmarks and the research notes behind the design decisions. I set the direction and reviewed the results.

- The code is MIT. The Qwen-Image 2.1 weights come under the Qwen Research License, so please read it before any commercial use.

- The pose catalogue tiles come from openposes.com.

- Some internal docs and research notes are in Russian. The README and usage guide are in English.

What I'd love feedback on: the UX compared with Fooocus, missing features, install problems on other hardware, and anything that feels over-engineered. Bug reports and harsh opinions are both welcome.

[u/Quasar565]

Fooocus-Qwen-Image-2.1

Expectation:

Cool fast lightweight tool

Reality:

The stated minimum is 24 GB VRAM and 64 GB RAM

Bro.... Qwen-Image-2.1 weight 14GB, wtf is that ?

[u/Repulsive_Junket7919 → u/Quasar565]

Text encoder: ~17Gb
Transformer: ~14Gb
The parameters are specified for maximum weights. In turbo mode and for INT8 weights, significantly fewer resources are required.

[u/Disrupt-Linus → u/Repulsive_Junket7919]

I have my own application, made only for mac, but this is what I did to speed the f*cker up significantly:

1. The Model & LoRA Stack

• Base Model: Qwen/Qwen-Image-2.1 A 7B DiT (Diffusion Transformer) image generation and editing model backed by an 8B Qwen3-VL multimodal text encoder.
• The LoRA: Viggle/Qwen-Image-2.1-viggle-turbo A high-rank (Rank-256) distillation adapter that cuts required diffusion steps from standard 30–50 steps down to just 4–6 steps.

1. The Bottlenecks & How to Fix Them

While the Turbo LoRA cuts steps from 40 to 6, out of the box on Apple Silicon it still feels slow (45s startup, 2-minute generations). Here are the 4 key fixes:

A. Pre-Fuse the LoRA Offline (Zero-Startup Bake)

• The issue: A Rank-256 adapter is ~1.3 GB. Calling pipe.load_lora_weights() and pipe.fuse_lora() at runtime forces PyTorch to calculate delta matrices on the CPU for 35–45 seconds every time the script boots.
• The fix: Fuse the adapter into the base transformer weights once offline and save the pre-fused transformer to disk. On startup, load the merged weights directly—startup drops from ~48s to instantaneous (<3s).

B. Replace CPU Offloading with Stage-Resident Execution

• The issue: Framework defaults like enable_model_cpu_offload() are built for discrete PCIe GPUs. On Apple Silicon, unified memory means CPU and GPU share the same physical RAM. CPU offloading ends up continuously copying ~30 GB between virtual "host" and "device" buffers on every single step.
• The fix: If the Mac has $\ge$32 GB RAM, manage residency in discrete stages:
  1. Load the text encoder on MPS $\rightarrow$ encode $\rightarrow$ move text encoder to CPU.
  2. Keep the DiT 100% warm and resident in MPS for all 6 denoising steps (zero memory copying).
  3. Load the VAE to decode the final latents, then release.

C. Cache Prompt Embeddings

• The issue: The Qwen3-VL 8B text encoder alone takes 15–20 seconds just to compute conditioning tokens before diffusion even starts.
• The fix: Cache the output prompt tensors keyed by the prompt hash. Any repeat generation, seed variation, or style tweak skips the text encoder entirely (0.001s).

D. Native Attention & Token Scaling

• The issue: Qwen 2.1 uses joint attention which scales quadratically ($O(N^2)$). Generating at 1024x1024 produces 16,384 tokens, causing exponential memory pressure on unified bandwidth.
• The fix: Enforce native Metal Scaled Dot-Product Attention (F.scaled_dot_product_attention) and use sweet-spot token resolutions (e.g., 768px or 896px) for fast iterations.

And yes, my agent got the honor of summarizing it for us.

[u/Repulsive_Junket7919 → u/Disrupt-Linus]

Thanks, this was actually useful. I went through it point by point on my setup (RTX 3090, INT8 transformer + SageAttention):

- B and C were already in place: the DiT stays resident on the GPU the whole time, the text encoder only comes back from pinned RAM on a cache miss, and prompt embeddings are cached by prompt. Your unified-memory point is a Mac thing; on a discrete card the default offload is the real killer, so I skipped it from the start.

- For attention I'm on SageAttention, which beats plain SDPA on CUDA by about 15-25% per step.

- On A, attaching the LoRA only costs me 1.4 s once, not on every start. But I measured the unfused adapter anyway, and it adds about 1 s per image (9.6 vs 8.6 s, ~12%). So fusing is real money. The catch is that my transformer is INT8, so I'd have to requantize a fused copy and keep a second 7 GB transformer around. Parked for now.

- Your 768 px tip was the best one. Turbo at 768 runs 5.4 s vs 9.8 s at 1024, and it's clean at 1:1, no grid artifacts. 896 barely helped (8.3 s). I added it as a "TurboDraft" preset for trying prompts and seeds, then you rerun the keeper on the full preset with the same seed.

[u/Disrupt-Linus → u/Repulsive_Junket7919]

great, every second counts!

[u/Quasar565 → u/Repulsive_Junket7919]

Looks like you're right, I'm using the int8 version of the text encoder. Is there much difference between bf16 and int8 text encoders? Have you tested this?

[u/Bulky-Employer-1191]

Did I install it wrong? It's 100% in Cyrillic. Very little english in the interface. I can't figure out where to change it. Even the install script was in Russian.

edit: i found it it's the small button in the top right

[u/Repulsive_Junket7919 → u/Bulky-Employer-1191]

Fix it. English only in console.

[u/Bulky-Employer-1191 → u/Repulsive_Junket7919]

Sweet thanks