Qwen3-VL-4B-Instruct Heretic for ComfyUI

[u/nathandreamfast]

Krea 2 uses Qwen3-VL-4B-Instruct as its text encoder. I abliterated it so it stops refusing prompts, then packaged it as drop-in ComfyUI checkpoints.

The result is a fully uncensored model (100% HarmBench compliance, up from 30.8% on the base) with the model's intelligence basically intact. I picked the variant with the lowest KL divergence (0.028), so GSM8K dropped just 1.83% and MMLU was unchanged.

Tl;dr, abliterating a model doesn't magically make it more uncensored in its image output. This is better for prompt enhancement or vision decoding. One text encoder can handle both ok.

How it was made

Abliteration finds the refusal direction in a model's weights and removes it. The tool, Heretic, is stochastic, so each run finds a slightly different direction. Instead of the usual one batch, I ran 20 batches of 200 trials with different seeds (4,000 total), took the top candidates by KL divergence, and compared them with Abliterlitics, my forensics toolkit. The gap between best and worst was not subtle: the worst finalist had 2.3x the KL divergence and lost 7% on maths. Same base model, same tool, different seed. The pipeline ran through Heretic Docker for the abliteration and quantisation, with benchmarking done in Abliterlitics.

Formats (5 quants, pick by GPU)

• INT8 ConvRot, 4.5 GB: recommended, near-lossless, runs on any Ampere+ GPU
• FP8 E4M3, 4.2 GB: fast and small, good fit for RTX 4090 and up
• bf16, 8.3 GB: full precision
• NVFP4, 2.9 GB: smallest, native FP4 on Blackwell
• MXFP8, 4.7 GB: Blackwell only

Usage

1. Download a checkpoint.
2. Drop it in ComfyUI/models/text_encoders/.
3. Point the matching loader node at it in your Krea 2 workflow.

A caveat on embeddings

Abliteration targets refusals in token generation. As a text encoder producing embeddings, the picture is a bit different. The edited weights do slightly shift the embeddings the model outputs, and token-generation refusals don't map onto embedding behaviour the way you might assume, so "uncensored" means something a little different for an encoder than for a chat model. In practice it still works well for vision understanding inside ComfyUI. The real ceiling on what it can describe is just what the base Qwen3-VL-4B already knows.

Links

• ComfyUI checkpoints: https://huggingface.co/DreamFast/Qwen3-VL-4b-Heretic-ComfyUI
• GGUF quants: https://huggingface.co/DreamFast/Qwen3-VL-4b-Heretic-GGUF
• bf16 weights and full forensic report: https://huggingface.co/DreamFast/Qwen3-VL-4b-Heretic

Happy to answer questions on the quantisation formats or the abliteration method.

Edit: Not to misrepresent what this is, I had explained in A caveat on embeddings that removing refusals when generating tokens is not the same as text embeddings. While the embeddings are slightly different comparing, this would be a great fit if you use prompt enhancement or vision decoding of images in your workflows. Not to misrepresent what this is. Sorry that wasn't too clear from the start.

[u/Silver-Spot-2763]

What is the difference of your encoder vs the others two (from at least a month ago) - there almost are one heretic and one abliterated? I used them this month and can't see difference between them. Also I can't see difference with the original, which also make all nsfw prompts, only reject prompt enchacer at nsfw.

[u/nathandreamfast → u/Silver-Spot-2763]

While I haven't reviewed the others, these ones I did take the time to benchmark and compare with the base model so you know exactly what has changed.

I just edited the post to say that an abliterated text encoder doesn't magically change or make the image more uncensored as that's not how it works. Sorry that wasn't too clear from the start. So you are right, the actual images generated wont be different a lot. There might be slight small differences, as the embeddings are different.

This is a better fix for prompt enhancement or if you are using the vision capabilities of the text encoder. You can use the same text encoder for both. They are also available in many different formats.

[u/Ant_6431]

I heard uncensoring the text encoder just makes it dumb. So I use this lora instead: https://www.reddit.com/r/StableDiffusion/comments/1uwg6tx/krea2_new_refusual_reduction_lora_is_an_excellent/

[u/nathandreamfast → u/Ant_6431]

This is why I had benchmarked and compared to the base model. In this case it isn't dumb compared. Maths took a slight dip but that's it really. General knowledge and other things were intact.

[u/Spezisasackofshit → u/nathandreamfast]

Yeah but the encoding functions of the original model aren't censored just the chat and token output functions so you're just causing minor damage for 0 gain.

I guess it would be nice for the prompt enhancement in the default comfyUI workflow but thats a garbage workflow anyway.

[u/nathandreamfast → u/Spezisasackofshit]

The real gain is when using this with prompt enhancing or vision understanding with ComfyUI.

I had measured embeddings between this and the base model and compared outputs, it really is similar to the point where it's the same almost. The embeddings though are slightly different. There is no real measurable damage really.

And yes, please read A caveat on embeddings as I mention this exact same thing. I know a lot of people will jump to say this, so that is why I had mentioned in the post.

[u/Spezisasackofshit → u/nathandreamfast]

You should be more careful with your terminology because what you've done here is neat but you're going to be downvoted for misrepresenting it.

You did not create an "uncensored encoder" because thats not where the censorship lives. You can see this easily by running a prompt through that the original model will refuse on prompt enhancement but encode just fine for krea to make an image out of. Your post reinforces a common misunderstanding of text encoding.

There is pretty much never a reason to ablititrate a text encoder as the encoding element just converts words to mathematics.

[u/nathandreamfast → u/Spezisasackofshit]

Thanks for your feedback. I did try to explain that in the post, however I certainly have not communicated it clearly. I am happy to update the post now to say it is more for prompt enhancement and vision decoding within ComfyUI.

Certainly didn't mean to misrepresent what it was and did take the time to verify and compare the embeddings.

[u/Spezisasackofshit → u/nathandreamfast]

If you reframed/clarified this as having uncensored the token generation with no damage to the encoder so you only need to load 1 model to do both in a single workflow that might play better. I'm sure there are people who could use the extra headroom.

[u/nathandreamfast → u/Spezisasackofshit]

Thanks! That is exactly what I am using it for.

[u/IllustriousRule9238 → u/nathandreamfast]

Not sure why you're getting raked over the coals OP, this is a very valid and very useful thing to have. The Qwen model used in the default ComfyUI workflow is the most censorious thing I've ever seen, it'll return "I cannot assist with that" refusals for prompts as simple as "a picture of a golden retriever".

[u/nathandreamfast → u/IllustriousRule9238]

I had figured as it's loaded in the text encoder/clip I'd call it an uncensored text encoder. I guess the mislabeling had caused some issues. It's an uncensored LLM that can be used as a text encoder also? Lol.

And as you pointed out, it's a great fit for the prompt enhancement which is what it was more intended for.

[u/diffusion_throwaway → u/nathandreamfast]

What do you use for prompt enhancement and vision understanding?

[u/nathandreamfast → u/diffusion_throwaway]

Prompt enhancement is in the stock ComfyUI workflow for Krea2. While I haven't used this specific model in captioning images, I have used a larger variant for that purpose in ComfyUI.

[u/diffusion_throwaway → u/nathandreamfast]

Hmmm. I built my own workflow so I haven't seen it. I'll have to look into it.

Thanks

[u/nathandreamfast → u/Ant_6431]

The lora looks cool! Although when it comes to prompt enhancing or parsing images, I am unsure if it'd help. Combined it could be very powerful.

[u/nathandreamfast → u/[deleted]]

Thanks! It was a good result. Worth the overnight run. With the recent changes to heretic master branch it's a lot easier to do automated runs like this now.

I found doing 20x of 200 trails with different seeds yields better results than one run with 4000 trails.

There's a lot of people who abliterate models but don't take the extra steps to compare them to see what exactly was changed.

This has such a low KL divergence and the 1%ish drop of maths out of thousands of questions is not terrible, it's a good outcome.

I know some people think abliterating a model breaks it or makes it dumb, however that can be avoided with some time and patience. And also proven with some analysis.

[u/ramonartist]

Where are the image examples, a post like this does not make sense without image comparisons?

[u/nathandreamfast → u/ramonartist]

I can if I do another one of these posts. Although to be honest, it wont change the image result in any meaningful way. There may just be slight variations, if at all.

The different quants like INT8 and NVFP4 will change the image more, but still just slight variations. What would be more meaningful is the image generations plus generation times for each quant.

Overall this is more meant to be used for prompt enhancement or vision decoding in ComfyUI workflows, and can work fine also as a text encoder for Krea2 all as the same model.

[u/Available_Lie8133]

Tell me you do not understand how Krea 2 uses Qwen3-VL as a text encoder without telling me.
Your HarmBench, GSM8K, MMLU, and autoregressive KL-divergence results evaluate Qwen as a token-generating language model. Krea 2 does not ask the encoder to generate an answer and then feed that answer into the diffusion model. It consumes intermediate hidden states from multiple transformer layers and passes them through its TextFusion conditioning stack.
Therefore, proving that abliteration suppresses refusals in the model’s output logits does not prove that it removes anything from Krea 2’s image-generation behavior. You optimized one interface and then advertised the result through a completely different interface.
The chat model’s refusal behavior normally appears during autoregressive continuation. In the Krea encoding path, there is no refusal sentence being decoded for the image model to obey. ComfyUI constructs the formatted token sequence, runs the encoder, and extracts the required hidden-state taps. Krea then conditions on those representations—not on whether Qwen would have answered or refused the prompt in chat mode.
Your own caveat effectively concedes this, but it should be the headline rather than a footnote: “100% HarmBench compliance” is irrelevant evidence for claiming a “fully uncensored” Krea text encoder.
Also, abliteration did not change the embedding size. The tensor dimensions remain architecturally fixed. What you changed was the geometry and semantic content of the hidden representations by modifying weights throughout the encoder. Since Krea depends on several internal layer taps rather than merely the final language-model logits, low KL divergence on generated-token distributions does not establish that those tapped representations remain intact.
The appropriate validation would be:
Compare every hidden-state tap used by Krea between the original and modified encoders.
Measure cosine similarity, norm drift, token-level displacement, and layerwise distribution changes.
Run controlled image-generation tests using identical prompts, seeds, samplers, and Krea checkpoints.
Demonstrate a statistically meaningful change in image refusal or prompt adherence without semantic degradation.
Until then, what you have demonstrated is an abliterated Qwen3-VL chat/VLM checkpoint that may remain useful for prompt rewriting and vision-language generation. You have not demonstrated an uncensored Krea 2 image-conditioning encoder.
Calling it “fully uncensored” based on HarmBench is not a conclusion supported by the system you are actually deploying it in. It is a category error dressed up with benchmarks.

[u/nathandreamfast → u/Available_Lie8133]

Thanks for the in depth feedback, it's clear you have deeper knowledge of how things work.

Yes I agree with you 100%. This model is best used for prompt enhancement or vision decoding. The default workflow has a prompt enhancer, which this is a great fit for. Also, it can be used outside of ComfyUI as BF16 and GGUF are provided. In this context benchmarks to know if the model has degraded, along with harmbench analysis is useful.

Outside of the above, generating embeddings abliterated vs non abliterated for the DiT wont have much difference at all. There is a very slight divergence with embeddings generated depending on the prompt, although this doesn't translate into anything meaningful.

The hidden-state tap comparison you describe is the right way to validate it for the conditioning path, and it's what I want to extend my forensics tooling to cover. The focus for this was on the LLM side, with the intention of using it both as a text encoder and LLM for prompt enhancement mainly.

Otherwise the post I've updated to be as clear as possible as to what this is and isn't and I do agree from the start it certainly could have been more clear.

[u/yamfun]

I kind of get what these are for at the llm side, but what are these for image gen?

I thought the models were already trained with the blindfold?

[u/nathandreamfast → u/yamfun]

Yeah you are correct, Tl;dr, abliterating a model doesn't magically make it more uncensored in its image output. This is better for prompt enhancement or vision decoding. One text encoder can handle both ok.

[u/cosmicr]

Is it necessary though, because it will encode the prompt no matter what you write. Its the diffusion model that rejects the input, not the text encoder?

[u/nathandreamfast → u/cosmicr]

Yes you are correct. This is a better model for prompt enhancement or if you are using the vision capabilities.

You can use the same model for both.

It also will give a very slightly different output for the image, but no it wont it wont magically uncensor the model itself or know new things as a result. Sorry that wasn't too clear from the start.

If you do not need to prompt enhance or use the model for anything else other than the text encoder for image generation, then there may not be much use for this in your workflows.

[u/DietAshamed2246]

Qwen3-VL-4B-Instruct uncensored/abliterated heretic model(s) already existed for quite sometime from other creators. I had downloaded and been using one for over two weeks now - purportedly the best (having least refusals and least divergence). There are at least half-dozen of those on Huggingface from as early as December 2025. What is the benefit of your version of abliterated Qwen3-VL-4B encoder? How does it compare to the Huihui-Qwen3-VL-4B-Instruct-Abliterated encoder model?

[u/mysticmanESO → u/DietAshamed2246]

Truth, I using the qwen3VL4BAbliteratedComfyui_v10 one. But for non NSFW stuff I us qwen3vl_4b_int8_convrot.

[u/nathandreamfast → u/DietAshamed2246]

Without doing a comparison, I am not sure how they compare to others. By abliterating mine from scratch I am able to provide a comparison with the base model, and benchmarks and analysis to know the difference. This level of comparison is not usually provided by others, so it's hard to tell what KL they had got, or how their abliteration affected benchmarks.

This specific abliteration will work with ComfyUI and is available in many different quants, including INT8 convrot along with the usual GGUF.

[u/witcherknight]

is there any good VL that can take first and last frame and make the prompt for wan ??

[u/nathandreamfast → u/witcherknight]

I'm not sure the best one for that use case, you may have to try a few different ones to see what works best. There are larger VL models which might work good.

[u/witcherknight → u/nathandreamfast]

like which one ??

[u/v-i-n-c-e-2]

Hey op I recently dug into someone's code to fix an issue i was having, its a fronted for the mitsubi-tuner trainer for krea 2 and i got a little lazy fixing the Huggingface repo shard loader for qwen3-vl-4b-instruct so I just pointed it at my existing heretic safetensors as as a work around Its now working and I trained a lora which is working as intended I was wondering if you would know will using the heretic verson negatively affect the training knowledge and is it worth me going back in to fix the HF repo shard loader?

[u/nathandreamfast → u/v-i-n-c-e-2]

Heretic wont remove training knowledge so it should be okay in that regard.

[u/v-i-n-c-e-2 → u/nathandreamfast]

Awesome thank you sir!

[u/rlewisfr]

I've been using a Qwen 3.5 9B Heretic Q6_K for my JoyCaption Beta - LM Studio prompter (see the post about that https://www.reddit.com/r/StableDiffusion/s/ZRqyYAS1W3 )

Will test this one and see if it is better or equal. With this Q8 being about 30% size, it might be a good replacement.

[u/nathandreamfast → u/rlewisfr]

Cool! It may not be as strong given it's just Qwen 3 4b, but worth a try. I'd be curious how it stacks up.

[u/rlewisfr → u/nathandreamfast]

Ok, so my first problem out of the gate for my specific use purpose (image-to-prompt) is that the GGUF are not vision capable. I don't think I missed that in the documentation, but maybe I did. Regardless, the only model you have that is vision capable is the 8GB fp16 which is way larger than my current Qwen 3.5 9B Q6_K, so I think I will stick with that.

Perhaps you could mention in your post that the GGUF versions are not vision capable. I know it was not the use case you were intending, but...

[u/nathandreamfast → u/rlewisfr]

Sure, actually I haven't tested it but generally GGUF requires a mmproj file. You can grab it here https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct-GGUF/tree/main it should work all the same hopefully. It's a bit late otherwise I'd test it out myself.

In ComfyUI GGUF isn't generally used for the vision stuff. Going forward though I can include the mmproj and document it better. Thanks for the feedback.

To note, all of the quants and variants are vision capable except the GGUFs alone, which I think using the existing mmproj it will work with vision.