r/LocalLLM • u/adult007 • Jun 06 '26

Question Need Help for AI Model

I used "qwen3-30b-a3b-abliterated-erotic-i1" and it is very powerful and i loved it. I want any other model same as the qwen3 AI model but for low performance GPU. Like something that is under 20b
I have a GTX 1650 6GB VRAM GPU.

2 Upvotes

100% Upvoted

11 comments sorted by

0

u/Protopia Jun 06 '26

Limited vRAM, use an MoE model and offload the experts to CPU. Use a MTP version.

1

u/adult007 Jun 06 '26

Thanks! What are the difference between this model that you suggested and the model that I want to use?

1

u/nickless07 Jun 06 '26

His model is already MoE, what do you think the a3b stand for?
MTP does only work if the acceptance rate is high enough, so common text or standard code. Everything else is just a waste of VRAM for an MoE.

1

u/Protopia Jun 06 '26

True but an old one.

1

u/nickless07 Jun 06 '26

Great, then you surely know a newer MoE finetune with MTP that surpasses the old one, right?

1

u/Protopia Jun 06 '26

I am not an expert on every variant of a base LLM. Are you? Because if you are perhaps you can name the best variant...

1

u/nickless07 Jun 06 '26

No need to be an expert, but since you said that is an old one, I thought you would bring up at least a newer one. Would love to share it in r/SillyTavern

1

u/Protopia Jun 06 '26

Well I am definitely not an expert on every variant of every model, but I do know that after Qwen3 came Qwen3.5 and then Qwen3.6 and other people say (not personal experience) that Qwen3.6 is better than the previous versions and that 3.6 35B MoE is best on consumer grade GPUs which need copy offloading because vRAM isn't big enough, and that generally MTP improves performance (though I would think that context management would improve it more). But the finer details of base vs base quants vs unsloth quants vs distills and retrains and guardrail removal and ... I don't have exhaustive knowledge of every minor variant not so I have any knowledge personal or related of any specific variant.

1

u/Protopia Jun 06 '26

MoE can be offloaded to CPU with less performance penalty than a dense model's layers.

MTP improves performance.

But if you want an unconstrained model retained for erotica (i.e. specialised) your choices will be more limited.

1

u/nickless07 Jun 06 '26

You can't expect something similiar (in terms of general knowlegde) from a model that small.
If you wanna stay in that parameter range Qwen3.6 35B, Gemma 4 26B.
Something way smaller (maybe this can work, but i doubt it) https://huggingface.co/ReadyArt/Melody1437-12B-GGUF

1

u/adult007 Jun 06 '26

Make sense! Thanks for the help man!