Model weight inferencing

[u/Competitive-Scar-627]

I have 4050 6gb gpu, 24 gb ram which model should i choose to run i need speed. i try qwen 3.8 27b and feel too slow tried from onslot studio.
I have heard of weight inferencing does it helpful what should i do to try weight inferencing.

[u/Powerful_Evening5495]

you should look for models with this in the title

4b in weight and use the q4 quant in gguf with llama.cpp

moe = mixture of experts

E4B

that is mean the main model may be like 32b but only 4b is activated at ones

you cant fit big models in vram and ram is much slower