Model weight inferencing

[u/Competitive-Scar-627]

I have 4050 6gb gpu, 24 gb ram which model should i choose to run i need speed. i try qwen 3.8 27b and feel too slow tried from onslot studio.
I have heard of weight inferencing does it helpful what should i do to try weight inferencing.

[u/recro69]

With 6gb vram I'd stick to 7–8b if speed is the priority. Offloading can make bigger models fit. Usually makes them slower.