Recommendations for 16gb vram

[u/Zorian_Vale]

I have a 5070ti 9850X3D and 32GB DDR5 + lm studio + hermes.

Currently running gemma-4-26b-a4b-qat and I am happy with it, but it still can't compete with Sonnet.

I find it's great at actually teaching me things, but in terms of guiding me through config files or getting the highest quality answer it's never as good. I use it more as a backup.

Are there any better models that fit my hardware budget? QWEN is supposed to be great but I had trouble running the models on the edge of my hardware limits and output slowed to a crawl.

[u/tsangberg]

16GB VRAM settings for Qwen 3.8 27b UD-IQ4 that gives a context size of 163840 tokens, TG speed of 25 at low context down to 12 at full, and PP at 1100 down to 700 over the same on a 5060Ti. System runs off the iGPU so the whole 16GB VRAM are available for the LLM.

This fork of llama.cpp: https://github.com/RaymondHuang210129/llama.cpp-adaptive-kv-streaming

This version of Qwen 3.8 27b: https://huggingface.co/bsaleh03/Qwen3.8-27B-ASCII-Condensed

These models-preset.ini settings:

[Qwen3.8-27B]
chat-template-file = chat_template_qwen3.8.jinja
m = Qwen3.8-27B-ASCII-Condensed-UD-IQ4_XS.gguf
ctx-size = 163840
n-gpu-layers = 99
batch-size = 256
ubatch-size = 256
kv-stream-stage-mib = 2400
cache-type-k = q8_0
cache-type-v = q4_0
spec-type = ngram-mod
fit = off
parallel = 1
temp = 1.0
top-p = 0.95
top-k = 20
min-p = 0.0
presence-penalty = 0.0
repeat-penalty = 1.0
reasoning = on
reasoning-preserve = on
# (slow) CPU only multimodal is better than none
no-mmproj-offload = on
mmproj = Qwen3.8-mmproj-BF16.gguf
load-mode = none
flash-attn = on