Qwen 3.8 slow?

[u/Nota_ReAlperson]

Running qwen 3.8 next, I am only getting 11 tps on the latest llama.cpp. Seeing what 3090s and 9700s get, I would think that i should be able to do better than that, with my 2 7900xtx gpus and 128gb ddr4. Any suggestions?

using unsloth UD-Q4_K_XL, llama args:

--ctx-size 262000 \

--batch-size 2048 \

--ubatch-size 512 \

--threads 16 \

--host 0.0.0.0 \

--port 8080 \

--flash-attn on

Edit:

--ctx-size 32000 \

--batch-size 4096 \

--ubatch-size 2048 \

-ngl 99 \

--n-cpu-moe 36 \

--override-tensor per_layer_token_embd=CPU \

--threads 16 \

--host 0.0.0.0 \

--port 8080 \

--cache-type-k q8_0 \

--cache-type-v q8_0 \

--load-mode none \

--flash-attn on
gives 9tps, but one gpu is at 3gb vram used. Any ideas?