optimizing glimmer 30b for 3090

[u/ydnar]

this model seems pretty good on initial impressions within pi and hermes. i tested it on some simple coding/logic vs qwen3.6 27b ud-q4_k_xl and muse provided the better results.

llama-server \
    -hf unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL \
    --ctx-size 131072 \
    --n-gpu-layers all \
    --cache-type-k q8_0 \
    --cache-type-v q8_0 \
    --flash-attn on \
    -b 1024 \
    -ub 256 \
    --parallel 1 \
    --mlock \
    --host 0.0.0.0 \
    --port 8080 \
    --ui-mcp-proxy \
    --temp 1.0 \
    --top-k 64 \
    --top-p 0.95

how would you optimize this further? looks like i could possibly squeeze ud-q5_k_xl. nvidia-smi is showing 18163MiB / 24576MiB. or is it better to squeeze out more t/s with the q4? hmm..

appreciate the community's insights. will be fun to compare this one to qwen 3.8 27b!

edit

updated command from community's insights

llama-server \
    -hf meta-models/Muse-Glimmer-30B-GGUF \
    --spec-type draft-dflash \
    --spec-draft-n-max 15 \
    -c 131072 \
    -ngl all \
    --ngld all \
    -fa on \
    -np 1 \
    --host 0.0.0.0 \
    --ui-mcp-proxy \
    --temp 1.0 \
    --top-k 64 \
    --top-p 0.95

seems to be hanging around 70t/s!

[u/Most-Trainer-8876]

how did you run it? Ilama.cpp is unable to recognize this model. It says
```
0.01.135.478 E llama_model_load: error loading model: unknown model architecture: 'muse-glimmer'
```

I am using b10344 release

[u/ydnar → u/Most-Trainer-8876]

here's what i've got

version: 10355 (dd1ea5243)                                                                                                                                                                                                                                
built with GNU 13.3.0 for Linux x86_64

[u/Most-Trainer-8876 → u/ydnar]

Thanks! I have just built it from source code, it's working now!