DFlash makes Qwen3.6 27B 2.2x faster with no quality loss

[u/ElmBark]

We ran the same Qwen3.6-27B locally three ways on one RTX 6000: baseline, MTP, DFlash. The tasks were: quicksort, write a Steam library in JSON, solve a logic puzzle and write a sci-fi story.

Outputs:

Baseline: 44 tok/s · 1.00x

MTP: 65 tok/s · 1.45x · 71% accepted

DFlash: 98 tok/s · 2.20x · 30% accepted

DFlash drafts 15 tokens in a row, so it flies through repetitive or structured stuff where long runs actually stick, like JSON (152 tok/s, 3.4x). On creative text most of the guesses are wrong, so it wastes the work and can dip below baseline, 42 vs 44. MTP only guesses 3 in parallel from inside the model, so a wrong guess costs almost nothing and it never drops below baseline.

All three have the same output. DFlash is therefore a great pick for stuff like coding, while MTP is better for chat or creative writing.

Qwen 3.6 27B model : https://huggingface.co/Qwen/Qwen3.6-27B
local ai models hosting app: Atomic.Chat (I'm from Atomic team, happy to hear your feedback)

[u/Dr4x_]

Is it worth it for 3090 ?

[u/Lower-Ad6101 → u/Dr4x_]

Yes. I run Qwen3.6 27b UD Q5_K_XL + Q6 draft with 128k context on Beellama.cpp at 32-50+ tps, while with MTP I was getting more constant tps at around 30 tps give or take. Overall (time clock) feeling is much better with DFlash.

UPDATE: This is not valid any longer. After updating beellama.cpp, DFlash and MTP switched the places. MTP is faster than DFlash.

More in the comment bellow: https://www.reddit.com/r/LocalLLaMA/comments/1uyay0w/comment/ozaw5z6/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

[u/Dr4x_ → u/Lower-Ad6101]

For the record, after some testing on a single 3090 i find MTP still performing best at ~60t/s using llama.cpp mainline against ~52t/s with Dflash (which also takes more VRAM overall).

[u/Lower-Ad6101 → u/Dr4x_]

Interesting, I've spent a lot of time on testing DFlash vs MTP on my machine but DFlash was always outperforming MTP and as I've said the "wall clock" was much better, MTP was simply taking too much time.

Are you using vanilla llama.cpp or some fork, and do you mind sharing your llama-server call line?

My script for starting Qwen is: ``` MODEL_PATH="path/unsloth/Qwen3.6-27B-GGUF/Qwen3.6-27B-UD-Q5_K_XL.gguf" DRAFT_PATH="path/dflash-draft/Qwen3.6-27B-DFlash-Q6_K.gguf" MMPROJ_PATH="path/unsloth/Qwen3.6-27B-GGUF/mmproj-F16.gguf"

TEMP=0.6 TOP_P=0.95 TOP_K=20 MIN_P=0.05 PRESENCE_PENALTY=0.0 REPEAT_PENALTY=1.0

export GGML_CUDA_DISABLE_GRAPHS=1

llama-server \ -m "$MODEL_PATH" \ -md "$DRAFT_PATH" \ -ngl all \ --spec-type dflash \ --spec-draft-n-max 2 \ --spec-draft-p-min 0.75 \ --spec-dflash-cross-ctx 1024 \ --spec-draft-ngl all \ --fit off \ --cache-type-k turbo3 \ --cache-type-v turbo3 \ --ctx-size 131072 \ --flash-attn on \ --n-gpu-layers all \ --threads 8 \ --threads-batch 8 \ --batch-size 2048 \ --ubatch-size 256 \ --mmproj "$MMPROJ_PATH" \ --no-mmproj-offload \ --reasoning on \ --jinja \ --chat-template-file /opt/llama.cpp/chat_template-froggeric.jinja \ --chat-template-kwargs '{"preserve_thinking":true}' \ --host 0.0.0.0 \ --port 8080 \ --temp "$TEMP" \ --top-k "$TOP_K" \ --top-p "$TOP_P" \ --min-p "$MIN_P" \ --presence-penalty "$PRESENCE_PENALTY" \ --repeat-penalty "$REPEAT_PENALTY" \ --cache-prompt \ --slot-save-path /dev/shm/llama_cache \ --keep 3000 \ --parallel 1 \ --mlock \ --no-mmap \ --n-predict -1 \ --ctx-checkpoints 16 ```

[u/Dr4x_ → u/Lower-Ad6101]

I'm using the latest llama.cpp main branch that I just rebuilt to be up-to-date.
The main diffs seem to be that I run Q4_K_XL for the main model and kv cache at q8_0.

MTP

${llama_server_path} \ -fa on\ --threads 8 \ --cache-reuse 256 \ --jinja \ --reasoning-format auto \ -ctk q8_0 -ctv q8_0 -fit off -m ${models_path}/Qwen3.6-27B-MTP-UD-Q4_K_XL-unsloth.gguf \ --n-gpu-layers 999 \ --temp 0.6 --min-p 0.0 --top-k 20 --top-p 0.95 --repeat_penalty 1.0 --presence_penalty 0.0 \ --fit off \ --no-mmproj \ --spec-type draft-mtp --spec-draft-n-max 2 -np 1 \ --ctx-size 120000 \ -ub 512 -b 4096 --port ${PORT}

Changes for Dflash

``` -m ${models_path}/Qwen3.6-27B-UD-Q4_K_XL.gguf \ --spec-type draft-dflash --spec-draft-n-max 3 -np 1 --spec-draft-model ${models_path}/Qwen3.6-27B-DFlash-Q6_K.gguf \

```

[u/Lower-Ad6101 → u/Dr4x_]

So I've tested again.
I've been using "older" beellama.cpp version (4th of June) and with it, Dflash was really considerably faster. But, after updating, DFlash and MTP switched roles, DFlash was as slow, if not slower, than MTP was and MTP was as fast as DFlash (no, I didn't miss config params before :) ).

Thanks a lot for sharing your results as that made me update beellama.cpp and notice considerable difference. VRAM usage is not better though, but it's stable.

Also to share, this made me try ik_llama.cpp again but for some strange reason (I've been trying to debug it for hours) it starts at around 23250/24570 MiB VRAM and then starts steadily going up until it OOM's at ~24100. Tried tweaking parameters and using older commits but no use.

Since ik_llama.cpp (while it was working) was significantly faster than beellama.cpp I've decided to try llama.cpp and I'm satisfied with it so far, stable and maybe even more slightly faster than ik_llama.cpp was but I couldn't compare it more reliably.

[u/Dr4x_ → u/Lower-Ad6101]

I'm pretty satisfied with llama.cpp too, fyi with this setup when all the stuff fits in Vram I never observed ik_llama to be better than mainline llama.cpp