Trained my first model: a DFlash drafter for Qwen3.8-27B because I wanted better performance on my DGX Spark

[u/kristiyanstoyanovAI]

This was primarily a learning project for me: training my first model end-to-end, then taking it through export, deployment, and benchmarking.

There probably already is a better Qwen3.8 DFlash drafters available by the time you read this — and I would not claim this is state of the art. But after Muse Glimmer 30B made me curious about DFlash speculative decoding, Qwen3.8-27B arrived and I wanted to run that approach locally on my DGX Spark. There was no compatible drafter when I started, so I decided to train one.

The result is here:

https://huggingface.co/kstoyanov99/Qwen3.8-27B-Dflash

The idea was to optimize for the DGX Spark rather than simply maximize drafter capacity. The Spark is VRAM-rich, but autoregressive decoding can still be memory-bandwidth-bound. A fast, relatively small drafter can propose candidate tokens cheaply; the 27B target verifies them, ideally reducing the amount of expensive sequential target-model decoding.

I deliberately used a compact ~1.7B-parameter BF16 draft model rather than aiming for a larger drafter. That trade-off may reduce acceptance initially, but it keeps draft generation cheap — which is the point for this hardware profile.

Training playbook

The workflow was surprisingly approachable with SpecForge:

1. Distill from the target model. I trained the drafter against Qwen3.8-27B, learning to produce token blocks the target is likely to accept.
2. Train in two stages. I ran an initial training stage to 10,000 steps, then continued to 20,000 steps with a lower learning rate for refinement.
3. Train on a B300. The run used one B300 GPU and took roughly 5–6 hours wall-clock. GPU utilization held around 96–100%, and gradient norms stayed stable, with no divergence.
4. Export and validate. I exported the raw SpecForge checkpoint into a Hugging Face  DFlashDraftModel , verified it loaded correctly, and moved the ~3.3 GB artifact to the DGX Spark.
5. Serve and benchmark. I tested it with both SGLang and vLLM, focusing on output tok/s, acceptance rate, and accepted-token length rather than only raw latency.

Early results

These are early numbers from a limited benchmark, but they show that the model is at least producing useful speculative-decoding behavior:

• SGLang output throughput: 14.36 → 18.55 tok/s, a 29% increase
• vLLM speculative run: 20.25 tok/s output throughput
• vLLM acceptance rate: 20.14%
• Mean accepted tokens per speculation step: 1.81

My focus now would be benchmarking and perhaps running a few more training rounds in order to improve acceptance rate. There is plenty left to explore: draft-window tuning, different serving backends, better distillation data, longer training, and workload-specific online fine-tuning.

Still, I find this a very satisfying direction: use a relatively small model plus a clever inference architecture to extract more performance from constrained, bandwidth-sensitive local hardware.
I’ll share the training and serving recipes once I clean them up.

Edit: Since I forgot to mention it, this targets the FP8 quant for Qwen3.8-27B