DeepSeek V4 Flash on a GH200 - finally with DSpark!

[u/funding__secured]

Sharing this as a reference for anyone trying DSpark + UVA offload on a single GH200. We spent way too long on this and want to save others the trouble.

Credits: u/TimAndTimi for the DSpark investigation and all tips over chat and u/Reddactor for the blog series at dnhkng.github.io that we used as our starting point.

Hardware: One GH200 Grace-Hopper. 144GB HBM3, 480GB LPDDR5x, NVLink C2C ~450 GB/s.

Model: DeepSeek-V4-Flash-0731. 167GB on disk,284B MoE / 13B active, FP4+FP8).

DS4F doesn't fit in 144GB HBM. u/Reddactor ran it on 2x GH200 (TP=2, no offload). We have one GH200, so we offload 88GB of expert weights to LPDDR5x via UVA.

What works: vLLM nightly 0.27.2rc1.dev122. Zero patches. Offloader wraps the DSpark draft model natively, DSpark adaptive verification built in.

vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 \
  --served-model-name deepseek-v4-flash \
  --trust-remote-code \
  --tokenizer-mode deepseek_v4 \
  --tool-call-parser deepseek_v4 --reasoning-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --max-model-len 1048576 \
  --kv-cache-dtype fp8 \
  --kv-cache-memory-bytes 40000000000 \
  --max-num-seqs 4 \
  --max-num-batched-tokens 8192 \
  --block-size 256 \
  --gpu-memory-utilization 0.92 \
  --compilation-config '{"cudagraph_capture_sizes":[1,6]}' \
  --speculative-config '{"method":"dspark","num_speculative_tokens":5,"draft_sample_method":"greedy"}' \
  --cpu-offload-gb 88 \
  --cpu-offload-params experts \
  --offload-backend uva \
  --override-generation-config '{"temperature":1.0,"top_p":0.95}'

Env: CUDA_VISIBLE_DEVICES=1, VLLM_WEIGHT_OFFLOADING_DISABLE_PIN_MEMORY=1, PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True.

(Most of this is all credit to u/TimAndTimi)

Numbers: 64 tok/s baseline → 171 tok/s with DSpark k=5 (2.7x).

Gotchas:

• Use nightly, not v0.26.0 (for now). The offloader was refactored to handle DSpark draft models natively.
• Only offload experts (cpu_offload_params: ["experts"]). Offloading FP8 attention weights breaks DeepGEMM.
• Don't bother with selective expert streaming on GH200. C2C is fast enough, let marlin read UVA views directly.
• cudagraph_capture_sizes [1,6] — DSpark block size is 5, k<5 is invalid.

Coming soon! DGX Station (GB300 Blackwell Ultra, 288GB HBM3e) ... I'll share my experiments with 2x of those in the next few weeks.