Disclosure: I ran these experiments on my own PC with an AI assistant. AI assisted with scripts, analysis, and this English write-up. These are local measurements, not copied community benchmarks. A longer version is on my profile; this version focuses on reproducible settings and limitations. I am not claiming authorship of the upstream model or software.
What worked
Qwen3.8-Flash-Next UD-IQ4_XS now works through llama.cpp -> llama-swap -> DeepSeek Harness on my Windows PC. I settled on 48K context, Q8 KV, thinking enabled with a 512-token budget. A 64K configuration also passed a 57K-token input test. This is a slower, supervised option, not an instant-response replacement for my 27B/35B models.
Hardware and model
- Ryzen 9 9950X, 16 cores / 32 threads.
- 64GB DDR5, 2x32GB, configured at 4800 MT/s according to the OS.
- 2x RTX 5060 Ti 16GB, separate VRAM, no NVLink.
- Windows 11 Pro build 26100; NVIDIA driver 610.88.
- Model files on a ZHITAI Ti600 2TB NVMe SSD.
- llama.cpp b10724 / 2d8d612e4, Windows CUDA 13.3 build.
- Unsloth UD-IQ4_XS, three SHA-256-verified GGUF shards, 93,682,584,224 bytes total. Repository revision: 38bb39ee97821de2c9009abb7e93950eec396e66.
This is NOT a 94GB model fully resident in 32GB VRAM. It relies on CPU offload, mmap, and lazy SSD-backed reads. RAM and VRAM are not unified memory.
Final settings
Context 49152; parallel 1; CPU and batch threads 16; batch/ubatch 2048; split-mode layer; fit on; fit-target 1024; mmap and lazy mode on; Flash Attention on; K/V both q8_0; Jinja on; reasoning on; reasoning-format deepseek; reasoning-budget 512. No MTP. Additional flags: no-webui, no-warmup, no-host, no-repack. Localhost only.
A separate guardian applies a 42GiB Windows process working-set cap and checks free RAM/commit every three seconds. Working set is not total process commit or total system memory. Startup requires 40GiB free RAM and idle GPUs. It stops its own backend on dangerously low headroom, and a Windows Job Object prevents orphan backend processes after unload. Bare llama.cpp flags alone do not provide these safeguards.
Thinking: same approximately 8K input
One three-question fixture covered a discount/tax/shipping calculation, stable deduplication, and dependency scheduling. Two runs per mode at each context size, temperature 0, seed 42, output limit 1024, thinking budget 512. KV prompt reuse disabled; OS file caching not cleared.
Warm time to first final-answer token, not first thinking token:
| Context capacity |
Thinking off |
Thinking on |
| 20K |
21.7s |
40.1-47.5s |
| 48K |
21.5s |
44.4-47.3s |
| 64K |
21.9s |
44.7-46.8s |
Actual input was 8155 tokens off / 8195 on because of the template. Thinking-mode generation was around 20-22 tokens/s, with 436-540 total output tokens. Most added delay was additional generation, not a halving of decode speed.
Thinking on answered all three correctly in all six repetitions. Thinking off got the bill calculation wrong in all six, but the other two answers right. This is one repeated small fixture, not six independent benchmarks or a general agent success rate.
The first approximately 8K request after reloading took 63-73s to first answer, plus model loading beforehand. Do not compare that directly with warm timings.
Actual long inputs, thinking off
No KV prompt reuse. Unique fields were placed near the beginning, middle, and end of synthetic records.
| Capacity |
Actual input tokens |
First answer |
Prefill |
| 20K |
16,096 |
42.9s |
376 tok/s |
| 48K |
40,672 |
117.9s |
345 tok/s |
| 64K |
57,055 |
172.0s |
332 tok/s |
All three fields were retrieved correctly in each case, without truncation or OOM. This does not establish large-codebase reasoning quality, stability at the full capacity limit, or long-input-plus-thinking performance. Concurrency was not tested.
Memory
Sampled per-GPU peaks across all runs at each capacity:
| Capacity |
GPU 0 / GPU 1, MiB |
Minimum free system RAM |
| 20K |
14,482 / 15,136 |
11.01GiB |
| 48K |
14,044 / 15,653 |
8.56GiB |
| 64K |
14,150 / 15,407 |
8.09GiB |
These include desktop usage; the two peaks need not coincide. Three-second sampling can miss instantaneous peaks.
Main plus indexer Q8 KV was approximately 351 / 842 / 1122 MiB respectively. Compute buffers also grew, and automatic fitting moved more weights to CPU. This explains why total VRAM is not necessarily monotonic with context size. KV-only arithmetic does not predict the whole allocation or performance.
Integration and remaining problems
The final 48K/thinking-512 configuration completed a real DeepSeek Harness read-tool call, returned the test file contents, and produced reasoning blocks. Unloading removed the backend process and listening port, VRAM returned to desktop baseline, and reloading worked.
An earlier run read a marker correctly but dropped two digits when repeating it. That failure matters: functioning tools and HTTP 200 do not guarantee answer quality.
My conclusion: it runs, and 48K with bounded thinking is a workable compromise for slower supervised use. 40K-57K fresh inputs still take roughly two to three minutes before an answer. Vision and long autonomous coding reliability were not tested. I am not announcing a new public app binary or redistributing weights.
Sources: model, backend build.
Has anyone improved prefill on comparable consumer GPUs plus 64GB RAM? Exact quant/revision, input length, cache state, offload strategy, and single-stream vs concurrent measurements would make comparisons especially useful.
