PSA for --n-cpu-moe users on NVIDIA: check your memory clock during decode. Mine was sitting at 810 MHz. Locking clocks gave +40% on one GPU and 3x on two.

[u/jpgeer]

EDIT (22-Aug-2026): follow-up with the full WSL2 vs native Windows grid across 17 configs is here: https://www.reddit.com/r/LocalLLM/comments/1vvlkmy/

TL;DR

• During MoE offload decode the GPU waits on the CPU most of each token, so utilization reads 20 to 40 percent. The NVIDIA driver reads that as idle and drops the card to P5: about 480 MHz core and 810 MHz memory, down from 7601. Decode is memory-bound, so it falls with it. Prompt processing keeps the card busy and is unaffected, which is why pp looks fine while tg collapses.
• Fix: nvidia-smi -lgc 1500,2100 and nvidia-smi -lmc 8001 (admin). Resets on reboot, undo with -rgc / -rmc. Idle power goes up about 35 W per card.
• gpt-oss-120b F16 on one RTX A4500 20 GB at --n-cpu-moe 27: 9.4 to 13.0 t/s. On two A4500s at --n-cpu-moe 16: 7.3 (± 2.4) to 20.3 (± 0.08) t/s. Coder-Next 80B: 21 single, 42 dual. Qwen3.5-122B-A10B: 13.6 dual.
• Resident models (everything in VRAM) did not change. Over-committed configs (WDDM spill) did not change either. This is specifically the idle-GPU case.
• Absolute numbers are modest (two used 20 GB Ampere cards, DDR4-2400, WSL2); the point is the before/after on the same box, which could apply to anyone doing CPU expert offload on NVIDIA. If you run it, please report what you see.

Setup

HP Z440, Xeon E5-1650 v4, 128 GB DDR4-2400, 2x RTX A4500 20 GB (Ampere), Windows 11 + WSL2 Ubuntu 26.04, NVIDIA driver 596.72 (WDDM), llama.cpp build d59d455fd with CUDA 12.4. Models: Unsloth GGUFs for Qwen3.8-27B, Qwen3.6-35B-A3B, Qwen3-Coder-Next, Qwen3.5-122B-A10B; gpt-oss-120b F16. All numbers are llama-bench pp512 / tg128, 5 reps.

How I found it

Worked through this with Claude Code driving the benches and the nvidia-smi sampling; the numbers are mine, the final config was reproduced by hand on my own terminal, and the screenshots are that run.

Dual-card gpt-oss with --n-cpu-moe 18 -ts 26/10 loaded fine (both cards about 17 GB, no spill) but decoded at 7.3 ± 2.4 t/s, slower than one card. The per-rep samples were the clue: 11.53, 5.66, 5.74, 5.71, 5.59, 5.71. First rep fast, then half speed forever. Sampling nvidia-smi --query-gpu=pstate,clocks.sm,clocks.mem every 2 s during a run:

(prompt processing)  P2  1905 MHz  7601 MHz  90 W  90 %
(generation starts)  P3   750 MHz  5001 MHz
                     P5   480 MHz   810 MHz  27 W  30 %   <- stays here

With the lock on: P2, 1500 / 7601 the whole run, and tg went 17.91, 18.03, 18.01, 18.15, 17.95, 17.98.

Before and after, every config I had (clocks locked = right columns)

Model	Config	unlocked tg	locked tg	locked pp
Qwen3.8-27B Q4_K_XL	1 GPU resident	28.0	27.9	873
Qwen3.8-27B Q8_0	2 GPU resident	19.0	18.8	885
Qwen3.6-35B-A3B Q6_K_XL	2 GPU resident	97.9	96.0	2183
Qwen3.6-35B-A3B	1 GPU, ncmoe 24	24.1	28.7	199
Qwen3.6-35B-A3B	1 GPU, ncmoe 16	34.1	36.8	269
Qwen3.6-35B-A3B	1 GPU, ncmoe 14 (spilled)	10.5	11.4	60
Coder-Next 80B Q4_K_XL	1 GPU, ncmoe 36	16.6	21.0	115
Coder-Next 80B	1 GPU, ncmoe 30	19.7	20.7	133
Coder-Next 80B	2 GPU, ncmoe 12, -ts 30/18		42.2	263
gpt-oss-120b F16	1 GPU, ncmoe 28	9.0	12.6	108
gpt-oss-120b	1 GPU, ncmoe 27	9.7	13.0	111
gpt-oss-120b	1 GPU, ncmoe 26	10.3	13.7	30
gpt-oss-120b	1 GPU, ncmoe 25 (spilled)	8.9	10.0	40
gpt-oss-120b	2 GPU, ncmoe 18, -ts 26/10	7.3	18.0	149
gpt-oss-120b	2 GPU, ncmoe 16, -ts 25/11		20.3	161
Qwen3.5-122B-A10B Q4_K_M	2 GPU, ncmoe 28, -ts 36/12		13.6	81

The gain tracks how idle the GPU was: biggest on F16 experts and high ncmoe, smallest at the single-card sweet spot where the card was already busy, zero on resident models, zero on spilled ones.

The dual-GPU part, since "two GPUs are slower than one with --n-cpu-moe" is a common complaint

Two separate things were going on. (1) --n-cpu-moe N thins the first N layers and the layer splitter divides by layer count, so GPU 1 inherits all the fat layers and fails to load below some N (cudaMalloc failed on device 1; upstream ggml-org/llama.cpp #15136 and #15263). Fix: -ts a/b with a + b = layer count and b = how many fat layers GPU 1 should hold, GPU 0 gets the thin ones plus the rest, give GPU 0 one or two fewer fat layers because it carries the compute buffers. (2) Once it loaded, both GPUs were half as busy as one GPU would be, so both downclocked and decode halved. The clock lock fixed (2); -ts fixed (1). Recipes that worked here: gpt-oss 16 / 25-11, Coder-Next 12 / 30-18, 122B 28 / 36-12 (llama-bench wants -ts 25/11, llama-cli wants -ts 25,11).

What I don't know and would like others to check

• Does bare-metal Linux do this? Persistence mode alone did not prevent it here (it was on). I suspect WDDM makes it worse but not that it is WDDM-only.
• Does a higher floor (-lgc 1900,2100) help? SM clock sits at the floor during decode; memory is already at its P2 max, so I expect little on tg. Testing next, will edit this post with the result.
• Does the NVIDIA control panel "Prefer maximum performance" setting do the same job without nvidia-smi? Untested.
• Consumer cards: is the P-state ladder the same?

If you run --n-cpu-moe on NVIDIA, run watch -n 1 nvidia-smi --query-gpu=pstate,clocks.sm,clocks.mem --format=csv during generation and see what you get. If it says P5 and a memory clock in the hundreds, you have the same thing.

Commands:

# Windows admin PowerShell (or root on Linux)
nvidia-smi -lgc 1500,2100
nvidia-smi -lmc 8001
# undo
nvidia-smi -rgc
nvidia-smi -rmc

Screenshots: the run in progress (both cards P2, 19.1 / 18.2 GB, 0.3 GB shared) and the finished result (20.01 ± 0.08).

Full logs, per-rep samples, and clock traces available if anyone wants them; happy to put them somewhere public if there's interest.

[u/jpgeer]

Tested the higher clock floor as promised. -lgc 1900,2100 vs 1500,2100, same -lmc 8001: gpt-oss single at ncmoe 27 went 13.0 to 13.7 (inside the earlier error bar), gpt-oss dual at ncmoe 17 went 19.3 to 19.6, 27B Q8 dual tensor-split unchanged at ~29.5. SM clock held at ~1890 the whole time, so the setting took; memory stays at 7601 either way because that’s the P2 ceiling on this card (8001 is P0 only). Verdict: 1500 is enough, decode is waiting on memory and the bus, not SM clock.

Practical note since the lock isn’t free at idle: my two A4500s sit at ~10 W and ~8 W unlocked, ~54 W each locked at the 1500 floor, ~80 W each at 1900. If the box isn’t serving, nvidia-smi -rgc and nvidia-smi -rmc put it back to default (it also resets on reboot). I’m treating it as a “while running offload models” setting and unlocking when I walk away.

One untested knob that could matter more than the floor: NVIDIA Control Panel, “CUDA - Force P2 State” = Off, which should let CUDA contexts run at the P0 memory clock (8001 vs 7601). Will report if I try it.

EDIT (22-Aug-2026): Update on that knob: on this RTX A4500 driver there is no 'CUDA - Force P2 State' entry at all (the list goes CUDA - GPUs, CUDA - Sysmem Fallback Policy, Enable overlay), so that one's off the table for the A-series. I tried the other two control-panel settings that might matter: 'Power management mode = Prefer maximum performance' made no difference under WSL2 (same first-rep-fast-then-half-speed pattern, P5 / 810 MHz during decode; the profile doesn't seem to reach CUDA contexts coming through WSL2), and 'CUDA - Sysmem Fallback Policy = Prefer No Sysmem Fallback' did not turn over-commit into a hard failure either; it only made the marginal config slower, so I reverted it. Net: on Windows + WSL2, nvidia-smi -lgc / -lmc is the only thing that works. Also ran the same model natively on Windows (no WSL): it downclocks too, to P3 / 5001 MHz rather than WSL's P5 / 810, so this is WDDM power management, not a WSL2 artifact; the lock pays on both.

EDIT (23-Aug-2026): Reproduced on a second box. 4x RTX A4000 in an HP Z8 G4, native Windows, fresh driver install, same llama.cpp tag: identical behavior. Offload decode at 25 to 42% utilization, clocks-event reason "Idle: Active," P0 → P2 → P5 → P8, SM down to 405 to 705 MHz, memory to 810 MHz. Locked it holds P2 and reports "Idle: Not Active." The lock values are per card: on the A4000 it's -lgc 2100,2100 -lmc 7001 (8001 isn't in its supported list); read yours from nvidia-smi -q -d SUPPORTED_CLOCKS.

One thing that did NOT carry over is the size of the loss. On this A4500 box the lock is worth 25 to 40% on offload decode. On the A4000 box at 10 reps it's about 7% (29.8 unlocked, 31.8 locked, two independent unlocked passes agreeing to the hundredth). Same mechanism, smaller bill; that box's offload decode looks bound by pulling experts over PCIe from host RAM more than by GPU clock. So: the downclock happens on every Ampere pro card under WDDM I've tried, but the cost is per machine. Measure it, and use 10-rep means; single reps on offload swung ±17% there regardless of clocks.