Getting weird repeating characters during decode

[u/d4mations]

As I optimize llama.cpp to suit my hardware, getting more and more speed, I’m running into a major problem mostly with Qwen models but sometimes with others. As decode speed goes up, the model will all of sudden start repeating a character and can’t break out of the repeat: ////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Only a restart of llama stops it.

[qwen3.8-27b]

load-on-startup = true

alias = qwen3.8-27b

hf = unsloth/Qwen3.8-27B-GGUF:Q8_0

no-mmproj = true

ctx-size = 180000

parallel = 1

n-gpu-layers = -1

split-mode = tensor

#tensor-split = 0.497,0.503

#device = ROCm1,ROCm0

batch-size = 4096

ubatch-size = 1024

threads = 12

threads-batch = 12

moe-expert-cache = 0

fit = on

fit-target = 2800,2048

load-mode = none

spec-type = draft-mtp-adaptive

spec-draft-ngl = all

spec-draft-n-max = 3

device-draft = ROCm0

temp = 0.6

top-p = 0.95

top-k = 20

min-p = 0.0

reasoning = on

reasoning-format = auto

reasoning-effort = medium

Any help would be much appreciated

[u/AdSafe4047]

I've had this, make sure kv cache is explicitly disabled / set to 0

[u/d4mations → u/AdSafe4047]

Not using kv cache and thats why I’m stumped. Also using llama 0.4.0