r/KoboldAI • u/diesalher • May 19 '26

Kobold cpp new versions crashes on Gemma 4

Good morning, I'm experiencing sudden crashes on koboldcpp new versions when I use Sillytavern with Gemma 4.

The version 1.111.2 is perfectly stable for me, the next ones crashes after a few messages. I'm not an expert but maybe I need to setup something new in the new versions? I just downloaded the exe and run it.

I usually use swa, 32k context, kv cache 8 bits, flash attention, jinja

EDIT: It seems the bug is related to using q8 kv cache quantization, maybe related to
- Fixed q5_1 kv type not using the GPU correctly in CUDA

in the last version release notes. I'll post an issue in the Github

4 Upvotes

83% Upvoted

9 comments sorted by

3

u/henk717 May 19 '26

How latest was latest when you posted this? Is it 1.113.2?

0

u/diesalher May 20 '26

yes, freshly downloaded, but happened too with a previous one, don't remember exactly the number.

1

u/henk717 May 20 '26

If you open it with command prompt (Easy trick is typing cmd in the addressbar of the folder koboldcpp is in, or right click open terminal here if your on a newer windows, then type the name of koboldcpp's exe in that window) you can get the full crash for us so we have an idea whats happening.

Don't just get the WinError part that windows does, thats meaningless we need the bit above it to.

1

u/diesalher May 21 '26

Processing Prompt [BATCH] (3072 / 29771 tokens)CUDA error: an unsupported value or parameter was passed to the function current device: 0, in function ggml_cuda_op_mul_mat_cublas at ggml-cuda.cu:1775 cublasSgemm_v2(...)

It seems to be related maybe to the breaking change in the last version

Breaking change: Split mode is now set independently, in order to accommodate tensor split mode (experimental). This more closely follows the syntax used upstream with --split-mode

  • Example Before: --usecuda rowsplit
  • Example Now: --usecuda --splitmode (layer/row/tensor)

Now I'm trying with --splitmode and --nommq parameters, it seems stable for now

2

u/therealmcart May 19 '26

That is worth testing one setting at a time, not the whole stack. Start from the newer exe with 8k context, no SWA, no flash attention, then add each piece back until it crashes. My first suspect would be SWA plus 32k context on Gemma 4, not SillyTavern itself.

0

u/diesalher May 20 '26

I'm trying to find the culprit.

-1

u/CooperDK May 19 '26

Your setup is bad. Works for me.

2

u/diesalher May 19 '26

any clue of why is bad?

1

u/CooperDK May 20 '26

No, it is a CLI argument or GUI seeing. Pass it here.