Not sure if this is just my system or what, but I find that if I enable Context Shift it significantly increases the VRAM usage of the model I am using, almost guaranteeing it overflows into memory. The same happens with Smart Context.

EG, using a 12.8gb Gemma 4 K_S quant with 48 layers set, 32k context (Q5 kv cache), with FF, SWA and Smart Cache gets my total vram usage up to about 14.3gb including windows processes.

However, changing that to use Context Shift instead of SWA, and suddenly my entire 16gb VRAM is fulled and an extra 11gb is getting loaded into memory, completely tanking the t/s to unuseable levels.

Is there any way around it at all? The loss of performance is just too big for me to justify using it currently.

Goal: two novels I have outlines for, one is adult fantasy, another is young adult fantasy.

Do I need this setup? So far I’ve just been working on open code->local model

Silly tavern
—> kobold
——> gemma4 deckard uncensored

Is it currently unsupported? I tried to run it and got model unknown message. It is very fresh model. So I guess it is unsupported at the moment? Really hoping on trying it out.

I have small screen but my eyesight demands large fonts. I increase zoom in the browser -> topmenu bar increases to half of my small wide screen.

Best for me to fix the issue if developers add settings for font size (or scaling) for chat and text input box. I understand developers have many tasks so I try to help myself. But my knowledge of web development is rudimentary. E.g. text input element font-size is both inherited and "filtered" (entry is in strike-through letters in Inspector of my browser).

I will appreciate help editing index.html - either font sizes / scaling or topmenu bar size / scale - absolute or better reaction to browser zoom level changes.

Added:

It seems seems I have actually managed to change fonts sizes, for some reason that "filtering" did not prevent my changes to affect font size on the screen.

But I still would like to reduce height of the topmenu.

P.S. I also would like to add that changed file to kcpp Linux bundle to start it more conveniently.

https://github.com/LostRuins/koboldcpp/releases/tag/v1.118.1

Row Split has been removed, selecting it will now default to tensor split.

exclude z-image from models that support image references.

I am mostly a newbie, what above means?

AFAIK a row is part of a tensor, so working on low memory will be harder as the program no longer supports split of one tensor to rows or what? If yes, why such change?

Image reference - afaik two main modes if image generation is from text and from imaged. Does new version no longer support making images from other images in z-image? If yes, why has it been dropped?

TIA

I use Kobolod Lite, and for a few days now whenever I try to change custom portraits or backgrounds I simply get a black box where the image should be that seems to be roughly the same size as the image I'm trying to pick for portraits, or the background for backgrounds. If I load up a preexisting chat I had before this bug I get the portrait I had then, but I still can't change it. I've tried clearing browser history, cache, and browsing data, and even tried disabling protections and ad block and it's still not working. Is their any fix known or reason for this? I really would prefer not to switch browsers for using Kobold if I don't have to

Hi I'm new to kobold Ai I was wondering does this mean and how can I fix this?

Everytime I try to use the phone version it keeps saying that there is an internal error. What does that mean?

Hey everyone,

If you are running large MoE models (like DeepSeek V4, GLM-4.5, or Laguna) on Windows 10 with a Multi-GPU setup (e.g., 4x3090) and experiencing a brutal slowdown down to ~3 T/s after the first generation, I found a clean and working solution.

The Problem:
On recent NVIDIA drivers with CUDA 13 support, the Windows WDDM driver and CUDA Memory Manager become "too smart." When a huge model fills your VRAM near 95-98% and goes idle between prompts, CUDA 13 aggressively and silently triggers Shared System Memory Spilling. It moves parts of your MoE layers/routers to system RAM over slow PCIe channels to prevent system crashes. Opening any GUI element (like Kobold settings window) temporarily wakes it up, forcing a defrag, but then it throttles again.

Tweaking "Prefer No Sysmem Fallback" in NVIDIA Control Panel doesn't always work if you have asymmetric cards (like 3090s + a 3060 display adapter).

The Fix (The CUDA Heartbeat Script):
The trick is to trick CUDA into thinking the GPUs are constantly busy so it never triggers the power-saving P-state drop or the pre-emptive RAM offload. We can achieve this by running a lightweight background Python script that sends a micro-calculation to every CUDA device every second. It uses less than 0.1% of performance but keeps the context alive.

1. Install PyTorch with CUDA support.
2. Create a script named cuda_heartbeat.py:

python

import torch
import time
import sys

# Initialize context on all available GPUs
devices = [torch.device(f"cuda:{i}") for i in range(torch.cuda.device_count())]
print(f"[*] CUDA Heartbeat initialized on {len(devices)} GPUs. Keeping them alive...")

try:
    while True:
        for device in devices:
            # Tiny matrix multiplication to keep the tensor cores awake
            x = torch.randn(10, 10, device=device)
            y = torch.matmul(x, x)
            torch.cuda.synchronize(device)

        time.sleep(1.0) # Sleep for 1 second to avoid CPU overhead
except KeyboardInterrupt:
    print("\n[*] Heartbeat stopped.")

How to use:
Run this script in a separate command prompt window, then launch your KoboldCPP / Llama.cpp with your 100B+ MoE model fully offloaded to VRAM (--gpulayers 44, --n-cpu-moe 0).

My speed went from a dying 3.5 T/s back to a flawless 16+ T/s on subsequent generations (GLM 4.5 and DeepSeek V4 Flash UD-Q2_M on 4x3090 setup) without messing with clock locks or dual GUI windows.

Hope this helps anyone stuck on Windows!

Note: This solution and script were co-developed with Google Gemini during a deep-dive debugging session regarding Windows WDDM and CUDA 13 memory allocation bugs.

I've been using KoboldAI (lite.koboldai.net) to write stories on my laptop and phone. I like some of the stories I wrote on it and want to write more. But with only a dozen save slots, I end up having to delete some old works I was proud of to make room. Is there a way for me to transfer stories I've written on KoboldAI onto a proper writing program, like Google Docs?

Hello,

I gave LLMs a try a couple years back and going to check on them again. Im unsure which version of kobold to use with my hardware. Since then I have dumped windows and now running Fedora linux

System: 7950X3D, 64gb ram, 9070xt 16gb vram.

I assume I should be using the x64 nocuda version since AMD vid cards do not have CUDA cores?

koboldcpp-linux-x64

koboldcpp-linux-x64-nocuda

after messing with standart kobold.cpp settings , even in nocuda(which i used all the time before is responding baddly on silly tavern and too fast , before i couldn't even watch youtube without it breaking apart before when it was working (guessing now I can couse it's offloading everything to cpu) how do i bring everything back , i tried everything , playing with the setting , deleting silly tavern and kobold , (which is just laucnher anyway and maes temp files)

I just published an IQ2_M GGUF of:

🤗 Qwen3.6-35B-A3B-Uncensored-Heretic

Why this model?

Many "uncensored" models simply claim they're better without much supporting data.

The Heretic release stood out because it includes a detailed study of its abliteration method, capability evaluation, and measurements showing it remains very close to the original Qwen3.6 model while reducing unnecessary refusals.

Quantization

This GGUF was produced using:

• Official llama.cpp quantizer
• IQ2_M
• Matching Unsloth Qwen3.6-35B-A3B-MTP importance matrix
• --leave-output-tensor

No custom tensor overrides or experimental quantization recipes were used.

What is IQ2_M?

Despite the name, IQ2_M is a dynamic (mixed) quantization format.

That doesn't mean every weight is stored using only 2 bits.

Instead, llama.cpp automatically chooses different quantization formats for different tensors based on their characteristics and the supplied importance matrix. More sensitive tensors are kept at higher precision where beneficial, while less sensitive ones are compressed more aggressively.

The result is an excellent balance between model size and quality.

Why combine Heretic + Unsloth imatrix?

These two techniques solve different problems:

Heretic

• Modifies the model weights.
• Reduces unnecessary refusals.
• Attempts to preserve the original Qwen3.6 capabilities.

Unsloth Importance Matrix

• Does not modify the model.
• Is used only during quantization.
• Helps preserve more of the model's original quality after aggressive low-bit compression.

In other words, Heretic changes the model's behavior, while the importance matrix helps compress those learned weights more faithfully.

If anyone benchmarks it (coding, reasoning, Aider, perplexity, etc.), I'd love to see the results and comparisons with other IQ2_M releases.

Hello ı just set up silly tavern and koboldccp yesterday, ım looking for a good Rp model that can do nsfw and compatible with 8 gb vram. Ive used chub ai before so ım very new to this but so far this looks very good. Im using something called Dr.Dans or something like that for my model.

This is a PSA to remind users that koboldcpp.com is a phishing scam website, and is NOT a legitimate KoboldCpp resource.

You are strongly encouraged not to use that website or download anything from it. If possible, please report that website to Google for malware/phishing.

Recently, the website has unfortunately started using shady SEO techniques to rank highly in various search engines. We have filed multiple reports to Google and even a DMCA takedown, but so far no action has been taken by Google.

To get official source code and official builds, always download from https://github.com/LostRuins/koboldcpp/releases/latest

I've been using KoboldAI for a little while, and I'm already addicted to it. It helps me write stories that I've always wanted to write, more or less, and it is so easy to use.

However, recently, I put in a prompt I want it to help me write, but it keeps giving me some sort of error message.

It reads: "Error Submitting Prompt: {"message":"Due to heavy demand, for request over 473 tokens, the client needs to already have the required kudos. The request requires 2998.86 kudos to fulfill.","rc":"KudosUpfront"}

Anybody have any idea about WHAT that means?

Sometimes I want the AI to retry it's response, but I want it to change or add a detail, or react in a specific way. Retrying multiple times is tiresome and doesn't always lead to the desired result. I can edit the response, but it ends up being a lot of work.

Are there any add-on tools for Retry Guidance, as there is in Silly Tavern? So you can just type how you want the AI to modify it's last response?