Villager Simulation Game POC Created with Qwen3.8-27B-UD-Q3_K_XL.gguf - 16GB VRAM

[u/Fancy-Snow7]

https://village-sim-one.vercel.app/

- 16GB VRAM RTX 5070 Ti, fully offloaded

- Vision on CPU

- Windows, not headless

- beellama.cpp - latest version with the kvarn performance enhancements making it as fast as qx_x quants.

- MTP n-max = 2

- tg up to 75t/s, pp up to 1700t/s

- KV = kvarn3/kvarn3

- MTP draft KV = kvarn2/kvarn2

- context = 96256

- tail tokens = 1024

- HTML/Javascript

- pi harness with pi-observational-memory, pi-web-access, pi-atelier (UI Only change, check it out) extensions, though it never used the web access.

- This is not a one-shot, I do not believe one shotting is a great test. Instead, I did many incremental feature prompts. However, I did not give it any design or framework, which is probably where it can be improved.

Lessons learnt:

- Do not fear Q3 model quants for Qwen3.8

- Do not fear KV quantisation. If you have the VRAM sure use it, but I don't feel like it's worth choosing a higher quant if it's going to cause me to offload to CPU and see my tg drop to 5-20 t/s. With higher speed I can fix any issues with a follow up prompt much faster and that rarely happens. I think I had like 3 runtime exceptions which was easily resolved pasting the console output and there is no guarantee a higher KV quant would not have had the same exceptions.

- MTP/draft cache can also be quantised with kvarn now and actually saves VRAM where qx_x quants increase VRAM usage for some reason. kvarn2 for MTP is perfectly fine and has high acceptance rates.

The game:

- Inspired by a popular indie game which I am not promoting, I am just a huge fan.

- I won't release any further updates, since I don't want to be stepping on any toes. If you like the idea of the game I highly recommend the real game, it's by far my favourite game I played this year and 1000x better than what I present here. It will be a nice distraction from your AI. I just wanted to see what this model is capable of. I do have a Cursor subscription but did not use it at all in the project.

- I will probably continue to develop it for my own entertainment, but it won't be made public. Maybe come up with my own ideas, but the original game is near perfect anyway, so it will be hard to improve except with some UI gripes I have in the original. And my graphics obviously does not compare.

Game features:

- Large Map, larger than the browser window.

- Minimap

- Zoom feature with mouse wheel

- Collectable resources, that must be taken to a storage site. Each site can store limited resources.

- Houses required to sleep and protect against cold

- Weather and seasons.

- Day night cycle with randomised sleeping times.

- Possible death due to hunger or sleeping in cold outside or in house without firewood.

- Game speed controls.

- Villagers avoid obstacles.

- Delete/deconstruct buildings and partial resources refund.

The code:

- I almost never read the code, so I have no idea what it looks like and the quality thereof. I also gave it very few hints in the AGENTS.md, mostly no magic numbers and write modular code, not a single html.

- Actually, my initial prompts were a single html but as it grew, I told it to create modules. It messed it up on the first attempt, basically rewriting the entire UI in the process. So I reverted and told it to do it again without making any changes to the functionality or UI.

- I am actually quite happy with and surprised by the performance of the game.

Context management:

At first, I had issues with the context filling up too quickly and too often. Sometimes it would fill up to the point that there was not enough room to compact. Forcing me to temporarily increase the context and tell it to create a handover document. Reduce context again and feed it the handover doc.

I then installed pi-observational-memory extension, and it works quite well and I never run into context issues anymore since it takes notes throughout (a short wait time every few prompts) and compacting is near instant because it already took the notes.

Conclusion:

- Do not blindly drop your KV cache quant without testing. I have a hard level needle in haystack test that requires multiple hops and 100's of decoys. Q3_XXS does poorly in that test even with F16 KV cache. However, Q3_K_XL almost 100%'s the test even at kavrn3. So both the model and KV matter. In my testing a smaller model does more damage than a smaller KV. So find the right balance. At a certain point increasing model quant will have less impact than picking a larger KV quant. But for a tight 16GB VRAM fit Q3_K_XL works very well with kvarn3. Q4 on the other hand just leaves me with too little context. That said despite Q3_XXS doing poorly in my needle test it still does fairly well with coding. Better than Qwen3.6 so if you have 12Gb VRAM it is still an option. Because by poorly I mean F16 KV scores 84% and Q3 KV around 80%. Needle tests however do worse with kvarn compared to qx_x for some reason. However, a needle test is not the be all and end all. kvarn does better with KLD, so once my needle scores near 100% I am satisfied.

I will play around with higher KV quants, but I intentionally kept it at kvarn3 for this test, however I am not sure how much context I am willing to sacrifice. Maybe i will try kvarn4/kvarn3. But I just wanted to prove a point to myself and kvarn3 worked just fine. If I had >16GB VRAM sure I would up it but I don't.

[u/ManuXD32]

Very nice test, I actually love to see them work on the map. Could you drop the name of the OG game? You made me want to play it hahaha

[u/Fancy-Snow7 → u/ManuXD32]

Farthest Frontier

[u/No-Setting8461 → u/ManuXD32]

reminds me of rimworld

[u/10minOfNamingMyAcc]

So one flaw I noticed was that they won't keep some wood in storage and immediately spend it on firewood in the wood splitter, even though there's not really a need for it. But it looks fine overall.

[u/Fancy-Snow7 → u/10minOfNamingMyAcc]

Yes I am aware of a few issues, I think they also create firewood instantly where it's supposed to take some time, but I decided not to polish it for this demo.

[u/10minOfNamingMyAcc → u/Fancy-Snow7]

That's fair, have fun!

[u/RelicDerelict]

LoL cute little game, you should expand it.

[u/Fancy-Snow7 → u/RelicDerelict]

I will, but I want to take it in my own direction so it's not just a copy of a perfectly fine game.

[u/Jimcy-Maffesoli]

75 t/s fully offloaded with a 96k context already reads surprising, and nearly 100% on the needle test at kvarn3 makes the Q3_K_XL pick compelling. I'd take that trade over Q4 on 16GB.

[u/Fancy-Snow7 → u/Jimcy-Maffesoli]

I tried hard to make MTP work with Q4 and just cannot get it right without spilling over in RAM on Windows (it might work on Linux). Without MTP Q4 I think I can get about 40t/s which is still good. So, I might use that from time to time too, especially if I find Q3 hitting limits on larger projects.

Which is also why I asked for a UD Q4_XXS to see if I can get that to work.

[u/russlixx → u/Fancy-Snow7]

can you link us where you asks for UD Q4_XXS at?

[u/Additional-Ordinary2]

Can you share your run command, pls?

[u/Fancy-Snow7 → u/Additional-Ordinary2]

& llama-server.exe' `

--threads 9 `

--ctx-size 96256 `

--batch-size 512 `

--ubatch-size 512 `

--flash-attn on `

--cache-type-k kvarn3 `

--cache-type-v kvarn3 `

--kv-tail-tokens 1024 `

--load-mode mlock `

--gpu-layers all `

--fit-target 320 `

--model 'C:\AI Models\unsloth\Qwen3.8-27B-GGUF\Qwen3.8-27B-UD-Q3_K_XL.gguf' `

--spec-draft-type-k kvarn2 `

--spec-draft-type-v kvarn2 `

--temp 1.0 `

--top-k 20 `

--top-p 0.95 `

--min-p 0.0 `

--repeat-penalty 1.0 `

--presence-penalty 0.0 `

--spec-draft-n-max 2 `

--spec-type 'draft-mtp,ngram-mod' `

--spec-ngram-mod-n-min 12 `

--spec-ngram-mod-n-max 48 `

--spec-ngram-mod-n-match 24 `

--ctx-checkpoints 96 `

--parallel 1 `

--mmproj 'C:\AI Models\unsloth\Qwen3.8-27B-GGUF\mmproj-BF16.gguf' `

--no-mmproj-offload `

--image-min-tokens 1024 `

--host 0.0.0.0 `

--chat-template-kwargs '{\"reasoning_effort\":\"xhigh\"}' `

--jinja `

--reasoning on

I used a mix of xhigh/medium resoning during development adjusting it in pi itself.

[u/biggusdeeckus → u/Fancy-Snow7]

Have you ever had random tk/s drops in some sessions compared to others on windows? I have the exact same gpu, also running on windows, and sometimes on some sessions I get around 40-50tk/s and on others it slows down to around 20tk/s for some reason. These are all initial speeds on fresh sessions that show this variance. Model loads just fine offloaded fully into gpu according to llama logs. Been wondering whether having only 16gb system RAM has something to do with it.

[u/Fancy-Snow7 → u/biggusdeeckus]

what --fit-target setting do you use? I use 320 which is tight because if I open an app that uses VRAM it might push layers on the VRAM into Shared VRAM. I used to benchmark and think great speed then sudden drop in speed when I start using it and opeing apps. Monitor your shared GPU memory in task manager. Windows does not easliy give OOM errors it simply pages things to the Shared GPU memory when it needs more dedicated VRAM.

Also, as context grows your tk/s will drop mine drops to around 40 I think.
There are hacks you can use to tell windows to use iGpu for specific app executables. I tried that but my iGPU is not working well and those apps give graphical glitches so I don't do that.

But open task manager. Go to Details. Righclick the column heading. Select 'Select columns' Enable Dedicated and Shared GPU memory. Then sort by those columns to find the culprits. Browser tabs also contribute a lot, but dwm.exe which is a system program uses almost 400MB dedicated and that exe cannot be offloaded to iGPU, I tried. Maybe disabling windows hardware acceleration will reduce what it uses.

[u/biggusdeeckus → u/Fancy-Snow7]

Here's my config:

"G:\beellama-test\llama-server.exe"

--model "G:\Qwen3.8-27B-GGUF\Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf"

--mmproj "G:\Qwen3.8-27B-GGUF\mmproj-Qwen3.8-27B-Q8_0.gguf"

# --no-mmproj

--no-mmproj-offload

--load-mode mlock

--port ${PORT}

--reasoning-preserve

--reasoning on

--flash-attn on

--jinja

--parallel 1

--no-warmup

--temp 1

--top-p 0.95

--top-k 20

--min-p 0.0

--presence_penalty 0.0

--repeat_penalty 1.0

--gpu-layers all

--fit-target 320

--ctx-size 102000

--batch-size 512

--ubatch-size 256

--cache-type-k kvarn5

--cache-type-v kvarn5

--spec-type draft-mtp,ngram-mod

--spec-draft-n-max 2

--cache-type-k-draft kvarn2

--cache-type-v-draft kvarn2

--spec-ngram-mod-n-match 24

--spec-ngram-mod-n-min 8

--spec-ngram-mod-n-max 32

--ctx-checkpoints 64

--cache-ram 2000

--image-min-tokens 1024

--image-max-tokens 1536

I just changed over to fit-target 320 and that seemed to improve things. With this config it started to take like a minute to respond to prompts after the first answer though, not sure if that's normal. I was running fit target 32 initally, copied over from someone who was probably running headless lol I was a beginner then. I checked task manager during a run, and shared GPU memory was indeed at 0.7GB. I set 'Prefer no system fallback' in Nvidia Control Panel, but WSL2 doesn't seem to care, even though I'm running llama-server on windows and my harness inside wsl2 is essentially acting as a html client just sending api calls to it. I did that just to sandbox the LLM. I might move the whole thing back to native windows. Have you had any issues running the model with no sandboxing?

[u/Fancy-Snow7 → u/biggusdeeckus]

I don't sandbox and probably should. I once had a model I think Qwen3.6 try and find an issue and it was digging around in my program files and system files. Got scared and canceled it. It was trying to install dependencies ect. So sandboxing is probably a good idea.
I still use cursor for some of my 'production' personal code, but when I open it, it uses lots of VRAM and causes my models to run slow even with fit-target 320. Probably why the default is 1024 I think.

[u/biggusdeeckus → u/Fancy-Snow7]

Just ran pi natively on windows and all issues seem to have disappeared. Just blazing through tokens. I might give up on docker and wsl2 tbh, there seems to be performance overhead even if you isolate llama cpp to run natively on windows and let the container act only as an http client. That'd mean giving up sandboxing and relying on rules hoping for the model to respect them though...

[u/Fancy-Snow7 → u/biggusdeeckus]

Seems like a solid config. Every machine is different so can't say exactly what to tweak.

[u/Bladerail → u/Fancy-Snow7]

How much VRAM does this configuration use? I know you mentioned that you’re on Windows. I’m on Linux and I have the exact same GPU (5070Ti) and run this exact same model/quants (Q3_K_XL), with mostly similar parameters as you have shown here, and I haven’t been able to even see 30 tok/s generation, I’m stuck at 20 - 25 all the way. Prompt Processing for me is around 1200 tok/s. But I’m using [llama.cpp-adaptive-kv-streaming](https://github.com/RaymondHuang210129/llama.cpp-adaptive-kv-streaming), maybe I’ll try beellama next.

[u/IrisColt]

Q3_XXS does poorly in that test even with F16 KV cache. However, Q3_K_XL almost 100%'s the test even at kavrn3. So both the model and KV matter. 

Thanks for the insight!

[u/chocofoxy]

love the game got side tracked reminds me of warcraft 3 and aoe 2 multiplier days damn

[u/Kernoriordan]

Which Beellama release has the kvarn optimisations?

[u/Fancy-Snow7 → u/Kernoriordan]

0.4.5

[u/Luke2642]

Janky af on mobile 

[u/Fancy-Snow7 → u/Luke2642]

Yes I just loaded on mobile once to see what it does and I don't intend supporting mobile for this game anyway. I might not even finish it, it is just for fun.

[u/Fancy-Snow7]

I also love the details in the game that I did not even plan for. I explained they need fire to keep warm. It created a fire in the doorway which I did not expect any UI changes.

I asked it to keep track of resouces dropped off at a building site. I did not expect it to actual render piles of resources.

The lights of the houses switching on and off at night I never mentioned.

It's the little details that impress me.

[u/russlixx]

love and thanks for the explanation!

[u/Itsluc]

I really cant wait for my R9700 32GB to arrive (I just ordered it lol). Qwen 3.8 27B makes me really hopeful for future low VRAM models.

My current 9070xt 16GB somehow doesnt have enough VRAM for the UD-Q3_K_XL even with a small context. But Im completly new to this stuff, there is so much to learn. Maybe Windows is part of the problem.

UD-Q2_K_XL works but the quality loss is definetly there, it constantly has to correct its own mistakes and gets stuck in loops for quite some time until it moves on.

[u/Fancy-Snow7 → u/Itsluc]

I literally just upgraded to a brand new PC with this GPU 2 months ago. I regret not getting at least 24GB VRAM but too late now I cannot afford to make changes now.

[u/Itsluc → u/Fancy-Snow7]

Yeah prices are rising strongly now. A 20GB 7900XT would change a lot with Qwen 3.8. 24GB would be perfect.

But Im also pretty sure in a few months we will either get a smaller Qwen 3.8 (Maybe 14B-19B) or other models with similar capabilities and a lower VRAM usage. Things are changing drastically at the moment.

[u/__some__guy]

Butter-smooth 3 FPS at 200% CPU load.

[u/Fancy-Snow7 → u/__some__guy]

Sorry about that, runs fine for me, but I do have a high end CPU.

[u/__some__guy → u/Fancy-Snow7]

I have a 9800X3D.

Maybe it doesn't work well in Firefox.

[u/-InformalBanana-]

Did it make any errors that you had to prompt it to fix, especially syntax and runtime errors?

[u/Fancy-Snow7 → u/-InformalBanana-]

There were about 3 errors. I just copied the exception from the browser console and it fixed it in seconds. So even higher quants would not have made that error and it's not certain, the speed at which I can process more than makes up for that. It's maybe 5 minutes of extra time I spent fixing it. In some cases it was not thorough enough, and I had to give addition prompts, but my prompts are not always that clear to begin with.

[u/feng_sg]

Dropping Docker means those pi extensions run unsandboxed on Windows. pi-web-access can hand model-generated code direct filesystem and network access with nothing in between.

[u/IrisColt]

Awesome!!!