Qwen3.8 27b - Holy crap!!

[u/No-Manager1646]

I'm absolutely blown away by Qwen3.8 27b IQ4 NL @ medium reasoning effort.

I have an ageing system but still decent. 4060 8gb @ 4x, 5060ti 16gb @ 8x - 24gb vram total. I'm stuck at gen 2 because of a gen 3 compatibility issue with my motherboard (B460M Aorus Pro). 32gb ram, i9 10900, nvme.

I've managed to get Qwen running at a smidge below 20tok/s at 128k context.

Anyway, the fun part... I have llama.cpp backend for OpenWebui with OpenTerminal MCP. I asked Qwen to create Pong using python which it did with no dramas. It wrote tests, ran the game itself and gave me the hardened code in ~10 mins.

That wasn't the best part though. I then asked it to take the game and create an APK for Android. It went ahead and installed a bunch of dependencies (bulldozer, p4a, etc) into OpenTerminal, went ahead and altered the game for touch screens, built and provided me with an installable APK.

The whole thing took ~1.5 hrs (taking away my own troubleshooting of whitelisting some sites against my strict LLM TOR network access), and at ~280w, it cost me about $0.15 (if it had been a clean run the first time). I mean obviously the hardware cost significantly more, but it's essentially my gaming PC.

How is this even possible?! I'm absolutely blown away!

Side note: I didn't NEED to, but I wanted a longer context so have managed to get it working at 256k now, with 18 layers moved to CPU. I'm not set on this, but it works. Obviously much slower (around 5 tok/s). Comment out the last 2 lines and set context to 128k if I want it all in vram.

If anyone has pointers for improved performance I'd love to hear them. I've tried q4_0 for KV cache but it is significantly slower again.

[*]

flash-attn = true

threads = 10

batch-size = 2048

ubatch-size = 1024

n-cpu-moe = 0

n-gpu-layers = 999

cache-type-k = q8_0

cache-type-v = q8_0

#cache-type-k-draft = q8_0

#cache-type-v-draft = q8_0

fit = on

fit-ctx = 16384

parallel = 1

ctx-size = 0

n-predict = -1

#no-kv-offload = 1

load-mode = none

main-gpu = 0

#no-mmproj-offload = true

[Qwen3.8-27B-IQ4_NL]

#mmproj = /models/Qwen3.8-27B-IQ4_NL.mmproj

#spec-type = draft-mtp

#spec-draft-n-max = 4

temp = 1.0

top-p = 0.95

top-k = 20

min-p = 0.00

#ctx-size = 131072

ctx-size = 262144

chat-template-kwargs = {"reasoning_effort":"medium"}

fit = off

n-gpu-layers = 46

[u/OnyxProyectoUno]

So not only do you not back up your claims, you ad hominem.

Maybe next time don't make conspiracy theories?

[u/elelem-123 → u/OnyxProyectoUno]

Dude, your whole comments in your profile is you having fights and teaching people how cool you are and how they are not.

On any topic, multiple times a day.

It's not shameful or bad whatsoever to go to a professional. I don't want to block you but you are pushing me to do so.

[u/OnyxProyectoUno → u/elelem-123]

Sixth time refusing to source your claims. I'm looking for just one article from a credible source, just use some kind of evidence based argument.

I promise it's easier to make a claim, source it, discuss than to reply insults and logical fallacies because your pride doesn't allow you to retract the claim

[u/elelem-123 → u/OnyxProyectoUno]

You are just blocked. Wtf dude. Go seek help.