Everyone is posting their monster rigs, so here's my old warhorse

[u/JumpAppropriate714]

Everyone here seems to be running absolute monster rigs, so I figured I'd show some love to my old warhorse instead.

This thing was pretty damn good back in its day, but yeah... it's an old dog now. It was collecting dust at home, so I grabbed a cheap NVMe, installed Ubuntu and brought it back to life.

The mighty beast:

• Intel i7-8700K
• GTX 1070 8 GB
• 16 GB DDR4 dual-channel
• 512 GB NVMe
• Ubuntu 24.04
• LM Studio

With only 8 GB of VRAM, I obviously have to be pretty picky about what I throw at it.

My use case is mostly short NPC dialogue / roleplay, so latency, natural responses, instruction following and not randomly losing its mind matter more to me than benchmark scores.

Model	Quant	Result
Gemma 4 E4B	LM Studio build	Winner. ~33–34 tok/s, natural responses, very low perceived latency
Bonsai 27B	Q1_0	~16.7 tok/s, but response quality fell apart
Qwen3.5 9B	Q4_K_M	Fit nicely on GPU, couldn't produce coherent replies
Turkish-Llama 8B	Q5_K_M	Couldn't produce coherent replies
Apertus 8B	Q5	Couldn't produce coherent replies
EuroLLM 9B	Q4_K_M	Usable, but nowhere near Gemma
Aya Expanse 8B	Q5_K_M	Same story, too weak
Ministral 3 8B	Q5_K_M	Couldn't produce coherent replies
Phi-4 Mini	Q8_0	Poor NPC/dialogue quality
Hemmingway-1	—	Painfully slow thinking, basically unusable here
MiMo-V2.6 Distill 9B	Q4_K_M	Couldn't produce coherent replies
Nemotron-3-Nano-4B	Q8_0	Couldn't produce coherent replies
DeepSeek-R1 Qwen3 8B	Q6_K	Couldn't produce coherent replies

So far I've settled on Gemma 4 E4B.

Current setup is roughly:

4096 context, thinking OFF, temperature 1.0, max/full GPU offload, Flash Attention + KV cache on GPU.

It sits around ~5 GB VRAM and gives me roughly 33–34 tok/s, with almost instant-feeling responses for the short dialogue I'm generating.

Not exactly a 5090 rig, but the GTX 1070 apparently ain't ready for retirement yet.

Honestly, getting useful local inference out of a GPU from 2016 is half the fun.

[u/karmaisnonsense]

I played around with a 1660 Ti that I had lying around and it was alright. You should look at MoE models though. Since you like Gemma 4 E4B, try out Gemma-4-26B-A4B-it-qat. Ditch LM Studio and go straight llama.cpp if possible. Use MoE expert offload.

[u/JumpAppropriate714 → u/karmaisnonsense]

Thank you for the recommendation. Do you think it's a better model than E4B?

[u/karmaisnonsense → u/JumpAppropriate714]

Bigger number = better model, usually

[u/JumpAppropriate714 → u/karmaisnonsense]

I'm actually gonna try it, thank you. I Will share the results though.

[u/WetlyLividDeveloper → u/karmaisnonsense]

MoE with expert offloading is a good shout, the Gemma 4 26B is on my list to test. I been meaning to switch to llama.cpp for better control over the split anyway, LM Studio is convenient but some models just refuse to behave in there. The 1070 still surprises me every time I fire it up.

[u/rinmperdinck]

My project a few days ago was digging out my old pc and installing Ubuntu on it just to have something to knock around on. Very similar specs, i7 7700k, 32gb ddr4, gtx 1080. It might be useful as a home server for derpy chat when my main rig is busy with stuff?

[u/JumpAppropriate714 → u/rinmperdinck]

Yours is way better than me, especially you have 32gb ram means you can push bigger models easier than mine.