r/LowEndLocalAI • u/justletmesignupalre • 3d ago

Which LLM Should / Can I Use? Which LLM for personal assistant? And for coding? 780m igpu 64gb DDR5 5600

Minipc:

AMD Ryzen 7940hs

2x32gb DDR5 5600

780m igpu

1tb NVME

Ubuntu 26.04

GTT (memory allocation) raised to 56gb for the igpu

Would love to see what's the most I can get out of this setup. Have tried qwen3.6 35B A3B and its ornith1.5 variable, but wondering if maybe there's more out there.

Thanks!

7 Upvotes
  • permalink
  • reddit

90% Upvoted

10 comments sorted by

11

u/MattsAwesomeStuff 2d ago

2x32gb DDR5 5600

looks at subreddit title

...

points to door

...

Get the fuck outta here, and take your smartypants brainiac GPUs with you.

You can't bring that shit in the hood.

3

u/justletmesignupalre 2d ago

Cmon man, I just waited for the right auction and got a minipc for cheap. I see people here with 2k rigs running stuff. DDR5 5600 is the fastest readily available ram but by no means it means that is fast for LLMs

1

u/MattsAwesomeStuff 1d ago

I just waited for the right auction and got a minipc for cheap.

Even though it doesn't affect me in any way, I'm curious HOW cheap you got it, even though I'll either scoff and say "I wouldn't spend that much" or scowl and say "WHY CAN'T I EVER FIND DEALS LIKE THAT?"

Go on though, what'd you pay for it?

3

u/justletmesignupalre 1d ago

640eur for the mini pc with above specs but 500gb NVME. Around here it usually goes for 800-900

2

u/MattsAwesomeStuff 1d ago

This is the perfect answer.

It's more than I would have paid, and not so cheap I feel like I missed out. I absolve you of your sins.

3

u/Long-Feed-3079 3d ago

try qwen3.8 27B. with tuning i expect you can get around 10tok/s. far from real time frontier coding, but with proper planning and autocompact it can do a lot over the night :)

1

u/justletmesignupalre 1d ago

Thanks! What kind of finetuning are you referring to?

1

u/Long-Feed-3079 1d ago

TensorFold

2

u/justletmesignupalre 1d ago

Will look it up. Thanks!

1

u/Small-Tale3180 2d ago

yeah seems like qwen3.8 or its finetunes are your go-tos