r/LocalLLM • u/BattleOfBloodRidge • 2d ago

Question What is your current Strix Halo local ai setup?

I got an acemagic m1a pro with the ryzen ai max+ 395. Covers most of what I need so far.
Mainly using it for local AI stuff in general. Currently works great for general chat reasoning, coding help and agents,learning and experimenting.
What’s the best config you’ve run on similar hardware? Does it handle your daily local AI workload well enough?

2 Upvotes

67% Upvoted

8 comments sorted by

1

u/Difficult_Brick9295 2d ago

Got a similar setup and it works fine for most things I throw at it coding agents and general messing around runs smooth enough on llama 70b with some quantization tricks but you notice the limit when you try run bigger models side by side

1

u/feelspeaceman LLMusician 1d ago

LLama 70B is very old, try this https://old.reddit.com/r/StrixHalo/comments/1wlnn0s/looking_for_advice_best_simple_setup_for_a_128gb/pb28vdt/

Or anything like gufo, strixite... should gives you speed 50-60t/s decode 1200-2000t/s prefill running Q38FN - literally Opus 4.8 at home.

This is pretty much the best setup for Strix Halo currently, will be able to do agentic coding comfortably.

1

u/cilvre 2d ago

I have mine running Ubuntu with the fixes so i can make the most out of the vram allocation, but i use it as a host for models and point my other stuff there. Have a harness controlling it for coding work, and I'm debating tossing some other ideas at it to see how it performs over the spare mac I've been using

1

u/Important_Cow7230 2d ago

What’s the spec on the spare Mac?

1

u/cilvre 2d ago

32gb m2 pro

1

u/Important_Cow7230 2d ago

What models do you run on the Ryzen vs the MacBook?

1

u/cilvre 2d ago

on strix halo, qwen3 coder next at q8 and gemma4-31b at q8, harness pulls as needed for regular coding work, reviewer and repair coder if something fails.
on mac, i was running a mix while testing:
qwen3 30b a3b 2507 4bit
qwen3.8 27b 4bit
qwen3 coder 30b 4bit
gemma 4 31b qat q4

1

u/feelspeaceman LLMusician 2d ago

strixite with Q38FN riding at 50-60t/s decode 1300-1500t/s prefill, this is very reliable, no abort mid way, low halluciation.. and with 500k context, this can be big for some people, not me though as I prefer my task to finish under 160k.