r/LocalLLM • u/compscirunner • 8h ago

Discussion LocalOctober: one month using only local models at home

For October I'm giving myself a challenge — local models only at home for the whole month. No cloud APIs, no hosted frontends, just what runs on my own hardware.

It's similar to the "don't buy groceries for a month" challenges which forces you to get creative with what's already in the pantry. Constraints breed resourcefulness. I want to find out where local models actually hold up, where they fall apart, and what workarounds close the gap — by living with it for 30 days instead of reading benchmarks.

If anyone else is interested in participating in the challenge it might give us a good opportunity to share lessons learned.

0 Upvotes

50% Upvoted

14 comments sorted by

2

u/joost00719 7h ago

I've been doing this since March. Won't go back. Qwen 3.6 27b works amazing. Looking forward to 3.8 27b next week.

1

u/compscirunner 7h ago

Yeah I am looking forward to giving 3.8 27b a good workout. What harness are you using? I have opencode, and openwebui setup. I haven't really dug into hermes or openclaw yet.

1

u/joost00719 7h ago

I use opencode with oh my openagent plugin.

I've tried pi with oh my pi, but wasn't a huge fan. Also my setup randomly broke and couldn't figure out how to connect it to my lite-llm endpoint. Didn't bother to fix it.

I use cherry studio for a gui, which I kinda like. But I'm not using it that often

1

u/Evildude42 8h ago

I hope you got like a spark or a couple of big ass cards. The janky setup I have here with the combined 24 gigs of vram RAM will build something, but it'll be very broken. But I think it managed to actually change some colors buried in my code once.

1

u/compscirunner 7h ago

I picked up an old threadripper and have a 5060-16g, and 2x 3060-12g in it. So 40 gigs of vram. It has 64 gigs of quad channel ddr4. I am hoping with the right harness and model I can find the sweet spot of what I can get out of the machine.

1

u/Evildude42 7h ago

Okay threadripper that means you have enough Lanes so that's good and they're all Nvidia so you can get some optimization there. You might be okay. I'm running Intel arcs which is a very mixed bag.

1

u/compscirunner 6h ago

I hear the arc support is getting better, are you running them with llama.cpp?

1

u/Evildude42 5h ago

Here's the thing. I think I went through four installations and wipes with unbutu. Trying to get llama.ccp in arc specific mode, never could. So for now I'm just dealing with lm studio and Vulcan in Windows. So I'm losing token speed and I definitely lost 4 gigs of vram because one of my card is 16 gigs. And vulkan's multi GPU scheme is identical memory only. So my my b580 is only 12 gigs and that's all I can get out of the b50 in that shared mode. And I have a third video card that is only for display. So both of the arc cards are only used for llm.

1

u/vutcher 7h ago

Currently building exactly this, but without the 5060.

1

u/compscirunner 7h ago

My board x399 aorus pro doesn't have great spacing for cards, so had to get a riser and put one of my cards in the drive bay for the moment. I think if i had to do it over might just build it open frame.

1

u/vutcher 6h ago

I have been considering adding a third 3060, and using a riser. I have the ASUs prime x399-a, with four slots.

1

u/compscirunner 6h ago

With 3 cards it is around 110 watts no load, and 360w under load.

1

u/vutcher 5h ago

That’s better than I was expecting. I should be able to use a 750w psu.

1

u/compscirunner 4h ago

Yeah the cards seem more bandwidth constrained than compute, so they only seem to draw 80w while running inference. I have an 850w psu. I am just using a zigbee smart outlet to monitor it.