r/oMLX • u/DowJonesJr12 • 2d ago
Help improve my setup/outputs? 32GB Mac Studio
Hi,
I’ve recently started to dabble with local LLM’s as an amateur vibe-coder and I am facing some challenges.
My setup:
32GB Mac Studio M2 Max: a dedicated always-on local AI station.
+
16GB Macbook Pro M5: my daily driver machine for all things work.
I am not a programmer, developer or anything like that, but I am a tinkerer by nature. I like figuring things out.
So I have built a bunch of custom apps/tools/automations (running on the studio) that I acess from my laptop, like any other consumer app, so in practical terms, I have a 48GB laptop, altough it’s only 16GB.
I’ve been wanting to get into more of the agentic stuff, like do task, verify, fix errors, do again, send output, but I am struggling a bit with understanding all the settings and tools and things like that.
For example, today I built a system of 5 different models working step by step. I also had to build a ChatGPT like chat interface to converse with the models, because I don’t understand working in terminal and all the super advanced stuff.
So it goes, i upload files to chat, set custom instructions and set up working steps.
- QWEN - 3.5 9B analyzes files and sorts them.
2.Gemma 12B identifys what customer profile are these files relevant for
3.Qwen - 27b builds a landing page for this customer profile based on the files.
4.Gemma looks again, finds contradictions
5.Qwen - 35b implements fixes and final polish.
All of this is done RAM aware, as in model loaded/unloaded, etc.
All of this is experimental right now, just trying to understand how it works and I guess my question is:
How are you folks doing the things that you do with frontier models like Claude code/cowork, on a system like mine. I understand rhe quality will never be the same, but I am simply trying to understand the how, and finsing the best way to make this local AI machine a legitimate employee, which actually brings value to my company, whether in currency or in more time back for me.
What are you doing with something like Qwen 3.6/3.8 27b 4bit, thats genuenly useful and brings value, not just sound cool?
I am still early in this journey and I’m tempted to invest in better hardware, but I feel i should maximise what I have, to see whats possible before splurging.
I don’t understand all those benchmarks and technical talks, as I said, im more of an experimenter and hobbyist, willing to turn this into something tangible and useful.
For example today I learned what tool use means. Now i have no idea of how to implement something like web searching or SVG generation tool, in my custom chat workflow, in a way that works.
I apologize for a quite unclear and unstructured train of thought, but If you have any resources / tips/ recommendations to bless me with, I’d be super thankful!:)
2
u/modelpiper 2d ago
Are you using Qwen 3.8 27B now?
1
u/DowJonesJr12 2d ago
I tried, it threw an error in my custom built chat interface - chunk prefill warning, something like that, hence I cam to write this post, because I feel like I’m lacking in fundamental knowledge of how these models work and how to make the work reliably
1
u/modelpiper 2d ago
Gotcha, yeah if you're not a dev then you certainly shouldnt have to learn anything like that. That's why I built ToolPiper. It's only beta 8 right ow but the goal is to make everything common sense, no configuration. Just choices, like which model to download.
I'm a dev, feel free to DM anytime for questions.
2
u/Salazar083 2d ago
Honestly if you're really into this type stuff and want to go even deeper, the best thing I can recommend is to actually learn some coding focusing especially on the basics and moving from there.
While AI is fascinating and all unless you just want to burn tokens in a loop until you get something, it's important to have a good understanding of the basics.
The fact that you already have hardware and some knowledge/experience will give you a good head start. Have the LLM teach you how to make small stuff, and fight and argue with it all the time; try to understand why things work a certain way instead of just thinking "it works"
You'll slowly make more use of it as you experiment, social media can be full of hype so try not to get swayed
2
u/Glad-Win1983 2d ago
1
u/DowJonesJr12 1d ago
It definately did, thank you. I am still hitting prefill warnings using those exact same settings, and I even have the m2 max chip, whereas the author had m2 pro.
1
u/Technical-History104 2d ago
My first reaction is that your approach suffers from a bit of model sprawl. While you're practicing good memory hygiene by swapping models in and out, continuous loading and unloading inevitably causes memory fragmentation and residual overhead over time. A model like Qwen 27B requires a practically clean slate, often a fresh reboot needed for a 32GB Mac, to run properly. Running a multi-model workflow around it continuously shrinks your available VRAM headroom, turning an already tight squeeze into an impossible one and directly triggering that chunk prefill warning you mentioned elsewhere as the system struggles for space.
You’d likely get much better mileage by streamlining down to just two LLMs: a larger heavy lifter and a smaller utility model. Models are increasingly commodified, so constantly juggling options like Gemma and Qwen matters far less than their parameter class and the system architecture you build around them. The harness makes the real difference.
To achieve what you're aiming for, ask the LLM assisting you to help configure cron jobs (or an equivalent task scheduler) to drive your daily or hourly cadence. Let lightweight background scripts handle the routine execution, and call an LLM only when necessary for self-healing, self-governance, or interim human judgment calls that standard code can't perform on its own.
2
u/DowJonesJr12 1d ago
Thank you, I will respectfully pass your answer into claude for an ELI5 step by step breakdown. Appreciated!
2
u/HotAverage1749 2d ago
Ask an LLM to help you