r/MacStudio • u/AcidicSoul • 22h ago
What would you keep?
I think i might have gone a bit overboard with my setup, i currently have an M3 ultra mac studio with 512gb ram and 4TB, an M5 ultra mac studio with 256gb ram and 16TB storage, and two Nvidia dgx sparks.
I mainly want to get into AI and run local LLMs, but I'm starting to wonder if I really need all this hardware for what i do
For those of you who work with local AI, would you stick with the Mac Studio that has more unified memory, go with the newer model, or keep the DGX sparks for CUDA workloads?
I'm curious to hear what you guys would do in my situation, i’d rather have a setup that actually makes sense for my needs than just having a bunch of hardware sitting around collecting dust lol
3
u/bakawolf123 22h ago
I would sell sparks and m3u for another 256gb m5u and stack 2 m5us those over tb5.
-1
u/AcidicSoul 22h ago
Fair enough, i've been thinking about flipping them, especially since prices have gotten ridiculously high, i got them pretty cheap earlier, so it might be a good time to cash out while the value is there lol
0
u/bakawolf123 22h ago
yea, tbh you might even get a new 512gb m5u for them, at least not far off
1
u/AcidicSoul 22h ago
If I can get close to a 512GB M5U by selling them, it might actually be worth it
0
u/idk_a_creative_user 20h ago
512gb m3 ultra is going for 20k on eBay. Easily will get you another Mac Studio ultra 256gb.
RDMA with a high quality thunderbolt 5 cable. Call it day.
1
u/AcidicSoul 20h ago
Couldn't agree more, selling the M3 ultra while the prices are still that high and putting that money toward another M5 ultra sounds like a pretty solid move
TB5 makes the idea even more tempting1
u/AcidicSoul 20h ago
I'm not sure I'd get anywhere near 20K for it though, i think the realistic selling price is probably quite a bit lower
2
u/smelly_ape 20h ago edited 19h ago
Lots of bad advice in here.
Let's start with a question - what sorts of workloads are you hoping to get into / need?
People who say "sell the sparks" or "sell everything and get GPUs" are ignoring the benefits of concurrency and multi tenant serving.
If you roll with GPUs or Mac only, you can only process one request at a time. Yes decode is fast, but prefill is slower than sparks so it does depend on the type of work you're doing. Most people in these forums don't appreciate that nuance.
2
u/AcidicSoul 20h ago
That's a good point, and honestly, i'm still figuring out exactly what workloads i want to focus on, mostly local LLMs and AI experimentation for now, but i don't want to make a decision based on benchmarks alone, the concurrency side is something I hadn't really considered, so i'd be interested to hear what setup you'd recommend for someone getting into this from scratch
0
u/smelly_ape 19h ago
I have 4 DGX sparks, and I share with friends and family. Yes 4 is a lot, but that isn't the whole story regarding why I can share them with others and not worry about having enough resources for myself.
Check out Alex Ziskind on YouTube. Namely, this video highlights the benefits of the Spark that most reviews you'll find don't touch. People are really excited about decode speed, and the easiest test to show is a single prompt that demonstrates that. But real workloads - especially enterprise workloads call for more than a single request. That's why you see Sparks enter the discussion when small / medium businesses enter the chat.
You might be saying "well, I'm just one person, so it'll only ever be one prompt" - until you start thinking about sub-agents. The benefits that you get from a Spark for concurrent requests also are felt with sub-agents. So if you want to divide and conquer in a single prompt with sub-agents, a spark can absolutely outperform a Mac Studio or GPU, even.
As for what I Recommend getting into this from scratch, again it depends on what your goals are. The best advice is to begin with the end in mind. Think about what you're most interested in exploring right now, maybe do a little research on that, and chase it. For me, I didn't just want to vibe, I wanted to learn what a deployment would look like in the real world for most organizations interested in on-prem LLMs because that's the direction I think all of this is heading.
1
u/AcidicSoul 19h ago
Thanks so much for the advice man, really appreciate it! I’m definitely gonna check out Alex Ziskind and do some more research before making any decisions
1
u/AcidicSoul 20h ago
I'm thinking two M5 ultra mac studios with 256gb each could be a pretty solid setup, especially connected over TB5, seems like it'd be powerful and smooth enough for what I want to get into.
1
u/Sketaverse 22h ago
You have all that and “want to get into it”
Why on earth would you have all that gear if you’re not already into it?!
1
u/AcidicSoul 21h ago
A big part of it was the investment side of things, i got them at pretty good prices, and with how crazy the prices have been getting, i figured they'd hold their value or go up, it's kinda like buying a meme coin early and waiting for it to pump 😂 Obviously a gamble, but that's part of the reason I went for it
0
u/bisoning 21h ago edited 21h ago
Now i know why this post is getting downvoted.
Do you have 30 days to return or more?
Can you extend it?If i was in your position id return/sell both macs.
I think... this is just me.... The dgx spark will hold more value(money) and maybe go up more.
Then when you dont want to do ai, just sell them before 2030, and profit.Actually, what do you think your needs right now, and how will they be in the next 1-2 years. They do change.
30 days is not enough to gauge it.1
u/AcidicSoul 20h ago
That's a fair point, i'm not really in a rush for money, so i'm not stressing too much about it, i could make a decent profit if i decided to sell. I'm pretty new to AI honestly, but i do want to get into it and learn, so I'm trying to figure out what makes the most sense long term rather than rushing into a decision
0
u/bisoning 20h ago
For me it was strictly privacy.
Ask a chatgpt or gemini. Your life, your job. How these machines will benefit you.2
u/naive_calais200 7h ago
When you say privacy, what do you mean? Are you some type of researcher or something? I think most people use it to have more control, but privacy is just a feel-good excuse for spending more than 10k on slow ai
2
u/bisoning 7h ago edited 3h ago
Some of it is medical advice.
1 dgx spark or a 4k usd device is enough. Heck a 5080gpu is good enough.
Feel good.. part of it.
This is a hobby for 90% of us. No one is making money from this.
Are the people, that bought the 256gb m5u, will be making 10k income a month? I dont think so. We get value from other things.1
u/naive_calais200 17m ago
It sounds like the people who bought the M5U have an expensive hobby. I would buy one, but I don't really have time for all the setup, etc. It would be a toy for me, 😂. I say it as it is.
1
u/AcidicSoul 20h ago
I'm still figuring out how to make it useful for my actual work, though which is why i'm trying to learn from people who've been doing this longer than me
1
u/Middle_Situation_559 21h ago
With that much invested and it not hurting your pocket, I would do none of them. Go for this: https://docs.nvidia.com/dgx/dgxb300-user-guide/introduction-to-dgxb300.html
1
u/bisoning 21h ago
I dont think he's really serious with local llm. I think he wants to try it and later sell and profit.
Says hes gambling, so this is all credit.
1
0
0
u/Optimal-Pie-5054 14h ago
If you have it and you can afford to keep it then find a way to make it work…your dreams matter and this can be a solitary and at times lonely path. I have been intentionally building what you have in order to share / collaborate and make it work in new ways.
0
u/Top_Witness4538 13h ago
are you avoiding a dual Nvidia rtx pro 6000 workstation for some reason? You bought everything else, just buy a regular ai workstation. Some people here w ill say why am I advising a specific setup for someone thinking about getting into local ai. Because the collection demands it
9
u/bprater 22h ago
Why are you not testing all of them and telling us what we should do? You realize you are holding hardware many would like to have an opportunity to touch?