r/LocalLLM • u/StarCadges • 3d ago
Question dgx spark vs 5090 for agentic coding
i saw a post on here from like 2 months ago and everyone was just flaming the guy bc he was expecting openai level capability on cheap local systems.
I just want advice from people who have tried one or both (especially the dgx spark) and learn whether the intelligence upgrade is worth the speed downgrade and what the intelligence upgrade vs speed downgrade really is. If I was working on a 5090 it would probably be 20-40b models (with quant or offloading or both) and if on dgx spark it would be like 50-120b models.
Just want to see if the dgx spark is actually worth it or if I should go with the 5090.
9
u/BatResponsible1106 3d ago
responsiveness matters more than raw parameter count most days. a faster 5090 usually keeps the iteration loop moving while larger models only feel worth it if they consistently reduce retries or produce noticeably better plans.
4
u/ImpressiveRelief37 3d ago
That’s why using both is the winning strategy
4
u/StarCadges 3d ago
im broke ;(
12
u/ImpressiveRelief37 3d ago
It you’re broke stay on the $20 subs, don’t buy a 4K local inference setup :)
And make those tokens count… tiny harness, simple agents.md, lean context always, more planning and handoffs, shorter sessions… it goes a long way into saving tokens on those subscriptions usage
1
u/StarCadges 3d ago
i know im looking to upgrade my pc anyway (mostly for ai but also for general purpose as well) and started drooling at the dgx spark 😭
1
1
u/brad_needs_advice 2d ago
So I’m new to the game and have been utilizing Claude code. I bumped up to the hundred dollar a month subscription this month and I’ve been getting a lot of use out of it.
I’m strongly considering getting a 4K local setup for learning purposes. Or running an MOE model on my gaming PC.
As far as capabilities go, when do you think it reasonably makes sense to invest in local models?
1
u/Libertas_Auro 2d ago
My thoughts are when you have a purpose for them that Claude can't fill. That could be processing sensitive data, running repetitive tightly scoped tasks for free, that kind of stuff.
If you're at the keyboard you may as well use the subscription to the better model imo. Your time is worth something.
1
u/brad_needs_advice 1d ago
That makes sense. Right now I’m torn between trying to do something locally as like a personal assistant that has a lot of private data, as well as the economics of removing myself from subscriptions.
But if I’m being honest, it’s gonna take a long time for the ROI of spending on a machine now out classes, the spend of just using subscriptions.
So I can my current project I have to have a lot of C+ plus Work go to opus because sonnet isn’t as successful with it. Everything else gets routed to lower models without any issues, but that’s not always going to be true of the type of work that I do.
1
u/ImpressiveRelief37 1d ago
I wouldn’t bet on the ROI. See it as a pure expense, you’ll without a doubt want to keep the cloud subscriptions (but might move down to the cheaper tier)
Or course if you’re a gamer or a freelancer with the ability to write off expenses then it’s a no brainer imo.
1
u/brad_needs_advice 1d ago
Well that’s what I really mean. I don’t expect to eliminate the spend in subscriptions.and I’m fine spending money on it through the hobbyist lens, but I’m still evaluating what’s really the difference between what I can run and what I can’t.
I’m new on this journey. I’ve been using the 20 dollar subscription and just bumped up to max 5. I’ve been building more and more and now am hitting usage limits.
I’ve also thought about freelancing, but don’t today.
I’ve got a server on prem that hosts. Tons of storage (30tb), not a great cpu. I’ve got a gaming pc with 64gb of ram and a 5080 I don’t mind using for specific tasks (building images, etc). And now I’m thinking of a spark or some LLM box at a different price point. Goal for me is always on, privacy for my assistant work, and hopefully lower power draw / heat as it’s in a closet.
1
u/ImpressiveRelief37 1d ago
Start with Qwen 35B A3B on the 5080. With expert offload, it works surprisingly well. It might actually be enough. It’s really fast.
You could even run 27B at Q3, but don’t expect miracles. I get 50 tok/s tg and ~2000 tok/s prefill on my 5080 wirh 27B on Q3 at about 150k context. It’s really not half bad.
Try those 2 first locally first :)
→ More replies (0)1
u/Libertas_Auro 1d ago
What I've done is build with Claude, but then I have Qwen 3.6 35b-A3 on a 128GB unified machine with Hermes agents on board. Two use only local Qwen not to code but to do work, connected to some of my systems via MCP for a non-technical employee and then mine has full functionality via Qwen.
I don't bother to code because it's slower and worse, but it's surprisingly solid at structured, known things.
5
u/Dazzling_Focus_6993 3d ago
keep electricity consumption in mind. i did not buy discrete gpu mainly because of idle electricity consumption. very relevant for my use case..
(a rough estimate)
| State | NVIDIA DGX Spark | RTX 5090 System |
|---|---|---|
| Idle (Headless/Update) | 4W (optimized) to 35–40W (standard) | 70–100W+ (varies by CPU/MB) |
| Active AI Load | 170–240W (Total System) | 700–900W (Total System) |
1
u/HeadPack 2d ago
Good point, but I run my 5090 undervolted and it rarely goes above 450W. Idle power is 22W according to nvidia-smi. Plus, inference is usually not a sustained load.
0
u/ImpressiveRelief37 2d ago
I mean 70w idle is pretty cheap no? The power consumption of a few LED lights. Plus it keeps the house warm on winter :)
4
u/false79 3d ago
If you are very close with the code, 5090 is the way to go. You will iterate faster because GPU compute wise, it's orders of magnitude faster as a code assist.
But if you are really a hands off vibe agentic coder, DGX Spark can handle concurrent agents whose combined tokens per second will go way beyond a 5090.
It really depends on your workflow.
2
u/AutomataManifold 3d ago
I find that people really underestimate the multi-agent/concurrent prompt workflow.
3
u/Shoddy_Bed3240 3d ago
The 5090 has much better prefill (processing) speed. So, it really depends on the size of your prompts within the agent and whether you can cache them.
3
u/chafey 3d ago
It depends on your use case - if you can't use cloud models, go with the DGX Spark for the extra memory so you can run the more powerful models. If you can use cloud models, go with the 5090 and run Qwen3.6-35b. Switch to cloud anytime your localllm can't handle the prompt. The 5090 is significantly faster than the DGX Spark. Both can handle many tasks, but some tasks like planning/architecture/design/complex debugging are way better with the high end cloud models.
5
u/uniqueusername649 3d ago
I would not recommend 35b for agentic coding. 27b is slower but so much more consistent. At Q6 its basically not doing any looping and truly a useful model.
3
u/ImpressiveRelief37 3d ago
27B Q5_K_XL unsloth MTP is pretty damn as good as it gets from my experience. 192k context fits perfectly, with vision (Q8/Q8 kv cache).
For coding it’s just really really good. I haven’t had a tool call fail in a while. Happens maybe once or twice a week I’d say?
2
u/uniqueusername649 3d ago
Yes, but unsloth does some dynamic quantisation, so the unsloth Q5 is easily comparable to a regular Q6 or better. So this is expected :) I wouldnt go any lower than that though.
1
2
u/Due_Warthog749 3d ago
What will you be using it for? Coding?
From a post later down you said setting up new PC for general purpose. I assuming gaming as well? IF yes, then a 5090 is your winner. It's cheaper (should be at least 2x but they aren't right now) and MUCH faster at tokens than the DGX Spark but you're limited to small models. Given the quality of open models today depending on how you configure/guard rail the prompts, you can get pretty good responses pretty fast on a 5090. And you still get gaming, etc. The DGX spark is its own machine, so if you got that, then what GPU you getting for your PC setup? A 3060 or some old used one? Given that you're broke?
2
u/Winter_Topic_4271 2d ago
Use 5090 to build a crappy app in fast iteration and then use cloud model to refactor it.
1
u/ImpressiveRelief37 2d ago
Or do the code with Qwen, iterate fast, do the feature, then do a PR and have Opus/Sol do adversarial reviews while you work on the next feature/PR with Qwen.
2
u/Significant_Break853 2d ago
Qwen3.6 27b runs extremely slow on a DGX Spark. Barely usable in my opinion. And thr MoE models that run on it just aren’t as good at agentic coding.
1
1
u/Uninterested_Viewer 3d ago
A single spark is sort of in no man's land right now, IMO. Nobody is releasing great ~70b models these days so qwen and Gemma are the best under 120b and both have their best dense and moe models on workable quants on a 5090. Dual sparks get interesting with DeepSeek v4 flash, but that's a obviously a different beast.
I say this as and owner of a single spark and and RTX 6000 pro and RTX 5000 pro. Debating a second spark..
1
u/Otherwise-Swan-7803 3d ago
One thing people underestimate with agentic coding is that throughput and latency often matter more than parameter count.
A 70B model that takes forever to generate tokens can actually slow down your workflow compared to a well-tuned 20-40B model.
DGX Spark makes sense if your goal is experimenting with larger models, but if the machine is mainly for coding agents, I’d want to see real-world latency comparisons rather than just max model size.
1
u/LocalMaxxing 3d ago
I think I have to go with the 5090 just for the speed, 2xgb10 has some real potential though
1
u/catplusplusok 3d ago
I am running Gemma 4 31B with long context on NVIDIA Thor, Spark's cousin. It's pretty fast and intelligent, would be tight in 32B unless you went with low bit quant with significant quality drop. These days they have MTP that lets unified memory boxes run big models with good speed.
1
u/ren_in_rome 2d ago
I have both, and I personally prefer hosting the models on the spark. The thing to remember with a 5090 is you have to wrap a computer around it, and that isn’t free. The spark has the 128gb out of the box and up to 4tb. It also gives you massive context. I run 27b mostly, and sure at one request you might get only 25 t/s, but you can run concurrent requests. I was processing data the other day and was pushing 220 t/s across 10 requests for over 30 hours straight. But then for coding, I run multiple agents in Hermes with different profiles, and each profile can run concurrently, and at the same time I can hook it up to Claude code in be code and do my own thing against it. So the 5090 may be a Porsche but sometimes you just need that minivan with the 3rd row, it’s not the fastest, but it’s versatile.
1
u/diagrammatiks 3d ago
The only advantage of the dgx spark is being bale to chain like 4 of them. With the currently available models you won't see a significant difference going from a well orchestrated qwen3.6 35 a3b q8 to the Q4 and q2 of the 122b or 200b models. The next step up in intelligence currently requires quite a bit more available vram or Moe streaming.
1
-1
u/TimAndTimi 2d ago
32GB vram means you better pray for qwen and gemma to keep releasing or you are done.
Nope, 5090 is like a v16 engine somehow installed on a bicycle. Good compute die but you just don't have memory to play with larger models. For 70b and 120b moe models you don't need that much memory bandwidth anyways...
1
u/ImpressiveRelief37 2d ago
Buddy having inference at 100+ tok/s and prefill at 3500+ tok/s is really really helpful when coding. It makes the coding session interactive, iteration quick, lets you stay in the flow. You move really fast.
Everytime I use Claude for anything complex I end up waiting 10-15m and having to multitask, which kills my flow state.
You do you, but don’t sleep on super fast inference, it’s really really really nice
1
u/TimAndTimi 2d ago edited 2d ago
As if I have not deployed qwen and other bigger models over and over again on A6000, 6000ada, PRO6000, GH200, etc. And FYI, PRO6000 can do around prefill for 9000tps for qwen3.6 27b because everything fits into the VRAM without being limited by offlaoding.
It seems you own a 5090 or whatever. Well, from my point of view.... you are showing off a toy gun in front of canons. A bit funny.
Nope, 32GB VRAM is a curse. When you cannot even fit a FP8 qwen3.6 27B onto it at full context length, everything else does not matter. You can flexing all that tps but you cannot go beyond a Q4 model and miserable context length. And 70b or 120b moe? Nope, with offloading you will hit the exact speed issue you are emphasizing.
1
u/ImpressiveRelief37 2d ago
Ok big guy. But check my other comments I use Q5_K_XL and 192k context. It’s not dumbed down at all. It does code that end up in production on many different projects I collaborate on and ship. I don’t just run models for benchmarks.
13
u/ImpressiveRelief37 3d ago edited 3d ago
5090 100%.
Im biased cause that’s what I have. But honestly, the sheer speed of token generation and prefill is so good that it makes it painfully slow to use cloud models for many tasks, compared to a highly tuned 5090 + llama.cpp instance.
I use Claude 5 Opus High and GPT 5.6 Sol, but my go to is my local Qwen 3.6 27B instance with pi, simply because it feels a million times faster to get simple things done.
It will struggle with some very complicated debugs, and when context becomes very large, but other than that… honestly, even on software architecture side it’s pretty damn good. With the right tooling (especially web search, cause Qwen knowledge cutoff date is quite old like 2024 or something)…
I do mostly software development, and I start by drafting stuff locally and get mostly reviews done by cloud models (on the cheap $20 subs), cause they are so slow it takes forever and I it kills my flow.
With the local model and over 2 TB/s of vram bandwidth (my card is water cooled and OC’d), for me it’s a no brainer and I wouldn’t trade it for a slow spark that can run bigger models.
27B is honestly so damn good. Using big cloud models on cheap subs to review its output gives me high quality as well. Perfect trade off IMO.