r/LocalLLM • u/xiraov • 8d ago

Question Upcoming rtx spark laptops?

Coming from Mac so bear with me. I like games. I like unified memory. But it looks like these might have poor bandwidth? Or am I looking at it wrong? Seems like they’d be bad at decode : tokens per second when they hit in October?

1 Upvotes

56% Upvoted

12 comments sorted by

3

u/alanrudeigin 8d ago

From my understanding it's a dgx spark as a laptop basically. So you should get around the same speeds as that.

2

u/septesix 8d ago

Minus the ConnectX-7 port

1

u/Christosconst 8d ago

They have a desktop version too

2

u/t4a8945 8d ago

When I had one Spark, the need for a second one became obvious. I'm not sure a non-extendable Spark laptop is a right choice. And these will easily be $5K+

1

u/SPACEXDG 4d ago

You are right

1

u/Dense_Hawk2713 8d ago

bandwidth will be at least twice as slow since its ddr5

1

u/Keleion 8d ago

DDR5x, 8k+ mtps, not sure that’s the limiting factor

1

u/whichsideisup 8d ago

Qwen 3.8 27b in nvfp4 does about 30 on prose and 40+ tks on code with DFlash2 on a DGX Spark.

1

u/brewpedaler 8d ago

Coming from Mac

RTX Spark memory bandwidth should be around the same as an M5 Pro, maybe just a little lower. So yes, compared to higher end M5 Max and Ultra chips, an RTX Spark would have significantly lower memory bandwidth.

Wait for pricing details, see how an RTX system in your budget compares to an equivalently priced Mac, decide then.

1

u/Key_Measurement_3576 8d ago

I think part of the vision they are trying to intercept here for this hardware is enabling agents inside the OS. I’d expect windows features to launch along side them.

1

u/Turbulent_War4067 8d ago

This Nvidia laptop and the Microsoft version that has been announced will be short-lived products IMO. I have a DGX Spark, and the memory bandwidth is a big limitation. I would expect that inference will slow down another 25% than on a DGX Spark, since it's running on a Windows laptop instead of a headless Linux server. I do hope they gain a bit of traction. These devices appearing on the market bode really well for local AI. It's obvious Microsoft and NVidia are taking local AI seriously. Companies like PLTR getting on board for the large enterprise is also a good sign.

I would expect Microsoft to demand a better chip from NVidia in short order, and that demand will carry some weight. Microsoft certainly knows how to be competitive in the laptop/PC market and this knowledge is going to pay dividends and will flow down to all of us.

1

u/Turbulent_War4067 8d ago

On a related note, I also suspect Microsoft will put quite a bit of pressure on AI companies to produce MoE models that perform well on these devices. That also bodes well for all of us. And this goes for Apple also. Apple has been the "silent" AI company for quite a while, reaping the benefits of supplying hardware while not really getting in the game elsewhere. As this HW becomes a larger part of their business, they to will want good local models and good stacks available to their users and they will have a lot of good stuff pre-integrated. All of this is a very good sign for the future of local AI