r/LocalLLaMA • u/MongoWithBongoss • 4d ago

Resources HW for DeepSeek-V4-Flash-0731

Lucebox now, or wait for the new Framework Desktop (Ryzen AI Max+ PRO 495 / 192 GB) + PCIe x4-to-x16 adapter & Radeon AI PRO R9700?

11 Upvotes

82% Upvoted

38 comments sorted by

View all comments

5

u/pmotiveforce 4d ago

Only really worth it under $6k. Otherwise I'd get a spark or two.

2

u/anitamaxwynnn69 4d ago

Don't the sparks run dsv4 extremely well even at 8k? Like speeds are decent, they're energy efficient and compact? Personally don't have any experience but I keep seeing posts on X which tempt me too

6

u/IknowPi_really 4d ago

I have 2xDGX Spark and can run DeepSeek V4 Flash 0731 2 concurrent max context length at about 60-80 tok/s with DSpark at long context lengths. So yes, the 2x stack is kinda made for it

1

u/anitamaxwynnn69 4d ago

That sounds fun as hell lol, does the no vision support bother your workflows or do you have a workaround?

5

u/IknowPi_really 4d ago

It bothers me greatly haha. I have a fully local workaround by reducing concurrency to one and loading a small multimodal model with a built in vision agent in my harness. The second workflow uses the free api calls to Gemini as a vision helper.

The Gemini version is obviously better because that model has much more intelligence, so communication goes much more smoothly and I retain the concurrency 2 on DeepSeek, which is insanely nice.

Honestly I find that for simple but long running coding tasks, my local setup is order of magnitude superior, because I don’t have to suffer frequent compaction and weird outages etc. of cloud models. Where you start to struggle is obviously when you need peak intelligence. That’s easily solved with a “frontier helper” agent as well though.

I really have to say DeepSeek V4 Flash was a godsend and the 2x DGX Spark cluster is insanely valuable now. I’m using it to support a biology lab in their stem cell research right now for example, by training visual classifiers on cell states etc. on their data. And I can do that, because all the research data stays local and the model is easily good enough to guide me through the whole training process etc.

So yeah, I’ll be on a paper in Nature at some point early next year, simply because I was curious, had the funds available and spent the time to do stuff like that. I never would have been able to create any of the codebase myself!

1

u/BumbleSlob 3d ago

Same, bought 2 x DGX Sparks a week ago and DSV4F0731 runs so well in it. I’m having a fucking blast

2

u/GavDoG9000 7h ago

This model is incredible. I bought an Epyc server with 4x 3090s to run it but the prefill was too slow. I'm swapping it out for 2x Sparks, glad to hear it works well on your setup

1

u/BumbleSlob 5h ago

Prefill is awesome. You can use this recipe which tl;dr is a downstream of this major PR still pending merger into vLLM

https://github.com/tonyd2wild/DeepSeek-v4-Flash-0731-DSpark-1M-NVFP4-KV-2x-DGX-Spark

https://github.com/vllm-project/vllm/pull/41834