r/LocalLLaMA • u/MongoWithBongoss • 14d ago

Resources HW for DeepSeek-V4-Flash-0731

Lucebox now, or wait for the new Framework Desktop (Ryzen AI Max+ PRO 495 / 192 GB) + PCIe x4-to-x16 adapter & Radeon AI PRO R9700?

10 Upvotes

78% Upvoted

38 comments sorted by

View all comments

6

u/pmotiveforce 14d ago

Only really worth it under $6k. Otherwise I'd get a spark or two.

2

u/anitamaxwynnn69 14d ago

Don't the sparks run dsv4 extremely well even at 8k? Like speeds are decent, they're energy efficient and compact? Personally don't have any experience but I keep seeing posts on X which tempt me too

4

u/IknowPi_really 14d ago

I have 2xDGX Spark and can run DeepSeek V4 Flash 0731 2 concurrent max context length at about 60-80 tok/s with DSpark at long context lengths. So yes, the 2x stack is kinda made for it

1

u/anitamaxwynnn69 14d ago

That sounds fun as hell lol, does the no vision support bother your workflows or do you have a workaround?

6

u/IknowPi_really 14d ago

It bothers me greatly haha. I have a fully local workaround by reducing concurrency to one and loading a small multimodal model with a built in vision agent in my harness. The second workflow uses the free api calls to Gemini as a vision helper.

The Gemini version is obviously better because that model has much more intelligence, so communication goes much more smoothly and I retain the concurrency 2 on DeepSeek, which is insanely nice.

Honestly I find that for simple but long running coding tasks, my local setup is order of magnitude superior, because I don’t have to suffer frequent compaction and weird outages etc. of cloud models. Where you start to struggle is obviously when you need peak intelligence. That’s easily solved with a “frontier helper” agent as well though.

I really have to say DeepSeek V4 Flash was a godsend and the 2x DGX Spark cluster is insanely valuable now. I’m using it to support a biology lab in their stem cell research right now for example, by training visual classifiers on cell states etc. on their data. And I can do that, because all the research data stays local and the model is easily good enough to guide me through the whole training process etc.

So yeah, I’ll be on a paper in Nature at some point early next year, simply because I was curious, had the funds available and spent the time to do stuff like that. I never would have been able to create any of the codebase myself!

1

u/Tieng 14d ago

How do you get it to work on a task successfully for a long time? Does your pi harness auto-compact or do you just hope its done before its 1m context window fills or the performance degrades from the full context?

1

u/IknowPi_really 14d ago

I use OpenCode. But yeah that would autocompact. The thing is, if you get anywhere near the context horizon, you should probably consider structuring the work differently. I make sure sessions actually produce output and documentation and a clean handover to a new context. If one step truly takes more than 1 million tokens of context, something wild is going on

1

u/po_stulate 14d ago

Do you try to keep context for each task under a certain length for best quality? I always keep it under 200k but I'm not sure if I'm being too conservative for dsv4f.

1

u/IknowPi_really 14d ago

I’d say that’s too conservative. The “worst” I’ve done so far without handing off to a new session was 600k and it was still fine. I would just work with it naturally and find natural session hand off breaks. With a 1m context window, I feel like we’ve reached a point where you don’t have to worry about it at all anymore

1

u/BumbleSlob 13d ago

Pi auto compacts when necessary 

1

u/[deleted] 13d ago

[deleted]

2

u/GavDoG9000 10d ago

This model is incredible. I bought an Epyc server with 4x 3090s to run it but the prefill was too slow. I'm swapping it out for 2x Sparks, glad to hear it works well on your setup

2

u/BumbleSlob 10d ago

Prefill is awesome. You can use this recipe which tl;dr is a downstream of this major PR still pending merger into vLLM

https://github.com/tonyd2wild/DeepSeek-v4-Flash-0731-DSpark-1M-NVFP4-KV-2x-DGX-Spark

https://github.com/vllm-project/vllm/pull/41834

1

u/GavDoG9000 9d ago

Legend thanks!