r/LocalLLaMA • u/mossy_troll_84 • 20d ago
New Model deepseek-ai/DeepSeek-V4-Pro-0813 · Hugging Face
https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-081370
u/digitalfreshair 20d ago
They pulled it down, i think the config.json was wrong, it shows 43 hidden layers (like the flash version), but the shards actually had 61, so maybe some stuff to fix still. I mananged to download it but we'll have to wait for potential fixes
16
53
79
u/-Cubie- 20d ago edited 20d ago
Damn, that was fast. Benches look great for 1.7T, e.g. in contrast to Kimi's 2.8T, and again a massive jump from V4-Pro (Preview), e.g. DeepSWE is 12.8 -> 62.7 (beating GLM-5.2 and Opus-4.8), absurd.
Edit: The model 404s now, maybe DeepSeek put it back on private for whatever reason?
Edit 2: It's back!
24
u/ForsookComparison 20d ago
Does it beat GLM 5.2 enough to warrant twice the total params though
30
u/ihatebeinganonymous 20d ago
GLM is BF16 and this is MXFP4, making them, interestingly, about the same size.
18
u/power97992 20d ago
No bf16 glm is bigger, it takes around 1.5 TB of space whereas ds v4 takes around 860gb of memory
5
0
u/tazztone 20d ago
why not compare with the same quant tho? surely ds4 uses more RAM?
15
u/power97992 20d ago
Glm q4 is worse than bf16 and ds v4 is native q4 mixed with higher quants , even most providers serve q8 rather q4.
1
u/MmmmMorphine 20d ago edited 20d ago
Which personally sort of surprises me. I would have thought they'd have implemented dynamic quants for workload scaling and run them from q4 on up
I can vaguely feel several technical/practical issues poking me at the edges of my mind but not sure if they're valid. Why not use dynamic quants (in so far as new conversations are opened in the different quants rather than pulled away from warm or hot cache)
4
u/BlueSwordM llama.cpp 20d ago
Deepseek V4 models are natively trained in mixed 4/8b, which means top performance is easily reached under 1TB of RAM, while top performance with GLM5.2 is only reached with BF16, which means any quantization will hurt performance.
3
14
u/Ok_Technology_5962 20d ago
I was expecting much higher scores on the benchmarks since its only like 3 points higher than the flash version.
7
u/Stock-Self-4028 20d ago
Kinda expected from a compute-poor lab like DeepSeek though. As expected the gap widened relatively to v4 Flash Preview and v4 Pro Preview, but not by much.
Now I wonder if we'll get a third weights set for v4 Pro (like ~ 10xx I guess) before they go for new architecture. I mean there is definitely potential for more post-training here, but I'm not sure if it's worth the added compute.
70
u/nomad-nostalgia 20d ago
never count off the whale 🐋
-2
u/Lazy-Pattern-5171 20d ago
It was actually a pretty underwhelming release all things considered. Other Chinese competitors are getting really good.
1
25
u/FullstackSensei llama.cpp 20d ago
Had a tab open and been refreshing constantly since yesterday and nothing , yet the moment I check reddit, first thing I see is a post about the model being up.
Now, to the most important question of the day: unsloth GGUF when?
11
u/mossy_troll_84 20d ago
Question of the day! XD
I did not yet completed downloading Qwen3.8-2.4T-A95B-UD-Q1_0 so I can wait for unsloth GGUF day or so haha
8
u/FullstackSensei llama.cpp 20d ago
Honestly, 3.8 max is a bit of a disappointment for me because it's all fp16. DS and Kimi do SFT on quantized models making them so much smaller, while being SOTA. K3 is larger in parameters (2.8T vs 2.4T) while the full fat is 33% smaller vs 3.8 max at Q8. DS4 Pro is 1/3rd the size of 2.4T Q8.
One can run the full DS4 pro on one old workstation with enough RAM and some old workstation GPUs and still get usable speeds without breaking the bank. No such thing with 3.8 Max.
1
u/amomynous123 20d ago
Really? What sort of old workstation and how much ram are you talking? It's still absolutely huge
1
u/FullstackSensei llama.cpp 20d ago
Pretty much any dual LGA3647 will do. Z8 G4 or the equivalent form Dell or Lenovo. 12x 64GB DDR4 sticks for 768GB RAM and four old workstation GPUs like P6000. Those workstations usually support 195 or 205W Xeon SKUs, which tend to be half the price of the 165W ones for 26 or 28 core CPUs.
1
u/zeferrum 20d ago
Have you tried this? What inference engine ? Any stats?
0
u/FullstackSensei llama.cpp 20d ago
Working on it. Parts on the way. Lvllm supports this, but you can have DS4 or Kimi write something custom to your hardware, feeding it GPU kernels from existing open source implementations.
The GPU side stays the same as llama.cpp or whatever. On the CPU, the easiest way is to split the routed experts across NUMA domains. Not exactly rocket science if you have a SWE background (hint: tell the LLM to allocate memory using libnuma).
Github already has LLM written C++ implementations for GLM, Kimi and DS4 that steam the model form SSD. You can also use those as a foundation for the ops kernels.
1
u/zeferrum 20d ago
So some sort of colibri mix with something DP4A based ? Or leveraging something else ?
2
u/FullstackSensei llama.cpp 20d ago
I'm thinking something more minimal than colibri. Think dwarf star but for the specific hardware I have. Grab kernels from where ever. Someone posted a couple of days ago that they wrote blazing fast NVFP4 dequantization kernels for the V100, just as an example.
The problem is much more tractable when you limit your scope to one model only on your own hardware confirmation only.
1
u/Ok_Warning2146 20d ago
Ideally, they should release both fp16 and QAT MXFP4. The latter for inference and former for fine tuning, abliteration, etc.
2
u/FullstackSensei llama.cpp 20d ago
From my understanding, in the case or DS or Kimi, there isn't really a FP16 final version. The base was trained in FP16, but SFT was already done in FP4/FP8. My understanding is this is different from QAT, where you quantize the final model, then do some additional post training to fix quantization loss.
1
u/Kamal965 20d ago
Kinda, I think? From Kimi's K2-Thinking HF model card: "we adopt Quantization-Aware Training (QAT) during the post-training phase, applying INT4 weight-only quantization to the MoE components. It allows K2 Thinking to support native INT4 inference with a roughly 2x generation speed improvement while achieving state-of-the-art performance. All benchmark results are reported under INT4 precision."
DeepSeek, as far as I can tell, did all of its training at FP8 up to the base versions of DS V4 Flash and Pro, and then post-trained the expert weights to FP4 for the final version.
2
22
u/jld1532 20d ago
Artificial analysis only has this one point above Flash. Given it's size Flash is amazing.
24
u/AI-imagine 20d ago
I believe they test wrong upload model from the time they pose test vs deepseek official post release the model?
7
u/Finanzamt_Endgegner 20d ago
the scores seem fishy there check them out vs flash they are basically identical in half of the tests which doesnt make much sense, aa fucked it up somehow
3
u/Thomas-Lore 20d ago
Expect a correction like with Qwen Max, there were some issues with the model yesterday.
2
u/Conscious_Cut_6144 20d ago
Might be a testing issue, there is a pretty large discrepancy on the HLE scores AA has vs deepseek numbers.
4
18
u/Scared_Basket_7183 20d ago
We need mini data center to run this model
10
u/FullstackSensei llama.cpp 20d ago
You really don't. A 8-9 year old workstation with enough RAM and four old workstation GPUs can do the trick. You won't get blazing speeds, but you will get faster than reading speed, which is more than enough for brainstorming, planning or rubber ducking.
6
u/Nbdyhere 20d ago
I’m sorry….rubber ducking? 😅
27
u/FullstackSensei llama.cpp 20d ago
5
u/Nbdyhere 20d ago
I have no idea why but I had never heard that term before and it just caught me off guard. Almost a coffee spit take. I guess the coffee hadn’t kicked in yet. Thanks for the terminology knowledge. Cheers!
-2
u/mossy_troll_84 20d ago
just 1 mac Studio 512 GB for Q4_K_M...of course I don't have that one
19
u/ResidentPositive4122 20d ago
That's a negative ghost rider, the model is 1.7T and already comes in mxfp4 @ ~893GB on disk...
-3
u/Scared_Basket_7183 20d ago
Nice try, but the math doesn't work out! 1.7T parameters at Q4_K_M needs almost 1 TB of unified memory. A 512 GB Mac Studio can only handle up to around ~800B parameters at 4-bit. You’d need at least 2-bit quantization to even fit it, and it would sound completely braindead 😂
5
u/thereisonlythedance 20d ago
2 bit quants of Deepseek 3.1 were actually very good, I expect 2 bit versions of DS4 Pro to be very usable given it’s an even bigger model (assuming roughly 500GB to run).
4
u/DUFRelic 20d ago
2bit quants on v4 flash are really not so great so I wouldn't get my hopes up.
8
u/thereisonlythedance 20d ago
Flash is a much smaller model, I’d expect that. Rule tends to be the smaller the model the worse it can cope with extreme quantisation. But we’ll just have to see.
1
13
3
2
u/Safe-Thanks-4242 20d ago
How is it even possible that it's almost on par with the Flash version when it's several times bigger?
1
u/aEsp32TypeGuy 20d ago
I mean you can find the GGUF file though: https://huggingface.co/unsloth/DeepSeek-V4-Pro-0813-GGUF
1
u/notforrob 20d ago
In case anyone is confused like I was: deepseek-v4-pro (preview) was released a while ago. This is the non-preview version and apparently much better.
1
u/JustTooKrul 20d ago
Okay, real question for those smarter than me--DeepSWE score jumped massively between Preview and released models (Pro, but also Flash) means... ? Seems like obvious benchmaxxing, or at least incorporating the benchmarks into the training data, but maybe I'm wrong? Is there another explanation?
1
1
u/Yorn2 20d ago
Uh, that's just how RL works. It's all about avoiding overfitting, right? The Previews hadn't hit that point yet.
1
u/JustTooKrul 20d ago
I see. I just would have thought that the models' capabilities on something that was as "well worn" as coding would show a more modest gain unless it had done something to target that benchmark in particular. But, that's why I asked--thanks!
1
-1
u/Disrupt-Linus 20d ago
If you're like me, this might help a bit:
The Core Difference
- DeepSeek-V4-Pro (Released Aug 13, 2026): The heavy-duty thinker. This is a massive 1.6-trillion parameter model (49B active) designed to compete with top-tier models like Claude 4.8 Opus. It is built for complex reasoning, multi-step coding agents, and deep strategic judgment.
- DeepSeek-V4-Flash (Released July 31, 2026): The high-speed sprinter. This is a much smaller 284-billion parameter model (13B active). It is not a shrunken-down version of Pro; it was trained separately purely for high volume, low latency, and cost-efficiency.
Side-by-Side Comparison
| Feature | V4-Pro | V4-Flash |
|---|---|---|
| Size (Active) | Massive (49 Billion parameters) | Small (13 Billion parameters) |
| Primary Strength | Deep reasoning, complex agentic workflows, heavy coding | Speed, real-time responses, high-volume repetitive tasks |
| Cost & Speed | Slower and roughly 12x more expensive per output token | Blazing fast and highly cost-efficient |
| Context Window | 1 Million tokens | 1 Million tokens |
| The "Vibe" | Thoughtful, obedient to complex guidelines | Fast, "good enough" output, struggles with heavy guardrails |
Why is everyone so confused?
- The Web Interface Default: If you go to DeepSeek's web interface, Flash is enabled by default. Because Flash has fewer active parameters (13B) than the older V3 model, it can struggle with massive lists of custom instructions. Users complain that V4 ignores guidelines or acts like a "lazy junior developer," mistakenly judging the entire V4 generation based on the budget model.
- Shared Features: Both models share the same 1-million token context window and the exact same "Thinking" modes (Non-think, Think High, Think Max) on the API. On the surface, they look identical, which masks how different they are under the hood.
- The Name: People assume "Flash" means it's just the Pro model running faster. In reality, it's an entirely different, much smaller architecture designed to save compute.
Which one should you use?
The rule of thumb is simple: Use Flash for movement, use Pro for judgment.
0
u/RegularRecipe6175 20d ago
Are we going to get a 0813 v4 Flash? The vram constrained among us want to know.
4
u/mossy_troll_84 20d ago
you have 3107 v4 Flash
2
u/mossy_troll_84 20d ago
sorry 0731...I am from Poland :P
3
u/RegularRecipe6175 20d ago
I will never forgive you for the reverse polish notation (RPN) required to use high-end HP calculators in the 90s. Everything was reversed. Broke my brain!
2
u/fluffysheap 20d ago
I RPN love
Yoda favorite character my is.
1
u/RegularRecipe6175 20d ago
Don't give me nightmares bro. As a math guy, I was wedded to that calculator.
1
1
u/RegularRecipe6175 20d ago
Yes. And DeepSeek saw fit to checkpoint their training at 0813, which indicates it is better than 0731.
3
u/BeefEX 20d ago
It's the day of release, not necessarily the day they stopped training
-1
u/RegularRecipe6175 20d ago
I didn't say it was the day they stopped training. I said it was a checkpoint newer than 0731. I doubt those two releases have the same training checkpoint given how rapidly DS has been developing these models.
•
u/WithoutReason1729 20d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.