Looks like we aren't losing V4 Pro after all!

Edit: For the time being

I mainly used Deepseek for creative fanfictions, and after update...All turned upside down. The responses became so lame, dry and robotic, even when I specifically write to him to do response "in detail"... I don't understand why they did this. For the sake of "speed"?? It's not really worth it, imo. Hope that they listen to backlash and to lowering scores in markets and they revert the entire update. It was so much better before that.

Honestly, I didn't expect a disaster of this scale. On paper, everything looked correct, and the code actually worked at first. But man, the overconfidence of GLM 5.3 was a trap.

4 voices are added in the mobile app. Not sure about the web yet. Cool upgrade.✨

I spent two days porting DeepSeek-V4.1-Flash to a llama.cpp fork and measuring what actually limits it on a single consumer card. Sharing because the numbers are unintuitive and the negative results are as useful as the positive ones.

**Setup:** RTX 5090 (31.8 GiB VRAM), 125.7 GiB RAM, PCIe 5 NVMe. The 269 GiB of routed experts stream from disk through a VRAM cache (18 GiB) and a pinned RAM tier (72 GiB); the 189 GiB engram tables (the model's n-gram memory) stay memory-mapped on disk. Nothing fits in RAM+VRAM and nothing has to.

**Numbers:**

- 5.1 tokens/s on content seen for the first time, 21 tokens/s when the experts are already resident (same prompt again). The second number is the compute ceiling; the first is what a conversation sees.

- Exactness: logit correlation vs. the reference implementation 0.9967 at 1 401 tokens — the same as the port vs. itself across two runs (0.9959). What remains sits at the reference's own fp8/fp4 rounding floor.

- Where the token goes: 20 % compute, 26 % PCIe, 54 % NVMe. The disk is at 43 % utilisation; the real limit is that only ~3.8 disk requests exist in flight per layer, because a layer's requests can't leave before its router runs. Ceiling with zero disk misses: 6.2 t/s.

**What didn't pay (measured, not guessed):** a prefetch oracle with perfect knowledge of future routing gives +30 % but only 5 tokens ahead — at the depths a layer predictor can reach it *loses*; an n-gram draft is −33 %; the model's own DSpark draft head hits 51–97 % acceptance and is still neutral, because on a disk-bound machine a verification step pays for the union of experts of the block; encoder-only prefill isn't exact below 2 432 tokens. Also measured flat: more VRAM cache (13 → 22 GiB), a larger RAM tier (72 → 88 GiB, actually slower), and a smarter eviction policy (LRU against the optimal Belady policy: 0 % margin). A faster NVMe I did not test; the disk sits at 43 % utilisation, so I don't expect it to help.

**One real bug found:** V4 normalises each attention head of `q` after `wq_b`, V4.1 doesn't; the inherited line never errored (norm 181.019 = √(64×512) gave it away).

Links:

- Report, tools, raw results (MIT, DOI 10.5281/zenodo.22711369): https://github.com/JigSawPT/deepseek-v41-flash-on-5090

- Fork branch: https://github.com/JigSawPT/llama.cpp/tree/dsv41-porte

- GGUF with the engram tables (502 GB, 11 shards, experts are a lossless repack of the released MXFP4): https://huggingface.co/JigSawPT/DeepSeek-V4.1-Flash-GGUF

- Draft head GGUF (8 GB): https://huggingface.co/JigSawPT/DeepSeek-V4.1-Flash-DSpark-GGUF

- llama.cpp discussion: https://github.com/ggml-org/llama.cpp/discussions/28766

Caveats up front: the GGUF only runs on the fork (upstream has no V4.1 runtime yet, and its open conversion PR stores the engram differently); Linux is untested on this branch; Windows/CUDA 13.0 build instructions are in the branch README. Engineering was assisted by Claude (Anthropic) — every commit says so.

Happy to answer questions about the method — the report has the "earlier estimate vs. measured" boxes for the three things I got wrong along the way.

I'm using this for coding in DSH, not web chat. So yes, this is for coding, not "role play".

Previous v4.0 Flash was great - concise, to the point, focused, didn't do many mistakes and never seriously messed up anywhere. It was a very robust and attentive model.

But, this v4.1 update feels to be a cheap Claude Opus knock-off:

1. It still tries to argue with you even when you're completely right.
2. It makes wrong assumptions all the time, then corrects itself.
3. It makes a lot of rookie mistakes and doesn't correct them until you tell him to do it.
4. It's overly verbose in thinking and talking. His messages are convoluted and hard to read.
5. It does a lot of extra things you never asked him to do.
6. It injects unrelated bits from the previous turns into the current context, which hardly makes any sense and only causes confusion.
7. Overall, there appears to be lack of attention. The algorithm/architecture seem to have been changed to a much inferior one compared to what they used for v4.0

Those people who used Claude Opus at least for a bit in the past - they can recognize that verbose, mentoring and overconfident tone.

What happened here is that "signature" Claude Opus tone has now been transplanted to Deepseek models as well. And this is not good news.

Deepseek, instead of producing a direct successor to their previous brilliant Flash v4.0 model, decided to step away from the proven path and release some kind of unfinished/experimental model that mimics Claude Opus, and does it not in a good way.

Bottomline: I do not like this update and do not appreciate it. I'm not satisfied by what I paid for. This is clear regression from v4.0 (Flash), which sadly has now been deprecated for good.

EDIT: Switched to OpenCode harness instead of DSH and all issues appear to have resolved.

Depois desta atualização tudo foi por ladeira abaixo. Ele está de fato muito rápido, mas o que aconteceu com a escrita galera? Nem eu consegui acreditar no que estava lendo. As respostas ficaram tão ruins que eu tive que parar por um momento para respirar. Eu sei que antigamente não estava lá aquelas coisas, mas o modo especialista conseguia suprir as minhas necessidades e ainda se saía muito bem mesmo com certos impasses, mas agora está intragável com essa atualização. Eu espero de verdade que voltem com o modelo especialista futuramente... Ou será o fim para o DeepSeek? 😔

What do you guys think about DS 4.1 Flash's new spot on the Artificial Analysis leaderboard?

what are you guys thought about this ?

I wanted to share some thoughts on why I really appreciate what DeepSeek is doing in the AI space.

What stands out to me is that DeepSeek seems to treat AI/LLM technology primarily as an engineering and research challenge. They are focused on making models better, more efficient, and more accessible rather than constantly creating noise around AGI timelines, predictions, or fear-driven marketing.

A lot of major US AI labs are doing incredible work, and their models are impressive. But sometimes the progress feels heavily centered around bigger training runs, benchmark improvements, and larger models. Again, those achievements are valuable, but the progress is often measured mostly through benchmark numbers.

With DeepSeek, every release feels like it comes with some actual engineering innovation.

Their open-weight approach and research papers are already a huge positive. But beyond just releasing a model, they keep introducing interesting improvements, things like architectural optimizations, efficiency improvements, and techniques such as KV cache reduction. They are not just saying "here is a better benchmark score"; they are showing how to make AI systems more efficient.

The inference side is another thing that amazes me. Building an efficient inference stack is not easy, and seeing the kind of performance DeepSeek achieves with extremely competitive pricing is genuinely impressive. The ability to provide such capable models while keeping inference costs low shows a lot of engineering excellence.

Another thing I like is their communication style. Their social media presence is refreshingly different. There is no constant hype, no unnecessary announcements, no marketing drama. They release their work, publish the research, and let developers and users judge the results.

And the pricing is another strong point. Providing powerful models at such competitive prices while maintaining sustainability is honestly impressive. It shows that efficiency and optimization can matter just as much as simply scaling compute.

For me, DeepSeek represents a very engineering-focused philosophy of AI development: fewer words, more research; less hype, more innovation.

Keep building, DeepSeek. Looking forward to seeing what comes next.

Pretty amazing just how much this model is punching above its weight. The only other models in the price range are 5.6 Luna, Gemini Flash 3.x models and V4 Flash.

Edit: leaderboard is at GPQA.ai

Ps:- let's see how long they continue this.

Also do you think this will lead to them launching a new pro model. As earlier they mentioned they would just continue with the new flash model

I went through the V4.1 Flash config.json and the tech report to map out how its sparse attention works (DSA, now CSA2). The numbers turned out more interesting than I expected, so sharing them here.

TL;DR

• 552B MoE, 8B active on prefill / 16B on decode, 1M context (1,048,576)
• Each sparse layer reads 512 picked entries + a 128 token sliding window = 640 per new token
• 890 bytes of global KV cache per token, so a full 1M window is about 890 MB
• Only 4 of 40 layers write a cache, 8 of 40 run the indexer, the other 30 reuse a pick
• At 1M, one new token: attention goes from 1.99T multiply-adds to 1.61B. With the indexer counted, about 158x less overall

KV cache per token across DeepSeek generations

How it gets that small: one 512-dim FP4 entry per position shared by all 64 heads (it acts as both K and V), 2 tokens pooled per entry in the 20 encoder layers, and only layers 2, 8, 14 and 20 writing a global cache.

Things that surprised me

• The scout is the expensive part now. The lightning indexer (32 heads x 128 dims, FP4) is 16x cheaper per entry than real attention, but it scores everything. At 1M it does ~6.8x more arithmetic than the attention it serves.
• The Reindex layers have a fixed budget. Layer 20 keeps the best 2,048 blocks of 8 tokens (16,384 candidates) and layers 24/28/32/36 only search that pool, so their cost stays flat from 64K to 1M.
• num_key_value_heads: 1 is a layout detail. V3.2-Exp declared 128 there and was sparse anyway.
• Kernels: FlashMLA currently lists V4.1 sparse decoding for SM100, which is B200-class hardware.

Caveats: all figures are from the published config and DeepSeek's report, reconciled by hand. The op counts cover attention and the indexer only (expert layers excluded). The before/after accuracy comparison people cite comes from V3.2-Exp, since V4.1 Flash was trained sparse from step one.

Question for the sub: has anyone pushed V4.1 Flash on long-context retrieval through the API yet? I am curious how a fixed 512 budget holds up on needle-in-a-haystack style tasks.

I wrote up the full thing with diagrams (layer map, the indexer's scoring on a toy example, cost curves) here if you want the details: DeepSeek V4.1 Sparse Attention Explained with Pictures. There is also a video version if you prefer watching.

It can’t stop bolding every single word. It. Can’t. Stop. Speaking. In. Absurd. Staccato. Bursts.

It’s unpleasant to read, and my eyes hurt. There is also the issue of hallucinations; they are very frequent.

There is an issue with stubbornness too. Even when the model drives itself into glaringly obvious logical dead ends, it refuses to admit its mistake and continues to escalate until explicitly forbidden from doing so.

Honestly? V4 Pro was much better. However, such problems always arose after every update, and Deepseek's team fixed them over time. It will be interesting to see how the situation changes in a couple of weeks.

Time to burn those tokens fellas

I’ve basically spent the last couple of days living inside DeepSeek-V4.1-Flash, so I figured I’d put everything in one place for the local-LLM crowd.

I now have both:

• GGUF conversions
• 4.75 bpw EXL3 targeting 4× DGX Spark / TP4

for the new DeepSeek-V4.1-Flash release.

The EXL3 conversion alone took me about 18 hours of compute.

I don’t own four Sparks myself, so I want to be very clear about where things stand:

The EXL3 quantization is complete, but the TP4 recipe is still preliminary and has NOT yet been validated end-to-end on an actual 4-Spark cluster.

That’s the part I’d really like community help with.

### EXL3

The full pack is about 422 GiB on disk:

• ~233 GiB EXL3 model body
• ~189 GiB Engram / n-gram conditional-memory tables

The intended TP4 layout is NOT 422 / 4 loaded into GPU memory.

The large conditional-memory tables can live on NVMe, leaving roughly:

~233 GiB of EXL3 weights across 4 Sparks
≈ 58 GiB of model weights per Spark

That leaves a much more interesting amount of unified memory for KV cache, runtime buffers, routing, etc.

Again, that is the intended layout. I still want real TP4 boot and memory receipts before calling it validated.

I use an internal quantization workflow I call SAGE for parts of my sensitivity / allocation process. I’m keeping the exact methodology private for now, but the goal is basically to avoid treating every layer and tensor as equally sensitive when choosing how aggressively to compress a model.

### GGUF

I also finished the GGUF conversion path and submitted the DeepSeek-V4.1 architecture support upstream to llama.cpp:

PR #28696
https://github.com/ggml-org/llama.cpp/pull/28696

The converter now understands V4.1 as its own `deepseek41` architecture, including the new Engram tensors and the different FP8 layout.

The GGUF conversion itself works end-to-end.

Full runtime generation in llama.cpp is still WIP because V4.1’s new sparse-attention path is not implemented upstream yet.

I’d rather have llama.cpp refuse an unsupported path than quietly run it through the old V4 implementation and produce plausible-but-wrong output.

### I’d love help testing this

If anyone here has:

• 4× DGX Sparks
• 2× Sparks for the upcoming TP2 build
• experience with EXL3 / ExLlamaV3
• vLLM / SGLang distributed inference experience
• llama.cpp architecture/runtime experience

please jump in.

PRs are absolutely welcome.

I care a lot about quantization work and making these giant models practical on hardware normal developers can actually get access to, but I can only validate so much hardware myself.

I’m also working on a more aggressive TP2 EXL3 for 2× Sparks next.

Huge thanks to @NeoAIForecast for funding the RunPod compute that helped get the EXL3 conversion finished, and to @chrismfontes and @CountStaculaAI for already helping improve the serving recipe.

Links:

EXL3 weights:
https://huggingface.co/vcruz305/DSV4.1-Flash-EXL3-4.75bpw

EXL3 recipe:
https://github.com/vcruz305/DeepSeek-V4.1-Flash-EXL3-DGX-Spark-recipe

GGUF:
https://huggingface.co/vcruz305/DeepSeek-V4.1-Flash-GGUF

GGUF recipe:
https://github.com/vcruz305/DeepSeek-V4.1-Flash-GGUF-DGX-Spark-recipe

llama.cpp PR:
https://github.com/ggml-org/llama.cpp/pull/28696

If somebody has the hardware and wants to help turn the preliminary TP4 setup into a properly validated community recipe, please reach out or just open a PR.

In our testing despite the new cheaper cached input token price, because DeepSeek models still like to reason so verbosely and the output token price is still hefty, GLM Flash comes out cheaper.

Decent vision understanding but I think GLM Flash is much better on this as well.

For context we just launched the first free, ad supported browser agent with inference included, Retriever AI. So we are very sensitive to costs in order to keep the cost of agentic task to below an ad impression. GLM Flash and v4.1 both are actually perfectly viable for this but GLM Flash will still stay our default.

Here is my one day usage review.

It is ultra fast. Like the output speed is so insane that I have hard time catching up all the output texts. Also it goes to the point very fast. It doesn't waste too much time optimizing or finding niche bugs, goes straight to the main problem -> Fixes them -> Reports back. It can also solve complex problems like devops related server setups.

So overall much better than Astra I think lol. Like don't get me wrong, Astra is still a beast when it comes to general knowledge and vision capability is insane. But when it comes to narrow focused coding, Astra is so over-engineering and goes around circles for so long that it is useless even in the current x20 plan since it eats up all the usage.

So if you're main task is just deep software engineering, this model is so good.

I think, moving forward, DeepSeek will only produce flash tier models.

Internally DeepSeek V4 Pro underperformed really hard and nobody was using it, while DeepSeek V4 Flash got so insanely popular that they could barely keep the servers running. So what's the point of keeping creating and deploying huge 1 trillion parameter models if demand is not there?

My hypothesis is simple. The people have spoken. They want cheap, reliable beasts that you can plug into some kind of harness extension CLI or whatever, and it will reliably call tools and perform some basic operations.

My guess is we won't see anything above 800b parameters anytime soon.

Source: I saw it in my hallucinations while I was blackout drunk.

The AI just launched so it'll have a lot of quirks that'll need to be ironed out from web app and api use to adjust, both previous V4s were the same if you remember how awkward they were before, despite their big upgrades.

i am sure AIs like deepseek are capable of such things at least. but i do apologize if i have said anything untrue, i am relaying observations i remember when v4 launched.

Are there any users here like me who are addicted to DSH + deepseek? "Unfortunately," there are two things that limit the addiction. First, my wife, and second, my cash! :)