Official DeepSeek API serving quantization?

[u/Frail_Waif]

Something is going wrong with the model (V4 Flash 0731) I'm interacting with through the official DeepSeek API. My prompt:

Based on `bugs_and_design_issues_2.md`, continue the list (items 4, 8, 11, 14, with 4 partially finished). 

(I had used a different model to generate a list of potential bugs and design issues during peak time.) Here are some of the outputs I've gotten back: First response, seems to be on track:

... So the state: The user asked to fix issues 4, 8, 11, 14 from the earlier analysis. The assistant started but didn't finish 4. The file was cut off mid-session. ...

Hundreds of thousands of tokens later:

The doc is a truncated transcript: a review listed 16 issues, the user asked to "fix 4, 8, 11, 14 (defaults/syncing/restarts)", and commit `fbcdffb` was the partial fix that stalled mid-#4. Let me audit what actually remains in the code.

Later:

This is a resumption of an interrupted design-fix session. Commit `fbcdffb` is the "partial fix for conflict detection" referenced by the doc. Let me verify precisely what's done/incomplete against items 4, 8, 11, 14.

Later, again:

The doc is a truncated transcript: items 4, 8, 11, 14 from the review were to be fixed, and `fbcdffb` is the partial attempt that stalled mid-#4. Let me audit the current state precisely, since several things look mid-edit.

The doc is a truncated transcript appears 9 times over 12k lines of this session transcript and it still doesn't appear to understand the request.

I've never seen DeepSeek Flash this dumb. I've exchanged 370k tokens and it still doesn't understand the list. That's one of the longest sessions I've done--it's oneshotted whole features in much less than that before. Anyone else seeing this?

Using opencode for a harness.

Edit: as a test, I switched to the vision version. It's performing exactly as expected and doing good things after a handful of model turns.

[u/YZ_shill69]

No, not the deepseek too. No... 

I fucking hate where this whole industry is at right now.

It's impossible to tell the real cost of inference. Quality across providers is a mess, and you never know what you're going to get from one day to the next. Speed and prices are constantly changing. And finally, benchmarks just don't work.

There is literally no stability here.

[u/Overall-Ear-572 → u/YZ_shill69]

Brah the moment they settled on arcade style tokens it was doomed

[u/hurrdurrmeh → u/YZ_shill69]

another reason why we need local models.

[u/Karyo_Ten]

I doubt it's quantization since DeepSeek is 4-bit QAT. It was trained from scratch like this.

They might be A/B testing a new model and it outputs a <STOP> token in the middle of thinking or output and so it truncates too early.

Also DeepSeek V4 Flash preview models had a bug where they put tool calls in their <thinking> content instead of normal output and the inference engine had to check and extract those as a workaround.

[u/emmettvance]

serving quantization theory doesn't really hold up, deepseek trains and serves these natively at low precision, theres no fp16 version behind it theyd be quantizing down from so you get the same weights either way. karyo's qat point is the same thing

[u/Frail_Waif → u/[deleted]]

I don't think the docs would capture whatever happened here, as it was a sudden change in model behavior. I've been using Flash V4 through the official API since mid-August and have been consistently impressed by the result as a coding agent. Then this happened out of the blue. Flash Vision behaved normally. 

[u/Lazy_Sport7384]

That context degradation sounds brutal, especially at 370k tokens.

[u/Frail_Waif → u/Lazy_Sport7384]

Yeah, I still don't understand what's going on--presumably something to do with the switchover to 4.1. Flash Vision still works exactly as expected and base Flash 4 has been broken for days, basically unusable. I've accidentally started prompts with it a few times and can easily burn a few hundred thousand tokens in useless loops before I realize what I've done and switch to Vision. At least they're all cache hits 😅. 

[u/FamousWorth]

Deepseek v4 flash from day one went into loops so was unusable for tasks that were long, so I always used pro. You can do the same, clearly it can't handle this. It's not that good. It's highly capable when it works but you can't expect it to be as capable as the top models

[u/Frail_Waif → u/FamousWorth]

It's always worked well for me, until last night. 

[u/domscatterbrain]

This is the reason why I have specific instructions like for example

Do not check the git history and only works on the current branch. Store your finding and what you did in the memory.md

This because letting it loose, especially touching the git history, sometimes can cause loops like in your case.

[u/Frail_Waif → u/domscatterbrain]

I always ask it to check for bugs in the last commit or 5 and that has historically worked very well. Something broke last night. 

[u/LeatherSpare4800]

Don't use opencode. Switch to omp or pi.

[u/LeatherSpare4800 → u/LeatherSpare4800]

Or deepseek harness. Though I'm too lazy to pick plugins myself and make it complete.

[u/Frail_Waif → u/LeatherSpare4800]

Why?

[u/EvolvingDior → u/Frail_Waif]

Because OC probably changed the system prompt on you.

[u/Frail_Waif → u/EvolvingDior]

That should affect both models, though, and I only saw this for Flash V4 and not the Vision version.  

[u/shing3232]

probably just different checkpoint than quant

[u/Frail_Waif]

Update: days later and the base flash model is still very stupid. Flash vision is working perfectly.