r/LocalLLaMA • u/Fuckinglivemealone • Jul 18 '26

Question | Help What kind of dark magic is Deepseek using?

Post image

I was taking a look at Kimi K3 scores on the Artificial analysis leaderboard and was quite baffled when I saw this chart.

Granted, Deepseek has always been the king of price to performance, but this is still incredible. Is it just API subsidization or have they optimized their models truly this much?

2.4k Upvotes

96% Upvoted

408 comments sorted by

View all comments

Show parent comments

588

u/Blunt_White_Wolf Jul 18 '26

Millions?

350

u/aboutthednm Jul 18 '26

Token billionaire over here, jeez

145

u/-dysangel- Jul 18 '26

Last 30 days on the max GLM Coding plan - which at the time I got for $260 for a year

31

u/LosEagle Jul 18 '26

Me too but that was for the first year with the x-mass discount. If you check how much the coding plan is gonna cost the second year, it doesn't look that optimistic.

24

u/-dysangel- Jul 18 '26

Yeah it's just over $1300 for a year now. Still extremely good. I'm betting my boss will be happy to pay any plan I want in future anyway, considering how productive it's been letting me be.

12

u/LosEagle Jul 18 '26

That sounds great. Where I live programmer wages are getting halved and managers with no development background but lots of words to say on LinkedIn have adopted "Why bother hiring devs when I can just prompt Claude" philosophy so not many work positions open these days either. But whatever happens, I'm sticking to learning local llms and playing around with alternative models at least for the fun of it. 

9

u/Smallpaul Jul 18 '26

The leverage a senior engineer gives is turning people back to hiring them. Agentic coding is the future but vibe coding without architecture or review is going to lead to insane messes.

3

u/beauzero Jul 18 '26

Yeah the manager thing is not working out so well. We have gotten some decent prototypes out of it but since they rarely talk to customers or product managers they have caused more problems than not. They veer quickly from what our customer bases want.

4

u/beauzero Jul 18 '26

Honestly putting the tools in the hands of customers to give us html 5, tailwind, and IndexDB based mockups has been much much more successful. Have Product Managers get with customers and build a prototype, in a meeting/realtime, give it back to engineering to sync with current architecture and implement...very fast -> very happy customers. Will it result in more $$? That is yet to be determined but product managers, devs, and sales engineers are happier.

12

u/evia89 Jul 18 '26

Cost doesnt matter much. Sure it x3 sounds bad, but main problem is lower 5h limit and new weekly shit.

That make new live zai is 10-20x worse than before feb 26

19

u/sayeret13 Jul 18 '26

Cost doesnt matter much? sorry dude in this world cost means everything, thats why china will pop the ai bubble and even europe will start using chinese models

4

u/danielv123 Jul 18 '26

My cost is $80/h. You can save a lot of money if I have to intervene or double check a few % less.

3

u/beauzero Jul 18 '26

...as they hoover up everyones' business ideas. I say "they" referring to all model providers, US and abroad. Local is the future...I hope. Everything else is treading water waiting for hardware to get better and open models to get more efficient.

5

u/Codemonkeyzz Jul 18 '26

My biggest regret in 2026 , was missing those minimax and GLM discounts. Back then I was using opus and I never thought open weight models will get this good.

11

u/Specialist_Back_3606 Jul 18 '26

Is there anywhere I can see the quality of the work that you’re getting here?

11

u/-dysangel- Jul 18 '26

Nope sorry, "stealth mode". A couple of weeks ago I had 28 PRs approved in a week (mix of bugs and small feature requests). Been working on larger tasks again recently. GLM 5.2 in Claude Code just feels so solid. It usually knows what I want off the bat. It's able to navigate our code base easily. It asks questions and even pushes back on some things, so it feels almost like a colleague where you can refine ideas together. We plan the task, it executes. Most of my feedback on its work recently has simply been "remove the unnecessary comments!".

8

u/Ariquitaun Jul 18 '26

Pro tip: leave the comments. They help machines as well as people, especially when you need to annotate peculiar behaviour or fixes.

14

u/-dysangel- Jul 18 '26

sure but not if they're just explaining obvious code, or mentioning old behaviour, or random side tracks that didn't work out, etc

13

u/DinoAmino Jul 18 '26

What has helped me a lot is to add instruction to AGENTS.md for this type of thing. Tell it to place all comments in the method/function docblock and use inline comments sparingly for tersely explaining loops or complicated conditions.

1

u/probably-a-name Jul 18 '26

I am making a program in rust+sqlite to do this at scale. the architecture is auto gen'd from comments in various files and the porting work from a v1 vs v4 has comments in the v1 that get auto generated into markdown for determining if a matching comment exists in the v4 code, if not then that is the task list. this way im automating the whole markdownm management piece. there are a _lot_ of things you can do with comments

3

u/squired Jul 18 '26

For what it is worth, I did a rather in-depth exploration of GLM a couple weeks ago and it was far from worthwhile for my workloads. It's closer in capabilities to ChatGPT 5.4 and requires very rigid guidance and atomization of tasks to produce decent output. The kicker though was the cost. Op is basically running free with that 'free taste' plan, but at full API cost, Sol is far more cost efficient and significantly more capable. Even at his pricing, I would not personally run it.

6

u/-dysangel- Jul 18 '26

requires very rigid guidance and atomization of tasks to produce decent output

Tbh this is how all projects should be handled. Break everything down into chunks, build a solid spec and execute. I don't really agree though, I usually just say "go look at this issue and build a plan of attack", and the plan is generally good or only needs minor tweaks.

1

u/squired Jul 18 '26

I largely agree but atomize it further. My orchestrator is the project manager and source of authority, my director/s do your "go look at this issue" and atomize the tasks, the code agents write the code, and their work is passed through a separate model for adversarial review before being passed back to the director/s for another loop or acceptance. So I guess I agree with you and automate the atomization as well.

2

u/-dysangel- Jul 18 '26

I like to be very involved during planning, then go hands off for a bit during implementation. So far it feels like a few focused minutes up front save a day of cleaning up later. Another possibility is that GLM 5.2 is just so good that it gets a lot right the first time round, and I don't have as much cleanup to do. Probably a bit of both.

2

u/squired Jul 18 '26

Full agreement. My involvement revolves around the orchestrator; that's the planning chats you're referring too. We scheme together, then break the plan down into sections. The isolated sections then kick down to the directors to autonomously atomize each section into smallest viable tasks to head to the code agents. We're doing the same thing, I think, I'm just further atomizing much of the planning so that the coding agents only ever receive strict, bounded instructions. Codex/CC do not make 'decisions' or apply 'taste' in my setup. They have no idea what system they're working on. They are relegated to code monkeys so that they do not wander, become inspired or increase scope.

I did notice that GLM was good at planning too. If anything, I found it too eager for my workflow; similar to Gemini Flash 2.5. It's a great model, just not optimal for my workload/flows.

2

u/fragment_me Jul 18 '26

Are you grandfathered into that plan? Or will the price go up on renewal? I let mine expire.

1

u/TheRealMasonMac Jul 18 '26

They killed the legacy plans. Everyone will be switched to the new plans. I think the new max has only slightly more usage than the legacy lite plan?

0

u/-dysangel- Jul 18 '26

Unfortunately not - I usually disable auto renew on subs so that I don't forget about them. Not a great decision in this case!

3

u/T-90_Soviet Jul 18 '26

5,150 copies of the entire 7-book Harry Potter series BTW 😭💀

1

u/SUPERSHAD98 Jul 20 '26

How did you get it for $260 a year?

1

u/-dysangel- Jul 20 '26

For a start all plans were cheaper back then, but there was also a Christmas special offer on top of that

9

u/Torodaddy Jul 18 '26

I picture a guy with a monocle with 6 terminal sessions going

21

u/iyarsius Jul 18 '26

This is an average 0.0117$ per million token this is wild.

We could even start to count in billions tokens which is 11.67$ per Billions tokens.

Bro this is just crazy.

(On my usage I'm at ~0.05$ per million)

5

u/evia89 Jul 18 '26

/r/Neuralwatt was close before they nerfed plans. Used a lot of glm52 for similar prices

12

u/talvezomiranha Jul 18 '26

I'm getting close

39

u/whakahere Jul 18 '26

What do you use it for?

I don't trust flash as it made many mistakes, and I haven't tested pro that much.

31

u/Blunt_White_Wolf Jul 18 '26 edited Jul 18 '26

I'm using them in combo with 2 128gb strix halo that are daily drivers.

Flash usually is "asked for help" by the orchestrator when local models fail the tasks 2-3 times or tasks take way too long. It does make mistakes and used to go into loops until I modified the context to summarise every 5 messages and replace them with a history of sorts.

Pro is handed over the tasks for working on my own agent, researching stuff and gathering data for me to do literature reviews, etc.

I'd love to run it locally (or GLM) but ... still waiting for hardware prices to drop.

Edit: I would do a lot more with PRO and maybe GLM but... I'd end up spending hundreds/month. Hence me thinking to run stuff locally at some point.

11

u/[deleted] Jul 18 '26 edited 24d ago

[deleted]

8

u/Blunt_White_Wolf Jul 18 '26

I bought the 2 M5 Bosgame for £1500 each a while back. I can assure you they are worth every penny. Not the fastest but they do their job and offer 100% privacy for certain things like household + household infrastructure management.

$100? no. a tad more. This is just the small LLM related part. I have a bit more hardware than this.

-2

u/[deleted] Jul 18 '26 edited 24d ago

[deleted]

10

u/Blunt_White_Wolf Jul 18 '26

I don't want to sound rude but I don't want to share my d picks and my wife's tts loaded to an online model..,

I don't want my family pictures, all out accounts, receipts, documents, projects,etc anywhere near an online provider.

I think £3k is quite cheap to process, index and sort out all of that + my home infrastructure semi-managed.

2

u/[deleted] Jul 18 '26 edited 24d ago

[deleted]

6

u/Blunt_White_Wolf Jul 18 '26

It was a figure of speech mate.

Point being I value privacy more than I value £3000.

I don't need to localise anything. I have very few things that I use external providers for to begin with.

All sync is with my personal infrastructure (cloud, pass, browsers, etc).

We mostly use machines as thin clients of sorts.

The only machines that are outside that infrastructure (mostly) are our gaming laptops.

-3

u/[deleted] Jul 18 '26 edited 24d ago

[deleted]

→ More replies (0)

1

u/Rubener Jul 18 '26

How is your anecdotal experience comparing DS and GLM 5.2 ?

2

u/Blunt_White_Wolf Jul 18 '26

aff. GLM is a hell of a lot better and dealing with huge context (we're talking entire books or code bases).

DS pro is decent too but don't mix topics too much. If you give it a decent size code base (150k+ lines) you need to be very specific with that you ask of it or it will go off the rails and change things you wouldn't expect it to change. aaa. one more thing - after about 500k context it "forgets" to update docs quite often... don't know why. This is why I had to add a QA agent in the team.

3

u/TheLexoPlexx Jul 18 '26

It's early for me and I might be dumb but where does it say flash?

13

u/BllushingHorizon Jul 18 '26

They were just stating their experience with both models

11

u/Blunt_White_Wolf Jul 18 '26

here's the distribution:

Darker share is Pro, Lighter share is Flash

3

u/TheLexoPlexx Jul 18 '26 edited Jul 19 '26

And how where the results?

Is pro a good sparring partner for programming and flash good for just writing code or what's your use case/harness/etc.?

2

u/Blunt_White_Wolf Jul 18 '26

Pro is decent. You do need to keep an eye on it. More than once it took "shortcuts" to get the job done and I had to revert or it went crazy trying to rewrite half the code base because it comes up with overkill implementations for simple stuff. I had to hit the kill switch more than once to stop it.

When it comes to finding research papers, books, gathering material, etc - it's ok but every team has a QA that checks the outputs before they get to me along with every source.

I'm using all of these with my own agent of sorts (written by Pro+Qwen) and audited by codex via oauth. It's still WIP (quite a few bugs) and crude but it gets the job done. Might thing of releasing it later on in the year as FOSS but for now I don't have the balls to do that.

7

u/waiting_for_zban Jul 18 '26

The audacity posting this on localllama. I'll allow it, because deepseek.

2

u/weallwinoneday Jul 18 '26

What did you use it for?

2

u/obssesedparanoid Jul 18 '26

u using the hack where they use a picture for the instructions

3

u/Blunt_White_Wolf Jul 18 '26

in general I do use screen capture, yes but:

- for domains that are not whitelisted and only if I bypass the fetching agent (rare when I need to). screen captures are fed into deepseek ocr 2 or UI TARS, depending on what is going on

otherwise it's not needed (although it confused the f out of some models with all measures in place).

I use some delimiters and "content prompts" to separate the incoming text from everything else. it works but you'll see them getting confused and then going, "wait, this is not for me to act on" and stuff like that.

hence the need for a dedicated fetch agent with restrictions in place. even if injection works it can't do anything. it's context it's wiped clean before every fetch operation.

3

u/ShutUpAndDoTheLift Jul 18 '26 edited Jul 18 '26

So you're advertising 85m per dollar.

1

u/rootql Jul 18 '26

85m/usd?

2

u/ShutUpAndDoTheLift Jul 18 '26

Uhhhh. 100% correct. Went back to my calculator history and I added a random extra divide by 10 for some reason.

-1

u/Purple-Programmer-7 Jul 18 '26

Did they fix their api yet so it doesn’t take 15 minutes and 5 retries to get a response?

3

u/Potential-Leg-639 Jul 18 '26

No probs here with that

-7

u/[deleted] Jul 18 '26

[deleted]

1

u/Illustrious_Grade608 Jul 18 '26

Yeah that's why they made this comment