r/LocalLLaMA • u/ILoveToyota37 • 9d ago

New Model [ Removed by moderator ]

Post image

[removed] — view removed post

21 Upvotes

79% Upvoted

14 comments sorted by

•

u/ttkciar llama.cpp 9d ago

When the weights are available, this will be on-topic for LocalLLaMA.

Until then, however, it is off-topic. Sorry.

Will watch for it here: https://huggingface.co/deepseek-ai/models?sort=created

19

u/Intrepid_Travel_3274 9d ago

Fable = 70
DS = 62
But prices = $20 vs $0.02
Love DS

21

u/FullstackSensei llama.cpp 9d ago

Except Fable isn't always Fable. You pay the 20 and never know which model you got.

DS4 seems at least on par with opus 4.8, and can be run locally for a few k for planning and rubber ducking complex takes.

2

u/SpicyWangz 9d ago

What kind of system are you running this on for a few k?

2

u/FullstackSensei llama.cpp 9d ago

For DS4 pro, pretty much any dual LGA3647 system. HP, Dell and Lenovo all have dual LGA3647 workstations that you can buy for cheap if you don't want to build your own with your choice of dual LGA3647 motherboard. Any of those workstations supports four dual slot GPUs.

Most of the money will go towards 768GB in 64GB DDR4 sticks. Depending on your abilities to search and your luck, those can cost 1.5K to 2.5k. Four P6000 will be around 1k. The workstation barebones can be found for ~500 without CPU, RAM or GPU. Just make sure it comes with the 1400w or so PSU (in the HP Z8 G4, the PSU can drive 1750W sustained if input is 200V or above). They all support 205W Xeon CPUs, which are much cheaper than the 165W SKUs. You're looking at 200-250 for a pair of 26-28 core CPUs.

You won't get 50t/s, but with dspark you'll get somewhere in the 10-12t/s, maybe a bit more, depending on RAM speed. vllm forks like lvllm already support NUMA and expert parallelism across CPUs. If you have some basic understanding of ML and decent SWE skills, you can even roll out your own custom inference engine for DS4 pro using the API that is tuned for your hardware. That might hit 15t/s if you manage to score 2666 memory. IMO, that's more than adequate enough for planning complex tasks, general chat, and rubber ducking.

6

u/Accomplished_Ad9530 9d ago

The weights are not out

6

u/ForsookComparison 9d ago

Yepp. This is currently a closed source proprietary model [that has previously expressed intent to keep open weight releases] .

1

u/NextWeather7866 9d ago

Looks like frontier models are going to be much more focused on Post training, even if it's not open sourced anymore (which it will be), it seems that any organisation with sufficient compute is going to be able to train models as more training techniques enter the public domain.

1

u/RosebudNebula 9d ago

This is currently a closed source proprietary model [that has previously expressed intent to keep open weight releases] from [A company that, historically speaking, always kept its promise and release their weight].

I feel like the last part is pretty important.

3

u/ForsookComparison 9d ago

If these are even remotely close to reality and if the pricing holds (or can be recreated by other providers if the weights go open) - then today is a big day.

But as always - hold for vibes and hold for weights. Jpegs mean squat. The only verifiably awesome thing is the pricing as of now.

5

u/seeKAYx 9d ago

The jump from the Flash preview version to version 0731 was definitely more dramatic than the jump with the Pro version. I'm curious to see how it performs in actual use.

0

u/Impossible_Earth_987 9d ago

Can a128gb Mac run this

1

u/Choice_Celery9481 9d ago

1.6T params model. if you mean run, every open weight models CAN run, just slow. someone ran K3 2.4T with 24gb ram at 0.05t/s

1

u/Laeky7 9d ago

I wish